Google Git
Sign in
chromium/external/github.com/python/cpython/0959142e4defcf7a9fcbbb228d2e2b97a074f7ea/./Doc/whatsnew/3.13.rst
blob: aa06e92afc9a43876b5a7ba59d3ceedd8d6f12f1 [file]
****************************
What's New In Python 3.13
****************************
:Editor: Thomas Wouters
.. Rules for maintenance:
* Anyone can add text to this document. Do not spend very much time
on the wording of your changes, because your text will probably
get rewritten to some degree.
* The maintainer will go through Misc/NEWS periodically and add
changes; it's therefore more important to add your changes to
Misc/NEWS than to this file.
* This is not a complete list of every single change; completeness
is the purpose of Misc/NEWS. Some changes I consider too small
or esoteric to include. If such a change is added to the text,
I'll just remove it. (This is another reason you shouldn't spend
too much time on writing your addition.)
* If you want to draw your new text to the attention of the
maintainer, add 'XXX' to the beginning of the paragraph or
section.
* It's OK to just add a fragmentary note about a change. For
example: "XXX Describe the transmogrify() function added to the
socket module." The maintainer will research the change and
write the necessary text.
* You can comment out your additions if you like, but it's not
necessary (especially when a final release is some months away).
* Credit the author of a patch or bugfix. Just the name is
sufficient; the e-mail address isn't necessary.
* It's helpful to add the issue number as a comment:
XXX Describe the transmogrify() function added to the socket
module.
(Contributed by P.Y. Developer in :gh:`12345`.)
This saves the maintainer the effort of going through the VCS log
when researching a change.
This article explains the new features in Python 3.13, compared to 3.12.
For full details, see the :ref:`changelog <changelog>`.
.. seealso::
:pep:`719` -- Python 3.13 Release Schedule
.. note::
Prerelease users should be aware that this document is currently in draft
form. It will be updated substantially as Python 3.13 moves towards release,
so it's worth checking back even after reading earlier versions.
Summary -- Release Highlights
=============================
.. This section singles out the most important changes in Python 3.13.
Brevity is key.
Python 3.13 beta is the pre-release of the next version of the Python
programming language, with a mix of changes to the language, the
implementation and the standard library. The biggest changes to the
implementation include a new interactive interpreter, and experimental
support for dropping the Global Interpreter Lock (:pep:`703`) and a
Just-In-Time compiler (:pep:`744`). The library changes contain removal of
deprecated APIs and modules, as well as the usual improvements in
user-friendliness and correctness.
.. PEP-sized items next.
Interpreter improvements:
* A greatly improved :ref:`interactive interpreter
<whatsnew313-better-interactive-interpreter>` and
:ref:`improved error messages <whatsnew313-improved-error-messages>`.
* Color support in the new :ref:`interactive interpreter
<whatsnew313-better-interactive-interpreter>`,
as well as in :ref:`tracebacks <whatsnew313-improved-error-messages>`
and :ref:`doctest <whatsnew313-doctest>` output. This can be disabled through the
:envvar:`PYTHON_COLORS` and |NO_COLOR|_ environment variables.
* :pep:`744`: A basic :ref:`JIT compiler <whatsnew313-jit-compiler>` was added.
It is currently disabled by default (though we may turn it on later).
Performance improvements are modest -- we expect to be improving this
over the next few releases.
* :pep:`667`: The :func:`locals` builtin now has
:ref:`defined semantics <whatsnew313-locals-semantics>` when mutating the
returned mapping. Python debuggers and similar tools may now more reliably
update local variables in optimized scopes even during concurrent code
execution.
New typing features:
* :pep:`696`: Type parameters (:data:`typing.TypeVar`, :data:`typing.ParamSpec`,
and :data:`typing.TypeVarTuple`) now support defaults.
* :pep:`702`: Support for marking deprecations in the type system using the
new :func:`warnings.deprecated` decorator.
* :pep:`742`: :data:`typing.TypeIs` was added, providing more intuitive
type narrowing behavior.
* :pep:`705`: :data:`typing.ReadOnly` was added, to mark an item of a
:class:`typing.TypedDict` as read-only for type checkers.
Free-threading:
* :pep:`703`: CPython 3.13 has experimental support for running with the
:term:`global interpreter lock` disabled when built with ``--disable-gil``.
See :ref:`Free-threaded CPython <free-threaded-cpython>` for more details.
Platform support:
* :pep:`730`: Apple's iOS is now an officially supported platform. Official
Android support (:pep:`738`) is in the works as well.
Removed modules:
* :ref:`PEP 594 <whatsnew313-pep594>`: The remaining 19 "dead batteries"
have been removed from the standard library:
:mod:`!aifc`, :mod:`!audioop`, :mod:`!cgi`, :mod:`!cgitb`, :mod:`!chunk`,
:mod:`!crypt`, :mod:`!imghdr`, :mod:`!mailcap`, :mod:`!msilib`, :mod:`!nis`,
:mod:`!nntplib`, :mod:`!ossaudiodev`, :mod:`!pipes`, :mod:`!sndhdr`, :mod:`!spwd`,
:mod:`!sunau`, :mod:`!telnetlib`, :mod:`!uu` and :mod:`!xdrlib`.
* Also removed were the :mod:`!tkinter.tix` and :mod:`!lib2to3` modules, and the
``2to3`` program.
Release schedule changes:
* :pep:`602` ("Annual Release Cycle for Python") has been updated:
* Python 3.9 - 3.12 have one and a half years of full support,
followed by three and a half years of security fixes.
* Python 3.13 and later have two years of full support,
followed by three years of security fixes.
New Features
============
.. _whatsnew313-better-interactive-interpreter:
A Better Interactive Interpreter
--------------------------------
On Unix-like systems like Linux or macOS as well as Windows, Python now
uses a new :term:`interactive` shell. When the user starts the
:term:`REPL` from an interactive terminal the interactive shell now
supports the following new features:
* Colorized prompts.
* Multiline editing with history preservation.
* Interactive help browsing using :kbd:`F1` with a separate command
history.
* History browsing using :kbd:`F2` that skips output as well as the
:term:`>>>` and :term:`...` prompts.
* "Paste mode" with :kbd:`F3` that makes pasting larger blocks of code
easier (press :kbd:`F3` again to return to the regular prompt).
* The ability to issue REPL-specific commands like :kbd:`help`, :kbd:`exit`,
and :kbd:`quit` without the need to use call parentheses after the command
name.
If the new interactive shell is not desired, it can be disabled via
the :envvar:`PYTHON_BASIC_REPL` environment variable.
The new shell requires :mod:`curses` on Unix-like systems.
For more on interactive mode, see :ref:`tut-interac`.
(Contributed by Pablo Galindo Salgado, Łukasz Langa, and
Lysandros Nikolaou in :gh:`111201` based on code from the PyPy project.
Windows support contributed by Dino Viehland and Anthony Shaw.)
.. _whatsnew313-improved-error-messages:
Improved Error Messages
-----------------------
* The interpreter now colorizes error messages when displaying tracebacks by default.
This feature can be controlled via the new :envvar:`PYTHON_COLORS` environment
variable as well as the canonical |NO_COLOR|_ and |FORCE_COLOR|_ environment
variables. See also :ref:`using-on-controlling-color`.
(Contributed by Pablo Galindo Salgado in :gh:`112730`.)
.. Apparently this how you hack together a formatted link:
.. |FORCE_COLOR| replace:: ``FORCE_COLOR``
.. _FORCE_COLOR: https://force-color.org/
.. |NO_COLOR| replace:: ``NO_COLOR``
.. _NO_COLOR: https://no-color.org/
* A common mistake is to write a script with the same name as a
standard library module. When this results in errors, we now
display a more helpful error message:
.. code-block:: shell-session
$ python random.py
Traceback (most recent call last):
File "/home/random.py", line 1, in <module>
import random; print(random.randint(5))
^^^^^^^^^^^^^
File "/home/random.py", line 1, in <module>
import random; print(random.randint(5))
^^^^^^^^^^^^^^
AttributeError: module 'random' has no attribute 'randint' (consider renaming '/home/random.py' since it has the same name as the standard library module named 'random' and the import system gives it precedence)
Similarly, if a script has the same name as a third-party
module it attempts to import, and this results in errors,
we also display a more helpful error message:
.. code-block:: shell-session
$ python numpy.py
Traceback (most recent call last):
File "/home/numpy.py", line 1, in <module>
import numpy as np; np.array([1,2,3])
^^^^^^^^^^^^^^^^^^
File "/home/numpy.py", line 1, in <module>
import numpy as np; np.array([1,2,3])
^^^^^^^^
AttributeError: module 'numpy' has no attribute 'array' (consider renaming '/home/numpy.py' if it has the same name as a third-party module you intended to import)
(Contributed by Shantanu Jain in :gh:`95754`.)
* When an incorrect keyword argument is passed to a function, the error message
now potentially suggests the correct keyword argument.
(Contributed by Pablo Galindo Salgado and Shantanu Jain in :gh:`107944`.)
>>> "better error messages!".split(max_split=1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
"better error messages!".split(max_split=1)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
TypeError: split() got an unexpected keyword argument 'max_split'. Did you mean 'maxsplit'?
* Classes have a new :attr:`~class.__static_attributes__` attribute, populated by the compiler,
with a tuple of names of attributes of this class which are assigned
through ``self.X`` from any function in its body. (Contributed by Irit Katriel
in :gh:`115775`.)
.. _whatsnew313-locals-semantics:
Defined mutation semantics for ``locals()``
-------------------------------------------
Historically, the expected result of mutating the return value of :func:`locals`
has been left to individual Python implementations to define.
Through :pep:`667`, Python 3.13 standardises the historical behaviour of CPython
for most code execution scopes, but changes
:term:`optimized scopes <optimized scope>` (functions, generators, coroutines,
comprehensions, and generator expressions) to explicitly return independent
snapshots of the currently assigned local variables, including locally
referenced nonlocal variables captured in closures.
This change to the semantics of :func:`locals` in optimized scopes also affects the default
behaviour of code execution functions that implicitly target ``locals()`` if no explicit
namespace is provided (such as :func:`exec` and :func:`eval`). In previous versions, whether
or not changes could be accessed by calling ``locals()`` after calling the code execution
function was implementation dependent. In CPython specifically, such code would typically
appear to work as desired, but could sometimes fail in optimized scopes based on other code
(including debuggers and code execution tracing tools) potentially resetting the shared
snapshot in that scope. Now, the code will always run against an independent snapshot of the
local variables in optimized scopes, and hence the changes will never be visible in
subsequent calls to ``locals()``. To access the changes made in these cases, an explicit
namespace reference must now be passed to the relevant function. Alternatively, it may make
sense to update affected code to use a higher level code execution API that returns the
resulting code execution namespace (e.g. :func:`runpy.run_path` when executing Python
files from disk).
To ensure debuggers and similar tools can reliably update local variables in
scopes affected by this change, :attr:`FrameType.f_locals <frame.f_locals>` now
returns a write-through proxy to the frame's local and locally referenced
nonlocal variables in these scopes, rather than returning an inconsistently
updated shared ``dict`` instance with undefined runtime semantics.
See :pep:`667` for more details, including related C API changes and deprecations. Porting
notes are also provided below for the affected :ref:`Python APIs <pep667-porting-notes-py>`
and :ref:`C APIs <pep667-porting-notes-c>`.
(PEP and implementation contributed by Mark Shannon and Tian Gao in
:gh:`74929`. Documentation updates provided by Guido van Rossum and
Alyssa Coghlan.)
Incremental Garbage Collection
------------------------------
* The cycle garbage collector is now incremental.
This means that maximum pause times are reduced
by an order of magnitude or more for larger heaps.
Support For Mobile Platforms
----------------------------
* iOS is now a :pep:`11` supported platform. ``arm64-apple-ios``
(iPhone and iPad devices released after 2013) and
``arm64-apple-ios-simulator`` (Xcode iOS simulator running on Apple Silicon
hardware) are now tier 3 platforms.
``x86_64-apple-ios-simulator`` (Xcode iOS simulator running on older x86_64
hardware) is not a tier 3 supported platform, but will be supported on a
best-effort basis.
See :pep:`730`: for more details.
(PEP written and implementation contributed by Russell Keith-Magee in
:gh:`114099`.)
.. _whatsnew313-jit-compiler:
Experimental JIT Compiler
=========================
When CPython is configured using the ``--enable-experimental-jit`` option,
a just-in-time compiler is added which may speed up some Python programs.
The internal architecture is roughly as follows.
* We start with specialized *Tier 1 bytecode*.
See :ref:`What's new in 3.11 <whatsnew311-pep659>` for details.
* When the Tier 1 bytecode gets hot enough, it gets translated
to a new, purely internal *Tier 2 IR*, a.k.a. micro-ops ("uops").
* The Tier 2 IR uses the same stack-based VM as Tier 1, but the
instruction format is better suited to translation to machine code.
* We have several optimization passes for Tier 2 IR, which are applied
before it is interpreted or translated to machine code.
* There is a Tier 2 interpreter, but it is mostly intended for debugging
the earlier stages of the optimization pipeline.
The Tier 2 interpreter can be enabled by configuring Python
with ``--enable-experimental-jit=interpreter``.
* When the JIT is enabled, the optimized
Tier 2 IR is translated to machine code, which is then executed.
* The machine code translation process uses a technique called
*copy-and-patch*. It has no runtime dependencies, but there is a new
build-time dependency on LLVM.
The ``--enable-experimental-jit`` flag has the following optional values:
* ``no`` (default) -- Disable the entire Tier 2 and JIT pipeline.
* ``yes`` (default if the flag is present without optional value)
-- Enable the JIT. To disable the JIT at runtime,
pass the environment variable ``PYTHON_JIT=0``.
* ``yes-off`` -- Build the JIT but disable it by default.
To enable the JIT at runtime, pass the environment variable
``PYTHON_JIT=1``.
* ``interpreter`` -- Enable the Tier 2 interpreter but disable the JIT.
The interpreter can be disabled by running with
``PYTHON_JIT=0``.
(On Windows, use ``PCbuild/build.bat --experimental-jit`` to enable the JIT
or ``--experimental-jit-interpreter`` to enable the Tier 2 interpreter.)
See :pep:`744` for more details.
(JIT by Brandt Bucher, inspired by a paper by Haoran Xu and Fredrik Kjolstad.
Tier 2 IR by Mark Shannon and Guido van Rossum.
Tier 2 optimizer by Ken Jin.)
.. _free-threaded-cpython:
Free-threaded CPython
=====================
CPython will run with the :term:`global interpreter lock` (GIL) disabled when
configured using the ``--disable-gil`` option at build time. This is an
experimental feature and therefore isn't used by default. Users need to
either compile their own interpreter, or install one of the experimental
builds that are marked as *free-threaded*. See :pep:`703` "Making the Global
Interpreter Lock Optional in CPython" for more detail.
Free-threaded execution allows for full utilization of the available
processing power by running threads in parallel on available CPU cores.
While not all software will benefit from this automatically, programs
designed with threading in mind will run faster on multicore hardware.
Work is still ongoing: expect some bugs and a substantial single-threaded
performance hit.
The free-threaded build still supports optionally running with the GIL
enabled at runtime using the environment variable :envvar:`PYTHON_GIL` or
the command line option :option:`-X gil`.
To check if the current interpreter is configured with ``--disable-gil``,
use ``sysconfig.get_config_var("Py_GIL_DISABLED")``. To check if the :term:`GIL`
is actually disabled in the running process, the :func:`!sys._is_gil_enabled`
function can be used.
C-API extension modules need to be built specifically for the free-threaded
build. Extensions that support running with the :term:`GIL` disabled should
use the :c:data:`Py_mod_gil` slot. Extensions using single-phase init should
use :c:func:`PyUnstable_Module_SetGIL` to indicate whether they support
running with the GIL disabled. Importing C extensions that don't use these
mechanisms will cause the GIL to be enabled, unless the GIL was explicitly
disabled with the :envvar:`PYTHON_GIL` environment variable or the
:option:`-X gil=0` option.
pip 24.1b1 or newer is required to install packages with C extensions in the
free-threaded build.
Other Language Changes
======================
* The :func:`exec` and :func:`eval` built-ins now accept their ``globals``
and ``locals`` namespace arguments as keywords.
(Contributed by Raphael Gaschignard in :gh:`105879`)
* Allow the *count* argument of :meth:`str.replace` to be a keyword.
(Contributed by Hugo van Kemenade in :gh:`106487`.)
* Compiler now strip indents from docstrings.
This will reduce the size of :term:`bytecode cache <bytecode>` (e.g. ``.pyc`` file).
For example, cache file size for ``sqlalchemy.orm.session`` in SQLAlchemy 2.0
is reduced by about 5%.
This change will affect tools using docstrings, like :mod:`doctest`.
(Contributed by Inada Naoki in :gh:`81283`.)
* The :func:`compile` built-in can now accept a new flag,
``ast.PyCF_OPTIMIZED_AST``, which is similar to ``ast.PyCF_ONLY_AST``
except that the returned ``AST`` is optimized according to the value
of the ``optimize`` argument.
(Contributed by Irit Katriel in :gh:`108113`).
* :mod:`multiprocessing`, :mod:`concurrent.futures`, :mod:`compileall`:
Replace :func:`os.cpu_count` with :func:`os.process_cpu_count` to select the
default number of worker threads and processes. Get the CPU affinity
if supported.
(Contributed by Victor Stinner in :gh:`109649`.)
* :func:`os.path.realpath` now resolves MS-DOS style file names even if
the file is not accessible.
(Contributed by Moonsik Park in :gh:`82367`.)
* Fixed a bug where a :keyword:`global` declaration in an :keyword:`except` block
is rejected when the global is used in the :keyword:`else` block.
(Contributed by Irit Katriel in :gh:`111123`.)
* Many functions now emit a warning if a boolean value is passed as
a file descriptor argument.
This can help catch some errors earlier.
(Contributed by Serhiy Storchaka in :gh:`82626`.)
* Added a new environment variable :envvar:`PYTHON_FROZEN_MODULES`. It
determines whether or not frozen modules are ignored by the import machinery,
equivalent of the :option:`-X frozen_modules <-X>` command-line option.
(Contributed by Yilei Yang in :gh:`111374`.)
* Add :ref:`support for the perf profiler <perf_profiling>` working without
frame pointers through the new environment variable
:envvar:`PYTHON_PERF_JIT_SUPPORT` and command-line option :option:`-X perf_jit
<-X>` (Contributed by Pablo Galindo in :gh:`118518`.)
* The new :envvar:`PYTHON_HISTORY` environment variable can be used to change
the location of a ``.python_history`` file.
(Contributed by Levi Sabah, Zackery Spytz and Hugo van Kemenade in
:gh:`73965`.)
* Add :exc:`PythonFinalizationError` exception. This exception derived from
:exc:`RuntimeError` is raised when an operation is blocked during
the :term:`Python finalization <interpreter shutdown>`.
The following functions now raise PythonFinalizationError, instead of
:exc:`RuntimeError`:
* :func:`_thread.start_new_thread`.
* :class:`subprocess.Popen`.
* :func:`os.fork`.
* :func:`os.forkpty`.
(Contributed by Victor Stinner in :gh:`114570`.)
* Added :attr:`!name` and :attr:`!mode` attributes for compressed
and archived file-like objects in modules :mod:`bz2`, :mod:`lzma`,
:mod:`tarfile` and :mod:`zipfile`.
(Contributed by Serhiy Storchaka in :gh:`115961`.)
* Allow controlling Expat >=2.6.0 reparse deferral (:cve:`2023-52425`)
by adding five new methods:
* :meth:`xml.etree.ElementTree.XMLParser.flush`
* :meth:`xml.etree.ElementTree.XMLPullParser.flush`
* :meth:`xml.parsers.expat.xmlparser.GetReparseDeferralEnabled`
* :meth:`xml.parsers.expat.xmlparser.SetReparseDeferralEnabled`
* :meth:`!xml.sax.expatreader.ExpatParser.flush`
(Contributed by Sebastian Pipping in :gh:`115623`.)
* The :func:`ssl.create_default_context` API now includes
:data:`ssl.VERIFY_X509_PARTIAL_CHAIN` and :data:`ssl.VERIFY_X509_STRICT`
in its default flags.
.. note::
:data:`ssl.VERIFY_X509_STRICT` may reject pre-:rfc:`5280` or malformed
certificates that the underlying OpenSSL implementation otherwise would
accept. While disabling this is not recommended, you can do so using::
ctx = ssl.create_default_context()
ctx.verify_flags &= ~ssl.VERIFY_X509_STRICT
(Contributed by William Woodruff in :gh:`112389`.)
* The :class:`configparser.ConfigParser` now accepts unnamed sections before named
ones if configured to do so.
(Contributed by Pedro Sousa Lacerda in :gh:`66449`.)
* :ref:`annotation scope <annotation-scopes>` within class scopes can now
contain lambdas and comprehensions. Comprehensions that are located within
class scopes are not inlined into their parent scope. (Contributed by
Jelle Zijlstra in :gh:`109118` and :gh:`118160`.)
* Classes have a new :attr:`!__firstlineno__` attribute,
populated by the compiler, with the line number of the first line
of the class definition.
(Contributed by Serhiy Storchaka in :gh:`118465`.)
* ``from __future__ import ...`` statements are now just normal
relative imports if dots are present before the module name.
(Contributed by Jeremiah Gabriel Pascual in :gh:`118216`.)
New Modules
===========
* None.
Improved Modules
================
argparse
--------
* Add parameter *deprecated* in methods
:meth:`~argparse.ArgumentParser.add_argument` and :meth:`!add_parser`
which allows to deprecate command-line options, positional arguments and
subcommands.
(Contributed by Serhiy Storchaka in :gh:`83648`.)
array
-----
* Add ``'w'`` type code (``Py_UCS4``) that can be used for Unicode strings.
It can be used instead of ``'u'`` type code, which is deprecated.
(Contributed by Inada Naoki in :gh:`80480`.)
* Add ``clear()`` method in order to implement ``MutableSequence``.
(Contributed by Mike Zimin in :gh:`114894`.)
ast
---
* The constructors of node types in the :mod:`ast` module are now stricter
in the arguments they accept, and have more intuitive behaviour when
arguments are omitted.
If an optional field on an AST node is not included as an argument when
constructing an instance, the field will now be set to ``None``. Similarly,
if a list field is omitted, that field will now be set to an empty list,
and if a :class:`!ast.expr_context` field is omitted, it defaults to
:class:`Load() <ast.Load>`.
(Previously, in all cases, the attribute would be missing on the newly
constructed AST node instance.)
If other arguments are omitted, a :exc:`DeprecationWarning` is emitted.
This will cause an exception in Python 3.15. Similarly, passing a keyword
argument that does not map to a field on the AST node is now deprecated,
and will raise an exception in Python 3.15.
These changes do not apply to user-defined subclasses of :class:`ast.AST`,
unless the class opts in to the new behavior by setting the attribute
:attr:`ast.AST._field_types`.
(Contributed by Jelle Zijlstra in :gh:`105858`, :gh:`117486`, and :gh:`118851`.)
* :func:`ast.parse` now accepts an optional argument *optimize*
which is passed on to the :func:`compile` built-in. This makes it
possible to obtain an optimized AST.
(Contributed by Irit Katriel in :gh:`108113`.)
asyncio
-------
* :meth:`asyncio.loop.create_unix_server` will now automatically remove
the Unix socket when the server is closed.
(Contributed by Pierre Ossman in :gh:`111246`.)
* :meth:`asyncio.DatagramTransport.sendto` will now send zero-length
datagrams if called with an empty bytes object. The transport flow
control also now accounts for the datagram header when calculating
the buffer size.
(Contributed by Jamie Phan in :gh:`115199`.)
* Add :meth:`asyncio.Server.close_clients` and
:meth:`asyncio.Server.abort_clients` methods which allow to more
forcefully close an asyncio server.
(Contributed by Pierre Ossman in :gh:`113538`.)
* :func:`asyncio.as_completed` now returns an object that is both an
:term:`asynchronous iterator` and a plain :term:`iterator` of awaitables.
The awaitables yielded by asynchronous iteration include original task or
future objects that were passed in, making it easier to associate results
with the tasks being completed.
(Contributed by Justin Arthur in :gh:`77714`.)
* When :func:`asyncio.TaskGroup.create_task` is called on an inactive
:class:`asyncio.TaskGroup`, the given coroutine will be closed (which
prevents a :exc:`RuntimeWarning` about the given coroutine being
never awaited).
(Contributed by Arthur Tacca and Jason Zhang in :gh:`115957`.)
* Improved behavior of :class:`asyncio.TaskGroup` when an external cancellation
collides with an internal cancellation. For example, when two task groups
are nested and both experience an exception in a child task simultaneously,
it was possible that the outer task group would hang, because its internal
cancellation was swallowed by the inner task group.
In the case where a task group is cancelled externally and also must
raise an :exc:`ExceptionGroup`, it will now call the parent task's
:meth:`~asyncio.Task.cancel` method. This ensures that a
:exc:`asyncio.CancelledError` will be raised at the next
:keyword:`await`, so the cancellation is not lost.
An added benefit of these changes is that task groups now preserve the
cancellation count (:meth:`asyncio.Task.cancelling`).
In order to handle some corner cases, :meth:`asyncio.Task.uncancel` may now
reset the undocumented ``_must_cancel`` flag when the cancellation count
reaches zero.
(Inspired by an issue reported by Arthur Tacca in :gh:`116720`.)
* Add :meth:`asyncio.Queue.shutdown` (along with
:exc:`asyncio.QueueShutDown`) for queue termination.
(Contributed by Laurie Opperman and Yves Duprat in :gh:`104228`.)
* Accept a tuple of separators in :meth:`asyncio.StreamReader.readuntil`,
stopping when one of them is encountered.
(Contributed by Bruce Merry in :gh:`81322`.)
base64
------
* Add :func:`base64.z85encode` and :func:`base64.z85decode` functions which allow encoding
and decoding Z85 data.
See `Z85 specification <https://rfc.zeromq.org/spec/32/>`_ for more information.
(Contributed by Matan Perelman in :gh:`75299`.)
copy
----
* Add :func:`copy.replace` function which allows to create a modified copy of
an object, which is especially useful for immutable objects.
It supports named tuples created with the factory function
:func:`collections.namedtuple`, :class:`~dataclasses.dataclass` instances,
various :mod:`datetime` objects, :class:`~inspect.Signature` objects,
:class:`~inspect.Parameter` objects, :ref:`code object <code-objects>`, and
any user classes which define the :meth:`!__replace__` method.
(Contributed by Serhiy Storchaka in :gh:`108751`.)
ctypes
------
* The layout of :ref:`bit fields <ctypes-bit-fields-in-structures-unions>` in
:class:`~ctypes.Structure` and :class:`~ctypes.Union` was improved to better
match platform defaults (GCC/Clang or MSC). In particular, fields no longer
overlap.
(Contributed by Matthias Görgens in :gh:`97702`.)
* A :attr:`ctypes.Structure._layout_` class attribute can be set
to help match a non-default ABI.
(Contributed by Petr Viktorin in :gh:`97702`.)
dbm
---
* Add :meth:`dbm.gnu.gdbm.clear` and :meth:`dbm.ndbm.ndbm.clear` methods that remove all items
from the database.
(Contributed by Donghee Na in :gh:`107122`.)
* Add new :mod:`dbm.sqlite3` backend, and make it the default :mod:`!dbm` backend.
(Contributed by Raymond Hettinger and Erlend E. Aasland in :gh:`100414`.)
dis
---
* Change the output of :mod:`dis` module functions to show logical
labels for jump targets and exception handlers, rather than offsets.
The offsets can be added with the new ``-O`` command line option or
the ``show_offsets`` parameter.
(Contributed by Irit Katriel in :gh:`112137`.)
* :meth:`~dis.get_instructions` no longer represents cache entries as
separate instructions. Instead, it returns them as part of the
:class:`~dis.Instruction`, in the new *cache_info* field. The
*show_caches* argument to :meth:`~dis.get_instructions` is
deprecated and no longer has any effect.
(Contributed by Irit Katriel in :gh:`112962`.)
.. _whatsnew313-doctest:
doctest
-------
* Color is added to the output by default.
This can be controlled via the new :envvar:`PYTHON_COLORS` environment
variable as well as the canonical |NO_COLOR|_ and |FORCE_COLOR|_ environment
variables. See also :ref:`using-on-controlling-color`.
(Contributed by Hugo van Kemenade in :gh:`117225`.)
* The :meth:`doctest.DocTestRunner.run` method now counts the number of skipped
tests. Add :attr:`doctest.DocTestRunner.skips` and
:attr:`doctest.TestResults.skipped` attributes.
(Contributed by Victor Stinner in :gh:`108794`.)
email
-----
* Headers with embedded newlines are now quoted on output.
The :mod:`~email.generator` will now refuse to serialize (write) headers
that are improperly folded or delimited, such that they would be parsed as
multiple headers or joined with adjacent data.
If you need to turn this safety feature off,
set :attr:`~email.policy.Policy.verify_generated_headers`.
(Contributed by Bas Bloemsaat and Petr Viktorin in :gh:`121650`.)
* :func:`email.utils.getaddresses` and :func:`email.utils.parseaddr` now return
``('', '')`` 2-tuples in more situations where invalid email addresses are
encountered instead of potentially inaccurate values. Add optional *strict*
parameter to these two functions: use ``strict=False`` to get the old
behavior, accept malformed inputs.
``getattr(email.utils, 'supports_strict_parsing', False)`` can be used to
check if the *strict* parameter is available.
(Contributed by Thomas Dwyer and Victor Stinner for :gh:`102988` to improve
the :cve:`2023-27043` fix.)
fractions
---------
* Formatting for objects of type :class:`fractions.Fraction` now supports
the standard format specification mini-language rules for fill, alignment,
sign handling, minimum width and grouping. (Contributed by Mark Dickinson
in :gh:`111320`.)
gc
--
* The cyclic garbage collector is now incremental, which changes the meanings
of the results of :meth:`gc.get_threshold` and :meth:`gc.set_threshold` as
well as :meth:`gc.get_count` and :meth:`gc.get_stats`.
* :meth:`gc.get_threshold` returns a three-item tuple for backwards compatibility.
The first value is the threshold for young collections, as before; the second
value determines the rate at which the old collection is scanned (the
default is 10, and higher values mean that the old collection is scanned more slowly).
The third value is meaningless and is always zero.
* :meth:`gc.set_threshold` ignores any items after the second.
* :meth:`gc.get_count` and :meth:`gc.get_stats`
return the same format of results as before.
The only difference is that instead of the results referring to
the young, aging and old generations, the results refer to the
young generation and the aging and collecting spaces of the old generation.
In summary, code that attempted to manipulate the behavior of the cycle GC may
not work exactly as intended, but it is very unlikely to be harmful.
All other code will work just fine.
glob
----
* Add :func:`glob.translate` function that converts a path specification with
shell-style wildcards to a regular expression.
(Contributed by Barney Gale in :gh:`72904`.)
importlib
---------
* Previously deprecated :mod:`importlib.resources` functions are un-deprecated:
* :func:`~importlib.resources.is_resource()`
* :func:`~importlib.resources.open_binary()`
* :func:`~importlib.resources.open_text()`
* :func:`~importlib.resources.path()`
* :func:`~importlib.resources.read_binary()`
* :func:`~importlib.resources.read_text()`
All now allow for a directory (or tree) of resources, using multiple positional
arguments.
For text-reading functions, the *encoding* and *errors* must now be given as
keyword arguments.
The :func:`~importlib.resources.contents()` remains deprecated in favor of
the full-featured :class:`~importlib.resources.abc.Traversable` API.
However, there is now no plan to remove it.
(Contributed by Petr Viktorin in :gh:`106532`.)
io
--
* The :class:`io.IOBase` finalizer now logs the ``close()`` method errors with
:data:`sys.unraisablehook`. Previously, errors were ignored silently by default,
and only logged in :ref:`Python Development Mode <devmode>` or on :ref:`Python
built on debug mode <debug-build>`.
(Contributed by Victor Stinner in :gh:`62948`.)
ipaddress
---------
* Add the :attr:`ipaddress.IPv4Address.ipv6_mapped` property, which returns the IPv4-mapped IPv6 address.
(Contributed by Charles Machalow in :gh:`109466`.)
* Fix ``is_global`` and ``is_private`` behavior in
:class:`~ipaddress.IPv4Address`,
:class:`~ipaddress.IPv6Address`,
:class:`~ipaddress.IPv4Network` and
:class:`~ipaddress.IPv6Network`.
itertools
---------
* Added a ``strict`` option to :func:`itertools.batched`.
This raises a :exc:`ValueError` if the final batch is shorter
than the specified batch size.
(Contributed by Raymond Hettinger in :gh:`113202`.)
marshal
-------
* Add the *allow_code* parameter in module functions.
Passing ``allow_code=False`` prevents serialization and de-serialization of
code objects which are incompatible between Python versions.
(Contributed by Serhiy Storchaka in :gh:`113626`.)
math
----
* A new function :func:`~math.fma` for fused multiply-add operations has been
added. This function computes ``x * y + z`` with only a single round, and so
avoids any intermediate loss of precision. It wraps the ``fma()`` function
provided by C99, and follows the specification of the IEEE 754
"fusedMultiplyAdd" operation for special cases.
(Contributed by Mark Dickinson and Victor Stinner in :gh:`73468`.)
mimetypes
---------
* Add the :func:`~mimetypes.guess_file_type` function which works with file path.
Passing file path instead of URL in :func:`~mimetypes.guess_type` is :term:`soft deprecated`.
(Contributed by Serhiy Storchaka in :gh:`66543`.)
mmap
----
* The :class:`mmap.mmap` class now has an :meth:`~mmap.mmap.seekable` method
that can be used when a seekable file-like object is required.
The :meth:`~mmap.mmap.seek` method now returns the new absolute position.
(Contributed by Donghee Na and Sylvie Liberman in :gh:`111835`.)
* :class:`mmap.mmap` now has a *trackfd* parameter on Unix; if it is ``False``,
the file descriptor specified by *fileno* will not be duplicated.
(Contributed by Zackery Spytz and Petr Viktorin in :gh:`78502`.)
* :class:`mmap.mmap` is now protected from crashing on Windows when the mapped memory
is inaccessible due to file system errors or access violations.
(Contributed by Jannis Weigend in :gh:`118209`.)
opcode
------
* Move ``opcode.ENABLE_SPECIALIZATION`` to ``_opcode.ENABLE_SPECIALIZATION``.
This field was added in 3.12, it was never documented and is not intended for
external usage. (Contributed by Irit Katriel in :gh:`105481`.)
* Removed ``opcode.is_pseudo``, ``opcode.MIN_PSEUDO_OPCODE`` and
``opcode.MAX_PSEUDO_OPCODE``, which were added in 3.12, were never
documented or exposed through ``dis``, and were not intended to be
used externally.
os
--
* Add :func:`os.process_cpu_count` function to get the number of logical CPUs
usable by the calling thread of the current process.
(Contributed by Victor Stinner in :gh:`109649`.)
* Add a low level interface for Linux's timer notification file descriptors
via :func:`os.timerfd_create`,
:func:`os.timerfd_settime`, :func:`os.timerfd_settime_ns`,
:func:`os.timerfd_gettime`, and :func:`os.timerfd_gettime_ns`,
:const:`os.TFD_NONBLOCK`, :const:`os.TFD_CLOEXEC`,
:const:`os.TFD_TIMER_ABSTIME`, and :const:`os.TFD_TIMER_CANCEL_ON_SET`
(Contributed by Masaru Tsuchiyama in :gh:`108277`.)
* :func:`os.cpu_count` and :func:`os.process_cpu_count` can be overridden through
the new environment variable :envvar:`PYTHON_CPU_COUNT` or the new command-line option
:option:`-X cpu_count <-X>`. This option is useful for users who need to limit
CPU resources of a container system without having to modify the container (application code).
(Contributed by Donghee Na in :gh:`109595`.)
* Add support of :func:`os.lchmod` and the *follow_symlinks* argument
in :func:`os.chmod` on Windows.
Note that the default value of *follow_symlinks* in :func:`!os.lchmod` is
``False`` on Windows.
(Contributed by Serhiy Storchaka in :gh:`59616`.)
* Add support of :func:`os.fchmod` and a file descriptor
in :func:`os.chmod` on Windows.
(Contributed by Serhiy Storchaka in :gh:`113191`.)
* :func:`os.posix_spawn` now accepts ``env=None``, which makes the newly spawned
process use the current process environment.
(Contributed by Jakub Kulik in :gh:`113119`.)
* :func:`os.posix_spawn` gains an :attr:`os.POSIX_SPAWN_CLOSEFROM` attribute for
use in ``file_actions=`` on platforms that support
:c:func:`!posix_spawn_file_actions_addclosefrom_np`.
(Contributed by Jakub Kulik in :gh:`113117`.)
* :func:`os.mkdir` and :func:`os.makedirs` on Windows now support passing a
*mode* value of ``0o700`` to apply access control to the new directory. This
implicitly affects :func:`tempfile.mkdtemp` and is a mitigation for
:cve:`2024-4030`. Other values for *mode* continue to be ignored.
(Contributed by Steve Dower in :gh:`118486`.)
os.path
-------
* Add :func:`os.path.isreserved` to check if a path is reserved on the current
system. This function is only available on Windows.
(Contributed by Barney Gale in :gh:`88569`.)
* On Windows, :func:`os.path.isabs` no longer considers paths starting with
exactly one (back)slash to be absolute.
(Contributed by Barney Gale and Jon Foster in :gh:`44626`.)
* Add support of *dir_fd* and *follow_symlinks* keyword arguments in
:func:`shutil.chown`.
(Contributed by Berker Peksag and Tahia K in :gh:`62308`)
pathlib
-------
* Add :exc:`pathlib.UnsupportedOperation`, which is raised instead of
:exc:`NotImplementedError` when a path operation isn't supported.
(Contributed by Barney Gale in :gh:`89812`.)
* Add :meth:`pathlib.Path.from_uri`, a new constructor to create a :class:`pathlib.Path`
object from a 'file' URI (``file://``).
(Contributed by Barney Gale in :gh:`107465`.)
* Add :meth:`pathlib.PurePath.full_match` for matching paths with
shell-style wildcards, including the recursive wildcard "``**``".
(Contributed by Barney Gale in :gh:`73435`.)
* Add :attr:`pathlib.PurePath.parser` class attribute that stores the
implementation of :mod:`os.path` used for low-level path parsing and
joining: either ``posixpath`` or ``ntpath``.
* Add *recurse_symlinks* keyword-only argument to :meth:`pathlib.Path.glob`
and :meth:`~pathlib.Path.rglob`.
(Contributed by Barney Gale in :gh:`77609`.)
* Add *follow_symlinks* keyword-only argument to :meth:`~pathlib.Path.is_file`,
:meth:`~pathlib.Path.is_dir`, :meth:`~pathlib.Path.owner`,
:meth:`~pathlib.Path.group`.
(Contributed by Barney Gale in :gh:`105793`, and Kamil Turek in
:gh:`107962`.)
* Return files and directories from :meth:`pathlib.Path.glob` and
:meth:`~pathlib.Path.rglob` when given a pattern that ends with "``**``". In
earlier versions, only directories were returned.
(Contributed by Barney Gale in :gh:`70303`.)
pdb
---
* Add ability to move between chained exceptions during post mortem debugging in :func:`~pdb.pm` using
the new ``exceptions [exc_number]`` command for Pdb. (Contributed by Matthias
Bussonnier in :gh:`106676`.)
* Expressions/statements whose prefix is a pdb command are now correctly
identified and executed.
(Contributed by Tian Gao in :gh:`108464`.)
* ``sys.path[0]`` will no longer be replaced by the directory of the script
being debugged when ``sys.flags.safe_path`` is set (via the :option:`-P`
command line option or :envvar:`PYTHONSAFEPATH` environment variable).
(Contributed by Tian Gao and Christian Walther in :gh:`111762`.)
* :mod:`zipapp` is supported as a debugging target.
(Contributed by Tian Gao in :gh:`118501`.)
* ``breakpoint()`` and ``pdb.set_trace()`` now enter the debugger immediately
rather than on the next line of code to be executed. This change prevents the
debugger from breaking outside of the context when ``breakpoint()`` is positioned
at the end of the context.
(Contributed by Tian Gao in :gh:`118579`.)
queue
-----
* Add :meth:`queue.Queue.shutdown` (along with :exc:`queue.ShutDown`) for queue
termination.
(Contributed by Laurie Opperman and Yves Duprat in :gh:`104750`.)
random
------
* Add a :ref:`command-line interface <random-cli>`.
(Contributed by Hugo van Kemenade in :gh:`118131`.)
re
--
* Rename :exc:`!re.error` to :exc:`re.PatternError` for improved clarity.
:exc:`!re.error` is kept for backward compatibility.
site
----
* :file:`.pth` files are now decoded by UTF-8 first, and then by the
:term:`locale encoding` if the UTF-8 decoding fails.
(Contributed by Inada Naoki in :gh:`117802`.)
sqlite3
-------
* A :exc:`ResourceWarning` is now emitted if a :class:`sqlite3.Connection`
object is not :meth:`closed <sqlite3.Connection.close>` explicitly.
(Contributed by Erlend E. Aasland in :gh:`105539`.)
* Add *filter* keyword-only parameter to :meth:`sqlite3.Connection.iterdump`
for filtering database objects to dump.
(Contributed by Mariusz Felisiak in :gh:`91602`.)
statistics
----------
* Add :func:`statistics.kde` for kernel density estimation.
This makes it possible to estimate a continuous probability density function
from a fixed number of discrete samples. Also added :func:`statistics.kde_random`
for sampling from the estimated probability density function.
(Contributed by Raymond Hettinger in :gh:`115863`.)
.. _whatsnew313-subprocess:
subprocess
----------
* The :mod:`subprocess` module now uses the :func:`os.posix_spawn` function in
more situations. Notably in the default case of ``close_fds=True`` on more
recent versions of platforms including Linux, FreeBSD, and Solaris where the
C library provides :c:func:`!posix_spawn_file_actions_addclosefrom_np`.
On Linux this should perform similar to our existing Linux :c:func:`!vfork`
based code. A private control knob :attr:`!subprocess._USE_POSIX_SPAWN` can
be set to ``False`` if you need to force :mod:`subprocess` not to ever use
:func:`os.posix_spawn`. Please report your reason and platform details in
the CPython issue tracker if you set this so that we can improve our API
selection logic for everyone.
(Contributed by Jakub Kulik in :gh:`113117`.)
sys
---
* Add the :func:`sys._is_interned` function to test if the string was interned.
This function is not guaranteed to exist in all implementations of Python.
(Contributed by Serhiy Storchaka in :gh:`78573`.)
tempfile
--------
* On Windows, the default mode ``0o700`` used by :func:`tempfile.mkdtemp` now
limits access to the new directory due to changes to :func:`os.mkdir`. This
is a mitigation for :cve:`2024-4030`.
(Contributed by Steve Dower in :gh:`118486`.)
time
----
* On Windows, :func:`time.monotonic()` now uses the
``QueryPerformanceCounter()`` clock to have a resolution better than 1 us,
instead of the ``GetTickCount64()`` clock which has a resolution of 15.6 ms.
(Contributed by Victor Stinner in :gh:`88494`.)
* On Windows, :func:`time.time()` now uses the
``GetSystemTimePreciseAsFileTime()`` clock to have a resolution better
than 1 μs, instead of the ``GetSystemTimeAsFileTime()`` clock which has a
resolution of 15.6 ms.
(Contributed by Victor Stinner in :gh:`63207`.)
tkinter
-------
* Add :mod:`tkinter` widget methods:
:meth:`!tk_busy_hold`, :meth:`!tk_busy_configure`,
:meth:`!tk_busy_cget`, :meth:`!tk_busy_forget`,
:meth:`!tk_busy_current`, and :meth:`!tk_busy_status`.
(Contributed by Miguel, klappnase and Serhiy Storchaka in :gh:`72684`.)
* The :mod:`tkinter` widget method :meth:`!wm_attributes` now accepts
the attribute name without the minus prefix to get window attributes,
e.g. ``w.wm_attributes('alpha')`` and allows to specify attributes and
values to set as keyword arguments, e.g. ``w.wm_attributes(alpha=0.5)``.
Add new optional keyword-only parameter *return_python_dict*: calling
``w.wm_attributes(return_python_dict=True)`` returns the attributes as
a dict instead of a tuple.
(Contributed by Serhiy Storchaka in :gh:`43457`.)
* Add new optional keyword-only parameter *return_ints* in
the :meth:`!Text.count` method.
Passing ``return_ints=True`` makes it always returning the single count
as an integer instead of a 1-tuple or ``None``.
(Contributed by Serhiy Storchaka in :gh:`97928`.)
* Add support of the "vsapi" element type in
the :meth:`~tkinter.ttk.Style.element_create` method of
:class:`tkinter.ttk.Style`.
(Contributed by Serhiy Storchaka in :gh:`68166`.)
* Add the :meth:`!after_info` method for Tkinter widgets.
(Contributed by Cheryl Sabella in :gh:`77020`.)
* Add the :class:`!PhotoImage` method :meth:`!copy_replace` to copy a region
from one image to other image, possibly with pixel zooming and/or
subsampling.
Add *from_coords* parameter to :class:`!PhotoImage` methods :meth:`!copy()`,
:meth:`!zoom()` and :meth:`!subsample()`.
Add *zoom* and *subsample* parameters to :class:`!PhotoImage` method
:meth:`!copy()`.
(Contributed by Serhiy Storchaka in :gh:`118225`.)
* Add the :class:`!PhotoImage` methods :meth:`!read` to read
an image from a file and :meth:`!data` to get the image data.
Add *background* and *grayscale* parameters to :class:`!PhotoImage` method
:meth:`!write`.
(Contributed by Serhiy Storchaka in :gh:`118271`.)
traceback
---------
* Add *show_group* parameter to :func:`traceback.TracebackException.format_exception_only`
to format the nested exceptions of a :exc:`BaseExceptionGroup` instance, recursively.
(Contributed by Irit Katriel in :gh:`105292`.)
* Add the field *exc_type_str* to :class:`~traceback.TracebackException`, which
holds a string display of the *exc_type*. Deprecate the field *exc_type*
which holds the type object itself. Add parameter *save_exc_type* (default
``True``) to indicate whether ``exc_type`` should be saved.
(Contributed by Irit Katriel in :gh:`112332`.)
types
-----
* :class:`~types.SimpleNamespace` constructor now allows specifying initial
values of attributes as a positional argument which must be a mapping or
an iterable of key-value pairs.
(Contributed by Serhiy Storchaka in :gh:`108191`.)
typing
------
* Add :func:`typing.get_protocol_members` to return the set of members
defining a :class:`typing.Protocol`. Add :func:`typing.is_protocol` to
check whether a class is a :class:`typing.Protocol`. (Contributed by Jelle Zijlstra in
:gh:`104873`.)
* Add :data:`typing.ReadOnly`, a special typing construct to mark
an item of a :class:`typing.TypedDict` as read-only for type checkers.
See :pep:`705` for more details.
* Add :data:`typing.NoDefault`, a sentinel object used to represent the defaults
of some parameters in the :mod:`typing` module. (Contributed by Jelle Zijlstra in
:gh:`116126`.)
unicodedata
-----------
* The Unicode database has been updated to version 15.1.0. (Contributed by
James Gerity in :gh:`109559`.)
venv
----
* Add support for adding source control management (SCM) ignore files to a
virtual environment's directory. By default, Git is supported. This is
implemented as opt-in via the API which can be extended to support other SCMs
(:class:`venv.EnvBuilder` and :func:`venv.create`), and opt-out via the CLI
(using ``--without-scm-ignore-files``). (Contributed by Brett Cannon in
:gh:`108125`.)
warnings
--------
* The new :func:`warnings.deprecated` decorator provides a way to communicate
deprecations to :term:`static type checkers <static type checker>` and
to warn on usage of deprecated classes and functions. A runtime deprecation
warning may also be emitted when a decorated function or class is used at runtime.
See :pep:`702`. (Contributed by Jelle Zijlstra in :gh:`104003`.)
xml.etree.ElementTree
---------------------
* Add the :meth:`!close` method for the iterator returned by
:func:`~xml.etree.ElementTree.iterparse` for explicit cleaning up.
(Contributed by Serhiy Storchaka in :gh:`69893`.)
zipimport
---------
* Gains support for ZIP64 format files. Everybody loves huge code right?
(Contributed by Tim Hatch in :gh:`94146`.)
.. Add improved modules above alphabetically, not here at the end.
Optimizations
=============
* :func:`textwrap.indent` is now ~30% faster than before for large input.
(Contributed by Inada Naoki in :gh:`107369`.)
* The :mod:`subprocess` module uses :func:`os.posix_spawn` in more situations
including the default where ``close_fds=True`` on many modern platforms. This
should provide a noteworthy performance increase launching processes on
FreeBSD and Solaris. See the :ref:`subprocess <whatsnew313-subprocess>`
section above for details.
(Contributed by Jakub Kulik in :gh:`113117`.)
* Several standard library modules have had their import times significantly
improved. For example, the import time of the :mod:`typing` module has been
reduced by around a third by removing dependencies on :mod:`re` and
:mod:`contextlib`. Other modules to enjoy import-time speedups include
:mod:`importlib.metadata`, :mod:`threading`, :mod:`enum`, :mod:`functools`
and :mod:`email.utils`.
(Contributed by Alex Waygood, Shantanu Jain, Adam Turner, Daniel Hollas and
others in :gh:`109653`.)
Removed Modules And APIs
========================
.. _whatsnew313-pep594:
PEP 594: dead batteries (and other module removals)
---------------------------------------------------
* :pep:`594` removed 19 modules from the standard library,
deprecated in Python 3.11:
* :mod:`!aifc`.
(Contributed by Victor Stinner in :gh:`104773`.)
* :mod:`!audioop`.
(Contributed by Victor Stinner in :gh:`104773`.)
* :mod:`!chunk`.
(Contributed by Victor Stinner in :gh:`104773`.)
* :mod:`!cgi` and :mod:`!cgitb`.
* ``cgi.FieldStorage`` can typically be replaced with
:func:`urllib.parse.parse_qsl` for ``GET`` and ``HEAD`` requests,
and the :mod:`email.message` module or :pypi:`multipart`
PyPI project for ``POST`` and ``PUT``.
* ``cgi.parse()`` can be replaced by calling :func:`urllib.parse.parse_qs`
directly on the desired query string, except for ``multipart/form-data``
input, which can be handled as described for ``cgi.parse_multipart()``.
* ``cgi.parse_header()`` can be replaced with the functionality in the
:mod:`email` package, which implements the same MIME RFCs. For example,
with :class:`email.message.EmailMessage`::
from email.message import EmailMessage
msg = EmailMessage()
msg['content-type'] = 'application/json; charset="utf8"'
main, params = msg.get_content_type(), msg['content-type'].params
* ``cgi.parse_multipart()`` can be replaced with the functionality in the
:mod:`email` package (e.g. :class:`email.message.EmailMessage` and
:class:`email.message.Message`) which implements the same MIME RFCs, or
with the :pypi:`multipart` PyPI project.
(Contributed by Victor Stinner in :gh:`104773`.)
* :mod:`!crypt` module and its private :mod:`!_crypt` extension.
The :mod:`hashlib` module is a potential replacement for certain use cases.
Otherwise, the following PyPI projects can be used:
* :pypi:`bcrypt`:
Modern password hashing for your software and your servers.
* :pypi:`passlib`:
Comprehensive password hashing framework supporting over 30 schemes.
* :pypi:`argon2-cffi`:
The secure Argon2 password hashing algorithm.
* :pypi:`legacycrypt`:
:mod:`ctypes` wrapper to the POSIX crypt library call and associated functionality.
* :pypi:`crypt_r`:
Fork of the :mod:`!crypt` module, wrapper to the :manpage:`crypt_r(3)` library
call and associated functionality.
(Contributed by Victor Stinner in :gh:`104773`.)
* :mod:`!imghdr`: use the projects :pypi:`filetype`,
:pypi:`puremagic`, or :pypi:`python-magic` instead.
The ``puremagic.what()`` function can be used to replace
the ``imghdr.what()`` function for all file formats that
were supported by ``imghdr``.
(Contributed by Victor Stinner in :gh:`104773`.)
* :mod:`!mailcap`.
The :mod:`mimetypes` module provides an alternative.
(Contributed by Victor Stinner in :gh:`104773`.)
* :mod:`!msilib`.
(Contributed by Zachary Ware in :gh:`104773`.)
* :mod:`!nis`.
(Contributed by Victor Stinner in :gh:`104773`.)
* :mod:`!nntplib`:
the :pypi:`nntplib` PyPI project can be used instead.
(Contributed by Victor Stinner in :gh:`104773`.)
* :mod:`!ossaudiodev`: use the
`pygame project <https://www.pygame.org/>`_ for audio playback.
(Contributed by Victor Stinner in :gh:`104780`.)
* :mod:`!pipes`: use the :mod:`subprocess` module instead.
(Contributed by Victor Stinner in :gh:`104773`.)
* :mod:`!sndhdr`: use the projects :pypi:`filetype`,
:pypi:`puremagic`, or :pypi:`python-magic` instead.
(Contributed by Victor Stinner in :gh:`104773`.)
* :mod:`!spwd`:
the :pypi:`python-pam` project can be used instead.
(Contributed by Victor Stinner in :gh:`104773`.)
* :mod:`!sunau`.
(Contributed by Victor Stinner in :gh:`104773`.)
* :mod:`!telnetlib`, use the projects :pypi:`telnetlib3` or
:pypi:`Exscript` instead.
(Contributed by Victor Stinner in :gh:`104773`.)
* :mod:`!uu`: the :mod:`base64` module is a modern alternative.
(Contributed by Victor Stinner in :gh:`104773`.)
* :mod:`!xdrlib`.
(Contributed by Victor Stinner in :gh:`104773`.)
* Remove the ``2to3`` program and the :mod:`!lib2to3` module,
deprecated in Python 3.11.
(Contributed by Victor Stinner in :gh:`104780`.)
* Remove the :mod:`!tkinter.tix` module, deprecated in Python 3.6. The
third-party Tix library which the module wrapped is unmaintained.
(Contributed by Zachary Ware in :gh:`75552`.)
configparser
------------
* Remove the undocumented :class:`!configparser.LegacyInterpolation` class,
deprecated in the docstring since Python 3.2,
and with a deprecation warning since Python 3.11.
(Contributed by Hugo van Kemenade in :gh:`104886`.)
importlib
---------
* Remove deprecated :meth:`~object.__getitem__` access for
:class:`!importlib.metadata.EntryPoint` objects.
(Contributed by Jason R. Coombs in :gh:`113175`.)
locale
------
* Remove ``locale.resetlocale()`` function deprecated in Python 3.11:
use ``locale.setlocale(locale.LC_ALL, "")`` instead.
(Contributed by Victor Stinner in :gh:`104783`.)
pathlib
-------
* Remove support for using :class:`pathlib.Path` objects as context managers.
This functionality was deprecated and made a no-op in Python 3.9.
re
--
* Remove undocumented, never working, and deprecated ``re.template`` function
and ``re.TEMPLATE`` flag (and ``re.T`` alias).
(Contributed by Serhiy Storchaka and Nikita Sobolev in :gh:`105687`.)
turtle
------
* Remove the :meth:`!turtle.RawTurtle.settiltangle` method,
deprecated in docs since Python 3.1
and with a deprecation warning since Python 3.11.
(Contributed by Hugo van Kemenade in :gh:`104876`.)
typing
------
* Namespaces ``typing.io`` and ``typing.re``, deprecated in Python 3.8,
are now removed. The items in those namespaces can be imported directly
from :mod:`typing`. (Contributed by Sebastian Rittau in :gh:`92871`.)
* Remove support for the keyword-argument method of creating
:class:`typing.TypedDict` types, deprecated in Python 3.11.
(Contributed by Tomas Roun in :gh:`104786`.)
unittest
--------
* Remove the following :mod:`unittest` functions, deprecated in Python 3.11:
* :func:`!unittest.findTestCases`
* :func:`!unittest.makeSuite`
* :func:`!unittest.getTestCaseNames`
Use :class:`~unittest.TestLoader` methods instead:
* :meth:`unittest.TestLoader.loadTestsFromModule`
* :meth:`unittest.TestLoader.loadTestsFromTestCase`
* :meth:`unittest.TestLoader.getTestCaseNames`
(Contributed by Hugo van Kemenade in :gh:`104835`.)
* Remove the untested and undocumented :meth:`!unittest.TestProgram.usageExit`
method, deprecated in Python 3.11.
(Contributed by Hugo van Kemenade in :gh:`104992`.)
urllib
------
* Remove *cafile*, *capath* and *cadefault* parameters of the
:func:`urllib.request.urlopen` function, deprecated in Python 3.6: pass
the *context* parameter instead. Use
:meth:`ssl.SSLContext.load_cert_chain` to load specific certificates, or
let :func:`ssl.create_default_context` select the system's trusted CA
certificates for you.
(Contributed by Victor Stinner in :gh:`105382`.)
webbrowser
----------
* Remove the untested and undocumented :mod:`webbrowser` :class:`!MacOSX` class,
deprecated in Python 3.11.
Use the :class:`!MacOSXOSAScript` class (introduced in Python 3.2) instead.
(Contributed by Hugo van Kemenade in :gh:`104804`.)
* Remove deprecated ``webbrowser.MacOSXOSAScript._name`` attribute.
Use :attr:`webbrowser.MacOSXOSAScript.name <webbrowser.controller.name>`
attribute instead.
(Contributed by Nikita Sobolev in :gh:`105546`.)
New Deprecations
================
* Removed chained :class:`classmethod` descriptors (introduced in
:gh:`63272`). This can no longer be used to wrap other descriptors
such as :class:`property`. The core design of this feature was flawed
and caused a number of downstream problems. To "pass-through" a
:class:`classmethod`, consider using the :attr:`!__wrapped__`
attribute that was added in Python 3.10. (Contributed by Raymond
Hettinger in :gh:`89519`.)
* :mod:`array`: :mod:`array`'s ``'u'`` format code, deprecated in docs since Python 3.3,
emits :exc:`DeprecationWarning` since 3.13
and will be removed in Python 3.16.
Use the ``'w'`` format code instead.
(Contributed by Hugo van Kemenade in :gh:`80480`.)
* :mod:`ctypes`: Deprecate undocumented :func:`!ctypes.SetPointerType`
function. :term:`Soft-deprecate <soft deprecated>` the :func:`ctypes.ARRAY`
function in favor of multiplication.
(Contributed by Victor Stinner in :gh:`105733`.)
* :mod:`decimal`: Deprecate non-standard format specifier "N" for
:class:`decimal.Decimal`.
It was not documented and only supported in the C implementation.
(Contributed by Serhiy Storchaka in :gh:`89902`.)
* :mod:`dis`: The ``dis.HAVE_ARGUMENT`` separator is deprecated. Check
membership in :data:`~dis.hasarg` instead.
(Contributed by Irit Katriel in :gh:`109319`.)
* :ref:`frame-objects`:
Calling :meth:`frame.clear` on a suspended frame raises :exc:`RuntimeError`
(as has always been the case for an executing frame).
(Contributed by Irit Katriel in :gh:`79932`.)
* :mod:`getopt` and :mod:`optparse` modules: They are now
:term:`soft deprecated`: the :mod:`argparse` module should be used for new projects.
Previously, the :mod:`optparse` module was already deprecated, its removal
was not scheduled, and no warnings was emitted: so there is no change in
practice.
(Contributed by Victor Stinner in :gh:`106535`.)
* :mod:`gettext`: Emit deprecation warning for non-integer numbers in
:mod:`gettext` functions and methods that consider plural forms even if the
translation was not found.
(Contributed by Serhiy Storchaka in :gh:`88434`.)
* :mod:`glob`: The undocumented :func:`!glob.glob0` and :func:`!glob.glob1`
functions are deprecated. Use :func:`glob.glob` and pass a directory to its
*root_dir* argument instead.
(Contributed by Barney Gale in :gh:`117337`.)
* :mod:`http.server`: :class:`http.server.CGIHTTPRequestHandler` now emits a
:exc:`DeprecationWarning` as it will be removed in 3.15. Process-based CGI
HTTP servers have been out of favor for a very long time. This code was
outdated, unmaintained, and rarely used. It has a high potential for both
security and functionality bugs. This includes removal of the ``--cgi``
flag to the ``python -m http.server`` command line in 3.15.
* :mod:`mimetypes`: Passing file path instead of URL in :func:`~mimetypes.guess_type` is
:term:`soft deprecated`. Use :func:`~mimetypes.guess_file_type` instead.
(Contributed by Serhiy Storchaka in :gh:`66543`.)
* :mod:`re`: Passing optional arguments *maxsplit*, *count* and *flags* in module-level
functions :func:`re.split`, :func:`re.sub` and :func:`re.subn` as positional
arguments is now deprecated. In future Python versions these parameters will be
:ref:`keyword-only <keyword-only_parameter>`.
(Contributed by Serhiy Storchaka in :gh:`56166`.)
* :mod:`pathlib`:
:meth:`pathlib.PurePath.is_reserved` is deprecated and scheduled for
removal in Python 3.15. Use :func:`os.path.isreserved` to detect reserved
paths on Windows.
* :mod:`platform`:
:func:`~platform.java_ver` is deprecated and will be removed in 3.15.
It was largely untested, had a confusing API,
and was only useful for Jython support.
(Contributed by Nikita Sobolev in :gh:`116349`.)
* :mod:`pydoc`: Deprecate undocumented :func:`!pydoc.ispackage` function.
(Contributed by Zackery Spytz in :gh:`64020`.)
* :mod:`sqlite3`: Passing more than one positional argument to
:func:`sqlite3.connect` and the :class:`sqlite3.Connection` constructor is
deprecated. The remaining parameters will become keyword-only in Python 3.15.
Deprecate passing name, number of arguments, and the callable as keyword
arguments for the following :class:`sqlite3.Connection` APIs:
* :meth:`~sqlite3.Connection.create_function`
* :meth:`~sqlite3.Connection.create_aggregate`
Deprecate passing the callback callable by keyword for the following
:class:`sqlite3.Connection` APIs:
* :meth:`~sqlite3.Connection.set_authorizer`
* :meth:`~sqlite3.Connection.set_progress_handler`
* :meth:`~sqlite3.Connection.set_trace_callback`
The affected parameters will become positional-only in Python 3.15.
(Contributed by Erlend E. Aasland in :gh:`107948` and :gh:`108278`.)
* :mod:`sys`: :func:`sys._enablelegacywindowsfsencoding` function.
Replace it with the :envvar:`PYTHONLEGACYWINDOWSFSENCODING` environment variable.
(Contributed by Inada Naoki in :gh:`73427`.)
* :mod:`tarfile`:
The undocumented and unused ``tarfile`` attribute of :class:`tarfile.TarFile`
is deprecated and scheduled for removal in Python 3.16.
* :mod:`traceback`: The field *exc_type* of :class:`traceback.TracebackException`
is deprecated. Use *exc_type_str* instead.
* :mod:`typing`:
* Creating a :class:`typing.NamedTuple` class using keyword arguments to denote
the fields (``NT = NamedTuple("NT", x=int, y=int)``) is deprecated, and will
be disallowed in Python 3.15. Use the class-based syntax or the functional
syntax instead. (Contributed by Alex Waygood in :gh:`105566`.)
* When using the functional syntax to create a :class:`typing.NamedTuple`
class or a :class:`typing.TypedDict` class, failing to pass a value to the
'fields' parameter (``NT = NamedTuple("NT")`` or ``TD = TypedDict("TD")``) is
deprecated. Passing ``None`` to the 'fields' parameter
(``NT = NamedTuple("NT", None)`` or ``TD = TypedDict("TD", None)``) is also
deprecated. Both will be disallowed in Python 3.15. To create a NamedTuple
class with 0 fields, use ``class NT(NamedTuple): pass`` or
``NT = NamedTuple("NT", [])``. To create a TypedDict class with 0 fields, use
``class TD(TypedDict): pass`` or ``TD = TypedDict("TD", {})``.
(Contributed by Alex Waygood in :gh:`105566` and :gh:`105570`.)
* :func:`typing.no_type_check_decorator` is deprecated, and scheduled for
removal in Python 3.15. After eight years in the :mod:`typing` module, it
has yet to be supported by any major type checkers.
(Contributed by Alex Waygood in :gh:`106309`.)
* :data:`typing.AnyStr` is deprecated. In Python 3.16, it will be removed from
``typing.__all__``, and a :exc:`DeprecationWarning` will be emitted when it
is imported or accessed. It will be removed entirely in Python 3.18. Use
the new :ref:`type parameter syntax <type-params>` instead.
(Contributed by Michael The in :gh:`107116`.)
* :ref:`user-defined-funcs`:
Assignment to a function's :attr:`~function.__code__` attribute where the new code
object's type does not match the function's type, is deprecated. The
different types are: plain function, generator, async generator and
coroutine.
(Contributed by Irit Katriel in :gh:`81137`.)
* :mod:`wave`: Deprecate the ``getmark()``, ``setmark()`` and ``getmarkers()``
methods of the :class:`wave.Wave_read` and :class:`wave.Wave_write` classes.
They will be removed in Python 3.15.
(Contributed by Victor Stinner in :gh:`105096`.)
.. Add deprecations above alphabetically, not here at the end.
.. include:: ../deprecations/pending-removal-in-3.14.rst
.. include:: ../deprecations/pending-removal-in-3.15.rst
.. include:: ../deprecations/pending-removal-in-3.16.rst
.. include:: ../deprecations/pending-removal-in-future.rst
CPython Bytecode Changes
========================
* The oparg of ``YIELD_VALUE`` is now ``1`` if the yield is part of a
yield-from or await, and ``0`` otherwise. The oparg of ``RESUME`` was
changed to add a bit indicating whether the except-depth is 1, which
is needed to optimize closing of generators.
(Contributed by Irit Katriel in :gh:`111354`.)
C API Changes
=============
New Features
------------
* You no longer have to define the ``PY_SSIZE_T_CLEAN`` macro before including
:file:`Python.h` when using ``#`` formats in
:ref:`format codes <arg-parsing-string-and-buffers>`.
APIs accepting the format codes always use ``Py_ssize_t`` for ``#`` formats.
(Contributed by Inada Naoki in :gh:`104922`.)
* The *keywords* parameter of :c:func:`PyArg_ParseTupleAndKeywords` and
:c:func:`PyArg_VaParseTupleAndKeywords` now has type :c:expr:`char * const *`
in C and :c:expr:`const char * const *` in C++, instead of :c:expr:`char **`.
It makes these functions compatible with arguments of type
:c:expr:`const char * const *`, :c:expr:`const char **` or
:c:expr:`char * const *` in C++ and :c:expr:`char * const *` in C
without an explicit type cast.
This can be overridden with the :c:macro:`PY_CXX_CONST` macro.
(Contributed by Serhiy Storchaka in :gh:`65210`.)
* Add :c:func:`PyImport_AddModuleRef`: similar to
:c:func:`PyImport_AddModule`, but return a :term:`strong reference` instead
of a :term:`borrowed reference`.
(Contributed by Victor Stinner in :gh:`105922`.)
* Add :c:func:`PyWeakref_GetRef` function: similar to
:c:func:`PyWeakref_GetObject` but returns a :term:`strong reference`, or
``NULL`` if the referent is no longer live.
(Contributed by Victor Stinner in :gh:`105927`.)
* Add :c:func:`PyObject_GetOptionalAttr` and
:c:func:`PyObject_GetOptionalAttrString`, variants of
:c:func:`PyObject_GetAttr` and :c:func:`PyObject_GetAttrString` which
don't raise :exc:`AttributeError` if the attribute is not found.
These variants are more convenient and faster if the missing attribute
should not be treated as a failure.
(Contributed by Serhiy Storchaka in :gh:`106521`.)
* Add :c:func:`PyMapping_GetOptionalItem` and
:c:func:`PyMapping_GetOptionalItemString`: variants of
:c:func:`PyObject_GetItem` and :c:func:`PyMapping_GetItemString` which don't
raise :exc:`KeyError` if the key is not found.
These variants are more convenient and faster if the missing key should not
be treated as a failure.
(Contributed by Serhiy Storchaka in :gh:`106307`.)
* Add fixed variants of functions which silently ignore errors:
- :c:func:`PyObject_HasAttrWithError` replaces :c:func:`PyObject_HasAttr`.
- :c:func:`PyObject_HasAttrStringWithError` replaces :c:func:`PyObject_HasAttrString`.
- :c:func:`PyMapping_HasKeyWithError` replaces :c:func:`PyMapping_HasKey`.
- :c:func:`PyMapping_HasKeyStringWithError` replaces :c:func:`PyMapping_HasKeyString`.
New functions return not only ``1`` for true and ``0`` for false, but also
``-1`` for error.
(Contributed by Serhiy Storchaka in :gh:`108511`.)
* If Python is built in :ref:`debug mode <debug-build>` or :option:`with
assertions <--with-assertions>`, :c:func:`PyTuple_SET_ITEM` and
:c:func:`PyList_SET_ITEM` now check the index argument with an assertion.
(Contributed by Victor Stinner in :gh:`106168`.)
* Add :c:func:`PyModule_Add` function: similar to
:c:func:`PyModule_AddObjectRef` and :c:func:`PyModule_AddObject` but
always steals a reference to the value.
(Contributed by Serhiy Storchaka in :gh:`86493`.)
* Add :c:func:`PyDict_GetItemRef` and :c:func:`PyDict_GetItemStringRef`
functions: similar to :c:func:`PyDict_GetItemWithError` but returning a
:term:`strong reference` instead of a :term:`borrowed reference`. Moreover,
these functions return -1 on error and so checking ``PyErr_Occurred()`` is
not needed.
(Contributed by Victor Stinner in :gh:`106004`.)
* Added :c:func:`PyDict_SetDefaultRef`, which is similar to
:c:func:`PyDict_SetDefault` but returns a :term:`strong reference` instead of
a :term:`borrowed reference`. This function returns ``-1`` on error, ``0`` on
insertion, and ``1`` if the key was already present in the dictionary.
(Contributed by Sam Gross in :gh:`112066`.)
* Add :c:func:`PyDict_ContainsString` function: same as
:c:func:`PyDict_Contains`, but *key* is specified as a :c:expr:`const char*`
UTF-8 encoded bytes string, rather than a :c:expr:`PyObject*`.
(Contributed by Victor Stinner in :gh:`108314`.)
* Added :c:func:`PyList_GetItemRef` function: similar to
:c:func:`PyList_GetItem` but returns a :term:`strong reference` instead of
a :term:`borrowed reference`.
* Add :c:func:`Py_IsFinalizing` function: check if the main Python interpreter is
:term:`shutting down <interpreter shutdown>`.
(Contributed by Victor Stinner in :gh:`108014`.)
* Add :c:func:`PyLong_AsInt` function: similar to :c:func:`PyLong_AsLong`, but
store the result in a C :c:expr:`int` instead of a C :c:expr:`long`.
Previously, it was known as the private function :c:func:`!_PyLong_AsInt`
(with an underscore prefix).
(Contributed by Victor Stinner in :gh:`108014`.)
* Python built with :file:`configure` :option:`--with-trace-refs` (tracing
references) now supports the :ref:`Limited API <limited-c-api>`.
(Contributed by Victor Stinner in :gh:`108634`.)
* Add :c:func:`PyObject_VisitManagedDict` and
:c:func:`PyObject_ClearManagedDict` functions which must be called by the
traverse and clear functions of a type using
:c:macro:`Py_TPFLAGS_MANAGED_DICT` flag. The `pythoncapi-compat project
<https://github.com/python/pythoncapi-compat/>`__ can be used to get these
functions on Python 3.11 and 3.12.
(Contributed by Victor Stinner in :gh:`107073`.)
* Add :c:func:`PyUnicode_EqualToUTF8AndSize` and :c:func:`PyUnicode_EqualToUTF8`
functions: compare Unicode object with a :c:expr:`const char*` UTF-8 encoded
string and return true (``1``) if they are equal, or false (``0``) otherwise.
These functions do not raise exceptions.
(Contributed by Serhiy Storchaka in :gh:`110289`.)
* Add :c:func:`PyThreadState_GetUnchecked()` function: similar to
:c:func:`PyThreadState_Get()`, but don't kill the process with a fatal error
if it is NULL. The caller is responsible to check if the result is NULL.
Previously, the function was private and known as
``_PyThreadState_UncheckedGet()``.
(Contributed by Victor Stinner in :gh:`108867`.)
* Add :c:func:`PySys_AuditTuple` function: similar to :c:func:`PySys_Audit`,
but pass event arguments as a Python :class:`tuple` object.
(Contributed by Victor Stinner in :gh:`85283`.)
* :c:func:`PyArg_ParseTupleAndKeywords` now supports non-ASCII keyword
parameter names.
(Contributed by Serhiy Storchaka in :gh:`110815`.)
* Add :c:func:`PyMem_RawMalloc`, :c:func:`PyMem_RawCalloc`,
:c:func:`PyMem_RawRealloc` and :c:func:`PyMem_RawFree` to the limited C API
(version 3.13).
(Contributed by Victor Stinner in :gh:`85283`.)
* Add :c:func:`PySys_Audit` and :c:func:`PySys_AuditTuple` functions to the
limited C API.
(Contributed by Victor Stinner in :gh:`85283`.)
* Add :c:func:`PyErr_FormatUnraisable` function: similar to
:c:func:`PyErr_WriteUnraisable`, but allow customizing the warning message.
(Contributed by Serhiy Storchaka in :gh:`108082`.)
* Add :c:func:`PyList_Extend` and :c:func:`PyList_Clear` functions: similar to
Python ``list.extend()`` and ``list.clear()`` methods.
(Contributed by Victor Stinner in :gh:`111138`.)
* Add :c:func:`PyDict_Pop` and :c:func:`PyDict_PopString` functions: remove a
key from a dictionary and optionally return the removed value. This is
similar to :meth:`dict.pop`, but without the default value and not raising
:exc:`KeyError` if the key is missing.
(Contributed by Stefan Behnel and Victor Stinner in :gh:`111262`.)
* Add :c:func:`Py_HashPointer` function to hash a pointer.
(Contributed by Victor Stinner in :gh:`111545`.)
* Add :c:func:`PyObject_GenericHash` function that implements the default
hashing function of a Python object.
(Contributed by Serhiy Storchaka in :gh:`113024`.)
* Add PyTime C API:
* :c:type:`PyTime_t` type.
* :c:var:`PyTime_MIN` and :c:var:`PyTime_MAX` constants.
* Add functions:
* :c:func:`PyTime_AsSecondsDouble`.
* :c:func:`PyTime_Monotonic`.
* :c:func:`PyTime_MonotonicRaw`.
* :c:func:`PyTime_PerfCounter`.
* :c:func:`PyTime_PerfCounterRaw`.
* :c:func:`PyTime_Time`.
* :c:func:`PyTime_TimeRaw`.
(Contributed by Victor Stinner and Petr Viktorin in :gh:`110850`.)
* Add :c:func:`PyLong_AsNativeBytes`, :c:func:`PyLong_FromNativeBytes` and
:c:func:`PyLong_FromUnsignedNativeBytes` functions to simplify converting
between native integer types and Python :class:`int` objects.
(Contributed by Steve Dower in :gh:`111140`.)
* Add :c:func:`PyType_GetFullyQualifiedName` function to get the type's fully
qualified name. Equivalent to ``f"{type.__module__}.{type.__qualname__}"``,
or ``type.__qualname__`` if ``type.__module__`` is not a string or is equal
to ``"builtins"``.
(Contributed by Victor Stinner in :gh:`111696`.)
* Add :c:func:`PyType_GetModuleName` function to get the type's module name.
Equivalent to getting the ``type.__module__`` attribute.
(Contributed by Eric Snow and Victor Stinner in :gh:`111696`.)
* Add support for ``%T``, ``%#T``, ``%N`` and ``%#N`` formats to
:c:func:`PyUnicode_FromFormat`: format the fully qualified name of an object
type and of a type: call :c:func:`PyType_GetModuleName`. See :pep:`737` for
more information.
(Contributed by Victor Stinner in :gh:`111696`.)
* Add :c:func:`Py_GetConstant` and :c:func:`Py_GetConstantBorrowed` functions
to get constants. For example, ``Py_GetConstant(Py_CONSTANT_ZERO)`` returns a
:term:`strong reference` to the constant zero.
(Contributed by Victor Stinner in :gh:`115754`.)
* Add :c:func:`PyType_GetModuleByDef` to the limited C API
(Contributed by Victor Stinner in :gh:`116936`.)
* Add two new functions to the C-API, :c:func:`PyRefTracer_SetTracer` and
:c:func:`PyRefTracer_GetTracer`, that allows to track object creation and
destruction the same way the :mod:`tracemalloc` module does. (Contributed
by Pablo Galindo in :gh:`93502`.)
* Add :c:func:`PyEval_GetFrameBuiltins`, :c:func:`PyEval_GetFrameGlobals`, and
:c:func:`PyEval_GetFrameLocals` to the C API. These replacements for
:c:func:`PyEval_GetBuiltins`, :c:func:`PyEval_GetGlobals`, and
:c:func:`PyEval_GetLocals` return :term:`strong references <strong reference>`
rather than borrowed references. (Added as part of :pep:`667`.)
* Add :c:type:`PyMutex` API, a lightweight mutex that occupies a single byte.
The :c:func:`PyMutex_Lock` function will release the GIL (if currently held)
if the operation needs to block.
(Contributed by Sam Gross in :gh:`108724`.)
Build Changes
=============
* The :file:`configure` option :option:`--with-system-libmpdec` now defaults
to ``yes``. The bundled copy of ``libmpdecimal`` will be removed in Python
3.15.
* Autoconf 2.71 and aclocal 1.16.4 are now required to regenerate
the :file:`configure` script.
(Contributed by Christian Heimes in :gh:`89886`.)
* SQLite 3.15.2 or newer is required to build the :mod:`sqlite3` extension module.
(Contributed by Erlend Aasland in :gh:`105875`.)
* Python built with :file:`configure` :option:`--with-trace-refs` (tracing
references) is now ABI compatible with the Python release build and
:ref:`debug build <debug-build>`.
(Contributed by Victor Stinner in :gh:`108634`.)
* Building CPython now requires a compiler with support for the C11 atomic
library, GCC built-in atomic functions, or MSVC interlocked intrinsics.
* The ``errno``, ``fcntl``, ``grp``, ``md5``, ``pwd``, ``resource``,
``termios``, ``winsound``,
``_ctypes_test``, ``_multiprocessing.posixshmem``, ``_scproxy``, ``_stat``,
``_statistics``, ``_testconsole``, ``_testimportmultiple`` and ``_uuid``
C extensions are now built with the :ref:`limited C API <limited-c-api>`.
(Contributed by Victor Stinner in :gh:`85283`.)
* ``wasm32-wasi`` is now a :pep:`11` tier 2 platform.
(Contributed by Brett Cannon in :gh:`115192`.)
* ``wasm32-emscripten`` is no longer a :pep:`11` supported platform.
(Contributed by Brett Cannon in :gh:`115192`.)
* Python now bundles the `mimalloc library <https://github.com/microsoft/mimalloc>`__.
It is licensed under the MIT license; see :ref:`mimalloc license <mimalloc-license>`.
The bundled mimalloc has custom changes, see :gh:`113141` for details.
(Contributed by Dino Viehland in :gh:`109914`.)
* On POSIX systems, the pkg-config (``.pc``) filenames now include the ABI
flags. For example, the free-threaded build generates ``python-3.13t.pc``
and the debug build generates ``python-3.13d.pc``.
Porting to Python 3.13
======================
This section lists previously described changes and other bugfixes
that may require changes to your code.
Changes in the Python API
-------------------------
* An :exc:`OSError` is now raised by :func:`getpass.getuser` for any failure to
retrieve a username, instead of :exc:`ImportError` on non-Unix platforms or
:exc:`KeyError` on Unix platforms where the password database is empty.
* The :mod:`threading` module now expects the :mod:`!_thread` module to have
an ``_is_main_interpreter`` attribute. It is a function with no
arguments that returns ``True`` if the current interpreter is the
main interpreter.
Any library or application that provides a custom ``_thread`` module
must provide ``_is_main_interpreter()``, just like the module's
other "private" attributes.
(See :gh:`112826`.)
* :class:`mailbox.Maildir` now ignores files with a leading dot.
(Contributed by Zackery Spytz in :gh:`65559`.)
* :meth:`pathlib.Path.glob` and :meth:`~pathlib.Path.rglob` now return both
files and directories if a pattern that ends with "``**``" is given, rather
than directories only. Users may add a trailing slash to match only
directories.
* The value of the :attr:`!mode` attribute of :class:`gzip.GzipFile` was
changed from integer (``1`` or ``2``) to string (``'rb'`` or ``'wb'``).
The value of the :attr:`!mode` attribute of the readable file-like object
returned by :meth:`zipfile.ZipFile.open` was changed from ``'r'`` to ``'rb'``.
(Contributed by Serhiy Storchaka in :gh:`115961`.)
* :class:`functools.partial` now emits a :exc:`FutureWarning` when it is
used as a method.
Its behavior will be changed in future Python versions.
Wrap it in :func:`staticmethod` if you want to preserve the old behavior.
(Contributed by Serhiy Storchaka in :gh:`121027`.)
.. _pep667-porting-notes-py:
* Calling :func:`locals` in an :term:`optimized scope` now produces an
independent snapshot on each call, and hence no longer implicitly updates
previously returned references. Obtaining the legacy CPython behaviour now
requires explicit calls to update the initially returned dictionary with the
results of subsequent calls to ``locals()``. Code execution functions that
implicitly target ``locals()`` (such as ``exec`` and ``eval``) must be
passed an explicit namespace to access their results in an optimized scope.
(Changed as part of :pep:`667`.)
* Calling :func:`locals` from a comprehension at module or class scope
(including via ``exec`` or ``eval``) once more behaves as if the comprehension
were running as an independent nested function (i.e. the local variables from
the containing scope are not included). In Python 3.12, this had changed
to include the local variables from the containing scope when implementing
:pep:`709`. (Changed as part of :pep:`667`.)
* Accessing :attr:`FrameType.f_locals <frame.f_locals>` in an
:term:`optimized scope` now returns a write-through proxy rather than a
snapshot that gets updated at ill-specified times. If a snapshot is desired,
it must be created explicitly with ``dict`` or the proxy's ``.copy()`` method.
(Changed as part of :pep:`667`.)
Changes in the C API
--------------------
* ``Python.h`` no longer includes the ``<ieeefp.h>`` standard header. It was
included for the ``finite()`` function which is now provided by the
``<math.h>`` header. It should now be included explicitly if needed. Remove
also the ``HAVE_IEEEFP_H`` macro.
(Contributed by Victor Stinner in :gh:`108765`.)
* ``Python.h`` no longer includes these standard header files: ``<time.h>``,
``<sys/select.h>`` and ``<sys/time.h>``. If needed, they should now be
included explicitly. For example, ``<time.h>`` provides the ``clock()`` and
``gmtime()`` functions, ``<sys/select.h>`` provides the ``select()``
function, and ``<sys/time.h>`` provides the ``futimes()``, ``gettimeofday()``
and ``setitimer()`` functions.
(Contributed by Victor Stinner in :gh:`108765`.)
* On Windows, ``Python.h`` no longer includes the ``<stddef.h>`` standard
header file. If needed, it should now be included explicitly. For example, it
provides ``offsetof()`` function, and ``size_t`` and ``ptrdiff_t`` types.
Including ``<stddef.h>`` explicitly was already needed by all other
platforms, the ``HAVE_STDDEF_H`` macro is only defined on Windows.
(Contributed by Victor Stinner in :gh:`108765`.)
* If the :c:macro:`Py_LIMITED_API` macro is defined, :c:macro:`!Py_BUILD_CORE`,
:c:macro:`!Py_BUILD_CORE_BUILTIN` and :c:macro:`!Py_BUILD_CORE_MODULE` macros
are now undefined by ``<Python.h>``.
(Contributed by Victor Stinner in :gh:`85283`.)
* The old trashcan macros ``Py_TRASHCAN_SAFE_BEGIN`` and ``Py_TRASHCAN_SAFE_END``
were removed. They should be replaced by the new macros ``Py_TRASHCAN_BEGIN``
and ``Py_TRASHCAN_END``.
A ``tp_dealloc`` function that has the old macros, such as::
static void
mytype_dealloc(mytype *p)
{
PyObject_GC_UnTrack(p);
Py_TRASHCAN_SAFE_BEGIN(p);
...
Py_TRASHCAN_SAFE_END
}
should migrate to the new macros as follows::
static void
mytype_dealloc(mytype *p)
{
PyObject_GC_UnTrack(p);
Py_TRASHCAN_BEGIN(p, mytype_dealloc)
...
Py_TRASHCAN_END
}
Note that ``Py_TRASHCAN_BEGIN`` has a second argument which
should be the deallocation function it is in. The new macros were
added in Python 3.8 and the old macros were deprecated in Python 3.11.
(Contributed by Irit Katriel in :gh:`105111`.)
* Functions :c:func:`PyDict_GetItem`, :c:func:`PyDict_GetItemString`,
:c:func:`PyMapping_HasKey`, :c:func:`PyMapping_HasKeyString`,
:c:func:`PyObject_HasAttr`, :c:func:`PyObject_HasAttrString`, and
:c:func:`PySys_GetObject`, which clear all errors which occurred when calling
them, now report them using :func:`sys.unraisablehook`.
You may replace them with other functions as
recommended in the documentation.
(Contributed by Serhiy Storchaka in :gh:`106672`.)
* :c:func:`!PyCode_GetFirstFree` is an unstable API now and has been renamed
to :c:func:`PyUnstable_Code_GetFirstFree`.
(Contributed by Bogdan Romanyuk in :gh:`115781`.)
.. _pep667-porting-notes-c:
* The effects of mutating the dictionary returned from :c:func:`PyEval_GetLocals` in an
:term:`optimized scope` have changed. New dict entries added this way will now *only* be
visible to subsequent :c:func:`PyEval_GetLocals` calls in that frame, as
:c:func:`PyFrame_GetLocals`, :func:`locals`, and
:attr:`FrameType.f_locals <frame.f_locals>` no longer access the same underlying cached
dictionary. Changes made to entries for actual variable names and names added via the
write-through proxy interfaces will be overwritten on subsequent calls to
:c:func:`PyEval_GetLocals` in that frame. The recommended code update depends on how the
function was being used, so refer to the deprecation notice on the function for details.
(Changed as part of :pep:`667`.)
* Calling :c:func:`PyFrame_GetLocals` in an :term:`optimized scope` now returns a
write-through proxy rather than a snapshot that gets updated at ill-specified times.
If a snapshot is desired, it must be created explicitly (e.g. with :c:func:`PyDict_Copy`)
or by calling the new :c:func:`PyEval_GetFrameLocals` API. (Changed as part of :pep:`667`.)
* :c:func:`!PyFrame_FastToLocals` and :c:func:`!PyFrame_FastToLocalsWithError`
no longer have any effect. Calling these functions has been redundant since
Python 3.11, when :c:func:`PyFrame_GetLocals` was first introduced.
(Changed as part of :pep:`667`.)
* :c:func:`!PyFrame_LocalsToFast` no longer has any effect. Calling this function
is redundant now that :c:func:`PyFrame_GetLocals` returns a write-through proxy
for :term:`optimized scopes <optimized scope>`. (Changed as part of :pep:`667`.)
Removed C APIs
--------------
* Remove many APIs (functions, macros, variables) with names prefixed by
``_Py`` or ``_PY`` (considered as private API). If your project is affected
by one of these removals and you consider that the removed API should remain
available, please open a new issue to request a public C API and
add ``cc @vstinner`` to the issue to notify Victor Stinner.
(Contributed by Victor Stinner in :gh:`106320`.)
* Remove functions deprecated in Python 3.9:
* ``PyEval_CallObject()``, ``PyEval_CallObjectWithKeywords()``: use
:c:func:`PyObject_CallNoArgs` or :c:func:`PyObject_Call` instead.
Warning: :c:func:`PyObject_Call` positional arguments must be a
:class:`tuple` and must not be ``NULL``, keyword arguments must be a
:class:`dict` or ``NULL``, whereas removed functions checked arguments type
and accepted ``NULL`` positional and keyword arguments.
To replace ``PyEval_CallObjectWithKeywords(func, NULL, kwargs)`` with
:c:func:`PyObject_Call`, pass an empty tuple as positional arguments using
:c:func:`PyTuple_New(0) <PyTuple_New>`.
* ``PyEval_CallFunction()``: use :c:func:`PyObject_CallFunction` instead.
* ``PyEval_CallMethod()``: use :c:func:`PyObject_CallMethod` instead.
* ``PyCFunction_Call()``: use :c:func:`PyObject_Call` instead.
(Contributed by Victor Stinner in :gh:`105107`.)
* Remove old buffer protocols deprecated in Python 3.0. Use :ref:`bufferobjects` instead.
* :c:func:`!PyObject_CheckReadBuffer`: Use :c:func:`PyObject_CheckBuffer` to
test if the object supports the buffer protocol.
Note that :c:func:`PyObject_CheckBuffer` doesn't guarantee that
:c:func:`PyObject_GetBuffer` will succeed.
To test if the object is actually readable, see the next example
of :c:func:`PyObject_GetBuffer`.
* :c:func:`!PyObject_AsCharBuffer`, :c:func:`!PyObject_AsReadBuffer`:
:c:func:`PyObject_GetBuffer` and :c:func:`PyBuffer_Release` instead:
.. code-block:: c
Py_buffer view;
if (PyObject_GetBuffer(obj, &view, PyBUF_SIMPLE) < 0) {
return NULL;
}
// Use `view.buf` and `view.len` to read from the buffer.
// You may need to cast buf as `(const char*)view.buf`.
PyBuffer_Release(&view);
* :c:func:`!PyObject_AsWriteBuffer`: Use
:c:func:`PyObject_GetBuffer` and :c:func:`PyBuffer_Release` instead:
.. code-block:: c
Py_buffer view;
if (PyObject_GetBuffer(obj, &view, PyBUF_WRITABLE) < 0) {
return NULL;
}
// Use `view.buf` and `view.len` to write to the buffer.
PyBuffer_Release(&view);
(Contributed by Inada Naoki in :gh:`85275`.)
* Remove the following old functions to configure the Python initialization,
deprecated in Python 3.11:
* ``PySys_AddWarnOptionUnicode()``: use :c:member:`PyConfig.warnoptions` instead.
* ``PySys_AddWarnOption()``: use :c:member:`PyConfig.warnoptions` instead.
* ``PySys_AddXOption()``: use :c:member:`PyConfig.xoptions` instead.
* ``PySys_HasWarnOptions()``: use :c:member:`PyConfig.xoptions` instead.
* ``PySys_SetPath()``: set :c:member:`PyConfig.module_search_paths` instead.
* ``Py_SetPath()``: set :c:member:`PyConfig.module_search_paths` instead.
* ``Py_SetStandardStreamEncoding()``: set :c:member:`PyConfig.stdio_encoding`
instead, and set also maybe :c:member:`PyConfig.legacy_windows_stdio` (on
Windows).
* ``_Py_SetProgramFullPath()``: set :c:member:`PyConfig.executable` instead.
Use the new :c:type:`PyConfig` API of the :ref:`Python Initialization
Configuration <init-config>` instead (:pep:`587`), added to Python 3.8.
(Contributed by Victor Stinner in :gh:`105145`.)
* Remove ``PyEval_ThreadsInitialized()``
function, deprecated in Python 3.9. Since Python 3.7, ``Py_Initialize()``
always creates the GIL: calling ``PyEval_InitThreads()`` does nothing and
``PyEval_ThreadsInitialized()`` always returned non-zero.
(Contributed by Victor Stinner in :gh:`105182`.)
* Remove ``PyEval_AcquireLock()`` and ``PyEval_ReleaseLock()`` functions,
deprecated in Python 3.2. They didn't update the current thread state.
They can be replaced with:
* :c:func:`PyEval_SaveThread` and :c:func:`PyEval_RestoreThread`;
* low-level :c:func:`PyEval_AcquireThread` and :c:func:`PyEval_RestoreThread`;
* or :c:func:`PyGILState_Ensure` and :c:func:`PyGILState_Release`.
(Contributed by Victor Stinner in :gh:`105182`.)
* Remove private ``_PyObject_FastCall()`` function:
use ``PyObject_Vectorcall()`` which is available since Python 3.8
(:pep:`590`).
(Contributed by Victor Stinner in :gh:`106023`.)
* Remove ``cpython/pytime.h`` header file: it only contained private functions.
(Contributed by Victor Stinner in :gh:`106316`.)
* Remove ``_PyInterpreterState_Get()`` alias to
:c:func:`PyInterpreterState_Get()` which was kept for backward compatibility
with Python 3.8. The `pythoncapi-compat project
<https://github.com/python/pythoncapi-compat/>`__ can be used to get
:c:func:`PyInterpreterState_Get()` on Python 3.8 and older.
(Contributed by Victor Stinner in :gh:`106320`.)
* The :c:func:`PyModule_AddObject` function is now :term:`soft deprecated`:
:c:func:`PyModule_Add` or :c:func:`PyModule_AddObjectRef` functions should
be used instead.
(Contributed by Serhiy Storchaka in :gh:`86493`.)
* Remove undocumented ``PY_TIMEOUT_MAX`` constant from the limited C API.
(Contributed by Victor Stinner in :gh:`110014`.)
Deprecated C APIs
-----------------
* Deprecate the old ``Py_UNICODE`` and ``PY_UNICODE_TYPE`` types: use directly
the :c:type:`wchar_t` type instead. Since Python 3.3, ``Py_UNICODE`` and
``PY_UNICODE_TYPE`` are just aliases to :c:type:`wchar_t`.
(Contributed by Victor Stinner in :gh:`105156`.)
* Deprecate old Python initialization functions:
* :c:func:`PySys_ResetWarnOptions`:
clear :data:`sys.warnoptions` and :data:`!warnings.filters` instead.
* :c:func:`Py_GetExecPrefix`: get :data:`sys.exec_prefix` instead.
* :c:func:`Py_GetPath`: get :data:`sys.path` instead.
* :c:func:`Py_GetPrefix`: get :data:`sys.prefix` instead.
* :c:func:`Py_GetProgramFullPath`: get :data:`sys.executable` instead.
* :c:func:`Py_GetProgramName`: get :data:`sys.executable` instead.
* :c:func:`Py_GetPythonHome`: get :c:member:`PyConfig.home` or
:envvar:`PYTHONHOME` environment variable instead.
Functions scheduled for removal in Python 3.15.
(Contributed by Victor Stinner in :gh:`105145`.)
* Deprecate the :c:func:`PyImport_ImportModuleNoBlock` function which is just
an alias to :c:func:`PyImport_ImportModule` since Python 3.3.
Scheduled for removal in Python 3.15.
(Contributed by Victor Stinner in :gh:`105396`.)
* Deprecate the :c:func:`PyWeakref_GetObject` and
:c:func:`PyWeakref_GET_OBJECT` functions, which return a :term:`borrowed
reference`: use the new :c:func:`PyWeakref_GetRef` function instead, it
returns a :term:`strong reference`. The `pythoncapi-compat project
<https://github.com/python/pythoncapi-compat/>`__ can be used to get
:c:func:`PyWeakref_GetRef` on Python 3.12 and older.
(Contributed by Victor Stinner in :gh:`105927`.)
* Deprecate the :c:func:`PyEval_GetBuiltins`, :c:func:`PyEval_GetGlobals`, and
:c:func:`PyEval_GetLocals` functions, which return a :term:`borrowed reference`.
Refer to the deprecation notices on each function for their recommended replacements.
(Soft deprecated as part of :pep:`667`.)
.. Add deprecations above alphabetically, not here at the end.
.. include:: ../deprecations/c-api-pending-removal-in-3.14.rst
.. include:: ../deprecations/c-api-pending-removal-in-3.15.rst
.. include:: ../deprecations/c-api-pending-removal-in-future.rst
Regression Test Changes
=======================
* Python built with :file:`configure` :option:`--with-pydebug` now
supports a :option:`-X presite=package.module <-X>` command-line
option. If used, it specifies a module that should be imported early
in the lifecycle of the interpreter, before ``site.py`` is executed.
(Contributed by Łukasz Langa in :gh:`110769`.)