As we get closer to Debian Bookworm's release, I thought I'd share one change in Python 3.11 that will surely affect many people.
Python 3.11 implements the new PEP 668, Marking Python base environments
as “externally managed”1. If you use
pip regularly on Debian, it's
likely you'll eventually hit the
× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
python3-xyz, where xyz is the package you are trying to
If you wish to install a non-Debian-packaged Python package,
create a virtual environment using python3 -m venv path/to/venv.
Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
sure you have python3-full installed.
If you wish to install a non-Debian packaged Python application,
it may be easiest to use pipx install xyz, which will manage a
virtual environment for you. Make sure you have pipx installed.
See /usr/share/doc/python3.11/README.venv for more information.
note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
With this PEP, Python tools can now distinguish between packages that have been
installed by the user with a tool like
pip and ones installed using a
distribution's package manager, like
This is generally great news: it was previously too easy to break a system by mixing the two types of packages. This PEP will simplify our role as a distribution, as well as improve the overall Python user experience in Debian.
Sadly, it's also likely this change will break some of your scripts, especially
CI that (legitimately) install packages via
pip alongside system packages.
For example, I use the following
gitlab-ci snippet to make sure my PRs don't
break my build process2:
- apt-get update && apt-get install -y flit python3-pip
- FLIT_ROOT_INSTALL=1 flit install
- metalfinder --help
With Python 3.11, this snippet will error out, as
pip will refuse to install
packages alongside the system's. The fix is to tell
pip it's OK to "break"
your system packages, either using the
--break-system-packages parameter, or
PIP_BREAK_SYSTEM_PACKAGES=1 environment variable3.
This, of course, is not something you should be using in production to restore
the old behavior! The "proper" way to fix this issue, as the
externally-managed-environment error message aptly (har har) informs you, is
to use virtual environments.
Kudos to our own Matthias Klose, Stefano Rivera and Elana Hashman, who worked on designing and implementing this PEP! ↩
Which is something that bit me before... You push some changes to your git repository, everything seems fine and all the tests pass, so you merge it and make a new git tag. When the time comes to build and upload this tag to PyPi, you find out some minor thing broke your build system (which you weren't testing) and you have to scramble to make a point-release to fix the issue. Sad! ↩
Don't go searching for this environment variable in
pip's code though, as you won't find it! All of
pip's command line options can be passed as env vars using the
PIP_<UPPER_LONG_NAME>format. Useful for tools that use