The way we build python environments is subtly broken. A python
environment should be semantically identical to a vanilla Python
installation in, say, /usr/local. The current implementation, however,
differs in two important ways. The first is that it's impossible to use
python packages from the environment in python virtual environments. The
second is that the nix-generated environment appears to be a venv, but
it's not.
This commit changes the way python environments are built:
* When generating wrappers for python executables, we inherit argv[0]
from the wrapper. This causes python to initialize its configuration
in the environment with all the correct paths.
* We remove the sitecustomize.py file from the base python package.
This file was used tweak the python configuration after it was
incorrectly initialized. That's no longer necessary.
The end result is that python environments no longer appear to be venvs,
and behave more like a vanilla python installation. In addition it's
possible to create a venv using an environment and use packages from
both the environment and the venv.
Without the change `test-pythonPackagesExtensions` test fails the eval
as:
$ nix build -f. pythonInterpreters.pypy39_prebuilt.tests --show-trace
error:
… while evaluating an attribute name
at pkgs/development/interpreters/python/tests.nix:151:16:
150| });
151| in pkgs_.${python.pythonAttr}.pkgs.foo;
| ^
152| });
error: value is null while a string was expected
This happens because `pypy39_prebuilt` exposes `pythonAttr` attribute,
but it has a `null` value. Fix the test to filter out prebuilt pypy.
The virtualenv mode is trying to be smart and checks whether it can
write to $out, which for some reason fails on darwin.
> virtualenv: error: argument dest: the destination . is not write-able at /nix/store
But what does work is creating it in the pwd and moving it to $out,
because mv just does what it is told.
Allow python3.tests to eval on other platforms than linux, by excluding
conda tests when not on linux, because they include alsa-lib, which is
linux-only.
Python package sets can be overridden by overriding an interpreter
and passing in `packageOverrides = self: super: {...};`. This is fine
in case you need a single interpreter, however, it does not help you
when you want to override all sets.
With this change it is possible to override all sets at once by
appending a list of "extensions" to `pythonPackagesExtensions`.
From reading the implementation you might wonder why a list is used, and
not
`lib.composeExtensions`? The reason is the latter requires knowledge of
the library function. This approach should be easier for most users
as it is similar to how we append to lists of e.g. inputs or patches
when overriding a derivation.
Sandro Jäckel