This commit was created by a combination of scripts and tools:
- an ast-grep script to prefix things in meta with `lib.`,
- a modified nixf-diagnose / nixf combination to remove unused `with
lib;`, and
- regular nixfmt.
Co-authored-by: Wolfgang Walther <walther@technowledgy.de>
As documented in https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#abi-tag.
This is useful to check whether a wheel is compatible with a certain interpreter. [Pyproject.nix](https://github.com/pyproject-nix/pyproject.nix) has [functions to perform wheel compatibility checking](https://pyproject-nix.github.io/pyproject.nix/lib/pypa.html#function-library-lib.pypa.isWheelFileCompatible) against a Python interpreter, and has computed interpreter ABI tags itself.
The recent addition of free threading (`python313FreeThreading`) complicates this by not being introspectable:
A GIL Python (non free-threaded) has an ABI tag `cp313` while the free-threaded Python has `cp313t`, but the package doesn't communicate whether `enableGIL` is true or false, leaving no way to compute the tag.
The same goes for if debugging support was added to the derivation: A `d` suffix would need to be added.
Additionally ABI tags has no defined format and can really only be accurately computed by having insight into how the ABI tags are used by a specific interpreter, meaning that the only correct place to compute ABI tags is within the context of a particular Python derivation.
While this has no immediate use within nixpkgs it could be used as a basis to provide compatibility assertions regarding wheel compat at eval time.
Without the change the eval fails as:
$ nix build --no-link -f. pythonInterpreters.pypy310_prebuilt.tests.test-packageOverrides
error:
… in the condition of the assert statement
at pkgs/development/interpreters/python/tests.nix:212:11:
211| in
212| assert myPython.pkgs.foobar == myPython.pkgs.numpy;
| ^
213| myPython.withPackages (ps: with ps; [ foobar ]);
… while evaluating the attribute 'outPath'
at lib/customisation.nix:421:7:
420| drv.drvPath;
421| outPath =
| ^
422| assert condition;
(stack trace truncated; use '--show-trace' to show the full, detailed trace)
error: attribute 'mainProgram' missing
at pkgs/by-name/me/meson/package.nix:75:52:
74| --replace-fail "PythonExternalProgram('python3', mesonlib.python_command)" \
75| "PythonExternalProgram('${python3.meta.mainProgram}', mesonlib.python_command)"
| ^
76| substituteInPlace mesonbuild/modules/python3.py \
In preparation for the deprecation of `stdenv.isX`.
These shorthands are not conducive to cross-compilation because they
hide the platforms.
Darwin might get cross-compilation for which the continued usage of `stdenv.isDarwin` will get in the way
One example of why this is bad and especially affects compiler packages
https://www.github.com/NixOS/nixpkgs/pull/343059
There are too many files to go through manually but a treewide should
get users thinking when they see a `hostPlatform.isX` in a place where it
doesn't make sense.
```
fd --type f "\.nix" | xargs sd --fixed-strings "stdenv.is" "stdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "stdenv'.is" "stdenv'.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "clangStdenv.is" "clangStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "gccStdenv.is" "gccStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "stdenvNoCC.is" "stdenvNoCC.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "inherit (stdenv) is" "inherit (stdenv.hostPlatform) is"
fd --type f "\.nix" | xargs sd --fixed-strings "buildStdenv.is" "buildStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "effectiveStdenv.is" "effectiveStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "originalStdenv.is" "originalStdenv.hostPlatform.is"
```
The way we build python environments is subtly broken. A python
environment should be semantically identical to a vanilla Python
installation in, say, /usr/local. The current implementation, however,
differs in two important ways. The first is that it's impossible to use
python packages from the environment in python virtual environments. The
second is that the nix-generated environment appears to be a venv, but
it's not.
This commit changes the way python environments are built:
* When generating wrappers for python executables, we inherit argv[0]
from the wrapper. This causes python to initialize its configuration
in the environment with all the correct paths.
* We remove the sitecustomize.py file from the base python package.
This file was used tweak the python configuration after it was
incorrectly initialized. That's no longer necessary.
The end result is that python environments no longer appear to be venvs,
and behave more like a vanilla python installation. In addition it's
possible to create a venv using an environment and use packages from
both the environment and the venv.
This commit adds a Nix-specific module that recursively adds paths that
are on `NIX_PYTHONPATH` to `sys.path`. In order to process possible
`.pth` files `site.addsitedir` is used.
The paths listed in `PYTHONPATH` are added to `sys.path` afterwards, but
they will be added before the entries we add here and thus take
precedence.
The reason for adding support for this environment variable is that we
can set it in a wrapper without breaking support for `PYTHONPATH`.
Patching numpy.distutils used to be required for pythonPackages.cython
to build on darwin. It was later accidentally disabled during one of the
refactorings, but that did not break cython. This change reinstantiates
the patch. It still applies, so it should be low maintenance and it can
still be useful.
These interpreters are prebuilt by upstream and patched using patchelf.
They are primarily added for testing purposes and development on the
non-prebuilt PyPy interpreters as it can speed up translation
significantly.
Emily