We are currently using an impure method to build a derivation for
playwrights browsers on darwin: We just call `playwright install`
during build-time. But this recently began to fail, with an error that
the azure certificate would be self-signed, while it was not. This is
presumably due to https://github.com/NixOS/nixpkgs/pull/257513
where we started to use the systems CA store for nodejs instead of
the bundled one, so we add cacert to fix it.
Maybe the whole impure method of downloading browsers on darwin
should be reconsidered in favor of following the same approach as
on Linux, but that would probably have the drawback that we'd loose
safari as one of the browsers.
I wanted to improve the generated luarocks config.
The current system appends a string to it, this commit allows to pass
arbitrary nix code and have it autoconverted to lua along with the
default settings in the luarocks config.
We have packages that use `meta.platforms = []` as a sort of synonym
for `broken = true`. Without this commit, the attrnames for those
jobs will end up in the list of attrnames which are expected to
build, even though they are not expected to build.
This commit temporarily adds pkgs/test/release to the
lib/tests/release.nix test suite, because ofborg already knows about
that entry point.
We should move the list of test entry points out of ofborg and into
a central place in nixpkgs:
https://github.com/NixOS/nixpkgs/issues/272591
Once we do that we won't need to have this ugly kludge in an
inappropriate place.
This derivation verifies that all jobset attributes can be evaluated
under tryEval without producing any non-catchable errors or causing
infinite recursion.
This file walks the entire nixpkgs tree and emits a superset of all
release attrnames in only 44 seconds on a 3ghz CPU, using 5 gbytes
of memory. By comparison, on the same CPU the `nix-env` hack used
by ofborg on every PR submission requires 41 *minutes* and peaks at
60 gbytes, even with checkMeta turned off. Full details below.
This is:
- 46x faster (or 2.1% of the elapsed time)
- 12.5x less memory (or 8.0% of the peak memory usage)
In order to replace the ofborg check, this list of attrnames must
then be post-filtered for platform-relevance. However, crucially,
the post-filtering can be done *in parallel* on multiple cores by
splitting the attrname list in to chunks. Generating the list of
attrnames cannot be parallelized because it is a single-threaded
cppnix task.
This PR also adds `recurseForDerivations` where necessary within
nixpkgs in order to make this possible -- it screens out various
non-tryEval-catchable failures and infinite recursions. Before
undraftifying, I will add an invocation of this command to the CI
tests, to ensure that the work performed here is not immediately
undone. My next PR will then add an additional CI check confirming
that the emitted attrpaths are in fact a superset of the release
attrpaths calculated by the slow-memory-hog ofborg method.
I have manually confirmed that this is the case at the tip commit of
this PR, but we need CI to make sure this remains true until ofborg
switches to this more-efficient method of calculation; at that point
the superset-check can be dropped.
According to GNU Time,
Command being timed: "nix-instantiate --eval --strict --json pkgs/top-level/release-attrpaths-superset.nix -A names"
User time (seconds): 44.88
System time (seconds): 8.09
Percent of CPU this job got: 99%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:53.20
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 4823028
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 3611240
Voluntary context switches: 113
Involuntary context switches: 949
Swaps: 0
File system inputs: 1480
File system outputs: 5944
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
Compared to release-outpaths.nix:
Command being timed: "nix-env -qaP --no-name --out-path --arg checkMeta false --argstr path /git/work/pr/release-outpaths -f pkgs/top-level/release-outpaths.nix"
User time (seconds): 2120.67
System time (seconds): 337.80
Percent of CPU this job got: 98%
Elapsed (wall clock) time (h:mm:ss or m:ss): 41:37.91
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 60171768
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 2
Minor (reclaiming a frame) page faults: 230608113
Voluntary context switches: 8876
Involuntary context switches: 22275
Swaps: 0
File system inputs: 62624
File system outputs: 72
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
The previous code did not apply any changes to the upstream defaults on being presented with an empty list.
This changes the code to use the above behaviour on a `null` value while an empty list is passed through as normal which yields a systemd configuration line with empty value which resets it to an empty value.
Signed-off-by: benaryorg <binary@benary.org>
