179 Commits

Author SHA1 Message Date
Eman Resu 3fbb3ae435 ci/eval: add opt-in instruction counting using perf
Currently unused in comparisons - just collecting the data for
future use.
2026-07-07 10:48:38 -04:00
Silvan Mosberger bf80cf0f2e ci/eval: Allow disallowed attrs to depend on other disallowed attrs
With a dependency chain like

    a -> b -> c

it was not possible to e.g. add a `meta.problems.removal` to b, because that would cause b to be entirely unavailable as a dependency.

This PR slightly lifts that restriction by allowing both a and b to be marked with such a problem without causing a CI failure.

This is useful for:
- Having a problem on a package that needs to be removed but still has dependents (which then also need a problem)
- Having a problem on a package that is aliased under another attribute, which counts as a dependency
2026-06-30 16:12:39 +02:00
Silvan Mosberger a90d993610 ci/eval: Allow preventing internal Nixpkgs use of certain problem kinds 2026-06-24 12:26:03 +02:00
Silvan Mosberger 19a31658dc ci/eval: Refactor attrpaths.nix to a more generic pre-eval.nix 2026-06-24 12:25:57 +02:00
Aliaksandr a100d091a0 ci/eval: name the base branch in the performance comparison summary
The performance comparison summary previously stated it compared against
"its pull request base branch (e.g., 'master')" regardless of the actual
base branch.

Plumb the base branch classification computed in prepare.js through the
eval workflow to the `compare` derivation so the summary names the real
base branch (e.g. "staging-25.11"). The local `eval.full` helper defaults
to "master", matching its touched-files convention.

Assisted-by: claude-code with claude-opus-4-8[1m]-high
2026-06-17 01:08:45 +03:00
Aliaksandr 25f9394b8a ci/eval/compare: show performance comparison even when package sets differ
Previously the eval comparison was skipped entirely when packages were
added or removed between revisions, leaving only a "stats were skipped"
message. cmp-stats already joins on the stats chunks present in both
revisions, so a comparison can still be produced. Always run it and add
a GitHub note caveating that the figures are approximate when the
package sets differ.

Assisted-by: claude-code with claude-opus-4-7[1m]-xhigh
2026-05-25 11:04:25 +03:00
zowoq c103da6a19 ci: add nixosTests.simple-container
nixosTests.simple-vm but using an nspawn container
2026-05-22 21:24:34 +10:00
zowoq 244b700403 nixosTests.simple-vm: rename from nixosTests.simple
renamed because we now have different kinds of tests
2026-05-22 21:24:28 +10:00
Michael Daniels 02bda2c4cf ci/eval/compare: set NixOS test rebuild label on all PRs that do so
I don't think the worry about confusion applies anymore because nowadays CI
will tell you if you target staging-nixos when you should be targeting staging.

On the other hand, I was confused why kernel updates didn't show up as
NixOS test rebuilds...
2026-04-23 17:16:39 -04:00
Kamil Monicz 8926c73f50 ci/eval/compare: Expose attrdiff by kernel and platform 2026-04-13 10:53:48 +00:00
dramforever 16b3779220 ci/eval/compare: Add test for nonexistent by-name attribute 2026-03-14 14:11:41 +08:00
dramforever ce70a9e6b4 ci/eval/compare/maintainers: Handle nonexistent attribute in changedByNameAttrPaths
This could happen if a new package was added in pkgs/by-name
2026-03-14 14:10:56 +08:00
Silvan Mosberger 3e93ae351e ci/eval/compare: Fix tests
https://github.com/NixOS/nixpkgs/commit/ca6ce52e6a2991753c2d63c2e96d3cf0c91fd07f
didn't update the tests accordingly
2026-03-14 00:15:49 +01:00
Silvan Mosberger ca6ce52e6a ci/eval/compare: Fix wrong Nixpkgs being used
See https://github.com/NixOS/nixpkgs/pull/488014#issuecomment-4009775318
and https://github.com/NixOS/nixpkgs/issues/497110
2026-03-13 16:53:43 +01:00
Silvan Mosberger b513ab8192 Reapply "ci: module maintainer review requests; nixos/modules: init meta.teams"
This reverts commit 2cc2e0225f.
2026-03-13 16:53:28 +01:00
Silvan Mosberger 2cc2e0225f Revert "ci: module maintainer review requests; nixos/modules: init meta.teams" 2026-03-06 08:41:24 +01:00
Silvan Mosberger 7fe44998ab ci/eval/compare/maintainers.nix: Refactor
Was more complicated than needed.
Tests still succeed
2026-03-02 10:59:25 +01:00
Silvan Mosberger ee016757f5 ci/eval/compare/maintainers.nix: Add tests 2026-03-02 10:59:22 +01:00
Silvan Mosberger c0d434bf40 ci/eval/compare/maintainers.nix: Improve interface and document 2026-03-02 10:59:18 +01:00
Silvan Mosberger 5991eab94f modules/generic/meta-maintainers.nix: Introduce meta.teams
And trigger review requests for it
2026-03-02 10:59:07 +01:00
Silvan Mosberger b486dfb1ab modules/generic/meta-maintainers.nix: Remove Nixpkgs prefix from module path
This is a functional change, but there's no known consumer of this
interface, and if we want to trigger review request for PRs it's
convenient for these paths to be absolute

ci: Trigger review requests for module maintainers
2026-03-02 10:58:24 +01:00
luojiyin dbb1854400 fix: typo packge -> package in maintainers.nix 2026-03-01 14:00:07 +08:00
Michael Daniels f89b3b4ffb {ci/eval,release}: migrate supported systems to top-level (#492103) 2026-02-28 00:21:52 +00:00
Emily 24d5ac84b3 {ci/eval,release}: migrate supported systems to top-level
The supported systems can differ across release branches. The original
file is kept until the GitHub CI is migrated.
2026-02-27 13:45:31 +00:00
Matt Sturgeon 92b637a63e ci/eval: only evaluate the NixOS test on Linux (#492148) 2026-02-27 10:40:40 +00:00
Emily ec070a136c config: add stub allowDeprecatedx86_64Darwin option 2026-02-22 18:52:16 +00:00
Emily 82ef104d8e ci/eval: only evaluate the NixOS test on Linux
It does evaluate for macOS – at least when I’m not busy adding a
warning to `x86_64-darwin` – but Darwin‐only changes to the test
driver won’t cause rebuilds on Hydra anyway.
2026-02-19 16:08:11 +00:00
Philip Taron 5712d1d024 ci/eval/outpaths.nix: add executable bit
This file has a nix-shell shebang but was never made executable.
2026-02-16 15:31:57 -08:00
Silvan Mosberger b0b2c24be9 ci: First-class team package maintainer review requests
Co-Authored-By: Alexander Bantyev <alexander.bantyev@tweag.io>
2026-02-07 13:52:46 +01:00
Michael Daniels 0d74c65927 Revert "ci: First-class team package maintainer review requests"
This reverts commit 193deb8533.
2026-02-05 21:50:27 -05:00
Philip Taron ba626b8a60 First-class GitHub team reviews (#456481) 2026-02-06 01:44:31 +00:00
Silvan Mosberger 193deb8533 ci: First-class team package maintainer review requests
Co-Authored-By: Alexander Bantyev <alexander.bantyev@tweag.io>
2026-02-05 14:52:47 +01:00
Michael Daniels 006ecdbdeb treewide: fix typos (#479869) 2026-01-24 21:36:44 +00:00
Michael Daniels 3711d8fc80 ci/eval/compare: don't try to get the attr of pkgs/by-name/README.md 2026-01-24 15:24:56 -05:00
Ben Siraphob c7e10647ea treewide: fix typos 2026-01-13 14:45:11 -05:00
Wolfgang Walther c0b6cc9387 ci/eval/compare: fix without owners
Even without relevant owners, the owners.txt file must be created,
otherwise the next job will fail.
2025-11-02 17:30:46 +01:00
Wolfgang Walther 3bef0dcf65 ci/request-reviews: move get-code-owners to eval/compare
This moves the parsing of ci/OWNERS into the Nix sandbox. We also get
rid of checking out the nixpkgs repo another time in the reviewers
workflow - we already have everything we need in the eval/compare job.

The creation of owners.txt in this way is only temporary, it should
eventually be moved further, similar to how maintainers.json is
currently migrating to a maintainer map for the whole repo stored on the
target branch as artifact.
2025-11-01 15:02:23 +01:00
Wolfgang Walther 0e65d6bc46 ci/github-script/labels: set maintainer labels from latest maintainer map (#457243) 2025-11-01 10:11:01 +00:00
Wolfgang Walther 6b5e6cbbee ci/github-script/labels: set maintainer labels from latest maintainer map
Instead of setting the maintainer-related labels based entirely on Eval
results, this uses the new maintainer map from the target branch. This
allows labeling PRs correctly, that had been created *before* a
contributor became a maintainer of the respective package.
2025-11-01 10:36:23 +01:00
Gaetan Lepage 0d5f9e6641 ci/eval: add extraNixpkgsConfig argument 2025-11-01 00:12:42 +01:00
Wolfgang Walther 11749394df ci/eval: avoid passing args around
There is no point in passing these arguments around between all the
different parts of the eval chain - these global settings should surely
not be modified between different steps.

This also makes it simpler to add new arguments like extra nixpkgs
configuration.
2025-10-31 20:03:17 +01:00
Wolfgang Walther 498574f238 workflows/eval: upload maintainer list of all packages as artifact
This creates a simple mapping of all packages to github ids of all their
maintainers. This is uploaded as an artifact and is then available for
download on the latest commit of each branch with a merge queue. This
will allow scheduled jobs to use this information for setting
maintainer-related labels, to request reviewers and to implement the
merge-bot.

The advantage over querying this information directly via Nix in each
case: The scheduled job does not need to install Nix and does not need
to checkout the target branch.

Compared to obtaining the maintainer information just for a single PR
during Eval, this will allow retroactively changing maintainers for a
package: For example, it allows to request a new maintainer as reviewer
for a PR that was created before they became maintainer, but is still
open - and similarly for maintainer labels and merge-bot rights.

None of these extensions are implemented by this PR, yet.
2025-10-31 10:59:01 +01:00
Wolfgang Walther 5c01bd9df6 ci/eval/utils: refactor to use lib.init 2025-10-31 10:58:55 +01:00
Wolfgang Walther 350a469f73 ci/eval/diff: avoid copying paths.json
Currently the `diff-<system>` artifacts are 6-7 MB in size - and almost
all of that is the `paths.json` file, which is only used to generate the
diff itself. This had been stored in the artifact previously for
debugging purposes. Ever since we moved to Cachix this is not required
anymore, since it's possible to run the same eval locally and thus fetch
the `eval.singleSystem` result, including `paths.json`, from Cachix.

This will be even more helpful when the next step adds `meta.json` -
which is magnitudes bigger than `paths.json`.
2025-10-31 10:58:50 +01:00
Wolfgang Walther b5af85e10e Revert "First-class GitHub team reviews" 2025-10-28 07:38:20 +00:00
Silvan Mosberger e6c822f51f workflows/reviewers: First-class team package maintainer review requests 2025-10-28 01:35:25 +01:00
Matt Sturgeon 9ea88ca354 ci/eval/compare/maintainers: simplify meta.position predicate 2025-10-22 21:11:19 +01:00
Wolfgang Walther 6d564102b7 ci/eval/compare/maintainers: fix maintainer pings without meta.position
In a recent change, the path matching was simplified in maintainers.nix.
This revealed a pre-existing logic bug: Packages without `meta.position`
would get an empty string as their file name. The change would then
cause this empty string to always be matched, which lead to maintainer
pings for these packages in seemingly random PRs, when some of their
dependencies were changed.
2025-10-22 21:06:53 +02:00
Wolfgang Walther 6a26c66335 ci/eval/compare/maintainers: disable aliases
We should never try to ping maintainers through package aliases, this
can only lead to errors. One example case is, where an attribute is a
throw alias, but then re-introduced in a PR. This would trigger the
throw. By disabling aliases, we can fallback gracefully.
2025-10-22 13:33:41 +02:00
Wolfgang Walther 2b7ea9379a ci/eval: disable GC warning
This came up more often recently, and breaks Eval because it prints on
stderr.
2025-10-18 14:06:09 +02:00