101 Commits

Author SHA1 Message Date
Aliaksandr a100d091a0 ci/eval: name the base branch in the performance comparison summary
The performance comparison summary previously stated it compared against
"its pull request base branch (e.g., 'master')" regardless of the actual
base branch.

Plumb the base branch classification computed in prepare.js through the
eval workflow to the `compare` derivation so the summary names the real
base branch (e.g. "staging-25.11"). The local `eval.full` helper defaults
to "master", matching its touched-files convention.

Assisted-by: claude-code with claude-opus-4-8[1m]-high
2026-06-17 01:08:45 +03:00
Aliaksandr 25f9394b8a ci/eval/compare: show performance comparison even when package sets differ
Previously the eval comparison was skipped entirely when packages were
added or removed between revisions, leaving only a "stats were skipped"
message. cmp-stats already joins on the stats chunks present in both
revisions, so a comparison can still be produced. Always run it and add
a GitHub note caveating that the figures are approximate when the
package sets differ.

Assisted-by: claude-code with claude-opus-4-7[1m]-xhigh
2026-05-25 11:04:25 +03:00
zowoq c103da6a19 ci: add nixosTests.simple-container
nixosTests.simple-vm but using an nspawn container
2026-05-22 21:24:34 +10:00
zowoq 244b700403 nixosTests.simple-vm: rename from nixosTests.simple
renamed because we now have different kinds of tests
2026-05-22 21:24:28 +10:00
Michael Daniels 02bda2c4cf ci/eval/compare: set NixOS test rebuild label on all PRs that do so
I don't think the worry about confusion applies anymore because nowadays CI
will tell you if you target staging-nixos when you should be targeting staging.

On the other hand, I was confused why kernel updates didn't show up as
NixOS test rebuilds...
2026-04-23 17:16:39 -04:00
Kamil Monicz 8926c73f50 ci/eval/compare: Expose attrdiff by kernel and platform 2026-04-13 10:53:48 +00:00
dramforever 16b3779220 ci/eval/compare: Add test for nonexistent by-name attribute 2026-03-14 14:11:41 +08:00
dramforever ce70a9e6b4 ci/eval/compare/maintainers: Handle nonexistent attribute in changedByNameAttrPaths
This could happen if a new package was added in pkgs/by-name
2026-03-14 14:10:56 +08:00
Silvan Mosberger 3e93ae351e ci/eval/compare: Fix tests
https://github.com/NixOS/nixpkgs/commit/ca6ce52e6a2991753c2d63c2e96d3cf0c91fd07f
didn't update the tests accordingly
2026-03-14 00:15:49 +01:00
Silvan Mosberger ca6ce52e6a ci/eval/compare: Fix wrong Nixpkgs being used
See https://github.com/NixOS/nixpkgs/pull/488014#issuecomment-4009775318
and https://github.com/NixOS/nixpkgs/issues/497110
2026-03-13 16:53:43 +01:00
Silvan Mosberger b513ab8192 Reapply "ci: module maintainer review requests; nixos/modules: init meta.teams"
This reverts commit 2cc2e0225f.
2026-03-13 16:53:28 +01:00
Silvan Mosberger 2cc2e0225f Revert "ci: module maintainer review requests; nixos/modules: init meta.teams" 2026-03-06 08:41:24 +01:00
Silvan Mosberger 7fe44998ab ci/eval/compare/maintainers.nix: Refactor
Was more complicated than needed.
Tests still succeed
2026-03-02 10:59:25 +01:00
Silvan Mosberger ee016757f5 ci/eval/compare/maintainers.nix: Add tests 2026-03-02 10:59:22 +01:00
Silvan Mosberger c0d434bf40 ci/eval/compare/maintainers.nix: Improve interface and document 2026-03-02 10:59:18 +01:00
Silvan Mosberger 5991eab94f modules/generic/meta-maintainers.nix: Introduce meta.teams
And trigger review requests for it
2026-03-02 10:59:07 +01:00
Silvan Mosberger b486dfb1ab modules/generic/meta-maintainers.nix: Remove Nixpkgs prefix from module path
This is a functional change, but there's no known consumer of this
interface, and if we want to trigger review request for PRs it's
convenient for these paths to be absolute

ci: Trigger review requests for module maintainers
2026-03-02 10:58:24 +01:00
luojiyin dbb1854400 fix: typo packge -> package in maintainers.nix 2026-03-01 14:00:07 +08:00
Silvan Mosberger b0b2c24be9 ci: First-class team package maintainer review requests
Co-Authored-By: Alexander Bantyev <alexander.bantyev@tweag.io>
2026-02-07 13:52:46 +01:00
Michael Daniels 0d74c65927 Revert "ci: First-class team package maintainer review requests"
This reverts commit 193deb8533.
2026-02-05 21:50:27 -05:00
Silvan Mosberger 193deb8533 ci: First-class team package maintainer review requests
Co-Authored-By: Alexander Bantyev <alexander.bantyev@tweag.io>
2026-02-05 14:52:47 +01:00
Michael Daniels 3711d8fc80 ci/eval/compare: don't try to get the attr of pkgs/by-name/README.md 2026-01-24 15:24:56 -05:00
Wolfgang Walther c0b6cc9387 ci/eval/compare: fix without owners
Even without relevant owners, the owners.txt file must be created,
otherwise the next job will fail.
2025-11-02 17:30:46 +01:00
Wolfgang Walther 3bef0dcf65 ci/request-reviews: move get-code-owners to eval/compare
This moves the parsing of ci/OWNERS into the Nix sandbox. We also get
rid of checking out the nixpkgs repo another time in the reviewers
workflow - we already have everything we need in the eval/compare job.

The creation of owners.txt in this way is only temporary, it should
eventually be moved further, similar to how maintainers.json is
currently migrating to a maintainer map for the whole repo stored on the
target branch as artifact.
2025-11-01 15:02:23 +01:00
Wolfgang Walther 6b5e6cbbee ci/github-script/labels: set maintainer labels from latest maintainer map
Instead of setting the maintainer-related labels based entirely on Eval
results, this uses the new maintainer map from the target branch. This
allows labeling PRs correctly, that had been created *before* a
contributor became a maintainer of the respective package.
2025-11-01 10:36:23 +01:00
Wolfgang Walther 5c01bd9df6 ci/eval/utils: refactor to use lib.init 2025-10-31 10:58:55 +01:00
Wolfgang Walther b5af85e10e Revert "First-class GitHub team reviews" 2025-10-28 07:38:20 +00:00
Silvan Mosberger e6c822f51f workflows/reviewers: First-class team package maintainer review requests 2025-10-28 01:35:25 +01:00
Matt Sturgeon 9ea88ca354 ci/eval/compare/maintainers: simplify meta.position predicate 2025-10-22 21:11:19 +01:00
Wolfgang Walther 6d564102b7 ci/eval/compare/maintainers: fix maintainer pings without meta.position
In a recent change, the path matching was simplified in maintainers.nix.
This revealed a pre-existing logic bug: Packages without `meta.position`
would get an empty string as their file name. The change would then
cause this empty string to always be matched, which lead to maintainer
pings for these packages in seemingly random PRs, when some of their
dependencies were changed.
2025-10-22 21:06:53 +02:00
Wolfgang Walther 6a26c66335 ci/eval/compare/maintainers: disable aliases
We should never try to ping maintainers through package aliases, this
can only lead to errors. One example case is, where an attribute is a
throw alias, but then re-introduced in a PR. This would trigger the
throw. By disabling aliases, we can fallback gracefully.
2025-10-22 13:33:41 +02:00
Wolfgang Walther f33892fcc9 ci/eval/compare/maintainers: ping by-name reviewers for pure refactors
This allows requesting reviewers for pure refactor PRs, which don't
cause a rebuild of the package. This is only possible for by-name,
because only here the package names can be inferred from the filenames.
2025-10-17 16:37:14 +02:00
Wolfgang Walther 0a5ffcdf31 ci/eval/compare/maintainers: ping by-name reviewers for all rebuilds
This adds support to ping maintainers when arbitrary files in by-name
are changed, as long as they still cause a rebuild. For example, this is
the case when changing .json files with version metadata. These were
previously not detected as belonging to the package, and didn't cause
maintainer pings.
2025-10-17 16:37:09 +02:00
Wolfgang Walther 7c865ddd46 ci/eval/compare/maintainers: refactor path matching
The only reason for the additional `lib.hasSuffix` check was, that the
`lib.removePrefix` was broken - it was never adjusted when porting this
from ofborg, so the relative path was wrong and no prefix ever removed,
since no packages are in `ci/`.

This additionally strips the leading `/`, so that `relevantFilenames`
will then have paths starting with `pkgs/...`, similar to how git
reports those paths in the `changedpathsjson` file. This allows simple
equality comparison.
2025-10-17 16:37:04 +02:00
Wolfgang Walther 79029e39a4 ci/eval/compare/maintainers: refactor using lib
Removes unused code and comments, uses shorter lib over builtins and
switches to `lib.importJSON`.

Readability is key.
2025-10-17 16:37:00 +02:00
Wolfgang Walther 44e4c1ba9f maintainers/scripts/get-maintainer-pings-between: drop
This has been broken for months, since all the interface to `ci/eval`
changed. Nobody noticed, so nobody is using it.
2025-10-16 13:36:42 +02:00
NAHO dec908b169 treewide: remove optional builtins prefixes from prelude functions
Remove optional builtins prefixes from prelude functions by running:

    builtins=(
      abort
      baseNameOf
      break
      derivation
      derivationStrict
      dirOf
      false
      fetchGit
      fetchMercurial
      fetchTarball
      fetchTree
      fromTOML
      import
      isNull
      map
      null
      placeholder
      removeAttrs
      scopedImport
      throw
      toString
      true
    )

    fd --type file --exec-batch sed --in-place --regexp-extended "
      s/\<builtins\.($(
        printf '%s\n' "${builtins[@]}" |
          paste --delimiter '|' --serial -
      ))\>/\1/g
    "

    nix fmt
2025-09-30 09:05:08 +02:00
Philip Taron 3d0e2841b8 ci.eval.compare: keep warnings as warnings rather than raising them as errors
This reverts commit f8210561f3 (ci.eval.compare: turn warnings into errors, 2025-09-16).

It turns out that there are normal math warnings and we don't want to block CI on the math coming out wrong.
2025-09-18 08:37:36 -07:00
Philip Taron d80d4a77b7 ci.eval.compare: split out equivalent values into their own table 2025-09-17 14:04:28 -07:00
Philip Taron fb1647ec6e ci.eval.compare: explain the various metrics under the --explain flag 2025-09-17 14:04:28 -07:00
Philip Taron c9860ef95c ci.eval.compare: remove the duplicate cpuTime key 2025-09-17 14:04:28 -07:00
Philip Taron 2817f79649 ci.eval.compare: put things with counts together 2025-09-17 14:04:28 -07:00
Philip Taron 210e3e1151 ci.eval.compare: put things with bytes together 2025-09-17 14:04:28 -07:00
Philip Taron 3edc1e204e ci.eval.compare: make the table format using tabulate not manually 2025-09-17 14:04:28 -07:00
Philip Taron 4eaa094f20 ci.eval.compare: sort time metrics first, then GC metrics, then everything else 2025-09-17 14:04:28 -07:00
Philip Taron e83e900874 ci.eval.compare: assert types in flatten_data 2025-09-17 14:04:28 -07:00
Philip Taron 9959a4e507 ci.eval.compare: delete unreferenced global 2025-09-17 14:04:28 -07:00
Philip Taron e4101ea3a9 ci.eval.compare: instead of manually tabulating, use tabulate 2025-09-17 14:04:28 -07:00
Philip Taron 7818a245f7 ci.eval.compare: support passing single files to cmp-stats 2025-09-17 14:04:27 -07:00
Philip Taron 4bc54e7a3a ci.eval.compare: allow before_vals == 1 but avoid the t-test 2025-09-17 14:04:27 -07:00