`lib.debug.runTests` provides a unit test evaluator for Nix, but its
results are returned in a raw and difficult-to-read form.
Currently, different callers output the results in various ways:
`builtins.throw (builtins.toJSON failures)` and `builtins.throw ("Tests
failed: " + lib.generators.toPretty { } failures)` are both used.
This change adds a new `lib.debug.throwTestFailures` function which
displays the results nicely before throwing an exception (or returns
`null` if no failures are given), unifying these disparate call-sites.
First, each failing test is pretty-printed in a `trace` message:
```
trace: FAIL testDerivation:
Expected: <derivation a>
Result: <derivation b>
```
Then, an exception is thrown containing the number of tests that failed
(and their names), followed by the raw JSON of the results (for parity
with previous usage, and because `lib.generators.toPretty` sometimes
omits information that `builins.toJSON` includes):
```
error:
… while evaluating the file '...':
… caused by explicit throw
at /nix/store/.../lib/debug.nix:528:7:
527| in
528| throw (
| ^
529| builtins.seq traceFailures (
error: 1 tests failed:
- testDerivation
[{"expected":"/nix/store/xh7kyqp69mxkwspmi81a94m9xx74r8dr-a","name":"testDerivation","result":"/nix/store/503l84nir4zw57d1shfhai25bxxn16c6-b"}]
```
Unfortunately, we don't have a good enough way to discern this
possibility. Technically we could add yet another attribute, but
that does not seem to be worth the cost at this point, in terms
of complexity and possibly even performance.
I take the blame for this one. I identified the need for the second
merge commit in review but didn't require it in the first merge.
These changes really should have been done in a single merge.
That would have prevented this situation.
This will allow the code for https://github.com/NixOS/nixpkgs/issues/447514
to make sure that the right team is requested for review,
even if it has been renamed in the meantime.
While the matching is currently based on the teams slug/name and not the
id, renames won't cause problems with `lib.teams`, because CI would
error if there's no match. Changing this can be future work.
The before and after of
nix-instantiate --eval -A lib.teams --strict --json | jq 'walk(if type == "array" then sort else . end)'
has been ensured to be negligible, only consisting of minor team
shortName and scope differences
