gador/nixpkgs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge 8620c0b5cc into haskell-updates

Browse Source
This commit is contained in:
nixpkgs-ci[bot]
2026-03-21 00:27:14 +00:00
committed by GitHub
parent e1dd933dfb 8620c0b5cc
commit 318dcde9b4
855 changed files with 16406 additions and 11755 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/build.yml
+1 -1
View File
@@ -67,7 +67,7 @@ jobs:
# Sandbox is disabled on MacOS by default.
extra_nix_config: sandbox = true
- uses: cachix/cachix-action@3ba601ff5bbb07c7220846facfa2cd81eeee15a1 # v16
- uses: cachix/cachix-action@1eb2ef646ac0255473d23a5907ad7b04ce94065c # v17
continue-on-error: true
with:
# The nixpkgs-gha cache should not be trusted or used outside of Nixpkgs and its forks' CI.
.github/workflows/check.yml
+1 -1
View File
@@ -87,7 +87,7 @@ jobs:
- uses: cachix/install-nix-action@1ca7d21a94afc7c957383a2d217460d980de4934 # v31.10.1
- uses: cachix/cachix-action@3ba601ff5bbb07c7220846facfa2cd81eeee15a1 # v16
- uses: cachix/cachix-action@1eb2ef646ac0255473d23a5907ad7b04ce94065c # v17
continue-on-error: true
with:
# The nixpkgs-gha cache should not be trusted or used outside of Nixpkgs and its forks' CI.
.github/workflows/eval.yml
+1 -1
View File
@@ -182,7 +182,7 @@ jobs:
- name: Install Nix
uses: cachix/install-nix-action@1ca7d21a94afc7c957383a2d217460d980de4934 # v31.10.1
- uses: cachix/cachix-action@3ba601ff5bbb07c7220846facfa2cd81eeee15a1 # v16
- uses: cachix/cachix-action@1eb2ef646ac0255473d23a5907ad7b04ce94065c # v17
continue-on-error: true
with:
# The nixpkgs-gha cache should not be trusted or used outside of Nixpkgs and its forks' CI.
.github/workflows/lint.yml
+2 -2
View File
@@ -72,7 +72,7 @@ jobs:
- uses: cachix/install-nix-action@1ca7d21a94afc7c957383a2d217460d980de4934 # v31.10.1
- uses: cachix/cachix-action@3ba601ff5bbb07c7220846facfa2cd81eeee15a1 # v16
- uses: cachix/cachix-action@1eb2ef646ac0255473d23a5907ad7b04ce94065c # v17
continue-on-error: true
with:
# The nixpkgs-gha cache should not be trusted or used outside of Nixpkgs and its forks' CI.
@@ -102,7 +102,7 @@ jobs:
- uses: cachix/install-nix-action@1ca7d21a94afc7c957383a2d217460d980de4934 # v31.10.1
- uses: cachix/cachix-action@3ba601ff5bbb07c7220846facfa2cd81eeee15a1 # v16
- uses: cachix/cachix-action@1eb2ef646ac0255473d23a5907ad7b04ce94065c # v17
continue-on-error: true
with:
# The nixpkgs-gha cache should not be trusted or used outside of Nixpkgs and its forks' CI.
doc/release-notes/rl-2605.section.md
+5 -1
View File
@@ -150,6 +150,8 @@
- `python3Packages.gradio` has been updated to version 6. See upstream's migration guide at https://www.gradio.app/main/guides/gradio-6-migration-guide.
- `python3Packages.pikepdf` no longer builds with mupdf support by default, which may be nice in Jupyter and iPython. Build with `withMupdf = true` if this is required.
- `vicinae` has been updated to v0.20. This includes, among several other breaking changes, a complete overhaul of the configuration system. For update instructions, see the [upstream configuration documentation](https://docs.vicinae.com/config#migration-from-v0-16-x-to-v0-17-x).
- The `man-pages` package's outputs have been split. The manual pages are installed into the `man` output, which is installed by default. Binaries (including `diffman-git`, `mansect`, `pdfman`, and `sortman`) are installed into the `out` output, which is not installed by default.
@@ -229,6 +231,8 @@
If your SQLite database is corrupted, the migration might fail and require [manual intervention](https://github.com/louislam/uptime-kuma/issues/5281).
See the [migration guide](https://github.com/louislam/uptime-kuma/wiki/Migration-From-v1-To-v2) for more information.
- The `libcxxhardeningextensive` hardening flag has been **disabled** by default. Enabling it by default in 25.11 was unintentional and may have had a negative effect on performance in some cases. `libcxxhardeningfast` remains enabled by default.
- Switch inhibitors were introduced, which add a pre-switch check that compares a list of strings between the previous and the new generation, and refuses to switch into the new generation when there is a difference between the two lists. This allows to avoid switching into a system when for instance the systemd version changed by adding `config.systemd.package.version` to the switch inhibitors for your system. You can still forcefully switch into any generation by setting `NIXOS_NO_CHECK=1`.
- GNU Taler has been updated to version 1.3.
@@ -239,7 +243,7 @@
- `fetchPnpmDeps` and `pnpmConfigHook` were added as top-level attributes, replacing the now deprecated `pnpm.fetchDeps` and `pnpm.configHook` attributes.
- `buildNpmPackage` now supports `npmDepsCacheVersion`. Set to `2` to enable packument caching, which fixes builds for projects using npm workspaces.
- `buildNpmPackage` now supports `npmDepsFetcherVersion` (and `fetchNpmDeps` now supports `fetcherVersion`). Set to `2` to enable packument caching, which fixes builds for projects using npm workspaces.
- Added `dell-bios-fan-control` package and service.
doc/stdenv/stdenv.chapter.md
+11 -11
View File
@@ -1631,13 +1631,11 @@ Adds the `-fzero-call-used-regs=used-gpr` compiler option. This causes the gener
This flag adds the `-fstack-clash-protection` compiler option, which causes growth of a program's stack to access each successive page in order. This should force the guard page to be accessed and cause an attempt to "jump over" this guard page to crash.
### Hardening flags disabled by default {#sec-hardening-flags-disabled-by-default}
#### `libcxxhardeningfast` {#libcxxhardeningfast}
The following flags are disabled by default and should be enabled with `hardeningEnable` for packages that take untrusted input like network services.
Adds the `-D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_FAST` compiler flag. This flag only has an effect on libc++ targets, and when defined, enables a set of assertions that prevent undefined behavior caused by violating preconditions of the standard library. libc++ provides several hardening modes, and this "fast" mode contains a set of security-critical checks that can be done with relatively little overhead in constant time.
#### `nostrictaliasing` {#nostrictaliasing}
This flag adds the `-fno-strict-aliasing` compiler option, which prevents the compiler from assuming code has been written strictly following the standard in regards to pointer aliasing and therefore performing optimizations that may be unsafe for code that has not followed these rules.
Disabling `libcxxhardeningfast` implies disablement of checks from `libcxxhardeningextensive`.
#### `strictflexarrays1` {#strictflexarrays1}
@@ -1647,6 +1645,14 @@ Enabling this flag on packages that still use length declarations of flexible ar
Disabling `strictflexarrays1` implies disablement of `strictflexarrays3`.
### Hardening flags disabled by default {#sec-hardening-flags-disabled-by-default}
The following flags are disabled by default and should be enabled with `hardeningEnable` for packages that take untrusted input like network services.
#### `nostrictaliasing` {#nostrictaliasing}
This flag adds the `-fno-strict-aliasing` compiler option, which prevents the compiler from assuming code has been written strictly following the standard in regards to pointer aliasing and therefore performing optimizations that may be unsafe for code that has not followed these rules.
#### `strictflexarrays3` {#strictflexarrays3}
This flag adds the `-fstrict-flex-arrays=3` compiler option, which reduces the cases the compiler treats as "flexible arrays" to only those declared with length as (the correct) `[]`. This increases the coverage of fortify checks, because such arrays declared as the trailing element of a structure can normally not have their intended length determined by the compiler.
@@ -1683,12 +1689,6 @@ Adds the `-D_GLIBCXX_ASSERTIONS` compiler flag. This flag only has an effect on
These checks may have an impact on performance in some cases.
#### `libcxxhardeningfast` {#libcxxhardeningfast}
Adds the `-D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_FAST` compiler flag. This flag only has an effect on libc++ targets, and when defined, enables a set of assertions that prevent undefined behavior caused by violating preconditions of the standard library. libc++ provides several hardening modes, and this "fast" mode contains a set of security-critical checks that can be done with relatively little overhead in constant time.
Disabling `libcxxhardeningfast` implies disablement of checks from `libcxxhardeningextensive`.
#### `libcxxhardeningextensive` {#libcxxhardeningextensive}
Adds the `-D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_EXTENSIVE` compiler flag. This flag only has an effect on libc++ targets, and when defined, enables a set of assertions that prevent undefined behavior caused by violating preconditions of the standard library. libc++ provides several hardening modes, and this "extensive" mode adds checks for undefined behavior that incur relatively little overhead but aren’t security-critical. The additional rigour impacts performance more than fast mode: benchmarking is recommended to determine if it is acceptable for a particular application.
lib/meta.nix
+2 -1
View File
@@ -434,8 +434,9 @@ rec {
getLicenseFromSpdxId =
licstr:
getLicenseFromSpdxIdOr licstr (
lib.warn "getLicenseFromSpdxId: No license matches the given SPDX ID: ${licstr}" {
lib.warn "getLicenseFromSpdxId: No license with the given SPDX ID found: ${licstr}" {
shortName = licstr;
spdxId = licstr;
}
);
maintainers/maintainer-list.nix
+19
View File
@@ -13697,6 +13697,13 @@
github = "karaolidis";
githubId = 46189100;
};
karlbeecken = {
name = "Karl Beecken";
email = "karl@beecken.berlin";
github = "karlbeecken";
githubId = 32744028;
keys = [ { fingerprint = "7577 13A4 9609 0C2F 51C4 018C B5C8 89A2 F195 28F6"; } ];
};
KarlJoad = {
email = "karl@hallsby.com";
github = "KarlJoad";
@@ -14176,6 +14183,12 @@
github = "klDen";
githubId = 5478260;
};
klea = {
email = "klea+nixos@iwnp.org";
github = "notklea";
githubId = 231780064;
name = "Klea";
};
klntsky = {
email = "klntsky@gmail.com";
name = "Vladimir Kalnitsky";
@@ -15050,6 +15063,12 @@
githubId = 31388299;
name = "Leonardo Eugênio";
};
lenianiva = {
name = "Leni Aniva";
email = "aniva@stanford.edu";
github = "lenianiva";
githubId = 107011294;
};
lenivaya = {
name = "Danylo Osipchuk";
email = "danylo.osipchuk@proton.me";
nixos/doc/manual/development/running-nixos-tests-interactively.section.md
+15 -4
View File
@@ -10,6 +10,17 @@ $ ./result/bin/nixos-test-driver
>>>
```
::: {.note}
Tests using `systemd-nspawn` container machines require root privileges to run interactively,
since the driver calls `systemd-nspawn` directly to start the containers:
```
$ sudo ./result/bin/nixos-test-driver
[...]
>>>
```
:::
::: {.note}
By executing the test driver in this way,
the VMs executed may gain network & Internet access via their backdoor control interface,
@@ -30,7 +41,7 @@ back into the test driver command line upon its completion. This allows
you to inspect the state of the VMs after the test (e.g. to debug the
test script).
## Shell access in interactive mode {#sec-nixos-test-shell-access}
## Shell access to VMs in interactive mode {#sec-nixos-test-shell-access}
The function `<yourmachine>.shell_interact()` grants access to a shell running
inside a virtual machine. To use it, replace `<yourmachine>` with the name of a
@@ -63,7 +74,7 @@ using:
Once the connection is established, you can enter commands in the socat terminal
where socat is running.
## SSH Access for test machines {#sec-nixos-test-ssh-access}
## SSH Access for test VMs {#sec-nixos-test-ssh-access}
An SSH-based backdoor to log into machines can be enabled with
@@ -149,10 +160,10 @@ must be configured to allow these connections.
## Reuse VM state {#sec-nixos-test-reuse-vm-state}
You can re-use the VM states coming from a previous run by setting the
`--keep-vm-state` flag.
`--keep-machine-state` flag.
```ShellSession
$ ./result/bin/nixos-test-driver --keep-vm-state
$ ./result/bin/nixos-test-driver --keep-machine-state
```
The machine state is stored in the `$TMPDIR/vm-state-machinename`
nixos/doc/manual/development/running-nixos-tests.section.md
+35 -1
View File
@@ -21,10 +21,44 @@ $ nix-store --read-log result
## System Requirements {#sec-running-nixos-tests-requirements}
NixOS tests require virtualization support.
NixOS tests using QEMU virtual machine [`nodes`](#test-opt-nodes) require virtualization support.
This means that the machine must have `kvm` in its [system features](https://nixos.org/manual/nix/stable/command-ref/conf-file.html?highlight=system-features#conf-system-features) list, or `apple-virt` in case of macOS.
These features are autodetected locally, but `apple-virt` is only autodetected since Nix 2.19.0.
Features of **remote builders** must additionally be configured manually on the client, e.g. on NixOS with [`nix.buildMachines.*.supportedFeatures`](https://search.nixos.org/options?show=nix.buildMachines.*.supportedFeatures&sort=alpha_asc&query=nix.buildMachines) or through general [Nix configuration](https://nixos.org/manual/nix/stable/advanced-topics/distributed-builds).
If you run the tests on a **macOS** machine, you also need a "remote" builder for Linux; possibly a VM. [nix-darwin](https://daiderd.com/nix-darwin/) users may enable [`nix.linux-builder.enable`](https://daiderd.com/nix-darwin/manual/index.html#opt-nix.linux-builder.enable) to launch such a VM.
NixOS tests using `systemd-nspawn` [`containers`](#test-opt-containers) require the Nix daemon to be
configured with the following settings:
```nix
{
nix.settings = {
auto-allocate-uids = true;
extra-system-features = [ "uid-range" ];
experimental-features = [
"auto-allocate-uids"
"cgroups"
];
};
}
```
See the documentation of the settings
[`auto-allocate-uids`](https://nix.dev/manual/nix/stable/command-ref/conf-file#conf-auto-allocate-uids),
[`uid-range`](https://nix.dev/manual/nix/stable/command-ref/conf-file.html?highlight=uid-range#conf-system-features), and
[`cgroups`](https://nix.dev/manual/nix/stable/development/experimental-features#xp-feature-cgroups)
for more information.
If the test uses both `systemd-nspawn` [`containers`](#test-opt-containers) and QEMU virtual machine [`nodes`](#test-opt-nodes)
and requires them share a common VLAN,
`/dev/net` must be present in the sandbox.
This allows them to be bridged over a TAP interface.
To make this path available, set the following option:
```nix
{
nix.settings.sandbox-paths = [ "/dev/net" ];
}
```
nixos/doc/manual/development/writing-nixos-tests.section.md
+262 -123
View File
@@ -4,15 +4,14 @@ A NixOS test is a module that has the following structure:
```nix
{
# One or more machines:
# QEMU virtual machines:
nodes = {
machine =
vm1 =
{ config, pkgs, ... }:
{
# ...
};
machine2 =
vm2 =
{ config, pkgs, ... }:
{
# ...
@@ -20,6 +19,20 @@ A NixOS test is a module that has the following structure:
# …
};
# systemd-nspawn containers:
containers = {
container1 =
{ config, pkgs, ... }:
{
# ...
};
container2 =
{ config, pkgs, ... }:
{
# ...
};
};
testScript = ''
Python code…
'';
@@ -27,12 +40,13 @@ A NixOS test is a module that has the following structure:
```
We refer to the whole test above as a test module, whereas the values
in [`nodes.<name>`](#test-opt-nodes) are NixOS modules themselves.
in [`nodes.<name>`](#test-opt-nodes) and [`containers.<name>`](#test-opt-containers)
are NixOS modules themselves.
The option [`testScript`](#test-opt-testScript) is a piece of Python code that executes the
test (described below). During the test, it will start one or more
virtual machines, the configuration of which is described by
the option [`nodes`](#test-opt-nodes).
test (described [below](#ssec-test-script)). During the test, it will start one or more
virtual machines and/or `systemd-nspawn` containers, the configuration of which is described by
the options [`nodes`](#test-opt-nodes) and [`containers`](#test-opt-containers), respectively.
An example of a single-node test is
[`login.nix`](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/login.nix).
@@ -42,6 +56,12 @@ when switching between consoles, and so on. An interesting multi-node test is
[`nfs/simple.nix`](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nfs/simple.nix).
It uses two client nodes to test correct locking across server crashes.
A test can contain both virtual machines and containers.
If configured to share a common VLAN,
they can reach each other over the network.
See [](https://github.com/applicative-systems/nixpkgs/blob/78100f9077ab50604ab9c9514e442bbc7ac7ca5b/nixos/tests/nixos-test-driver/containers.nix)
for an example of this and [](#sec-running-nixos-tests-requirements) for the system requirements for this scenario.
## Calling a test {#sec-calling-nixos-tests}
Tests are invoked differently depending on whether the test is part of NixOS or lives in a different project.
@@ -90,13 +110,54 @@ pkgs.testers.runNixOSTest {
`runNixOSTest` returns a derivation that runs the test.
## Configuring the nodes {#sec-nixos-test-nodes}
## Test machines {#ssec-nixos-test-machines}
There are a few special NixOS options for test VMs:
A NixOS test usually consists of one or more test machines. Each machine is either a
QEMU virtual machine or a `systemd-nspawn` container.
`virtualisation.memorySize`
QEMU virtual machines are defined in the
[`nodes`](#test-opt-nodes) attribute set, whereas `systemd-nspawn` containers are defined in the
[`containers`](#test-opt-containers) attribute set.
: The memory of the VM in MiB (1024×1024 bytes).
To set NixOS options for all machines in the test, use the attribute
[`defaults`](#test-opt-defaults). These options are applied to both virtual machines
and containers. You can set separate defaults for virtual machines and containers
using the attributes [`nodeDefaults`](#test-opt-nodeDefaults) and
[`containerDefaults`](#test-opt-containerDefaults), respectively.
### Virtual machines vs. containers {#sec-nixos-test-vms-vs-containers}
QEMU virtual machines and `systemd-nspawn` containers offer different
trade-offs which make them suitable for different use cases.
Some advantages of containers over virtual machines are:
- Containers share the kernel of the host system; they are
significantly faster to start up than virtual machines.
- Containers are more lightweight in terms of resource usage, which
allows running more of them in parallel on a single host.
- Containers can easily be run in virtualised environments, e.g., CI systems.
- Containers allow direct bind-mounting of host device nodes, which enables
testing of GPU code (CUDA), for example.
Some advantages of virtual machines over containers are:
- Virtual machines run a separate kernel, which allows testing kernel features
(kernel modules, etc.).
- Virtual machines support testing graphical applications on X11.
- Virtual machines allow testing NixOS modules that use systemd's namespacing options (such as `ProtectSystem=` or `MountAPIVFS=`).
- Virtual machines allow testing [`spcialisation`](options.html#opt-specialisation).
(Switching to a specialisation requires the creation of SUID/SGID wrappers, which is disallowed in `systemd-nspawn` within the Nix sandbox.)
- Virtual machines allow the execution of `setuid` binaries.
Refer to the sections on [QEMU virtual machines](#ssec-nixos-test-qemu-vms)
and [systemd-nspawn containers](#ssec-nixos-test-nspawn-containers) below
for more details on configuring each type of machine.
### Configuring test machines {#sec-nixos-test-machines-config}
The following special NixOS option can be used to configure
machines in a NixOS test, whether they are virtual machines or containers:
`virtualisation.vlans`
@@ -104,6 +165,35 @@ There are a few special NixOS options for test VMs:
[`nat.nix`](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nat.nix)
for an example.
#### Configuring `systemd-nspawn` containers {#ssec-nixos-test-nspawn-containers}
Some options are specific to `systemd-nspawn` containers:
`virtualisation.systemd-nspawn.options`
: A list of additional command-line options to pass to
`systemd-nspawn` when starting the container. For example, to
bind mount a directory from the host into the container, you could
use: `[ "--bind=/host/dir:/container/dir" ]`.
For more options, see the module
[`nspawn-container`](https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/nspawn-container/default.nix).
Note that the paths used in `--bind` or `--bind-ro` options have to be accessible from within the Nix sandbox.
Use the Nix option
[`sandbox-paths`](https://nix.dev/manual/nix/stable/command-ref/conf-file#conf-sandbox-paths)
and/or the module [`programs.nix-required-mounts`](#opt-programs.nix-required-mounts.enable) on the host
to add additional paths to the sandbox.
#### Configuring QEMU virtual machines {#ssec-nixos-test-qemu-vms}
Some options are specific to QEMU virtual machines:
`virtualisation.memorySize`
: The memory of the VM in MiB (1024×1024 bytes).
`virtualisation.writableStore`
: By default, the Nix store in the VM is not writable. If you enable
@@ -114,13 +204,15 @@ There are a few special NixOS options for test VMs:
For more options, see the module
[`qemu-vm.nix`](https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/qemu-vm.nix).
## Writing the test script {#ssec-test-script}
The test script is a sequence of Python statements that perform various
actions, such as starting VMs, executing commands in the VMs, and so on.
Each virtual machine is represented as an object stored in the variable
`name` if this is also the identifier of the machine in the declarative
config. If you specified a node `nodes.machine`, the following example starts the
machine, waits until it has finished booting, then executes a command
and checks that the output is more-or-less correct:
actions, such as starting machines, executing commands in them, and so on. For
example, if you specified a virtual machine in `nodes.machine`, there will be
a Python variable `machine` available in the test script that represents that
virtual machine. The following example would start the machine, wait until it
has finished booting, and then execute a command and check that the output is
more-or-less correct:
```py
machine.start()
@@ -139,17 +231,20 @@ start_all()
Under the variable `t`, all assertions from [`unittest.TestCase`](https://docs.python.org/3/library/unittest.html) are available.
If the hostname of a node contains characters that can't be used in a
If the hostname of a machine contains characters that can't be used in a
Python variable name, those characters will be replaced with
underscores in the variable name, so `nodes.machine-a` will be exposed
to Python as `machine_a`.
## Machine objects {#ssec-machine-objects}
### Methods available on machine objects {#ssec-machine-objects}
The following methods are available on machine objects:
The following methods are available on machine objects (like `machine` in
the examples above):
@PYTHON_MACHINE_METHODS@
### Testing user units {#ssec-testing-user-units}
To test user units declared by `systemd.user.services` the optional
`user` argument can be used:
@@ -162,6 +257,84 @@ machine.wait_for_unit("xautolock.service", "x-session-user")
This applies to `systemctl`, `get_unit_info`, `wait_for_unit`,
`start_job` and `stop_job`.
### Failing tests early {#ssec-failing-tests-early}
To fail tests early when certain invariants are no longer met (instead of waiting for the build to time out), the decorator `polling_condition` is provided. For example, if we are testing a program `foo` that should not quit after being started, we might write the following:
```py
@polling_condition
def foo_running():
machine.succeed("pgrep -x foo")
machine.succeed("foo --start")
machine.wait_until_succeeds("pgrep -x foo")
with foo_running:
... # Put `foo` through its paces
```
`polling_condition` takes the following (optional) arguments:
`seconds_interval`
: specifies how often the condition should be polled:
```py
@polling_condition(seconds_interval=10)
def foo_running():
machine.succeed("pgrep -x foo")
```
`description`
: is used in the log when the condition is checked. If this is not provided, the description is pulled from the docstring of the function. These two are therefore equivalent:
```py
@polling_condition
def foo_running():
"check that foo is running"
machine.succeed("pgrep -x foo")
```
```py
@polling_condition(description="check that foo is running")
def foo_running():
machine.succeed("pgrep -x foo")
```
### Adding Python packages to the test script {#ssec-python-packages-in-test-script}
When additional Python libraries are required in the test script, they can be
added using the parameter `extraPythonPackages`. For example, you could add
`numpy` like this:
```nix
{
extraPythonPackages = p: [ p.numpy ];
nodes = { };
# Type checking on extra packages doesn't work yet
skipTypeCheck = true;
testScript = ''
import numpy as np
assert str(np.zeros(4)) == "[0. 0. 0. 0.]"
'';
}
```
In that case, `numpy` is chosen from the generic `python3Packages`.
### Linting and type checking test scripts {#ssec-test-script-checks}
Test scripts are automatically linted with
[Pyflakes](https://pypi.org/project/pyflakes/) and type-checked with
[Mypy](https://mypy.readthedocs.io/en/stable/).
If there are any linting or type checking errors, the test will
fail to evaluate.
For faster dev cycles it's also possible to disable the code-linters
(this shouldn't be committed though):
@@ -209,76 +382,6 @@ way:
}
```
## Failing tests early {#ssec-failing-tests-early}
To fail tests early when certain invariants are no longer met (instead of waiting for the build to time out), the decorator `polling_condition` is provided. For example, if we are testing a program `foo` that should not quit after being started, we might write the following:
```py
@polling_condition
def foo_running():
machine.succeed("pgrep -x foo")
machine.succeed("foo --start")
machine.wait_until_succeeds("pgrep -x foo")
with foo_running:
... # Put `foo` through its paces
```
`polling_condition` takes the following (optional) arguments:
`seconds_interval`
: specifies how often the condition should be polled:
```py
@polling_condition(seconds_interval=10)
def foo_running():
machine.succeed("pgrep -x foo")
```
`description`
: is used in the log when the condition is checked. If this is not provided, the description is pulled from the docstring of the function. These two are therefore equivalent:
```py
@polling_condition
def foo_running():
"check that foo is running"
machine.succeed("pgrep -x foo")
```
```py
@polling_condition(description="check that foo is running")
def foo_running():
machine.succeed("pgrep -x foo")
```
## Adding Python packages to the test script {#ssec-python-packages-in-test-script}
When additional Python libraries are required in the test script, they can be
added using the parameter `extraPythonPackages`. For example, you could add
`numpy` like this:
```nix
{
extraPythonPackages = p: [ p.numpy ];
nodes = { };
# Type checking on extra packages doesn't work yet
skipTypeCheck = true;
testScript = ''
import numpy as np
assert str(np.zeros(4)) == "[0. 0. 0. 0.]"
'';
}
```
In that case, `numpy` is chosen from the generic `python3Packages`.
## Overriding a test {#sec-override-nixos-test}
The NixOS test framework returns tests with multiple overriding methods.
@@ -297,7 +400,7 @@ The NixOS test framework returns tests with multiple overriding methods.
: Evaluates the test with additional NixOS modules and/or arguments.
`module`
: A NixOS module to add to all the nodes in the test. Sets test option [`extraBaseModules`](#test-opt-extraBaseModules).
: A NixOS module to add to all the machines in the test. Sets test option [`extraBaseModules`](#test-opt-extraBaseModules).
`specialArgs`
: An attribute set of arguments to pass to all NixOS modules. These override the existing arguments, as well as any `_module.args.<name>` that the modules may define. Sets test option [`node.specialArgs`](#test-opt-node.specialArgs).
@@ -345,7 +448,52 @@ list-id: test-options-list
source: @NIXOS_TEST_OPTIONS_JSON@
```
## Accessing VMs in the sandbox with SSH {#sec-test-sandbox-breakpoint}
## Debugging test machines {#sec-test-sandbox-breakpoint}
You can set the [`enableDebugHook`](#test-opt-enableDebugHook) option to pause
a test on the first failure and have it print instructions on how to enter the
sandbox shell of the test. Suppose you have the following test module:
```nix
{
name = "foo";
nodes.machine = { };
enableDebugHook = true;
sshBackdoor.enable = true;
testScript = ''
start_all()
machine.succeed("false") # this will fail
'';
}
```
The test will fail with an output like this:
```
vm-test-run-foo> !!! Breakpoint reached, run 'sudo /nix/store/eeeee-attach/bin/attach <PID>'
```
You can then enter the sandbox shell:
```
$ sudo /nix/store/eeeee-attach/bin/attach <PID>
bash#
```
There, you can attach to a [`pdb`](https://docs.python.org/3/library/pdb.html) session
to step through the Python test script:
```
bash# telnet 127.0.0.1 4444
pdb$
```
Note that it is also possible to set breakpoints in the test script using `debug.breakpoint()`.
### SSH access to test VMs {#sec-test-vm-ssh-access}
::: {.note}
For debugging with SSH access into the machines, it's recommended to try using
@@ -356,24 +504,15 @@ This feature is mostly intended to debug flaky test failures that aren't
reproducible elsewhere.
:::
As explained in [](#sec-nixos-test-ssh-access), it's possible to configure an
SSH backdoor based on AF_VSOCK. This can be used to SSH into a VM of a running
build in a sandbox.
This can be done when something in the test fails, e.g.
If you set the [`sshBackdoor.enable`](#test-opt-sshBackdoor.enable) option,
QEMU virtual machines will open an SSH backdoor based on AF_VSOCK
(see [](#sec-nixos-test-ssh-access)).
Once you are in the sandbox shell, you can access the VMs (for example, `machine`)
with SSH over vsock:
```nix
{
nodes.machine = { };
sshBackdoor.enable = true;
enableDebugHook = true;
testScript = ''
start_all()
machine.succeed("false") # this will fail
'';
}
```
bash# ssh -F ./ssh_config vsock/3
```
For the AF_VSOCK feature to work, `/dev/vhost-vsock` is needed in the sandbox
@@ -383,24 +522,24 @@ which can be done with e.g.
nix-build -A nixosTests.foo --option sandbox-paths /dev/vhost-vsock
```
This will halt the test execution on a test-failure and print instructions
on how to enter the sandbox shell of the VM test. Inside, one can log into
e.g. `machine` with
```
ssh -F ./ssh_config vsock/3
```
As described in [](#sec-nixos-test-ssh-access), the numbers for vsock start at
`3` instead of `1`. So the first VM in the network (sorted alphabetically) can
be accessed with `vsock/3`.
Alternatively, it's possible to explicitly set a breakpoint with
`debug.breakpoint()`. This also has the benefit, that one can step through
`testScript` with `pdb` like this:
### SSH access to test containers {#sec-test-container-ssh-access}
If you set the [`sshBackdoor.enable`](#test-opt-sshBackdoor.enable) option,
each `systemd-nspawn` container will open an SSH backdoor.
Once the container starts,
it will print instructions on how to log into the container via SSH.
If the test fails,
attach to the sandbox as described above,
and then use the provided SSH command to log into the container.
For example:
```
$ sudo /nix/store/eeeee-attach <id>
bash# telnet 127.0.0.1 4444
pdb$ …
$ sudo /nix/store/eeeee-attach <PID>
bash# ssh -o User=root -o ProxyCommand="socat - UNIX-CLIENT:/run/systemd/nspawn/unix-export/machine/ssh" bash
[root@machine:~]# hostname
machine
```
nixos/doc/manual/redirects.json
+52 -3
View File
@@ -88,6 +88,9 @@
"module-virtualisation-xen-introduction": [
"index.html#module-virtualisation-xen-introduction"
],
"sec-nixos-test-vms-vs-containers": [
"index.html#sec-nixos-test-vms-vs-containers"
],
"sec-override-nixos-test": [
"index.html#sec-override-nixos-test"
],
@@ -100,6 +103,51 @@
"sec-wireless-imperative": [
"index.html#sec-wireless-imperative"
],
"sec-test-container-ssh-access": [
"index.html#sec-test-container-ssh-access"
],
"sec-test-vm-ssh-access": [
"index.html#sec-test-vm-ssh-access"
],
"ssec-all-machine-objects": [
"index.html#ssec-all-machine-objects"
],
"ssec-nixos-test-machines": [
"index.html#ssec-nixos-test-machines"
],
"ssec-nixos-test-nspawn-containers": [
"index.html#ssec-nixos-test-nspawn-containers"
],
"ssec-nixos-test-qemu-vms": [
"index.html#ssec-nixos-test-qemu-vms"
],
"ssec-nspawn-machine-objects": [
"index.html#ssec-nspawn-machine-objects"
],
"ssec-qemu-machine-objects": [
"index.html#ssec-qemu-machine-objects"
],
"ssec-test-script": [
"index.html#ssec-test-script"
],
"ssec-test-script-checks": [
"index.html#ssec-test-script-checks"
],
"ssec-testing-user-units": [
"index.html#ssec-testing-user-units"
],
"test-opt-containerDefaults": [
"index.html#test-opt-containerDefaults"
],
"test-opt-containers": [
"index.html#test-opt-containers"
],
"test-opt-extraBaseModules": [
"index.html#test-opt-extraBaseModules"
],
"test-opt-nodeDefaults": [
"index.html#test-opt-nodeDefaults"
],
"test-opt-rawTestDerivationArg": [
"index.html#test-opt-rawTestDerivationArg"
],
@@ -2037,7 +2085,8 @@
"sec-call-nixos-test-outside-nixos": [
"index.html#sec-call-nixos-test-outside-nixos"
],
"sec-nixos-test-nodes": [
"sec-nixos-test-machines-config": [
"index.html#sec-nixos-test-machines-config",
"index.html#sec-nixos-test-nodes"
],
"ssec-machine-objects": [
@@ -2070,8 +2119,8 @@
"test-opt-enableOCR": [
"index.html#test-opt-enableOCR"
],
"test-opt-extraBaseModules": [
"index.html#test-opt-extraBaseModules"
"test-opt-extraBaseNodeModules": [
"index.html#test-opt-extraBaseNodeModules"
],
"test-opt-extraDriverArgs": [
"index.html#test-opt-extraDriverArgs"
nixos/doc/manual/release-notes/rl-2605.section.md
+4
View File
@@ -247,6 +247,8 @@ See <https://github.com/NixOS/nixpkgs/issues/481673>.
- `services.caddy` now supports setting `httpPort` and `httpsPort` and opening them in the firewall via `openFirewall`.
- `boot.initrd.secrets` is now deprecated in favour of `boot.initrd.secretPaths` and `boot.initrd.extraSecretsHook`.
- The latest available version of Nextcloud is v33 (available as `pkgs.nextcloud33`). The installation logic is as follows:
- If [`services.nextcloud.package`](#opt-services.nextcloud.package) is specified explicitly, this package will be installed (**recommended**)
- If [`system.stateVersion`](#opt-system.stateVersion) is >=26.05, `pkgs.nextcloud33` will be installed by default.
@@ -306,3 +308,5 @@ See <https://github.com/NixOS/nixpkgs/issues/481673>.
```
**Do not set this globally!** This makes your setup inherently less secure.
- `services.radicle` now supports importing the private key and passphrase as systemd creds.
nixos/lib/systemd-unit-options.nix
-4
View File
@@ -3,12 +3,8 @@
let
inherit (systemdUtils.lib)
assertValueOneOf
automountConfig
checkUnitConfig
makeJobScript
mountConfig
serviceConfig
unitConfig
unitNameType
;
nixos/lib/test-driver/default.nix
+7
View File
@@ -19,9 +19,12 @@
qemu_test,
setuptools,
socat,
systemd,
tesseract4,
util-linux,
vde2,
enableNspawn ? false,
enableOCR ? false,
extraPythonPackages ? (_: [ ]),
}:
@@ -51,8 +54,12 @@ buildPythonApplication {
netpbm
qemu_pkg
socat
util-linux
vde2
]
++ lib.optionals enableNspawn [
systemd
]
++ lib.optionals enableOCR [
imagemagick_light
tesseract4
nixos/lib/test-driver/src/extract-docstrings.py
+61 -14
View File
@@ -42,6 +42,43 @@ some_function(param1, param2)
"""
def function_docstrings(functions: list[ast.FunctionDef]) -> str | None:
"""Extracts docstrings from a list of function definitions."""
documented_functions = [f for f in functions if ast.get_docstring(f) is not None]
if not documented_functions:
return None
docstrings = []
for function in documented_functions:
docstr = ast.get_docstring(function)
assert docstr is not None
args = ", ".join(a.arg for a in function.args.args[1:])
args = f"({args})"
docstr = "\n".join(f" {line}" for line in docstr.strip().splitlines())
docstrings.append(f"{function.name}{args}\n\n:{docstr[1:]}\n")
return "\n".join(docstrings)
def machine_methods(
class_name: str, class_definitions: list[ast.ClassDef]
) -> list[ast.FunctionDef]:
"""Given a class name and a list of class definitions, returns the list of function definitions
for the class matching the given name.
"""
machine_class = next(filter(lambda x: x.name == class_name, class_definitions))
assert machine_class is not None
function_definitions = [
node for node in machine_class.body if isinstance(node, ast.FunctionDef)
]
function_definitions.sort(key=lambda x: x.name)
return function_definitions
def main() -> None:
if len(sys.argv) != 2:
print(f"Usage: {sys.argv[0]} <path-to-test-driver>")
@@ -49,25 +86,35 @@ def main() -> None:
module = ast.parse(Path(sys.argv[1]).read_text())
class_definitions = (node for node in module.body if isinstance(node, ast.ClassDef))
class_definitions = [node for node in module.body if isinstance(node, ast.ClassDef)]
machine_class = next(filter(lambda x: x.name == "Machine", class_definitions))
assert machine_class is not None
base_machine_methods = machine_methods("BaseMachine", class_definitions)
base_method_names = {method.name for method in base_machine_methods}
function_definitions = [
node for node in machine_class.body if isinstance(node, ast.FunctionDef)
qemu_machine_methods = [
method
for method in machine_methods("QemuMachine", class_definitions)
if method.name not in base_method_names
]
function_definitions.sort(key=lambda x: x.name)
for function in function_definitions:
docstr = ast.get_docstring(function)
if docstr is not None:
args = ", ".join(a.arg for a in function.args.args[1:])
args = f"({args})"
nspawn_machine_methods = [
method
for method in machine_methods("NspawnMachine", class_definitions)
if method.name not in base_method_names
]
docstr = "\n".join(f" {line}" for line in docstr.strip().splitlines())
print(f"{function.name}{args}\n\n:{docstr[1:]}\n")
print("#### Generic machine objects {#ssec-all-machine-objects} \n")
print(function_docstrings(base_machine_methods))
print("#### QEMU VM objects {#ssec-qemu-machine-objects}\n")
print(
function_docstrings(qemu_machine_methods)
or "No methods specific to QEMU virtual machines."
)
print("#### `systemd-nspawn` container objects {#ssec-nspawn-machine-objects}\n")
print(
function_docstrings(nspawn_machine_methods)
or "No methods specific to `systemd-nspawn` containers."
)
if __name__ == "__main__":
nixos/lib/test-driver/src/test_driver/__init__.py
Executable → Regular
+77 -18
View File
@@ -1,6 +1,8 @@
import argparse
import os
import sys
import time
import warnings
from pathlib import Path
import ptpython.ipython
@@ -16,7 +18,7 @@ from test_driver.logger import (
class EnvDefault(argparse.Action):
"""An argpars Action that takes values from the specified
"""An argparse Action that takes values from the specified
environment variable as the flags default value.
"""
@@ -55,9 +57,15 @@ def writeable_dir(arg: str) -> Path:
def main() -> None:
arg_parser = argparse.ArgumentParser(prog="nixos-test-driver")
arg_parser.add_argument(
"-K",
"--keep-vm-state",
help="re-use a VM state coming from a previous run",
help=argparse.SUPPRESS,
dest="keep_machine_state",
action="store_true",
)
arg_parser.add_argument(
"-K",
"--keep-machine-state",
help="re-use a machine state coming from a previous run",
action="store_true",
)
arg_parser.add_argument(
@@ -71,13 +79,37 @@ def main() -> None:
help="Enable interactive debugging breakpoints for sandboxed runs",
)
arg_parser.add_argument(
"--start-scripts",
metavar="START-SCRIPT",
"--vm-names",
metavar="VM-NAME",
action=EnvDefault,
envvar="startScripts",
envvar="vmNames",
nargs="*",
help="names of participating virtual machines",
)
arg_parser.add_argument(
"--vm-start-scripts",
metavar="VM-START-SCRIPT",
action=EnvDefault,
envvar="vmStartScripts",
nargs="*",
help="start scripts for participating virtual machines",
)
arg_parser.add_argument(
"--container-names",
metavar="CONTAINER-NAME",
action=EnvDefault,
envvar="containerNames",
nargs="*",
help="names of participating containers",
)
arg_parser.add_argument(
"--container-start-scripts",
metavar="CONTAINER-START-SCRIPT",
action=EnvDefault,
envvar="containerStartScripts",
nargs="*",
help="start scripts for participating containers",
)
arg_parser.add_argument(
"--vlans",
metavar="VLAN",
@@ -97,8 +129,8 @@ def main() -> None:
arg_parser.add_argument(
"-o",
"--output_directory",
help="""The path to the directory where outputs copied from the VM will be placed.
By e.g. Machine.copy_from_vm or Machine.screenshot""",
help="""The path to the directory where outputs copied from the machine will be placed.
By e.g. NspawnMachine.copy_from_machine or QemuMachine.screenshot""",
default=Path.cwd(),
type=writeable_dir,
)
@@ -122,6 +154,12 @@ def main() -> None:
args = arg_parser.parse_args()
if "--keep-vm-state" in sys.argv:
warnings.warn(
"The flag '--keep-vm-state' is deprecated. Use '--keep-machine-state' instead.",
DeprecationWarning,
)
output_directory = args.output_directory.resolve()
logger = CompositeLogger([TerminalLogger()])
@@ -131,21 +169,33 @@ def main() -> None:
if args.junit_xml:
logger.add_logger(JunitXMLLogger(output_directory / args.junit_xml))
if not args.keep_vm_state:
logger.info("Machine state will be reset. To keep it, pass --keep-vm-state")
if not args.keep_machine_state:
logger.info(
"Machine state will be reset. To keep it, pass --keep-machine-state"
)
debugger: DebugAbstract = DebugNop()
if args.debug_hook_attach is not None:
debugger = Debug(logger, args.debug_hook_attach)
assert len(args.vm_names) == len(args.vm_start_scripts), (
f"the number of vm names and vm start scripts must be the same: {args.vm_names} vs. {args.vm_start_scripts}"
)
assert len(args.container_names) == len(args.container_start_scripts), (
f"the number of container names and container start scripts must be the same: {args.container_names} vs. {args.container_start_scripts}"
)
with Driver(
args.start_scripts,
args.vlans,
args.testscript.read_text(),
output_directory,
logger,
args.keep_vm_state,
args.global_timeout,
vm_names=args.vm_names,
vm_start_scripts=args.vm_start_scripts,
container_names=args.container_names,
container_start_scripts=args.container_start_scripts,
vlans=args.vlans,
tests=args.testscript.read_text(),
out_dir=output_directory,
logger=logger,
keep_machine_state=args.keep_machine_state,
global_timeout=args.global_timeout,
debug=debugger,
) as driver:
if offset := args.dump_vsocks:
@@ -170,7 +220,16 @@ def generate_driver_symbols() -> None:
in user's test scripts. That list is then used by pyflakes to lint those
scripts.
"""
d = Driver([], [], "", Path(), CompositeLogger([]))
d = Driver(
vm_names=[],
vm_start_scripts=[],
container_names=[],
container_start_scripts=[],
vlans=[],
tests="",
out_dir=Path(),
logger=CompositeLogger([]),
)
test_symbols = d.test_symbols()
with open("driver-symbols", "w") as fp:
fp.write(",".join(test_symbols.keys()))
nixos/lib/test-driver/src/test_driver/driver.py
+119 -30
View File
@@ -1,6 +1,7 @@
import os
import re
import signal
import subprocess
import sys
import tempfile
import threading
@@ -16,7 +17,12 @@ from colorama import Style
from test_driver.debug import DebugAbstract, DebugNop
from test_driver.errors import MachineError, RequestedAssertionFailed
from test_driver.logger import AbstractLogger
from test_driver.machine import Machine, NixStartScript, retry
from test_driver.machine import (
BaseMachine,
NspawnMachine,
QemuMachine,
retry,
)
from test_driver.polling_condition import PollingCondition
from test_driver.vlan import VLan
@@ -63,7 +69,8 @@ class Driver:
tests: str
vlans: list[VLan]
machines: list[Machine]
machines_qemu: list[QemuMachine]
machines_nspawn: list[NspawnMachine]
polling_conditions: list[PollingCondition]
global_timeout: int
race_timer: threading.Timer
@@ -72,12 +79,15 @@ class Driver:
def __init__(
self,
start_scripts: list[str],
vm_names: list[str],
vm_start_scripts: list[str],
container_names: list[str],
container_start_scripts: list[str],
vlans: list[int],
tests: str,
out_dir: Path,
logger: AbstractLogger,
keep_vm_state: bool = False,
keep_machine_state: bool = False,
global_timeout: int = 24 * 60 * 60 * 7,
debug: DebugAbstract = DebugNop(),
):
@@ -94,25 +104,95 @@ class Driver:
vlans = list(set(vlans))
self.vlans = [VLan(nr, tmp_dir, self.logger) for nr in vlans]
def cmd(scripts: list[str]) -> Iterator[NixStartScript]:
for s in scripts:
yield NixStartScript(s)
self.polling_conditions = []
self.machines = [
Machine(
start_command=cmd,
keep_vm_state=keep_vm_state,
name=cmd.machine_name,
self.machines_qemu = [
QemuMachine(
name=name,
start_command=vm_start_script,
keep_machine_state=keep_machine_state,
tmp_dir=tmp_dir,
callbacks=[self.check_polling_conditions],
out_dir=self.out_dir,
logger=self.logger,
)
for cmd in cmd(start_scripts)
for name, vm_start_script in zip(vm_names, vm_start_scripts)
]
if len(container_start_scripts) > 0:
self._init_nspawn_environment()
self.machines_nspawn = [
NspawnMachine(
name=name,
start_command=container_start_script,
tmp_dir=tmp_dir,
logger=self.logger,
keep_machine_state=keep_machine_state,
callbacks=[self.check_polling_conditions],
out_dir=self.out_dir,
)
for name, container_start_script in zip(
container_names,
container_start_scripts,
)
]
def _init_nspawn_environment(self) -> None:
assert os.geteuid() == 0, (
f"systemd-nspawn requires root to work. You are {os.geteuid()}"
)
# set up prerequisites for systemd-nspawn containers.
# these are not guaranteed to be set up in the Nix sandbox.
# if running interactively as root, these will already be set up.
# check if /run is writable by root
if not os.access("/run", os.W_OK):
Path("/run").mkdir(parents=True, exist_ok=True)
subprocess.run(["mount", "-t", "tmpfs", "none", "/run"], check=True)
Path("/run/netns").mkdir(parents=True, exist_ok=True)
# check if /var/run is a symlink to /run
if not (os.path.exists("/var/run") and os.path.samefile("/var/run", "/run")):
Path("/var").mkdir(parents=True, exist_ok=True)
subprocess.run(["ln", "-s", "/run", "/var/run"], check=True)
# check if /sys/fs/cgroup is mounted as cgroup2
with open("/proc/mounts", encoding="utf-8") as mounts:
for line in mounts:
parts = line.split()
if len(parts) >= 3 and parts[1] == "/sys/fs/cgroup":
if parts[2] == "cgroup2":
break
else:
Path("/sys/fs/cgroup").mkdir(parents=True, exist_ok=True)
subprocess.run(
["mount", "-t", "cgroup2", "none", "/sys/fs/cgroup"], check=True
)
# systemd-nspawn requires that /etc/os-release exists
# It supports SYSTEMD_NSPAWN_CHECK_OS_RELEASE=0, but that
# would try to "fix" it by bind mounting, which is worse.
if not os.path.isfile("/etc/os-release"):
subprocess.run(["touch", "/etc/os-release"], check=True)
# ensure /etc/machine-id exists and is non-empty
if (
not os.path.isfile("/etc/machine-id")
or os.path.getsize("/etc/machine-id") == 0
):
subprocess.run(
["systemd-machine-id-setup"], check=True
) # set up /etc/machine-id
@property
def machines(self) -> list[QemuMachine | NspawnMachine]:
machines = self.machines_qemu + self.machines_nspawn
# Sort the machines by name for consistency with `nodesAndContainers` in <nixos/lib/testing/network.nix>.
machines.sort(key=lambda machine: machine.name)
return machines
def __enter__(self) -> "Driver":
return self
@@ -148,7 +228,8 @@ class Driver:
general_symbols = dict(
start_all=self.start_all,
test_script=self.test_script,
machines=self.machines,
machines_qemu=self.machines_qemu,
machines_nspawn=self.machines_nspawn,
vlans=self.vlans,
driver=self,
log=self.logger,
@@ -161,7 +242,7 @@ class Driver:
serial_stdout_off=self.serial_stdout_off,
serial_stdout_on=self.serial_stdout_on,
polling_condition=self.polling_condition,
Machine=Machine, # for typing
BaseMachine=BaseMachine, # for typing
t=AssertionTester(),
debug=self.debug,
)
@@ -186,14 +267,14 @@ class Driver:
def dump_machine_ssh(self, offset: int) -> None:
print("SSH backdoor enabled, the machines can be accessed like this:")
print(
f"{Style.BRIGHT}Note:{Style.RESET_ALL} this requires {Style.BRIGHT}systemd-ssh-proxy(1){Style.RESET_ALL} to be enabled (default on NixOS 25.05 and newer)."
f"{Style.BRIGHT}Note:{Style.RESET_ALL} vsocks require {Style.BRIGHT}systemd-ssh-proxy(1){Style.RESET_ALL} to be enabled (default on NixOS 25.05 and newer)."
)
names = [machine.name for machine in self.machines]
longest_name = len(max(names, key=len))
for num, name in enumerate(names, start=offset + 1):
longest_name = len(max((machine.name for machine in self.machines), key=len))
for index, machine in enumerate(self.machines, start=offset + 1):
name = machine.name
spaces = " " * (longest_name - len(name) + 2)
print(
f" {name}:{spaces}{Style.BRIGHT}ssh -o User=root vsock/{num}{Style.RESET_ALL}"
f" {name}:{spaces}{Style.BRIGHT}{machine.ssh_backdoor_command(index)}{Style.RESET_ALL}"
)
def test_script(self) -> None:
@@ -252,8 +333,16 @@ class Driver:
def start_all(self) -> None:
"""Start all machines"""
with self.logger.nested("start all VMs"):
threads = []
for machine in self.machines:
machine.start()
# Create a thread for each machine's start method
t = threading.Thread(target=machine.start, name=f"start-{machine.name}")
threads.append(t)
t.start()
# Wait for all startup threads to complete before proceeding
for t in threads:
t.join()
def join_all(self) -> None:
"""Wait for all machines to shut down"""
@@ -279,19 +368,19 @@ class Driver:
start_command: str,
*,
name: str | None = None,
keep_vm_state: bool = False,
) -> Machine:
keep_machine_state: bool = False,
) -> BaseMachine:
"""
Create a `QemuMachine`. This currently only supports qemu "nodes", not containers.
"""
tmp_dir = get_tmp_dir()
cmd = NixStartScript(start_command)
name = name or cmd.machine_name
return Machine(
return QemuMachine(
tmp_dir=tmp_dir,
out_dir=self.out_dir,
start_command=cmd,
start_command=start_command,
name=name,
keep_vm_state=keep_vm_state,
keep_machine_state=keep_machine_state,
logger=self.logger,
)
nixos/lib/test-driver/src/test_driver/machine/__init__.py
+754 -385
View File
File diff suppressed because it is too large Load Diff
nixos/lib/test-driver/src/test_driver/vlan.py
+13
View File
@@ -50,6 +50,8 @@ class VLan:
pid: int
fd: io.TextIOBase
plug_process: subprocess.Popen
logger: AbstractLogger
def __repr__(self) -> str:
@@ -58,6 +60,7 @@ class VLan:
def __init__(self, nr: int, tmp_dir: Path, logger: AbstractLogger):
self.nr = nr
self.socket_dir = tmp_dir / f"vde{self.nr}.ctl"
self.tap_name = f"vde-tap{self.nr}"
self.logger = logger
# TODO: don't side-effect environment here
@@ -114,6 +117,13 @@ class VLan:
if "1000 Success" in line:
break
# This is needed to allow systemd-nspawn containers to communicate
# with VMs connected to the VLAN.
self.logger.info(f"creating tap interface {self.tap_name}")
self.plug_process = subprocess.Popen(
["vde_plug2tap", "-s", self.socket_dir, self.tap_name],
)
assert (self.socket_dir / "ctl").exists(), "cannot start vde_switch"
self.logger.info(f"running vlan (pid {self.pid}; ctl {self.socket_dir})")
@@ -122,4 +132,7 @@ class VLan:
self.logger.info(f"kill vlan (pid {self.pid})")
assert self.process.stdin is not None
self.process.stdin.close()
if self.plug_process:
self.plug_process.terminate()
self.plug_process.wait()
self.process.terminate()
nixos/lib/test-script-prepend.py
+5 -4
View File
@@ -4,7 +4,7 @@
from test_driver.debug import DebugAbstract
from test_driver.driver import Driver
from test_driver.vlan import VLan
from test_driver.machine import Machine
from test_driver.machine import BaseMachine, NspawnMachine, QemuMachine
from test_driver.logger import AbstractLogger
from typing import Callable, Iterator, ContextManager, Optional, List, Dict, Any, Union
from typing_extensions import Protocol
@@ -34,8 +34,9 @@ class CreateMachineProtocol(Protocol):
start_command: str | dict,
*,
name: Optional[str] = None,
keep_vm_state: bool = False,
) -> Machine:
keep_machine_state: bool = False,
**kwargs: Any, # to allow usage of deprecated keep_vm_state
) -> BaseMachine:
raise Exception("This is just type information for the Nix test driver")
@@ -43,7 +44,7 @@ start_all: Callable[[], None]
subtest: Callable[[str], ContextManager[None]]
retry: RetryProtocol
test_script: Callable[[], None]
machines: List[Machine]
machines: List[BaseMachine]
vlans: List[VLan]
driver: Driver
log: AbstractLogger
nixos/lib/testing-python.nix
+1
View File
@@ -56,6 +56,7 @@ pkgs.lib.throwIf (args ? specialArgs)
{
machine ? null,
nodes ? { },
containers ? { },
testScript,
enableOCR ? false,
globalTimeout ? (60 * 60),
nixos/lib/testing/driver.nix
+35 -13
View File
@@ -14,18 +14,17 @@ let
qemu_pkg = config.qemu.package;
imagemagick_light = hostPkgs.imagemagick_light.override { inherit (hostPkgs) libtiff; };
tesseract4 = hostPkgs.tesseract4.override { enableLanguages = [ "eng" ]; };
enableNspawn = config.containers != { };
# We want `pkgs.systemd`, *not* `python3Packages.system`.
systemd = hostPkgs.systemd;
};
vlans = map (
m: (m.virtualisation.vlans ++ (lib.mapAttrsToList (_: v: v.vlan) m.virtualisation.interfaces))
) (lib.attrValues config.nodes);
) ((lib.attrValues config.nodes) ++ (lib.attrValues config.containers));
vms = map (m: m.system.build.vm) (lib.attrValues config.nodes);
nodeHostNames =
let
nodesList = map (c: c.system.name) (lib.attrValues config.nodes);
in
nodesList ++ lib.optional (lib.length nodesList == 1 && !lib.elem "machine" nodesList) "machine";
containers = map (m: m.system.build.nspawn) (lib.attrValues config.containers);
pythonizeName =
name:
@@ -38,8 +37,22 @@ let
uniqueVlans = lib.unique (builtins.concatLists vlans);
vlanNames = map (i: "vlan${toString i}: VLan;") uniqueVlans;
pythonizedNames = map pythonizeName nodeHostNames;
machineNames = map (name: "${name}: Machine;") pythonizedNames;
vmMachineNames = map (c: c.system.name) (lib.attrValues config.nodes);
containerMachineNames = map (c: c.system.name) (lib.attrValues config.containers);
theOnlyMachine =
let
exactlyOneMachine = lib.length (lib.attrValues config.nodes) == 1;
allMachineNames = map (c: c.system.name) (lib.attrValues config.allMachines);
in
lib.optional (exactlyOneMachine && !lib.elem "machine" allMachineNames) "machine";
pythonizedVmNames = map pythonizeName (vmMachineNames ++ theOnlyMachine);
vmMachineTypeHints = map (name: "${name}: QemuMachine;") pythonizedVmNames;
pythonizedContainerNames = map pythonizeName containerMachineNames;
containerMachineTypeHints = map (name: "${name}: NspawnMachine;") pythonizedContainerNames;
withChecks = lib.warnIf config.skipLint "Linting is disabled";
@@ -62,12 +75,16 @@ let
''
mkdir -p $out/bin
vmStartScripts=($(for i in ${toString vms}; do echo $i/bin/run-*-vm; done))
vmNames=(${lib.escapeShellArgs vmMachineNames})
vmStartScripts=(${lib.escapeShellArgs (map lib.getExe vms)})
containerNames=(${lib.escapeShellArgs containerMachineNames})
containerStartScripts=(${lib.escapeShellArgs (map lib.getExe containers)})
${lib.optionalString (!config.skipTypeCheck) ''
# prepend type hints so the test script can be type checked with mypy
cat "${../test-script-prepend.py}" >> testScriptWithTypes
echo "${toString machineNames}" >> testScriptWithTypes
echo "${toString vmMachineTypeHints}" >> testScriptWithTypes
echo "${toString containerMachineTypeHints}" >> testScriptWithTypes
echo "${toString vlanNames}" >> testScriptWithTypes
echo -n "$testScript" >> testScriptWithTypes
@@ -90,7 +107,9 @@ let
echo "See https://nixos.org/manual/nixos/stable/#test-opt-skipLint"
PYFLAKES_BUILTINS="$(
echo -n ${lib.escapeShellArg (lib.concatStringsSep "," pythonizedNames)},
echo -n ${
lib.escapeShellArg (lib.concatStringsSep "," (pythonizedVmNames ++ pythonizedContainerNames))
},
cat ${lib.escapeShellArg "driver-symbols"}
)" ${hostPkgs.python3Packages.pyflakes}/bin/pyflakes $out/test-script
''}
@@ -98,7 +117,10 @@ let
# set defaults through environment
# see: ./test-driver/test-driver.py argparse implementation
wrapProgram $out/bin/nixos-test-driver \
--set startScripts "''${vmStartScripts[*]}" \
--set vmStartScripts "''${vmStartScripts[*]}" \
--set vmNames "''${vmNames[*]}" \
--set containerStartScripts "''${containerStartScripts[*]}" \
--set containerNames "''${containerNames[*]}" \
--set testScript "$out/test-script" \
--set globalTimeout "${toString config.globalTimeout}" \
--set vlans '${toString vlans}' \
nixos/lib/testing/network.nix
+82 -50
View File
@@ -1,11 +1,13 @@
{ lib, nodes, ... }:
testModuleArgs@{
lib,
...
}:
let
inherit (lib)
attrNames
concatMap
concatMapAttrsStringSep
concatMapStrings
flip
forEach
head
listToAttrs
@@ -20,22 +22,15 @@ let
zipLists
;
nodeNumbers = listToAttrs (zipListsWith nameValuePair (attrNames nodes) (range 1 254));
nodeNumbers = listToAttrs (
zipListsWith nameValuePair (attrNames testModuleArgs.config.allMachines) (range 1 254)
);
networkModule =
{
config,
nodes,
pkgs,
...
}:
{ config, ... }:
let
qemu-common = import ../qemu-common.nix { inherit (pkgs) lib stdenv; };
interfaces = lib.attrValues config.virtualisation.allInterfaces;
interfacesNumbered = zipLists interfaces (range 1 255);
# Automatically assign IP addresses to requested interfaces.
assignIPs = lib.filter (i: i.assignIP) interfaces;
ipInterfaces = forEach assignIPs (
@@ -56,17 +51,6 @@ let
}
);
qemuOptions = lib.flatten (
forEach interfacesNumbered (
{ fst, snd }: qemu-common.qemuNICFlags snd fst.vlan config.virtualisation.test.nodeNumber
)
);
udevRules = forEach interfaces (
interface:
# MAC Addresses for QEMU network devices are lowercase, and udev string comparison is case-sensitive.
''SUBSYSTEM=="net",ACTION=="add",ATTR{address}=="${toLower (qemu-common.qemuNicMac interface.vlan config.virtualisation.test.nodeNumber)}",NAME="${interface.name}"''
);
networkConfig = {
networking.hostName = mkDefault config.virtualisation.test.nodeName;
@@ -80,33 +64,51 @@ let
optionalString (ipInterfaces != [ ])
(head (head ipInterfaces).value.ipv6.addresses).address;
# Put the IP addresses of all VMs in this machine's
# /etc/hosts file. If a machine has multiple
# interfaces, use the IP address corresponding to
# the first interface (i.e. the first network in its
# virtualisation.vlans option).
networking.extraHosts = flip concatMapStrings (attrNames nodes) (
m':
# Generate /etc/hosts including every remote's primary IP addresses
# (whichever VLAN they may belong to) as well as all IP addresses from
# VLANs that both the local machine and the remote machine share.
networking.extraHosts =
let
config = nodes.${m'};
hostnames =
optionalString (
config.networking.domain != null
) "${config.networking.hostName}.${config.networking.domain} "
+ "${config.networking.hostName}\n";
localVlans = config.virtualisation.vlans;
in
optionalString (
config.networking.primaryIPAddress != ""
) "${config.networking.primaryIPAddress} ${hostnames}"
+ optionalString (
config.networking.primaryIPv6Address != ""
) "${config.networking.primaryIPv6Address} ${hostnames}"
);
concatMapAttrsStringSep "" (
mName: remoteConfig:
let
remoteInterfaces = remoteConfig.networking.interfaces;
sharedIps = lib.flatten (
lib.mapAttrsToList (
ifaceName: ifaceCfg:
let
remoteIfaceMeta = remoteConfig.virtualisation.allInterfaces."${ifaceName}" or { };
vlanId = remoteIfaceMeta.vlan or null;
in
if vlanId != null && builtins.elem vlanId localVlans then
builtins.map (addr: addr.address) ifaceCfg.ipv4.addresses
++ builtins.map (addr: addr.address) ifaceCfg.ipv6.addresses
else
[ ]
) remoteInterfaces
);
virtualisation.qemu.options = qemuOptions;
boot.initrd.services.udev.rules = concatMapStrings (x: x + "\n") udevRules;
# We also want to test router protocols that enable connections
# between nodes even if they don't share a VLAN, so we include
# the primary IPs of all machines in the hosts file.
primaryIPs = [
remoteConfig.networking.primaryIPAddress
remoteConfig.networking.primaryIPv6Address
];
allReachableIps = lib.lists.uniqueStrings (sharedIps ++ primaryIPs);
hostnames =
optionalString (
remoteConfig.networking.domain != null
) "${remoteConfig.networking.hostName}.${remoteConfig.networking.domain} "
+ "${remoteConfig.networking.hostName}\n";
in
builtins.concatStringsSep "" (map (ip: "${ip} ${hostnames}") allReachableIps)
) testModuleArgs.config.allMachines;
};
in
{
key = "network-interfaces";
@@ -117,6 +119,31 @@ let
};
};
qemuNetworkModule =
{ config, pkgs, ... }:
let
qemu-common = import ../qemu-common.nix { inherit (pkgs) lib stdenv; };
interfaces = lib.attrValues config.virtualisation.allInterfaces;
interfacesNumbered = zipLists interfaces (range 1 255);
qemuOptions = lib.flatten (
forEach interfacesNumbered (
{ fst, snd }: qemu-common.qemuNICFlags snd fst.vlan config.virtualisation.test.nodeNumber
)
);
udevRules = map (
interface:
# MAC Addresses for QEMU network devices are lowercase, and udev string comparison is case-sensitive.
''SUBSYSTEM=="net",ACTION=="add",ATTR{address}=="${toLower (qemu-common.qemuNicMac interface.vlan config.virtualisation.test.nodeNumber)}",NAME="${interface.name}"''
) interfaces;
in
{
virtualisation.qemu.options = qemuOptions;
boot.initrd.services.udev.rules = concatMapStrings (x: x + "\n") udevRules;
};
nodeNumberModule = (
regular@{ config, name, ... }:
{
@@ -127,7 +154,7 @@ let
# We need to force this in specialisations, otherwise it'd be
# readOnly = true;
description = ''
The `name` in `nodes.<name>`; stable across `specialisations`.
The `name` in `nodes.<name>` and `containers.<name>`; stable across `specialisations`.
'';
};
virtualisation.test.nodeNumber = mkOption {
@@ -136,7 +163,7 @@ let
readOnly = true;
default = nodeNumbers.${config.virtualisation.test.nodeName};
description = ''
A unique number assigned for each node in `nodes`.
A unique number assigned for each machine in `nodes` and `containers`.
'';
};
@@ -172,5 +199,10 @@ in
nodeNumberModule
];
};
extraBaseNodeModules = {
imports = [
qemuNetworkModule
];
};
};
}
nixos/lib/testing/nixos-test-base.nix
+3 -2
View File
@@ -7,7 +7,6 @@ let
in
{
imports = [
../../modules/virtualisation/qemu-vm.nix
../../modules/testing/test-instrumentation.nix # !!! should only get added for automated test runs
{
key = "no-manual";
@@ -32,7 +31,9 @@ in
# This is mostly a Hydra optimization, so we don't rebuild all the tests every time switch-to-configuration-ng changes.
key = "no-switch-to-configuration";
system.switch.enable = mkDefault (
config.isSpecialisation || config.specialisation != { } || config.virtualisation.installBootLoader
config.isSpecialisation
|| config.specialisation != { }
|| (!config.boot.isContainer && config.virtualisation.installBootLoader)
);
}
)
nixos/lib/testing/nodes.nix
+157 -40
View File
@@ -2,7 +2,6 @@ testModuleArgs@{
config,
lib,
hostPkgs,
nodes,
options,
...
}:
@@ -12,12 +11,9 @@ let
literalExpression
literalMD
mapAttrs
mkDefault
mkIf
mkMerge
mkOption
mkForce
optional
optionalAttrs
types
;
@@ -49,15 +45,11 @@ let
./nixos-test-base.nix
{
key = "nodes";
_module.args.nodes = config.nodesCompat;
_module.args = {
inherit (config) containers;
nodes = config.nodesCompat;
};
}
(
{ config, ... }:
{
virtualisation.qemu.package = testModuleArgs.config.qemu.package;
virtualisation.host.pkgs = hostPkgs;
}
)
(
{ options, ... }:
{
@@ -73,6 +65,62 @@ let
testModuleArgs.config.extraBaseModules
];
};
baseQemuOS = baseOS.extendModules {
modules = [
../../modules/virtualisation/qemu-vm.nix
config.nodeDefaults
{
key = "base-qemu";
virtualisation.qemu.package = testModuleArgs.config.qemu.package;
virtualisation.host.pkgs = hostPkgs;
}
testModuleArgs.config.extraBaseNodeModules
];
};
baseNspawnOS = baseOS.extendModules {
modules = [
../../modules/virtualisation/nspawn-container
config.containerDefaults
(
{ pkgs, ... }:
{
key = "base-nspawn";
# PAM requires setuid and doesn't work in the build sandbox.
# https://github.com/NixOS/nix/blob/959c244a1265f4048390f3ad21679219d7b27a99/src/libstore/unix/build/linux-derivation-builder.cc#L63
services.openssh.settings.UsePAM = false;
# Networking for tests is statically configured by default.
# dhcpcd times out after blocking for a long time, which slows down tests.
# See https://github.com/NixOS/nixpkgs/pull/478109#discussion_r2867570799
networking.useDHCP = lib.mkDefault false;
# Disable Info manual directory generation to prevent build failures.
#
# Context: 'install-info' (from texinfo) is triggered during system-path
# generation to index manuals, but it requires 'gzip' in the $PATH to
# decompress them.
# When 'networking.useDHCP' is set to false, transitive dependencies
# (like dhcpcd or other network tools) that normally pull 'gzip' into
# the system environment are removed. This leaves 'install-info'
# stranded without 'gzip', causing the 'system-path' derivation to fail.
# Since nspawn containers are typically minimal, disabling 'info'
# is a cleaner fix than explicitly adding 'gzip' to systemPackages.
documentation.info.enable = lib.mkDefault false;
# Gross, insecure hack to make login work. See above.
security.pam.services.login = {
text = ''
auth sufficient ${pkgs.linux-pam}/lib/security/pam_permit.so
account sufficient ${pkgs.linux-pam}/lib/security/pam_permit.so
password sufficient ${pkgs.linux-pam}/lib/security/pam_permit.so
session sufficient ${pkgs.linux-pam}/lib/security/pam_permit.so
'';
};
}
)
];
};
# TODO (lib): Dedup with run.nix, add to lib/options.nix
mkOneUp = opt: f: lib.mkOverride (opt.highestPrio - 1) (f opt.value);
@@ -109,25 +157,74 @@ in
node.type = mkOption {
type = types.raw;
default = baseOS.type;
default = baseQemuOS.type;
internal = true;
};
nodes = mkOption {
type = types.lazyAttrsOf config.node.type;
default = { };
visible = "shallow";
description = ''
An attribute set of NixOS configuration modules.
An attribute set of NixOS configuration modules representing QEMU vms that can be started during a test.
The configurations are augmented by the [`defaults`](#test-opt-defaults) option.
They are assigned network addresses according to the `nixos/lib/testing/network.nix` module.
A few special options are available, that aren't in a plain NixOS configuration. See [Configuring the nodes](#sec-nixos-test-nodes)
A few special options are available, that aren't in a plain NixOS configuration. See [Configuring virtual machines](#ssec-nixos-test-qemu-vms)
'';
};
container.type = mkOption {
type = types.raw;
default = baseNspawnOS.type;
internal = true;
};
containers = mkOption {
type = types.lazyAttrsOf config.container.type;
default = { };
visible = "shallow";
description = ''
An attribute set of NixOS configuration modules representing systemd-nspawn containers that can be started during a test.
The configurations are augmented by the [`defaults`](#test-opt-defaults) option.
They are assigned network addresses according to the `nixos/lib/testing/network.nix` module.
A few special options are available, that aren't in a plain NixOS configuration. See [Configuring containers](#ssec-nixos-test-nspawn-containers)
'';
};
allMachines = mkOption {
readOnly = true;
internal = true;
description = ''
Basically a merge of [{option}`nodes`](#test-opt-nodes) and [{option}`containers`](#test-opt-containers).
This ensures that there are no name collisions between nodes and containers.
'';
default =
let
overlappingNames = lib.intersectLists (lib.attrNames config.nodes) (
lib.attrNames config.containers
);
in
lib.throwIfNot (overlappingNames == [ ])
"The following names are used in both `nodes` and `containers`: ${lib.concatStringsSep ", " overlappingNames}"
(config.nodes // config.containers);
};
defaults = mkOption {
description = ''
NixOS configuration that is applied to all [{option}`nodes`](#test-opt-nodes) and [{option}`containers`](#test-opt-containers).
'';
type = types.deferredModule;
default = { };
};
nodeDefaults = mkOption {
description = ''
NixOS configuration that is applied to all [{option}`nodes`](#test-opt-nodes).
'';
@@ -135,7 +232,23 @@ in
default = { };
};
containerDefaults = mkOption {
description = ''
NixOS configuration that is applied to all [{option}`containers`](#test-opt-containers).
'';
type = types.deferredModule;
default = { };
};
extraBaseModules = mkOption {
description = ''
NixOS configuration that, like [{option}`defaults`](#test-opt-defaults), is applied to all [{option}`nodes`](#test-opt-nodes) and [{option}`containers`](#test-opt-containers) and can not be undone with [`specialisation.<name>.inheritParentConfig`](https://search.nixos.org/options?show=specialisation.%3Cname%3E.inheritParentConfig&from=0&size=50&sort=relevance&type=packages&query=specialisation).
'';
type = types.deferredModule;
default = { };
};
extraBaseNodeModules = mkOption {
description = ''
NixOS configuration that, like [{option}`defaults`](#test-opt-defaults), is applied to all [{option}`nodes`](#test-opt-nodes) and can not be undone with [`specialisation.<name>.inheritParentConfig`](https://search.nixos.org/options?show=specialisation.%3Cname%3E.inheritParentConfig&from=0&size=50&sort=relevance&type=packages&query=specialisation).
'';
@@ -145,7 +258,7 @@ in
node.pkgs = mkOption {
description = ''
The Nixpkgs to use for the nodes.
The Nixpkgs to use for the nodes and containers.
Setting this will make the `nixpkgs.*` options read-only, to avoid mistakenly testing with a Nixpkgs configuration that diverges from regular use.
'';
@@ -160,7 +273,7 @@ in
description = ''
Whether to make the `nixpkgs.*` options read-only. This is only relevant when [`node.pkgs`](#test-opt-node.pkgs) is set.
Set this to `false` when any of the [`nodes`](#test-opt-nodes) needs to configure any of the `nixpkgs.*` options. This will slow down evaluation of your test a bit.
Set this to `false` when any of the [`nodes`](#test-opt-nodes) or [{option}`containers`](#test-opt-containers) need to configure any of the `nixpkgs.*` options. This will slow down evaluation of your test a bit.
'';
type = types.bool;
default = config.node.pkgs != null;
@@ -188,6 +301,7 @@ in
};
config = {
_module.args.containers = config.containers;
_module.args.nodes = config.nodesCompat;
nodesCompat = mapAttrs (
name: config:
@@ -201,6 +315,7 @@ in
) config.nodes;
passthru.nodes = config.nodesCompat;
passthru.containers = config.containers;
extraDriverArgs = mkIf config.sshBackdoor.enable [
"--dump-vsocks=${toString config.sshBackdoor.vsockOffset}"
@@ -211,33 +326,35 @@ in
nixpkgs.pkgs = config.node.pkgs;
imports = [ ../../modules/misc/nixpkgs/read-only.nix ];
})
(mkIf config.sshBackdoor.enable (
let
inherit (config.sshBackdoor) vsockOffset;
in
{ config, ... }:
{
services.openssh = {
enable = true;
settings = {
PermitRootLogin = "yes";
PermitEmptyPasswords = "yes";
};
(mkIf config.sshBackdoor.enable {
services.openssh = {
enable = true;
settings = {
PermitRootLogin = "yes";
PermitEmptyPasswords = "yes";
};
};
security.pam.services.sshd = {
allowNullPassword = true;
};
virtualisation.qemu.options = [
"-device vhost-vsock-pci,guest-cid=${
toString (config.virtualisation.test.nodeNumber + vsockOffset)
}"
];
}
))
security.pam.services.sshd = {
allowNullPassword = true;
};
})
];
nodeDefaults = mkIf config.sshBackdoor.enable (
let
inherit (config.sshBackdoor) vsockOffset;
in
{ config, ... }:
{
virtualisation.qemu.options = [
"-device vhost-vsock-pci,guest-cid=${
toString (config.virtualisation.test.nodeNumber + vsockOffset)
}"
];
}
);
# Docs: nixos/doc/manual/development/writing-nixos-tests.section.md
/**
See https://nixos.org/manual/nixos/unstable#sec-override-nixos-test
nixos/lib/testing/run.nix
+4
View File
@@ -2,6 +2,7 @@
config,
hostPkgs,
lib,
containers,
options,
...
}:
@@ -96,12 +97,15 @@ in
requiredSystemFeatures = [
"nixos-test"
]
# Containers use systemd-nspawn, which requires pid 0 inside of the sandbox.
++ lib.optional (builtins.length (lib.attrNames containers) > 0) "uid-range"
++ lib.optional isLinux "kvm"
++ lib.optional isDarwin "apple-virt";
nativeBuildInputs = lib.optionals config.enableDebugHook [
hostPkgs.openssh
hostPkgs.inetutils
hostPkgs.socat # to allow SSH backdoor connections for systemd-nspawn containers
];
buildCommand = ''
nixos/lib/testing/testScript.nix
+2 -1
View File
@@ -56,11 +56,12 @@ in
# reuse memoized config
v
) config.nodesCompat;
containers = config.containers;
}
else
config.testScript;
defaults =
nodeDefaults =
{ config, name, ... }:
{
# Make sure all derivations referenced by the test
nixos/modules/config/system-path.nix
+1
View File
@@ -214,6 +214,7 @@ in
postBuild = ''
# Remove wrapped binaries, they shouldn't be accessible via PATH.
find $out/bin -maxdepth 1 -name ".*-wrapped" -type l -delete
find $out/bin -maxdepth 1 -name ".*-wrapped_*" -type l -delete
if [ -x $out/bin/glib-compile-schemas -a -w $out/share/glib-2.0/schemas ]; then
$out/bin/glib-compile-schemas $out/share/glib-2.0/schemas
nixos/modules/config/unix-odbc-drivers.nix
+3 -3
View File
@@ -1,5 +1,5 @@
{ config, lib, ... }:
# unixODBC drivers (this solution is not perfect.. Because the user has to
# unixodbc drivers (this solution is not perfect.. Because the user has to
# ask the admin to add a driver.. but it's simple and works
let
@@ -17,11 +17,11 @@ in
environment.unixODBCDrivers = lib.mkOption {
type = lib.types.listOf lib.types.package;
default = [ ];
example = lib.literalExpression "with pkgs.unixODBCDrivers; [ sqlite psql ]";
example = lib.literalExpression "with pkgs.unixodbcDrivers; [ sqlite psql ]";
description = ''
Specifies Unix ODBC drivers to be registered in
{file}`/etc/odbcinst.ini`. You may also want to
add `pkgs.unixODBC` to the system path to get
add `pkgs.unixodbc` to the system path to get
a command line client to connect to ODBC databases.
'';
};
nixos/modules/services/continuous-integration/jenkins/job-builder.nix
+1 -1
View File
@@ -175,7 +175,7 @@ in
(umask 0077; printf "${cfg.accessUser}:@password_placeholder@" >"$auth_file")
"${pkgs.replace-secret}/bin/replace-secret" "@password_placeholder@" "$access_token_file" "$auth_file"
if ! "${pkgs.jenkins}/bin/jenkins-cli" -s "$jenkins_url" -auth "@$auth_file" reload-configuration; then
if ! "${pkgs.jenkins}/bin/jenkins-cli" -http -s "$jenkins_url" -auth "@$auth_file" reload-configuration; then
echo "error: failed to reload configuration"
exit 1
fi
nixos/modules/services/home-automation/home-assistant.nix
+10
View File
@@ -566,6 +566,16 @@ in
'';
};
finalPackage = mkOption {
default = package;
internal = true;
readOnly = true;
type = types.package;
description = ''
The final Home Assistant package which is being used in the service.
'';
};
openFirewall = mkOption {
default = false;
type = types.bool;
nixos/modules/services/misc/radicle.nix
+49 -20
View File
@@ -2,6 +2,7 @@
config,
lib,
pkgs,
utils,
...
}:
let
@@ -15,6 +16,11 @@ let
RAD_HOME = HOME;
};
credentials = {
privateKey = "xyz.radicle.node.secret";
privateKeyPassphrase = "xyz.radicle.node.passphrase";
};
# Convenient wrapper to run `rad` in the namespaces of `radicle-node.service`
rad-system = pkgs.writeShellScriptBin "rad-system" ''
set -o allexport
@@ -131,16 +137,34 @@ in
services.radicle = {
enable = lib.mkEnableOption "Radicle Seed Node";
package = lib.mkPackageOption pkgs "radicle-node" { };
privateKeyFile = lib.mkOption {
# Note that a key encrypted by systemd-creds is not a path but a str.
type = with lib.types; either path str;
privateKey = lib.mkOption {
type = with lib.types; nullOr (either path str);
default = null;
description = ''
Absolute file path to an SSH private key,
An SSH private key (as an absolute file path or Systemd credential name),
usually generated by `rad auth`.
If it contains a colon (`:`) the string before the colon
is taken as the credential name
and the string after as a path encrypted with `systemd-creds`.
If set to the default value of `null`, radicle will import the private key from a credential
named `${credentials.privateKey}`.
If configured as a credential name it will be imported via `ImportCredential=` in the service configuration.
Refer to the systemd-creds documentation for more details <https://systemd.io/CREDENTIALS/>
'';
};
privateKeyPassphrase = lib.mkOption {
type = with lib.types; nullOr str;
default = null;
description = ''
A passphrase for an SSH private key (as a Systemd credential name),
usually provided on generation of the key with `rad auth`.
If set to the default value of `null`, radicle will optionally import the passphrase from a
credential named `${credentials.privateKeyPassphrase}`.
If the passphrase is not set, radicle will prompt for it.
If configured as a credential name it will be imported via `ImportCredential=` in the service configuration.
Refer to the systemd-creds documentation for more details <https://systemd.io/CREDENTIALS/>
'';
};
publicKey = lib.mkOption {
@@ -304,23 +328,28 @@ in
# Give only access to the private key to radicle-node.
{
serviceConfig =
let
keyCred = builtins.split ":" "${cfg.privateKeyFile}";
in
if lib.length keyCred > 1 then
if cfg.privateKey == null then
{
LoadCredentialEncrypted = [ cfg.privateKeyFile ];
# Note that neither %d nor ${CREDENTIALS_DIRECTORY} works in BindReadOnlyPaths=
BindReadOnlyPaths = [
"/run/credentials/radicle-node.service/${lib.head keyCred}:${env.RAD_HOME}/keys/radicle"
];
ImportCredential = [ credentials.privateKey ];
}
else if lib.types.path.check cfg.privateKey then
{
LoadCredential = [ "${credentials.privateKey}:${cfg.privateKey}" ];
}
else
{
LoadCredential = [ "radicle:${cfg.privateKeyFile}" ];
BindReadOnlyPaths = [
"/run/credentials/radicle-node.service/radicle:${env.RAD_HOME}/keys/radicle"
];
ImportCredential = [ "${cfg.privateKey}:${credentials.privateKey}" ];
};
}
{
serviceConfig =
if cfg.privateKeyPassphrase == null then
{
ImportCredential = [ credentials.privateKeyPassphrase ];
}
else
{
ImportCredential = [ "${cfg.privateKeyPassphrase}:${credentials.privateKeyPassphrase}" ];
};
}
];
nixos/modules/services/networking/iscsi/root-initiator.nix
+1 -1
View File
@@ -86,7 +86,7 @@ in
description = ''
Append an additional file's contents to `/etc/iscsid.conf`. Use a non-store path
and store passwords in this file. Note: the file specified here must be available
in the initrd, see: `boot.initrd.secrets`.
in the initrd, see: `boot.initrd.secretPaths`.
'';
default = null;
type = nullOr str;
nixos/modules/services/networking/murmur.nix
+59 -24
View File
@@ -7,6 +7,8 @@
let
cfg = config.services.murmur;
acmeHostDir = config.security.acme.certs."${cfg.tls.useACMEHost}".directory;
forking = cfg.logToFile;
configFile = pkgs.writeText "murmurd.ini" ''
database=${cfg.stateDir}/murmur.sqlite
@@ -41,9 +43,9 @@ let
${lib.optionalString (cfg.registerHostname != "") "registerHostname=${cfg.registerHostname}"}
certrequired=${lib.boolToString cfg.clientCertRequired}
${lib.optionalString (cfg.sslCert != null) "sslCert=${cfg.sslCert}"}
${lib.optionalString (cfg.sslKey != null) "sslKey=${cfg.sslKey}"}
${lib.optionalString (cfg.sslCa != null) "sslCA=${cfg.sslCa}"}
${lib.optionalString (cfg.tls.certPath != null) "sslCert=${cfg.tls.certPath}"}
${lib.optionalString (cfg.tls.keyPath != null) "sslKey=${cfg.tls.keyPath}"}
${lib.optionalString (cfg.tls.caPath != null) "sslCA=${cfg.tls.caPath}"}
${lib.optionalString (cfg.dbus != null) "dbus=${cfg.dbus}"}
@@ -58,6 +60,12 @@ in
"murmur"
"logFile"
] "This option has been superseded by services.murmur.logToFile")
(lib.mkRenamedOptionModule [ "services" "murmur" "sslCa" ] [ "services" "murmur" "tls" "caPath" ])
(lib.mkRenamedOptionModule [ "services" "murmur" "sslKey" ] [ "services" "murmur" "tls" "keyPath" ])
(lib.mkRenamedOptionModule
[ "services" "murmur" "sslCert" ]
[ "services" "murmur" "tls" "certPath" ]
)
];
options = {
@@ -237,22 +245,41 @@ in
clientCertRequired = lib.mkEnableOption "requiring clients to authenticate via certificates";
sslCert = lib.mkOption {
type = lib.types.nullOr lib.types.path;
default = null;
description = "Path to your SSL certificate.";
};
tls = {
certPath = lib.mkOption {
type = lib.types.nullOr lib.types.path;
default = if (cfg.tls.useACMEHost != null) then "${acmeHostDir}/cert.pem" else null;
defaultText = lib.literalMD "If {option}`services.murmur.tls.useACMEHost` is set, defaults to what's provided by the ACME module.";
description = "Path to your TLS certificate.";
};
sslKey = lib.mkOption {
type = lib.types.nullOr lib.types.path;
default = null;
description = "Path to your SSL key.";
};
keyPath = lib.mkOption {
type = lib.types.nullOr lib.types.path;
default = if (cfg.tls.useACMEHost != null) then "${acmeHostDir}/key.pem" else null;
defaultText = lib.literalMD "If {option}`services.murmur.tls.useACMEHost` is set, defaults to what's provided by the ACME module.";
description = "Path to your TLS key.";
};
sslCa = lib.mkOption {
type = lib.types.nullOr lib.types.path;
default = null;
description = "Path to your SSL CA certificate.";
caPath = lib.mkOption {
type = lib.types.nullOr lib.types.path;
default = if (cfg.tls.useACMEHost != null) then "${acmeHostDir}/chain.pem" else null;
defaultText = lib.literalMD "If {option}`services.murmur.tls.useACMEHost` is set, defaults to what's provided by the ACME module.";
description = "Path to your TLS CA certificate.";
};
useACMEHost = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "mumble.example.com";
description = ''
Host of an existing Let's Encrypt certificate to use for TLS.
Make sure that the certificate directory is readable by the
`murmur` user or group. *Note that this option does not
create any certificates and it doesn't add subdomains to
existing ones – you will need to create them manually using
{option}`security.acme.certs`.*
'';
};
};
extraConfig = lib.mkOption {
@@ -316,10 +343,18 @@ in
allowedUDPPorts = [ cfg.port ];
};
security.acme.certs = lib.mkIf (cfg.tls.useACMEHost != null) {
"${cfg.tls.useACMEHost}".reloadServices = [ "murmur.service" ];
};
systemd.services.murmur = {
description = "Murmur Chat Service";
wantedBy = [ "multi-user.target" ];
after = [ "network.target" ];
after = [
"network.target"
]
++ lib.optional (cfg.tls.useACMEHost != null) "acme-${cfg.tls.useACMEHost}.service";
wants = lib.mkIf (cfg.tls.useACMEHost != null) [ "acme-${cfg.tls.useACMEHost}.service" ];
preStart = ''
${pkgs.envsubst}/bin/envsubst \
-o /run/murmur/murmurd.ini \
@@ -422,14 +457,14 @@ in
${lib.optionalString cfg.logToFile ''
/var/log/murmur/murmurd.log rw,
''}
${lib.optionalString (cfg.sslCert != null) ''
${cfg.sslCert} r,
${lib.optionalString (cfg.tls.certPath != null) ''
${cfg.tls.certPath} r,
''}
${lib.optionalString (cfg.sslKey != null) ''
${cfg.sslKey} r,
${lib.optionalString (cfg.tls.keyPath != null) ''
${cfg.tls.keyPath} r,
''}
${lib.optionalString (cfg.sslCa != null) ''
${cfg.sslCa} r,
${lib.optionalString (cfg.tls.caPath != null) ''
${cfg.tls.caPath} r,
''}
${lib.optionalString (cfg.dbus != null) ''
dbus bus=${cfg.dbus},
nixos/modules/services/web-apps/dawarich.nix
+1
View File
@@ -701,6 +701,7 @@ in
meta.maintainers = with lib.maintainers; [
diogotcorreia
tmarkus
];
}
nixos/modules/services/web-apps/olivetin.nix
+11 -1
View File
@@ -17,7 +17,17 @@ in
options.services.olivetin = {
enable = lib.mkEnableOption "OliveTin";
package = lib.mkPackageOption pkgs "olivetin" { };
package = lib.mkOption {
type = lib.types.package;
description = "The olivetin package to use.";
default =
if lib.versionAtLeast config.system.stateVersion "26.05" then pkgs.olivetin-3k else pkgs.olivetin;
defaultText = lib.literalExpression ''
if lib.versionAtLeast config.system.stateVersion "26.05"
then pkgs.olivetin-3k
else pkgs.olivetin
'';
};
user = lib.mkOption {
type = lib.types.str;
nixos/modules/system/activation/bootspec.nix
+4 -2
View File
@@ -14,7 +14,9 @@ let
children = lib.mapAttrs (
childName: childConfig: childConfig.configuration.system.build.toplevel
) config.specialisation;
hasAtLeastOneInitrdSecret = lib.length (lib.attrNames config.boot.initrd.secrets) > 0;
hasInitrdSecrets =
(lib.length (lib.attrNames config.boot.initrd.secretPaths) > 0)
|| (config.boot.initrd.extraSecretsHook != "");
schemas = {
v1 = rec {
filename = "boot.json";
@@ -33,7 +35,7 @@ let
// lib.optionalAttrs config.boot.initrd.enable {
initrd = "${config.system.build.initialRamdisk}/${config.system.boot.loader.initrdFile}";
}
// lib.optionalAttrs hasAtLeastOneInitrdSecret {
// lib.optionalAttrs hasInitrdSecrets {
initrdSecrets = "${config.system.build.initialRamdiskSecretAppender}/bin/append-initrd-secrets";
};
}
nixos/modules/system/boot/clevis.nix
+2 -2
View File
@@ -99,8 +99,8 @@ in
sed -i $out/bin/clevis-decrypt-tpm2 -e 's,tpm2_,tpm2 ,'
'';
secrets = lib.mapAttrs' (
name: value: lib.nameValuePair "/etc/clevis/${name}.jwe" value.secretFile
secretPaths = lib.mapAttrs' (
name: value: lib.nameValuePair "/etc/clevis/${name}.jwe" { source = value.secretFile; }
) cfg.devices;
systemd = {
nixos/modules/system/boot/initrd-openvpn.nix
+3 -3
View File
@@ -29,7 +29,7 @@ in
};
boot.initrd.network.openvpn.configuration = mkOption {
type = types.path; # Same type as boot.initrd.secrets
type = types.path; # Same type as boot.initrd.secretPaths.*.source
description = ''
The configuration file for OpenVPN.
@@ -74,8 +74,8 @@ in
"${pkgs.glibc}/lib/libnss_dns.so.2"
];
boot.initrd.secrets = {
"/etc/initrd.ovpn" = cfg.configuration;
boot.initrd.secretPaths = {
"/etc/initrd.ovpn".source = cfg.configuration;
};
# openvpn --version would exit with 1 instead of 0
nixos/modules/system/boot/initrd-ssh.nix
+2 -2
View File
@@ -302,8 +302,8 @@ in
fi
'';
boot.initrd.secrets = listToAttrs (
map (path: nameValuePair (initrdKeyPath path) path) cfg.hostKeys
boot.initrd.secretPaths = listToAttrs (
map (path: nameValuePair (initrdKeyPath path) { source = path; }) cfg.hostKeys
);
# Systemd initrd stuff
nixos/modules/system/boot/kernel.nix
+1 -1
View File
@@ -414,7 +414,7 @@ in
ln -s ${initrdPath} $out/initrd
${optionalString (config.boot.initrd.secrets != { }) ''
${optionalString (config.boot.initrd.secretPaths != { }) ''
ln -s ${config.system.build.initialRamdiskSecretAppender}/bin/append-initrd-secrets $out
''}
nixos/modules/system/boot/loader/grub/grub.nix
+6 -6
View File
@@ -935,21 +935,21 @@ in
'')
(mkRemovedOptionModule [ "boot" "loader" "grub" "extraInitrd" ] ''
This option has been replaced with the bootloader agnostic
boot.initrd.secrets option. To migrate to the initrd secrets system,
boot.initrd.secretPaths option. To migrate to the initrd secrets system,
extract the extraInitrd archive into your main filesystem:
# zcat /boot/extra_initramfs.gz | cpio -idvmD /etc/secrets/initrd
/path/to/secret1
/path/to/secret2
then replace boot.loader.grub.extraInitrd with boot.initrd.secrets:
then replace boot.loader.grub.extraInitrd with boot.initrd.secretPaths:
boot.initrd.secrets = {
"/path/to/secret1" = "/etc/secrets/initrd/path/to/secret1";
"/path/to/secret2" = "/etc/secrets/initrd/path/to/secret2";
boot.initrd.secretPaths = {
"/path/to/secret1".source = "/etc/secrets/initrd/path/to/secret1";
"/path/to/secret2".source = "/etc/secrets/initrd/path/to/secret2";
};
See the boot.initrd.secrets option documentation for more information.
See the boot.initrd.secretPaths option documentation for more information.
'')
];
nixos/modules/system/boot/loader/grub/install-grub.pl
+1 -1
View File
@@ -482,7 +482,7 @@ sub addEntry {
die "failed to create initrd secrets $!\n";
} else {
say STDERR "warning: failed to create initrd secrets for \"$name\", an older generation";
say STDERR "note: this is normal after having removed or renamed a file in `boot.initrd.secrets`";
say STDERR "note: this is normal after having modified or removed an entry in `boot.initrd.secretPaths`";
}
}
# Check whether any secrets were actually added
nixos/modules/system/boot/loader/systemd-boot/systemd-boot-builder.py
+1 -1
View File
@@ -202,7 +202,7 @@ def write_entry(profile: str | None, generation: int, specialisation: str | None
print("warning: failed to create initrd secrets "
f'for "{title} - Configuration {generation}", an older generation', file=sys.stderr)
print("note: this is normal after having removed "
"or renamed a file in `boot.initrd.secrets`", file=sys.stderr)
"or modified an entry in `boot.initrd.secretPaths`", file=sys.stderr)
entry_file = BOOT_MOUNT_POINT / "loader/entries" / generation_conf_filename(profile, generation, specialisation)
tmp_path = entry_file.with_suffix(".tmp")
kernel_params = "init=%s " % bootspec.init
nixos/modules/system/boot/stage-1.nix
+154 -38
View File
@@ -148,23 +148,22 @@ let
# Copy secrets if needed.
#
# TODO: move out to a separate script; see #85000.
${optionalString (!config.boot.loader.supportsInitrdSecrets) (
concatStringsSep "\n" (
mapAttrsToList (
dest: source:
let
source' = if source == null then dest else source;
in
''
mkdir -p $(dirname "$out/secrets/${dest}")
# Some programs (e.g. ssh) doesn't like secrets to be
# symlinks, so we use `cp -L` here to match the
# behaviour when secrets are natively supported.
cp -Lr ${source'} "$out/secrets/${dest}"
''
) config.boot.initrd.secrets
)
)}
${optionalString (!config.boot.loader.supportsInitrdSecrets) ''
${concatStringsSep "\n" (
mapAttrsToList (_: scfg: ''
mkdir -p $(dirname "$out/secrets${scfg.path}")
# Some programs (e.g. ssh) doesn't like secrets to be
# symlinks, so we use `cp -L` here to match the
# behaviour when secrets are natively supported.
# The assertion further up in this file (stage-1.nix)
# checks that all secretPaths are Nix store paths set via
# boot.initrd.secretPaths.*.source if the bootloader doesn't
# support initrd secrets.
cp -Lr ${scfg.source} "$out/secrets${scfg.path}"
'') config.boot.initrd.secretPaths
)}
${config.boot.initrd.extraSecretsHook}
''}
${config.boot.initrd.extraUtilsCommands}
@@ -436,7 +435,9 @@ let
exit 0
fi
${lib.optionalString (config.boot.initrd.secrets == { }) "exit 0"}
${lib.optionalString (
config.boot.initrd.secretPaths == { } && config.boot.initrd.extraSecretsHook == ""
) "exit 0"}
export PATH=${pkgs.coreutils}/bin:${pkgs.cpio}/bin:${pkgs.gzip}/bin:${pkgs.findutils}/bin
@@ -451,17 +452,25 @@ let
${lib.concatStringsSep "\n" (
mapAttrsToList (
dest: source:
_: scfg:
let
source' = if source == null then dest else toString source;
prefix = lib.optionalString scfg.intermediateSecretsDir "/.initrd-secrets";
in
''
mkdir -p $(dirname "$tmp/.initrd-secrets/${dest}")
cp -a ${source'} "$tmp/.initrd-secrets/${dest}"
mkdir -p $(dirname "$tmp${prefix}${scfg.path}")
(
export out="$tmp${prefix}${scfg.path}"
${scfg.generateSecretCommand}
)
''
) config.boot.initrd.secrets
) config.boot.initrd.secretPaths
)}
(
cd "$tmp"
${config.boot.initrd.extraSecretsHook}
)
# mindepth 1 so that we don't change the mode of /
(cd "$tmp" && find . -mindepth 1 | xargs touch -amt 197001010000 && find . -mindepth 1 -print0 | sort -z | cpio --quiet -o -H newc -R +0:+0 --reproducible --null) | \
${compressorExe} ${lib.escapeShellArgs initialRamdisk.compressorArgs} >> "$1"
@@ -650,21 +659,113 @@ in
boot.initrd.secrets = mkOption {
default = { };
type = types.attrsOf (types.nullOr types.path);
visible = false;
description = ''
Secrets to append to the initrd. The attribute name is the
path the secret should have inside the initrd, the value
is the path it should be copied from (or null for the same
path inside and out).
Secrets to append to the initrd. This option has been deprecated in
favour of `boot.initrd.secretPaths`.
'';
example = literalExpression ''
{ "/etc/dropbear/dropbear_rsa_host_key" =
./secret-dropbear-key;
}
'';
};
boot.initrd.secretPaths = mkOption {
default = { };
type = types.attrsOf (
types.submodule (
{ config, name, ... }:
{
options = {
path = mkOption {
type = types.path;
default = name;
description = ''
The path the secret should be placed at in the initrd. Defaults
to the attribute name.
'';
};
intermediateSecretsDir = mkOption {
type = types.bool;
default = true;
description = ''
By default, the secrets will be copied over to the
`/.initrd-secrets` dir at initrd generation time, and then copied
over to their final location at boot time. This is because initrd secrets
that are supposed to be placed in `/run` would be overridden by
the tmpfs mount over `/run` otherwise.
Set this option to `false` to skip this intermediate step and
place the secret at its final location straightaway.
'';
};
source = mkOption {
type = types.nullOr types.path;
default = null;
description = ''
The absolute path on the filesystem to copy the secret from.
'';
example = "/var/lib/secrets/initrd/ssh_host_ed25519_key";
};
generateSecretCommand = mkOption {
type = types.path;
description = ''
The command to run to generate the secret. It should write
the secret to `$out`.
This is useful if you have a more advanced secrets provisioning
mechanism.
'';
example = ''
pkgs.writeShellScript "generate-secret" '''
''${lib.getExe pkgs.age} -d -i /etc/ssh/ssh_host_ed25519_key -o "$out" ''${./secret.age}
'''
'';
};
};
config = {
generateSecretCommand = lib.mkIf (config.source != null) (
pkgs.writeShellScript "copy-secret" ''
cp -Lr ${config.source} "$out"
''
);
};
}
)
);
description = ''
Secret paths to append to the initrd. The attribute name is the
path the secret should have inside the initrd.
Note that `nixos-rebuild switch` will generate the initrd
also for past generations, so if secrets are moved or deleted
you will also have to garbage collect the generations that
use those secrets.
'';
example = literalExpression ''
{ "/etc/dropbear/dropbear_rsa_host_key" =
./secret-dropbear-key;
}
example = {
"/etc/ssh/ssh_host_ed25519_key".source = "/var/lib/secrets/initrd/ssh_host_ed25519_key";
};
};
boot.initrd.extraSecretsHook = mkOption {
default = "";
type = types.lines;
description = ''
Extra commands to be executed after the initrd secrets generation phase.
This script should place files into the current workdir. These files
will then be copied over to the initrd to the corresponding absolute
paths, e.g. `etc/ssh/ssh_host_ed25519_key` will be copied over to
`/etc/ssh/ssh_host_ed25519_key`.
'';
example = ''
# Generate a new SSH host key for every generation.
ssh-keygen -f etc/ssh/ssh_host_ed25519_key
'';
};
@@ -746,15 +847,18 @@ in
assertion =
!config.boot.loader.supportsInitrdSecrets
-> all (
source: builtins.isPath source || (builtins.isString source && hasPrefix builtins.storeDir source)
) (attrValues config.boot.initrd.secrets);
scfg:
builtins.isPath scfg.source
|| (builtins.isString scfg.source && hasPrefix builtins.storeDir scfg.source)
) (attrValues config.boot.initrd.secretPaths);
message = ''
boot.initrd.secrets values must be unquoted paths when
using a bootloader that doesn't natively support initrd
secrets, e.g.:
When using a bootloader that doesn't natively support initrd secrets,
all `boot.initrd.secretPaths` values must be defined via
`boot.initrd.secretsPaths.*.source`, and the `source` values must be
unquoted paths, e.g.
boot.initrd.secrets = {
"/etc/secret" = /path/to/secret;
boot.initrd.secretPaths = {
"/etc/secret".source = /path/to/secret;
};
Note that this will result in all secrets being stored
@@ -763,6 +867,18 @@ in
}
];
warnings = lib.optional (config.boot.initrd.secrets != { }) ''
The option `boot.initrd.secrets` has been deprecated in favour of `boot.initrd.secretPaths`.
'';
# Backwards compatibility to the legacy `boot.initrd.secrets` option.
boot.initrd.secretPaths = lib.mapAttrs' (dest: source: {
# The legacy boot.initrd.secrets option didn't type-check the attr
# names, so we need to optionally prepend a slash.
name = "${lib.optionalString (!lib.hasPrefix "/" dest) "/"}${dest}";
value.source = if dest != null then source else dest;
}) config.boot.initrd.secrets;
system.build = mkMerge [
{
inherit
nixos/modules/system/boot/systemd/initrd-secrets.nix
+17 -8
View File
@@ -10,14 +10,21 @@
# Copy secrets into the initrd if they cannot be appended
boot.initrd.systemd.contents = lib.mkIf (!config.boot.loader.supportsInitrdSecrets) (
lib.mapAttrs' (
dest: source:
lib.nameValuePair "/.initrd-secrets/${dest}" { source = if source == null then dest else source; }
) config.boot.initrd.secrets
_: scfg:
let
prefix = lib.optionalString scfg.intermediateSecretsDir "/.initrd-secrets";
in
lib.nameValuePair "${prefix}${scfg.path}" { inherit (scfg) source; }
) config.boot.initrd.secretPaths
);
# Copy secrets to their respective locations
boot.initrd.systemd.services.initrd-nixos-copy-secrets =
lib.mkIf (config.boot.initrd.secrets != { })
lib.mkIf
(
(builtins.any (x: x.intermediateSecretsDir) (builtins.attrValues config.boot.initrd.secretPaths))
|| config.boot.initrd.extraSecretsHook != ""
)
{
description = "Copy secrets into place";
# Run as early as possible
@@ -34,10 +41,12 @@
# drop this service, we'd mount the /run tmpfs over the secret, making it
# invisible in stage 2.
script = ''
for secret in $(cd /.initrd-secrets; find . -type f -o -type l); do
mkdir -p "$(dirname "/$secret")"
cp "/.initrd-secrets/$secret" "/$secret"
done
if [ -d /.initrd-secrets ]; then
for secret in $(cd /.initrd-secrets; find . -type f -o -type l); do
mkdir -p "$(dirname "/$secret")"
cp "/.initrd-secrets/$secret" "/$secret"
done
fi
'';
serviceConfig = {
nixos/modules/tasks/lvm.nix
+4 -4
View File
@@ -51,7 +51,7 @@ in
config = mkMerge [
{
# minimal configuration file to make lvmconfig/lvm2-activation-generator happy
environment.etc."lvm/lvm.conf".text = "config {}";
environment.etc."lvm/lvm.conf".text = lib.mkBefore "config {}";
}
(mkIf cfg.enable {
systemd.tmpfiles.packages = [ cfg.package.out ];
@@ -59,6 +59,8 @@ in
systemd.packages = [ cfg.package ];
services.udev.packages = [ cfg.package.out ];
environment.etc."lvm/lvm.conf".text =
"global/lvresize_fs_helper_executable = ${pkgs.lvm2.scripts}/libexec/lvresize_fs_helper";
})
(mkIf config.boot.initrd.services.lvm.enable {
# We need lvm2 for the device-mapper rules
@@ -71,9 +73,7 @@ in
systemd.sockets."dm-event".wantedBy = [ "sockets.target" ];
systemd.services."lvm2-monitor".wantedBy = [ "sysinit.target" ];
environment.etc."lvm/lvm.conf".text = ''
dmeventd/executable = "${cfg.package}/bin/dmeventd"
'';
environment.etc."lvm/lvm.conf".text = "dmeventd/executable = ${cfg.package}/bin/dmeventd";
services.lvm.package = mkDefault pkgs.lvm2_dmeventd;
})
(mkIf cfg.boot.thin.enable {
nixos/modules/testing/test-instrumentation.nix
+16 -2
View File
@@ -86,7 +86,8 @@ in
options.testing = {
backdoor = lib.mkEnableOption "backdoor service in stage 2" // {
default = true;
# See assertion below for why the backdoor doesn't work with containers.
default = !config.boot.isContainer;
};
initrdBackdoor = lib.mkEnableOption ''
@@ -105,7 +106,20 @@ in
{
assertion = cfg.initrdBackdoor -> config.boot.initrd.systemd.enable;
message = ''
testing.initrdBackdoor requires boot.initrd.systemd.enable to be enabled.
`testing.initrdBackdoor` requires `boot.initrd.systemd.enable` to be enabled.
'';
}
{
assertion = config.boot.isContainer -> !cfg.backdoor;
message = ''
`testing.backdoor` uses virtio console, which does not work with
containers (we use `nsenter` instead).
'';
}
{
assertion = config.boot.isContainer -> !cfg.initrdBackdoor;
message = ''
`testing.initrdBackdoor` does not work with containers as there is no initrd.
'';
}
];
nixos/modules/virtualisation/guest-networking-options.nix
+1 -1
View File
@@ -71,7 +71,7 @@ in
virtualisation.vlans = lib.mkOption {
type = types.listOf types.ints.unsigned;
default = if cfg.interfaces == { } then [ 1 ] else [ ];
defaultText = lib.literalExpression "if cfg.interfaces == {} then [ 1 ] else [ ]";
defaultText = lib.literalExpression "if config.virtualisation.interfaces == {} then [ 1 ] else [ ]";
example = [
1
2
nixos/modules/virtualisation/nspawn-container/default.nix
+42
View File
@@ -70,6 +70,45 @@ in
config = {
boot.isNspawnContainer = true;
assertions = [
{
assertion = config.specialisation == { };
message = ''
Setting 'specialisation' is disallowed for systemd-nspawn container configurations.
Activating a specialisation requires creating SUID wrappers (e.g., for 'sudo'),
which is prohibited within the Nix build sandbox where the test is run.
'';
}
{
# Check every interface defined in allInterfaces.
# Containers try to create a bridge "${config.system.name}-${interfaceName}"
assertion = lib.all (
iface:
let
hostName = "${config.system.name}-${iface.name}";
in
lib.stringLength hostName <= 15
) (lib.attrValues cfg.allInterfaces);
message =
let
offendingInterfaces = lib.filter (
iface: lib.stringLength "${config.system.name}-${iface.name}" > 15
) (lib.attrValues cfg.allInterfaces);
offenderList = map (
i:
"${config.system.name}-${i.name} (${toString (lib.stringLength "${config.system.name}-${i.name}")} chars)"
) offendingInterfaces;
in
''
The following generated host interface names exceed the Linux 15-character limit:
${lib.concatStringsSep "\n " offenderList}
Please shorten 'config.system.name' or the interface names in 'virtualisation.interfaces'.
'';
}
];
# TODO(arianvp): Remove after https://github.com/NixOS/nixpkgs/pull/480686 is merged
console.enable = true;
@@ -94,6 +133,9 @@ in
# > kind of unit allocation or registration with systemd-machined.
"--keep-unit"
"--register=no"
# Send a READY=1 notification to a socket when the container is fully booted.
"--notify-ready=yes"
];
system.build.nspawn =
nixos/modules/virtualisation/nspawn-container/run-nspawn/src/run_nspawn/__init__.py
+29
View File
@@ -68,7 +68,9 @@ def ensure_vlan_bridge(vlan: int) -> typing.Generator[str, None, None]:
ipv6_addr = f"2001:db8:{vlan}::fe/64"
bridge_name = f"br{vlan}"
tap_name = f"vde-tap{vlan}"
bridge_path = Path("/sys/class/net") / bridge_name
tap_path = Path("/sys/class/net") / tap_name
try:
# To avoid racing against other nspawn containers that also
# need this vlan, grab an exclusive lock.
@@ -80,6 +82,19 @@ def ensure_vlan_bridge(vlan: int) -> typing.Generator[str, None, None]:
run_ip("addr", "add", ipv4_addr, "dev", bridge_name)
run_ip("addr", "add", ipv6_addr, "dev", bridge_name)
if tap_path.exists():
logger.info(f"attaching {tap_name} to {bridge_name}")
run_ip("link", "set", tap_name, "master", bridge_name)
run_ip("link", "set", tap_name, "up")
else:
logger.warning(
f"TAP {tap_name} not found; container will be isolated from VDE"
)
if not Path("/dev/net").exists():
logger.warning(
"A common reason for this is that /dev/net is not available in the Nix sandbox. Try adding /dev/net to extra-sandbox-paths."
)
yield bridge_name
finally:
# To avoid racing against other nspawn containers that also
@@ -126,6 +141,7 @@ def mk_veth(
def run(
container_name: str,
root_dir_str: str,
shared_dir_str: typing.Optional[str],
interfaces: dict,
nspawn_options: list[str],
init: str,
@@ -166,12 +182,19 @@ def run(
flush=True,
)
shared_dir = Path(shared_dir_str) if shared_dir_str else None
cp = subprocess.Popen(
[
"@systemd-nspawn@",
*nspawn_options,
f"--directory={root_dir}",
f"--network-namespace-path={netns.path}",
*(
[f"--bind={shared_dir}:/tmp/shared"]
if shared_dir is not None
else []
),
init,
*cmdline,
],
@@ -218,6 +241,11 @@ def main():
required=True,
help="Path to container root directory (overridable with RUN_NSPAWN_ROOT_DIR)",
)
arg_parser.add_argument(
"--shared-dir",
required=False,
help="Path to a shared directory to bind-mount into the container at /tmp/shared (overridable with RUN_NSPAWN_SHARED_DIR)",
)
arg_parser.add_argument(
"--interfaces-json",
dest="interfaces",
@@ -239,6 +267,7 @@ def main():
run(
container_name=args.container_name,
root_dir_str=os.getenv("RUN_NSPAWN_ROOT_DIR", default=args.root_dir),
shared_dir_str=os.getenv("RUN_NSPAWN_SHARED_DIR", default=args.shared_dir),
interfaces=args.interfaces,
nspawn_options=nspawn_options,
init=args.init,
nixos/tests/all-tests.nix
+2 -3
View File
@@ -168,6 +168,7 @@ in
node-name = runTest ./nixos-test-driver/node-name.nix;
busybox = runTest ./nixos-test-driver/busybox.nix;
console-log = runTest ./nixos-test-driver/console-log.nix;
containers = runTest ./nixos-test-driver/containers.nix;
driver-timeout =
pkgs.runCommand "ensure-timeout-induced-failure"
{
@@ -1138,9 +1139,6 @@ in
nixos-rebuild-target-host = runTest {
imports = [ ./nixos-rebuild-target-host.nix ];
};
nixos-rebuild-target-host-interrupted = runTest {
imports = [ ./nixos-rebuild-target-host-interrupted.nix ];
};
nixpkgs = pkgs.callPackage ../modules/misc/nixpkgs/test.nix { inherit evalMinimalConfig; };
nixpkgs-config-allow-unfree =
pkgs.callPackage ../modules/misc/nixpkgs/test-nixpkgs-config-allow-unfree.nix
@@ -1649,6 +1647,7 @@ in
teleports = runTest ./teleports.nix;
temporal = runTest ./temporal.nix;
terminal-emulators = handleTest ./terminal-emulators.nix { };
test-containers-bittorrent = runTest ./test-containers-bittorrent.nix;
thanos = runTest ./thanos.nix;
thelounge = handleTest ./thelounge.nix { };
tiddlywiki = runTest ./tiddlywiki.nix;
nixos/tests/bootspec.nix
+1 -1
View File
@@ -128,7 +128,7 @@ in
environment.systemPackages = [ pkgs.jq ];
# It's probably the case, but we want to make it explicit here.
boot.initrd.enable = true;
boot.initrd.secrets."/some/example" = pkgs.writeText "example-secret" "test";
boot.initrd.secretPaths."/some/example".source = pkgs.writeText "example-secret" "test";
};
testScript = ''
nixos/tests/firefox.nix
+2 -2
View File
@@ -46,7 +46,7 @@
@contextmanager
def record_audio(machine: Machine):
def record_audio(machine: BaseMachine):
"""
Perform actions while recording the
machine audio output.
@@ -56,7 +56,7 @@
machine.systemctl("stop audio-recorder")
def wait_for_sound(machine: Machine):
def wait_for_sound(machine: BaseMachine):
"""
Wait until any sound has been emitted.
"""
nixos/tests/initrd-luks-empty-passphrase.nix
+1 -1
View File
@@ -47,7 +47,7 @@ in
};
};
virtualisation.rootDevice = "/dev/mapper/cryptroot";
boot.initrd.secrets."/etc/cryptroot.key" = keyfile;
boot.initrd.secretPaths."/etc/cryptroot.key".source = keyfile;
};
specialisation.boot-luks-missing-keyfile.configuration = {
nixos/tests/initrd-secrets-changing.nix
+6 -6
View File
@@ -21,16 +21,16 @@ testing.makeTest {
boot.loader.grub.device = "/dev/vda";
boot.initrd.secrets = {
"/test" = secret1InStore;
"/run/keys/test" = secret1InStore;
boot.initrd.secretPaths = {
"/test".source = secret1InStore;
"/run/keys/test".source = secret1InStore;
};
boot.initrd.postMountCommands = "cp /test /mnt-root/secret-from-initramfs";
specialisation.secrets2System.configuration = {
boot.initrd.secrets = lib.mkForce {
"/test" = secret2InStore;
"/run/keys/test" = secret2InStore;
boot.initrd.secretPaths = lib.mkForce {
"/test".source = secret2InStore;
"/run/keys/test".source = secret2InStore;
};
};
};
nixos/tests/initrd-secrets.nix
+20 -6
View File
@@ -21,14 +21,26 @@ let
{ ... }:
{
virtualisation.useBootLoader = true;
boot.initrd.secrets = {
"/test" = secretInStore;
boot.initrd.secretPaths = {
"/test" = {
source = secretInStore;
intermediateSecretsDir = false;
};
# This should *not* need to be copied in postMountCommands
"/run/keys/test" = secretInStore;
"/run/keys/test1".source = secretInStore;
"/run/keys/test2".generateSecretCommand = pkgs.writeShellScript "copy-secret" ''
cp ${secretInStore} "$out"
'';
};
boot.initrd.extraSecretsHook = ''
mkdir -p etc/secrets
cp ${secretInStore} etc/secrets/test2
'';
boot.initrd.postMountCommands = ''
cp /test /mnt-root/secret-from-initramfs
cp /test /mnt-root/secret-from-initramfs-1
cp /etc/secrets/test2 /mnt-root/secret-from-initramfs-2
'';
boot.initrd.compressor = compressor;
# zstd compression is only supported from 5.9 onwards. Remove when 5.10 becomes default.
@@ -39,8 +51,10 @@ let
start_all()
machine.wait_for_unit("multi-user.target")
machine.succeed(
"cmp ${secretInStore} /secret-from-initramfs",
"cmp ${secretInStore} /run/keys/test",
"cmp ${secretInStore} /secret-from-initramfs-1",
"cmp ${secretInStore} /secret-from-initramfs-2",
"cmp ${secretInStore} /run/keys/test1",
"cmp ${secretInStore} /run/keys/test2",
)
'';
};
nixos/tests/installer.nix
+3 -3
View File
@@ -67,7 +67,7 @@ let
boot.loader.systemd-boot.enable = true;
''}
boot.initrd.secrets."/etc/secret" = "/etc/nixos/secret";
boot.initrd.secretPaths."/etc/secret".source = "/etc/nixos/secret";
${optionalString clevisTest ''
boot.kernelParams = [ "console=tty0" "ip=192.168.1.1:::255.255.255.0::eth1:none" ];
@@ -1385,7 +1385,7 @@ in
};
# Full disk encryption (root, kernel and initrd encrypted) using GRUB, GPT/UEFI,
# LVM-on-LUKS and a keyfile in initrd.secrets to enter the passphrase once
# LVM-on-LUKS and a keyfile in initrd.secretPaths to enter the passphrase once
fullDiskEncryption = makeInstallerTest "fullDiskEncryption" {
createPartitions = ''
installer.succeed(
@@ -1419,7 +1419,7 @@ in
boot.loader.grub.enableCryptodisk = true;
boot.loader.efi.efiSysMountPoint = "/boot/efi";
boot.initrd.secrets."/luks.key" = "/etc/nixos/luks.key";
boot.initrd.secretPaths."/luks.key" = "/etc/nixos/luks.key";
boot.initrd.luks.devices.crypt =
{ device = "/dev/vda2";
keyFile = "/luks.key";
nixos/tests/jenkins-cli.nix
+1 -1
View File
@@ -25,7 +25,7 @@
assert "http://0.0.0.0:8080" in machine.succeed("echo $JENKINS_URL")
machine.succeed(
"jenkins-cli -auth admin:$(cat /var/lib/jenkins/secrets/initialAdminPassword)"
"jenkins-cli -http -auth admin:$(cat /var/lib/jenkins/secrets/initialAdminPassword)"
)
'';
}
nixos/tests/jenkins.nix
+2 -2
View File
@@ -118,7 +118,7 @@
master.wait_until_succeeds("test -f ${jenkinsHome}/jobs/folder-1/jobs/job-2/config.xml")
# Verify that jenkins also sees the jobs.
out = master.succeed("${pkgs.jenkins}/bin/jenkins-cli -s ${jenkinsUrl} -auth admin:$(cat ${jenkinsHome}/secrets/initialAdminPassword) list-jobs")
out = master.succeed("${pkgs.jenkins}/bin/jenkins-cli -http -s ${jenkinsUrl} -auth admin:$(cat ${jenkinsHome}/secrets/initialAdminPassword) list-jobs")
jobs = [x.strip() for x in out.splitlines()]
# Seeing jobs inside folders requires the Folders plugin
# (https://plugins.jenkins.io/cloudbees-folder/), which we don't have
@@ -135,7 +135,7 @@
master.wait_until_fails("test -f ${jenkinsHome}/jobs/folder-1/jobs/job-2/config.xml")
# Verify that jenkins also sees the jobs as removed.
out = master.succeed("${pkgs.jenkins}/bin/jenkins-cli -s ${jenkinsUrl} -auth admin:$(cat ${jenkinsHome}/secrets/initialAdminPassword) list-jobs")
out = master.succeed("${pkgs.jenkins}/bin/jenkins-cli -http -s ${jenkinsUrl} -auth admin:$(cat ${jenkinsHome}/secrets/initialAdminPassword) list-jobs")
jobs = [x.strip() for x in out.splitlines()]
assert jobs == [], f"jobs != []: {jobs}"
'';
nixos/tests/nixos-rebuild-target-host-interrupted.nix
-232
View File
@@ -1,232 +0,0 @@
{ hostPkgs, ... }:
# This test recreates a remote deployment scenario where the connection
# between deployer and target is closed during the deployment - in this
# case because the connection goes over a 'reverse ssh' tunnel service
# that has changes that are being deployed.
# This is not seamless (the deployer doesn't get to see the logs after
# the disconnect), but is a lot better than the old behaviour, where
# the switch was aborted and the connection never restored.
{
name = "nixos-rebuild-target-host-interrupted";
# TODO: remove overlay from nixos/modules/profiles/installation-device.nix
# make it a _small package instead, then remove pkgsReadOnly = false;.
node.pkgsReadOnly = false;
nodes = {
deployer =
{
nodes,
lib,
pkgs,
...
}:
let
inherit (import ./ssh-keys.nix pkgs) snakeOilPrivateKey snakeOilPublicKey;
in
{
imports = [
../modules/profiles/installation-device.nix
];
nix.settings = {
substituters = lib.mkForce [ ];
hashed-mirrors = null;
connect-timeout = 1;
};
system.includeBuildDependencies = true;
virtualisation = {
cores = 2;
memorySize = 3072;
};
services.openssh.enable = true;
users.users.root.openssh.authorizedKeys.keys = [ nodes.target.system.build.publicKey ];
system.extraDependencies = [
# so that it doesn't need to be built inside the test
pkgs.nixVersions.latest
];
system.build.privateKey = snakeOilPrivateKey;
system.build.publicKey = snakeOilPublicKey;
system.switch.enable = true;
services.getty.autologinUser = lib.mkForce "root";
};
target =
{
nodes,
lib,
pkgs,
...
}:
let
inherit (import ./ssh-keys.nix pkgs) snakeOilPrivateKey snakeOilPublicKey;
targetConfig = {
documentation.enable = false;
services.openssh.enable = true;
system.build.privateKey = snakeOilPrivateKey;
system.build.publicKey = snakeOilPublicKey;
users.users.root.openssh.authorizedKeys.keys = [ nodes.deployer.system.build.publicKey ];
users.users.alice.openssh.authorizedKeys.keys = [ nodes.deployer.system.build.publicKey ];
users.users.bob.openssh.authorizedKeys.keys = [ nodes.deployer.system.build.publicKey ];
users.users.alice.extraGroups = [ "wheel" ];
users.users.bob.extraGroups = [ "wheel" ];
# Disable sudo for root to ensure sudo isn't called without `--sudo`
security.sudo.extraRules = lib.mkForce [
{
groups = [ "wheel" ];
commands = [ { command = "ALL"; } ];
}
{
users = [ "alice" ];
commands = [
{
command = "ALL";
options = [ "NOPASSWD" ];
}
];
}
];
nix.settings.trusted-users = [ "@wheel" ];
services.autossh-ng.sessions.will-be-interrupted-by-rebuild = {
user = "root";
destination = "deployer";
extraArguments = "-R2222:localhost:22";
hostKeyChecking = false;
};
};
in
{
imports = [ ./common/user-account.nix ];
config = lib.mkMerge [
targetConfig
{
system.build = {
inherit targetConfig;
};
system.switch.enable = true;
networking.hostName = "target";
}
];
};
};
testScript =
{ nodes, ... }:
let
sshConfig = builtins.toFile "ssh.conf" ''
UserKnownHostsFile=/dev/null
StrictHostKeyChecking=no
'';
targetConfigJSON = hostPkgs.writeText "target-configuration.json" (
builtins.toJSON nodes.target.system.build.targetConfig
);
targetNetworkJSON = hostPkgs.writeText "target-network.json" (
builtins.toJSON nodes.target.system.build.networkConfig
);
configFile =
hostname:
hostPkgs.writeText "configuration.nix" # nix
''
{ lib, pkgs, modulesPath, ... }: {
imports = [
(modulesPath + "/virtualisation/qemu-vm.nix")
(modulesPath + "/testing/test-instrumentation.nix")
(modulesPath + "/../tests/common/user-account.nix")
(lib.modules.importJSON ./target-configuration.json)
(lib.modules.importJSON ./target-network.json)
./hardware-configuration.nix
];
boot.loader.grub = {
enable = true;
device = "/dev/vda";
forceInstall = true;
};
# needed to make NIX_SSHOPTS work for nix-copy-closure
# 2.31.3 (current default) break, 2.32.6 and 2.33.3 (current latest) work
# let's use the default here again once the fix has made it there.
nix.package = pkgs.nixVersions.latest;
# We're changing the '-E' parameter to the new hostname here,
# not because we care about the logs, but because we want to
# force the scenario where the connection is broken during the
# deployment (because the autossh-ng service is stopped and
# started):
services.autossh-ng.sessions.will-be-interrupted-by-rebuild.extraArguments = "-R2222:localhost:22 -E ${hostname}";
# this will be asserted to validate the switch happened:
networking.hostName = "${hostname}";
}
'';
in
# python
''
start_all()
target.wait_for_open_port(22)
deployer.wait_until_succeeds("ping -c1 target")
deployer.succeed("install -Dm 600 ${nodes.deployer.system.build.privateKey} ~root/.ssh/id_ecdsa")
deployer.succeed("install ${sshConfig} ~root/.ssh/config")
target.succeed("nixos-generate-config")
target.succeed("install -Dm 600 ${nodes.target.system.build.privateKey} ~root/.ssh/id_ecdsa")
deployer.succeed("scp alice@target:/etc/nixos/hardware-configuration.nix /root/hardware-configuration.nix")
target.wait_for_unit("autossh-ng-will-be-interrupted-by-rebuild.service")
deployer.copy_from_host("${configFile "config-1-deployed"}", "/root/configuration-1.nix")
deployer.copy_from_host("${configFile "config-2-deployed"}", "/root/configuration-2.nix")
deployer.copy_from_host("${targetNetworkJSON}", "/root/target-network.json")
deployer.copy_from_host("${targetConfigJSON}", "/root/target-configuration.json")
with subtest("Deploy to alice@target via reverse ssh"):
deployer.wait_for_unit("multi-user.target")
# Uses TTY/send_chars instead of deployer.succeed to set NIX_SSHOPTS
deployer.send_chars("NIX_SSHOPTS=\"-p 2222\" nixos-rebuild switch -I nixos-config=/root/configuration-1.nix --target-host alice@localhost --sudo\n")
# the connection breaks, but the 'switch' should now continue in the background:
deployer.wait_until_tty_matches("1", "error: while running command with remote sudo")
def deployed(last_try: bool) -> bool:
target_hostname = deployer.succeed("ssh alice@target cat /etc/hostname", timeout=20).rstrip()
if last_try:
print(f"Still seeing hostname {target_hostname}")
return target_hostname == "config-1-deployed"
retry(deployed)
with subtest("Deploy to bob@target via reverse ssh with password-based sudo"):
deployer.wait_for_unit("multi-user.target")
# Uses TTY/send_chars instead of deployer.succeed to set NIX_SSHOPTS and for ask-sudo-password
deployer.send_chars("NIX_SSHOPTS=\"-p 2222\" nixos-rebuild switch -I nixos-config=/root/configuration-2.nix --target-host bob@localhost --ask-sudo-password\n")
deployer.wait_until_tty_matches("1", "password for bob")
deployer.send_chars("${nodes.target.users.users.bob.password}\n")
# the connection breaks, but the 'switch' should now continue in the background:
deployer.wait_until_tty_matches("1", "error: while running command with remote sudo")
def deployed(last_try: bool) -> bool:
target_hostname = deployer.succeed("ssh alice@target cat /etc/hostname", timeout=20).rstrip()
if last_try:
print(f"Still seeing hostname {target_hostname}")
return target_hostname == "config-2-deployed"
retry(deployed)
'';
}
nixos/tests/nixos-test-driver/containers.nix
+77
View File
@@ -0,0 +1,77 @@
{ pkgs, ... }:
{
name = "containers";
meta.maintainers = with pkgs.lib.maintainers; [ jfly ];
nodes = {
n1 = {
virtualisation.vlans = [ 1 ];
};
n2 = {
virtualisation.vlans = [
2
];
};
};
containers = {
c1 = {
virtualisation.vlans = [ 1 ];
};
c2 = {
virtualisation.vlans = [ 2 ];
};
c12 = {
virtualisation.vlans = [
1
2
];
};
};
testScript = /* python */ ''
c1.start()
c2.start()
c12.start()
c1.succeed("echo hello > /hello.txt")
c1.copy_from_machine("/hello.txt")
c1.systemctl("start network-online.target")
c2.systemctl("start network-online.target")
c12.systemctl("start network-online.target")
c1.wait_for_unit("network-online.target")
c2.wait_for_unit("network-online.target")
c12.wait_for_unit("network-online.target")
# Confirm containers in vlan 1 can talk to each other.
c1.succeed("ping -c 1 c12")
c12.succeed("ping -c 1 c1")
# Confirm containers in vlan 2 can talk to each other.
c2.succeed("ping -c 1 c12")
c12.succeed("ping -c 1 c2")
# Confirm containers in separate vlans cannot talk to each other.
c1.fail("ping -c 1 -W 1 c2")
n1.start()
n2.start()
n1.systemctl("start network-online.target")
n2.systemctl("start network-online.target")
n1.wait_for_unit("network-online.target")
n2.wait_for_unit("network-online.target")
# Confirm containers and nodes in the same vlan can talk to each other.
c1.succeed("ping -c 1 n1")
n1.succeed("ping -c 1 c1")
c2.succeed("ping -c 1 n2")
n2.succeed("ping -c 1 c2")
# Confirm containers and nodes in different vlans cannot talk to each other.
c1.fail("ping -c 1 -W 1 n2")
n1.fail("ping -c 1 -W 1 c2")
c2.fail("ping -c 1 -W 1 n1")
n2.fail("ping -c 1 -W 1 c1")
'';
}
nixos/tests/olivetin.nix
+13 -3
View File
@@ -1,4 +1,4 @@
{ lib, ... }:
{ config, lib, ... }:
{
name = "olivetin";
@@ -41,16 +41,26 @@
testScript = ''
import json
import shlex
machine.wait_for_unit("olivetin.service")
machine.wait_for_open_port(8000)
response = json.loads(machine.succeed("curl http://localhost:8000/api/StartActionByGetAndWait/hello_world"))
def start_action(action):
if "${config.nodes.machine.services.olivetin.package.releaseSeries}" == "2k":
cmd = f"curl http://localhost:8000/api/StartActionByGetAndWait/{action}"
else:
req = {"actionId": action}
cmd = f"curl -H 'Content-Type: application/json' http://localhost:8000/api/olivetin.api.v1.OliveTinApiService/StartActionAndWait -d {shlex.quote(json.dumps(req))}"
return json.loads(machine.succeed(cmd))
response = start_action("hello_world")
assert response["logEntry"]["exitCode"] == 0
assert response["logEntry"]["output"] == "Hello World!"
assert machine.succeed("cat /tmp/result") == "Hello World!"
response = json.loads(machine.succeed("curl http://localhost:8000/api/StartActionByGetAndWait/secret"))
response = start_action("secret")
assert response["logEntry"]["exitCode"] == 0
assert machine.succeed("cat /tmp/result2") == "secret"
'';
nixos/tests/radicle-ci-broker.nix
+4 -1
View File
@@ -19,9 +19,12 @@ in
meta.maintainers = with lib.maintainers; [ defelo ];
nodes.seed = {
virtualisation.credentials = {
"xyz.radicle.node.secret".source = "${seed-ssh-keys.snakeOilEd25519PrivateKey}";
};
services.radicle = {
enable = true;
privateKeyFile = seed-ssh-keys.snakeOilEd25519PrivateKey;
publicKey = seed-ssh-keys.snakeOilEd25519PublicKey;
node.openFirewall = true;
settings = {
nixos/tests/radicle.nix
+4 -1
View File
@@ -76,9 +76,12 @@ in
{
imports = [ commonHostConfig ];
virtualisation.credentials = {
"xyz.radicle.node.secret".source = "${seed-ssh-keys.snakeOilEd25519PrivateKey}";
};
services.radicle = {
enable = true;
privateKeyFile = seed-ssh-keys.snakeOilEd25519PrivateKey;
publicKey = seed-ssh-keys.snakeOilEd25519PublicKey;
node = {
openFirewall = true;
nixos/tests/systemd-initrd-luks-keyfile.nix
+1 -1
View File
@@ -38,7 +38,7 @@ in
};
};
virtualisation.rootDevice = "/dev/mapper/cryptroot";
boot.initrd.secrets."/etc/cryptroot.key" = keyfile;
boot.initrd.secretPaths."/etc/cryptroot.key".source = keyfile;
};
};
nixos/tests/test-containers-bittorrent.nix
+215
View File
@@ -0,0 +1,215 @@
# This test runs a Bittorrent tracker on one machine, and verifies
# that two client machines can download the torrent using
# `aria2c'. The first client (behind a NAT router) downloads
# from the initial seeder running on the tracker. Then we kill the
# initial seeder. The second client downloads from the first client,
# which only works if the first client successfully uses the UPnP-IGD
# protocol to poke a hole in the NAT.
# We use aria2 as the initial seeder because transmission
# fails in the sandbox because of systemd hardening settings,
# namely MountAPIVFS=yes, so we get the following error:
# $ journalctl --unit transmission.service
# (n-daemon)[417]: transmission.service: Failed to create destination mount point node '/run/transmission/run/host/.os-release-stage/', ignoring: Read-only file system
# (n-daemon)[417]: transmission.service: Failed to mount /run/systemd/propagate/.os-release-stage to /run/transmission/run/host/.os-release-stage/: No such file or directory
# (n-daemon)[417]: transmission.service: Failed to set up mount namespacing: /run/host/.os-release-stage/: No such file or directory
# (n-daemon)[417]: transmission.service: Failed at step NAMESPACE spawning /nix/store/zfksw9bllp95pl45d1nxmpd2lks42bkj-transmission-4.0.6/bin/transmission-daemon: No such file or directory
# systemd[1]: transmission.service: Main process exited, code=exited, status=226/NAMESPACE
{ lib, hostPkgs, ... }:
let
# Some random file to serve.
file = hostPkgs.hello.src;
internalRouterAddress = "192.168.3.1";
internalClient1Address = "192.168.3.2";
# cannot use documentation networks (198.51.100.0/24 or 192.0.2.0/24) here
# because miniupnpd recognizes them as such and refuses to work with them
# https://github.com/miniupnp/miniupnp/blob/2a74cb2f27cacf06d2b50c187e8f90aa1f5c2528/miniupnpd/miniupnpd.c#L998
externalRouterAddress = "80.100.100.1";
externalClient2Address = "80.100.100.2";
externalTrackerAddress = "80.100.100.3";
download-dir = "/tmp/aria2-downloads";
peerConfig =
{ pkgs, ... }:
{
environment.systemPackages = [
pkgs.aria2
pkgs.transmission_4 # only needed for transmission-create
];
};
in
{
name = "bittorrent";
meta = {
maintainers = [
lib.maintainers.kmein
];
};
containers = {
tracker =
{ pkgs, ... }:
{
imports = [ peerConfig ];
virtualisation.vlans = [ 1 ];
networking.firewall.enable = false;
networking.interfaces.eth1.ipv4.addresses = [
{
address = externalTrackerAddress;
prefixLength = 24;
}
];
# We need Apache on the tracker to serve the torrents.
services.httpd = {
enable = true;
virtualHosts = {
"torrentserver.org" = {
adminAddr = "foo@example.org";
documentRoot = "/tmp";
};
};
};
services.opentracker.enable = true;
};
router =
{ pkgs, containers, ... }:
{
virtualisation.vlans = [
1
2
];
networking.nat.enable = true;
networking.nat.internalInterfaces = [ "eth2" ];
networking.nat.externalInterface = "eth1";
networking.firewall.enable = true;
networking.firewall.trustedInterfaces = [ "eth2" ];
networking.interfaces.eth0.ipv4.addresses = [ ];
networking.interfaces.eth1.ipv4.addresses = [
{
address = externalRouterAddress;
prefixLength = 24;
}
];
networking.interfaces.eth2.ipv4.addresses = [
{
address = internalRouterAddress;
prefixLength = 24;
}
];
networking.nftables.enable = true;
services.miniupnpd = {
enable = true;
externalInterface = "eth1";
internalIPs = [ "eth2" ];
appendConfig = ''
ext_ip=${externalRouterAddress}
'';
};
};
client1 =
{ pkgs, containers, ... }:
{
imports = [ peerConfig ];
environment.systemPackages = [ pkgs.miniupnpc ];
virtualisation.vlans = [ 2 ];
networking.interfaces.eth0.ipv4.addresses = [ ];
networking.interfaces.eth1.ipv4.addresses = [
{
address = internalClient1Address;
prefixLength = 24;
}
];
networking.defaultGateway = internalRouterAddress;
networking.firewall.enable = false;
};
client2 =
{ pkgs, ... }:
{
imports = [ peerConfig ];
virtualisation.vlans = [ 1 ];
networking.interfaces.eth0.ipv4.addresses = [ ];
networking.interfaces.eth1.ipv4.addresses = [
{
address = externalClient2Address;
prefixLength = 24;
}
];
networking.firewall.enable = false;
};
};
testScript =
{ containers, ... }:
''
start_all()
# Wait for network and miniupnpd.
router.systemctl("start network-online.target")
router.wait_for_unit("network-online.target")
router.wait_for_unit("miniupnpd")
# Create the torrent.
tracker.succeed("mkdir -p ${download-dir}")
tracker.succeed(
"cp ${file} ${download-dir}/test.tar.bz2"
)
tracker.succeed(
"transmission-create ${download-dir}/test.tar.bz2 --private --tracker http://${externalTrackerAddress}:6969/announce --outfile /tmp/test.torrent"
)
tracker.succeed("chmod 644 /tmp/test.torrent")
# Start the tracker
tracker.systemctl("start network-online.target")
tracker.wait_for_unit("network-online.target")
tracker.wait_for_unit("opentracker.service")
tracker.wait_for_open_port(6969)
# --- Start the initial seeder using aria2 ---
# https://stackoverflow.com/a/44528978
tracker.execute(
"aria2c --enable-dht=false --seed-time=999 --dir=${download-dir} "
"-V --seed-ratio=0.0 "
"/tmp/test.torrent >/dev/null &"
)
# --- Wait until the tracker shows we are seeding ---
tracker.wait_until_succeeds("curl -s http://localhost:6969/stats | grep -q 'serving 1 torrents'")
# Now we should be able to download from the client behind the NAT.
tracker.wait_for_unit("httpd")
def connect_from(machine):
machine.systemctl("start network-online.target")
machine.wait_for_unit("network-online.target")
machine.execute(
"aria2c --enable-dht=false --seed-time=999 --dir=${download-dir} "
"http://${externalTrackerAddress}/test.torrent >/dev/null &"
)
machine.wait_until_succeeds(
"cmp ${download-dir}/test.tar.bz2 ${file}"
) # Wait for download to finish and verify
connect_from(client1)
# --- Bring down the initial seeder ---
tracker.succeed("pkill aria2c")
# Now download from the second client. This can only succeed if
# the first client created a NAT hole in the router.
connect_from(client2)
'';
}
pkgs/applications/audio/ladspa-plugins/default.nix
+2 -2
View File
@@ -5,7 +5,7 @@
autoreconfHook,
automake,
fftw,
ladspaH,
ladspa-header,
libxml2,
pkg-config,
perlPackages,
@@ -41,7 +41,7 @@ stdenv.mkDerivation (finalAttrs: {
];
buildInputs = [
fftw
ladspaH
ladspa-header
libxml2
];
pkgs/applications/audio/non/default.nix
+2 -2
View File
@@ -9,7 +9,7 @@
ntk,
libjack2,
libsndfile,
ladspaH,
ladspa-header,
liblo,
libsigcxx,
lrdf,
@@ -37,7 +37,7 @@ stdenv.mkDerivation {
ntk
libjack2
libsndfile
ladspaH
ladspa-header
liblo
libsigcxx
lrdf
pkgs/applications/audio/zynaddsubfx/default.nix
+2 -2
View File
@@ -21,7 +21,7 @@
alsa-lib,
dssiSupport ? false,
dssi,
ladspaH,
ladspa-header,
jackSupport ? true,
libjack2,
ossSupport ? true,
@@ -108,7 +108,7 @@ stdenv.mkDerivation (finalAttrs: {
++ lib.optionals alsaSupport [ alsa-lib ]
++ lib.optionals dssiSupport [
dssi
ladspaH
ladspa-header
]
++ lib.optionals jackSupport [ libjack2 ]
++ lib.optionals portaudioSupport [ portaudio ]
pkgs/applications/editors/vim/plugins/non-generated/blink-cmp/0001-pin-frizbee-0.6.0.patch
-29
View File
@@ -1,29 +0,0 @@
diff --git a/Cargo.lock b/Cargo.lock
index 35bb10e..71c79eb 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -144,9 +144,9 @@ checksum = "48c757948c5ede0e46177b7add2e67155f70e33c07fea8284df6576da70b3719"
[[package]]
name = "frizbee"
-version = "0.7.0"
+version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "4d024031f1a5bc5f19917baa0b618f1067610e35ba23e9f105653fcb27e74f5c"
+checksum = "c3365720de81dac18e889afa72f5907aa061c975548da68e2400c056ebc94aec"
dependencies = [
"multiversion",
"rayon",
diff --git a/Cargo.toml b/Cargo.toml
index 392d1bb..c776c7d 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -9,7 +9,7 @@ crate-type = ["cdylib"]
[dependencies]
regex = "1.11.2"
-frizbee = "0.7.0"
+frizbee = "0.6.0"
mlua = { version = "0.11.3", features = ["module", "luajit"] }
thiserror = "2.0.16"
blake3 = "1.8.2"
pkgs/applications/editors/vim/plugins/non-generated/blink-cmp/default.nix
+3 -9
View File
@@ -8,24 +8,18 @@
gitMinimal,
}:
let
version = "1.9.1";
version = "1.10.1";
src = fetchFromGitHub {
owner = "Saghen";
repo = "blink.cmp";
tag = "v${version}";
hash = "sha256-GgodXdWpQoF2z1g1/WvnSpfuhskw0aMcOoyZM5l66q8=";
hash = "sha256-y8f+bmPkb3M6DzcUkJMxd2woDLoBYslne7aB8A0ejCk=";
};
blink-fuzzy-lib = rustPlatform.buildRustPackage {
inherit version src;
pname = "blink-fuzzy-lib";
cargoHash = "sha256-Qdt8O7IGj2HySb1jxsv3m33ZxJg96Ckw26oTEEyQjfs=";
# NOTE: The only change in frizbee 0.7.0 was nixpkgs incompatible rust semantic changes
# Patch just reverts https://github.com/saghen/blink.cmp/commit/cc824ec85b789a54d05241389993c6ab8c040810
cargoPatches = [
./0001-pin-frizbee-0.6.0.patch
];
cargoHash = "sha256-3o2n4xwNF9Fc3VlPKf3lnvmN7FVus5jQB8gcXXwz50c=";
nativeBuildInputs = [ gitMinimal ];
pkgs/applications/editors/vscode/extensions/anthropic.claude-code/default.nix
+2 -2
View File
@@ -8,8 +8,8 @@ vscode-utils.buildVscodeMarketplaceExtension {
mktplcRef = {
name = "claude-code";
publisher = "anthropic";
version = "2.1.78";
hash = "sha256-A8i7yU4W8Fjp1h7EFZKAEdyoqoKBVzJDvEZ+Y06dBXg=";
version = "2.1.79";
hash = "sha256-vQuSSpBcvd7XRTeprk8sMZmdRU6JiwPSmIQyBs94I5M=";
};
postInstall = ''
pkgs/applications/editors/vscode/extensions/jjk.jjk/default.nix
+2 -2
View File
@@ -7,8 +7,8 @@ vscode-utils.buildVscodeMarketplaceExtension {
mktplcRef = {
name = "jjk";
publisher = "jjk";
version = "0.9.1";
hash = "sha256-7CK2fmYVAd12kLTnq3vwDmgL22Tmi9Ljt9+tpXqRWuo=";
version = "0.9.3";
hash = "sha256-wkHMZTLi3dDV6JQdfJ4hI5uwGAlCmKwg+v8Z9RMU1wU=";
};
meta = {
changelog = "https://github.com/keanemind/jjk/releases";
pkgs/applications/editors/vscode/extensions/kilocode.kilo-code/default.nix
+12 -67
View File
@@ -1,80 +1,25 @@
{
lib,
stdenvNoCC,
fetchFromGitHub,
pnpm,
fetchPnpmDeps,
pnpmConfigHook,
nodejs,
vscode-utils,
nix-update-script,
vscode-extension-update-script,
}:
let
vsix = stdenvNoCC.mkDerivation (finalAttrs: {
name = "kilo-code-${finalAttrs.version}.vsix";
pname = "kilo-code-vsix";
version = "4.124.0";
src = fetchFromGitHub {
owner = "Kilo-Org";
repo = "kilocode";
tag = "v${finalAttrs.version}";
hash = "sha256-Dy0dd07pWsSbrO6BX7GEYf7CunXD0itaeIFRv9mQJks=";
};
pnpmDeps = fetchPnpmDeps {
inherit (finalAttrs) pname version src;
fetcherVersion = 2;
hash = "sha256-hxgzmJD+Sl7E+ape1M1/Xl8XLtAhtht3AE45zHFctsQ=";
};
nativeBuildInputs = [
nodejs
pnpmConfigHook
pnpm
];
buildPhase = ''
runHook preBuild
node --run build
runHook postBuild
'';
installPhase = ''
runHook preInstall
cp ./bin/kilo-code-$version.vsix $out
runHook postInstall
'';
});
in
vscode-utils.buildVscodeExtension (finalAttrs: {
pname = "kilo-code";
inherit (finalAttrs.src) version;
vscodeExtPublisher = "kilocode";
vscodeExtName = "Kilo-Code";
vscodeExtUniqueId = "${finalAttrs.vscodeExtPublisher}.${finalAttrs.vscodeExtName}";
src = vsix;
passthru = {
vsix = finalAttrs.src;
updateScript = nix-update-script {
attrPath = "vscode-extensions.kilocode.kilo-kode.vsix";
};
vscode-utils.buildVscodeMarketplaceExtension {
mktplcRef = {
publisher = "kilocode";
name = "Kilo-Code";
version = "7.0.51";
hash = "sha256-1NzwFTFM1gkTcrAVbP6wNctePMAGdy2T9UDn24xhixM=";
};
passthru.updateScript = vscode-extension-update-script { };
meta = {
description = "Open Source AI coding assistant for planning, building, and fixing code";
homepage = "https://kilocode.ai";
homepage = "https://kilo.ai";
downloadPage = "https://marketplace.visualstudio.com/items?itemName=kilocode.Kilo-Code";
license = lib.licenses.asl20;
license = lib.licenses.mit;
sourceProvenance = with lib.sourceTypes; [ fromSource ];
maintainers = with lib.maintainers; [ xiaoxiangmoe ];
};
})
}
pkgs/applications/editors/vscode/extensions/ms-azuretools.vscode-bicep/default.nix
+2 -2
View File
@@ -10,8 +10,8 @@ vscode-utils.buildVscodeMarketplaceExtension {
mktplcRef = {
publisher = "ms-azuretools";
name = "vscode-bicep";
version = "0.38.33";
hash = "sha256-gmSUPHdbxXu5jUASsbu+yVO2ZdVBo5+uQNeLdTsvQVU=";
version = "0.41.2";
hash = "sha256-8k2de208t/ZAVJzxkjd0qcqgVx523hEWWe5d1uvthFU=";
};
buildInputs = [
pkgs/applications/editors/vscode/extensions/reditorsupport.r/default.nix
+2 -2
View File
@@ -14,8 +14,8 @@ vscode-utils.buildVscodeMarketplaceExtension {
mktplcRef = {
name = "r";
publisher = "reditorsupport";
version = "2.8.6";
hash = "sha256-T/Qh0WfTfXMzPonbg9NMII5qFptfNoApFFiZCT5rR3Y=";
version = "2.8.7";
hash = "sha256-pA3/81UYrieDfGYn1fVI6KY9B7A5KAhGIzftZtzXQVc=";
};
nativeBuildInputs = [
jq
pkgs/applications/editors/vscode/extensions/wgsl-analyzer.wgsl-analyzer/default.nix
+2 -2
View File
@@ -10,8 +10,8 @@ vscode-utils.buildVscodeMarketplaceExtension {
mktplcRef = {
name = "wgsl-analyzer";
publisher = "wgsl-analyzer";
version = "0.11.141";
hash = "sha256-egd9B5mS5pqzDWVry3dEQKfnxT4zI0RdLMJ/x5n6Nek=";
version = "0.11.262";
hash = "sha256-a2TVwTmxP9wBt0tMkQcVCyzM0RoihGag56ITd+xjtl8=";
};
nativeBuildInputs = [
pkgs/applications/emulators/libretro/cores/beetle-psx.nix
+3 -3
View File
@@ -8,13 +8,13 @@
}:
mkLibretroCore {
core = "mednafen-psx" + lib.optionalString withHw "-hw";
version = "0-unstable-2026-03-06";
version = "0-unstable-2026-03-20";
src = fetchFromGitHub {
owner = "libretro";
repo = "beetle-psx-libretro";
rev = "3ea167a60bc37bd0c257592bb9a7f559a50465c4";
hash = "sha256-gSNsTV1w7i6bTLngU/Zbo7bwLmb+Bu26JwUre6Rj4qc=";
rev = "8b5dea564b687dd07d91cea48615fab213d1e2c9";
hash = "sha256-vIEjwhE86TsXbAPk6IbMlWWFWE2Nn6R/+gBe9mMmio4=";
};
extraBuildInputs = lib.optionals withHw [
pkgs/applications/emulators/libretro/cores/fbneo.nix
+3 -3
View File
@@ -5,13 +5,13 @@
}:
mkLibretroCore {
core = "fbneo";
version = "0-unstable-2026-03-08";
version = "0-unstable-2026-03-17";
src = fetchFromGitHub {
owner = "libretro";
repo = "fbneo";
rev = "14ff80a2e0611d039321a3ac0dd76bf6b4e3210f";
hash = "sha256-L6KYyEb95L9rDnaMVh49afaWxsshTy3eujsTQWbPfl0=";
rev = "baafb100b487f2ac06f9e78ac322e3ecf36b8924";
hash = "sha256-46hVbQN8QO1FNm56wJ7Q323blUWV9sn529tMwdAOhW8=";
};
makefile = "Makefile";
pkgs/applications/emulators/wine/base.nix
+1 -1
View File
@@ -174,7 +174,7 @@ stdenv.mkDerivation (
++ lib.optional cupsSupport pkgs.cups
++ lib.optional dbusSupport pkgs.dbus
++ lib.optional cairoSupport pkgs.cairo
++ lib.optional odbcSupport pkgs.unixODBC
++ lib.optional odbcSupport pkgs.unixodbc
++ lib.optional netapiSupport pkgs.samba4
++ lib.optional cursesSupport pkgs.ncurses
++ lib.optional vaSupport pkgs.libva
pkgs/applications/networking/browsers/chromium/common.nix
+4
View File
@@ -758,6 +758,10 @@ let
${ungoogler}/utils/domain_substitution.py apply -r ${ungoogler}/domain_regex.list -f ${ungoogler}/domain_substitution.list -c ./ungoogled-domsubcache.tar.gz .
'';
# Sadly, Chromium is not even -fstrict-flex-array=1 clean
# See https://github.com/NixOS/nixpkgs/issues/499982#issuecomment-4062355720
hardeningDisable = [ "strictflexarrays1" ];
llvmCcAndBintools = symlinkJoin {
name = "llvmCcAndBintools";
paths = [
pkgs/applications/networking/cluster/nomad/default.nix
+2 -3
View File
@@ -1,7 +1,6 @@
{
lib,
buildGoModule,
buildGo124Module,
buildGo125Module,
fetchFromGitHub,
nixosTests,
@@ -96,7 +95,7 @@ rec {
};
nomad_1_10 = generic {
buildGoModule = buildGo124Module;
buildGoModule = buildGo125Module;
version = "1.10.5";
hash = "sha256-NFH++oYWb6vQN6cOPByscI/ZBWDNy4YbcLiBMO3/jVU=";
vendorHash = "sha256-QcTw9kKwoHIvXZoxfDohFG+sBs8OLvYPeygygDClsn8=";
@@ -108,7 +107,7 @@ rec {
};
nomad_1_9 = generic {
buildGoModule = buildGo124Module;
buildGoModule = buildGo125Module;
version = "1.9.7";
hash = "sha256-U02H6DPr1friQ9EwqD/wQnE2Fm20OE5xNccPDJfnsqI=";
vendorHash = "sha256-9GnwqkexJAxrhW9yJFaDTdSaZ+p+/dcMuhlusp4cmyw=";
pkgs/applications/networking/cluster/terraform-providers/providers.json
+2 -2
View File
@@ -851,11 +851,11 @@
"vendorHash": "sha256-HzBj7TSjiv/Ov10KOt5V4oQaKK7CKjBjxpSrhmOl1lQ="
},
"loafoe_htpasswd": {
"hash": "sha256-1o2kgeTFxegzOgGXWP4OYZ3uC3WbAkCXPqScMvVpHr0=",
"hash": "sha256-1HCvAGWsYlcYCA8iOmBb/AawxHPLuoxxQWLzNy0x79M=",
"homepage": "https://registry.terraform.io/providers/loafoe/htpasswd",
"owner": "loafoe",
"repo": "terraform-provider-htpasswd",
"rev": "v2.0.0",
"rev": "v2.1.0",
"spdx": "MIT",
"vendorHash": "sha256-1gzJdcSvFPE8Spkwxe24U7DUyrz4qpDoFVHm6vCWF6A="
},
pkgs/applications/networking/instant-messengers/telegram/telegram-desktop/cstring-includes.patch
+74
View File
@@ -0,0 +1,74 @@
diff --git a/src/api/video/nv12_buffer.cc b/src/api/video/nv12_buffer.cc
index ca9dcd867..89d28f23c 100644
--- a/src/api/video/nv12_buffer.cc
+++ b/src/api/video/nv12_buffer.cc
@@ -16,6 +16,8 @@
#include "third_party/libyuv/include/libyuv/convert.h"
#include "third_party/libyuv/include/libyuv/scale.h"
+#include <cstring>
+
namespace webrtc {
namespace {
diff --git a/src/audio/utility/channel_mixer.cc b/src/audio/utility/channel_mixer.cc
index 0f1e66387..33b771b0c 100644
--- a/src/audio/utility/channel_mixer.cc
+++ b/src/audio/utility/channel_mixer.cc
@@ -15,6 +15,8 @@
#include "rtc_base/logging.h"
#include "rtc_base/numerics/safe_conversions.h"
+#include <cstring>
+
namespace webrtc {
ChannelMixer::ChannelMixer(ChannelLayout input_layout,
diff --git a/src/modules/audio_processing/aec3/alignment_mixer.cc b/src/modules/audio_processing/aec3/alignment_mixer.cc
index 7f076dea8..ffd7242b5 100644
--- a/src/modules/audio_processing/aec3/alignment_mixer.cc
+++ b/src/modules/audio_processing/aec3/alignment_mixer.cc
@@ -10,6 +10,7 @@
#include "modules/audio_processing/aec3/alignment_mixer.h"
#include <algorithm>
+#include <cstring>
#include "rtc_base/checks.h"
diff --git a/src/modules/desktop_capture/linux/wayland/shared_screencast_stream.cc b/src/modules/desktop_capture/linux/wayland/shared_screencast_stream.cc
index 7ef1a030e..5b9ab7137 100644
--- a/src/modules/desktop_capture/linux/wayland/shared_screencast_stream.cc
+++ b/src/modules/desktop_capture/linux/wayland/shared_screencast_stream.cc
@@ -18,6 +18,7 @@
#include <spa/param/video/format-utils.h>
#include <sys/mman.h>
+#include <cstring>
#include <vector>
#include "absl/memory/memory.h"
diff --git a/src/modules/video_coding/utility/ivf_file_reader.cc b/src/modules/video_coding/utility/ivf_file_reader.cc
index 4c08ca613..f82f2bfcb 100644
--- a/src/modules/video_coding/utility/ivf_file_reader.cc
+++ b/src/modules/video_coding/utility/ivf_file_reader.cc
@@ -10,6 +10,7 @@
#include "modules/video_coding/utility/ivf_file_reader.h"
+#include <cstring>
#include <string>
#include <vector>
diff --git a/src/net/dcsctp/packet/bounded_byte_writer.h b/src/net/dcsctp/packet/bounded_byte_writer.h
index d754549e4..bf5e3ed42 100644
--- a/src/net/dcsctp/packet/bounded_byte_writer.h
+++ b/src/net/dcsctp/packet/bounded_byte_writer.h
@@ -12,6 +12,7 @@
#define NET_DCSCTP_PACKET_BOUNDED_BYTE_WRITER_H_
#include <algorithm>
+#include <cstring>
#include "api/array_view.h"
pkgs/applications/networking/instant-messengers/telegram/telegram-desktop/tg_owt.nix
+3
View File
@@ -47,6 +47,9 @@ stdenv.mkDerivation {
# fix build with abseil 202508
# upstream PR: https://github.com/desktop-app/tg_owt/pull/164
./abseil-202508.patch
# fix build with latest glibc
# upstream PR: https://github.com/desktop-app/tg_owt/pull/172
./cstring-includes.patch
];
postPatch = lib.optionalString stdenv.hostPlatform.isLinux ''
pkgs/applications/networking/instant-messengers/telegram/telegram-desktop/unwrapped.nix
+2 -2
View File
@@ -19,7 +19,7 @@
ffmpeg_6,
protobuf,
openal-soft,
minizip,
minizip-ng,
range-v3,
tl-expected,
hunspell,
@@ -74,7 +74,7 @@ stdenv.mkDerivation (finalAttrs: {
xxHash
ffmpeg_6
openal-soft
minizip
minizip-ng
range-v3
tl-expected
rnnoise
pkgs/applications/office/libreoffice/default.nix
+2 -2
View File
@@ -77,7 +77,7 @@
clucene-core_2,
libcdr,
lcms2,
unixODBC,
unixodbc,
sane-backends,
mythes,
libexttextcat,
@@ -481,7 +481,7 @@ stdenv.mkDerivation (finalAttrs: {
libpq
python3
sane-backends
unixODBC
unixodbc
util-linux
which
xmlsec
pkgs/applications/science/math/cplex/default.nix
+2 -2
View File
@@ -13,7 +13,7 @@
libsecret,
openjdk,
sqlite,
unixODBC,
unixodbc,
gtk2,
libxtst,
glibcLocales,
@@ -57,7 +57,7 @@ stdenv.mkDerivation rec {
gsettings-desktop-schemas
gtk2
sqlite
unixODBC
unixodbc
libxtst
glibcLocales
];
pkgs/applications/science/math/wolfram-engine/default.nix
+2 -2
View File
@@ -14,7 +14,7 @@
ncurses,
opencv4,
openssl,
unixODBC,
unixodbc,
xkeyboard_config,
libxtst,
libxrender,
@@ -69,7 +69,7 @@ stdenv.mkDerivation rec {
opencv4
openssl
(lib.getLib stdenv.cc.cc)
unixODBC
unixodbc
xkeyboard_config
libxml2
libuuid
pkgs/applications/video/kodi/addons/trakt/default.nix
+2 -2
View File
@@ -10,11 +10,11 @@
buildKodiAddon rec {
pname = "trakt";
namespace = "script.trakt";
version = "3.6.1";
version = "3.8.2";
src = fetchzip {
url = "https://mirrors.kodi.tv/addons/${lib.toLower rel}/${namespace}/${namespace}-${version}.zip";
sha256 = "sha256-ZlBucYYRA1cL5c0H1jhXeKE1itReZe2gAJYFFxuUebo=";
sha256 = "sha256-75neHPVWpHhzMIOfNFvvX/Xqy3n1DO3SGg16zv/r9dU=";
};
propagatedBuildInputs = [
pkgs/build-support/bintools-wrapper/default.nix
+1 -1
View File
@@ -49,12 +49,12 @@
"format"
"fortify"
"fortify3"
"libcxxhardeningextensive"
"libcxxhardeningfast"
"pic"
"relro"
"stackclashprotection"
"stackprotector"
"strictflexarrays1"
"strictoverflow"
"zerocallusedregs"
],
pkgs/build-support/buildenv/builder.pl
Executable → Regular
View File
pkgs/build-support/fetchdarcs/builder.sh
+1 -1
View File
@@ -20,7 +20,7 @@ set -o noglob
success=
for repository in $repositories; do
echo "Trying to clone $repository $tagtext into $out …"
if darcs clone --lazy $tagflags "$repository" "$out"; then
if darcs clone --lazy --no-cache $tagflags "$repository" "$out"; then
# remove metadata, because it can change
rm -rf "$out/_darcs"
success=1
pkgs/build-support/fetchdarcs/nix-prefetch-darcs
+2 -6
View File
@@ -27,7 +27,7 @@ usage() {
while [ $# -gt 0 ]; do
case "$1" in
--quiet)
quiet=1; shift 1 ;;
quiet=1; shift 1 ;;
--name)
name="$2"; shift 2 ;;
--repo)
@@ -127,11 +127,7 @@ if [ -z "$final_path" ]; then
cd "$tmp_clone"
# Do not print Darcs progress to stdout (else stdout isn’t parsable JSON)
if [ -t 1 ]; then
darcs clone $clone_args "$repository" "$name" >/dev/tty
else
darcs clone $clone_args "$repository" "$name" >/dev/null
fi
darcs clone "$clone_args" "$repository" "$name" 1>&2
# Will put the current Darcs context into the store.
new_context="$tmp_clone/${name}-context.txt"
darcs log --repodir="$tmp_clone/$name" --context > "$new_context"
pkgs/build-support/node/prefetch-npm-deps/src/parse/lock.rs
+51 -12
View File
@@ -11,6 +11,36 @@ use std::{
};
use url::Url;
fn warn_unresolved_packages(
packages_without_resolved: &[(String, Package)],
total_count: usize,
) {
let missing_count = packages_without_resolved.len();
if missing_count == 0 {
return;
}
let percentage = (missing_count as f64 / total_count as f64) * 100.0;
eprintln!(
"warning: {} out of {} packages ({:.1}%) are missing 'resolved' URLs and will not be cached.",
missing_count, total_count, percentage
);
eprintln!("warning: Packages without 'resolved' URLs:");
for (name, _) in packages_without_resolved.iter().take(10) {
eprintln!("warning: - {}", name.trim_start_matches("node_modules/"));
}
if missing_count > 10 {
eprintln!("warning: ... and {} more", missing_count - 10);
}
if percentage > 50.0 {
eprintln!(
"warning: More than 50% of packages are missing 'resolved' URLs. This may indicate an issue with the lockfile.\n\
warning: This is a known issue with some npm versions. See: https://github.com/npm/cli/issues/6301\n\
warning: Consider regenerating upstream's lockfile with: npm install --package-lock-only (sending an upstream PR is best)"
);
}
}
pub(super) fn packages(content: &str) -> anyhow::Result<Vec<Package>> {
let lockfile: Lockfile = serde_json::from_str(content)?;
@@ -20,18 +50,27 @@ pub(super) fn packages(content: &str) -> anyhow::Result<Vec<Package>> {
to_new_packages(lockfile.dependencies.unwrap_or_default(), &initial_url)?
}
2 | 3 => lockfile
.packages
.unwrap_or_default()
.into_iter()
.filter(|(n, p)| !n.is_empty() && matches!(p.resolved, Some(UrlOrString::Url(_))))
.map(|(n, p)| Package {
// Use the package's own name if present (for aliases like string-width-cjs -> string-width),
// otherwise extract from the lockfile key
name: Some(p.name.unwrap_or(n)),
..p
})
.collect(),
2 | 3 => {
let (packages_with_resolved, packages_without_resolved): (Vec<_>, Vec<_>) = lockfile
.packages
.unwrap_or_default()
.into_iter()
.filter(|(n, _)| !n.is_empty())
.partition(|(_, p)| matches!(p.resolved, Some(UrlOrString::Url(_))));
let total_count = packages_with_resolved.len() + packages_without_resolved.len();
warn_unresolved_packages(&packages_without_resolved, total_count);
packages_with_resolved
.into_iter()
.map(|(n, p)| Package {
// Use the package's own name if present (for aliases like string-width-cjs -> string-width),
// otherwise extract from the lockfile key
name: Some(p.name.unwrap_or(n)),
..p
})
.collect()
}
_ => bail!(
"We don't support lockfile version {}, please file an issue.",
lockfile.version
pkgs/build-support/rust/fetch-cargo-vendor-util.py
+101 -36
View File
@@ -13,6 +13,7 @@ from typing import Any, TypedDict, cast
from urllib.parse import unquote
import requests
import tomli_w
from requests.adapters import HTTPAdapter, Retry
eprint = functools.partial(print, file=sys.stderr)
@@ -172,8 +173,12 @@ def get_manifest_metadata(manifest_path: Path) -> dict[str, Any]:
return json.loads(output)
def try_get_crate_manifest_path_from_mainfest_path(manifest_path: Path, crate_name: str) -> Path | None:
metadata = get_manifest_metadata(manifest_path)
def try_get_crate_manifest_path_from_manifest_path(manifest_path: Path, crate_name: str) -> Path | None:
try:
metadata = get_manifest_metadata(manifest_path)
except subprocess.CalledProcessError:
eprint(f"Warning: cargo metadata failed for {manifest_path}, skipping")
return None
for pkg in metadata["packages"]:
if pkg["name"] == crate_name:
@@ -183,11 +188,15 @@ def try_get_crate_manifest_path_from_mainfest_path(manifest_path: Path, crate_na
def find_crate_manifest_in_tree(tree: Path, crate_name: str) -> Path:
# in some cases Cargo.toml is not located at the top level, so we also look at subdirectories
manifest_paths = tree.glob("**/Cargo.toml")
# Scan all Cargo.toml files; sort by depth/path to make ordering deterministic
# and prefer less-nested manifests first.
manifest_paths = sorted(
tree.glob("**/Cargo.toml"),
key=lambda path: (len(path.parts), str(path)),
)
for manifest_path in manifest_paths:
res = try_get_crate_manifest_path_from_mainfest_path(manifest_path, crate_name)
res = try_get_crate_manifest_path_from_manifest_path(manifest_path, crate_name)
if res is not None:
return res
@@ -256,6 +265,21 @@ def extract_crate_tarball_contents(tarball_path: Path, crate_out_dir: Path) -> N
subprocess.check_output(cmd)
def make_git_source_selector(source_info: GitSourceInfo) -> dict[str, str]:
selector = {}
selector["git"] = source_info["url"]
if source_info["type"] is not None:
selector[source_info["type"]] = source_info["value"]
return selector
def make_registry_source_selector(source: str) -> dict[str, str]:
registry = source[9:] if source.startswith("registry+") else source
selector = {}
selector["registry"] = registry
return selector
def create_vendor(vendor_staging_dir: Path, out_dir: Path) -> None:
lockfile_path = vendor_staging_dir / "Cargo.lock"
out_dir.mkdir(exist_ok=True)
@@ -264,14 +288,67 @@ def create_vendor(vendor_staging_dir: Path, out_dir: Path) -> None:
cargo_lock_toml = load_toml(lockfile_path)
lockfile_version = get_lockfile_version(cargo_lock_toml)
config_lines = [
'[source.vendored-sources]',
'directory = "@vendor@"',
'[source.crates-io]',
'replace-with = "vendored-sources"',
]
source_to_ind: dict[str, str] = {}
source_config = {}
next_registry_ind = 0
next_git_ind = 0
def add_source_replacement(
orig_key: str,
orig_selector: dict[str, str],
vendored_key: str,
vendored_dir: str
) -> None:
source_config[vendored_key] = {}
source_config[vendored_key]["directory"] = vendored_dir
source_config[orig_key] = orig_selector
source_config[orig_key]["replace-with"] = vendored_key
# we reserve registry index 0 for crates-io
source_to_ind["registry+https://github.com/rust-lang/crates.io-index"] = "registry-0"
source_to_ind["sparse+https://index.crates.io/"] = "registry-0"
add_source_replacement(
orig_key="crates-io",
orig_selector={}, # there is an internal selector defined for the `crates-io` source
vendored_key="vendored-source-registry-0",
vendored_dir="@vendor@/source-registry-0"
)
next_registry_ind += 1
for pkg in cargo_lock_toml["package"]:
# ignore local dependencies
if "source" not in pkg.keys():
continue
source: str = pkg["source"]
if source in source_to_ind:
continue
if source.startswith("git+"):
ind = f"git-{next_git_ind}"
next_git_ind += 1
source_info = parse_git_source(source, lockfile_version)
selector = make_git_source_selector(source_info)
elif source.startswith("registry+") or source.startswith("sparse+"):
ind = f"registry-{next_registry_ind}"
next_registry_ind += 1
selector = make_registry_source_selector(source)
else:
raise Exception(f"Can't process source: {source}.")
source_to_ind[source] = ind
add_source_replacement(
orig_key=f"original-source-{ind}",
orig_selector=selector,
vendored_key=f"vendored-source-{ind}",
vendored_dir=f"@vendor@/source-{ind}"
)
config_path = out_dir / ".cargo" / "config.toml"
config_path.parent.mkdir()
with open(config_path, "wb") as config_file:
tomli_w.dump({"source": source_config}, config_file)
seen_source_keys = set()
for pkg in cargo_lock_toml["package"]:
# ignore local dependenices
@@ -279,13 +356,15 @@ def create_vendor(vendor_staging_dir: Path, out_dir: Path) -> None:
continue
source: str = pkg["source"]
dir_name = f"{pkg["name"]}-{pkg["version"]}"
crate_out_dir = out_dir / dir_name
source_ind = source_to_ind[source]
crate_dir_name = f"{pkg["name"]}-{pkg["version"]}"
source_dir_name = f"source-{source_ind}"
crate_out_dir = out_dir / source_dir_name / crate_dir_name
crate_out_dir.parent.mkdir(exist_ok=True)
if source.startswith("git+"):
source_info = parse_git_source(pkg["source"], lockfile_version)
source_info = parse_git_source(source, lockfile_version)
git_sha_rev = source_info["git_sha_rev"]
git_tree = vendor_staging_dir / "git" / git_sha_rev
@@ -296,23 +375,13 @@ def create_vendor(vendor_staging_dir: Path, out_dir: Path) -> None:
with open(crate_out_dir / ".cargo-checksum.json", "w") as f:
json.dump({"files": {}}, f)
source_key = source[0:source.find("#")]
if source_key in seen_source_keys:
continue
seen_source_keys.add(source_key)
config_lines.append(f'[source."{source_key}"]')
config_lines.append(f'git = "{source_info["url"]}"')
if source_info["type"] is not None:
config_lines.append(f'{source_info["type"]} = "{source_info["value"]}"')
config_lines.append('replace-with = "vendored-sources"')
elif source.startswith("registry+"):
elif source.startswith("registry+") or source.startswith("sparse+"):
filename = f"{pkg["name"]}-{pkg["version"]}.tar.gz"
tarball_path = vendor_staging_dir / "tarballs" / filename
# TODO: change this when non-crates-io registries are supported
dir_name = "tarballs"
tarball_path = vendor_staging_dir / dir_name / filename
extract_crate_tarball_contents(tarball_path, crate_out_dir)
@@ -323,10 +392,6 @@ def create_vendor(vendor_staging_dir: Path, out_dir: Path) -> None:
else:
raise Exception(f"Can't process source: {source}.")
(out_dir / ".cargo").mkdir()
with open(out_dir / ".cargo" / "config.toml", "w") as config_file:
config_file.writelines(line + "\n" for line in config_lines)
def main() -> None:
subcommand = sys.argv[1]
pkgs/build-support/rust/fetch-cargo-vendor.nix
+1
View File
@@ -27,6 +27,7 @@ let
with python3Packages;
[
requests
tomli-w
]
++ requests.optional-dependencies.socks; # to support socks proxy envs like ALL_PROXY in requests
flakeIgnore = [
pkgs/build-support/rust/fetchcargo-default-config.toml
Executable → Regular
View File

Some files were not shown because too many files have changed in this diff Show More