diff --git a/nixos/doc/manual/administration/rollback.section.md b/nixos/doc/manual/administration/rollback.section.md index 290d685a2a18..4e596fde8894 100644 --- a/nixos/doc/manual/administration/rollback.section.md +++ b/nixos/doc/manual/administration/rollback.section.md @@ -12,7 +12,7 @@ system has booted, you can make the selected configuration the default for subsequent boots: ```ShellSession -# /run/current-system/bin/switch-to-configuration boot +# /run/current-system/bin/apply boot ``` Second, you can switch to the previous configuration in a running @@ -25,11 +25,11 @@ system: This is equivalent to running: ```ShellSession -# /nix/var/nix/profiles/system-N-link/bin/switch-to-configuration switch +# /nix/var/nix/profiles/system-N-link/bin/apply switch ``` -where `N` is the number of the NixOS system configuration. To get a -list of the available configurations, do: +where `N` is the number of the NixOS system configuration to roll back to. +To get a list of the available configurations, run: ```ShellSession $ ls -l /nix/var/nix/profiles/system-*-link diff --git a/nixos/doc/manual/development/non-switchable-systems.section.md b/nixos/doc/manual/development/non-switchable-systems.section.md index a51e8233f30b..a8cbcdd865fd 100644 --- a/nixos/doc/manual/development/non-switchable-systems.section.md +++ b/nixos/doc/manual/development/non-switchable-systems.section.md @@ -16,6 +16,6 @@ profile: The most notable deviation of this profile from a standard NixOS configuration is that after building it, you cannot switch *to* the configuration anymore. The profile sets `config.system.switch.enable = false;`, which excludes -`switch-to-configuration`, the central script called by `nixos-rebuild`, from +`apply` and `switch-to-configuration`, the central scripts called by `nixos-rebuild`, from your system. Removing this script makes the image lighter and slightly more secure. diff --git a/nixos/doc/manual/development/what-happens-during-a-system-switch.chapter.md b/nixos/doc/manual/development/what-happens-during-a-system-switch.chapter.md index 28c06f999dac..a3e921569caa 100644 --- a/nixos/doc/manual/development/what-happens-during-a-system-switch.chapter.md +++ b/nixos/doc/manual/development/what-happens-during-a-system-switch.chapter.md @@ -5,8 +5,8 @@ This chapter explains some of the internals of this command to make it simpler for new module developers to configure their units correctly and to make it easier to understand what is happening and why for curious administrators. -`nixos-rebuild`, like many deployment solutions, calls `switch-to-configuration` -which resides in a NixOS system at `$out/bin/switch-to-configuration`. The +`nixos-rebuild`, like many deployment solutions, calls `apply` (or for NixOS older than 24.11, `switch-to-configuration`) +which resides in a NixOS system at `$out/bin/apply`. The script is called with the action that is to be performed like `switch`, `test`, `boot`. There is also the `dry-activate` action which does not really perform the actions but rather prints what it would do if you called it with `test`. diff --git a/nixos/doc/manual/installation/installing-from-other-distro.section.md b/nixos/doc/manual/installation/installing-from-other-distro.section.md index 2d9818e6805c..0b8dbd78c392 100644 --- a/nixos/doc/manual/installation/installing-from-other-distro.section.md +++ b/nixos/doc/manual/installation/installing-from-other-distro.section.md @@ -247,7 +247,7 @@ The first steps to all these are the same: ```ShellSession $ sudo mv -v /boot /boot.bak && - sudo /nix/var/nix/profiles/system/bin/switch-to-configuration boot + sudo /nix/var/nix/profiles/system/bin/apply boot ``` Cross your fingers, reboot, hopefully you should get a NixOS prompt! diff --git a/nixos/doc/manual/release-notes/rl-2411.section.md b/nixos/doc/manual/release-notes/rl-2411.section.md index a0600247bcd1..8123f2ca52e9 100644 --- a/nixos/doc/manual/release-notes/rl-2411.section.md +++ b/nixos/doc/manual/release-notes/rl-2411.section.md @@ -50,6 +50,16 @@ If you experience any issues, please report them. The original Perl script is deprecated and is planned for removal in the 25.05 release. It will remain accessible until then by setting `system.switch.enableNg` to `false`. +- Built NixOS configurations now have a `$toplevel/bin/apply` script. + Unlike `switch-to-configuration`, it is capable of performing a complete `switch` operation. + If you call `switch-to-configuration` directly, you are recommended to use `apply` instead, and remove your call to `nix-env --profile /nix/var/nix/profiles/system --set $toplevel` or similar. + It will run the switch operation as a systemd unit if available, as `nixos-rebuild switch` would. + + Benefits include: + - The `apply` script reduces the roundtrips required when performing a remote deployment with `nixos-rebuild switch --target-host HOST`. + - Developers and power users can now update NixOS in a single call. + - Alternative NixOS deployment methods have feature parity with `nixos-rebuild`, and NixOS can evolve all of its switching logic in one place. + - Support for mounting filesystems from block devices protected with [dm-verity](https://docs.kernel.org/admin-guide/device-mapper/verity.html) was added through the `boot.initrd.systemd.dmVerity` option. diff --git a/nixos/lib/testing/nixos-test-base.nix b/nixos/lib/testing/nixos-test-base.nix index c8b15a0ecce4..3793fd5b9819 100644 --- a/nixos/lib/testing/nixos-test-base.nix +++ b/nixos/lib/testing/nixos-test-base.nix @@ -23,7 +23,7 @@ in }; } ({ config, ... }: { - # Don't pull in switch-to-configuration by default, except when specialisations or early boot shenanigans are involved. + # Don't pull in apply and switch-to-configuration by default, except when specialisations or early boot shenanigans are involved. # This is mostly a Hydra optimization, so we don't rebuild all the tests every time switch-to-configuration-ng changes. key = "no-switch-to-configuration"; system.switch.enable = mkDefault (config.isSpecialisation || config.specialisation != {} || config.virtualisation.installBootLoader); diff --git a/nixos/modules/system/activation/apply/apply.sh b/nixos/modules/system/activation/apply/apply.sh new file mode 100644 index 000000000000..c494b76b7c29 --- /dev/null +++ b/nixos/modules/system/activation/apply/apply.sh @@ -0,0 +1,163 @@ +#!@bash@ + + +# This is the NixOS apply script, typically located at +# +# ${config.system.build.toplevel}/bin/apply +# +# This script is responsible for managing the profile link and calling the +# appropriate scripts for its subcommands, such as switch, boot, and test. + + +set -euo pipefail + +toplevel=@toplevel@ + +subcommand= + +installBootloader= +specialisation= +profile=/nix/var/nix/profiles/system + +log() { + echo "$@" >&2 +} + +die() { + log "NixOS apply error: $*" + exit 1 +} + +usage() { + log "NixOS apply invocation error: $*" +cat >&2 </dev/null; then + # We're not loaded into the test.sh, so we run main. + parse_args "$@" + main +fi diff --git a/nixos/modules/system/activation/apply/checks.nix b/nixos/modules/system/activation/apply/checks.nix new file mode 100644 index 000000000000..bb509b9210cb --- /dev/null +++ b/nixos/modules/system/activation/apply/checks.nix @@ -0,0 +1,51 @@ +# Run: +# nix-build -A nixosTests.apply +# +# These are not all tests. See also nixosTests. + +{ + lib, + stdenvNoCC, + testers, + ... +}: + +let + fileset = lib.fileset.unions [ + ./test.sh + ./apply.sh + ]; +in + +{ + unitTests = stdenvNoCC.mkDerivation { + name = "nixos-apply-unit-tests"; + src = lib.fileset.toSource { + root = ./.; + inherit fileset; + }; + dontBuild = true; + checkPhase = '' + ./test.sh + ''; + installPhase = '' + touch $out + ''; + }; + + shellcheck = + (testers.shellcheck { + src = lib.fileset.toSource { + # This makes the error messages include the full path + root = ../../../../..; + inherit fileset; + }; + }).overrideAttrs + { + postUnpack = '' + for f in $(find . -type f); do + substituteInPlace $f --replace @bash@ /usr/bin/bash + done + ''; + }; +} diff --git a/nixos/modules/system/activation/apply/test.sh b/nixos/modules/system/activation/apply/test.sh new file mode 100755 index 000000000000..02ab01f69bea --- /dev/null +++ b/nixos/modules/system/activation/apply/test.sh @@ -0,0 +1,176 @@ +#!/usr/bin/env bash +# shellcheck disable=SC2317 disable=SC2031 +# False positives: +# SC2317: Unreachable code: TEST_* +# SC2031: was modified in a subshell. That change might be lost. +# We have a lot of that, and that's expected. + +# This is a unit test script for the NixOS apply script. +# It can be run quickly with the following command: +# +# ./test.sh +# +# Alternatively, run the following to run all tests and checks +# +# TODO +# + +set -euo pipefail +# set -x + +apply="${BASH_SOURCE[0]%/*}/apply.sh" +# source_apply() { + +run_parse_args() { + bash -c "source $apply;"' parse_args "$@"' -- "$@" +} + +TEST_parse_args_none() { + if errout="$(run_parse_args 2>&1)"; then + test_fail "apply without arguments should fail" + elif [[ $? -ne 1 ]]; then + test_fail "apply without arguments should exit with code 1" + fi + grep -F "no subcommand specified" <<<"$errout" >/dev/null +} + +TEST_parse_args_switch() { + ( + # shellcheck source=nixos/modules/system/activation/apply/apply.sh + source "$apply"; + parse_args switch; + [[ $subcommand == switch ]] + [[ $specialisation == "" ]] + [[ $profile == "" ]] + ) +} + +TEST_parse_args_boot() { + ( + # shellcheck source=nixos/modules/system/activation/apply/apply.sh + source "$apply"; + parse_args boot; + [[ $subcommand == boot ]] + [[ $specialisation == "" ]] + [[ $profile == "" ]] + ) +} + +TEST_parse_args_test() { + ( + # shellcheck source=nixos/modules/system/activation/apply/apply.sh + source "$apply"; + parse_args test; + [[ $subcommand == test ]] + [[ $specialisation == "" ]] + [[ $profile == "" ]] + ) +} + +TEST_parse_args_dry_activate() { + ( + # shellcheck source=nixos/modules/system/activation/apply/apply.sh + source "$apply"; + parse_args dry-activate; + [[ $subcommand == dry-activate ]] + [[ $specialisation == "" ]] + [[ $profile == "" ]] + ) +} + +TEST_parse_args_unknown() { + if errout="$(run_parse_args foo 2>&1)"; then + test_fail "apply with unknown subcommand should fail" + fi + grep -F "unexpected argument or flag: foo" <<<"$errout" >/dev/null +} + +TEST_parse_args_switch_specialisation_no_arg() { + if errout="$(run_parse_args switch --specialisation 2>&1)"; then + test_fail "apply with --specialisation without argument should fail" + fi + grep -F "missing argument for --specialisation" <<<"$errout" >/dev/null +} + +TEST_parse_args_switch_specialisation() { + ( + # shellcheck source=nixos/modules/system/activation/apply/apply.sh + source "$apply"; + parse_args switch --specialisation low-power; + [[ $subcommand == switch ]] + [[ $specialisation == low-power ]] + [[ $profile == "" ]] + ) +} + +TEST_parse_args_switch_profile() { + ( + # shellcheck source=nixos/modules/system/activation/apply/apply.sh + source "$apply"; + parse_args switch --profile /nix/var/nix/profiles/system; + [[ $subcommand == switch ]] + [[ $specialisation == "" ]] + [[ $profile == /nix/var/nix/profiles/system ]] + ) +} + + + +# Support code + +test_fail() { + echo "TEST FAILURE: $*" >&2 + exit 1 +} + +test_print_trace() { + local frame=${1:0} + local caller + # shellcheck disable=SC2207 disable=SC2086 + while caller=( $(caller $frame) ); do + echo " in ${caller[1]} at ${caller[2]}:${caller[0]}" + frame=$((frame+1)); + done +} +test_on_err() { + echo "ERROR running: ${BASH_COMMAND}" >&2 + test_print_trace 1 >&2 +} + +test_init() { + trap 'test_on_err' ERR +} + +test_find() { + declare -F | grep -o 'TEST_.*' | sort +} + +test_run_tests() { + local status=0 + for test in $(test_find); do + set +e + ( + set -eEuo pipefail + trap 'test_on_err' ERR + $test + ) + r=$? + set -e + if [[ $r == 0 ]]; then + echo "ok: $test" + else + echo "TEST FAIL: $test"; status=1; + fi + done + if [[ $status == 0 ]]; then + echo "All good" + else + echo + echo "TEST SUITE FAILED" + fi + exit $status +} + +# Main +test_init +test_run_tests diff --git a/nixos/modules/system/activation/specialisation.nix b/nixos/modules/system/activation/specialisation.nix index 41491b67ff03..2b1fed44bbed 100644 --- a/nixos/modules/system/activation/specialisation.nix +++ b/nixos/modules/system/activation/specialisation.nix @@ -42,7 +42,7 @@ in (e.g. `fewJobsManyCores`) at runtime, run: ``` - sudo /run/current-system/specialisation/fewJobsManyCores/bin/switch-to-configuration test + sudo /run/current-system/specialisation/fewJobsManyCores/bin/apply test ``` ''; type = types.attrsOf (types.submodule ( diff --git a/nixos/modules/system/activation/switch-to-configuration.pl b/nixos/modules/system/activation/switch-to-configuration.pl index 4beca4f0a42a..622e25bc10e4 100755 --- a/nixos/modules/system/activation/switch-to-configuration.pl +++ b/nixos/modules/system/activation/switch-to-configuration.pl @@ -80,12 +80,9 @@ if ("@localeArchive@" ne "") { if (!defined($action) || ($action ne "switch" && $action ne "boot" && $action ne "test" && $action ne "dry-activate")) { print STDERR <<"EOF"; +error: Unknown action $action Usage: $0 [switch|boot|test|dry-activate] - -switch: make the configuration the boot default and activate now -boot: make the configuration the boot default -test: activate the configuration, but don\'t make it the boot default -dry-activate: show what would be done if this configuration were activated +Consider calling `apply` instead of `switch-to-configuration`. EOF exit(1); } diff --git a/nixos/modules/system/activation/switchable-system.nix b/nixos/modules/system/activation/switchable-system.nix index d1326a18e5fe..bb3f7a7d4a78 100644 --- a/nixos/modules/system/activation/switchable-system.nix +++ b/nixos/modules/system/activation/switchable-system.nix @@ -40,7 +40,30 @@ in }; }; + options.system.apply.enable = lib.mkOption { + type = lib.types.bool; + default = config.system.switch.enable; + internal = true; + description = '' + Whether to include the `bin/apply` script. + + Disabling puts `nixos-rebuild` in a legacy mode that won't be maintained + and removes cheap and useful functionality. It's also slower over ssh. + This should only be used for testing the `nixos-rebuild` command, to + pretend that the configuration is an old NixOS. + ''; + }; + config = lib.mkMerge [ + (lib.mkIf config.system.apply.enable { + system.activatableSystemBuilderCommands = '' + mkdir -p $out/bin + substitute ${./apply/apply.sh} $out/bin/apply \ + --subst-var-by bash ${lib.getExe pkgs.bash} \ + --subst-var-by toplevel ''${!toplevelVar} + chmod +x $out/bin/apply + ''; + }) (lib.mkIf (config.system.switch.enable && !config.system.switch.enableNg) { warnings = [ '' @@ -54,7 +77,7 @@ in ]; system.activatableSystemBuilderCommands = '' - mkdir $out/bin + mkdir -p $out/bin substitute ${./switch-to-configuration.pl} $out/bin/switch-to-configuration \ --subst-var out \ --subst-var-by toplevel ''${!toplevelVar} \ @@ -86,7 +109,7 @@ in ( source ${pkgs.buildPackages.makeWrapper}/nix-support/setup-hook - mkdir $out/bin + mkdir -p $out/bin ln -sf ${lib.getExe pkgs.switch-to-configuration-ng} $out/bin/switch-to-configuration wrapProgram $out/bin/switch-to-configuration \ --set OUT $out \ diff --git a/nixos/modules/system/activation/top-level.nix b/nixos/modules/system/activation/top-level.nix index 6abbd4b673c0..4ac7bda0cc2f 100644 --- a/nixos/modules/system/activation/top-level.nix +++ b/nixos/modules/system/activation/top-level.nix @@ -49,8 +49,8 @@ let # Putting it all together. This builds a store path containing # symlinks to the various parts of the built configuration (the # kernel, systemd units, init scripts, etc.) as well as a script - # `switch-to-configuration' that activates the configuration and - # makes it bootable. See `activatable-system.nix`. + # `bin/apply` that activates the configuration and + # makes it bootable. See `activatable-system.nix` and `switchable-system.nix`. baseSystem = pkgs.stdenvNoCC.mkDerivation ({ name = "nixos-system-${config.system.name}-${config.system.nixos.label}"; preferLocalBuild = true; diff --git a/nixos/tests/all-tests.nix b/nixos/tests/all-tests.nix index 8b674bfb7342..c01401b7e044 100644 --- a/nixos/tests/all-tests.nix +++ b/nixos/tests/all-tests.nix @@ -129,6 +129,7 @@ in { apfs = runTest ./apfs.nix; appliance-repart-image = runTest ./appliance-repart-image.nix; appliance-repart-image-verity-store = runTest ./appliance-repart-image-verity-store.nix; + apply = pkgs.callPackage ../modules/system/activation/apply/checks.nix { }; apparmor = handleTest ./apparmor.nix {}; archi = handleTest ./archi.nix {}; aria2 = handleTest ./aria2.nix {}; @@ -703,7 +704,17 @@ in { nixos-generate-config = handleTest ./nixos-generate-config.nix {}; nixos-rebuild-install-bootloader = handleTestOn ["x86_64-linux"] ./nixos-rebuild-install-bootloader.nix {}; nixos-rebuild-specialisations = runTestOn ["x86_64-linux"] ./nixos-rebuild-specialisations.nix; + nixos-rebuild-specialisations-legacy = runTestOn ["x86_64-linux"] { + name = mkForce "nixos-rebuild-specialisations-legacy"; + imports = [ ./nixos-rebuild-specialisations.nix ]; + extraBaseModules = { system.apply.enable = false; }; + }; nixos-rebuild-target-host = runTest ./nixos-rebuild-target-host.nix; + nixos-rebuild-target-host-legacy = runTest { + name = mkForce "nixos-rebuild-target-host-legacy"; + imports = [ ./nixos-rebuild-target-host.nix ]; + extraBaseModules = { system.apply.enable = false; }; + }; nixpkgs = pkgs.callPackage ../modules/misc/nixpkgs/test.nix { inherit evalMinimalConfig; }; nixseparatedebuginfod = handleTest ./nixseparatedebuginfod.nix {}; node-red = handleTest ./node-red.nix {}; diff --git a/pkgs/by-name/sw/switch-to-configuration-ng/src/src/main.rs b/pkgs/by-name/sw/switch-to-configuration-ng/src/src/main.rs index 61932cb55591..7996d1454a65 100644 --- a/pkgs/by-name/sw/switch-to-configuration-ng/src/src/main.rs +++ b/pkgs/by-name/sw/switch-to-configuration-ng/src/src/main.rs @@ -940,10 +940,7 @@ fn do_user_switch(parent_exe: String) -> anyhow::Result<()> { fn usage(argv0: &str) -> ! { eprintln!( r#"Usage: {} [switch|boot|test|dry-activate] -switch: make the configuration the boot default and activate now -boot: make the configuration the boot default -test: activate the configuration, but don't make it the boot default -dry-activate: show what would be done if this configuration were activated +Consider calling `apply` instead of `switch-to-configuration`. "#, argv0 ); diff --git a/pkgs/os-specific/linux/nixos-rebuild/nixos-rebuild.sh b/pkgs/os-specific/linux/nixos-rebuild/nixos-rebuild.sh index 1ab5c8346f33..6236c923c8a9 100755 --- a/pkgs/os-specific/linux/nixos-rebuild/nixos-rebuild.sh +++ b/pkgs/os-specific/linux/nixos-rebuild/nixos-rebuild.sh @@ -220,9 +220,9 @@ buildHostCmd() { if [ -z "$buildHost" ]; then runCmd "$@" elif [ -n "$remoteNix" ]; then - runCmd ssh $SSHOPTS "$buildHost" "${c[@]}" env PATH="$remoteNix":'$PATH' "$@" + runCmd ssh $SSHOPTS "$buildHost" "${c[@]}" env PATH="$remoteNix":'$PATH' "${@@Q}" else - runCmd ssh $SSHOPTS "$buildHost" "${c[@]}" "$@" + runCmd ssh $SSHOPTS "$buildHost" "${c[@]}" "${@@Q}" fi } @@ -237,7 +237,7 @@ targetHostCmd() { if [ -z "$targetHost" ]; then runCmd "${c[@]}" "$@" else - runCmd ssh $SSHOPTS "$targetHost" "${c[@]}" "$@" + runCmd ssh $SSHOPTS "$targetHost" "${c[@]}" "${@@Q}" fi } @@ -790,7 +790,6 @@ if [ -z "$rollback" ]; then pathToConfig="$(nixFlakeBuild "$flake#$flakeAttr.config.system.build.toplevel" "${extraBuildFlags[@]}" "${lockFlags[@]}")" fi copyToTarget "$pathToConfig" - targetHostSudoCmd nix-env -p "$profile" --set "$pathToConfig" elif [[ "$action" = test || "$action" = build || "$action" = dry-build || "$action" = dry-activate ]]; then if [[ -z $buildingAttribute ]]; then pathToConfig="$(nixBuild $buildFile -A "${attr:+$attr.}config.system.build.toplevel" "${extraBuildFlags[@]}")" @@ -841,12 +840,56 @@ else # [ -n "$rollback" ] fi +hasApplyScript= +# If we're doing a deployment-like action, we need to know whether the config has +# an apply script. NixOS versions >= 24.11 should be deployed with toplevel/bin/apply. +if [[ "$action" = switch || "$action" = boot || "$action" = test || "$action" = dry-activate ]]; then + hasApplyScriptOut="$(targetHostCmd sh -c "if test -e $pathToConfig/bin/apply; then echo __has-apply-script__; elif test -e $pathToConfig/bin; then echo __has-no-apply-script__; else echo $pathToConfig is gone; fi + ")" + # SSH can be messy (e.g. when user has a shell rc file that prints to stdout) + # So we only check for the substring + case "$hasApplyScriptOut" in + *__has-apply-script__*) + hasApplyScript=1 + ;; + *__has-no-apply-script__*) + hasApplyScript= + ;; + *) + # Unlikely + echo "$hasApplyScriptOut" 1>&2 + log "error: $pathToConfig could not be read" + exit 1 + ;; + esac +fi + +# switch|boot|test|dry-activate +# # If we're not just building, then make the new configuration the boot # default and/or activate it now. -if [[ "$action" = switch || "$action" = boot || "$action" = test || "$action" = dry-activate ]]; then - # Using systemd-run here to protect against PTY failures/network - # disconnections during rebuild. - # See: https://github.com/NixOS/nixpkgs/issues/39118 +if [[ -n "$hasApplyScript" ]] \ + && [[ "$action" = switch || "$action" = boot || "$action" = test || "$action" = dry-activate ]]; then + cmd=("$pathToConfig/bin/apply" "$action" "--profile" "$profile") + if [[ -n "$specialisation" ]]; then + cmd+=("--specialisation" "$specialisation") + fi + if [[ -n "$installBootloader" ]]; then + cmd+=("--install-bootloader") + fi + targetHostSudoCmd "${cmd[@]}" +elif [[ "$action" = switch || "$action" = boot || "$action" = test || "$action" = dry-activate ]]; then + # Legacy, without apply script, NixOS < 24.11 + + if [[ "$action" = switch || "$action" = boot ]]; then + if [[ -z "$rollback" ]]; then + : # We've already switched it so that hasApplyScript would check the right $pathToConfig + else + targetHostSudoCmd nix-env -p "$profile" --set "$pathToConfig" + fi + fi + + # Legacy logic to support deploying NixOS <24.11; see hasApplyScript cmd=( "systemd-run" "-E" "LOCALE_ARCHIVE" # Will be set to new value early in switch-to-configuration script, but interpreter starts out with old value