From 80e3fd91a92149cb6b463ebe32d5859f89fa6cda Mon Sep 17 00:00:00 2001 From: nicoo Date: Thu, 8 Aug 2024 16:17:13 +0000 Subject: [PATCH 1/5] doc/build-helpers: document `runCommandWith` Co-authored-by: Johannes Kirschbauer --- .../trivial-build-helpers.chapter.md | 72 ++++++++++++++++++- .../trivial-builders/default.nix | 13 +--- 2 files changed, 73 insertions(+), 12 deletions(-) diff --git a/doc/build-helpers/trivial-build-helpers.chapter.md b/doc/build-helpers/trivial-build-helpers.chapter.md index 5d4ede836a1b..0c7c3f899a73 100644 --- a/doc/build-helpers/trivial-build-helpers.chapter.md +++ b/doc/build-helpers/trivial-build-helpers.chapter.md @@ -3,6 +3,76 @@ Nixpkgs provides a variety of wrapper functions that help build commonly useful derivations. Like [`stdenv.mkDerivation`](#sec-using-stdenv), each of these build helpers creates a derivation, but the arguments passed are different (usually simpler) from those required by `stdenv.mkDerivation`. + +## `runCommandWith` {#trivial-builder-runCommandWith} + +The function `runCommandWith` returns a derivation built using the specified command(s), in a specified environment. + +It is the underlying base function of all `runCommand*` variants. +The general behavior is controlled via a single attribute set passed +as the first argument, and allows specifying `stdenv` freely. + +### Type {#trivial-builder-runCommandWith-Type} + +``` +runCommandWith :: { + name :: name; + stdenv? :: Derivation; + runLocal? :: Bool; + derivationArgs? :: { ... }; +} -> String -> Derivation +``` + +### Inputs {#trivial-builder-runCommandWith-Inputs} + +`name` (String) +: The derivation's name, which Nix will append to the store path; see [`mkDerivation`](#sec-using-stdenv). + +`runLocal` (Boolean) +: If set to `true` this forces the derivation to be built locally. Remote substitutes and distributed builds, won't be used. + Its effect is to set [`preferLocalBuild = true`][preferLocalBuild] and [`allowSubstitutes = false`][allowSubstitutes]. + + ::: {.note} + This prevents the use of substitutors, so only set `runLocal` (or use `runCommandLocal`) when certain the user will + always have a builder for the `system` of the derivation. This should be true for most trivial use cases + (e.g., just copying some files to a different location or adding symlinks) because there the `system` + is usually the same as `builtins.currentSystem`. + ::: + +`stdenv` (Derivation) +: The [standard environment](#chap-stdenv) to use, defaulting to `pkgs.stdenv` + +`derivationArgs` (Attribute set) +: Additional arguments for [`mkDerivation`](#sec-using-stdenv). + +`buildCommand` (String) +: Shell commands to run in the derivation builder. + + ::: {.note} + You have to create a file or directory `$out` for Nix to be able to run the builder successfully. + ::: + +[allowSubstitutes]: https://nixos.org/nix/manual/#adv-attr-allowSubstitutes +[preferLocalBuild]: https://nixos.org/nix/manual/#adv-attr-preferLocalBuild + +::: {.example #ex-runcommandwith} +# Invocation of `runCommandWith` + +```nix +runCommandWith { + name = "example"; + derivationArgs.nativeBuildInputs = [ cowsay ]; +} '' + cowsay > $out < AttrSet -> String -> Derivation` @@ -58,7 +128,7 @@ This works just like `runCommand`. The only difference is that it also provides Variant of `runCommand` that forces the derivation to be built locally, it is not substituted. This is intended for very cheap commands (<1s execution time). It saves on the network round-trip and can speed up a build. ::: {.note} -This sets [`allowSubstitutes` to `false`](https://nixos.org/nix/manual/#adv-attr-allowSubstitutes), so only use `runCommandLocal` if you are certain the user will always have a builder for the `system` of the derivation. This should be true for most trivial use cases (e.g., just copying some files to a different location or adding symlinks) because there the `system` is usually the same as `builtins.currentSystem`. +This sets [`allowSubstitutes` to `false`][allowSubstitutes], so only use `runCommandLocal` if you are certain the user will always have a builder for the `system` of the derivation. This should be true for most trivial use cases (e.g., just copying some files to a different location or adding symlinks) because there the `system` is usually the same as `builtins.currentSystem`. ::: ## Writing text files {#trivial-builder-text-writing} diff --git a/pkgs/build-support/trivial-builders/default.nix b/pkgs/build-support/trivial-builders/default.nix index fc6f07fdd11b..1939b24ccec3 100644 --- a/pkgs/build-support/trivial-builders/default.nix +++ b/pkgs/build-support/trivial-builders/default.nix @@ -36,17 +36,8 @@ rec { # `runCommandCCLocal` left out on purpose. # We shouldn’t force the user to have a cc in scope. - # TODO: Move documentation for runCommandWith to the Nixpkgs manual - /* - Generalized version of the `runCommand`-variants - which does customized behavior via a single - attribute set passed as the first argument - instead of having a lot of variants like - `runCommand*`. Additionally it allows changing - the used `stdenv` freely and has a more explicit - approach to changing the arguments passed to - `stdenv.mkDerivation`. - */ + # Docs in doc/build-helpers/trivial-build-helpers.chapter.md + # See https://nixos.org/manual/nixpkgs/unstable/#trivial-builder-runCommandWith runCommandWith = let # prevent infinite recursion for the default stdenv value From 482d6eaab2803e5d3be84124d0677d9dd5234a29 Mon Sep 17 00:00:00 2001 From: nicoo Date: Thu, 8 Aug 2024 16:36:03 +0000 Subject: [PATCH 2/5] doc/build-helpers: refactor the paragraphs about `runCommand{,CC}` Co-authored-by: Johannes Kirschbauer --- .../trivial-build-helpers.chapter.md | 39 ++++++++++--------- 1 file changed, 21 insertions(+), 18 deletions(-) diff --git a/doc/build-helpers/trivial-build-helpers.chapter.md b/doc/build-helpers/trivial-build-helpers.chapter.md index 0c7c3f899a73..3d2c7f166710 100644 --- a/doc/build-helpers/trivial-build-helpers.chapter.md +++ b/doc/build-helpers/trivial-build-helpers.chapter.md @@ -73,32 +73,39 @@ runCommandWith { ::: -## `runCommand` {#trivial-builder-runCommand} +## `runCommand` and `runCommandCC` {#trivial-builder-runCommand} -`runCommand :: String -> AttrSet -> String -> Derivation` +The function `runCommand` returns a derivation built using the specified command(s), in the `stdenvNoCC` environment. -The result of `runCommand name drvAttrs buildCommand` is a derivation that is built by running the specified shell commands. +`runCommandCC` is similar but uses the default compiler environment. To minimize dependencies, `runCommandCC` +should only be used when the build command needs a C compiler. -By default `runCommand` runs in a stdenv with no compiler environment, whereas [`runCommandCC`](#trivial-builder-runCommandCC) uses the default stdenv, `pkgs.stdenv`. +### Type {#trivial-builder-runCommand-Type} -`name :: String` -: The name that Nix will append to the store path in the same way that `stdenv.mkDerivation` uses its `name` attribute. +``` +runCommand :: String -> AttrSet -> String -> Derivation +runCommandCC :: String -> AttrSet -> String -> Derivation +``` -`drvAttr :: AttrSet` -: Attributes to pass to the underlying call to [`stdenv.mkDerivation`](#chap-stdenv). +### Input {#trivial-builder-runCommand-Input} -`buildCommand :: String` -: Shell commands to run in the derivation builder. +While the type signature(s) differ from `runCommandWith`, individual arguments with the same name will have the same type and meaning: + +`name` (String) +: The derivation's name + +`derivationArgs` (Attribute set) +: Additional parameters passed to [`mkDerivation`] + +`buildCommand` (String) +: The command(s) run to build the derivation. - ::: {.note} - You have to create a file or directory `$out` for Nix to be able to run the builder successfully. - ::: ::: {.example #ex-runcommand-simple} # Invocation of `runCommand` ```nix -(import {}).runCommand "my-example" {} '' +runCommand "my-example" {} '' echo My example command is running mkdir $out @@ -119,10 +126,6 @@ By default `runCommand` runs in a stdenv with no compiler environment, whereas [ ``` ::: -## `runCommandCC` {#trivial-builder-runCommandCC} - -This works just like `runCommand`. The only difference is that it also provides a C compiler in `buildCommand`'s environment. To minimize your dependencies, you should only use this if you are sure you will need a C compiler as part of running your command. - ## `runCommandLocal` {#trivial-builder-runCommandLocal} Variant of `runCommand` that forces the derivation to be built locally, it is not substituted. This is intended for very cheap commands (<1s execution time). It saves on the network round-trip and can speed up a build. From e3d7e7f2a7e54978dfeccb95bb0fb9b6761fc0f7 Mon Sep 17 00:00:00 2001 From: nicoo Date: Thu, 8 Aug 2024 16:48:57 +0000 Subject: [PATCH 3/5] doc/build-helpers: add note relating `runCommand` and `runCommandWith` --- .../trivial-build-helpers.chapter.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/doc/build-helpers/trivial-build-helpers.chapter.md b/doc/build-helpers/trivial-build-helpers.chapter.md index 3d2c7f166710..42e36821d9ec 100644 --- a/doc/build-helpers/trivial-build-helpers.chapter.md +++ b/doc/build-helpers/trivial-build-helpers.chapter.md @@ -126,6 +126,24 @@ runCommand "my-example" {} '' ``` ::: +::: {.note} +`runCommand name derivationArgs buildCommand` is equivalent to +```nix +runCommandWith { + inherit name derivationArgs; + stdenv = stdenvNoCC; +} buildCommand +``` + +Likewise, `runCommandCC name derivationArgs buildCommand` is equivalent to +```nix +runCommandWith { + inherit name derivationArgs; +} buildCommand +``` +::: + + ## `runCommandLocal` {#trivial-builder-runCommandLocal} Variant of `runCommand` that forces the derivation to be built locally, it is not substituted. This is intended for very cheap commands (<1s execution time). It saves on the network round-trip and can speed up a build. From 2b8a6a7e43e007f1bd40c515c1ecb84238638b6d Mon Sep 17 00:00:00 2001 From: nicoo Date: Thu, 8 Aug 2024 16:44:41 +0000 Subject: [PATCH 4/5] doc/build-helpers: refactor the paragraph about `runCommandLocal` --- .../trivial-build-helpers.chapter.md | 27 ++++++++++--------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/doc/build-helpers/trivial-build-helpers.chapter.md b/doc/build-helpers/trivial-build-helpers.chapter.md index 42e36821d9ec..6471395f78a0 100644 --- a/doc/build-helpers/trivial-build-helpers.chapter.md +++ b/doc/build-helpers/trivial-build-helpers.chapter.md @@ -29,11 +29,12 @@ runCommandWith :: { : The derivation's name, which Nix will append to the store path; see [`mkDerivation`](#sec-using-stdenv). `runLocal` (Boolean) -: If set to `true` this forces the derivation to be built locally. Remote substitutes and distributed builds, won't be used. +: If set to `true` this forces the derivation to be built locally, not using [substitutes] nor remote builds. + This is intended for very cheap commands (<1s execution time) which can be sped up by avoiding the network round-trip(s). Its effect is to set [`preferLocalBuild = true`][preferLocalBuild] and [`allowSubstitutes = false`][allowSubstitutes]. ::: {.note} - This prevents the use of substitutors, so only set `runLocal` (or use `runCommandLocal`) when certain the user will + This prevents the use of [substituters][substituter], so only set `runLocal` (or use `runCommandLocal`) when certain the user will always have a builder for the `system` of the derivation. This should be true for most trivial use cases (e.g., just copying some files to a different location or adding symlinks) because there the `system` is usually the same as `builtins.currentSystem`. @@ -54,6 +55,8 @@ runCommandWith :: { [allowSubstitutes]: https://nixos.org/nix/manual/#adv-attr-allowSubstitutes [preferLocalBuild]: https://nixos.org/nix/manual/#adv-attr-preferLocalBuild +[substituter]: https://nix.dev/manual/nix/latest/glossary#gloss-substituter +[substitutes]: https://nix.dev/manual/nix/2.23/glossary#gloss-substitute ::: {.example #ex-runcommandwith} # Invocation of `runCommandWith` @@ -80,16 +83,22 @@ The function `runCommand` returns a derivation built using the specified command `runCommandCC` is similar but uses the default compiler environment. To minimize dependencies, `runCommandCC` should only be used when the build command needs a C compiler. +`runCommandLocal` is also similar to `runCommand`, but forces the derivation to be built locally. +See the note on [`runCommandWith`] about `runLocal`. + +[`runCommandWith`]: #trivial-builder-runCommandWith + ### Type {#trivial-builder-runCommand-Type} ``` -runCommand :: String -> AttrSet -> String -> Derivation -runCommandCC :: String -> AttrSet -> String -> Derivation +runCommand :: String -> AttrSet -> String -> Derivation +runCommandCC :: String -> AttrSet -> String -> Derivation +runCommandLocal :: String -> AttrSet -> String -> Derivation ``` ### Input {#trivial-builder-runCommand-Input} -While the type signature(s) differ from `runCommandWith`, individual arguments with the same name will have the same type and meaning: +While the type signature(s) differ from [`runCommandWith`], individual arguments with the same name will have the same type and meaning: `name` (String) : The derivation's name @@ -144,14 +153,6 @@ runCommandWith { ::: -## `runCommandLocal` {#trivial-builder-runCommandLocal} - -Variant of `runCommand` that forces the derivation to be built locally, it is not substituted. This is intended for very cheap commands (<1s execution time). It saves on the network round-trip and can speed up a build. - -::: {.note} -This sets [`allowSubstitutes` to `false`][allowSubstitutes], so only use `runCommandLocal` if you are certain the user will always have a builder for the `system` of the derivation. This should be true for most trivial use cases (e.g., just copying some files to a different location or adding symlinks) because there the `system` is usually the same as `builtins.currentSystem`. -::: - ## Writing text files {#trivial-builder-text-writing} Nixpkgs provides the following functions for producing derivations which write text files or executable scripts into the Nix store. From 9e5d56e8c64e00f6c69070952fbf76aae5f0497a Mon Sep 17 00:00:00 2001 From: nicoo Date: Mon, 19 Aug 2024 19:13:20 +0000 Subject: [PATCH 5/5] doc/build-helpers: forward-link `runCommand*` in `runCommandWith` --- doc/build-helpers/trivial-build-helpers.chapter.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/doc/build-helpers/trivial-build-helpers.chapter.md b/doc/build-helpers/trivial-build-helpers.chapter.md index 6471395f78a0..8af68845202f 100644 --- a/doc/build-helpers/trivial-build-helpers.chapter.md +++ b/doc/build-helpers/trivial-build-helpers.chapter.md @@ -8,10 +8,14 @@ Like [`stdenv.mkDerivation`](#sec-using-stdenv), each of these build helpers cre The function `runCommandWith` returns a derivation built using the specified command(s), in a specified environment. -It is the underlying base function of all `runCommand*` variants. -The general behavior is controlled via a single attribute set passed +It is the underlying base function of all [`runCommand*` variants]. +The general behavior is controlled via a single attribute set passed as the first argument, and allows specifying `stdenv` freely. +The following [`runCommand*` variants] exist: `runCommand`, `runCommandCC`, and `runCommandLocal`. + +[`runCommand*` variants]: #trivial-builder-runCommand + ### Type {#trivial-builder-runCommandWith-Type} ```