rustc allows code to perform compile-time tests against target_env [0],
which is a function of the target triple. There isn't provision in
Nixpkgs for learning target_env, however. That doesn't affect rustc's
evaluation of cfg guards in Rust code - rustc knows perfectly well what
target_env is - but it _does_ affect the env vars passed to a build
script set by buildRustCrate, which is presently hard-coded to gnu, and
also affects any Nix code looking to branch on target_env.
Being able to access target_env is very relevant to, for example, WASI
as (using Rust terminology) wasm32-wasip1 and wasm32-wasip2 differ only
by target_env, with identical target_arch and target_os values. Properly
reflecting target_env may also fix some random musl issues if you're a
bit lucky.
This commit adds a new attr, rust.platform.env, which reflects the
target_env that rustc will set for the target, and wires it up in
buildRustCrate. In isolation, this change mostly only affects build
scripts checking target_env, but crate2nix will greatly benefit from
being able to accurately resolve the dep graph (which can also vary
depending on target_env).
The target triple -> target_env function resists Kolmogorov compression:
it's irregular and, though there are some patterns, there are lots of
special cases. So, I have done the stupidest possible thing and scraped
out all of the targets with non-empty target_env values and dumped that
into an attrset. This attrset will progressively get out of date as
rustc adds new platforms, but updating it should be simple enough - I've
included the generation script as a comment.
There are some other configuration options not being reflected in Nix. I
have left those alone, but, in the future, maybe this can be extended to
just dumping all of them into an attrset and then reflecting them in
rust.platform. It might even make sense to convert the existing code
producing rust.platform to just looking up from an attrset and to 'do
what rustc does'.
I would have liked to have added a test targetting a platform with a
non-empty non-GNU target_env, but all of the yaks were quite hairy.
Fixes https://github.com/NixOS/nixpkgs/issues/436832
[0]: https://doc.rust-lang.org/reference/conditional-compilation.html#r-cfg.target_env
Scope:
- Combination of
- Textual matches of "baseNameOf (toString"
- Redundant toString calls I've found with my latest
"lazy paths" nix branch as they force lazy fetches into
the store. More info and new PR soon.
- Only cases I believe are worthwhile or easily determined
I've determined the validity by
- testing llvmPackages instantiation
- figuring out which types can pass into any particular
toString call - "human fuzzy type checker"
Behavior considerations by type:
- `path`: converted back to a string *without* context
`baseNameOf` does not copy things to the store on its own,
equivalent to its behavior for string inputs
- `null`: converted to `""` -> may be valid input!
ok if "" would not have been acceptable anyway
- `string` itself: passed through identically -> trivial
- `attrset` with `outPath`: same coercion as built into
the `baseNameOf` function -> trivial
- other atomic types: generally not sensible inputs to
`baseNameOf` -> fuzzy but true
With the addition of `remap-path-prefix` to buildRustCrate, we now
can pull in two rustc variants when cross compiling, since the rustc
that is string-interpolated for passing the remap flag is not pulled
from the proper package-set. Depending on the host platform used, this
rustc variant may not even build. In order to only pull in one variant,
and to properly mask the rustc used for building with the
remap-path-prefix flag, rustc is passed explicitly from pkgsBuildHost.
The crates.io API server's 1 req/sec rate limit currently surfaces as
intermittent HTTP 403 errors when vendoring lockfiles. Switch to the CDN
endpoint as recommended by upstream (rust-lang/crates.io#13482), mirroring
the fix already applied to fetchCargoVendor in #512735.
fetchurl is content-addressed by sha256, so the URL change does not affect
any downstream store paths.
Fixes#524979
buildRustCrate calls rustc directly, so there is no built-in equivalent
of `cargo clippy`. The capLints doc already hints at the existing
workaround — overriding the `rust` attribute with a wrapper script that
sniffs argv for build_script_build and routes everything else to
clippy-driver. That works because clippy-driver IS rustc internally
(rustc_driver plus lint passes) and produces compatible rlibs and
metadata, but the wrapper is fragile and every downstream reinvents it.
Add a `useClippy` boolean (default false) that swaps the lib/bin/test
compilation to clippy-driver while keeping plain rustc for build.rs —
the same lib-vs-build-script split as extraRustcOptsForBuildRs, since
build scripts are typically auto-generated and clippy noise there is
not actionable. The clippy-driver binary comes from a new `clippy`
parameter, auto-filled from pkgs.clippy by callPackage and overridable
alongside `rust` for toolchain bundles that ship their own.
Implemented by introducing a RUSTC_DRIVER shell variable in
build-crate.nix that lib.sh's build_lib/build_bin consume; the
build_script_build path in configure-crate.nix never sees it.
Note that the default `capLints = "allow"` cap suppresses clippy lints
along with everything else, so `useClippy` is usually paired with
`capLints = "warn"` and `-W clippy::*`/`-D warnings` flags via
extraRustcOpts. Documented in the manual.
Test fixtures cover the three load-bearing properties: a denied clippy
lint fails the build (proving clippy-driver ran), the default capLints
silences clippy lints just like rustc lints, and an rlib emitted by
clippy-driver is link-compatible with a plain-rustc dependent.
buildRustCrate passes `--remap-path-prefix=$NIX_BUILD_TOP=/` so the
sandbox build directory is not embedded in compiled artifacts. The
resulting source path is `/$sourceRoot/src/...` — which works out to
`/<crateName>-<version>/src/...` only because fetchCrate happens to
unpack to a directory named after the crate.
When `src` is supplied directly — `lib.fileset.toSource` (always
`source`), `lib.cleanSource ./.`, a flake's `self`, or any
`<hash>-source` store path — stdenv unpacks to `$NIX_BUILD_TOP/source/`,
so every such crate ends up with `/source/src/lib.rs`. The crate
identity is gone from panic backtraces, `file!()` expansions, debuginfo,
and llvm-cov coverage maps. Workspace builds via crate2nix that source
each member with a fileset see all members collapse to the same prefix,
making per-crate coverage and backtrace attribution impossible without a
per-crate remap override.
Add a second, more specific remap of `$NIX_BUILD_TOP/$sourceRoot` to
`/<crateName>-<version>`. rustc applies `--remap-path-prefix` flags
last-match-wins, so this prefix wins for everything under the source
root — which in a buildRustCrate build is effectively everything that
would otherwise embed a sandbox path, including `OUT_DIR` (placed at
`$sourceRoot/target/build/` by configure-crate.nix). The broader
`$NIX_BUILD_TOP=/` remap stays as a fallback for any path outside
`$sourceRoot`.
This is safe with respect to the original remap's purpose:
- Reproducibility: still maps the per-build sandbox path to a fixed,
build-independent string. `crateName` and `version` are already
derivation inputs, so the result is deterministic.
- Closure size: still no store paths in the embedded strings.
- fetchCrate sources: `$sourceRoot` is `<crateName>-<version>` there,
so the new remap rewrites `/<crateName>-<version>` to itself — the
output is byte-for-byte identical. Only custom-`src` crates change,
and they change from a degenerate `/source/...` to the same shape
fetchCrate crates already had.
- The remap only affects path *strings* embedded in output (DWARF
`DW_AT_comp_dir`/`DW_AT_name`, `file!()`, `Location::file()`, LLVM
coverage filename tables). It does not change which files are read
or compiled.
Users who depend on the old `/source/...` prefix can restore it via
`extraRustcOpts`, which is appended after these flags and therefore
wins under last-match-wins.
Cargo links [dev-dependencies] only when building test or bench
targets. buildRustCrate models normal deps (`dependencies`) and
build-script deps (`buildDependencies`) but has no equivalent: callers
that set `buildTests = true` for a crate with dev-deps must compute the
dev-dep set themselves and splice it into `dependencies` via .override.
Add a `devDependencies` parameter that is appended to the linked
dependency list only when `buildTests = true`. The append happens at the
single point where `dependencies` is normalized, so it threads through
to the --extern flags, the deps directory in configurePhase, the
metadata hash, and propagated build inputs without further changes.
When `buildTests = false` the parameter is dropped, so existing lib/bin
derivations are byte-identical to before — no rebuilds for users that
don't pass it.
Includes a test case in build-rust-crate/test that fails without the
change (`can't find crate for dev_dep`).
zstd-sys: set ZSTD_SYS_USE_PKG_CONFIG so build.rs probes the system
libzstd via pkg-config instead of compiling the vendored C amalgamation
from scratch in every consumer.
fuser: with the default-on libfuse feature, build.rs discovers libfuse
via pkg-config and has no vendored fallback, so it fails in the sandbox
without pkg-config and fuse3 in scope.
Proc-macro crates are host dylibs that rustc dlopen()s. Instrumentation
flags passed via extraRustcOpts (e.g. -Zsanitizer=address,
-Cinstrument-coverage) leave unresolved runtime symbols in those dylibs
and break the build. Cargo avoids this by not applying RUSTFLAGS to host
artifacts; buildRustCrate already has extraRustcOptsForBuildRs for build
scripts, so add the analogous knob for proc-macros.
Defaults to null, which falls back to extraRustcOpts so existing callers
are unchanged. Set to [] to opt proc-macros out when applying
sanitizer/coverage flags tree-wide via crateOverrides.
Commit c4f5dfad0d was intended as a stylistic cleanup, but the trailing \ it
removed from pkgs/build-support/rust/lib/default.nix was not stylistic: it is a
shell line continuation that joins the cross-only env var block to the cargo
command added by consumers. Without it, `CC_*`, `CXX_*`, and
`CARGO_TARGET_*_LINKER` are silently not passed to cargo when
rustTargetPlatform != rustHostPlatform.
This in turn can break cross compilation of Rust packages.
Fix it by adding back the trailing \
Fixes: #497857
Signed-off-by: Gilberto Bertin <me@jibi.io>
Add completePropagatedBuildInputs that collects propagatedBuildInputs
from a crate and all its transitive Rust dependencies, analogous to how
completeDeps chains .rlib paths. The collected inputs are appended to
buildInputs so native library deps (e.g. boost) declared on a library
crate automatically propagate to binary crates that depend on it,
without requiring repetition in every downstream crate override.
The Makefile generated for parallel binary builds used the binary names
directly as make targets. This caused make to silently skip a build
whenever a file or directory with the same name existed in the crate
source root — e.g. a crate with a `cli` binary and a `cli/` directory
would "succeed" without producing the binary.
Using the binary name as a target also risked collision with the `all`
meta-target and with make metacharacters (`$`, `:`, `#`) in user-supplied
crateName or bin.path values.
Switch to synthetic numeric targets (b0, b1, …) declared .PHONY, and
escape `$` in the recipe arguments.
Collect binary targets into a bash associative array (name→path), then
generate a Makefile on the fly and pipe it to `make -j`. Make handles
parallel execution and the jobserver protocol natively, so rustc
invocations share the same token pool via MAKEFLAGS.
Also fix build_bin error propagation: add `|| return 1` to the rustc
invocation so failures are not silently swallowed when the crate name
has no hyphens and the mv rename is skipped.
nixpkgs-ci[bot]