#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343
#### Incorporates
- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: 5e4ec82181/gnu/packages/gcc.scm (L253)
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: 2f1948af9c/pkgs/stdenv/linux/default.nix (L380)
97 lines
4.4 KiB
Nix
97 lines
4.4 KiB
Nix
{ lib
|
|
, stdenv
|
|
, langC
|
|
, langCC
|
|
, langJit
|
|
}:
|
|
|
|
let
|
|
enableLibGccOutput = (with stdenv; targetPlatform == hostPlatform) && !langJit;
|
|
in
|
|
(pkg: pkg.overrideAttrs (previousAttrs: lib.optionalAttrs ((!langC) || langJit || enableLibGccOutput) {
|
|
outputs = previousAttrs.outputs ++ lib.optionals enableLibGccOutput [ "libgcc" ];
|
|
# This is a separate phase because gcc assembles its phase scripts
|
|
# in bash instead of nix (we should fix that).
|
|
preFixupPhases = (previousAttrs.preFixupPhases or []) ++ [ "preFixupLibGccPhase" ];
|
|
preFixupLibGccPhase =
|
|
# delete extra/unused builds of libgcc_s in non-langC builds
|
|
# (i.e. libgccjit, gnat, etc) to avoid potential confusion
|
|
lib.optionalString (!langC) ''
|
|
rm -f $out/lib/libgcc_s.so*
|
|
''
|
|
|
|
# TODO(amjoseph): remove the `libgcc_s.so` symlinks below and replace them
|
|
# with a `-L${gccForLibs.libgcc}/lib` in cc-wrapper's
|
|
# `$out/nix-support/cc-flags`. See also:
|
|
# - https://github.com/NixOS/nixpkgs/pull/209870#discussion_r1130614895
|
|
# - https://github.com/NixOS/nixpkgs/pull/209870#discussion_r1130635982
|
|
# - https://github.com/NixOS/nixpkgs/commit/404155c6acfa59456aebe6156b22fe385e7dec6f
|
|
#
|
|
# move `libgcc_s.so` into its own output, `$libgcc`
|
|
+ lib.optionalString enableLibGccOutput (''
|
|
# move libgcc from lib to its own output (libgcc)
|
|
mkdir -p $libgcc/lib
|
|
mv $lib/lib/libgcc_s.so $libgcc/lib/
|
|
mv $lib/lib/libgcc_s.so.1 $libgcc/lib/
|
|
ln -s $libgcc/lib/libgcc_s.so $lib/lib/
|
|
ln -s $libgcc/lib/libgcc_s.so.1 $lib/lib/
|
|
''
|
|
#
|
|
# Nixpkgs ordinarily turns dynamic linking into pseudo-static linking:
|
|
# libraries are still loaded dynamically, exactly which copy of each
|
|
# library is loaded is permanently fixed at compile time (via RUNPATH).
|
|
# For libgcc_s we must revert to the "impure dynamic linking" style found
|
|
# in imperative software distributions. We must do this because
|
|
# `libgcc_s` calls `malloc()` and therefore has a `DT_NEEDED` for `libc`,
|
|
# which creates two problems:
|
|
#
|
|
# 1. A circular package dependency `glibc`<-`libgcc`<-`glibc`
|
|
#
|
|
# 2. According to the `-Wl,-rpath` flags added by Nixpkgs' `ld-wrapper`,
|
|
# the two versions of `glibc` in the cycle above are actually
|
|
# different packages. The later one is compiled by this `gcc`, but
|
|
# the earlier one was compiled by the compiler *that compiled* this
|
|
# `gcc` (usually the bootstrapFiles). In any event, the `glibc`
|
|
# dynamic loader won't honor that specificity without namespaced
|
|
# manual loads (`dlmopen()`). Once a `libc` is present in the address
|
|
# space of a process, that `libc` will be used to satisfy all
|
|
# `DT_NEEDED`s for `libc`, regardless of `RUNPATH`s.
|
|
#
|
|
# So we wipe the RUNPATH using `patchelf --set-rpath ""`. We can't use
|
|
# `patchelf --remove-rpath`, because at least as of patchelf 0.15.0 it
|
|
# will leave the old RUNPATH string in the file where the reference
|
|
# scanner can still find it:
|
|
#
|
|
# https://github.com/NixOS/patchelf/issues/453
|
|
#
|
|
# Note: we might be using the bootstrapFiles' copy of patchelf, so we have
|
|
# to keep doing it this way until both the issue is fixed *and* all the
|
|
# bootstrapFiles are regenerated, on every platform.
|
|
#
|
|
# This patchelfing is *not* effectively equivalent to copying
|
|
# `libgcc_s` into `glibc`'s outpath. There is one minor and one
|
|
# major difference:
|
|
#
|
|
# 1. (Minor): multiple builds of `glibc` (say, with different
|
|
# overrides or parameters) will all reference a single store
|
|
# path:
|
|
#
|
|
# /nix/store/xxx...xxx-gcc-libgcc/lib/libgcc_s.so.1
|
|
#
|
|
# This many-to-one referrer relationship will be visible in the store's
|
|
# dependency graph, and will be available to `nix-store -q` queries.
|
|
# Copying `libgcc_s` into each of its referrers would lose that
|
|
# information.
|
|
#
|
|
# 2. (Major): by referencing `libgcc_s.so.1`, rather than copying it, we
|
|
# are still able to run `nix-store -qd` on it to find out how it got
|
|
# built! Most importantly, we can see from that deriver which compiler
|
|
# was used to build it (or if it is part of the unpacked
|
|
# bootstrap-files). Copying `libgcc_s.so.1` from one outpath to
|
|
# another eliminates the ability to make these queries.
|
|
#
|
|
+ ''
|
|
patchelf --set-rpath "" $libgcc/lib/libgcc_s.so.1
|
|
'');
|
|
}))
|