Commit 88cfc54552 ("nix-prefetch-git: disable maintenance mode via
environment variables") disabled maintenance.auto (git's new
auto-maintenance) via GIT_CONFIG_*, which propagates to submodules. But
the legacy auto-gc path (gc.auto plus detached gc.autoDetach) is still
enabled, and it keeps repacking .git/modules/<sub>/objects after a nested
submodule reports "checked out", racing the later `rm -rf .git`:
- rm empties objects/pack, the detached gc drops a fresh pack in, and
rmdir fails with "Directory not empty" (exit 123); or
- removal partially completes, leaving varying .git remnants, yielding a
non-deterministic NAR hash.
Disable gc.auto/gc.autoDetach alongside maintenance.auto, using the same
GIT_CONFIG_* mechanism so the settings reach every git invocation and its
children (including nested submodule fetches). Fold the repeated
boilerplate into a small git_config_env helper.
The successful output is unchanged (.git is removed either way), so
pinned fetchgit hashes stay valid; this only makes removal deterministic.
Fixes#524215.
Assisted-by: Claude Code (claude-opus-4-8)
The commit 011471c7f2 ("fetchgit: disable git maintenance") disabled
git's maintenance mode, but that does not affect git repositories with
submodules.
Switch to using the GIT_CONFIG_* environment variables. This is done in
a way to make the approach more generic, but also kinda not the best
("insane") so that it works with any future change, even with
`impureEnvVars`.
Maybe `git -c maintenance.auto=false` can work but I cannot figure out
how to use that here. I would prefer disabling maintenance mode in Git's
derivation itself but did not find any build-time knob to toggle that.
By default, Git commands will trigger background processes to perform
repository maintenance. As of Git v2.54.0, and in particular as of
452b12c2e0 (builtin/maintenance: use "geometric" strategy by default,
2026-02-24), Git has changed the strategy it uses to decide whether
maintenance needs performing, and as a result will kick off background
maintenance processes much more often.
These background processes will potentially change the contents of the
`.git` directory while `nix-prefetch-git` is deleting it. This results in
effectively random failures, when files are added to the `.git`
directory during the `rm -rf` operation (causing it to fail) or after
the operation has completed (causing the `.git` directory to be
recreated and resulting in a FOD hash mismatch).
This also causes problems for the determinism of Git's own tests.
They've resolved this in 09505b1115 (t: fix races caused by background
maintenance, 2026-02-24) by having the maintenance processes run in the
foreground. The discussion in that commit explains they considered just
disabling maintenance entirely, but opted for the foreground option
because disabling maintenance would mean the tests looked even less like
how most Git users work with Git.
`nix-prefetch-git` is nothing like most Git users. In almost every
case, the `.git` directory will be deleted during or immediately after
the script is called. The repositories certainly don't benefit from
ongoing maintenance. To avoid the non-determinism created by the
background maintenance processes, just disable maintenance entirely.
This is done with `git config` after initialising the repository, rather
than `git config --global` earlier in the script, as `git config
--global` will fail if `fetchgit` was called with a `gitConfig`
argument.
It would be possible to set this option with
`GIT_CONFIG_[COUNT|KEY_<n>|VALUE_<n>]` environment variables; I've opted
against that because (a) they're less familiar to most people who might
want to come to edit this script in future and (b) because if someone is
using those values themselves via `impureEnvVars` or similar, making our
configuration play nicely with someone else's would require some fairly
complex logic.
I haven't made this a configurable option: any use of `nix-prefetch-git`
will run with maintenance disabled, as I cannot think of a use case
where it would be useful. If there is such a use case, it would be
straightforward to extend `nix-prefetch-git` and `fetchgit` to allow
users to change the behaviour.
Fixes#524215.
When leaveDotGit == true,
restore the cloning behaviour before
commit 7e085677f9 ("nix-prefetch-git: dont't fetch tags when deep clone unless leaving .git")
to preserve the fragile hashes of .git sources.
Clean up fetchTagsCompat,
as fetchTagsCompat is no longer needed
after restoring the old tag-fetching flags.
Adds a new argument to `nix-prefetch-git`: `--no-add-path`. It disables
adding the path to the store. It is useful when working with a read-only
store.
With this argument nix-prefetch-git will make a subdirectory of the Git
repository a root of the resulting store path. This is helpful for
dealing with monorepos where many projects are in separate directories
and don't need a new source hash every time the monorepo is updated.
The corresponding fetchgit change follows.
For projects that have `submodule.$name.update` set to something other
than the default value of "checkout", this change will ensure those
submodules are fetched as well.
Added fetchTags feature to fetchgit, explicit and clear support for
fetching all tags after the source tree fetch completes. Doing this at
build-time in the fetcher is required for packages that invoke commands
like 'git describe' which require tags, and since the nix store is
read-only by design, it is not possible to git fetch --tags at
activation- or run-time. This feature may have been possible by
specifying a postFetch option including calling git fetch --tags,
however doing so obfuscates the solution to this very real problem.
Explicit support for fetching tags should be a first class citizen just
like fetching other refs.
Since nix 2.20, `nix-store --add-fixed` doesn't accept paths where the
parent directory is a symlink. On macOS, /tmp is a symlink to
/private/tmp, which causes a "'/tmp' is a symlink" error:
```
$ nix run github:nixos/nixpkgs/24.11-beta#nix-prefetch-git -- --url https://github.com/IFTTT/polo.git --rev 316aa2ac210a45a7fc400ab921831493d5dd21b8 --hash sha256
Initialized empty Git repository in /private/tmp/git-checkout-tmp-1Bf9bIv7/polo-316aa2a/.git/
remote: Enumerating objects: 51, done.
remote: Counting objects: 100% (51/51), done.
remote: Compressing objects: 100% (42/42), done.
remote: Total 51 (delta 8), reused 19 (delta 5), pack-reused 0 (from 0)
Unpacking objects: 100% (51/51), 19.57 KiB | 541.00 KiB/s, done.
From https://github.com/IFTTT/polo
* branch HEAD -> FETCH_HEAD
Switched to a new branch 'fetchgit'
removing `.git'...
error: path '/tmp' is a symlink
```
Avoid this by resolving /tmp to a real directory in all the prefetch scripts
This envvar is also added to lib.proxyImpureEnvVars since it's
typically required for https proxies.
This change also updates fetchgit and go module fetching to use this
envvar. NIX_GIT_SSL_CAINFO is still supported for backwards
compatibility in fetchgit.
Some programs (e.g. nvfetcher) parse the output of nix-prefetch-git as
JSON. However, there is a pushd/popd command in nix-prefetch-git that
outputs the directory name, breaking the structure of the result JSON.
We suppressed it with `>/dev/null`.
Passingly fixes a mixuse of tab and spaces.
This script needs to support being run both as part of a `fetchgit`
derivation and as a standalone, command-line tool.
The use of `$NIX_BUILD_TOP` only works when used in `fetchgit` but not when
invoked as a standalone tool. Instead we try to respect `$NETRC` so that
the command-line invocation behaves more like standard tools and the
`fetchgit` derivation can explicitly set `$NETRC` when `netrcPhase` is used
to avoid all ambiguity.
This fixes a regression introduced in https://github.com/NixOS/nixpkgs/pull/256628
which broke fetching with private = true through a netrc file.
Tested locally with a really special github enterprise.
nix-prefetch-git is either run as part of a build, usually sandboxed,
or outside a build, unsandboxed, to prefetch something that will later
be used in a build. It's important that the latter use produces
hashes that can be reproduced by the former.
One way that they can differ is if the user's git config does
something that changes the result of git clone. I ran into this,
because my global git config automatically enables git-lfs, whereas
nix-prefetch-git otherwise only uses git-lfs if specifically
requested. This led to very confusing hash mismatches.
The fetchgit function in nixpkgs sets the leaveDotGit argument to true
if deepClone is set to true. nix-prefetch-git did behave differently. It
would not assume --leave-dotGit if --deepClone is specified. With this
change the inconsistency is addressed by assuming --leave-dotGit if
--deepClone is specified.
It's hugely inefficient as we can't use shallow cloning (--depth=1).
This has been tested and adapted for quite a few hosts fetchgit is used on in
Nixpkgs. For those where fetching the hash directly doesn't work (most notably
git.savannah.gnu.org), we simply fall back to the old method.
The reason for this change is simply to avoid the following messages
that are unnecessary and can be confusing (and these messages will be
repeated for each submodule):
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint: git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint: git branch -m <name>
With this change the behaviour remains unchanged (apart from the
suppressed "warning" in the console output of course) and therefore this
doesn't cause any hashes to change and by default nix-prefetch-git uses
the "fetchgit" branch anyway (branchName can be set to override the
default):
Switched to a new branch 'fetchgit'
For that reason the initial branch name doesn't matter anyway and since
we're not relying on / hardcoding "master" we could simply switch to
"main" (which seems most common nowadays). See [0] for more details on
why this wouldn't break anything.
However, since the initial branch name doesn't matter and to avoid any
additional risks it was "decided" to keep using "master" (s. #113313).
[0]: https://github.com/NixOS/nixpkgs/pull/113313#issuecomment-780589516
Philip Taron