maintainers/scripts/update: Allow updating in (reverse) topological order

Previously, when updating multiple packages, we just updated them in arbitrary order. However, when some of those packages depended on each other, it could happen that some of the intermediary commits would not build because of version constraints on dependencies.

If we want each commit in the history to build when feasible, we need to consider four different scenarios:

1. Updated dependant is compatible with both the old and the new version of the dependency. Order of commits does not matter. But updating dependents first (i.e. reverse topological order) is useful since it allows building each package on the commit that updates it with minimal rebuilds.
2. Updated dependant raises the minimal dependency version. Dependency needs to be updated first (i.e. topological order).
3. Old dependant sets the maximal dependency version. Dependant needs to be updated first (i.e. reverse topological order).
4. Updated dependant depends on exact version of dependency and they are expected to be updated in lockstep. The earlier commit will be broken no matter the order.

This change allows selecting the order of updates to facilitate the first three scenarios. Since most package sets only have loose version constraints, the reverse topological order will generally be the most convenient. In major package set updates like bumping GNOME release, there will be exceptions (e.g. libadwaita typically requires GTK 4 from the same release) but those were probably in broken order before as well.

The downside of this feature is that it is quite slow – it requires instantiating each package and then querying Nix store for requisites.
It may also fail to detect dependency if there are multiple variants of the package and dependant uses a different one than the canonical one.

Testing with:

    env GNOME_UPDATE_STABILITY=unstable NIX_PATH=nixpkgs=$HOME/Projects/nixpkgs nix-shell maintainers/scripts/update.nix --arg predicate '(path: pkg: path == ["gnome-shell"] || path == ["mutter"] || path == ["glib"] || path == ["gtk3"] || path == ["pango"] || path == ["gnome-text-editor"])' --argstr order reverse-topological --argstr commit true --argstr max-workers 4
This commit is contained in:
Jan Tojnar
2025-03-02 17:33:18 +01:00
parent 1d2c1810eb
commit ce96c79779
2 changed files with 222 additions and 13 deletions

View File

@@ -16,6 +16,7 @@
keep-going ? null,
commit ? null,
skip-prompt ? null,
order ? null,
}:
let
@@ -217,6 +218,18 @@ let
to skip prompt:
--argstr skip-prompt true
By default, the updater will update the packages in arbitrary order. Alternately, you can force a specific order based on the packages dependency relations:
- Reverse topological order (e.g. {"gnome-text-editor", "gimp"}, {"gtk3", "gtk4"}, {"glib"}) is useful when you want checkout each commit one by one to build each package individually but some of the packages to be updated would cause a mass rebuild for the others. Of course, this requires that none of the updated dependents require a new version of the dependency.
--argstr order reverse-topological
- Topological order (e.g. {"glib"}, {"gtk3", "gtk4"}, {"gnome-text-editor", "gimp"}) is useful when the updated dependents require a new version of updated dependency.
--argstr order topological
Note that sorting requires instantiating each package and then querying Nix store for requisites so it will be pretty slow with large number of packages.
'';
# Transform a matched package into an object for update.py.
@@ -241,7 +254,8 @@ let
lib.optional (max-workers != null) "--max-workers=${max-workers}"
++ lib.optional (keep-going == "true") "--keep-going"
++ lib.optional (commit == "true") "--commit"
++ lib.optional (skip-prompt == "true") "--skip-prompt";
++ lib.optional (skip-prompt == "true") "--skip-prompt"
++ lib.optional (order != null) "--order=${order}";
args = [ packagesJson ] ++ optionalArgs;