linux: new workflow for kernel changes
See https://github.com/NixOS/nixpkgs/issues/437208#issuecomment-3288623669 Depends on https://github.com/NixOS/org/pull/172 As documented below, the idea is to essentially group all changes rebuilding all VM tests with kernel updates and merge them together into `master` whenever the Linux kernels get updated. This documents the workflow of updates in the nixpkgs manual. While at it, I removed the README from the packages because * it's horribly outdated * I didn't even know it exists which confirms that its discoverability was very poor and added the relevant portions into the nixpkgs manual as well.
This commit is contained in:
Maximilian Bosch
2
.github/workflows/periodic-merge-6h.yml
vendored
2
.github/workflows/periodic-merge-6h.yml
vendored
@@ -35,6 +35,8 @@ jobs:
|
||||
into: staging-next
|
||||
- from: staging-next
|
||||
into: staging
|
||||
- from: master
|
||||
into: staging-nixos
|
||||
uses: ./.github/workflows/periodic-merge.yml
|
||||
with:
|
||||
from: ${{ matrix.pairs.from }}
|
||||
|
||||
@@ -430,20 +430,22 @@ gitGraph
|
||||
|
||||
Here's an overview of the different branches:
|
||||
|
||||
| branch | `master` | `staging-next` | `staging` |
|
||||
| --- | --- | --- | --- |
|
||||
| Used for development | ✔️ | ❌ | ✔️ |
|
||||
| Built by Hydra | ✔️ | ✔️ | ❌ |
|
||||
| [Mass rebuilds][mass-rebuild] | ❌ | ⚠️ Only to fix Hydra builds | ✔️ |
|
||||
| Critical security fixes | ✔️ for non-mass-rebuilds | ✔️ for mass-rebuilds | ❌ |
|
||||
| Automatically merged into | `staging-next` | `staging` | - |
|
||||
| Manually merged into | - | `master` | `staging-next` |
|
||||
| branch | `master` | `staging-next` | `staging` | [`staging-nixos`](#test-driver-rebuild) |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| Used for development | ✔️ | ❌ | ✔️ | ✔️ |
|
||||
| Built by Hydra | ✔️ | ✔️ | ❌ | ❌ |
|
||||
| [Mass rebuilds][mass-rebuild] | ❌ | ⚠️ Only to fix Hydra builds | ✔️ | ❌[^1] |
|
||||
| Critical security fixes | ✔️ for non-mass-rebuilds | ✔️ for mass-rebuilds | ❌ | ✔️ |
|
||||
| Automatically merged into | `staging-next` & `staging-nixos` | `staging` | - | - |
|
||||
| Manually merged into | - | `master` | `staging-next` | `master` |
|
||||
|
||||
The staging workflow is used for all stable branches with corresponding names:
|
||||
- `master`/`release-YY.MM`
|
||||
- `staging`/`staging-YY.MM`
|
||||
- `staging-next`/`staging-next-YY.MM`
|
||||
|
||||
[^1]: Except changes that cause no more rebuilds than kernel updates
|
||||
|
||||
# Conventions
|
||||
|
||||
## Branch conventions
|
||||
@@ -495,6 +497,26 @@ In order to help the decision, CI automatically assigns [`rebuild` labels](https
|
||||
As a rule of thumb, if the number of rebuilds is **500 or more**, consider targeting the `staging` branch instead of `master`; if the number is **1000 or more**, the pull request causes a mass rebuild, and should target the `staging` branch.
|
||||
See [previously merged pull requests to the staging branches](https://github.com/NixOS/nixpkgs/issues?q=base%3Astaging+-base%3Astaging-next+is%3Amerged) to get a sense for what changes are considered mass rebuilds.
|
||||
|
||||
Please note that changes to the Linux kernel are an exception to this rule.
|
||||
These PRs go to `staging-nixos`, see [the next section for more context](#changes-rebuilding-all-tests).
|
||||
|
||||
### Changes rebuilding all NixOS tests
|
||||
[test-driver-rebuild]: #changes-rebuilding-all-tests
|
||||
|
||||
Changes causing a rebuild of all NixOS tests get a special [`10.rebuild-nixos-tests`](https://github.com/NixOS/nixpkgs/issues?q=state%3Aopen%20label%3A10.rebuild-nixos-tests) label.
|
||||
These changes pose a significant impact on the build infrastructure.
|
||||
|
||||
Hence, these PRs should either target a `staging`-branch or `staging-nixos`, provided one of following conditions applies:
|
||||
|
||||
* The label `10.rebuild-nixos-tests` is set, or
|
||||
* The PR is a change affecting the Linux kernel.
|
||||
|
||||
The branch gets merged whenever mainline kernel updates or critical security fixes land on the branch.
|
||||
This usually happens on a weekly basis.
|
||||
|
||||
Backports are not handled by such a branch.
|
||||
The relevant PRs from this branch must be backported manually.
|
||||
|
||||
## Commit conventions
|
||||
[commit-conventions]: #commit-conventions
|
||||
|
||||
|
||||
@@ -8,6 +8,7 @@ const typeConfig = {
|
||||
release: ['development', 'primary'],
|
||||
staging: ['development', 'secondary'],
|
||||
'staging-next': ['development', 'secondary'],
|
||||
'staging-nixos': ['development', 'secondary'],
|
||||
'haskell-updates': ['development', 'secondary'],
|
||||
nixos: ['channel'],
|
||||
nixpkgs: ['channel'],
|
||||
@@ -19,6 +20,7 @@ const orderConfig = {
|
||||
master: 0,
|
||||
release: 1,
|
||||
staging: 2,
|
||||
'staging-nixos': 2,
|
||||
'haskell-updates': 3,
|
||||
'staging-next': 4,
|
||||
}
|
||||
|
||||
@@ -105,3 +105,56 @@ $ make -C $dev/lib/modules/*/build M=$(pwd)/drivers/net/ethernet/mellanox module
|
||||
```
|
||||
|
||||
:::
|
||||
|
||||
## Maintainer information {#sec-linux-kernel-maintainer-information}
|
||||
|
||||
### Updating kernels {#sec-linux-updates}
|
||||
|
||||
Updating all kernels can be done with the following script:
|
||||
|
||||
```ShellSession
|
||||
$ pkgs/os-specific/linux/kernel/update.sh
|
||||
```
|
||||
|
||||
The change gets submitted like this:
|
||||
|
||||
* File a PR against `staging-nixos`.
|
||||
* Add a `backport release-XX.XX` label for an automated backport.
|
||||
We don't expect many other changes on that branch to require a backport, hence there's no such branch for stable.
|
||||
By using an additional PR, we get the automatic backport against stable without manual cherry-picks.
|
||||
* Merge into `staging-nixos`.
|
||||
* File as PR from `staging-nixos` against `master`.
|
||||
* When all status checks are green, merge.
|
||||
|
||||
### Add a new (major) version of the Linux kernel {#sec-linux-add-new-kernel-version}
|
||||
|
||||
* When running `./pkgs/os-specific/linux/kernel/update.sh`, new kernel majors get discovered automatically.
|
||||
* Prepare all Nix expressions for the new kernel
|
||||
* Instantiate the new kernel in `pkgs/top-level/linux-kernels.nix` in the `kernels`-section.
|
||||
```nix
|
||||
{
|
||||
linux_X_Y = callPackage ../os-specific/linux/kernel/mainline.nix {
|
||||
branch = "X.Y";
|
||||
kernelPatches = [
|
||||
# any new patches required (it makes to look which patches are used by its predecessor)
|
||||
];
|
||||
};
|
||||
}
|
||||
```
|
||||
* Instantiate the package-set in `vanillaPackages`:
|
||||
```nix
|
||||
{
|
||||
linux_X_Y = recurseIntoAttrs (packagesFor kernels.linux_X_Y);
|
||||
}
|
||||
```
|
||||
* Update `linux_latest` to the new attribute.
|
||||
* __SQUASH__ the changes into the `linux: init at …` commit.
|
||||
* If a new hardened is available:
|
||||
* Instantiate a `linux_X_Y_hardened = hardenedKernelsFor kernels.linux_X_Y { };` in `kernels` and
|
||||
`linux_X_Y_hardened = hardenedKernelFor kernels.linux_X_Y { };` in the `packages`-section.
|
||||
* Make sure to remove the hardened variant of the previous kernel version unless it's LTS.
|
||||
We only support the latest and latest LTS version of hardened.
|
||||
* If no new hardened kernel is available:
|
||||
* Keep the previously latest kernel until its mainline counterpart gets removed.
|
||||
After that `linux_hardened` points to the latest LTS supported by hardened.
|
||||
* __SQUASH__ the changes into the `linux_X_Y_hardened: init at …` commit.
|
||||
|
||||
@@ -4306,6 +4306,15 @@
|
||||
"sec-linux-kernel": [
|
||||
"index.html#sec-linux-kernel"
|
||||
],
|
||||
"sec-linux-kernel-maintainer-information": [
|
||||
"index.html#sec-linux-kernel-maintainer-information"
|
||||
],
|
||||
"sec-linux-updates": [
|
||||
"index.html#sec-linux-updates"
|
||||
],
|
||||
"sec-linux-add-new-kernel-version": [
|
||||
"index.html#sec-linux-add-new-kernel-version"
|
||||
],
|
||||
"ex-overriding-kernel-derivation": [
|
||||
"index.html#ex-overriding-kernel-derivation"
|
||||
],
|
||||
|
||||
@@ -1,47 +0,0 @@
|
||||
# How to add a new (major) version of the Linux kernel to Nixpkgs:
|
||||
|
||||
1. Copy the old Nix expression (e.g., `linux-2.6.21.nix`) to the new one (e.g., `linux-2.6.22.nix`) and update it.
|
||||
|
||||
2. Add the new kernel to the `kernels` attribute set in [`linux-kernels.nix`](../../../top-level/linux-kernels.nix) (e.g., create an attribute `kernel_2_6_22`).
|
||||
|
||||
3. Update the kernel configuration:
|
||||
|
||||
1. While in the Nixpkgs repository, enter the development shell for that kernel:
|
||||
|
||||
```console
|
||||
$ nix-shell -A linuxKernel.kernels.linux_2_6_22
|
||||
```
|
||||
|
||||
2. Unpack the kernel:
|
||||
|
||||
```console
|
||||
[nix-shell]$ pushd $(mktemp -d)
|
||||
[nix-shell]$ unpackPhase
|
||||
```
|
||||
|
||||
3. For each supported platform (`i686`, `x86_64`, `uml`) do the following:
|
||||
|
||||
1. Make a copy from the old config (e.g., `config-2.6.21-i686-smp`) to the new one (e.g., `config-2.6.22-i686-smp`).
|
||||
|
||||
2. Copy the config file for this platform (e.g., `config-2.6.22-i686-smp`) to `.config` in the unpacked kernel source tree.
|
||||
|
||||
3. Run `make oldconfig ARCH={i386,x86_64,um}` and answer all questions. (For the uml configuration, also add `SHELL=bash`.) Make sure to keep the configuration consistent between platforms (i.e., don’t enable some feature on `i686` and disable it on `x86_64`).
|
||||
|
||||
4. If needed, you can also run `make menuconfig`:
|
||||
|
||||
```ShellSession
|
||||
$ nix-shell -p ncurses pkg-config
|
||||
$ make menuconfig ARCH=arch
|
||||
```
|
||||
|
||||
5. Copy `.config` over the new config file (e.g., `config-2.6.22-i686-smp`).
|
||||
|
||||
4. Test building the kernel:
|
||||
|
||||
```ShellSession
|
||||
nix-build -A linuxKernel.kernels.kernel_2_6_22
|
||||
```
|
||||
|
||||
If it compiles, ship it! For extra credit, try booting NixOS with it.
|
||||
|
||||
5. It may be that the new kernel requires updating the external kernel modules and kernel-dependent packages listed in the `linuxPackagesFor` function in `linux-kernels.nix` (such as the NVIDIA drivers, AUFS, etc.). If the updated packages aren’t backwards compatible with older kernels, you may need to keep the older versions around.
|
||||
Reference in New Issue
Block a user