Make sure that JIT is actually available when using
services.postgresql = {
enable = true;
enableJIT = true;
package = pkgs.postgresql_15;
};
The current behavior is counter-intuitive because the docs state that
`enableJIT = true;` is sufficient even though it wasn't in that case
because the declared package doesn't have the LLVM dependency.
Fixed by using `package.withJIT` if `enableJIT = true;` and
`package.jitSupport` is `false`.
Also updated the postgresql-jit test to test for that case.
This is useful if your postgresql version is dependant on
`system.stateVersion` and not pinned down manually. Then it's not
necessary to find out which version exactly is in use and define
`package` manually, but just stay with what NixOS provides as default:
$ nix-instantiate -A postgresql
/nix/store/82fzmb77mz2b787dgj7mn4a8i4f6l6sn-postgresql-14.7.drv
$ nix-instantiate -A postgresql_jit
/nix/store/qsjkb72fcrrfpsszrwbsi9q9wgp39m50-postgresql-14.7.drv
$ nix-instantiate -A postgresql.withJIT
/nix/store/qsjkb72fcrrfpsszrwbsi9q9wgp39m50-postgresql-14.7.drv
$ nix-instantiate -A postgresql.withJIT.withoutJIT
/nix/store/82fzmb77mz2b787dgj7mn4a8i4f6l6sn-postgresql-14.7.drv
I.e. you can use postgresql with JIT (for complex queries only[1]) like
this:
services.postgresql = {
enable = true;
enableJIT = true;
};
Performing a new override instead of re-using the `_jit`-variants for
that has the nice property that overlays for the original package apply
to the JIT-enabled variant, i.e.
with import ./. {
overlays = [
(self: super: {
postgresql = super.postgresql.overrideAttrs (_: { fnord = "snens"; });
})
];
};
postgresql.withJIT.fnord
still gives the string `snens` whereas `postgresql_jit` doesn't have the
attribute `fnord` in its derivation.
[1] https://www.postgresql.org/docs/current/runtime-config-query.html#GUC-JIT-ABOVE-COST
Closes#150801
Note: I decided against resuming directly on #150801 because the
conflict was too big (and resolving it seemed too error-prone to me).
Also the `this`-refactoring could be done in an easier manner, i.e. by
exposing JIT attributes with the correct configuration. More on that
below.
This patch creates variants of the `postgresql*`-packages with JIT[1]
support. Please note that a lot of the work was derived from previous
patches filed by other contributors, namely dasJ, andir and abbradar,
hence the co-authored-by tags below.
Effectively, the following things have changed:
* For JIT variants an LLVM-backed stdenv with clang is now used as
suggested by dasJ[2]. We need LLVM and CLang[3] anyways to build the
JIT-part, so no need to mix this up with GCC's stdenv. Also, using the
`dev`-output of LLVM and clang's stdenv for building (and adding llvm
libs as build-inputs) seems more cross friendly to me (which will
become useful when cross-building for JIT-variants will actually be
supported).
* Plugins inherit the build flags from the Makefiles in
`$out/lib/pgxs/src` (e.g. `-Werror=unguarded-availability-new`). Since
some of the flags are clang-specific (and stem from the use of the
CLang stdenv) and don't work on gcc, the stdenv of `pkgs.postgresql`
is passed to the plugins. I.e., plugins for non-JIT variants are built
with a gcc stdenv on Linux and plugins for JIT variants with a clang
stdenv.
Since `plv8` hard-codes `gcc` as `$CC` in its Makefile[4], I marked it
as broken for JIT-variants of postgresql only.
* Added a test-matrix to confirm that JIT works fine on each
`pkgs.postgresql_*_jit` (thanks Andi for the original test in
#124804!).
* For each postgresql version, a new attribute
`postgresql_<version>_jit` (and a corresponding
`postgresqlPackages<version>JitPackages`) are now exposed for better
discoverability and prebuilt artifacts in the binary cache.
* In #150801 the `this`-argument was replaced by an internal recursion.
I decided against this approach because it'd blow up the diff even
more which makes the readability way harder and also harder to revert
this if necessary.
Instead, it is made sure that `this` always points to the correct
variant of `postgresql` and re-using that in an additional
`.override {}`-expression is trivial because the JIT-variant is
exposed in `all-packages.nix`.
* I think the changes are sufficiently big to actually add myself as
maintainer here.
* Added `libxcrypt` to `buildInputs` for versions <v13. While
building things with an LLVM stdenv, these versions complained that
the extern `crypt()` symbol can't be found. Not sure what this is
exactly about, but since we want to switch to libxcrypt for `crypt()`
usage anyways[5] I decided to add it. For >=13 it's not relevant
anymore anyways[6].
* JIT support doesn't work with cross-compilation. It is attempted to
build LLVM-bytecode (`%.bc` is the corresponding `make(1)`-rule) for
each sub-directory in `backend/` for the JIT apparently, but with a
$(CLANG) that can produce binaries for the build, not the host-platform.
I managed to get a cross-build with JIT support working with
`depsBuildBuild = [ llvmPackages.clang ] ++ buildInputs`, but
considering that the resulting LLVM IR isn't platform-independent this
doesn't give you much. In fact, I tried to test the result in a VM-test,
but as soon as JIT was used to optimize a query, postgres would
coredump with `Illegal instruction`.
A common concern of the original approach - with llvm as build input -
was the massive increase of closure size. With the new approach of using
the LLVM stdenv directly and patching out references to the clang drv in
`$out` the effective closure size changes are:
$ nix path-info -Sh $(nix-build -A postgresql_14)
/nix/store/kssxxqycwa3c7kmwmykwxqvspxxa6r1w-postgresql-14.7 306.4M
$ nix path-info -Sh $(nix-build -A postgresql_14_jit)
/nix/store/xc7qmgqrn4h5yr4vmdwy56gs4bmja9ym-postgresql-14.7 689.2M
Most of the increase in closure-size stems from the `lib`-output of
LLVM
$ nix path-info -Sh /nix/store/5r97sbs5j6mw7qnbg8nhnq1gad9973ap-llvm-11.1.0-lib
/nix/store/5r97sbs5j6mw7qnbg8nhnq1gad9973ap-llvm-11.1.0-lib 349.8M
which is why this shouldn't be enabled by default.
While this is quite much because of LLVM, it's still a massive
improvement over the simple approach of adding llvm/clang as
build-inputs and building with `--with-llvm`:
$ nix path-info -Sh $(nix-build -E '
with import ./. {};
postgresql.overrideAttrs ({ configureFlags ? [], buildInputs ? [], ... }: {
configureFlags = configureFlags ++ [ "--with-llvm" ];
buildInputs = buildInputs ++ [ llvm clang ];
})' -j0)
/nix/store/i3bd2r21c6c3428xb4gavjnplfqxn27p-postgresql-14.7 1.6G
Co-authored-by: Andreas Rammhold <andreas@rammhold.de>
Co-authored-by: Janne Heß <janne@hess.ooo>
Co-authored-by: Nikolay Amiantov <ab@fmap.me>
[1] https://www.postgresql.org/docs/current/jit-reason.html
[2] https://github.com/NixOS/nixpkgs/pull/124804#issuecomment-864616931
& https://github.com/NixOS/nixpkgs/pull/150801#issuecomment-1467868321
[3] This fails with the following error otherwise:
```
configure: error: clang not found, but required when compiling --with-llvm, specify with CLANG=
```
[4] https://github.com/plv8/plv8/blob/v3.1.5/Makefile#L14
[5] https://github.com/NixOS/nixpkgs/pull/181764
[6] c45643d618
This is not needed anymore because the version is EOL for almost a year
now and we don't even have the packages anymore, only the attributes for
compatibility for upgrades from older NixOS versions.
What the code was trying to do was helpfully add a directory and
extension if none were specified, but it did this by checking whether
the filename was composed of a very limited character set that didn't
even include dashes.
With this change, the intention of the code is clearer, and I can put
dashes in my screenshot names.
The keyd package already exists, but without a systemd service.
Keyd requires write access to /var/run to create its socket. Currently
the directory it uses can be changed with an environment variable, but
the keyd repo state suggests that this may turn into a compile-time
option. with that set, and some supplementary groups added, we can run
the service under DynamicUser.
Co-authored-by: pennae <82953136+pennae@users.noreply.github.com>
The single option tries to do too much work, which just ends up confusing people.
So:
- don't force the console font, the kernel can figure this out as of #210205
- don't force the systemd-boot mode, it's an awkward mode that's not supported
on most things and will break flicker-free boot
- add a separate option for the xorg cursor scaling trick and move it under the xorg namespace
- add a general `fonts.optimizeForVeryHighDPI` option that explicitly says what it does
- alias the old option to that
- don't set any of those automatically in nixos-generate-config
