From 7b2c5ede9acc5894b09d7ae0219bb77b6eeb2b5b Mon Sep 17 00:00:00 2001
From: zimbatm <zimbatm@zimbatm.com>
Date: Thu, 16 Apr 2026 14:43:33 +0200
Subject: [PATCH] pkgs-lib/formats: use yj instead of remarshal for
 toml.generate
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

`remarshal` is a Python application; evaluating its derivation forces a
large slice of `python3Packages` (rich, ruamel-yaml, cbor2, tomlkit and
their transitive build/check inputs — ~150 distinct python-modules,
~2700 derivations). 95 NixOS modules use `formats.toml`, so any system
enabling one of those services pays this on every `nixos-rebuild`.

`yj` is a small Go binary that does the same JSON→TOML conversion. For
all values accepted by `(formats.toml{}).type` the output is
semantically identical (verified via `tomllib`).

Reproducer (best of 3, `NIX_SHOW_STATS=1 nix-instantiate -E
'(pkgs.formats.toml{}).generate "x" {a.b=1;}'`):

                        cpuTime  nrFunctionCalls   nrThunks  drv-closure
    before (remarshal)    0.69s          634,117  1,184,499   2722 paths
    after  (yj)           0.36s          271,308    566,981   1481 paths
                           -49%             -57%       -52%         -46%

Runtime closure of the converter: remarshal 202.3 MiB → yj 5.2 MiB.

Differences in output:
- yj emits intermediate table headers for deeply-nested keys (`[a]`
  `[a.b]` `[a.b.c]` rather than just `[a.b.c]`). Valid TOML, parses
  identically.
- yj renders very small/large floats as decimals rather than scientific
  notation (`1e-300` → 300-digit literal). Valid TOML, parses
  identically.
- Direct callers of `generate` that pass `null` (which the format type
  already rejects) get key-dropped instead of a build-time error.
  NixOS modules are unaffected since the option type filters this at
  eval time; several modules already `filterAttrs (v: v != null)` for
  exactly this reason.

Tested:
- `nix-build pkgs/pkgs-lib/tests -A formats` passes
- arrays of tables, nested tables, string escaping, int64, unicode
  keys/values, mixed scalar+table siblings round-trip through `tomllib`
  identically to remarshal output
---
 pkgs/pkgs-lib/formats.nix       | 6 +++---
 pkgs/pkgs-lib/tests/formats.nix | 2 ++
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/pkgs/pkgs-lib/formats.nix b/pkgs/pkgs-lib/formats.nix
index 46a0dfcfcfe9..9478a11f83f6 100644
--- a/pkgs/pkgs-lib/formats.nix
+++ b/pkgs/pkgs-lib/formats.nix
@@ -461,16 +461,16 @@ optionalAttrs allowAliases aliases
       generate =
         name: value:
         pkgs.callPackage (
-          { runCommand, remarshal }:
+          { runCommand, yj }:
           runCommand name
             {
-              nativeBuildInputs = [ remarshal ];
+              nativeBuildInputs = [ yj ];
               value = builtins.toJSON value;
               passAsFile = [ "value" ];
               preferLocalBuild = true;
             }
             ''
-              json2toml "$valuePath" "$out"
+              yj -jt < "$valuePath" > "$out"
             ''
         ) { };
 
diff --git a/pkgs/pkgs-lib/tests/formats.nix b/pkgs/pkgs-lib/tests/formats.nix
index c50a92174436..d51b5d2f272b 100644
--- a/pkgs/pkgs-lib/tests/formats.nix
+++ b/pkgs/pkgs-lib/tests/formats.nix
@@ -701,6 +701,8 @@ runBuildTests {
       [attrs]
       foo = "foo"
 
+      [level1]
+      [level1.level2]
       [level1.level2.level3]
       level4 = "deep"
     '';