nixos/grafana: update instructions on secret rotation

Closes #495278

Turns out that the documentation about secrets[1] is actually a little
misleading and I let myself fool by it: the rotation is about moving
to envelope-based encryption, i.e. a way where a key is generated and
stored in the DB that is encrypted with `secret_key` and used to
decrypt actual secrets.

In fact, there's no official way to rotate this on a running instance.
The reason this wasn't noticed so far is because I'd expect most people
to

* use provisioning to set up their datasources (or)
* talk to a local Prometheus (or else) API that is exposed on the
  machine only and thus no authentication is needed if you trust that
  machine enough.

On top, the encryption is unauthenticated so only changing secret_key
appears to work, but uses gibberish as password for authentication
against data-sources.

So effectively there are two ways to migrate (short of a fresh setup):

* keep the old secret and call it a day. IMHO this is fine if the setup
  is e.g. a single-node machine where both the secret and the DB are on
  the same file-system, in the same data-dir and protected by the same
  permissions.

* use a 3rd-party rotation tool[2]. I ensured by hand on a test-instance
  that following the instructions works and you end up with a freshly
  generated secret_key that you need to deploy. With that key deployed,
  the credentials still decrypt to the same plaintext.

[1] https://grafana.com/docs/grafana/latest/setup-grafana/configure-security/configure-database-encryption/#envelope-encryption
[2] https://github.com/erooke/grafana-secretkey-rotation-tool/tree/d9dc788902fa5185e15cb15ce6129f7237ab6138

nixos/doc/manual/release-notes/rl-2605.section.md

@@ -121,11 +121,17 @@ of pulling the upstream container image from Docker Hub. If you want the old beh
 - `services.oauth2-proxy.clientSecret` and `services.oauth2-proxy.cookie.secret` have been replaced with `services.oauth2-proxy.clientSecretFile` and `services.oauth2-proxy.cookie.secretFile` respectively. This was done to ensure secrets don't get made world-readable.
 
 - [`services.grafana.settings.security.secret_key`](#opt-services.grafana.settings.security.secret_key) doesn't have a
-  default value anymore. Please generate your own key or hard-code the old one explicitly.
+  default value anymore. Please generate your own key or hard-code the old one ("SW2YcwTIb9zpOOhoPsMm") explicitly.
   See the [upstream docs](https://grafana.com/docs/grafana/latest/setup-grafana/configure-grafana/#secret_key) and
   the [instructions on how to rotate](https://grafana.com/docs/grafana/latest/setup-grafana/configure-security/configure-database-encryption/#re-encrypt-secrets)
   for further information.
 
+  Please do note that there's no official way to rotate. On a single-node instance with the database and the secret-key being
+  on the same filesystem with the same permissions for Grafana only to read it's most likely OK to keep using the old key.
+
+  If you need to rotate, a [3rd-party tool, `grafana-secretkey-rotation-tool`](https://github.com/erooke/grafana-secretkey-rotation-tool/tree/d9dc788902fa5185e15cb15ce6129f7237ab6138) is a tested option.
+  When using a secret for this value, make sure to use [Grafana's variable expansion to inject secrets](https://grafana.com/docs/grafana/latest/setup-grafana/configure-grafana/#variable-expansion).
+
 - Ethercalc and its associated module have been removed, as the package is unmaintained and cannot be installed from source with npm now.
 
 - `services.cgit` before always had the git-http-backend and its "export all" setting enabled, which sidestepped any access control configured in cgit's settings. Now you have to make a decision and either enable or disable `services.cgit.gitHttpBackend.checkExportOkFiles` (or disable the git-http-backend).

nixos/modules/services/monitoring/grafana.nix

@@ -2070,6 +2070,11 @@ in
           for more information.
 
           See https://grafana.com/docs/grafana/latest/setup-grafana/configure-security/configure-database-encryption/#re-encrypt-secrets on how to re-encrypt.
+
+          As stated in the NixOS changelog for 26.05, there's no official way to rotate.
+          Either hard-code the old key ("SW2YcwTIb9zpOOhoPsMm") if your setup doesn't have any secrets in the DB that need
+          special protection or perform a rotation with a 3rd-party tool
+          (https://github.com/erooke/grafana-secretkey-rotation-tool/tree/d9dc788902fa5185e15cb15ce6129f7237ab6138).
         '';
       }
     ];