In my last post, I shared how to get a working instance of Nitter deployed on NixOS, but requested advice on how to best automatically provision the guest_accounts.json runtime secret file on the target server.

A number of folks reached out to me on Mastodon (thanks @[email protected], @[email protected], @[email protected] and @[email protected]!) to suggest that I use agenix to copy encrypted files to the server and decrypt them in non-world readable directories, and then use systemd’s LoadCredentials option to make them available to the nitter service.

Honestly, it took me a while to understand what I was reading on the agenix README page, which is why I thought I’d do this additional technical write-up specifically for people who need to use agenix for the first time to provision runtime secrets for a systemd service.

Installing Agenix and Making File References⌗

The first step is to add the input to our flake.nix file and add the agenix binary the your system, this part is easy enough to follow along with on the official README.

Next, we need to create a secrets.nix file; the README suggests creating this in a secrets subdirectory, so we’ll go with that.

let
  nitter_server_key = "ssh-rsa AAAA....";
  keys = [nitter_server_key];
in {
  "guest_accounts.json.age".publicKeys = keys;
}

In this file we add the SSH public key of a corresponding private key that we expect to be somewhere on the target server (these are typically already generated for us thanks to the default value of services.openssh.hostKeys).

If you have multiple keys on multiple machines, you can give them different variable names and collect them all in the keys list (if you will be working with secrets that need to be deployed to multiple machines).

Finally, we associate those public keys to a file reference. Notice that we haven’t actually created any encrypted files yet. We are just making a reference, and stating that the private keys that correspond to the given public keys can be used to decrypt whatever file is eventually associated with that reference.

Creating Encrypted Files⌗

Once we have the reference for a file that will be titled guest_accounts.json.age, we can run agenix -e guest_accounts.json.age, which will make our $EDITOR open.

Here we can paste in our various Twitter guest account JSON objects. Once done, we save the file and exit the editor. We now have an encrypted guest_accounts.json.age file! Make sure to git add it.

Getting the Encrypted Files on a NixOS Server⌗

In our flake.nix file we can start by putting together a little helper:

{
  agenixSecrets = {
    userHome ? "/home/<YOUR_MOST_COMMON_USERNAME>",
    files,
  }: {
    age = {
      identityPaths = [
        "/etc/ssh/ssh_host_ed25519_key"
        "/etc/ssh/ssh_host_rsa_key"
        "${userHome}/.ssh/id_rsa"
      ];

      secrets = files;
    };
  };
}

This is a Nix function which takes a userHome optional argument, which has a default, and files, which is an object where we link our file references in secrets.nix to actual encrypted files created by the agenix -e command.

By default, age.identityPaths is populated with the keys created by services.openssh.hostKeys, which are one RSA key and one ed25519 key. I have another SSH RSA key that I typically use for my main user account which migrates with me from machine to machine which I also add to this list right at the end.

This is built with the userHome variable because this typically differs on Linux and macOS machines, with the format of the latter being /Users/<YOUR_USERNAME>. This lets us always default to Linux but gives us the choice to add an override if we ever want to pass a secret encrypted with agenix to a macOS machine.

{
  modules = [
    ./machines/remote-server.nix
    (agenixSecrets {
      files = {
        "guest_accounts.json".file = ./secrets/guest_accounts.json.age;
      };
    })
  ];
}

We can call this agenixSecrets helper in the modules of our target server definition, omitting the userHome argument if it’s unnecessary, and passing the files objects which references our newly created guest_accounts.json.age encrypted file.

The key on the left hand side is the name of the file that will be outputted on the server (this can be whatever you want!), and we set the file property on that key to the path of the encrypted file in our flake repository.

The process so far looks like this:

  • Step 1: "guest_accounts.json.age" reference in secrets.nix => linked with one or more public keys
  • Step 2: guest_accounts.json.age file => encrypted for the public keys linked in step 1 with the agenix -e guest_accounts.json.age command
  • Step 3: "guest_accounts.json" in flake.nix => linked with the encrypted files created in step 2

This means that the file that will be added to the /nix/store on the target server will still be encrypted. This is important because /nix/store is world-readable, and typically we don’t want every user to be able to dig in there and find secrets they shouldn’t have access to.

If we build and apply these changes to the target server now, we will find our decrypted file present at /run/agenix/guest_accounts.json. Progress!

Passing Encrypted Files to a Systemd Service⌗

Now, for the final piece of the puzzle: making the guest_accounts.json file available to our nitter systemd service. We can go back to the snippet from the previous post and update it like this:

{
  systemd.services.nitter.serviceConfig.LoadCredential = [
    "guest_account.json:${config.age.secrets."guest_accounts.json".path}"
  ];

  systemd.services.nitter.serviceConfig.Environment = [
    "NITTER_CONF_FILE=/var/lib/private/nitter/nitter.conf"
    "NITTER_ACCOUNTS_FILE=%d/guest_account.json"
  ];
}

First we want to override the serviceConfig of services.nitter to add the LoadCredential option. Most NixOS services run using systemd’s DynamicUser option, which means that they won’t have access to files owned by root in /run/agenix.

We use LoadCredential to tell systemd to load the credential at the path on the right hand side of the : to a file accessible by the DynamicUser of this service with the filename given on the left hand side.

Then, we can finally update the NITTER_ACCOUNTS_FILE environment variable to point to this file. %d is a templating feature provided by systemd that will always resolve to the directory where any loaded credentials are placed. This is typically a directory like /run/credentials/your-service.service, but the less hard-coding we have to do, the better.

We are now ready to apply all of these changes, and have a working instance nitter running on our NixOS server with an automatically provisioned runtime secret file!

Just don’t forget to remove the file that you manually placed in the /var/lib/private/nitter.service directory if you followed along with the previous post.

If you have any questions you can reach out to me on Twitter and Mastodon.

If you’re interested in what I read to come up with solutions like this one, you can subscribe to my Software Development RSS feed.

If you’d like to watch me writing code while explaining what I’m doing, you can also subscribe to my YouTube channel.