Introduction
I’ve been using NixOS for a while now, and my biggest issue was that I have a lot of machines. Which leads to having a lot of different systems on my flake because of that hardware. The normal solution would be to use the module system, but a new issue arises when we add nix-darwin. We suddenly start to fail eval over issues because some modules don’t exist that are in the normal NixOS module system. So my new issue is that I can no longer use my small abstraction over lib.nixosSystem
. I would have to expand the abstraction to include lib.darwinSystem
since I can no longer unconditionally import all modules I use if they don’t exist in nix-darwin? But what if I don’t want to do that? What if I want to write my own lib.nixosSystem
or my own lib.darwinSystem
? What if I call it mkSystem
. Well that’s exactly what I did. And this article covers how I got to that point and how my custom “builder” later evolved into easy-hosts.
The research
To start we should read the documentation for lib.evalModules
. From which we find that there are 4 arguments, but 3 that we really care about. Those being modules
, specialArgs
and class
. The modules
argument is a list of modules, which are files, functions or attrsets that will be merged and then evaluated. The specialArgs
argument is a attrset of arguments that are passed to the modules, but are not evaluated with the file structure in mind. The class
argument is a nominal type that ensures that only compatible modules are imported. This is a very important argument, as it allows us to have different modules for different systems.
However, I’m not much a fan of reading documentation. So instead lets dig into the source code and pick it apart. To find out what we need to get our custom lib.nixosSystem
working. To do this I identified 4 main files in the nixpkgs repository:
These files may seem somewhat arbitrary, but they are listed in the order they are called. The flake.nix
file has our lib.nixosSystem
function that calls our nixos/lib/eval-config.nix
file which is a light wrapper around our final lib/modules.nix
. So lets walk through those files in order and see what exactly they do.
flake.nix
This file contains our lib.nixosSystem
function, which takes args
as an argument. The documentation lists a set of known arguments being modules
, specialArgs
and modulesLocation
, it also specifies some additional legacy arguments system
and pkgs
(both of which are now redundant).
The lib.nixosSystem
then imports the nixos/lib/eval-config.nix
file whilst passing lib
, and the remaining args
to it. It also sets system
to null
as well as adding nixpkgs.flake.source
to nixpkgs output derivation.
nixos/lib/eval-config.nix
This file immediately points us to the fact that it is a “light wrapper” around lib.evalModules
. This file also has a large collection of arguments most of which will be the defaults. A good example of this is baseModules
which defaults to a list of modules from the nixpkgs repo. The most important arguments from this file are specialArgs
, lib
and modules
. For the most part these come from the prior flake.nix
file.
As we read down the file we notice that there are two additional modules that are going to be added. These are the pkgsModule
and the modulesModule
. These appear to be pretty strange names at first, but the pkgsModule
will set nixpkgs.system
if system
was not null, and will set nixpkgs.pkgs
if pkgs
is not null. The modulesModule
will add config._module.args
as an attrset of noUserModules
, baseModules
, extraModules
and modules
. So now we know some of the arguments that are given to lib.evalModules
lets see what that does.
nixos/lib/eval-config-minimal.nix
This file is a small wrapper upon lib.evalModules
, but it gives us a little bit of guidance on how to use the class
argument. As well as showing us the default for modulePath
which is going to be passed as a special arg.
lib/modules.nix
This file won’t add much to our understanding of lib.evalModules
, but it still is the definition of the function, so it’s good to keep in mind if we have any issues later down the line.
The implementation
Now that we have our key inputs of class
, modules
and specialArgs
we can start implementing our own lib.nixosSystem
.
Getting the basics
Let us start in our very own flake.nix
by writing the following code. This will give us a basic template to work with and you, the reader, knows how to start.
{ inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
outputs = inputs: let mkSystem = import ./mksystem.nix { inherit inputs; }; in { nixosConfigurations = { mySystem = mkSystem { }; }; };}
Now that we have a very bare bone flake.nix
we can get started on the mkSystem
function. Let’s also create the mksystem.nix
file. We are going to add some basic args that we know we are going to need later such as modules
, specialArgs
and class
. And we are going to add some defaults to these args such that we don’t get any errors if they are not provided.
{ inputs ? throw "No inputs provided", lib ? inputs.nixpkgs.lib, ...}:{ modules ? [ ], specialArgs ? { }, class ? "nixos",}: lib.evalModules { inherit modules specialArgs class;}
Adding modulesPath
Now that we have our basic template down. Let’s start by adding the modulesPath
, most people probably recognize this from when they first installed nix and read their hardware-configuration.nix
file and saw something along the lines of modulesPath + /installer/scan/not-detected.nix
. That is why we are starting with this, so that our hardware-configuration.nix
will work.
The key to this is going to be including modulesPath
in our specialArgs
. This is because specialArgs
should only be used with arguments that need to be evaluated when resolving module structure.
{ inputs, lib ? inputs.nixpkgs.lib, ...}:{ modules ? [ ], specialArgs ? { }, class ? "nixos",}: let modulesPath = "${inputs.nixpkgs}/nixos/modules";inlib.evalModules { inherit modules class;
# here we are merging the user provided specialArgs with the modulesPath specialArgs = { inherit modulesPath; } // specialArgs;}
So close and yet so far
Only adding modulePath
is a bit useless however, so we can’t exactly replace our lib.nixosSystem
’s yet. So let’s work on that. We are going to start by importing baseModules
, this will provide us with a base set of modules that provide abstractions over configuring your system. We can use the modulePath
and get the module list.
{ inputs, lib ? inputs.nixpkgs.lib, ...}:{ modules ? [ ], specialArgs ? { }, class ? "nixos",}: let modulesPath = "${inputs.nixpkgs}/nixos/modules";
baseModules = import "${modulesPath}/module-list.nix";inlib.evalModules { inherit class;
specialArgs = { inherit modulesPath; } // specialArgs;
modules = baseModules ++ modules;}
It actually works?
We now have a mostly functional replacement. Depending on your configuration may actually work as it is now! To keep progressing we are going to have to go back to the modulesModule
from earlier. we need this such that some nixpkgs modules will work, one of these is the documentation module which will be a hard module to ignore, when so many people use it.
To fix this we are going to introduce a new module which contains config._module.args
and takes a set of attrs that will be passed to each module. I’m sure most of you recognize these when writing a module and adding something like { pkgs, config, ... }
to the top of a file. The config._module.args
option should be used when trying to pass arguments to all modules, but should not be evaluated with file structure in mind.
{ inputs, lib ? inputs.nixpkgs.lib, ...}:{ modules ? [ ], specialArgs ? { }, class ? "nixos",}: let modulesPath = "${inputs.nixpkgs}/nixos/modules";
baseModules = import "${modulesPath}/module-list.nix";inlib.evalModules { inherit class;
specialArgs = { inherit modulesPath; } // specialArgs;
modules = baseModules ++ modules ++ [ { config._module.args = { inherit baseModules modules; }; } ];}
Adding some of our own modules
Even better, now we have completely replaced lib.nixosSystem
with our own mkSystem
function. But let’s be real. That’s not enough for us. We should start abstracting some common themes between our systems. Some big examples of this are networking.hostName
and nixpkgs.hostPlatform
. And while were at it lets also re-add the nixpkgs.flake.source
from the original lib.nixoSystem
, as well as adding inputs
as a special arg. As most people do this anyway, I think it’s a safe assumption we should add it. For further reading about passing inputs to modules check nobbz’s blog on getting inputs to flake modules.
{ inputs, lib ? inputs.nixpkgs.lib, ...}:name:{ modules ? [ ], specialArgs ? { }, class ? "nixos",}: let modulesPath = "${inputs.nixpkgs}/nixos/modules";
baseModules = import "${modulesPath}/module-list.nix";inlib.evalModules { inherit class;
specialArgs = { inherit inputs modulesPath; } // specialArgs;
modules = baseModules ++ modules ++ [ { config. = { _module.args = { inherit baseModules modules; };
networking.hostName = name;
nixpkgs.flake.source = inputs.nixpkgs.outPath; }; } ];}
I just added name
as an additional argument to our mkSystem
function. This allows us to set the hostname of our system. The way I opted to write it allows for us to use mapAttrs
on our nixosConfigurations
. This will mean that we need to change how the original flake.nix
works though.
{ inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
outputs = inputs: let mkSystem = import ./mksystem.nix { inherit inputs; }; in { nixosConfigurations = builtins.mapAttrs mkSystem { mySystem = { }; }; };}
Furthermore, notice how I lied about settings nixpkgs.hostPlatform
. If your curious why, maybe you should read my last blog post about it. (Shameless plug)
Darwin compatibility
One of the main reasons I wanted to make this was to support lib.darwinSystem
. So let’s address that now.
To introduce Darwin support we are allowing users to set the class
argument to darwin
from there we can determine what modules to import. As a result of this you may notice that Darwin has a different set of modules which introduced some new options to set for this system type. This includes nixpkgs.source
and darwinVersionSuffix
and darwinRevision
. Some of these are for commands like darwin-version
.
You may also notice that we had to add system = eval.config.system.build.toplevel
back into the final eval produced by our Darwin eval. This is required to swap to the configuration, otherwise it won’t work at all. This is because the system
output is used by darwin-rebuild
, to identify the final build output.
{ inputs, lib ? inputs.nixpkgs.lib, ...}:name:{ modules ? [ ], specialArgs ? { }, class ? "nixos",}: let # this is new? what is it? # I'm glad you asked, this is a nice way of checking if we have our darwin and nixpkgs inputs nixpkgs = inputs.nixpkgs or (throw "No nixpkgs input found"); darwin = inputs.darwin or inputs.nix-darwin or (throw "No nix-darwin input found");
modulesPath = if class == "darwin" then "${darwin}/modules" else "${nixpkgs}/nixos/modules";
baseModules = import "${modulesPath}/module-list.nix";
eval = lib.evalModules { inherit class;
specialArgs = { inherit inputs modulesPath; } // specialArgs;
modules = baseModules ++ modules ++ [ { config. = { _module.args = { inherit baseModules modules; };
networking.hostName = name;
nixpkgs.flake.source = nixpkgs.outPath; }; } ] ++ lib.optionals (class == "darwin") [ { config = { nixpkgs.source = nixpkgs.outPath;
system = { checks.verifyNixPath = false;
darwinVersionSuffix = ".${darwin.shortRev or darwin.dirtyShortRev or "dirty"}"; darwinRevision = darwin.rev or darwin.dirtyRev or "dirty"; }; }; } ]; };in if class == "darwin" then (eval // { system = eval.config.system.build.toplevel; }) else eval;
The final touch
The final and maybe the best bit is adding inputs'
. For those who are unaware of flake-parts, you probably are not aware of the greatness that is inputs'
. The diff below shows the advantage of using inputs'
over inputs
for accessing packages.
inputs.input-name.packages.${pkgs.stdenv.hostPlatform.system}.package-nameinputs'.input-name.packages.package-name
Is that not awesome? So how can we replicate that for ourselves?
What we will need to do is map over all inputs, and their outputs and select the output dependent on the host platform, if a system dependent output exists, otherwise it will leave it as is. We can achieve that with the following code:
inputs' = lib.mapAttrs (_: lib.mapAttrs (_: v: v.${config.nixpkgs.hostPlatform} or v)) inputs;
Or if you are using flake-parts, you may prefer using the following code instead:
withSystem config.nixpkgs.hostPlatform ({ inputs', ... }: { inherit inputs'; });
So let’s add that to our mkSystem
function.
{ inputs, lib ? inputs.nixpkgs.lib, ...}:name:{ modules ? [ ], specialArgs ? { }, class ? "nixos",}: let nixpkgs = inputs.nixpkgs or (throw "No nixpkgs input found"); darwin = inputs.darwin or inputs.nix-darwin or (throw "No nix-darwin input found");
modulesPath = if class == "darwin" then "${darwin}/modules" else "${nixpkgs}/nixos/modules";
baseModules = import "${modulesPath}/module-list.nix";
eval = lib.evalModules { inherit class;
specialArgs = { inherit inputs modulesPath; } // specialArgs;
modules = baseModules ++ modules ++ [ ({ config, ... }: { config = { _module.args = { inherit baseModules modules;
inputs' = lib.mapAttrs (_: lib.mapAttrs (_: v: v.${config.nixpkgs.hostPlatform} or v)) inputs; };
networking.hostName = name;
nixpkgs.flake.source = nixpkgs.outPath; }; }) ] ++ lib.optionals (class == "darwin") [ { config = { nixpkgs.source = nixpkgs.outPath;
system = { checks.verifyNixPath = false;
darwinVersionSuffix = ".${darwin.shortRev or darwin.dirtyShortRev or "dirty"}"; darwinRevision = darwin.rev or darwin.dirtyRev or "dirty"; }; }; } ]; };in if class == "darwin" then (eval // { system = eval.config.system.build.toplevel; }) else eval;
Conclusion
And that’s it! We have a fully functional mkSystem
function that can replace both lib.nixosSystem
and lib.darwinSystem
. This was quite the task, and although this blog post seems to reduce the quite simple. I’ve spent a lot of time on this, both when researching how to create the custom builder and writing and maintaining the latest rendition in the form of a flake module called easy-hosts. If you enjoyed this post, please consider donating on ko-fi or github sponsors.