Introduction
I've been using NixOS for a while now, and my biggest issue was that I have a
lot of machines. Which leads to having a lot of different
systems on my
flake because of that hardware. The
normal solution would be to use the module system, but a new issue arises when
we add nix-darwin. We suddenly start to
fail
eval over issues because some modules don't exist that are in the normal NixOS
module system. So my new issue is that I can no longer use my small abstraction
over
lib.nixosSystem.
I would have to expand the abstraction to include
lib.darwinSystem
since I can no longer unconditionally import all modules I use if they don't
exist in nix-darwin? But what if I don't want to do that? What if I want to
write my own lib.nixosSystem or my own lib.darwinSystem? What if I call it
mkSystem.
Well that's exactly what I did. And this article covers how I got to that point
and how my custom “builder” later evolved into
easy-hosts.
The research
To start we should read the documentation for lib.evalModules.
From which we find that there are 4 arguments, but 3 that we really care about.
Those being modules, specialArgs and class. The modules argument is a
list of modules, which are files, functions or attrsets that will be merged and
then evaluated. The specialArgs argument is a attrset of arguments that are
passed to the modules, but are not evaluated with the file structure in mind.
The class argument is a nominal type that ensures that only compatible
modules are imported. This is a very important argument, as it allows us to
have different modules for different systems.
However, I'm not much a fan of reading documentation. So instead lets dig into
the source code and pick it apart. To find out what we need to get our custom
lib.nixosSystem working. To do this I identified 4 main files in the nixpkgs
repository:
These files may seem somewhat arbitrary, but they are listed in the order they
are called. The flake.nix file has our lib.nixosSystem function that calls
our nixos/lib/eval-config.nix file which is a light wrapper around our final
lib/modules.nix. So lets walk through those files in order and see what
exactly they do.
flake.nix
This file contains our lib.nixosSystem function, which takes args as an
argument. The
documentation
lists a set of known arguments being modules, specialArgs and
modulesLocation, it also specifies some additional legacy arguments system
and pkgs (both of which are now redundant).
The lib.nixosSystem then imports the nixos/lib/eval-config.nix file whilst
passing lib, and the remaining args to it. It also sets system to
null as well as adding nixpkgs.flake.source to nixpkgs output derivation.
nixos/lib/eval-config.nix
This file immediately points us to the fact that it is a “light wrapper” around
lib.evalModules. This file also has a large collection of arguments most of
which will be the defaults. A good example of this is baseModules which
defaults to a list of modules from the nixpkgs repo. The most important
arguments from this file are specialArgs, lib and modules. For the
most part these come from the prior flake.nix file.
As we read down the file we notice that there are two additional modules that
are going to be added. These are the pkgsModule and the modulesModule.
These appear to be pretty strange names at first, but the pkgsModule will
set nixpkgs.system if system was not null, and will set
nixpkgs.pkgs if pkgs is not null. The modulesModule will add
config._module.args as an attrset of noUserModules, baseModules,
extraModules and modules. So now we know some of the arguments that are
given to lib.evalModules lets see what that does.
nixos/lib/eval-config-minimal.nix
This file is a small wrapper upon lib.evalModules, but it gives us a little
bit of guidance on how to use the class argument. As well as showing us the
default for modulePath which is going to be passed as a special arg.
lib/modules.nix
This file won't add much to our understanding of lib.evalModules, but it
still is the definition of the function, so it's good to keep in mind if we have
any issues later down the line.
The implementation
Now that we have our key inputs of class, modules and specialArgs we can
start implementing our own lib.nixosSystem.
Getting the basics
Let us start in our very own flake.nix by writing the following code. This
will give us a basic template to work with and you, the reader, knows how to
start.
{
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
outputs = inputs: let
mkSystem = import ./mksystem.nix { inherit inputs; };
in {
nixosConfigurations = {
mySystem = mkSystem { };
};
};
}
Now that we have a very bare bone flake.nix we can get started on the
mkSystem function. Let's also create the mksystem.nix file. We are going to
add some basic args that we know we are going to need later such as modules,
specialArgs and class. And we are going to add some defaults to these args
such that we don't get any errors if they are not provided.
{
inputs ? throw "No inputs provided",
lib ? inputs.nixpkgs.lib,
...
}:
{
modules ? [ ],
specialArgs ? { },
class ? "nixos",
}: lib.evalModules {
inherit modules specialArgs class;
}
Adding modulesPath
Now that we have our basic template down. Let's start by adding the
modulesPath, most people probably recognize this from when they first
installed nix and read their hardware-configuration.nix file and saw
something along the lines of modulesPath + /installer/scan/not-detected.nix.
That is why we are starting with this, so that our hardware-configuration.nix
will work.
The key to this is going to be including modulesPath in our specialArgs.
This is because specialArgs should only be used with arguments that need to
be evaluated when resolving module structure.
{
inputs,
lib ? inputs.nixpkgs.lib,
...
}:
{
modules ? [ ],
specialArgs ? { },
class ? "nixos",
}: let
modulesPath = "${inputs.nixpkgs}/nixos/modules";
in
lib.evalModules {
inherit modules class;
# here we are merging the user provided specialArgs with the modulesPath
specialArgs = { inherit modulesPath; } // specialArgs;
}
So close and yet so far
Only adding modulePath is a bit useless however, so we can't exactly replace
our lib.nixosSystem's yet. So let's work on that. We are going to start by
importing baseModules, this will provide us with a base set of modules that
provide abstractions over configuring your system. We can use the modulePath
and get the module list.
{
inputs,
lib ? inputs.nixpkgs.lib,
...
}:
{
modules ? [ ],
specialArgs ? { },
class ? "nixos",
}: let
modulesPath = "${inputs.nixpkgs}/nixos/modules";
baseModules = import "${modulesPath}/module-list.nix";
in
lib.evalModules {
inherit class;
specialArgs = { inherit modulesPath; } // specialArgs;
modules = baseModules ++ modules;
}
It actually works?
We now have a mostly functional replacement. Depending on your configuration
may actually work as it is now! To keep progressing we are going to have to go
back to the modulesModule from earlier. we need this such that some nixpkgs
modules will work, one of these is the documentation module
which will be a hard module to ignore, when so many people use it.
To fix this we are going to introduce a new module which contains
config._module.args and takes a set of attrs that will be passed to each
module. I'm sure most of you recognize these when writing a module and adding
something like { pkgs, config, ... } to the top of a file. The
config._module.args option should be used when trying to pass arguments to
all modules, but should not be evaluated with file structure in mind.
{
inputs,
lib ? inputs.nixpkgs.lib,
...
}:
{
modules ? [ ],
specialArgs ? { },
class ? "nixos",
}: let
modulesPath = "${inputs.nixpkgs}/nixos/modules";
baseModules = import "${modulesPath}/module-list.nix";
in
lib.evalModules {
inherit class;
specialArgs = { inherit modulesPath; } // specialArgs;
modules = baseModules ++ modules ++ [
{
config._module.args = {
inherit baseModules modules;
};
}
];
}
Adding some of our own modules
Even better, now we have completely replaced lib.nixosSystem with our own
mkSystem function. But let's be real. That's not enough for us. We should
start abstracting some common themes between our systems. Some big examples of
this are networking.hostName and nixpkgs.hostPlatform. And while were at it
lets also re-add the nixpkgs.flake.source from the original lib.nixoSystem,
as well as adding inputs as a special arg. As most people do this anyway,
I think it's a safe assumption we should add it. For further reading about
passing inputs to modules check nobbz's blog on getting inputs to flake modules.
{
inputs,
lib ? inputs.nixpkgs.lib,
...
}:
name:
{
modules ? [ ],
specialArgs ? { },
class ? "nixos",
}: let
modulesPath = "${inputs.nixpkgs}/nixos/modules";
baseModules = import "${modulesPath}/module-list.nix";
in
lib.evalModules {
inherit class;
specialArgs = { inherit inputs modulesPath; } // specialArgs;
modules = baseModules ++ modules ++ [
{
config. = {
_module.args = {
inherit baseModules modules;
};
networking.hostName = name;
nixpkgs.flake.source = inputs.nixpkgs.outPath;
};
}
];
}
I just added name as an additional argument to our mkSystem function. This
allows us to set the hostname of our system. The way I opted to write it allows
for us to use mapAttrs on our nixosConfigurations. This will mean that we
need to change how the original flake.nix works though.
{
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
outputs = inputs: let
mkSystem = import ./mksystem.nix { inherit inputs; };
in {
nixosConfigurations = builtins.mapAttrs mkSystem {
mySystem = { };
};
};
}
Furthermore, notice how I lied about settings nixpkgs.hostPlatform. If your curious why, maybe you
should read my last blog post about it. (Shameless plug)
Darwin compatibility
One of the main reasons I wanted to make this was to support
lib.darwinSystem. So let's address that now.
To introduce Darwin support we are allowing users to set the class
argument to darwin from there we can determine what modules to import.
As a result of this you may notice that Darwin has a different set of
modules which introduced some new options to set for this system type. This
includes nixpkgs.source and darwinVersionSuffix and darwinRevision. Some
of these are for commands like darwin-version.
You may also notice that we had to add system = eval.config.system.build.toplevel
back into the final eval produced by our Darwin eval. This is required to swap
to the configuration, otherwise it won't work at all. This is because the
system output is used by darwin-rebuild, to identify the final build
output.
{
inputs,
lib ? inputs.nixpkgs.lib,
...
}:
name:
{
modules ? [ ],
specialArgs ? { },
class ? "nixos",
}: let
# this is new? what is it?
# I'm glad you asked, this is a nice way of checking if we have our darwin and nixpkgs inputs
nixpkgs = inputs.nixpkgs or (throw "No nixpkgs input found");
darwin = inputs.darwin or inputs.nix-darwin or (throw "No nix-darwin input found");
modulesPath = if class == "darwin" then "${darwin}/modules" else "${nixpkgs}/nixos/modules";
baseModules = import "${modulesPath}/module-list.nix";
eval = lib.evalModules {
inherit class;
specialArgs = { inherit inputs modulesPath; } // specialArgs;
modules = baseModules ++ modules ++ [
{
config. = {
_module.args = {
inherit baseModules modules;
};
networking.hostName = name;
nixpkgs.flake.source = nixpkgs.outPath;
};
}
] ++ lib.optionals (class == "darwin") [
{
config = {
nixpkgs.source = nixpkgs.outPath;
system = {
checks.verifyNixPath = false;
darwinVersionSuffix = ".${darwin.shortRev or darwin.dirtyShortRev or "dirty"}";
darwinRevision = darwin.rev or darwin.dirtyRev or "dirty";
};
};
}
];
};
in
if class == "darwin" then (eval // { system = eval.config.system.build.toplevel; }) else eval;
The final touch
The final and maybe the best bit is adding inputs'. For those who are unaware
of flake-parts, you probably are not aware of the greatness that is inputs'.
The diff below shows the advantage of using inputs' over inputs for
accessing packages.
- inputs.input-name.packages.${pkgs.stdenv.hostPlatform.system}.package-name
+ inputs'.input-name.packages.package-name
Is that not awesome? So how can we replicate that for ourselves?
What we will need to do is map over all inputs, and their outputs and select
the output dependent on the host platform, if a system dependent output exists,
otherwise it will leave it as is. We can achieve that with the following code:
inputs' = lib.mapAttrs (_: lib.mapAttrs (_: v: v.${config.nixpkgs.hostPlatform} or v)) inputs;
Or if you are using flake-parts, you may prefer using the following code instead:
withSystem config.nixpkgs.hostPlatform ({ inputs', ... }: { inherit inputs'; });
So let's add that to our mkSystem function.
{
inputs,
lib ? inputs.nixpkgs.lib,
...
}:
name:
{
modules ? [ ],
specialArgs ? { },
class ? "nixos",
}: let
nixpkgs = inputs.nixpkgs or (throw "No nixpkgs input found");
darwin = inputs.darwin or inputs.nix-darwin or (throw "No nix-darwin input found");
modulesPath = if class == "darwin" then "${darwin}/modules" else "${nixpkgs}/nixos/modules";
baseModules = import "${modulesPath}/module-list.nix";
eval = lib.evalModules {
inherit class;
specialArgs = { inherit inputs modulesPath; } // specialArgs;
modules = baseModules ++ modules ++ [
({ config, ... }: {
config = {
_module.args = {
inherit baseModules modules;
inputs' = lib.mapAttrs (_: lib.mapAttrs (_: v: v.${config.nixpkgs.hostPlatform} or v)) inputs;
};
networking.hostName = name;
nixpkgs.flake.source = nixpkgs.outPath;
};
})
] ++ lib.optionals (class == "darwin") [
{
config = {
nixpkgs.source = nixpkgs.outPath;
system = {
checks.verifyNixPath = false;
darwinVersionSuffix = ".${darwin.shortRev or darwin.dirtyShortRev or "dirty"}";
darwinRevision = darwin.rev or darwin.dirtyRev or "dirty";
};
};
}
];
};
in
if class == "darwin" then
(eval // { system = eval.config.system.build.toplevel; })
else
eval;
Conclusion
And that's it! We have a fully functional mkSystem function that can replace
both lib.nixosSystem and lib.darwinSystem. This was quite the task, and
although this blog post seems to reduce the quite simple. I've spent a lot of
time on this, both when researching how to create the custom builder and
writing and maintaining the latest rendition in the form of a flake module
called easy-hosts. If you enjoyed
this post, please consider donating on ko-fi
or github sponsors.
]]>
