+++
date = "2021-11-22T09:55:32+00:00"
publishdate = "2023-12-29T07:08:55+00:00"
title = "NixOS Pins and Needles"
slug = "nixos-pins-and-needles"
author = "Thedro"
tags = ["nix"]
type = "posts"
summary =  "NixOS is a Linux distribution built around the nix package manager."
draft =  ""
syntax =  "1"
toc =  ""
updated =  ""
+++

![Three terminals arranged on a NixOS desktop
workspace](/images/nixos-pins-and-needles.png " A terminal desktop workspace on NixOS.")

[`NixOS`](https://nixos.org/) is a
[Linux distribution](https://en.wikipedia.org/wiki/Linux_distribution) built
around the `nix` package manager. Since
[writing my first article on `NixOS`,](https://www.thedroneely.com/posts/running-nixos-linux-containers/)
it has quickly become my personal `OS` (Operating System) of choice for desktops,
servers, and {{< sidenote mark="routers." set="left" >}} Replaced my
[pfSense](https://github.com/pfsense/pfsense#overview) and
[OPNsense](https://github.com/opnsense/core#opnsense-gui-and-system-management)
routers with `NixOS`. {{< /sidenote >}} The transition process to become
_almost_ as {{< sidenote mark="productive" set="right" >}} Good old cowboy style
system administration. {{< /sidenote >}} as my imperative environments has been
marred by a lot of pins and needles on the usability front, but it has been
worth it.

For context, you can use a single `configuration.nix` to build machines which is
super convenient, but my inventory is diverse and has at least
[five machines.](https://www.thedroneely.com/posts/nixops-towards-the-final-frontier/#nixops-deployment-and-commands)
Configuring these machines to play nice together ultimately means having to
control complexity --- and that's not always fun.

This is a fairly opinionated and under the hood take on using `NixOS`, you
shouldn't take any of this as advice --- I'm not a
[functional](https://en.wikipedia.org/wiki/Functional_programming) or
[`LISP`](<https://en.wikipedia.org/wiki/Lisp_(programming_language)>)
programming expert. _Lasciate ogni speranza, voi ch'entrate._

## Testing Single Files Quickly

My inventory includes some low end hardware so rebuilding a small
subset of machines is {{< sidenote mark="slow" set="right" >}} What software
today isn't slow? Fast feedback loops are hard to create nowadays.
{{< /sidenote >}} in `NixOS`. This
[hacky script of mine](https://github.com/tdro/dotfiles/blob/2eb824499ed7b4d76b29e5068c0d4e0c2f83adaa/.local/bin/nixos-test#L1)
dry builds single files in isolation and foregoes the need of rebuilding entire
configuration sets (with some trade--offs), saving precious time. Before, I'd
usually have to wait for a complete
[`nixos-rebuild`](https://www.mankier.com/8/nixos-rebuild) or
[`nixops`](https://github.com/NixOS/nixops/blob/3128b4ca31fe1e7930ce67a115eb131aa1b0b57d/doc/manual/overview.rst#overview)
evaluation, only to discover missing `imports` or other syntax errors. Single
file evaluation allows working on small sections of a larger set of
configurations quickly without rebuilding or activating machines.

{{< video poster="/images/nixos-pins-and-needles-nixos-test.png" source="/videos/nixos-pins-and-needles-nixos-test.mp4" options="loop muted" width="854" >}}
Using my hacky script `nixos-test` to quickly evaluate small files and even a
full configuration. {{< /video >}}

[`GNU` `Guix`](https://guix.gnu.org/en/about/) (as in _geeks_) is currently
superior to `NixOS` in this regard with commands like
[`guix system`](https://guix.gnu.org/manual/en/html_node/Invoking-guix-system.html)
that provide this type of fast inspection. If `Guix` is interesting to you,
{{< sidenote mark="check" set="left" >}}
[Emacs](https://www.gnu.org/software/emacs/) and
[Guile Scheme](https://www.gnu.org/software/guile/). {{< /sidenote >}} out this
[cool video introduction.](https://odysee.com/@SystemCrafters:e/5-reasons-to-try-guix-in-2022:f)

## Searching For Packages and Files

In most Linux distributions you can search to see which package owns a file. For
example, on [`Arch Linux`](https://wiki.archlinux.org/title/Arch_Linux) you
would run `pacman -Fy <file>` to search the
{{< sidenote mark="Arch" set="right" >}} Random Aside: It appears that
[Valve's Steam Deck](https://www.steamdeck.com/en/tech#specs) is based on
[Arch Linux](https://archlinux.org/). That's interesting. {{< /sidenote >}}
database for a package that contains the `<file>` you're seeking. This is great
when [compiling](https://www.thedroneely.com/posts/installing-isso-from-source/)
or writing a program. On `NixOS`, my preference is to use
[`nix-locate`](https://github.com/bennofs/nix-index#readme) to find packages and
files. `NixOS`
[package search](https://search.nixos.org/packages?channel=unstable&from=0&size=50&sort=relevance&type=packages&query=git)
and naming conventions can be unintuitive. To ease my pain, the ideas of
[this shell script](https://github.com/Shopify/comma/blob/85f8d8dfe5c1d773ab2e0b1768d5004b515adc81/%2C#L1)
are adapted for my specific purposes. My
[hacky version](https://github.com/tdro/dotfiles/blob/3a733d76f1155a04461e31e7ff6270256f6a335a/.local/bin/%2C#L1)
sets up an alternate home environment, drops the desired program into a
temporary
[`nix-shell`](https://nixos.org/manual/nix/unstable/command-ref/nix-shell.html)
with [`fish`](https://fishshell.com/) and prints some useful paths afterwards.

{{< video poster="/images/nixos-pins-and-needles-package-searching.png" source="/videos/nixos-pins-and-needles-package-searching.mp4" options="loop muted" width="854" >}}
Poking around [VLC's](https://www.videolan.org/vlc/) package files.
{{< /video >}}

## Running Configurations

`NixOS` comes with powerful
[virtual machine integration testing modules.](https://nix.dev/tutorials/integration-testing-using-virtual-machines)
I'm a bit more lazy and like to jump directly into a virtual machine from my
shell with [`QEMU`](https://www.qemu.org/) (sometimes called Quick Emulator) to
try out a configuration. The
[`nixos-generate`](https://www.mankier.com/8/nixos-generate-config) command
generates different system output formats for a specified configuration. The
[`ISO`](https://en.wikipedia.org/wiki/Optical_disc_image)
{{< sidenote mark="format" set="right" >}} Optical disc image. {{< /sidenote >}}
is my favorite. Often times I'll spin up a system in the cloud as a read only
`ISO`.

```shell
$ nixos-generate --format iso --configuration server.nix --option builders ''
$ qemu-system-x86_64 -nographic -enable-kvm -m 1024 -cdrom nixos.iso
```

{{< video poster="/images/nixos-pins-and-needles-running-configurations.png" source="/videos/nixos-pins-and-needles-running-configurations.mp4" options="loop muted" width="854" >}}
Trying out a configuration with a virtual machine. {{< /video >}}

## DRY is Evil Incarnate

The functional ecosystem has taught me that
{{< sidenote mark="clever" set="right" >}} I'm not that smart anyway.
{{< /sidenote >}} programming isn't worth the pain.
[_Don't repeat yourself_](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself)
(`DRY`) _is_ the devil incarnate in my book, especially in a functional
programming context. After throwing away my
[first directory structure,](https://www.thedroneely.com/posts/nixops-towards-the-final-frontier/#directory-structure)
I've since implemented what may be a kind of naive
[stratified design approach](https://www.youtube.com/watch?v=GbZpTHg0KfQ) to
prevent myself from wasting {{< sidenote mark="time" set="left" >}} Sometimes
coming back to my module configurations leave me baffled. {{< /sidenote >}}
pulling useful logic out of deep abstractions or forcing
[my poorly](https://www.thedroneely.com/posts/writing-nixos-modules-and-switching-to-cgit/#module-interface)
crafted [`NixOS` modules](https://nixos.wiki/wiki/Module) to work for a purpose
that it was not designed for.

[`Nixpkgs,`](https://github.com/NixOS/nixpkgs) the `NixOS` implementation, lets
you spin up system abstractions in your sleep and while this is good for writing
packages {{< sidenote mark="quickly," set="right" >}} The domain specific
tooling of `NixOS` and `Guix` is amazing for writing massive amounts of system
packages quickly. {{< /sidenote >}} if your hastily made abstractions are poor
--- you'll be banished to decomposing the world. My self made rule now is that
anything that does not require `imports` and does something useful is a basic
knowledge piece. Every singular piece of knowledge is saved, and whether used or
not can be combined to create higher functionality independent of itself. These
knowledge pieces should be easy to change, check, and reuse. For that simple
reason, duplication is good, really really good, and as a bonus --- sharing
working snippets with others is easier.

{{< video poster="/images/nixos-pins-and-needles-no-dry.png" source="/videos/nixos-pins-and-needles-no-dry.mp4" options="loop muted" width="854" >}}
WET: Write everything twice. {{< /video >}}

## Modules Should Do One Thing

Or rather --- it seems useful to distinguish between lower level modules
(modules containing [`systemd`](https://wiki.archlinux.org/title/systemd)
modules only) and higher level modules (modules containing multiple modules).
The high level interfacing inside a module is often too hard to pick apart.
Situations can arise where you try to do something with a module that is out of
its scope. The reality is that a lot of `NixOS` modules have two or more smaller
modules struggling to get out. Sometimes you'll have to reduce upstream modules
down to its basics to get what you want done --- this can be painful.

My use cases typically require running multiple copies of the same module. The
easiest way to achieve this is to keep custom made modules as small as possible
(just a `systemd` service and basic configuration). It's impossible to satisfy
all use cases so everything else ends up as a test case of combinations that can
be used as higher level modules. "Lower" level modules are easier to understand
and manipulate too, since they are just an indirect version of plain `systemd`
files.

{{< video poster="/images/nixos-pins-and-needles-modules.png" source="/videos/nixos-pins-and-needles-modules.mp4" options="loop muted" width="854" >}}
The `lib.attrsets.mapAttrs'` function is great for duplicating `systemd`
services. {{< /video >}}

## Separate System and User Environments

As a user, if I can avoid a `nixos-rebuild`, `nixops` deploy,
[`sudo`,](https://man.archlinux.org/man/sudo.8.en) or
[`doas`](https://man.archlinux.org/man/doas.1.en) --- sign me up. My user
packages are split from the system side with
[`config.nix`](https://www.thedroneely.com/git/thedroneely/dotfiles/tree/.config/nixpkgs/config.nix#n1)
abuse and
[`nix-env`.](https://nixos.org/manual/nix/unstable/command-ref/nix-env.html) The
`nix` community's
[home manager](https://github.com/nix-community/home-manager#readme) is nice,
but avoided because the less dependencies the better when connecting my home
folder to another Linux distribution. My programming style incorporates the
[Unix shell](https://en.wikipedia.org/wiki/Unix_shell) mostly as an
[integrated development environment](https://en.wikipedia.org/wiki/Integrated_development_environment)
(`IDE`), so my
[`dotfiles`](https://www.thedroneely.com/git/thedroneely/dotfiles/tree/) and
default presets for linting and such are all that's necessary. The new
[`nix flake`](https://nixos.org/manual/nix/unstable/command-ref/new-cli/nix3-flake.html#flake-format)
feature puts in the guard rails for proper reproducibility, but I don't think
I'll be using that for some time.

## Imperative to Declarative User Experience

Once in a while I'll take a look at the discussions on social media around the
`NixOS`, `Guix`, and functional operating system community. Recent discussions
seem to center around
[the lack of popularity of` NixOS` and `GNU` `Guix`](https://old.reddit.com/r/linuxquestions/comments/qqh55d/why_arent_systems_like_nixos_and_gnu_guix_more/)
and their complexity.

The unpopularity of `NixOS` and `GNU Guix` sounds about right. Functional Linux
distributions will not become _generally_ popular anytime
{{< sidenote mark="soon" set="left" >}} Sometimes getting popular too fast has
more drawbacks than benefits anyway. {{< /sidenote >}} due to misaligned
developer and user experience expectations. Popularity may be limited to
self--hosting, [developer operations,](https://en.wikipedia.org/wiki/DevOps) and
system administration circles.

I'm fairly certain that `nixos-generate-config` is the command with the best
user experience (`UX`) on the whole system --- everything else is pins and
needles. `NixOS` needs more imperative to declarative
{{< sidenote mark="workflows." set="right" >}} End users don't want long form
"documentation" --- what they _really_ want is either copy and paste style short
documentation, or imperative commands that generate plug and play configuration.
In the end, as the joke goes --- users are programmers who believe anything you
say. {{< /sidenote >}} The `NixOS` install procedure achieves this workflow with
two separate commands. The first command generates a declarative configuration
and the second implements it. If distribution installation speed runs were a
thing, then `NixOS` would have a leg up especially if a `configuration.nix` is
baked into a custom installer with automatic formatting options like
`fileSystems.<name>.autoFormat`.

```shell
$ nixos-generate-config --root /mnt
$ nixos-install
```

In my experience, a casual `NixOS` user doesn't even want to open the
`configuration.nix` to manually write stuff and connect modules together unless
absolutely necessary. In their world --- configuration blocks should simply be
"generated" so that they can just copy and paste things.

> "Why can't it just generate the configs like it did to install my system?"
{{< footer >}}A casual user{{< /footer >}}

[Docker,](https://github.com/docker/compose#readme)
[Kubernetes,](https://github.com/kubernetes/kubernetes#readme)
[Tailwind,](https://github.com/tailwindlabs/tailwindcss#readme) and
[React](https://github.com/facebook/react#readme) all have the same
{{< sidenote mark="design" set="left" >}} Synergy --- but I like to call it copy
and paste--ability. Applications with high copy and paste--ability are always
popular. {{< /sidenote >}} incentives that ensure popularity. It's not
necessarily the underlying technology that makes a software ecosystem popular or
"good", but the ease of the higher level tooling to create, share, and copy from
a near endless library of pre--defined documents, templates and components. In
short, it's very little effort for great reward --- that's the nature of the
beast. Most users abhor starting _from scratch_ and because of that it's sort of
easy to anticipate what will become popular --- `NixOS` and `GNU Guix` are not
there yet. Searching the
[`NixOS` options index](https://search.nixos.org/options) to build up
configuration is _almost_ like starting from scratch in a `docker` compose.

## Conclusion

Future iterations of `NixOS`, `Guix` and its derivatives can only get more
robust. {{< sidenote mark="Immutability" set="left" >}} Unchanging state.
{{< /sidenote >}} may be rejected in most real world applications, but
{{< sidenote mark="declarativity" set="left" >}} Describing what a result should
be. {{< /sidenote >}} will definitely become the name of the game.
{{< sidenote mark="Imperative" set="right" >}} Describing how you want to get to
a result. {{< /sidenote >}} styles will undoubtedly always be around.

If easily composed and shared declarative tooling (infrastructure as code)
becomes the dominant paradigm in user space, then issues in the Linux desktop
space can be resolved elegantly by lassoing the majority of desktop
configuration types and quirks. The
[`NixOS` hardware quirks repository](https://github.com/NixOS/nixos-hardware) is
brilliant.

Interestingly enough, the `nixpkgs` repository is a very
{{< sidenote mark="accessible" set="left" >}} Hosted on
[GitHub.](https://github.com/NixOS/nixpkgs/issues) {{< /sidenote >}} way of
getting a general overview of the current state of the upstream software
development world. Every dirty hack, every non--standard project setup, every
network call during installation --- the sandbox sees it all.

Finally, The `nix` language is just great in my opinion, without functions, it
reads like [`JSON`](https://www.json.org/json-en.html) (`JavaScript` Object
Notation) with comments minus the dangling comma shenanigans. It can be
[used as metadata](https://www.thedroneely.com/posts/nixops-towards-the-final-frontier/#nixops-and-nix-channel)
or as a language. If I could ever manage to get my `NixOS` configuration under
control (specifically secrets and metadata), I'd open source it, but for the
time being, I just don't have the time.