+++
date = "2020-05-27T05:26:17+00:00"
publishdate = "2023-12-29T07:08:55+00:00"
title = "Declarative User Package Management in NixOS"
slug = "declarative-user-package-management-in-nixos"
author = "Thedro"
tags = ["nix"]
type = "posts"
summary =  "NixOS is a Linux distribution built around the Nix package manager. The configuration of the entire system is declarative allowing for immutability and reproducibility."
draft =  ""
syntax =  "1"
toc =  ""
updated =  "2022-02-20"
+++

{{< image source="/images/declarative-user-package-management-in-nixos.png" title="NixOS" >}}
NixOS {{< /image >}}

[NixOS](https://nixos.org/) is a Linux distribution built around the `nix`
package manager. The configuration of the entire system is
{{< sidenote mark="declarative" set="right" >}}In theory, NixOS is the easiest
Linux distribution to use because of its declarative nature. In
theory...{{< /sidenote>}} allowing for immutability and reproducibility.

Let's take a look at managing a user's package environment declaratively in
NixOS version `20.03`. The {{< sidenote mark="nature" set="left" >}}The `nix`
expression language is not particularly difficult. It's the scope and
flexibility of the abstraction layer that makes the initial NixOS learning
experience **painful**.{{< /sidenote>}} of NixOS dictates that there are
numerous declarative methods. We can use attribute sets, build environments, or
even community {{< sidenote mark="tools" set="right" >}}Home manager is a system
that manages a user's environment. It's similar to the system's
`configuration.nix` setup.{{< /sidenote>}} like
[home manager.](https://github.com/nix-community/home-manager#home-manager-using-nix)

## Declarative Package Management

We will
[combine](https://nixos.org/nixpkgs/manual/#sec-declarative-package-management)
the command 
[`nix-env`](https://nixos.org/manual/nix/unstable/command-ref/nix-env.html#description) 
with the user's `config.nix` to achieve a declarative
package setup. The user specific configuration is located at
`~/.config/nixpkgs/config.nix`. Create this file if it does not exist.

The user's `nixpkgs` configuration `config.nix` places us in a context 
{{< sidenote mark="similar" set="left" >}}
This may be
[familiar](https://nixos.org/nixpkgs/manual/#chap-packageconfig) to those who
configure the system from `configuration.nix`.
{{< /sidenote>}} 
to `nixpkgs.config`. Populate `config.nix` with multiple build environments and
install [packages](https://search.nixos.org/packages) using `nix-env`. This can
be achieved more optimally with [`nix
flake`](https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake.html#flake-format)
or you could use
[`nix-shell`](https://nixos.org/manual/nix/stable/command-ref/nix-shell.html#description)
for projects and temporary environments. The following is a basic `config.nix`
that makes 
{{< sidenote mark="use" set="right" >}}
Beware of package name collisions that result in infinite recursion. Notice
that `Awesome` (with a capital) is used instead of `awesome` in one of the
overrides.
{{< /sidenote>}}
of [package
overriding.](https://nixos.org/guides/nix-pills/nixpkgs-overriding-packages.html#idm140737319627440)
Check out my `config.nix`
[here.](https://www.thedroneely.com/git/thedroneely/dotfiles/tree/.config/nixpkgs)

```nix {options="hl_lines=4"}
{
  allowUnfree = true;

  packageOverrides = pkgs: with pkgs; {

    Awesome = pkgs.buildEnv {
      name = "awesome";
      paths = [
        awesome
        lxappearance
        paper-gtk-theme
      ];
    };

    Golang = pkgs.buildEnv {
      name = "golang";
      paths = [
        go
      ];
    };

    PHP = pkgs.buildEnv {
      name = "php";
      paths = [
        php74
      ];
    };

    C = pkgs.buildEnv {
      name = "c";
      paths = [
        gnumake
        meson
        ninja
        gcc
      ];
    };
  };
}
```

Install the above build sets in `config.nix` using `nix-env -Air`. The `-A` flag
is for the attribute path, `-i` to install the packages and `-r` for removing
all {{< sidenote mark="undeclared" set="right" >}}This is
[like](https://nixos.wiki/index.php?title=Cheatsheet&useskin=vector) running an
`apt install` or `pacman -Syu` that intelligently erases all user packages
before installing.{{< /sidenote>}} packages.

```shell
nix-env -Air nixos.Awesome nixos.Golang nixos.PHP nixos.C
```

Distribution independent installations of `nix` default to the `nixpkgs`
{{< sidenote mark="prefix" set="left" >}}Each package contains the prefix
`nixos`. This is the default NixOS channel containing the packages. Run
`nix-channel --list` to view the current user's channel
prefixes.{{< /sidenote>}} as the default channel name.

```shell
nix-env -Air nixpkgs.Awesome nixpkgs.Golang nixpkgs.PHP nixpkgs.C
```

Querying the installed packages with `nix-env -q` shows that there are four
packages installed each containing the programs declared in `config.nix`.

```shell
$ nix-env -q
awesome
c
golang
php
```

One could exploit the recursive aspect and group packages further by any metric.
Below we add another set that groups the above declarations by machine.

```nix {options="hl_lines=4 6-9"}
{
  allowUnfree = true;

  packageOverrides = pkgs: with pkgs; rec {

    Machine1 = pkgs.buildEnv {
      name = "machine1";
      paths = [ Awesome Golang PHP C ];
    };

    # Package declarations for Awesome, Golang, PHP, & C ...
  };
}
```

{{< sidenote mark="Installing" set="right" >}} This should evaluate quicker than
the first install command. {{< /sidenote>}} and querying this package with
`nix-env` shows that there is one package installed containing a program set of
other program sets declared within `config.nix`.

```shell
$ nix-env -Air nixos.Machine1
$ nix-env -q
machine1
```

This is immensely beneficial for use cases that prioritize keeping `root` system
build times {{< sidenote mark="small." set="left" >}} Faster configuration
switches. {{< /sidenote>}} Users can install packages independent of the
system's `configuration.nix` declaratively. The `nix` store also allows
{{< sidenote mark="selectively" set="right" >}}This gives the option of only
exposing the binaries from `/bin`.{{< /sidenote>}} exposing program paths to the
user's environment.

```nix {options="hl_lines=13"}
{
  allowUnfree = true;

  packageOverrides = pkgs: with pkgs; {

    Awesome = pkgs.buildEnv {
      name = "awesome";
      paths = [
        awesome
        lxappearance
        paper-gtk-theme
      ];
      pathsToLink = [ "/etc" "/share" "/bin" ];
    };
  };
}
```

Instantly 
{{< sidenote mark="remove" set="left" >}} 
There's actually no need to remove or uninstall packages in declarative package
management. The main idea is to either
[rollback](https://nixos.org/manual/nix/unstable/command-ref/nix-env/rollback.html)
bad changes or
[delete](https://nixos.org/manual/nix/unstable/command-ref/nix-env/delete-generations.html)
and
[switch](https://nixos.org/manual/nix/unstable/command-ref/nix-env/switch-generation.html)
to newer or older generations.
{{< /sidenote>}} 
all packages with `nix-env -Air` by switching to an empty environment.

```shell
nix-env -Air
```

List previous environment switches with `--list-generations`. You can further
manipulate the environment lineage by using the generation arguments in the
[`nix-env` manual](https://nixos.org/manual/nix/unstable/command-ref/nix-env.html#operation---list-generations).

```shell
$ nix-env --list-generations
368 2021-08-13 23:16:36
369 2021-08-22 00:01:38
370 2021-08-22 00:05:47
371 2021-08-22 00:23:03
```

## Unstable Declarative Package Management

The above is nice and all, but what if a package is not in the `nixos` stable
channel? We can add the {{< sidenote mark="unstable" set="right" >}}The unstable
channel is based on the [master repository](https://github.com/NixOS/nixpkgs)
and contains more packages than the stable channel.{{< /sidenote>}} channel from
the system side, but what happens if its too unstable?

Since this is `NixOS` --- we can mix and match. In `config.nix` bring in the
`unstable` channel directly. We {{< sidenote mark="install" set="left" >}}This
is wonderful. Getting a quick and clean install of programs like
[Ungoogled Chromium](https://github.com/Eloston/ungoogled-chromium#ungoogled-chromium)
isn't so easy in other environments.{{< /sidenote>}} `ungoogled-chromium` from
the {{< sidenote mark="unstable" set="right" >}}The paranoid user can set any
package archive and
[pin specific versions.](https://nixos.wiki/wiki/FAQ/Pinning_Nixpkgs)
{{< /sidenote>}} channel as an example.

```nix {options="hl_lines=1-2 21"}
let
  unstable = import (builtins.fetchTarball "https://releases.nixos.org/nixos/unstable/nixos-21.03pre265961.891f607d530/nixexprs.tar.xz") {};

  # Or lock at a specific commit to stop moving targets.
  # unstable = import (builtins.fetchTarball {
  #  url = "https://releases.nixos.org/nixos/unstable/nixos-21.03pre265961.891f607d530/nixexprs.tar.xz";
  #  sha256 = "1hwwb4n15bbqxnbqffq4kfb369vz65sq74p537fqdp6i4ywpqsyh"; }) {};
in

{
  allowUnfree = true;

  packageOverrides = pkgs: with pkgs; {

    Awesome = pkgs.buildEnv {
      name = "awesome";
      paths = [
        awesome
        lxappearance
        paper-gtk-theme
        unstable.ungoogled-chromium
      ];
      pathsToLink = [ "/etc" "/share" "/bin" ];
    };
  };
}
```

## Advanced Declarative Package Management

Here's a scenario: You're reading some [Lua](https://www.lua.org/) code and you
need a `lua` formatting tool. It looks like
[LuaFormatter](https://github.com/Koihik/LuaFormatter#readme) will do the job, but that
package is neither in the `stable` or `unstable` channel. You install `luarocks`
--- the [`lua` package manager,](https://luarocks.org/) but something
{{< sidenote mark="odd" set="right" >}}`luaformatter` attempts to compile its
submodules from the read only `nix` path of `luarocks`.{{< /sidenote>}} happens
when it tries to install `luaformatter`. What now?

Create a {{< sidenote mark="derivation" set="left" >}}This is similar to writing
a `PKGBUILD` or `APKBUILD`. The ease of writing a `derivation` depends on
[upstream's](https://en.wikipedia.org/wiki/Upstream_(software_development))
build and install complexity.{{< /sidenote>}} for `luaformatter` that
`config.nix` can call and build to expose the package to the user's environment.
Create a `packages` directory and a folder containing `default.nix` at
`~/.config/nixpkgs/packages/luaformatter`.

```nix {options="hl_lines=12"}
{ lib, stdenv, fetchFromGitHub, cmake }:

stdenv.mkDerivation rec {
  pname = "LuaFormatter";
  version = "1.3.3";

  src = fetchFromGitHub {
    sha256 = "1dfqsh6v8brnwzg3lgi7228lw08qqfy4ghbjyvwn7mr82fy1xcnd";
    rev = version;
    repo = pname;
    owner = "Koihik";
    fetchSubmodules = true;
  };

  buildInputs = [ cmake ];

  meta = with lib; {
    inherit (src.meta) homepage;
    description = "Code formatter for Lua";
    license = licenses.asl20;
    platforms = platforms.linux;
  };
}
```

Test the `derivation` using `nix-build`. Run
{{< sidenote mark="this" set="left" >}} There is nothing special about this
expression. The `default.nix` here is in the correct form for `callPackage` to
understand. A derivation can contain _anything_ --- though I prefer a
`default.nix` to always return a package, a `shell.nix` to return a shell, a
`module.nix` a module, a `flake.nix` a flake, and so on. {{< /sidenote>}} `nix`
{{< sidenote mark="expression" set="right" >}}This is how the `nixpkgs`
repository
[calls most derivations](https://github.com/NixOS/nixpkgs/blob/f7ad47a8f62e28e840e0f11177b2f7d6a16d29b2/pkgs/top-level/agda-packages.nix#L22).
This format sets you up for a
[pull request](https://github.com/NixOS/nixpkgs/pulls) if that's your
thing.{{< /sidenote>}} with the `--expr` argument inside the folder containing
`default.nix`.


```shell
nix-build -E 'with import <nixpkgs> {}; callPackage ./default.nix {}'
```

Use the `--keep-failed` (`-K`) argument to save the temporary build folder on
failure. This allows testing fixups in the failed build environment.

```shell
nix-build -K -E 'with import <nixpkgs> {}; callPackage ./default.nix {}'
```

Call the `derivation` if the build succeeds with `pkgs.callPackage` from your
`config.nix`

```nix {options="hl_lines=6 15"}
let
  unstable = import (builtins.fetchTarball "https://releases.nixos.org/nixos/unstable/nixos-21.03pre265961.891f607d530/nixexprs.tar.xz") {};
in {
  allowUnfree = true;

  packageOverrides = pkgs: with pkgs; {

    Awesome = pkgs.buildEnv {
      name = "awesome";
      paths = [
        awesome
        lxappearance
        paper-gtk-theme
        unstable.ungoogled-chromium
        (callPackage ./packages/luaformatter/default.nix {})
      ];
      pathsToLink = [ "/etc" "/share" "/bin" ];
    };
  };
}
```

Think of a `derivation` as an abstraction of the typical application build
process. In the above `derivation` we imported `stdenv` --- the standard
environment that contains common tools and dependencies most programs need. The
`fetchFromGitHub` {{< sidenote mark="class" set="left" >}}Or rather an attribute
set containing attributes.{{< /sidenote>}} contains methods that extend the
standard environment allowing us to pull in repositories the
{{< sidenote mark="`nix`" set="right" >}}Nix builds in a sand--boxed environment by
default, so arbitrary network connections are not allowed outside the scope of
attributes.{{< /sidenote>}} way. The application `luaformatter` requires
`cmake`, so it is sourced as one of the `buildInputs` --- another attribute
within the `derivation` context.

A `derivation` can be extended in multiple ways, allowing us to
{{< sidenote mark="custom" set="left" >}}Most applications and libraries are
already built for us.{{< /sidenote>}} build any application. The `nixpkgs`
repository has an
[overview](https://github.com/NixOS/nixpkgs/tree/master/doc/languages-frameworks)
of building artifacts in popular environments like
[Python.](https://github.com/NixOS/nixpkgs/blob/master/doc/languages-frameworks/python.section.md)
In tricky application builds, step down a level of abstraction to fix it up
using the derivation's
[phase attributes.](https://nixos.org/nixpkgs/manual/#sec-stdenv-phases)

Finally a `derivation` can be paired with a
[module](https://nixos.wiki/wiki/Module) allowing us to hide away the above work
into a simple {{< sidenote mark="set" set="right" >}}Writing a module is a story
[for another day.](/posts/writing-nixos-modules-and-switching-to-cgit/){{< /sidenote>}}
of attributes that a user can easily understand. This is what most users care
about. The following is a _made up_ example from the simple derivation above.

```nix
{
  programs.awesome.enable = true;
  programs.awesome.luaFormat = true;

  # Or

  programs.awesome = {
    enable = true;
    luaFormat = true;
  };
}
```