Running NixOS in a Linux Container

Note: Rerunning the code in this article seems to suggest an ambiguity between the container init and the init path lxc expects across multiple NixOS versions. See this thread for a possible fix.
NixOS Home Page
NixOS Home Page

NixOS is seen as the holy grail of My favorite configuration stack is Ansible + Shell + Alpine management and system administration. It’s what many call a purely functional distribution [pdf]. The state is declarative — entire systems, deployment artifacts, or services can be built from a single file.

Let’s try out NixOS in a This is assuming you have a working LXC or LXD setup with networking. Container (LXC). The NixOS Hydra builds (aarch64) provide container images but they don’t seem to Symlinking issues when extracting the rootfs. for this use case.

Bootstrapping

The best way to get a NixOS root file system is to bootstrap using nixos-generate. This will require a live NixOS environment. Download the latest NixOS minimal image and boot into the live environment using preferred means.

Inside the live environment bootstrap a Linux container’s rootfs using nixos-generate.

shell
nixos-generate -f lxc

If the command nixos-generate is not found — install nixos-generators using nix-env.

shell
nix-env -iA nixos.nixos-generators

Once completed, this command will print a hashed path like /nix/store/3ipfpzhk4dllwhcnldsbfldi1favyxsm-tarball/nix-support/hydra-build-products. This file contains the location of the rootfs archive.

shell
$ cat /nix/store/3ipfpzhk4dllwhcnldsbfldi1favyxsm-tarball/nix-support/hydra-build-products
file system-tarball /nix/store/3ipfpzhk4dllwhcnldsbfldi1favyxsm-tarball/tarball/nixos-system-x86_64-linux.tar.xz

Save the generated archive nixos-system-x86_64-linux.tar.xz and exit the live environment.

Setup the Linux Container

Now that we have a clean rootfs archive. Create an empty linux container and rootfs directory. Extract nixos-system-x86_64-linux.tar.xz to rootfs.

shell
lxc-create -n nixos -t none
cd /var/lib/lxc/nixos
mkdir rootfs
tar -xvf nixos-system-x86_64-linux.tar.xz -C rootfs/

Let’s massage our This has been tested on Debian and Arch Linux. for NixOS at /var/lib/lxc/nixos/config. Our entry point is /sbin/init and we must mount /proc to run nixos-rebuild.

ini
# Distribution configuration
lxc.include = /usr/share/lxc/config/common.conf
lxc.arch = linux64

# Container specific configuration
lxc.rootfs.path = dir:/var/lib/lxc/nixos/rootfs
lxc.uts.name = nixos

# Network configuration
lxc.net.0.type = veth
lxc.net.0.link = lxcbr0
lxc.net.0.flags = up
lxc.net.0.hwaddr = 00:16:3e:01:77:76
lxc.apparmor.profile = generated
lxc.apparmor.allow_nesting = 1

# NixOS configuration
lxc.init.cmd = /sbin/init
lxc.mount.entry = proc mnt/proc proc create=dir 0 0

Setup and Run NixOS

Now NixOS
Container Setup NixOS Container Setup the container and enter the shell using lxc-attach.

shell
lxc-start -n nixos
lxc-attach -n nixos

The default /etc/nixos/configuration.nix is empty. Tell NixOS that we are a container by adding boot.isContainer = true.

nix
{ config, pkgs, ... }:

{
  imports = [ ];
  boot.isContainer = true;
}

Now run a Firewall errors about iptables IPv6 filters mean you either run an old kernel or you need to sudo modprobe ip6table_filter of the system. The upgrade flag is necessary on the first rebuild to download the channels.

shell
 nixos-rebuild switch --upgrade

We now have a base NixOS system in a Linux container. You can build reproducible desktops, servers, deployment artifacts — anything, using your own configuration.nix and consulting the NixOS options index. This approach is great because when you make something it just works — all the time.

16 January 2020 — Written
17 August 2022 — Updated
Thedro Neely — Creator
running-nixos-linux-containers.md — Article

More Content

Openring

Web Ring

Comments

References

  1. https://thedroneely.com/git/
  2. https://thedroneely.com/
  3. https://thedroneely.com/posts/
  4. https://thedroneely.com/projects/
  5. https://thedroneely.com/about/
  6. https://thedroneely.com/contact/
  7. https://thedroneely.com/abstracts/
  8. https://ko-fi.com/thedroneely
  9. https://thedroneely.com/tags/lxc/
  10. https://thedroneely.com/tags/nix/
  11. https://thedroneely.com/posts/running-nixos-linux-containers/#isso-thread
  12. https://thedroneely.com/posts/rss.xml
  13. https://github.com/nix-community/nixos-generators/issues/79#issuecomment-822073364
  14. https://thedroneely.com/images/running-nixos-linux-containers.png
  15. https://nixos.org/
  16. https://en.wikipedia.org/wiki/Ansible_%28software%29
  17. https://en.wikipedia.org/wiki/Unix_shell
  18. https://alpinelinux.org/about/
  19. https://nixos.org/~eelco/pubs/nixos-icfp2008-final.pdf
  20. https://linuxcontainers.org/
  21. https://hydra.nixos.org/job/nixos/release-22.05/nixos.containerTarball.x86_64-linux
  22. https://hydra.nixos.org/job/nixos/release-22.05/nixos.containerTarball.aarch64-linux
  23. https://thedroneely.com/posts/running-nixos-linux-containers/#bootstrapping
  24. https://nixos.org/download.html#download-nixos
  25. https://thedroneely.com/posts/running-nixos-linux-containers/#code-block-2491222
  26. https://thedroneely.com/posts/running-nixos-linux-containers/#code-block-57ce775
  27. https://thedroneely.com/posts/running-nixos-linux-containers/#code-block-4812c40
  28. https://thedroneely.com/posts/running-nixos-linux-containers/#setup-the-linux-container
  29. https://thedroneely.com/posts/running-nixos-linux-containers/#code-block-70ec8a9
  30. https://thedroneely.com/posts/running-nixos-linux-containers/#code-block-d12c54e
  31. https://thedroneely.com/posts/running-nixos-linux-containers/#setup-and-run-nixos
  32. https://thedroneely.com/images/running-nixos-linux-containers.gif
  33. https://thedroneely.com/posts/running-nixos-linux-containers/#code-block-5e53d1c
  34. https://thedroneely.com/posts/running-nixos-linux-containers/#code-block-62a110d
  35. https://thedroneely.com/posts/running-nixos-linux-containers/#code-block-f891066
  36. https://nixos.org/nixos/options.html
  37. https://www.thedroneely.com/posts/running-nixos-linux-containers.md
  38. https://thedroneely.com/posts/a-few-abstracts/
  39. https://thedroneely.com/posts/hugo-is-good/
  40. https://thedroneely.com/posts/writing-strategy/
  41. https://git.sr.ht/~sircmpwn/openring
  42. https://drewdevault.com/2022/11/12/In-praise-of-Plan-9.html
  43. https://drewdevault.com/
  44. https://mxb.dev/blog/the-indieweb-for-everyone/
  45. https://mxb.dev/
  46. https://www.taniarascia.com/simplifying-drag-and-drop/
  47. https://www.taniarascia.com/
  48. https://thedroneely.com/posts/running-nixos-linux-containers#isso-thread
  49. https://thedroneely.com/posts/running-nixos-linux-containers#bootstrapping
  50. https://thedroneely.com/posts/running-nixos-linux-containers#code-block-2491222
  51. https://thedroneely.com/posts/running-nixos-linux-containers#code-block-57ce775
  52. https://thedroneely.com/posts/running-nixos-linux-containers#code-block-4812c40
  53. https://thedroneely.com/posts/running-nixos-linux-containers#setup-the-linux-container
  54. https://thedroneely.com/posts/running-nixos-linux-containers#code-block-70ec8a9
  55. https://thedroneely.com/posts/running-nixos-linux-containers#code-block-d12c54e
  56. https://thedroneely.com/posts/running-nixos-linux-containers#setup-and-run-nixos
  57. https://thedroneely.com/posts/running-nixos-linux-containers#code-block-5e53d1c
  58. https://thedroneely.com/posts/running-nixos-linux-containers#code-block-62a110d
  59. https://thedroneely.com/posts/running-nixos-linux-containers#code-block-f891066
  60. https://thedroneely.com/posts/nixos-in-the-wild/
  61. https://thedroneely.com/posts/literate-programming/
  62. https://thedroneely.com/posts/good-evil-and-the-law/
  63. https://thedroneely.com/archives/posts/
  64. https://thedroneely.com/projects/personal-portfolio/
  65. https://thedroneely.com/posts/trying-out-this-website/
  66. https://drewdevault.com/2022/09/16/Open-source-matters.html
  67. https://mxb.dev/blog/make-free-stuff/
  68. https://thedroneely.com/sitemap.xml
  69. https://thedroneely.com/index.json
  70. https://thedroneely.com/resume/
  71. https://gitlab.com/tdro
  72. https://github.com/tdro
  73. https://codeberg.org/tdro
  74. https://thedroneely.com/analytics
  75. https://thedroneely.com/posts/running-nixos-linux-containers#
  76. https://creativecommons.org/licenses/by-sa/2.0/
  77. https://thedroneely.com/git/thedroneely/thedroneely.com
  78. https://opensource.org/licenses/GPL-3.0
  79. https://www.thedroneely.com/
  80. https://thedroneely.com/posts/running-nixos-linux-containers/#