Introduction
Hello everyone 
, we’d like to present our research project on improving the reproducibility of scientific software using Nix and NixOS.
We (Simon and Max) are two students doing our masters degree in software engineering at the University of Stuttgart.
We have conducted a case study on preCICE and many of its official adapters.
The repository including a report can be found here: GitHub - precice/nix-packages: Official preCICE adapters and solvers packaged with the Nix package manager. See https://precice.discourse.group/t/precice-goes-nix-os-improving-reproducibility-of-scientific-software/.
The report itself can be found here as a PDF.
TL;DR: We did a proof-of-concept of how scientific software can be built reproducibly, so scientific experiments using preCICE can be reproduced with the same software stack reliably when providing only a Git commit hash of our repository.
What is Nix/NixOS?
Nix is a functional, declarative package manager aiming for reproducibility.
Package builds are made by calling functions and providing inputs such as the source code and dependencies for the package.
If the same inputs are supplied to the function, the same outputs can be expected, i.e. making the build reproducible.
The outputs of the build functions can be pretty much anything, but are in most cases the compiled package binaries.
There is a large upstream Git repository called nixpkgs, specifying many different Nix derivations (build instructions) for several thousand packages.
Nix allows using substituters which are binary caches.
Those caches, like the upstream cache at https://cache.nixos.org, can serve outputs that were compiled by the hydra build cluster, e.g., the upstream one at https://hydra.nixos.org.
Packages available in the upstream cache can be searched using the Nix package search.
Nix can be leveraged as a general purpose build tool to not only build packages, but also systemd service files or images, making it possible to build whole operating systems like for example NixOS.
Any configuration options of the VM can be found in the upstream options search.
We analyzed the preCICE demo VM and recreated it as a NixOS based image.
The VM is built as an ISO, a Nix path and a Vagrant VirtualBox image.
We have already gathered some years of Nix/NixOS experience, as we use NixOS as our daily drivers.
Case study
To see how well Nix can be used to build and package scientific software reliably and reproducibly, we conducted a case study in the context of the preCICE distribution v2211.0.
The adapters and solvers are diverse in their project structures, build systems, programming languages and project sizes, so our case study is quite representative for multi-component research software like preCICE.
As we wanted to reproduce the preCICE demo VM, we were mainly interested in the tools, adapters including their solvers contained in the preCICE VM.
So we went ahead and packaged ASTE, python-bindings, CalculiX-adapter, code-aster and code_aster-adapter, dealii and dealii-adapter, DUNE and DUNE-adapter, FEniCS and fenics-adapter, OpenFOAM and openfoam-adapter, SU2 and su2-adapter.
The softwares preCICE and precice-config-visualizer were already packaged upstream.
Having packaged the different pieces of software, we see that Nix can successfully handle all kinds of scientific software distributions.
However, there is a quite large gap regarding packagability of the different source codes, mainly due to the build system used.
Custom build systems (like those used by OpenFOAM or code_aster) may require a disproportional amount of additional work to package them for any system.
On the other hand, packages that use common, industry proven package managers, like CMake and autotools make it easier for developers of the software to build their software as well as package managers to package it.
Why is this relevant?
You could ask yourself, “Cool, but why shouldn’t I simply use something like a VM or Docker?”.
Our research project was mainly focused on the reproducibility of packaging scientific software, which is quite trivial with Nix once you managed to write a Nix derivation defining the package.
With Docker, reproducibility is not achievable easily, sometimes not even achievable at all.
Docker uses a Dockerfile which contains build instructions to generate an image.
With a VM, you can shut it down and copy the virtual disk file as an image.
Some tools like Vagrant automate this task.
These reusable images can be distributed, so if copied onto two separate machines, an instance of the image will be the same on both machines.
The only downside of this is, that the images are not reproducible nor expandable.
If you want to reproduce the same Docker image of a Dockerfile that was generated one week ago, you simply cannot reproduce it in many cases, as package managers like apt fetch their sources from debian mirrors.
When the software versions of the upstream mirrors move onwards, apt fetches a newer version of the package.
The same is true for Vagrant provisioned VMs.
So if a Dockerfile specifies RUN apt update && apt install python3 you cannot tell if you just installed python3.7 or python3.10 (which can make a huge difference).
Nix allows you to exactly pin the version of a package (at Git commit level) including all its dependencies.
So if you use the preCICE package from the upstream nixpkgs repository at commit 842f85514f2e289c1709f99880ad4ed939662208, you will always get the exact same software output (including all dependencies of preCICE), therefore ensuring reproducibility.
This could enhance the reproducibility of scientific computations dramatically.
Want to try it out?
You can try out our nixified version of preCICE and its adapters by following our getting started section in the readme file of the repo.
We also provide a setup.sh script that allows the installation of Nix on the Uni-Stuttgart Simtech HPC cluster, so you can build software faster.
To build the preCICE demo VM with NixOS as a base, you can run any of the following commands if you have Nix installed:
sudo nix build --extra-experimental-features "nix-command flakes" github:precice/nix-packages#vm
# You can replace `vm` with
# - vm-light -- to build a lightweight qemu VM imsage
# - vagrant-vbox-image -- to build a vagrant virtual box image
# - vagrant-vbox-image-light -- to build a vagrant virtual box image of the lightweight configuration
# - iso -- to build a bootable iso
# - iso-light -- to build a bootable iso of the lightweight configuration
A bootable iso file will be linked to the current directory under ./result/.
If you have any questions about the implementation or on how to reproduce environments/setup the project, feel free to post here (or if it’s rather technical as an issue in the repository).
Any feedback is welcome!