In our quest to create the world's greatest development suite we think long and hard about how to make the best tools, offering the greatest value, with the least complexity possible, to our users. But we are all products of our environment, and our tools too are products of theirs. So, in an effort to support them, we investigated the best way to create the perfect environment for our tools, which in turn creates the best environment for our users. After taking that step back and analysing the contenders, one solution stood out above all others: Nix Package Manager.
Nix is a Game Changer
The Nix Package Manager has a number of unique features that provide a much more stable and smoother installation experience when managing different environments and installing multiple dependencies. Being a purely functional language means that output only depends on the input of the function, nothing else affects it. By using Nix we ensure it is impossible to introduce any impurities into our installation package.
Binary Distribution
The smoother experience is due to Nix using binaries when installing software, when they are available. This eliminates the need to compile from source every time. Nix channels contain these previously compiled binary versions of a package from where downloads are sourced. This means that you do not need to download and compile a package, it's dependencies, and pray, but instead you download and untar a tested working tarball from a Nix channel.
Multiple Versions
Increased stability is supported through a generation system. Each package installation creates a new generation of your environment which can be easily rolled back should an error occur. This functionality provides great continuity and stability across builds, because it means that if a download ever breaks a build, or introduces an error, you simply rollback to a previous working generation and suffer no down time while you figure out what went wrong.
Isolated Development Environment
The Nix shell supports creating isolated development environments that also have their own generation roll back system. This allows you to completely compartmentalize your system into many stable custom environments that can contain varying versions of the same package. Also, this system encourages experimentation as you can play with different optimization setups while keeping your stable version running without interference.
We chose Nix for managing this over virtualenv for a number of reasons, but mainly for the following:
virtualenvonly works for Python packages and system packages are not included.virtualenvinherits the installation shortcomings of the Python packaging tools, such apipandsetuptools.virtualenvrecently dropped Python 2.6 support.- Nix is under active development with a great community.
Atomic Upgrades
This is where Nix comes into a league of its own. If an install is not successful your development environment will not change, and if an install is successful but you are not pleased with it you can simply rollback to the previous working generation. Additionally, you can then use the Nix Garbage Collection function which allows you to quickly clean up your environments and remove ones that are not used.
Creating Isolated Environments
Creating an isolated environment in Nix is trivial, as can be seen from the following example which takes you from installing Nix to setting up a new Sphinx documentation set in just a few lines:
brian:nix brian$ bash <(curl https://nixos.org/nix/install) brian:nix brian$ which sphinx brian:nix brian$ nix-shell -p python27Packages.sphinx [nix-shell:~/nix]$ which sphinx [nix-shell:~/nix]$ /nix/store/2w1bkshjijjazm0j3fcqm9rv67kjmq7b-bash-4.2-p45/sphinx [nix-shell:~/nix]$ sphinx-quickstart Welcome to the Sphinx 1.2 quickstart utility. Please enter values for the following settings (just press Enter to accept a default value, if one is given in brackets). Enter the root path for documentation. > Root path for the documentation [.]: . . build succeeded. Build finished. The HTML pages are in _build/html. [nix-shell:~/nix]$ ls -al total 28 drwxr-xr-x 9 brian staff 306 Aug 11 15:23 . drwxr-xr-x+ 43 brian staff 1462 Aug 11 15:07 .. -rw-r--r-- 1 brian staff 6762 Aug 11 15:23 Makefile drwxr-xr-x 4 brian staff 136 Aug 11 15:23 _build drwxr-xr-x 2 brian staff 68 Aug 11 15:23 _static drwxr-xr-x 2 brian staff 68 Aug 11 15:23 _templates -rw-r--r-- 1 brian staff 8091 Aug 11 15:23 conf.py -rw-r--r-- 1 brian staff 423 Aug 11 15:23 index.rst -rw-r--r-- 1 brian staff 6701 Aug 11 15:23 make.bat
RhodeCode Enterprise and Nix
What does a purely functional package manager mean for RhodeCode Enterprise?
It means that because Nix treats our RhodeCode Enterprise packages like values in a purely functional manner when they are built, there is no interference and that packages never change after they have been created.
Also, when our, almost ready for show time, Nix based installer runs, instead of putting everything in a global /usr folder, RhodeCode Enterprise and all of its packages are stored in a separate location on the filesystem. This allows for multiple versions of RhodeCode Enterprise and it's dependencies to be installed with the guarantee that no existing dependencies are broken.
The atomic upgrades and rollbacks also enable each installation to be cancelled without the risk of corrupting any existing packages or configurations.
In-house, Nix enables us to test our setup multiple times consistently, even on different machines, saving us time trying to figure out differences between two machines or installed packages. This ability to manage and track RhodeCode Enterprise dependencies gives us better reproducibility, as well as traceability, allowing us to see the complete and exact changes that might have caused a system to fail.
Nix also enforces discipline on us when dealing with software dependencies. A dependency or external package can be easily installed and forgotten, and it can happen that such dependencies are introduced without anyone realising it, until it is too late and you are looking for a bug you think should not be possible.
This discipline that Nix enforces, because you must specify all dependencies, takes time to get used to, but once a build is working you are guaranteed that all the necessary dependencies have been successfully included and we know exactly what we have running in your build.
RhodeCode Enterprise is build on Python and Nix supports Linux, OSX and FreeBSD platforms. This gives us great cross-platform installer functionality and we are almost set to release it into the wild. Once we cross the Windows hurdle, then it will be ready for some beta-testing.
Brian