Setting Up Cuirass Locally
Recently I've been trying to get a PR merged with some fixes for Lua. I thought this would be a pretty straightforward thing to merge, but it turns out that modifying the Lua packages leads to 990 packages needing to be rebuilt. This is more than the 300 limit for a merge to master, so instead of merging my changes directly Andreas has kindly pushed them to a lua-team branch and queued it up behind go-team, gnome-team, and rust-team.
While I'm waiting for that, I have some more drastic changes that I'd like to try out (e.g. packaging Lua 5.5.0, and using it as the default Lua for building packages). In order to do that, I'd like to be able to test my changes before the lua-team branch gets to the front of the queue. I'm not a committer to Guix, so instead I figured I'd spin up Cuirass on my local machine to help me try things out.
Why Cuirass?
It's worth spending a moment on why I want to use Cuirass, rather than just doing the builds using Guix directly. guix refresh -l already gives you a list of the packages that you need to build to test your changes, why not just use that with guix build?
There are two reasons why I want to use Cuirass: live feedback, and tracking changes over time.
For live feedback, Cuirass gives me more information about what's going on that guix build does. Even though I'm only running on my local machine, seeing the list of scheduled/succeeded/failed builds, and in-progress builds, is helpful for me to keep track of where things are up to. It lets me get started investigating build failures right away, without interrupting the rest of the builds.
For tracking changes over time, Cuirass helps me by reporting not just whether a package has built or not, but whether that has changed since the last build. This tells me whether my changes are breaking things, or whether things were already broken before I started. I need to know this so I know where to spend effort trying to fix things.
Given Cuirass is now a sunk cost for me, we're going to proceed with the assumption that this was a good idea. 🙂
Initial Setup
Setting up Cuirass on a Guix system is shockingly easy. I just added these three services to my operating-system's services:
(service cuirass-service-type
(cuirass-configuration
(specifications
#~(list (specification
(name "lua-team")
(build '(manifests #$(local-file "lua-manifest.scm")))
(channels (list (channel
(name 'guix) ; has to be called guix for pull to work
(url "https://git.sr.ht/~czan/guix") ; my unlisted fork
(branch "lua-team")))))))))
(service postgresql-service-type
(postgresql-configuration
(postgresql (specification->package "postgresql@16"))))
This sets up a local instance of the cuirass-service-type configured to poll the listed channels and build things locally whenever anything changes.
The manifest that I wrote looks for all the packages which (transitively) depend on one of the Lua version. I took inspiration for the code for guix refresh -l, and this is the result::
;; lua-manifest.scm
(use-modules (guix)
(guix packages)
(gnu packages)
(guix profiles)
(guix monads)
(guix graph)
(guix scripts graph)
(ice-9 match)
(gnu packages lua))
(define packages
(list lua-5.4 lua lua-5.2 lua-5.1 luajit))
(packages->manifest
(with-store store
(run-with-store store
(mlet %store-monad ((edges (node-back-edges %bag-node-type
(package-closure (all-packages)))))
(return (node-transitive-edges packages edges))))))
Problem: not authenticated
The first problem I ran into was that Guix will fail a pull for the guix channel if it tries to authenticate the commits but can't. Since I'm not a committer, I can't authenticate my commits without doing some more work (setting up a key, adding to the keyring, signing a commit, setting up an introduction, ...). I didn't really want to do that, so I hunted through the Cuirass manual and found the authenticate-channels? property. Setting it to #f does an unauthenticated pull. Dangerous in some circumstances, but acceptable for what I'm doing.
(service cuirass-service-type
(cuirass-configuration
(specifications
#~(list (specification
;; ... As above ...
;; I'm not a committer, so we can't authenticate my commits
(properties `((authenticate-channels? . #f))))))))
Problem: evaluations failing
I got the pulls working, but then the evaluations kept failing. It turns out the path listed in the build option need to exist in the pulled channels. In my first attempt I passed a (local-file ...) call, but this did not work at all.
I had to commit my manifest into the repository and reference it in the build option:
(service cuirass-service-type
(cuirass-configuration
(specifications
#~(list (specification
(name "lua-team")
(build '(manifests "etc/teams/lua/lua-manifest.scm"))
;; ... As above ...
)))))
Problem: no builds running
The next problem I ran into was that this configuration didn't run any builds. It would poll the git repository, and successfully pull the changes and build Guix, but it would never get to building any packages. I couldn't see any reason for it not working, but the documentation for the local build mode also had this warning:
When Cuirass is used to build a large amount of jobs, the remote build mechanism described below should be preferred.
I figured I should try that. Again, configuring this in a Guix system to just use the local machine as a builder is easy. I just added the remote-server setting to my cuirass-configuration:
(service cuirass-service-type
(cuirass-configuration
(remote-server (cuirass-remote-server-configuration))
(specifications
;; ... As above ...
)))
and I added one more service to my list of services:
(service cuirass-remote-worker-service-type
(cuirass-remote-worker-configuration))
With that, Cuirass started to build my packages locally. I saw my CPUs get busy, and all was well.
Problem: killed my machine
Unfortunately, the default configuration for the Cuirass remote worker is to run a single build, and to let it use all the cores for its build. My machine has an i7-8700 with 6 cores running 12 threads, and 32GiB of RAM. It turns out that building some Guix packages will cause that configuration to lock up (I assume it's swapping, but I didn't investigate at all).
I looked through the Cuirass manual to try to find some way to set the equivalent of --cores in the builds it runs, but I couldn't find anything. I had a look at the Cuirass source where the parameter is set, which itself comes from dividing the number of CPU cores by the number of workers. This assumes that the Cuirass workers can take up all the cores on the machine (which is not ideal for the computer that I use as my desktop), but it at least gives me enough control to avoid the builds locking up the whole machine.
Once again, the configuration for this is straightforward:
(service cuirass-remote-worker-service-type
(cuirass-remote-worker-configuration
(workers 6)))
Now I have six concurrent builds, with each using two threads. This has kept my memory usage under 50% thus far, and builds are continuing unimpeded.
Final configuration
The end result of my service configuration looks like this:
(service cuirass-service-type
(cuirass-configuration
(remote-server (cuirass-remote-server-configuration))
(specifications
#~(list (specification
(name "lua-team")
(build '(manifests "etc/teams/lua/lua-manifest.scm"))
(channels (list (channel
(name 'guix)
(url "https://git.sr.ht/~czan/guix")
(branch "lua-team"))))
(properties `((authenticate-channels? . #f))))))))
(service postgresql-service-type
(postgresql-configuration
(postgresql (specification->package "postgresql@16"))))
(service cuirass-remote-worker-service-type
(cuirass-remote-worker-configuration
(workers 6)))
Want to help?
If you're interested in helping to maintain Lua packages in Guix (because I have no idea what I'm doing), get in touch on guix-devel in the thread I started. I'm particularly interested in getting a Guix committer on board, so I don't have to keep bothering Andreas. (Thanks Andreas!)