Hi Zelphir > Hello Guix Users! > > In the past I thought I had it figured out, thanks to help from someone on > the > mailing list, how to run Ocaml projects in a reproducible (hashes/checksums > verified!) way. > > It is December and I thought I would give Ocaml a try at Advent of Code this > year. However, I learned, that my previous project structure/setup no longer > works. I thought: "Why not check out how people, who don't use or know guix > do > this?" and looked into opam. Long story short: Unfortunately, it does not > provide a lock file that one could commit into a repo to ensure > reproducibility. > Even the tool opam-lock merely pins version numbers without hashes. > Disappointing. So I am back to trying to fix my guix using project setup. > Last > stop before giving up on Ocaml for another year or so. In the following I > will > describe my project setup, and why it hopefully is reproducible not only on > my > machine, but also on other machines. > > *The guix part* > > I have the following files: > > (1) guix-env/channels.scm: Specifying the exact guix channel that is used, > and > thereby locking down hashes, which are specified in the guix repository > itself. > > ~~~~ > (list (channel > (name 'guix) > (url"https://git.guix.gnu.org/guix.git";) > (branch "master") > (commit > "7c6d8a6224cf3209efa179dbe1509759a580cb05") > (introduction > (make-channel-introduction > "9edb3f66fd807b096b48283debdcddccfea34bad" > (openpgp-fingerprint > "BBB0 2DDF 2CEA F6A8 0D1D E643 A2A0 6DF2 A33A 54FA"))))) > ~~~~ > > (2) guix-env/manifest.scm: Specifying the package, that I want to use for my > project. > > ~~~~ > (specifications->manifest > '("[email protected]" > "ocaml-utop" > "dune" > "bash" > "make" > "ocaml-findlib" > "ocaml-zarith")) > ~~~~
You have actually two versions of ocaml now in your setup: 5.3.0 and 4.14 because you included version 5.3.0 explicitely but ocaml-utop is built with ocaml which is set to version 4.14 in the file ocaml.scm: (define-public ocaml ocaml-4.14). You have to be careful pinning a certain version. Normally in guix the convention is per a specific git commit there is a specific ocaml version (in your case 4.14) from which every dependency is built from. If you happen to need another version you need to rebuild all the dependecies with this version. A quick fix for your problem is to drop the version pinning of ocaml and just write ocaml instead of [email protected] explicitely. > (3) Makefile: Used as a task runner, basically, to avoid having to type > complicated commands. > > ~~~~ Makefile (variable definitions) ~~~~ > .POSIX: > .RECIPEPREFIX = > > .DELETE_ON_ERROR: > > MAKEFLAGS += --warn-undefined-variables > MAKEFLAGS += --no-builtin-rules > > SHELL ::= guix time-machine --channels=guix-env/channels.scm -- shell --check > --manifest=guix-env/manifest.scm -- bash -c > .SHELLFLAGS ::= -Euxo pipefail -c > > BUILD_DIR ::= _build > > OCAML_PACKAGES ::= zarith > > ... > ~~~~ > > Here one can see, that I am setting the `SHELL` environment variable to using > `bash -c`, which will evaluate the actual commands that are defined later in > Makefile targets. `bash` runs in the `guix shell`, that is created in a `guix > time-machine` call, which refers to the `guix-env/channels.scm` file, to make > sure, that I am using exactly the versions defined in that commit of the guix > repository, implying that I am using exactly the hashes defined on that > commit > of the guix repository. That should make things reproducible and make it so > that > when I run things on another machine, the same software, the exact same > packages > are used. > > I also define a list of Ocaml packages I am using, currently only one: > `zarith` > which is important for big integers. > > Then come the actual targets: > > ~~~~ Makefile (targets) ~~~~ > ... > > .PHONY: repl > repl: >> utop > > > .PHONY: shell > shell: >> bash > > ... > ~~~~ > > The target `shell` is merely for debugging purposes, or for when I need to > run a > command in a shell that has the dependencies installed. > > When I run `make shell` and in that resulting shell run `ocaml --version`, I > get > the output: `The OCaml toplevel, version 5.3.0`. Great so far! > > *(Issue 1)* > > One problem is with `utop`. Somehow it does not use the correct Ocaml version > and I don't know where it gets another Ocaml version even from. When I run > `make > repl` I get a `utop` REPL that says: > > ~~~~ utop on command line ~~~~ > ───────────┬───────────────────────────────────────────────────────────────────┬ > │ Welcome to utop version %%VERSION%% (using OCaml version > 4.14.1)! │ > > └───────────────────────────────────────────────────────────────────┘ > > Type #utop_help for help about using utop. > ~~~~ > > What? Why is it using Ocaml version 4.14.1?? Where does it get that from? No > idea. It shouldn't be using anything but the installed packages. Maybe it > internally depends on an older Ocaml version implicitly, and guix installs > that, > so that utop can run at all, satisfying utop's dependencies? But on > https://hpc.guix.info/package/ocaml-utop I click the link to > https://codeberg.org/guix/guix/src/commit/23dbcfaef682c41e9cb2a6aea9ad69feea65f26a/gnu/packages/ocaml.scm#L5660, > > where I see the inputs: > > ~~~~ ocaml.scm ~~~~ > (native-inputs > (list ocaml-cppo)) > (propagated-inputs > (list ocaml-lambda-term > ocaml-logs > ocaml-lwt > ocaml-lwt-react > ocaml-react > ocaml-zed)) > ~~~~ > > There I at least don't see any direct dependency on an older Ocaml version. > Might be one of those packages listed there is only available for older Ocaml > version. I don't know. > > But this issue is not the main issue. The main issue is with other targets. > The > Makefile also has targets to run an Ocaml file: > > ~~~~ Makefile (targets for running ocaml things) ~~~~ > %.byte: %.ml >> ocamlfind ocamlc $(foreach OCAML_PACKAGE,$(OCAML_PACKAGES),-package >> $(OCAML_PACKAGE)) -linkpkg -o $*.byte $< > > > # Simply running depends on building the byte files, which > # are to be considered intermediate files. > .PRECIOUS: %.byte > %: %.byte >> ./[email protected] > ~~~~ > > Here I define a target `%`. This target will be used when I write things like > `make main`. It depends on `%.byte`, which means that `main` depends on > `main.byte`. The other target `%.byte` describes how to build that. It > depends > on there being a `%.ml` which in case of `main` would be `main.ml`. It runs > the > long command written there, which makes use of `ocamlfind`. The hint to use > `ocamlfind` is the decisive hint someone gave me on this mailing list some > time > ago. If there is a better way, please let me know. I need to be able to > specify > arbitrary packages, that I install using guix, by writing them in the > `guix-env/manifest.scm` file. > > *(Issue 2)* > > So lets see what actually happens, when I try to run some `main.ml` file with > some code: > > ~~~~ main.ml ~~~~ > (* In REPL instead use: > #require "zarith";; *) > > open Z > ~~~~ > > `make main`: > > ~~~~ make main result ~~~~ > ocamlfind ocamlc -package zarith -linkpkg -o main.byte main.ml > guix shell: checking the environment variables visible from shell > '/bin/bash'... > guix shell: All is good! The shell gets correct environment variables. > + ocamlfind ocamlc -package zarith -linkpkg -o main.byte main.ml > findlib: [WARNING] Package unix has multiple definitions in > /gnu/store/q1jdsy1y8kzk18iy4snb9c85m6awl2jp-profile/lib/ocaml/unix/META, > /gnu/store/q1jdsy1y8kzk18iy4snb9c85m6awl2jp-profile/lib/ocaml/site-lib/unix/META, > > /gnu/store/yjik976n23235nhkr0amkrymb6kyfkxs-ocaml-findlib-1.9.5/lib/ocaml/site-lib/unix/META > findlib: [WARNING] Package threads has multiple definitions in > /gnu/store/q1jdsy1y8kzk18iy4snb9c85m6awl2jp-profile/lib/ocaml/threads/META, > /gnu/store/q1jdsy1y8kzk18iy4snb9c85m6awl2jp-profile/lib/ocaml/site-lib/threads/META, > > /gnu/store/yjik976n23235nhkr0amkrymb6kyfkxs-ocaml-findlib-1.9.5/lib/ocaml/site-lib/threads/META > File "main.ml", line 1: > Error: > /gnu/store/q1jdsy1y8kzk18iy4snb9c85m6awl2jp-profile/lib/ocaml/site-lib/zarith/z.cmi > is not a compiled interface for this version of OCaml. > It seems to be for an older version of OCaml. > make: *** [Makefile:23: main.byte] Error 2 > ~~~~ > > For starters, the `findlib` warnings seem to be worrisome. Why does findlib > ever > search somewhere, where it finds multiple definitions? I don't remember this > issue from this project setup appearing 1 or 2 years ago. This might already > be > the cause for the next part of the issue. > > ~~~~ > Error: > /gnu/store/q1jdsy1y8kzk18iy4snb9c85m6awl2jp-profile/lib/ocaml/site-lib/zarith/z.cmi > is not a compiled interface for this version of OCaml. > It seems to be for an older version of OCaml. > ~~~~ > > Well, seems like it is not using the correct package, or Ocaml version > specified > in manifest.scm, or the package got compiled using another Ocaml version, or > I > don't know what's going on and what is going wrong. > > The issue with utop showing another version is already suspect. Maybe somehow > the `zarith` package got compiled using that older version of Ocaml too! > > But I don't know how to check that. Or how to stop Guix from doing that. Or > how > to nail down the versions in a way that doesn't allow this to happen. > > I already tried running `guix gc` to maybe delete old `guix shell`s, but to > no > avail. > > Here is a repository, which contains the project setup I described: > https://codeberg.org/ZelphirKaltstahl/advent-of-code-2025/src/commit/aa7d26056fc0ff442978d4bdebe615d533e54db4. > > Here is my `guix --version` output, in case it is relevant: > > ~~~~ guix --version output ~~~~ > guix (GNU Guix) 7c6d8a6224cf3209efa179dbe1509759a580cb05 > Copyright (C) 2025 the Guix authors > License GPLv3+: GNU GPL version 3 or later<http://gnu.org/licenses/gpl.html> > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. > ~~~~ > > Here is my OS: > > ~~~~ lsb_release -a ~~~~ > No LSB modules are available. > Distributor ID: Debian > Description: Debian GNU/Linux 12 (bookworm) > Release: 12 > Codename: bookworm > ~~~~ > > So my questions are at this point: > > (1) How do people make reproducible Ocaml project setups? I don't mean only > version numbers ... It must be hashes and verifying those hashes. Otherwise I > don't think I will pursue things further with Ocaml. > > (2) How to fix my guix-using Ocaml project setup, so that I can run my code, > including ocaml-* packages that I install via guix? > > I am aware, that I could also create a guix profile. However, I don't think > that > will change things and I am fine with a guix shell. All I want is something > reproducible, whether that is a guix profile or a guix shell, doesn't really > matter to me. guix shells are also cached, so it is not like I need to > rebuild > the guix shell at every call of my Makefile targets either. > > I am at the end of my wits. If anyone can point me to a reproducible project > setup, or tell me what I am doing wrong in my setup, that would be great : ) > > Best regards, > Zelphir > > -- > repositories:https://codeberg.org/ZelphirKaltstahl
