Hi everyone,
As I've been thinking about our packaging problem, I've realized that
Conda packaging might be an easy solution that solves many needs. Conda,
and its compatible alternatives Mamba, Micromamba, and Pixi, is both a
package manager and an environment manager designed to handle all
languages. It's widely used by machine learning practitioners, but also
in many other fields. It grew out of the need for easy installation of
complicated Fortran dependencies of Python machine learning libraries,
but now it can handle many use cases. Unlike Nix, its environments are
highly mutable, but this gives it the advantage that most software will
work completely fine in a Conda environment, provided that the software
does not try to use any non-Conda package mangers.
Conda allows the user to easily create self-contained environments. Each
environment can either be created on the command line and populated with
packages by the user, or created from a YAML file describing which
repositories the packages should be downloaded from and the versions of
the packages that the user needs, along with the environment's name.
Then the user can run `conda activate MyEnvironment`, and all the tools
that were installed in the environment will now be on their $PATH, ahead
of any other tools installed by the system package manager with the same
name. The $PYTHONPATH and other language-specific PATHs are also set by
activating an environment. Conda supports Linux (glibc), Windows, and
macOS, but there isn't much stopping it from being used on other
platforms other than the fact that there are no prebuilt binary packages
for other platforms. If you want to try out Conda, the easiest way is to
install Miniforge3 (https://conda-forge.org/miniforge/), which will give
you a basic environment with conda and mamba installed, configured to
install packages from the large conda-forge repository.
For us, Conda will allow us to build a single package that works on all
Conda-supported Linux distributions, and the user will be able to have
multiple versions of GNUstep installed at the same time. rattler-build
(https://github.com/prefix-dev/rattler-build) comes with many tools that
will help us build reproducible Conda packages. Projects like FreeCAD
have used Conda to provide end-user AppImages, meaning that we could
easily create AppImages based on our conda packages for any GNUstep app.
Conda is a widely-used technology, so many developers will be familiar
with it, allowing them to use GNUstep without being scared by building
GNUstep from source and possibly messing up their system.
I think the most difficult part of this is just to learn how to make
conda packages, but working with the conda-forge maintainers and with
the rattler-build maintainers might be useful. Additionally, the Conda
ecosystem has had three stages of tooling -- the `conda` stage (with the
`conda` package manager and the `conda-build` build system, sponsored by
Anaconda Inc and written in Python), the `mamba` stage (with the `mamba`
package manager and the `boa` build system, an independent project
written in C++), and the `rattler` stage (with the `pixi` package
manager and the `rattler-build` build system, sponsored by prefix.dev
and written in Rust). While the `conda` and `mamba` stages are still
here, people are transitioning to `rattler`. Each stage's tools havve
typically been significantly faster than the previous, but migrations
are slow. However, these choices are not rather large choices for us, as
they all enjoy a high degree of compatibility between each other.
Conda could provide us the ability to easily package for most systems,
provide the ability to have multiple GNUstep installations without
having to mess with GNUstep Make's configuration, make it easy for
developers to get the libraries they need, and give us the tools
necessary to turn our conda packages into self-contained AppImages (and
potentially other formats) which can be provided for end users' use.
Thanks! I'm happy to recieve anyone's thoughts or questions, as I'm not
sure if I explained it well.
