Another "hilarious" consequence of attempting to side-step "the guix" way. If I just follow the `pip install` steps I end up with some C++ dependency (blspy I believe) complaining that it can't find libstdc++. Quick `locate` search unsurfaced it somewhere under /gnu/store, but of course binaries (like binary wheels) pre-built this way are completely oblivious to Linux machines (or I suppose Guix) that don't have the usual /lib or /lib64 or whatever. So yeah, looks like its guix way or no way. Like I mentioned earlier, could use some guidance in packaging this.
On Fri, 16 Apr 2021 at 09:57, Vladilen Kozin <[email protected]> wrote: > > Hi all. > > Could a kind soul handhold me while I try to port a python package to > guix please. I am not a python programmer, so not on first name basis > with python ecosystem and new to guix - double trouble. > > I've made some progress, but could use a bit of your help. Current > state of affairs is tracked in this repo: > https://github.com/vkz/ze-guix > > Repo and package I'm trying to import: > https://github.com/Chia-Network/chia-blockchain > > Both it and all of its dependencies are open source, most on pypi but > sadly some are only available there as binary wheels. Building from > source entirely maybe complicated since it very quickly escalates to > multiple build systems e.g. chiapos (proof of space lib) and chiavdf > are C++ and rely on cmake, clvm_rs (their homegrown lisp) is written > in Rust etc. > > Conceptually, installation is very simple and basically relies on > binary wheels and amounts to: > - checkout repo, > - init and update submodule(s) - really only mozilla-ca, > - create virtual env and activate, > - upgrade to the latest pip to use binary wheels, > - install deps and the package with pip > > You can see the steps I extracted from their install.sh here: > https://github.com/vkz/ze-guix/blob/master/packages/manual-install.log > > Now, guix obviously heads and shoulders above virtualenv or at least > it should be, however trying to build from source escalates quickly as > I've mentioned. So, first question is if I could define a package by > "replicating" the above essentially binary-only steps but replace > virtualenv with e.g. guix profile or smth. This should be possible to > do in code - as a proper package with some rudimentary build system > that amounts to building everything inside a separate profile or maybe > just pulling binary wheels into a directory in store and then adding > that to whatever Python or pip expect and use as their PATH > equivalent. IMO doing something trivial like this should be easy. To > be sure this approach has security and "freedom" implications, but > this is orthogonal to having something that works installed quickly. > Building from source is very often involved borderline crazy. Could > someone kindly teach me how to do this "binary" only packaging. > > Now, to building from source. > > First step I tried was to `guix import pypi -r`. This fails because > not all deps have source pushed to pypi and only offer binary wheels. > IMO `guix import` has no business failing like this or even at all for > whatever reason. It is best effort tool to begin with that should > result in a template that I can fill in. If there are errors - skip > the dependency, report it, but please continue. > > Since, recursive `guix import` failed me I had to manually track deps > and essentially call `guix import` for each one. Result you can see > here: > https://github.com/vkz/ze-guix/blob/master/packages/chia.scm > > This won't build for several reasons. Not all deps are there, some > probably shouldn't be like win32 something or other. Another reason, > I'd have to figure out how to build the inputs that require cmake and > rust and python (some all at once). And this is where I could really > use help. Examples as mentioned earlier are these dependencies: > https://github.com/Chia-Network/chiapos > https://github.com/Chia-Network/clvm_rs > maybe others - it is a fairly big project and I've not even touched > NPM/node that they rely on for their GUI client. That's another scary > can of worms I'm not prepared to peek inside at this time. > > Ok, so really two questions: > - how can I build a "binary" only python package that simply relies on > binary wheels e.g. from pypi, > - how to deal with multi-build systems e.g. python + cmake or python + > rust, maybe someone could think of a good educational (so, not too > convoluted) example of such package already in source tree. > > Latter would probably go much faster and teach better if done as > pair-programming exercise, but that maybe a lot to ask. If you're up > for it, I'm game though. > > Thank you > -- > Best regards > Vlad Kozin -- Best regards Vlad Kozin
