Pjotr Prins <pjotr.publi...@thebird.nl> skribis: > What will it be like with 15K packages? We will get there. We can > actually try it now by doubling the package tree - anyone wants to try > and create a simulation? I.e., not just double the tree, make sure > there are cross references between the two graphs by modifying some of > the inputs at random. > > This leads to the following thought: why don't we create a 'lazy' > build. There are multiple ways to go about it (I think). One would be > to parse scheme files for package names and only compile those that > are needed when someone invokes a guix command (and have not been > compiled yet). Or generate a meta list for a source tree.
Note that there are several things we could do: • Not compile gnu/packages/*.scm at all and instead turn on Guile’s auto-compilation. As things are currently, the first ‘guix package’ invocation would take ages though. • Never compile gnu/packages/*.scm and instead interpret it, though that’s currently relatively slow and probably more memory-consuming. I think with ‘wip-pull-reload’ (I’ll resume work on it, I promise!) things should already be nicer, and then, we should keep improving the compiler (Andy already significantly improved CPU consumption in Guile 2.2.3). > Or subcategorize packages so only those packages get included that are > asked for (assuming there are no deeper dependencies). For example, > few people need the bioinformatics packages. We could have the sub > section of the graph split out and have people do: > > guix package --topic=bio -i samtools > > for example and compile the contents gnu/packages/bio/ directory when > that happens the first time for a specific checkout. I think that should be the last resort, but yeah. Another thing we could do to speed up lookup-by-name is to maintain a cache that maps package names to modules (currently we always traverse all the package modules with ‘fold-packages’.) > I think scalability is a good goal and instant compilation another ;). > A few years back it just took 30 seconds to build Guix. Yep, scalability is this year’s challenge! Ludo’.