On Wed, Jul 05, 2017 at 08:49:52PM -0700, Chris Marusich wrote: > "Feng Shu" <[email protected]> writes: > > > Now I have found that 'guix pull' is too slow, > > I need 3 hours to compile guix, is it possible to speed it ? > > 3 hours is longer than it usually takes on my Lenovo X200 laptop with 2 > cores, an SSD, and 8 GB of RAM. Usually, for me, it seems to take about > 15 minutes.
Guix pull takes a while and building Guix in general is slow too. I find I have to rebuild quite often (updating trunk, moving between machines) and with having my own guix publish server building Guix is the bottle neck(!) From our discussion on Friday I gather it is going to get a lot worse with the growing number of packages. It scales probably worse than linear. What will it be like with 15K packages? We will get there. We can actually try it now by doubling the package tree - anyone wants to try and create a simulation? I.e., not just double the tree, make sure there are cross references between the two graphs by modifying some of the inputs at random. This leads to the following thought: why don't we create a 'lazy' build. There are multiple ways to go about it (I think). One would be to parse scheme files for package names and only compile those that are needed when someone invokes a guix command (and have not been compiled yet). Or generate a meta list for a source tree. Or subcategorize packages so only those packages get included that are asked for (assuming there are no deeper dependencies). For example, few people need the bioinformatics packages. We could have the sub section of the graph split out and have people do: guix package --topic=bio -i samtools for example and compile the contents gnu/packages/bio/ directory when that happens the first time for a specific checkout. This would be a lot more scalable and the main package section could be fairly small and build fast. Sectioning the graph may be hard (you'd be inclined to section off languages and window managers), but I think it can be dictated by whether a sub graph can live on its own. And if we allow packages to reference packages in a sub graph (triggering the build) then anything goes - the JVM subsystem could be in one and Perl CPAN in another. I think scalability is a good goal and instant compilation another ;). A few years back it just took 30 seconds to build Guix. Pj.
