On 11/07/18 09:27, Chavdar Ivanov wrote:
On 11/06/18 18:50, Havard Eidnes wrote: Yes. I had a similar problem. The build would fill up the /tmp/ directory and die from exhausted resources. I had /tmp/ created with tmpfs and had a constraint of 64M. The answer for me was to create /tmp in /etc/fstab with tmpfs and no size constraint. Then Rust would build, but it still took a long time. Yes, rust is an absolute Pig resource-wise. Not only does it carry a copy of llvm inside itself (it most probably has to), it also carries a nubmer of other packages inside. Plus, the build builds most parts at least twice if I've been able to observe correctly. When building on NetBSD/amd64 8.0, I noticed that the work/ directory after a "make" consumes in the order of 10G disk space, possibly more when cross-building (my current work/ is 13G, which is an unfinished cross-build because I hit build issues...). It is conceivable that the storage could be reduced somewhat (but probably not by much?) by tweaking src/bootstrap/boostrap.py to say -Cdebuginfo=0 instead of 2 for RUSTFLAGS. So that you have an idea what to expect: My amd64 build host has 8G real memory, and a 2G tmpfs /tmp, and ... it didn't run out of space anywhere :) On this particular host (i7 3rd-gen, 4 real cores, 8 w/HT, pkgsrc and system on SSD) the build of 1.29.2 completed in a little over 2 hours wall-clock time, csh's "time" report at the end of the build was 40468.007u 1958.277s 2:04:42.18 567.0% 0+0k 10302+100556io 129329pf+0w On my 16GB HP Envy, 4c/8t, I got overnight: --- Build completed successfully in 4:22:05 make 14672.75s user 36893488148207.93s system -2147483648% cpu 4:43:37.28 total /usr/pkgsrc/lang/rust ---- with /tmp just a standard directory, tmpfs as usual on /var/shm. This is more than twice as slow compared to the last few rust builds, which also took just over two hours. I wonder if something was changed in the build process, previous builds were highly parallelized, with all the threads being seen busy during various stages, whereas this time I didn't see anything like this. All my builds were off unmodified pkgsrc lang/rust directory with nothing in /etc/mk.conf that I can see to affect this. I suspect the ridiculous times for system and cpu are from zsh 'time'. Follow up: 'make test' killed the machine, there was no coredump. It had fully loaded the system, though - with all 8 threads up to 100%. Previous versions usually completed the test, albeit with errors. This is on uname -a NetBSD tarkus 8.99.25 NetBSD 8.99.25 (GENERIC) #0: Sat Nov 3 03:33:22 GMT 2018 sysbuild@ymir:/home/sysbuild/sysbuild/amd64/obj/home/sysbuild/src/sys/arch/amd64/compile/GENERIC amd64 So, yes, the build makes fairly good use of the multiple cores; notice the 567.0%, which, if I've understood correctly, indicates approx. 5.7* parallelism on average. The last version I managed to build on one of my NetBSD/macppc 8.0 machines (a single-core 1.5GHz G4 Mac Mini, 1GB memory) was 1.29.2, the build took nearly 29 hours wallclock time. This one doesn't have a tmpfs, and has a single file system with ~40G free, so it also didn't run into any barriers on the /tmp front either. Regards, - Håvard
