Re: How much memory do I need for buildworld?

[Nick Hibma]

 
 Nov 22 16:55:13 mercury kernel: swap_pager: out of swap space
 Nov 22 16:55:13 mercury kernel: swap_pager_getswapspace(16): failed
 Nov 22 16:55:13 mercury kernel: pid 22841 (tblgen), uid 0, was killed:
 out of swap space
 
 This machine has 256MB of RAM and one 64MB swap partition.
 
 This is most likely the problem: you need more RAM for this particular
 instance of tblgen.  On my -CURRENT i386 box, it takes ~369MiB of RSS to
 build the X86 disassembler tables.
 
 I'm surprised you didn't run into OOM problems earlier, with so little
 memory.  For such router like machines, it is obviously easier to do
 the build on a fast desktop machine, then install over NFS, or rsync
 /usr/src and /usr/obj to the target machine.

I suggest you have a look at NanoBSD in /usr/src/tools/tools/nanobsd.sh. It 
will build a disk image for you in roughly 1 hour on a fast machine with 4+ 
processors by doing a complete world. It allows you to regenerate the image and 
update a running image easily. If you keep your changes separated in /cfg you 
will find that you can actually work really quickly in this setup (even though 
not being able to install packages on the image directly is sometimes a bit of 
a nuisance).

We've wrapped that script with a lot of our own stuff and go from initial 
config to running in VM in less than 5 minutes, 2.5 minutes for an update after 
that (using an SSD on the host and FreeBSD in a single processor VM generating 
the image).

Nick


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: How much memory do I need for buildworld?

[Dimitry Andric]

On 22 Nov 2014, at 16:51, Rostislav Krasny rosti@gmail.com wrote:
 I've a fresh FreeBSD 10.1 installed on an old 32-bit machine. I've
 checked out revision 274850 of the base sources and ran 'make
 buildworld'. After some time it has failed:
 
 === lib/clang/libllvmx86disassembler (depend)
 tblgen -gen-disassembler  -I
 /usr/src/lib/clang/libllvmx86disassembler/../../../contrib/llvm/include
 -I 
 /usr/src/lib/clang/libllvmx86disassembler/../../../contrib/llvm/lib/Target/X86
 -d X86GenDisassemblerTables.inc.d -o X86GenDisassemblerTables.inc.h
 /usr/src/lib/clang/libllvmx86disassembler/../../../contrib/llvm/lib/Target/X86/X86.td
 *** Signal 9
 
 Stop.
 make[4]: stopped in /usr/src/lib/clang/libllvmx86disassembler
 *** Error code 1
 
 
 According to /var/log/messages it was killed because of out of memory:
 
 Nov 22 16:55:13 mercury kernel: swap_pager: out of swap space
 Nov 22 16:55:13 mercury kernel: swap_pager_getswapspace(16): failed
 Nov 22 16:55:13 mercury kernel: pid 22841 (tblgen), uid 0, was killed:
 out of swap space
 
 This machine has 256MB of RAM and one 64MB swap partition.

This is most likely the problem: you need more RAM for this particular
instance of tblgen.  On my -CURRENT i386 box, it takes ~369MiB of RSS to
build the X86 disassembler tables.

I'm surprised you didn't run into OOM problems earlier, with so little
memory.  For such router like machines, it is obviously easier to do
the build on a fast desktop machine, then install over NFS, or rsync
/usr/src and /usr/obj to the target machine.


 Previously
 I used FreeBSD 7.4-CURRENT on this machine with two swap partitions of
 64MB each and never had such a problem. I use it as a router, i.e.
 there is no X and no other memory greedy processes.

Yes, 7.4 is much smaller, and does not have any big C++ programs in it,
so a buildworld is more likely to succeed on small memory machines.


 Could it be related to an llvm bug fixed by following commit?
 
 http://svnweb.freebsd.org/base?view=revisionrevision=274696

No, this is unlikely.  This problem only occurred 1) in the compiler
itself, 2) very specific ports, and 3) when compiling with debug info
enabled.

-Dimitry



signature.asc
Description: Message signed with OpenPGP using GPGMail