Re: Bulk buils space requirements
On Fri, Oct 22, 2010 at 11:23 AM, Matthew Dillon dil...@apollo.backplane.com wrote: If you are going to use tmpfs then configure at least 16G of swap space. Up to 32G of swap can be configured with a default i386 kernel (and up to 512G for a x86-64 kernel by default). 64-bit Installer gave error saying only 8GB of swap can be configured :-( How do I configure more? after install? so should I leave free space on my disk for that? thanks --Siju
Re: Bulk buils space requirements
On 10/22/2010 7:53, Matthew Dillon wrote: If you are going to use tmpfs then configure at least 16G of swap space. Up to 32G of swap can be configured with a default i386 kernel (and up to 512G for a x86-64 kernel by default). Was this raised recently? Seems the installer wasn't adjusted to it. I'll do it then. Sascha
Re: Bulk buils space requirements
On 11/18/2010 11:03, Siju George wrote: On Fri, Oct 22, 2010 at 11:23 AM, Matthew Dillon dil...@apollo.backplane.com wrote: If you are going to use tmpfs then configure at least 16G of swap space. Up to 32G of swap can be configured with a default i386 kernel (and up to 512G for a x86-64 kernel by default). 64-bit Installer gave error saying only 8GB of swap can be configured :-( How do I configure more? after install? so should I leave free space on my disk for that? It would be better if bugs were reported on bugs@ instead of on us...@. Regards, Sascha
Re: Bulk buils space requirements
: :On 10/22/2010 7:53, Matthew Dillon wrote: : If you are going to use tmpfs then configure at least 16G of : swap space. Up to 32G of swap can be configured with a default : i386 kernel (and up to 512G for a x86-64 kernel by default). : :Was this raised recently? Seems the installer wasn't adjusted to it. :I'll do it then. : :Sascha I think it just comes down to recognizing the utility of tmpfs and the need for more swap space to accomodate it. For the installer I think the auto-generated swap size should be based on the size of the disk, leaving in the minimum based on the amount of memory. so e.g. we do not want to configure 32G of swap if the hard drive is only 64G. Say, no more than 10% of the disk (maybe even no more than 5%). I'm not sure what the best calculation should be. -Matt Matthew Dillon dil...@backplane.com
Bulk buils space requirements
HI, I got around 250 GB free on my desktop. I would like to try out a bulkbuild of pkgsrc ;-) will that space be enough? is reading http://www.netbsd.org/docs/pkgsrc/bulk.html enough or should I be knowing some dfly specific things? also how long will it take on an x86 port with 3 GB RAM dmesg here http://pastie.org/1237437 thanks --Siju
Re: Bulk buils space requirements
* Siju George wrote: HI, I got around 250 GB free on my desktop. I would like to try out a bulkbuild of pkgsrc ;-) will that space be enough? is reading http://www.netbsd.org/docs/pkgsrc/bulk.html enough or should I be knowing some dfly specific things? This is clearly enough. The official document speaks about roughly 35GB, so if you add another 10GB you should be fine. Cheers Matthias
Re: Bulk buils space requirements
On Thu, October 21, 2010 2:36 am, Siju George wrote: HI, I got around 250 GB free on my desktop. I would like to try out a bulkbuild of pkgsrc ;-) will that space be enough? I have some scripts that work as a wrapper around the bulk builds I do; this may be more than you need, but I'd like to see if they make sense to someone who is not me: http://www.shiningsilence.com/simplepbulk/ The two caveats for you: - Bulk builds take a week on decent hardware; you're building over 10,000 packages, after all. The limited_list option in the pbulk config can let you limit it to certain packages, which you may want to do. - If you're doing this on Hammer, keep an eye on disk usage. It generates a huge amount of disk activity if you build everything, and Hammer will happily keep track of all those changes. I've filled terabyte disks unintentionally by performing multiple builds.
Re: Bulk buils space requirements
On Fri, Oct 22, 2010 at 1:10 AM, Justin C. Sherrill jus...@shiningsilence.com wrote: On Thu, October 21, 2010 2:36 am, Siju George wrote: HI, I got around 250 GB free on my desktop. I would like to try out a bulkbuild of pkgsrc ;-) will that space be enough? I have some scripts that work as a wrapper around the bulk builds I do; this may be more than you need, but I'd like to see if they make sense to someone who is not me: http://www.shiningsilence.com/simplepbulk/ Ok thanks :-) The two caveats for you: - Bulk builds take a week on decent hardware; you're building over 10,000 packages, after all. The limited_list option in the pbulk config can let you limit it to certain packages, which you may want to do. ok, is there a problem if a build is stopped in between? Can it resume from that point later. I won't be able to keep this system up for a week continously :-( - If you're doing this on Hammer, keep an eye on disk usage. It generates a huge amount of disk activity if you build everything, and Hammer will happily keep track of all those changes. I've filled terabyte disks unintentionally by performing multiple builds. Is it possible to mount the /bulk pfs i made for it with nohistory ? Other wise I guess I will have to run hammer cleanup /bulk occassionally ? thanks :-) --Siju
Re: Bulk buils space requirements
:ok, is there a problem if a build is stopped in between? Can it resume :from that point later. :I won't be able to keep this system up for a week continously :-( Once you've done the initial setup you can set it up to pickup where it left off, yah. :... : unintentionally by performing multiple builds. : : :Is it possible to mount the /bulk pfs i made for it with nohistory ? :Other wise I guess I will have to run hammer cleanup /bulk occassionally ? : :thanks :-) : :--Siju Just setting the retention to a few days in 'hammer viconfig /bulk' and giving it a good 25 minutes to prune should do it: snapshots 1d 2d prune 1d 30m Even better, run a tmpfs for your /bulk/pbulk_chroot/usr/pkg, because that is the directory tree that the bulk build continuously tears down and rebuilds for each package. Once you've done the initial setup and it has created the chroot, then you can do the mount_tmpfs. If you are going to use tmpfs then configure at least 16G of swap space. Up to 32G of swap can be configured with a default i386 kernel (and up to 512G for a x86-64 kernel by default). It will take some time to get used to the scripts, during which you will probably be telling it to do it from scratch quite a bit. But once you get used to the script you can get an incremental build/rebuild going fairly easily. -Matt Matthew Dillon dil...@backplane.com
