Re: [gentoo-user] Re: vm still grinding away at compiling webkitgtk-2.14.2

[John Covici]

On Mon, 21 Nov 2016 03:57:02 -0500,
Neil Bothwick wrote:
> 
> [1  ]
> On Mon, 21 Nov 2016 08:47:13 +, Peter Humphrey wrote:
> 
> > > On my i7 machine with 16g of ram, webkit-gtk has got to take 4/5 hours
> > > to compile.  This is using an ssd.  
> > 
> > That vindicates my choice of an NVMe SSD for this box, which is also an
> > i7 but with 32GB. Genlop shows it takes between 23 and 50 minutes here,
> > depending on what else I'm doing I suppose. What a difference a direct
> > connection makes!
> 
> There's more then the SSD to this. I get a similar result on my AMD9590
> box with an NVMe SSD, but my i5 laptop with 8GB and a SATA SSD takes
> "only" around 2-3 hours, on a system that should be significantly slower
> than John's.

Well, one other thing, I always do -j1 for the make options, too many
times things don't work right if I try higher numbers.

-- 
Your life is like a penny.  You're going to lose it.  The question is:
How do
you spend it?

 John Covici
 cov...@ccs.covici.com

Re: [gentoo-user] Re: vm still grinding away at compiling webkitgtk-2.14.2

[Neil Bothwick]

On Mon, 21 Nov 2016 08:47:13 +, Peter Humphrey wrote:

> > On my i7 machine with 16g of ram, webkit-gtk has got to take 4/5 hours
> > to compile.  This is using an ssd.  
> 
> That vindicates my choice of an NVMe SSD for this box, which is also an
> i7 but with 32GB. Genlop shows it takes between 23 and 50 minutes here,
> depending on what else I'm doing I suppose. What a difference a direct
> connection makes!

There's more then the SSD to this. I get a similar result on my AMD9590
box with an NVMe SSD, but my i5 laptop with 8GB and a SATA SSD takes
"only" around 2-3 hours, on a system that should be significantly slower
than John's.


-- 
Neil Bothwick

Code: (n.) a means of concealing bugs favored by programmers.
  (v.) the process of concealing bugs by programming.


pgpLeZsx_leqs.pgp
Description: OpenPGP digital signature

Re: [gentoo-user] Re: vm still grinding away at compiling webkitgtk-2.14.2

[Peter Humphrey]

On Sunday 20 Nov 2016 09:48:07 John Covici wrote:
> On Sun, 20 Nov 2016 09:25:58 -0500,
> 
> Harry Putnam wrote:
> > Rich Freeman  writes:
> > > IMO over-committing CPU isn't actually THAT bad.  The CPU obviously
> > > gets divided n ways, but that's as far as it goes.  There isn't that
> > > much overhead switching between VMs (though there certainly is some).
> > 
> > [...]
> > 
> > Thanks for the fuller picture and putting in the time to write it.
> 
> On my i7 machine with 16g of ram, webkit-gtk has got to take 4/5 hours
> to compile.  This is using an ssd.

That vindicates my choice of an NVMe SSD for this box, which is also an i7 but 
with 32GB. Genlop shows it takes between 23 and 50 minutes here, depending on 
what else I'm doing I suppose. What a difference a direct connection makes!

-- 
Regards
Peter

Re: [gentoo-user] Re: vm still grinding away at compiling webkitgtk-2.14.2

[J. Roeleveld]

On Sunday, November 20, 2016 09:06:26 AM Rich Freeman wrote:
> On Sun, Nov 20, 2016 at 8:45 AM, Harry Putnam  wrote:
> > "J. Roeleveld"  writes:
> >> Also, overcommitting CPUs has a bad influence on performance,
> >> especially if the host wants to use all cores as well.
> > 
> > That is what I asked advice about.  What do you call
> > `overcommitting'.  For example with only 1 Vbox vm started and no
> > serious work being done by the windwos-10 os.  On an HP xw8600 with
> > older 2x Xeon 5.60 3.00Ghz with 32 GB ram
> 
> IMO over-committing CPU isn't actually THAT bad.  The CPU obviously
> gets divided n ways, but that's as far as it goes.  There isn't that
> much overhead switching between VMs (though there certainly is some).

True, it's not too bad. The problem is, however, that the host-OS is special 
as it has to manage access to the various resources. For this reason, I tend 
to dedicate specific CPU-cores to the host.

> Over-committing RAM on the other hand can definitely cause more
> serious issues, because then you're dealing with swap.  Dividing 1 CPU
> 3 ways gives you 1/3rd of a CPU (but collectively the 3 VMs are
> putting out close to a full CPU's worth of work).  If you're
> over-committing RAM and you go into swap, then the performance of all
> your hosts might drop considerably, adding up to WAY less than the
> total your box is capable of.

VMWare actually did this worse with their desktop version a few years ago. Not 
sure if they still do this.
What they did was to synchronize RAM with an on-disk copy. Only way to disable 
that was to edit the VM-config file using undocumented flags. I stopped using 
VMWare shortly after that.

> If your host is windows then this isn't an option for you (seriously,
> you should re-consider that), but if you could use a linux host
> another solution is containers.  In general they are FAR more flexible
> around RAM use and of course RAM tends to be the most precious
> commodity when you're running guests of any kind.  With a container
> you don't have to pre-allocate the RAM, so if Gentoo needs 8GB of RAM
> right now it isn't as big a deal because in 15min when you're done
> building it will go back down to 100MB or whatever it actually needs
> to run.

Containers are nice, I agree. Although I tend to see it more as a much better 
implementation of a chroot-jail.

> Back when I was running Gentoo VMs I would typically set the RAM use
> to something fairly minimal (think ~1GB or less).  Then when I was
> doing updates I'd up the setting to basically all the free RAM on my
> host and allocate multiple CPU cores to it, then mount a tmpfs on
> /var/tmp.  When I was done building I'd shrink it back down to a
> normal config.  And I wouldn't be doing builds on multiple hosts at
> once.

Or use a dedicated build-server/VM. And only install binary packages on the 
VMs. This has the benefit of speeding up updates on the VMs themselves.

> These days with containers I just run emerge on a few at a time and I
> don't worry about it (still with /var/tmp on a tmpfs in each).  Now, I
> wouldn't go building chromium and libreoffice in multiple containers
> at once that way, but for typical server-like guests very few packages
> use THAT much RAM.

That would be a good way to stress-test hardware though. :)
Especially to check if the cooling is working as expected.

--
Joost

Re: [gentoo-user] Re: vm still grinding away at compiling webkitgtk-2.14.2

[Rich Freeman]

On Sun, Nov 20, 2016 at 11:00 AM, Alec Ten Harmsel
 wrote:
> On Sun, Nov 20, 2016 at 10:32:25AM -0500, Harry Putnam wrote:
>> Rich Freeman  writes:
>>
>> > Are you building in a tmpfs?  That would perform better than an ssd
>> > and would be much less wear on your flash besides.  Of course, some
>> > packages do take a while to build.  I don't notice as much now that I
>> > do most of my building from cron, but it can be painful when you have
>> > dependency chains or soname changes.
>>
>> I hope this isn't more low grade density on my part but you do mean a
>> tmpfs on the vm right?
>>
>
> I'm not Rich but I'm sure that's what he means. I have an SSD, and using
> a tmpfs for building speeds up builds significantly - probably 10-15%.
>
> This will mean that you'll need a significant amount of memory allocated
> to the VM. Mounting a tmpfs defaults to half of the memory available to
> the machine, which seems like a decent rule of thumb. If you give the VM
> 8GB of memory, the tmpfs will have 4GB of space.
>

Well, I was directing it more to John who brought up building on an
ssd (which should make fairly little difference if you're doing the
build in a tmpfs, though I'm sure it would speed up the install/clean,
and it probably would make a difference for very short package builds;
once the build is running the stuff on your ssd should be rapidly
cached anyway).

But, yes, I would DEFINITELY use a tmpfs in a VM if you can manage the
RAM.  A VM disk will perform even worse than a regular drive and there
is no need to go writing all those object files only to delete them at
the end.

You can control the space allocated to a tmpfs via a mount option.  Of
course you need to reserve RAM for the build itself, you could very
well want more than half of your RAM going to the tmpfs.  Memory for
tmpfs is only consumed when it is in use, so allowing more space use
isn't a problem as long as you don't have things that will just dump
files in the tmpfs and leave them lying around.  Your other option
would be something like zram if you're really desperate, but that
takes a bit more work and I think its allocation is fixed.

-- 
Rich

Re: [gentoo-user] Re: vm still grinding away at compiling webkitgtk-2.14.2

[Alec Ten Harmsel]

On Sun, Nov 20, 2016 at 10:32:25AM -0500, Harry Putnam wrote:
> Rich Freeman  writes:
> 
> > Are you building in a tmpfs?  That would perform better than an ssd
> > and would be much less wear on your flash besides.  Of course, some
> > packages do take a while to build.  I don't notice as much now that I
> > do most of my building from cron, but it can be painful when you have
> > dependency chains or soname changes.
> 
> I hope this isn't more low grade density on my part but you do mean a
> tmpfs on the vm right?
> 

I'm not Rich but I'm sure that's what he means. I have an SSD, and using
a tmpfs for building speeds up builds significantly - probably 10-15%.

This will mean that you'll need a significant amount of memory allocated
to the VM. Mounting a tmpfs defaults to half of the memory available to
the machine, which seems like a decent rule of thumb. If you give the VM
8GB of memory, the tmpfs will have 4GB of space.

Alec

[gentoo-user] Re: vm still grinding away at compiling webkitgtk-2.14.2

[Harry Putnam]

Rich Freeman  writes:

> Are you building in a tmpfs?  That would perform better than an ssd
> and would be much less wear on your flash besides.  Of course, some
> packages do take a while to build.  I don't notice as much now that I
> do most of my building from cron, but it can be painful when you have
> dependency chains or soname changes.

I hope this isn't more low grade density on my part but you do mean a
tmpfs on the vm right?

Re: [gentoo-user] Re: vm still grinding away at compiling webkitgtk-2.14.2

[Rich Freeman]

On Sun, Nov 20, 2016 at 9:48 AM, John Covici  wrote:
> On Sun, 20 Nov 2016 09:25:58 -0500,
> Harry Putnam wrote:
>>
>> Rich Freeman  writes:
>>
>> > IMO over-committing CPU isn't actually THAT bad.  The CPU obviously
>> > gets divided n ways, but that's as far as it goes.  There isn't that
>> > much overhead switching between VMs (though there certainly is some).
>>
>> [...]
>>
>> Thanks for the fuller picture and putting in the time to write it.
>>
>>
>
>
> On my i7 machine with 16g of ram, webkit-gtk has got to take 4/5 hours
> to compile.  This is using an ssd.
>

Are you building in a tmpfs?  That would perform better than an ssd
and would be much less wear on your flash besides.  Of course, some
packages do take a while to build.  I don't notice as much now that I
do most of my building from cron, but it can be painful when you have
dependency chains or soname changes.

-- 
Rich

Re: [gentoo-user] Re: vm still grinding away at compiling webkitgtk-2.14.2

[John Covici]

On Sun, 20 Nov 2016 09:25:58 -0500,
Harry Putnam wrote:
> 
> Rich Freeman  writes:
> 
> > IMO over-committing CPU isn't actually THAT bad.  The CPU obviously
> > gets divided n ways, but that's as far as it goes.  There isn't that
> > much overhead switching between VMs (though there certainly is some).
> 
> [...]
> 
> Thanks for the fuller picture and putting in the time to write it.
> 
> 


On my i7 machine with 16g of ram, webkit-gtk has got to take 4/5 hours
to compile.  This is using an ssd.

-- 
Your life is like a penny.  You're going to lose it.  The question is:
How do
you spend it?

 John Covici
 cov...@ccs.covici.com

[gentoo-user] Re: vm still grinding away at compiling webkitgtk-2.14.2

[Harry Putnam]

Rich Freeman  writes:

> IMO over-committing CPU isn't actually THAT bad.  The CPU obviously
> gets divided n ways, but that's as far as it goes.  There isn't that
> much overhead switching between VMs (though there certainly is some).

[...]

Thanks for the fuller picture and putting in the time to write it.

Re: [gentoo-user] Re: vm still grinding away at compiling webkitgtk-2.14.2

[Rich Freeman]

On Sun, Nov 20, 2016 at 8:45 AM, Harry Putnam  wrote:
> "J. Roeleveld"  writes:
>
>> Also, overcommitting CPUs has a bad influence on performance,
>> especially if the host wants to use all cores as well.
>
> That is what I asked advice about.  What do you call
> `overcommitting'.  For example with only 1 Vbox vm started and no
> serious work being done by the windwos-10 os.  On an HP xw8600 with
> older 2x Xeon 5.60 3.00Ghz with 32 GB ram
>

IMO over-committing CPU isn't actually THAT bad.  The CPU obviously
gets divided n ways, but that's as far as it goes.  There isn't that
much overhead switching between VMs (though there certainly is some).

Over-committing RAM on the other hand can definitely cause more
serious issues, because then you're dealing with swap.  Dividing 1 CPU
3 ways gives you 1/3rd of a CPU (but collectively the 3 VMs are
putting out close to a full CPU's worth of work).  If you're
over-committing RAM and you go into swap, then the performance of all
your hosts might drop considerably, adding up to WAY less than the
total your box is capable of.

If your host is windows then this isn't an option for you (seriously,
you should re-consider that), but if you could use a linux host
another solution is containers.  In general they are FAR more flexible
around RAM use and of course RAM tends to be the most precious
commodity when you're running guests of any kind.  With a container
you don't have to pre-allocate the RAM, so if Gentoo needs 8GB of RAM
right now it isn't as big a deal because in 15min when you're done
building it will go back down to 100MB or whatever it actually needs
to run.

In general Gentoo doesn't need a lot of RAM to operate, it is a fairly
minimal distro in general.  Now, obviously if you're running webkit
then you're running it as more of a desktop and that is going to
consume a lot more RAM than a console-based system, on any distro.  If
anything you could still tweak USE flags and CFLAGS to reduce your RAM
footprint compared to most distros (whether this is worthwhile is
another matter).  However, the one exception to this is when you're
building things.  Ideally when you're building you want to use lots of
cores, which means more gcc instances using more RAM each, and you
want to be doing all of this on a tmpfs that uses even more RAM.

Back when I was running Gentoo VMs I would typically set the RAM use
to something fairly minimal (think ~1GB or less).  Then when I was
doing updates I'd up the setting to basically all the free RAM on my
host and allocate multiple CPU cores to it, then mount a tmpfs on
/var/tmp.  When I was done building I'd shrink it back down to a
normal config.  And I wouldn't be doing builds on multiple hosts at
once.

These days with containers I just run emerge on a few at a time and I
don't worry about it (still with /var/tmp on a tmpfs in each).  Now, I
wouldn't go building chromium and libreoffice in multiple containers
at once that way, but for typical server-like guests very few packages
use THAT much RAM.


-- 
Rich

[gentoo-user] Re: vm still grinding away at compiling webkitgtk-2.14.2

[Harry Putnam]

"J. Roeleveld"  writes:

> On November 20, 2016 6:21:40 AM GMT+01:00, Alan McKinnon 
>  wrote:
>>On 20/11/2016 02:59, Harry Putnam wrote:
>>> An emerge of webkitgtk-2.14.2 has been running for over 2hrs.  I
>>> wonder if I am starving my vbox vm of gentoo with 3GB or ram.
>>> 
>>> Top's cpu usage is fluxuating between 45% and 99% during this compile
>>> 
>>> Portage pulling 99% by itself at times.
>>> 
>>> I thought 3GB was a bit high for a vm compared to what I've given
>>debian
>>> vms... can anyone give some guidance about how much ram to allow a
>>> gentoo vm in Vbox?
>>> 
>>> I'm on a windows 10 host with 32 GB ram but running a number of vms.
>>> 
>>> 
>>
>>Depends on what you are compiling. webkitgtk is a brute, so is
>>thunderbird, libreoffice and other usual culprits. Those sorts of build
>>systems need as much ram as you can spare whereas the rest of the time
>>vm itself needs as much as it needs to do whatever it is you want it to
>>do.
>>
>>What hypervisor are you using, and can you use memory ballooning?
>
> MS Windows desktop OSes are not the best platforms to run multiple VMs on.

I'm just experimenting with several OS's ... The Vbox vms I run for
actual work are hosted on Solaris x86 (openindiana)

> Also, overcommitting CPUs has a bad influence on performance,
> especially if the host wants to use all cores as well.

That is what I asked advice about.  What do you call
`overcommitting'.  For example with only 1 Vbox vm started and no
serious work being done by the windwos-10 os.  On an HP xw8600 with
older 2x Xeon 5.60 3.00Ghz with 32 GB ram

[gentoo-user] Re: vm still grinding away at compiling webkitgtk-2.14.2

[Harry Putnam]

Alan McKinnon  writes:

> On 20/11/2016 02:59, Harry Putnam wrote:
>> An emerge of webkitgtk-2.14.2 has been running for over 2hrs.  I
>> wonder if I am starving my vbox vm of gentoo with 3GB or ram.
>> 
>> Top's cpu usage is fluxuating between 45% and 99% during this compile
>> 
>> Portage pulling 99% by itself at times.
>> 
>> I thought 3GB was a bit high for a vm compared to what I've given debian
>> vms... can anyone give some guidance about how much ram to allow a
>> gentoo vm in Vbox?
>> 
>> I'm on a windows 10 host with 32 GB ram but running a number of vms.
>> 
>> 
>
> Depends on what you are compiling. webkitgtk is a brute, so is
> thunderbird, libreoffice and other usual culprits. Those sorts of build
> systems need as much ram as you can spare whereas the rest of the time
> vm itself needs as much as it needs to do whatever it is you want it to do.
>
> What hypervisor are you using, and can you use memory ballooning?

Sorry I meant to include that info... virtualbox-5.1.8