Re: [gentoo-user] emerge --emptytree : how to ?

[Dale]

Walter Dnes wrote:
> On Sat, Oct 14, 2017 at 06:25:35AM -0500, Dale wrote
>
>> Some of my make.conf entries.  You may not need all of these so edit out
>> what you don't want or change values if you need to.  I have a four core
>> CPU. 
>>
>> FEATURES="-usersync -userpriv -usersandbox buildpkg sandbox parallel-fetch"
>>
>> MAKEOPTS="-j5"
>   There is some controversy over setting MAKEOPTS=${number of threads}
> possibly being better than MAKEOPTS=${number of threads} + 1
> https://blogs.gentoo.org/ago/2013/01/14/makeopts-jcore-1-is-not-the-best-optimization/
>


I tried different settings before that one and settled on the one that
works best.  It may not work on someone else's system but it does fine
here.  One thing about most of my settings, they've been tested pretty
well and work well on this system at least. 

Since I have a four core system, I usually set everything to number of
cores plus one which should lead to a always busy CPU.  So far, it has
worked out that way for the most part.  If I had enough memory, I might
could up that but it could lead to another problem I had before lowering
it to that number.  Compiles that fail.  It's been a while but at one
point, some packages had to have -j 1 to compile.  That is rare nowadays
I guess but since the settings I have works for me, I'll likely leave
them like they are. 

The one thing I did have to change recently, not compiling some large
packages on tmpfs.  If it was just one package I have enough memory. 
However, sometimes it would be two and sometimes even three that were
large.  The main culprits were Seamonkey, Firefox and Libreoffice.  I
think I added a couple other large ones in just to be sure but other
than that, these settings have been around for a good long while. 

I might add, I've read that blog before.  I also read the comments where
others had different results and pointed out some issues with the blog
points.  The blogger used kdelibs as a test case.  Thing is, I compile
more than just kdelibs here.  Most of the time I start my emerges before
I go to sleep.  I put Gkrellm where I can see it and I watch what the
CPUs are doing.  Generally speaking, the CPU cores stay busy the whole
time.  There are a few times that it is not at 100% but generally it
is.  If anything, since it does have times where it isn't at 100%, I may
need to up that number by one to see if that keeps it more busy.  Thing
is, in the past, it didn't help any. 

Thanks for the info. 

Dale

:-)  :-) 

Re: [gentoo-user] emerge --emptytree : how to ?

[R0b0t1]

On Sun, Oct 15, 2017 at 8:51 PM, Walter Dnes  wrote:
> On Sat, Oct 14, 2017 at 06:25:35AM -0500, Dale wrote
>
>> Some of my make.conf entries.  You may not need all of these so edit out
>> what you don't want or change values if you need to.  I have a four core
>> CPU.
>>
>> FEATURES="-usersync -userpriv -usersandbox buildpkg sandbox parallel-fetch"
>>
>> MAKEOPTS="-j5"
>
>   There is some controversy over setting MAKEOPTS=${number of threads}
> possibly being better than MAKEOPTS=${number of threads} + 1
> https://blogs.gentoo.org/ago/2013/01/14/makeopts-jcore-1-is-not-the-best-optimization/
>

I received the same results when I tested it myself. On modern servers
especially, RAM access speed seems to be matched to core processing
capability very well.

Strangely, adding to the build thread count doesn't seem to help when
disk IO is the bottleneck. It is conceivable that it could but in
practice the location of build data and files seems to be disperse
enough that there are no great access optimizations, and each read
blocks individually.

Cheers,
 R0b0t1

Re: [gentoo-user] emerge --emptytree : how to ?

[Walter Dnes]

On Sat, Oct 14, 2017 at 06:25:35AM -0500, Dale wrote

> Some of my make.conf entries.  You may not need all of these so edit out
> what you don't want or change values if you need to.  I have a four core
> CPU. 
> 
> FEATURES="-usersync -userpriv -usersandbox buildpkg sandbox parallel-fetch"
> 
> MAKEOPTS="-j5"

  There is some controversy over setting MAKEOPTS=${number of threads}
possibly being better than MAKEOPTS=${number of threads} + 1
https://blogs.gentoo.org/ago/2013/01/14/makeopts-jcore-1-is-not-the-best-optimization/

-- 
Walter Dnes 
I don't run "desktop environments"; I run useful applications

Re: [gentoo-user] emerge --emptytree : how to ?

[Dale]

Helmut Jarausch wrote:
> Hi,
> I think I'm in need of doing 
> emerge --emptytree ...
> for the first time.
> Can I do it on a running Gentoo system?
> I expected it will take several days to complete.
> This poses some problems to me.
> First, I have to shut down my machine overnight.
> Second, I haven't made good experience with --keep-going
> nor with --resume.
> What can I do if 'emerge -e ...' fails by itself or because I have to
> shut my machine down?
>
> Many thanks for some hints,
> Helmut
>
>


I do this on occasion when some update makes things go weird.  I'll look
for anyone else having the issue and if not, then I do a emerge -e world
to see if it helps. 

First, I have entries in make.conf to help make it so that it doesn't
affect what I'm doing.  I use ionice, -j and friends to do that.  I also
use --keep-going as well.  Generally, I can't tell it is doing
anything.  There is a few packages that it slows things down for a few
minutes.  It doesn't do it for the whole compile process, just a few
minutes of it.  Firefox, Libreoffice is two that I recall. I'll post
some of my make.conf items below. 

Second, you can skip certain programs, large ones for example.  You can
for example add this:  --exclude libreoffice  That will let it skip
libreoffice but keep in mind, some dependencies may be skipped as well,
if nothing else depends on them.  I haven't tested that but that's my
thinking.  Maybe someone else has more ideas on that. 

Third, I'm almost certain --resume works even after a reboot.  Just keep
in mind, if it was in the middle of a package compile, it likely will
start over from scratch.  That's my experience at least. 

Some of my make.conf entries.  You may not need all of these so edit out
what you don't want or change values if you need to.  I have a four core
CPU. 

FEATURES="-usersync -userpriv -usersandbox buildpkg sandbox parallel-fetch"

MAKEOPTS="-j5"

EMERGE_DEFAULT_OPTS="--with-bdeps y --backtrack=100 --keep-going -v -j5
--quiet-build=n -1 --unordered-display"

PORTAGE_NICENESS=5

PORTAGE_IONICE_COMMAND="ionice -c 3 -p \${PID}"

As for the command I use, emerge -ea world.  If you have to shutdown for
a while, once you reboot, try emerge --resume and see if it works.  It
should.  I've done it before but its been a good while back.

Hope that helps.

Dale

:-)  :-)