Re: [gentoo-user] Intel ucode updates for ME issues?

[taii...@gmx.com]

On 11/23/2017 12:47 AM, R0b0t1 wrote:


I think the information I outlined is a pretty good argument for assuming the ME
can not be disabled.

Even if true, there's not much to be done about it anyway
Yeah it certainly can't be disabled (I argue this point on a regular 
basis to no avail), as in non functional as it is involved in the 
pre-BIOS-boot process.
A certain low-morals company claims that they "disable" it with 
me_cleaner (they also infer they made it) but that is impossible.


To me disabled is no electricity flowing through it/physically 
disconnected and that couldn't be the case without enough money and 
resources to the point where one could simply make a POWER laptop with 
the current lot of POWER9 CPU's (ie: downclock and do some power saving 
engineering) - so de-facto impossible.

Re: [gentoo-user] Intel ucode updates for ME issues?

[R0b0t1]

On Wed, Nov 22, 2017 at 10:36 PM, taii...@gmx.com  wrote:
> On 11/22/2017 11:16 PM, R0b0t1 wrote:
>
>> Does anyone have more information on this? Has anything been
>> published? I'm interested in exploiting my own computers so I can
>> control the ME.
>
> It seems that it is the same people who figured out HAP mode but they
> haven't made a blog update I would ask on the coreboot mailinglist, there
> are some very smart people there.
>
> Although I doubt you will find any real information anywhere at all due to
> the recent "white hat" tendency to restrict the real nuts and bolts info and
> utilities to wealthy corporations instead of us peons who *gasp* might do
> something "bad" with it/don't have lots of money to pay for a "premier"
> support account.
>

This does make me sad. In a case such as this it makes the most sense
to me that the details be released so people who want to control their
devices are allowed to do so before the holes are patched.

> I am curious as to why you wish to do this, considering you can buy a libre
> firmware owner controlled motherboard with better functionality (ex:
> OpenBMC) than any me/psp board for only $250 and $100 for a FX-8310
> equivalent cpu.
>

I attempted to use some vPro/ME functionality and found it broken or
unsuable. So, I suppose I want access to the ME so I can use it for
what it was advertised to do. Currently I have not gotten it to do any
of those things, and its security is unprovable.

> On 11/22/2017 11:18 PM, R0b0t1 wrote:
>
>> On Wed, Nov 22, 2017 at 6:03 PM, taii...@gmx.com  wrote:
>>>
>>> Using ME cleaner would also solve the issue and you wouldn't need any
>>> more
>>> firmware updates when the next "bug" comes around.
>>>
>> Intel ME has been found to remain active after being disabled, and
>> some motherboards that do not ship as "vPro enabled" and consequently
>> haven't had the licensing paid for certain features have been found
>> with those same features enabled. I own an Asus laptop which is
>> affected. Some Asus forum post reported that there's a Java-based SOAP
>> webserver listening on the port associated with Intel ME. Intel ME is
>> not visible to the BIOS, and so it can't be turned any more "off."
>
> I understand the limitations of me_cleaner, although in this case it would
> in fact solve the problems as all the currently *publicly* discovered "bugs"
> are all ME feature exploits (and the features are removed by me_cleaner)
> rather than exploits of the ME kernel although I am certain that one is on
> the way.
>
> Believe me I know what I am talking about, I regularly provide support on
> the coreboot mailinglist and I own a variety of devices that are owner
> controlled with libre firmware (and of course no ME/PSP).

Well, at no point did I question your aptitude, but I think the
information I outlined is a pretty good argument for assuming the ME
can not be disabled.

Even if true, there's not much to be done about it anyway.

Cheers,
 R0b0t1

Re: [gentoo-user] Intel ucode updates for ME issues?

[taii...@gmx.com]

On 11/22/2017 11:16 PM, R0b0t1 wrote:


Does anyone have more information on this? Has anything been
published? I'm interested in exploiting my own computers so I can
control the ME.
It seems that it is the same people who figured out HAP mode but they 
haven't made a blog update I would ask on the coreboot mailinglist, 
there are some very smart people there.


Although I doubt you will find any real information anywhere at all due 
to the recent "white hat" tendency to restrict the real nuts and bolts 
info and utilities to wealthy corporations instead of us peons who 
*gasp* might do something "bad" with it/don't have lots of money to pay 
for a "premier" support account.


I am curious as to why you wish to do this, considering you can buy a 
libre firmware owner controlled motherboard with better functionality 
(ex: OpenBMC) than any me/psp board for only $250 and $100 for a FX-8310 
equivalent cpu.


On 11/22/2017 11:18 PM, R0b0t1 wrote:


On Wed, Nov 22, 2017 at 6:03 PM, taii...@gmx.com  wrote:

Using ME cleaner would also solve the issue and you wouldn't need any more
firmware updates when the next "bug" comes around.


Intel ME has been found to remain active after being disabled, and
some motherboards that do not ship as "vPro enabled" and consequently
haven't had the licensing paid for certain features have been found
with those same features enabled. I own an Asus laptop which is
affected. Some Asus forum post reported that there's a Java-based SOAP
webserver listening on the port associated with Intel ME. Intel ME is
not visible to the BIOS, and so it can't be turned any more "off."
I understand the limitations of me_cleaner, although in this case it 
would in fact solve the problems as all the currently *publicly* 
discovered "bugs" are all ME feature exploits (and the features are 
removed by me_cleaner) rather than exploits of the ME kernel although I 
am certain that one is on the way.


Believe me I know what I am talking about, I regularly provide support 
on the coreboot mailinglist and I own a variety of devices that are 
owner controlled with libre firmware (and of course no ME/PSP).

Re: [gentoo-user] Intel ucode updates for ME issues?

[R0b0t1]

On Wed, Nov 22, 2017 at 6:03 PM, taii...@gmx.com  wrote:
> Using ME cleaner would also solve the issue and you wouldn't need any more
> firmware updates when the next "bug" comes around.
>

Intel ME has been found to remain active after being disabled, and
some motherboards that do not ship as "vPro enabled" and consequently
haven't had the licensing paid for certain features have been found
with those same features enabled. I own an Asus laptop which is
affected. Some Asus forum post reported that there's a Java-based SOAP
webserver listening on the port associated with Intel ME. Intel ME is
not visible to the BIOS, and so it can't be turned any more "off."

Cheers,
 R0b0t1

Re: [gentoo-user] Intel ucode updates for ME issues?

[R0b0t1]

On Tue, Nov 21, 2017 at 11:42 PM, Adam Carter  wrote:
> I notice that an update for sys-firmware/intel-microcode just come through
> on ~amd64, does that address the ME issues?
>

No. As a sidenote, microcode updates can only remove or patch out
functionality. They can't modify functionality in complex ways.

> http://www.zdnet.com/article/intel-weve-found-severe-bugs-in-secretive-management-engine-affecting-millions/
>

Does anyone have more information on this? Has anything been
published? I'm interested in exploiting my own computers so I can
control the ME.

> Or will my NUC need a firmware update?

It is possible that this can't be fixed. Early versions of ME (at
least) had secret Huffman decoding tables designed into the ASIC that
were used to decompress the firmware. I am not sure if it is possible
to change these.

Cheers,
 R0b0t1

Re: [gentoo-user] Re: FYI: Daily / weekly / monthly cron jobs run twice on DST - non-DST transition

[John Campbell]

I don't know if this has been suggested yet, but I run cron on UTC,
which doesn't do daylight saving time. It's an option in cronie to set
the TZ for crontab.  I just have to transcode times from local to UTC
when setting up the job.

Re: [gentoo-user] Intel ucode updates for ME issues?

[taii...@gmx.com]

On 11/22/2017 12:42 AM, Adam Carter wrote:


I notice that an update for sys-firmware/intel-microcode just come through
on ~amd64, does that address the ME issues?

http://www.zdnet.com/article/intel-weve-found-severe-bugs-in-secretive-management-engine-affecting-millions/

Or will my NUC need a firmware update?

That would be "solved"[1] via a firmware update, microcode update is 
microcode - only for the cpu.
If you don't get one for your hardware due to the vendor saying it is 
"too old" (to scam you to buy a new motherboard for no reason) you can 
bisect the BIOS update and add it yourself (ask on the coreboot 
mailinglist how to do this for more info) not too difficult.


Using ME cleaner would also solve the issue and you wouldn't need any 
more firmware updates when the next "bug" comes around.



[1] Intel ME/AMD PSP will always be full of security "bugs" as they are 
designed to be an uber backdoor for god knows who - one can avoid this 
via getting either a slightly older x86-64 setup such as 
KCMA-D8/KGPE-D16 opteron motherboards (RYF libre firmware and a libre 
bmc firmware is available for them they also don't need microcode updats 
for series 2 CPU's), a g505S laptop (open source init firmware 
available) or a TALOS 2 server/workstation (POWER9, very very high 
performance high end server hardware with the usual price for that level 
of performance but you get libre firmware AND libre hardware RYF 
certification pending on release)

Re: [gentoo-user] is multi-core really worth it?

[Neil Bothwick]

On Wed, 22 Nov 2017 09:57:29 +0100, David Haller wrote:

> Hm. What's this EMERGE_DEFAULT_OPTS though? ;)) And
> how can I override them if not wanted? .oO( gotta read up on that )Oo.

man make.conf
man emerge
emerge --ignore-default-opts


-- 
Neil Bothwick

The trouble with life is that you are halfway through it before you
realize it's a "do it yourself" thing.


pgpQ6sEEnLpbF.pgp
Description: OpenPGP digital signature

Re: [gentoo-user] is multi-core really worth it?

[J. Roeleveld]

On Wednesday, November 22, 2017 3:34:56 PM CET Wols Lists wrote:
> On 22/11/17 14:11, Tsukasa Mcp_Reznor wrote:
> > You won't get build failures or dependency problems, portage is built to
> > handle emerging multiple packages that do not depend on each other
> > simultaneously.
> > it will not ever build a dependency and the main program at the same time.
> 
> Are you sure?
> 
> You *shouldn't* get problems, but I'm sure I have hit that problem, and
> I think it's down to buggy ebuilds.

buggy, as in, at least one dependency missed.

> Starting the emerge again fixes it, because cocked-up dependencies will
> sort themselves out first time round, and the second time the problem
> has gone away.

2nd time the dependency might have been installed already.
Sometime it hasn't.

This can also cause a failure when not using parallel builds for exactly the 
same reason.

--
Joost

Re: [gentoo-user] is multi-core really worth it?

[Wols Lists]

On 22/11/17 14:11, Tsukasa Mcp_Reznor wrote:
> 
> You won't get build failures or dependency problems, portage is built to
> handle emerging multiple packages that do not depend on each other
> simultaneously.
> it will not ever build a dependency and the main program at the same time.

Are you sure?

You *shouldn't* get problems, but I'm sure I have hit that problem, and
I think it's down to buggy ebuilds.

Starting the emerge again fixes it, because cocked-up dependencies will
sort themselves out first time round, and the second time the problem
has gone away.

Cheers,
Wol

Re: [gentoo-user] is multi-core really worth it?

[Tsukasa Mcp_Reznor]

From: Raffaele Belardi 
Sent: Wednesday, November 22, 2017 8:12 AM
To: gentoo-user@lists.gentoo.org
Subject: Re: [gentoo-user] is multi-core really worth it?

Jeremi Piotrowski wrote:
> That being said: if you do a world rebuild you will have lots of packages
> that spend ~40 seconds doing their autoconf run, only to build 2-3 sources
> files. On an 8-core machine at work, I get good results using parallel
> emerge jobs (emerge -jX). For your 6-core AMD CPU (assuming it actually
> has 12 threads) I'd start with 'emerge -j3' and MAKEOPTS='-j12 -l16'.
> That should get you a nice speedup, but may require a bit more ram.

emerge -j3 is something I did not think of, I'll try it. But won't that break 
portage's
carefully crafted package dependencies? I suppose you could get occasional 
build failures?

I'm using MAKEOPTS=-j7, I thought 2 threads per CPU (hyperthreading?) was an 
Intel thing only.

raffaele


There are alot of good tweaks to get by long configure sections, I have an octa 
core, so for normal updates I use -
MAKEOPTS="-j8" # for 8 cores being used per package
EMERGE_DEFAULT_OPTS="--jobs 2 --load-average 4" # only starts 2nd job if load 
average is low.

You won't get build failures or dependency problems, portage is built to handle 
emerging multiple packages that do not depend on each other simultaneously.
it will not ever build a dependency and the main program at the same time.

PORTAGE_NICENESS="19" # best setting ever, I do some gaming while updating, you 
mainly only notice install phase if your are loading while the disk queue is 
full of writes.

Re: [gentoo-user] is multi-core really worth it?

[J. Roeleveld]

On Wednesday, November 22, 2017 2:12:31 PM CET Raffaele Belardi wrote:
> Jeremi Piotrowski wrote:
> > That being said: if you do a world rebuild you will have lots of packages
> > that spend ~40 seconds doing their autoconf run, only to build 2-3 sources
> > files. On an 8-core machine at work, I get good results using parallel
> > emerge jobs (emerge -jX). For your 6-core AMD CPU (assuming it actually
> > has 12 threads) I'd start with 'emerge -j3' and MAKEOPTS='-j12 -l16'.
> > That should get you a nice speedup, but may require a bit more ram.
> 
> emerge -j3 is something I did not think of, I'll try it. But won't that
> break portage's carefully crafted package dependencies? I suppose you could
> get occasional build failures?

No, it won't.
If it has multiple packages that can be built (taking dependencies into 
account) it will. If all packages in the queue are waiting for a single 
package to be installed, all those will wait till that package is finished.

> I'm using MAKEOPTS=-j7, I thought 2 threads per CPU (hyperthreading?) was an
> Intel thing only.

I thought so too.

Don't forget to add the "load-average" option to both, otherwise there is a 
very likely chance you will have 3 * 7 = 21 build processes running at the 
same time.

Also, I have had situations where using "-j" actually caused some issues where 
it was picked up by the wrong process or it has a different meaning for some 
other process. Specifying the long version (--jobs) has been reliable for me.

--
Joost

Re: [gentoo-user] is multi-core really worth it?

[Raffaele Belardi]

Jeremi Piotrowski wrote:
> That being said: if you do a world rebuild you will have lots of packages
> that spend ~40 seconds doing their autoconf run, only to build 2-3 sources
> files. On an 8-core machine at work, I get good results using parallel
> emerge jobs (emerge -jX). For your 6-core AMD CPU (assuming it actually
> has 12 threads) I'd start with 'emerge -j3' and MAKEOPTS='-j12 -l16'.
> That should get you a nice speedup, but may require a bit more ram.

emerge -j3 is something I did not think of, I'll try it. But won't that break 
portage's
carefully crafted package dependencies? I suppose you could get occasional 
build failures?

I'm using MAKEOPTS=-j7, I thought 2 threads per CPU (hyperthreading?) was an 
Intel thing only.

raffaele

Re: [gentoo-user] is multi-core really worth it?

[Peter Humphrey]

On Wednesday, 22 November 2017 08:57:29 GMT David Haller wrote:
> Hello,
> 
> On Wed, 22 Nov 2017, J. Roeleveld wrote:
> >On Wednesday, November 22, 2017 8:48:08 AM CET David Haller wrote:
> >> So, your emerge is mostly IO and "emerge"-threads bound. Solution:
> >> adjust your build-threads[1], and then adjust your emerge jobs! See
> >> '--jobs' in 'man emerge'. Can't find where to set a default in a
> >> config-file for that ATM, but you could always alias/function/script
> >
> >> emerge to something else, e.g. put this in your ~/.bashrc:
> >Instead of all these aliases, simply edit your make.conf ( /etc/portage/
> >make.conf )
> >
> >And edit (or add) the following lines:
> >MAKEOPTS="--jobs 24 --load-average 48"
> >EMERGE_DEFAULT_OPTS="--jobs 24 --load-average 48"
> 
> Well, that alias/function/script is mostly for all the other (default)
> options I want ;) Hm. What's this EMERGE_DEFAULT_OPTS though? ;)) And
> how can I override them if not wanted? .oO( gotta read up on that )Oo.

Options passed on the command line (as in an alias) override any others. I 
assume a function would be equivalent to an alias here, but I haven't 
checked as I don't use them.

There used to be debates on the efficiency of alias versus function, but if 
they're only taking nanoseconds to execute, what does it matter?

-- 
Regards,
Peter.

Re: [gentoo-user] Re: is multi-core really worth it?

[David Haller]

Hello,

On Wed, 22 Nov 2017, Martin Vaeth wrote:
>David Haller  wrote:
>> autotools is _by far_the best both from a users and a packagers view.
>
>I do not agree. Its main advantage is that it is compatible with
>most existing unix systems (but I am already not so sure whether
>this also holds if you also want to compile for windows, powerpc,
>etc.)

Aye.

>> cmake [...] qmake
>
>I agree, these are horrible. The best build system currently
>appears to be meson.
>
>> equivalent to "./configure --help"
>
>For meson, it is "cat meson_options.txt", and there is a clear
>distinction between general options and project specific ones.

I've not yet "enconutered" meson... How is that meson_options.txt
maintained? Automatically or by hand? If the former: yay! If the
latter, treat it as non-existant...

>> transparent and easily hackable
>
>Hacking autotools is a nightmare: Things are often hidden in
>subprojects, sometimes combined with project specific hacks,
>generating/updating necessary configure files somewhere within the
>projects tree etc.

Well, now you're quite exaggerating! Projects using subprojects with
distinct autotools stack are broken by design and that is _NOT_ a
fault of autotools. And it's not "often". And then, usually it's
projects pulling in e.g. a "local" copy of e.g. "ffmpeg". _That_ is
broken and a nightmare. But then again, it shows the "hackability" of
autotools. With all "bad" side-effects. And on that I _DO_ agree quite
affirmatively! But then again, nobody keeps devs from using local
copies of qmake/cmake/meson/whatnot subprojects, in a otherwise
autotools/cmake/qmake/meson project and you'll have the same horrors...

>And after each change you have to run autoreconf, often with
>compatibility problems of autoconf/automake/gettext/... versions etc.

You'll get use to that. And you can get around by directly patching
configure and Makefile.in files. It _IS_ up to you! And that's one of
the points I like about autotools. You can even go around and first
run ./configure and _then_ patch the Makefiles and whatever like the
generated config.h. Or just set some ENV-vars or defines. That's the
flexibility of autotools. You can choose quite exactly where you want
to mess about with the delivered self-contained build. You can change
(or "mess with") _ANYTHING AT ANY STAGE_ of the process. As bad as
autotools are, that's something, I've not seen with any other
build-system yet. Be they self contained (IIRC: bjam anyone?) or
generators like imake/qmake, and cmake, what's that? It generates
makefiles but ... *gah*

>With meson, there is an absolutely strict separation between
>the distributed files and the generated/output files which are
>always in a fresh dir (and thus are _always_ produced).
>When hacking up, you need to modify only the *.meson files
>and do not have to worry about re-generating any other data.
>
>This sounds like I am a meson fanboy. I am not; actually, I dislike
>a lot of its design decisions. But compared to autotools, cmake,
>and qmake, it did a lot of things right.

Welcome to the club of build-system-haters! :)

I do not know meson yet, but, as far as I remember, it's rather
"closed" to interference from "packagers". I think I will rectify my
ignorance. But tell me:

- can you influence large parts of the build by just using ENV-vars or
  easy options? 'DESTDIR="/foo"' is the prime example but doesn't
  count, that's _too_ obvious. With autotools, you're free to fiddle
  with any var in the generated Makefiles...

- project specific './configure --help' (see above). Python's
  setuptools have that IIRC clumsily via 'python setup.py help' or
  some such, and that's usually rather disappointing regarding project
  specific options...

- etc. pp. I'm to tired...

As a packager, I just love autotools. As an author, autotools is a
piece of cake (you need _very_ few lines in your
configure.ac/Makefile.am, BTDT, esp. if you put your stuff into
subdirs and can use make's wildcard function for source/header etc. 
stuff). Well, but plainly implemented, Makefile.am are just some few
lines, with some clever "includes" (see the magic libreoffice does
with "plain" makefiles) you could probably prune it down to 2-3 lines
or so ...

But! Yes, I will look into meson and try building one of my "fully
autotooled" stuff as a meson project.. At least, I'll learn how to
munge meson stuff to my liking :)

I'm always curious about new build systems as a packager/ebuilder, and
I had a case recently, where I got a build failure due to some
build-system stuff, and I could not track it down. And I've about
15-20 years experience of compiling and packaging all kinds of weird
stuff. But this time, no go. Until I acutally used 'strace -e file'! 
WHAT THE *FSCK*! ISTR it might have been cmake, pulling in some file
from under /usr/. That's _NOT_ transparent. Debugging the build via
strace? Hey? Anyone home? And no, no debug option did help. And when
it comes to packageing, "I'm a Vet, 

Re: [gentoo-user] is multi-core really worth it?

[David Haller]

Hello,

On Wed, 22 Nov 2017, J. Roeleveld wrote:
>On Wednesday, November 22, 2017 8:48:08 AM CET David Haller wrote:
>> So, your emerge is mostly IO and "emerge"-threads bound. Solution:
>> adjust your build-threads[1], and then adjust your emerge jobs! See
>> '--jobs' in 'man emerge'. Can't find where to set a default in a
>> config-file for that ATM, but you could always alias/function/script
>> emerge to something else, e.g. put this in your ~/.bashrc:
>
>Instead of all these aliases, simply edit your make.conf ( /etc/portage/
>make.conf )
>
>And edit (or add) the following lines:
>MAKEOPTS="--jobs 24 --load-average 48"
>EMERGE_DEFAULT_OPTS="--jobs 24 --load-average 48"

Well, that alias/function/script is mostly for all the other (default)
options I want ;) Hm. What's this EMERGE_DEFAULT_OPTS though? ;)) And
how can I override them if not wanted? .oO( gotta read up on that )Oo.

-dnh

-- 
   85: Hot plugable
  Glüht beim Einstecken auf (Martin Neumann)

[gentoo-user] Re: is multi-core really worth it?

[Martin Vaeth]

David Haller  wrote:
> autotools is _by far_the best both from a users and a packagers view.

I do not agree. Its main advantage is that it is compatible with
most existing unix systems (but I am already not so sure whether
this also holds if you also want to compile for windows, powerpc,
etc.)

> cmake [...] qmake

I agree, these are horrible. The best build system currently
appears to be meson.

> equivalent to "./configure --help"

For meson, it is "cat meson_options.txt", and there is a clear
distinction between general options and project specific ones.

> transparent and easily hackable

Hacking autotools is a nightmare: Things are often hidden in
subprojects, sometimes combined with project specific hacks,
generating/updating necessary configure files somewhere within the
projects tree etc.
And after each change you have to run autoreconf, often with
compatibility problems of autoconf/automake/gettext/... versions etc.

With meson, there is an absolutely strict separation between
the distributed files and the generated/output files which are
always in a fresh dir (and thus are _always_ produced).
When hacking up, you need to modify only the *.meson files
and do not have to worry about re-generating any other data.

This sounds like I am a meson fanboy. I am not; actually, I dislike
a lot of its design decisions. But compared to autotools, cmake,
and qmake, it did a lot of things right.

Re: [gentoo-user] is multi-core really worth it?

[David Haller]

Hello,

On Wed, 22 Nov 2017, Jeremi Piotrowski wrote:
>That being said: if you do a world rebuild you will have lots of packages

that (even without the fetch) spend 40 seconds setting up the emerge
(unpack, prepare (plus eautoreconf))...

>that spend ~40 seconds doing their autoconf run, only to build 2-3 sources
>files.

Well, yes, autoreconf can be slow, configure itself less, but as one
writing both .spec files for rpms and .ebuilds, autotools is _by far_
the best both from a users and a packagers view.

Where's e.g. cmake's equvialent to "./configure --help"? Take defaults
or dig through whatnot spreadout cmake-files to find what -D arguments
to use... *bwah* Or qmake. Or whatnot. Yes. autotools sucks. A lot. 
But it is transparent and easily hackable[1], and the rest sucks a lot
worse.

-dnh

[1] do a "grep" on 'sed .. configure{,.ac,.in}' in ebuilds and patches
in ${FILESDIR}/ to configure{,.ac,.in} in the portage tree, and/or
sedding/patching *.m4 files ... That's definitely "hacking" the
autotools build process which should not be done (a autotools
tarball is "selfcontained"). But it's so easy, it is done quite
often, instead of patching up the result, moving files around etc. 
So easy and convenient, there's even "eautoreconf", even though
that takes (single-threaded) time to run. But rightly so. Upstream
is "notoriously" bad... The hoops you have to jump through for a
clean package with autotools are "walkthroughs" compared to the
stuff needed for cmake or qmake or ... I recently just gave up on
a (qmake) package, I just could not find what *.prf, *.pri
*whatnot file I needed to patch up to do as I needed... And I'm
not a newbie at patching that stuff.

And if you're adventurous, go and try patch up libreoffice or
mozillen ... ;) It's a pain. (BTW: I'm on the former, patching out
a lot of deps, like, when was the last time, if ever, you needed
to get a WordPerfect or Zoner Draw file to load?) Yes, all nicely
"useflagged", 406 lines of -U0 diff ATM with a 576 lines original
ebuild, but not ready yet ;) And yes, I'm even on the
"libstaroffice" stuff, and I _do_ have some of those file to load
and convert for myself...

-- 
>Someone from the society for prevention of cruelty to paper is sure to come
>after me if I suggest ramming it up a HR droid's ass.
Actually, it's that Lumber Cartel (TINLC) Directive (TAND) regarding
inappropriate use of their products.   -- in asr

Re: [gentoo-user] is multi-core really worth it?

[J. Roeleveld]

On Wednesday, November 22, 2017 8:48:08 AM CET David Haller wrote:
> Hello,
> 
> On Wed, 22 Nov 2017, Raffaele Belardi wrote:
> >rebuilding system and world with gcc-7.2.0 on a 6-core AMD CPU I have
> >the impression that most of the ebuilds limit parallel builds to 1, 2
> >or 3 threads.
> 
> Most should build with "many" cores, but some might be limited, but ...
> 
> >I'm aware it is only an impression, I did not spend the
> >night monitoring the process, but nevertheless every time I checked
> >the load was very low.
> 
> I guess you're mostly observing stalls because the (single-thread)
> overhead of emerge, configure (etc.), which can also be IO-bound...
> The actual builds probably goes so fast on your box, you hardly
> notice unless they're rather big(gish).
> 
> Some numbers for comparison, and I generally build with just one
> thread in the background on my 2 core "old" Athlon II X2 250:
> dev-libs/boost took a good 45mins to build with one thread. Divide by
> 6 for your cores, divide again by about 2-3 for your newer cores and
> you're in the 3:45min range for such a "biggie" as boost. Most other
> stuff won't even peg your meter.
> 
> So, your emerge is mostly IO and "emerge"-threads bound. Solution:
> adjust your build-threads[1], and then adjust your emerge jobs! See
> '--jobs' in 'man emerge'. Can't find where to set a default in a
> config-file for that ATM, but you could always alias/function/script
> emerge to something else, e.g. put this in your ~/.bashrc:

Instead of all these aliases, simply edit your make.conf ( /etc/portage/
make.conf )

And edit (or add) the following lines:
MAKEOPTS="--jobs 24 --load-average 48"
EMERGE_DEFAULT_OPTS="--jobs 24 --load-average 48"

Adjust the values to match your system, the above works fine on my desktop.
CPU: Intel(R) Core(TM) i7-5820K CPU @ 3.30GHz
And 32GB ram.

--
Joost

Re: [gentoo-user] is multi-core really worth it?

[Neil Bothwick]

On Wed, 22 Nov 2017 08:26:01 +0100, Jeremi Piotrowski wrote:

> That being said: if you do a world rebuild you will have lots of
> packages that spend ~40 seconds doing their autoconf run, only to build
> 2-3 sources files. On an 8-core machine at work, I get good results
> using parallel emerge jobs (emerge -jX). For your 6-core AMD CPU
> (assuming it actually has 12 threads) I'd start with 'emerge -j3' and
> MAKEOPTS='-j12 -l16'. That should get you a nice speedup, but may
> require a bit more ram.

Instead of specifying a number of jobs with "-j 3" you can let portage
work it out. On a similar system, I use 

EMERGE_DEFAULT_OPTS="--jobs --load-average 12"


-- 
Neil Bothwick

"An investment in knowledge always pays the best interest." - Benjamin
Franklin


pgpASw8EtmgsC.pgp
Description: OpenPGP digital signature