On Sun, Oct 08, 2017 at 11:31:43AM -0500, Bruce Dubbs wrote:
> Ken Moffat via blfs-book wrote:
> > On Sun, Oct 08, 2017 at 02:28:28AM -0000, via blfs-book wrote:
> > > Author: dj
> > > Date: Sat Oct 7 19:28:28 2017
> > > New Revision: 19304
> > >
> > > Log:
> > > Fix SBU value (ninja -j1) for systemd.
> > >
> >
> > The last time I looked at ninja, it used the number of cores plus 2
> > (-j3 on single processor, -j6 on 4 cores). There is merit in
> > documenting a -j1 time for a package which builds quickly, but
> > (unless things have changed a lot) the book's instructions will use
> > something more than one processor (e.g. watch 'top' and count the
> > percentages - I think they will show a total percentage approaching
> > 100 * number of cores.
> >
> > If I'm right, this might be one package where using the value for a
> > machine with 4 CPUs (as a commonly-available base) even for a quick
> > package might be worthwhile - we normally only force -j1 where we
> > have to becuase of broken Makefiles.
> >
> > My interest in this is that I'm thinking about extending the
> > Editors' Guide to include details of reducing the number of
> > processors available (taksset) when building on machines with many
> > cores.
>
> I agree that 4 cores are commonly available. Using -j1 is reasonable for
> consistency. More than -j4 is not (normally). As long as we document the
> number of cores used using higher levels of parallelism I think it is OK,
> but the default should still be 1.
>
I was thinking of the disparity between the command (which will use all
cores), and the quoted SBU for only 1 job.
> My rule of thumb is that if the package takes less than 0.5 SBU at -j1, then
> I don't use higher levels.
>
> I note that we use -j8 in two places: libreoffice (still 41 SBU) and gimp
> help (only if building all languages).
>
> -- Bruce
>
I hadn't realised, or had forgotten, that we use -j8. Maybe I'm
even to 'blame'.
> P.S. I really don't like ninja using all processors. In the case of my
> laptop (i7 processor), a long-ish build (e.g. gcc) results in very high
> temperatures. That package needs to introduce an environment variable to
> limit the number of cores instead of insisting on a command line entry.
>
I'll try to get to the Editor's Guide this week. In the meantime -
Identify which term is where you will be building (e.g. in Xorg run
top in that term, grep the output of ps aux for ' top' (if
qtwebengine has been invoked, it may have top as part of some
switches shown by ps aux), identify the pts/NN number. Then quit
top, so that the term is running bash.
Now grep for 'pts/NN ' and look for the bash process's pid.
Limit the number of cores with tasksel, e.g. to limit it to cores
4-7 'tasksel -pc 4-7 pid-of-bash'.
In the term where bash is running, use nproc to prove it can only
see 4 cores. Build. If you use top you should see the jobs moving
around those 4 cores, the other cores will only get used by other
tasks (top, xorg, other terms, vim, whatever) which in this example
can also use the 4 cores used by the build.
Note: linux moves jobs around to improve thermal management - if you
specify specific cores, you should use different cores each time.
Also, for CPUs with multithreading it probably makes sense to use
the pairs of cores (0,1 2,3 4,5 etc) rather than e.g. 0,3 which will
use two pairs of real cores for only two jobs.
For ninja, -jN is easier (-j6 on an 8-core machine probably uses 6
cores, so -j4 on the commandline will be easier).
Like a few other people, my experience with 'make -jN' is that using
more jobs than you have cores will at best make a minimal
improvement in compilation time, and typically just stresses the
system. Often -jN+1 on consumer hardware is slower than -jN.
My interest in this is because of rust, which reads the number of
cores and decides to use them all. Before discovering tasksel I
took CPUs offline.
ĸen
--
Truth, in front of her huge walk-in wardrobe, selected black leather
boots with stiletto heels for such a barefaced truth.
- Unseen Academicals
--
http://lists.linuxfromscratch.org/listinfo/blfs-book
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page
