On 19-02-08 21:16:02, Stuart Henderson wrote:
> On 2019/02/08 19:12, Marc Espie wrote:
> > On Fri, Feb 08, 2019 at 12:10:02AM -0500, Ted Unangst wrote:
> > > Go tries to use NCPU cpus. Unfortunately, half of them are turned off
> > > because
> > > hw.smt=0 by default, and then go spends a lot of time fighting against
> > > itself.
> > >
> > > The diff below, against go/src/runtime, changes to use the number of CPUs
> > > online. It's possible for this number to change, and thus become stale,
> > > but
> > > that's unlikely, and not the default.
> > >
> > > (This sysctl was added in 6.4.)
> > >
> > > --- os_openbsd.go.orig Fri Feb 8 00:02:27 2019
> > > +++ os_openbsd.go Fri Feb 8 00:06:21 2019
> > > @@ -85,8 +85,8 @@
> > > _KERN_OSREV = 3
> > >
> > > _CTL_HW = 6
> > > - _HW_NCPU = 3
> > > _HW_PAGESIZE = 7
> > > + _HW_NCPUONLINE = 25
> > > )
> > >
> > > func sysctlInt(mib []uint32) (int32, bool) {
> > > @@ -101,7 +101,7 @@
> > >
> > > func getncpu() int32 {
> > > // Fetch hw.ncpu via sysctl.
> > > - if ncpu, ok := sysctlInt([]uint32{_CTL_HW, _HW_NCPU}); ok {
> > > + if ncpu, ok := sysctlInt([]uint32{_CTL_HW, _HW_NCPUONLINE}); ok {
> > > return int32(ncpu)
> > > }
> > > return 1
>
> IIRC upstream had some requirement for supporting the previous OS version
> for their builders, so might be better to have fallback code in case
> _HW_NCPUONLINE isn't available.
This is correct - I've landed a diff that just switches to hw.ncpuonline,
however
a different version will go upstream.
> > Is there at least a knob to turn that off entirely ?
> >
> > Like, specifically, when you are bulk-building ports, you do not want go
> > to hog on every cpu while it's not alone.
> >
> > Parallelizing everything is not always (very often not actually) a win.
>
> On 2019/02/08 19:55, Otto Moerbeek wrote:
> > The GOMACPROCS env var is what you're looking for. If you set it to 1
> > go will use 1 core at max.
>
> There are two different things.
>
> go.port.mk's build command line (used for some but not all ports) calls
> "go install -p ${MAKE_JOBS}", that controls how many parallel processes
> are used. (some like sysutils/prometheus use a different build system).
>
> GOMAXPROCS (X not C) is different - number of threads within one process.
It is the number of threads that can simultaneously execute user-level Go code.
There may be more or less threads than this in a Go process, depending on
whether there is a need for concurrency and/or if threads are blocked in
system calls.
> So if both are in play maybe you get N(smp_cores) processes * N(smp_cores)
> threads ...
