Re: go uses too many CPUs
On 19-02-08 21:16:02, Stuart Henderson wrote:
> On 2019/02/08 19:12, Marc Espie wrote:
> > On Fri, Feb 08, 2019 at 12:10:02AM -0500, Ted Unangst wrote:
> > > Go tries to use NCPU cpus. Unfortunately, half of them are turned off
> > > because
> > > hw.smt=0 by default, and then go spends a lot of time fighting against
> > > itself.
> > >
> > > The diff below, against go/src/runtime, changes to use the number of CPUs
> > > online. It's possible for this number to change, and thus become stale,
> > > but
> > > that's unlikely, and not the default.
> > >
> > > (This sysctl was added in 6.4.)
> > >
> > > --- os_openbsd.go.origFri Feb 8 00:02:27 2019
> > > +++ os_openbsd.go Fri Feb 8 00:06:21 2019
> > > @@ -85,8 +85,8 @@
> > > _KERN_OSREV = 3
> > >
> > > _CTL_HW = 6
> > > - _HW_NCPU = 3
> > > _HW_PAGESIZE = 7
> > > + _HW_NCPUONLINE = 25
> > > )
> > >
> > > func sysctlInt(mib []uint32) (int32, bool) {
> > > @@ -101,7 +101,7 @@
> > >
> > > func getncpu() int32 {
> > > // Fetch hw.ncpu via sysctl.
> > > - if ncpu, ok := sysctlInt([]uint32{_CTL_HW, _HW_NCPU}); ok {
> > > + if ncpu, ok := sysctlInt([]uint32{_CTL_HW, _HW_NCPUONLINE}); ok {
> > > return int32(ncpu)
> > > }
> > > return 1
>
> IIRC upstream had some requirement for supporting the previous OS version
> for their builders, so might be better to have fallback code in case
> _HW_NCPUONLINE isn't available.
This is correct - I've landed a diff that just switches to hw.ncpuonline,
however
a different version will go upstream.
> > Is there at least a knob to turn that off entirely ?
> >
> > Like, specifically, when you are bulk-building ports, you do not want go
> > to hog on every cpu while it's not alone.
> >
> > Parallelizing everything is not always (very often not actually) a win.
>
> On 2019/02/08 19:55, Otto Moerbeek wrote:
> > The GOMACPROCS env var is what you're looking for. If you set it to 1
> > go will use 1 core at max.
>
> There are two different things.
>
> go.port.mk's build command line (used for some but not all ports) calls
> "go install -p ${MAKE_JOBS}", that controls how many parallel processes
> are used. (some like sysutils/prometheus use a different build system).
>
> GOMAXPROCS (X not C) is different - number of threads within one process.
It is the number of threads that can simultaneously execute user-level Go code.
There may be more or less threads than this in a Go process, depending on
whether there is a need for concurrency and/or if threads are blocked in
system calls.
> So if both are in play maybe you get N(smp_cores) processes * N(smp_cores)
> threads ...
Re: go uses too many CPUs
On 2019/02/08 19:12, Marc Espie wrote:
> On Fri, Feb 08, 2019 at 12:10:02AM -0500, Ted Unangst wrote:
> > Go tries to use NCPU cpus. Unfortunately, half of them are turned off
> > because
> > hw.smt=0 by default, and then go spends a lot of time fighting against
> > itself.
> >
> > The diff below, against go/src/runtime, changes to use the number of CPUs
> > online. It's possible for this number to change, and thus become stale, but
> > that's unlikely, and not the default.
> >
> > (This sysctl was added in 6.4.)
> >
> > --- os_openbsd.go.orig Fri Feb 8 00:02:27 2019
> > +++ os_openbsd.go Fri Feb 8 00:06:21 2019
> > @@ -85,8 +85,8 @@
> > _KERN_OSREV = 3
> >
> > _CTL_HW = 6
> > - _HW_NCPU = 3
> > _HW_PAGESIZE = 7
> > + _HW_NCPUONLINE = 25
> > )
> >
> > func sysctlInt(mib []uint32) (int32, bool) {
> > @@ -101,7 +101,7 @@
> >
> > func getncpu() int32 {
> > // Fetch hw.ncpu via sysctl.
> > - if ncpu, ok := sysctlInt([]uint32{_CTL_HW, _HW_NCPU}); ok {
> > + if ncpu, ok := sysctlInt([]uint32{_CTL_HW, _HW_NCPUONLINE}); ok {
> > return int32(ncpu)
> > }
> > return 1
IIRC upstream had some requirement for supporting the previous OS version
for their builders, so might be better to have fallback code in case
_HW_NCPUONLINE isn't available.
> Is there at least a knob to turn that off entirely ?
>
> Like, specifically, when you are bulk-building ports, you do not want go
> to hog on every cpu while it's not alone.
>
> Parallelizing everything is not always (very often not actually) a win.
On 2019/02/08 19:55, Otto Moerbeek wrote:
> The GOMACPROCS env var is what you're looking for. If you set it to 1
> go will use 1 core at max.
There are two different things.
go.port.mk's build command line (used for some but not all ports) calls
"go install -p ${MAKE_JOBS}", that controls how many parallel processes
are used. (some like sysutils/prometheus use a different build system).
GOMAXPROCS (X not C) is different - number of threads within one process.
So if both are in play maybe you get N(smp_cores) processes * N(smp_cores)
threads ...
Re: go uses too many CPUs
On Fri, Feb 08, 2019 at 07:12:25PM +0100, Marc Espie wrote:
> On Fri, Feb 08, 2019 at 12:10:02AM -0500, Ted Unangst wrote:
> > Go tries to use NCPU cpus. Unfortunately, half of them are turned off
> > because
> > hw.smt=0 by default, and then go spends a lot of time fighting against
> > itself.
> >
> > The diff below, against go/src/runtime, changes to use the number of CPUs
> > online. It's possible for this number to change, and thus become stale, but
> > that's unlikely, and not the default.
> >
> > (This sysctl was added in 6.4.)
> >
> > --- os_openbsd.go.orig Fri Feb 8 00:02:27 2019
> > +++ os_openbsd.go Fri Feb 8 00:06:21 2019
> > @@ -85,8 +85,8 @@
> > _KERN_OSREV = 3
> >
> > _CTL_HW = 6
> > - _HW_NCPU = 3
> > _HW_PAGESIZE = 7
> > + _HW_NCPUONLINE = 25
> > )
> >
> > func sysctlInt(mib []uint32) (int32, bool) {
> > @@ -101,7 +101,7 @@
> >
> > func getncpu() int32 {
> > // Fetch hw.ncpu via sysctl.
> > - if ncpu, ok := sysctlInt([]uint32{_CTL_HW, _HW_NCPU}); ok {
> > + if ncpu, ok := sysctlInt([]uint32{_CTL_HW, _HW_NCPUONLINE}); ok {
> > return int32(ncpu)
> > }
> > return 1
>
> Is there at least a knob to turn that off entirely ?
>
> Like, specifically, when you are bulk-building ports, you do not want go
> to hog on every cpu while it's not alone.
>
> Parallelizing everything is not always (very often not actually) a win.
>
The GOMACPROCS env var is what you're looking for. If you set it to 1
go will use 1 core at max.
-Otto
Re: go uses too many CPUs
On Fri, Feb 08, 2019 at 12:10:02AM -0500, Ted Unangst wrote:
> Go tries to use NCPU cpus. Unfortunately, half of them are turned off because
> hw.smt=0 by default, and then go spends a lot of time fighting against itself.
>
> The diff below, against go/src/runtime, changes to use the number of CPUs
> online. It's possible for this number to change, and thus become stale, but
> that's unlikely, and not the default.
>
> (This sysctl was added in 6.4.)
>
> --- os_openbsd.go.origFri Feb 8 00:02:27 2019
> +++ os_openbsd.go Fri Feb 8 00:06:21 2019
> @@ -85,8 +85,8 @@
> _KERN_OSREV = 3
>
> _CTL_HW = 6
> - _HW_NCPU = 3
> _HW_PAGESIZE = 7
> + _HW_NCPUONLINE = 25
> )
>
> func sysctlInt(mib []uint32) (int32, bool) {
> @@ -101,7 +101,7 @@
>
> func getncpu() int32 {
> // Fetch hw.ncpu via sysctl.
> - if ncpu, ok := sysctlInt([]uint32{_CTL_HW, _HW_NCPU}); ok {
> + if ncpu, ok := sysctlInt([]uint32{_CTL_HW, _HW_NCPUONLINE}); ok {
> return int32(ncpu)
> }
> return 1
Is there at least a knob to turn that off entirely ?
Like, specifically, when you are bulk-building ports, you do not want go
to hog on every cpu while it's not alone.
Parallelizing everything is not always (very often not actually) a win.
Re: go uses too many CPUs
On Fri, Feb 08, 2019 at 12:10:02AM -0500, Ted Unangst wrote:
> Go tries to use NCPU cpus. Unfortunately, half of them are turned off because
> hw.smt=0 by default, and then go spends a lot of time fighting against itself.
>
> The diff below, against go/src/runtime, changes to use the number of CPUs
> online. It's possible for this number to change, and thus become stale, but
> that's unlikely, and not the default.
>
> (This sysctl was added in 6.4.)
Yes, please,
-Otto
>
> --- os_openbsd.go.origFri Feb 8 00:02:27 2019
> +++ os_openbsd.go Fri Feb 8 00:06:21 2019
> @@ -85,8 +85,8 @@
> _KERN_OSREV = 3
>
> _CTL_HW = 6
> - _HW_NCPU = 3
> _HW_PAGESIZE = 7
> + _HW_NCPUONLINE = 25
> )
>
> func sysctlInt(mib []uint32) (int32, bool) {
> @@ -101,7 +101,7 @@
>
> func getncpu() int32 {
> // Fetch hw.ncpu via sysctl.
> - if ncpu, ok := sysctlInt([]uint32{_CTL_HW, _HW_NCPU}); ok {
> + if ncpu, ok := sysctlInt([]uint32{_CTL_HW, _HW_NCPUONLINE}); ok {
> return int32(ncpu)
> }
> return 1
>
Re: go uses too many CPUs
How horrible.
We had about 20 things depending on the old mix of "all cpus all the time",
and this one fell through the cracks.
> Go tries to use NCPU cpus. Unfortunately, half of them are turned off because
> hw.smt=0 by default, and then go spends a lot of time fighting against itself.
>
> The diff below, against go/src/runtime, changes to use the number of CPUs
> online. It's possible for this number to change, and thus become stale, but
> that's unlikely, and not the default.
>
> (This sysctl was added in 6.4.)
>
> --- os_openbsd.go.origFri Feb 8 00:02:27 2019
> +++ os_openbsd.go Fri Feb 8 00:06:21 2019
> @@ -85,8 +85,8 @@
> _KERN_OSREV = 3
>
> _CTL_HW = 6
> - _HW_NCPU = 3
> _HW_PAGESIZE = 7
> + _HW_NCPUONLINE = 25
> )
>
> func sysctlInt(mib []uint32) (int32, bool) {
> @@ -101,7 +101,7 @@
>
> func getncpu() int32 {
> // Fetch hw.ncpu via sysctl.
> - if ncpu, ok := sysctlInt([]uint32{_CTL_HW, _HW_NCPU}); ok {
> + if ncpu, ok := sysctlInt([]uint32{_CTL_HW, _HW_NCPUONLINE}); ok {
> return int32(ncpu)
> }
> return 1
>
go uses too many CPUs
Go tries to use NCPU cpus. Unfortunately, half of them are turned off because
hw.smt=0 by default, and then go spends a lot of time fighting against itself.
The diff below, against go/src/runtime, changes to use the number of CPUs
online. It's possible for this number to change, and thus become stale, but
that's unlikely, and not the default.
(This sysctl was added in 6.4.)
--- os_openbsd.go.orig Fri Feb 8 00:02:27 2019
+++ os_openbsd.go Fri Feb 8 00:06:21 2019
@@ -85,8 +85,8 @@
_KERN_OSREV = 3
_CTL_HW = 6
- _HW_NCPU = 3
_HW_PAGESIZE = 7
+ _HW_NCPUONLINE = 25
)
func sysctlInt(mib []uint32) (int32, bool) {
@@ -101,7 +101,7 @@
func getncpu() int32 {
// Fetch hw.ncpu via sysctl.
- if ncpu, ok := sysctlInt([]uint32{_CTL_HW, _HW_NCPU}); ok {
+ if ncpu, ok := sysctlInt([]uint32{_CTL_HW, _HW_NCPUONLINE}); ok {
return int32(ncpu)
}
return 1
