Re: Default options for -threaded

[Carter Schonwald]

Yup! Let’s do it. Efficient io and compute during ffi computation sound
good to me

On Fri, Jan 4, 2019 at 10:22 AM Matthew Pickering <
matthewtpicker...@gmail.com> wrote:

> Two years seems a good amount of time for any objectors.
>
> https://ghc.haskell.org/trac/ghc/ticket/16126#ticket
>
> On Fri, Oct 21, 2016 at 5:35 PM Simon Marlow  wrote:
> >
> > On 8 October 2016 at 17:55, Ben Gamari  wrote:
> >>
> >> loneti...@gmail.com writes:
> >>
> >> > Hi All,
> >> >
> >> > A user on https://ghc.haskell.org/trac/ghc/ticket/11054 has asked why
> >> > -N -qa isn’t the default for -threaded.
> >> >
> >> I'm not sure that scheduling on all of the cores on the user's machine
> by
> >> default is a good idea, especially given that our users have
> >> learned to expect the existing default. Enabling affinity by default
> >> seems reasonable if we have evidence that it helps the majority of
> >> applications, but we would first need to introduce an additional
> >> flag to disable it.
> >
> >
> > Affinity is almost always a bad idea in my experience.
> >
> >>
> >> In general I think -N1 is a reasonable default as it acknowledges the
> >> fact that deploying parallelism is not something that can be done
> >> blindly in many (most?) applications. To make effective use of
> >> parallelism the user needs to understand their hardware, their
> >> application, and its interaction with the runtime system and configure
> >> the RTS appropriately.
> >>
> >
> > Agree on keeping -N1.
> >
> > Related to this, I think it's about time we made -threaded the default.
> We could add a -single-threaded option to get back the old behaviour.
> >
> > There is a small overhead to using -threaded, but -threaded is also
> required to make a lot of things work (e.g. waitForProcess in a
> multithreaded program, not to mention parallelism).
> >
> > Anyone interested in doing this?
> >
> > Cheers
> > Simon
> >
> >
> >>
> >> Of course, this is just my two-cents.
> >>
> >> Cheers,
> >>
> >> - Ben
> >>
> >> ___
> >> ghc-devs mailing list
> >> ghc-devs@haskell.org
> >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> >>
> >
> > ___
> > ghc-devs mailing list
> > ghc-devs@haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Default options for -threaded

[Matthew Pickering]

Two years seems a good amount of time for any objectors.

https://ghc.haskell.org/trac/ghc/ticket/16126#ticket

On Fri, Oct 21, 2016 at 5:35 PM Simon Marlow  wrote:
>
> On 8 October 2016 at 17:55, Ben Gamari  wrote:
>>
>> loneti...@gmail.com writes:
>>
>> > Hi All,
>> >
>> > A user on https://ghc.haskell.org/trac/ghc/ticket/11054 has asked why
>> > -N -qa isn’t the default for -threaded.
>> >
>> I'm not sure that scheduling on all of the cores on the user's machine by
>> default is a good idea, especially given that our users have
>> learned to expect the existing default. Enabling affinity by default
>> seems reasonable if we have evidence that it helps the majority of
>> applications, but we would first need to introduce an additional
>> flag to disable it.
>
>
> Affinity is almost always a bad idea in my experience.
>
>>
>> In general I think -N1 is a reasonable default as it acknowledges the
>> fact that deploying parallelism is not something that can be done
>> blindly in many (most?) applications. To make effective use of
>> parallelism the user needs to understand their hardware, their
>> application, and its interaction with the runtime system and configure
>> the RTS appropriately.
>>
>
> Agree on keeping -N1.
>
> Related to this, I think it's about time we made -threaded the default.  We 
> could add a -single-threaded option to get back the old behaviour.
>
> There is a small overhead to using -threaded, but -threaded is also required 
> to make a lot of things work (e.g. waitForProcess in a multithreaded program, 
> not to mention parallelism).
>
> Anyone interested in doing this?
>
> Cheers
> Simon
>
>
>>
>> Of course, this is just my two-cents.
>>
>> Cheers,
>>
>> - Ben
>>
>> ___
>> ghc-devs mailing list
>> ghc-devs@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Default options for -threaded

[Simon Marlow]

On 8 October 2016 at 17:55, Ben Gamari  wrote:

> loneti...@gmail.com writes:
>
> > Hi All,
> >
> > A user on https://ghc.haskell.org/trac/ghc/ticket/11054 has asked why
> > -N -qa isn’t the default for -threaded.
> >
> I'm not sure that scheduling on all of the cores on the user's machine by
> default is a good idea, especially given that our users have
> learned to expect the existing default. Enabling affinity by default
> seems reasonable if we have evidence that it helps the majority of
> applications, but we would first need to introduce an additional
> flag to disable it.
>

Affinity is almost always a bad idea in my experience.


> In general I think -N1 is a reasonable default as it acknowledges the
> fact that deploying parallelism is not something that can be done
> blindly in many (most?) applications. To make effective use of
> parallelism the user needs to understand their hardware, their
> application, and its interaction with the runtime system and configure
> the RTS appropriately.
>
>
Agree on keeping -N1.

Related to this, I think it's about time we made -threaded the default.  We
could add a -single-threaded option to get back the old behaviour.

There is a small overhead to using -threaded, but -threaded is also
required to make a lot of things work (e.g. waitForProcess in a
multithreaded program, not to mention parallelism).

Anyone interested in doing this?

Cheers
Simon



> Of course, this is just my two-cents.
>
> Cheers,
>
> - Ben
>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Default options for -threaded

[Eric Seidel]

Ah, I'm sorry, I believe I was thinking of -qm, which is supposed to
prevent threads from being moved. I forgot these were separate options!
And the latest version of the User's Guide includes a comment about -qm

> This option is probably only of use for concurrent programs that explicitly 
> schedule threads onto CPUs with Control.Concurrent.forkOn.

which is exactly what I had to do.

On Mon, Oct 10, 2016, at 03:34, Phyx wrote:
> Oh, this is surprising, I must admit I haven't tried forkIO, but with
> forkOS is doesn't move the threads across capabilities.
> 
> Do you know if this is by design or a bug?
> 
> On Sat, Oct 8, 2016 at 6:13 PM, Eric Seidel  wrote:
> 
> > I would prefer keeping -N1 as a default, especially now that the number
> > of capabilities can be set at runtime. Programs can then use the more
> > common -j flag to enable parallelism.
> >
> > Regarding -qa, I was experimenting with it over the summer and found its
> > behavior a bit surprising. It did prevent threads from being moved
> > between capabilities, but it also forced all of the threads (created
> > with forkIO) to be *spawned* on the same capability, which was
> > unexpected. So -N -qa was, in my experience, equivalent to -N1!
> >
> > On Sat, Oct 8, 2016, at 09:55, Ben Gamari wrote:
> > > loneti...@gmail.com writes:
> > >
> > > > Hi All,
> > > >
> > > > A user on https://ghc.haskell.org/trac/ghc/ticket/11054 has asked why
> > > > -N -qa isn’t the default for -threaded.
> > > >
> > > I'm not sure that scheduling on all of the cores on the user's machine by
> > > default is a good idea, especially given that our users have
> > > learned to expect the existing default. Enabling affinity by default
> > > seems reasonable if we have evidence that it helps the majority of
> > > applications, but we would first need to introduce an additional
> > > flag to disable it.
> > >
> > > In general I think -N1 is a reasonable default as it acknowledges the
> > > fact that deploying parallelism is not something that can be done
> > > blindly in many (most?) applications. To make effective use of
> > > parallelism the user needs to understand their hardware, their
> > > application, and its interaction with the runtime system and configure
> > > the RTS appropriately.
> > >
> > > Of course, this is just my two-cents.
> > >
> > > Cheers,
> > >
> > > - Ben
> > > ___
> > > ghc-devs mailing list
> > > ghc-devs@haskell.org
> > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> > > Email had 1 attachment:
> > > + signature.asc
> > >   1k (application/pgp-signature)
> > ___
> > ghc-devs mailing list
> > ghc-devs@haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> >
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Default options for -threaded

[Phyx]

Oh, this is surprising, I must admit I haven't tried forkIO, but with
forkOS is doesn't move the threads across capabilities.

Do you know if this is by design or a bug?

On Sat, Oct 8, 2016 at 6:13 PM, Eric Seidel  wrote:

> I would prefer keeping -N1 as a default, especially now that the number
> of capabilities can be set at runtime. Programs can then use the more
> common -j flag to enable parallelism.
>
> Regarding -qa, I was experimenting with it over the summer and found its
> behavior a bit surprising. It did prevent threads from being moved
> between capabilities, but it also forced all of the threads (created
> with forkIO) to be *spawned* on the same capability, which was
> unexpected. So -N -qa was, in my experience, equivalent to -N1!
>
> On Sat, Oct 8, 2016, at 09:55, Ben Gamari wrote:
> > loneti...@gmail.com writes:
> >
> > > Hi All,
> > >
> > > A user on https://ghc.haskell.org/trac/ghc/ticket/11054 has asked why
> > > -N -qa isn’t the default for -threaded.
> > >
> > I'm not sure that scheduling on all of the cores on the user's machine by
> > default is a good idea, especially given that our users have
> > learned to expect the existing default. Enabling affinity by default
> > seems reasonable if we have evidence that it helps the majority of
> > applications, but we would first need to introduce an additional
> > flag to disable it.
> >
> > In general I think -N1 is a reasonable default as it acknowledges the
> > fact that deploying parallelism is not something that can be done
> > blindly in many (most?) applications. To make effective use of
> > parallelism the user needs to understand their hardware, their
> > application, and its interaction with the runtime system and configure
> > the RTS appropriately.
> >
> > Of course, this is just my two-cents.
> >
> > Cheers,
> >
> > - Ben
> > ___
> > ghc-devs mailing list
> > ghc-devs@haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> > Email had 1 attachment:
> > + signature.asc
> >   1k (application/pgp-signature)
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Default options for -threaded

[Phyx]

Oops, sorry, only just now seen this. It seems my overly aggressive filters
couldn't decide where to put the email :)

I do agree to some extend with this. I'd prefer if I made a mistake for my
system not to hang. The one downside to this default though is that you
can't just hand a program over to user and have it run at full capabilities.

If it possible to set this from inside a program? My guess is no, since by
the time you get to main the rts is already initialized?

Would a useful alternative be to provide a compile flag that would change
the default? e.g. opt-in? Since now there is a small burden on the end user.

Cheers,
Tamar

On Sat, Oct 8, 2016 at 5:55 PM, Ben Gamari  wrote:

> loneti...@gmail.com writes:
>
> > Hi All,
> >
> > A user on https://ghc.haskell.org/trac/ghc/ticket/11054 has asked why
> > -N -qa isn’t the default for -threaded.
> >
> I'm not sure that scheduling on all of the cores on the user's machine by
> default is a good idea, especially given that our users have
> learned to expect the existing default. Enabling affinity by default
> seems reasonable if we have evidence that it helps the majority of
> applications, but we would first need to introduce an additional
> flag to disable it.
>
> In general I think -N1 is a reasonable default as it acknowledges the
> fact that deploying parallelism is not something that can be done
> blindly in many (most?) applications. To make effective use of
> parallelism the user needs to understand their hardware, their
> application, and its interaction with the runtime system and configure
> the RTS appropriately.
>
> Of course, this is just my two-cents.
>
> Cheers,
>
> - Ben
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Default options for -threaded

[Eric Seidel]

I would prefer keeping -N1 as a default, especially now that the number
of capabilities can be set at runtime. Programs can then use the more
common -j flag to enable parallelism.

Regarding -qa, I was experimenting with it over the summer and found its
behavior a bit surprising. It did prevent threads from being moved
between capabilities, but it also forced all of the threads (created
with forkIO) to be *spawned* on the same capability, which was
unexpected. So -N -qa was, in my experience, equivalent to -N1!

On Sat, Oct 8, 2016, at 09:55, Ben Gamari wrote:
> loneti...@gmail.com writes:
> 
> > Hi All,
> >
> > A user on https://ghc.haskell.org/trac/ghc/ticket/11054 has asked why 
> > -N -qa isn’t the default for -threaded.
> >
> I'm not sure that scheduling on all of the cores on the user's machine by
> default is a good idea, especially given that our users have 
> learned to expect the existing default. Enabling affinity by default
> seems reasonable if we have evidence that it helps the majority of
> applications, but we would first need to introduce an additional
> flag to disable it.
> 
> In general I think -N1 is a reasonable default as it acknowledges the
> fact that deploying parallelism is not something that can be done
> blindly in many (most?) applications. To make effective use of
> parallelism the user needs to understand their hardware, their
> application, and its interaction with the runtime system and configure
> the RTS appropriately.
> 
> Of course, this is just my two-cents.
> 
> Cheers,
> 
> - Ben
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> Email had 1 attachment:
> + signature.asc
>   1k (application/pgp-signature)
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs