Re: Default options for -threaded
Yup! Let’s do it. Efficient io and compute during ffi computation sound good to me On Fri, Jan 4, 2019 at 10:22 AM Matthew Pickering < matthewtpicker...@gmail.com> wrote: > Two years seems a good amount of time for any objectors. > > https://ghc.haskell.org/trac/ghc/ticket/16126#ticket > > On Fri, Oct 21, 2016 at 5:35 PM Simon Marlow wrote: > > > > On 8 October 2016 at 17:55, Ben Gamari wrote: > >> > >> loneti...@gmail.com writes: > >> > >> > Hi All, > >> > > >> > A user on https://ghc.haskell.org/trac/ghc/ticket/11054 has asked why > >> > -N -qa isn’t the default for -threaded. > >> > > >> I'm not sure that scheduling on all of the cores on the user's machine > by > >> default is a good idea, especially given that our users have > >> learned to expect the existing default. Enabling affinity by default > >> seems reasonable if we have evidence that it helps the majority of > >> applications, but we would first need to introduce an additional > >> flag to disable it. > > > > > > Affinity is almost always a bad idea in my experience. > > > >> > >> In general I think -N1 is a reasonable default as it acknowledges the > >> fact that deploying parallelism is not something that can be done > >> blindly in many (most?) applications. To make effective use of > >> parallelism the user needs to understand their hardware, their > >> application, and its interaction with the runtime system and configure > >> the RTS appropriately. > >> > > > > Agree on keeping -N1. > > > > Related to this, I think it's about time we made -threaded the default. > We could add a -single-threaded option to get back the old behaviour. > > > > There is a small overhead to using -threaded, but -threaded is also > required to make a lot of things work (e.g. waitForProcess in a > multithreaded program, not to mention parallelism). > > > > Anyone interested in doing this? > > > > Cheers > > Simon > > > > > >> > >> Of course, this is just my two-cents. > >> > >> Cheers, > >> > >> - Ben > >> > >> ___ > >> ghc-devs mailing list > >> ghc-devs@haskell.org > >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > >> > > > > ___ > > ghc-devs mailing list > > ghc-devs@haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Default options for -threaded
Two years seems a good amount of time for any objectors. https://ghc.haskell.org/trac/ghc/ticket/16126#ticket On Fri, Oct 21, 2016 at 5:35 PM Simon Marlow wrote: > > On 8 October 2016 at 17:55, Ben Gamari wrote: >> >> loneti...@gmail.com writes: >> >> > Hi All, >> > >> > A user on https://ghc.haskell.org/trac/ghc/ticket/11054 has asked why >> > -N -qa isn’t the default for -threaded. >> > >> I'm not sure that scheduling on all of the cores on the user's machine by >> default is a good idea, especially given that our users have >> learned to expect the existing default. Enabling affinity by default >> seems reasonable if we have evidence that it helps the majority of >> applications, but we would first need to introduce an additional >> flag to disable it. > > > Affinity is almost always a bad idea in my experience. > >> >> In general I think -N1 is a reasonable default as it acknowledges the >> fact that deploying parallelism is not something that can be done >> blindly in many (most?) applications. To make effective use of >> parallelism the user needs to understand their hardware, their >> application, and its interaction with the runtime system and configure >> the RTS appropriately. >> > > Agree on keeping -N1. > > Related to this, I think it's about time we made -threaded the default. We > could add a -single-threaded option to get back the old behaviour. > > There is a small overhead to using -threaded, but -threaded is also required > to make a lot of things work (e.g. waitForProcess in a multithreaded program, > not to mention parallelism). > > Anyone interested in doing this? > > Cheers > Simon > > >> >> Of course, this is just my two-cents. >> >> Cheers, >> >> - Ben >> >> ___ >> ghc-devs mailing list >> ghc-devs@haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Default options for -threaded
On 8 October 2016 at 17:55, Ben Gamari wrote: > loneti...@gmail.com writes: > > > Hi All, > > > > A user on https://ghc.haskell.org/trac/ghc/ticket/11054 has asked why > > -N -qa isn’t the default for -threaded. > > > I'm not sure that scheduling on all of the cores on the user's machine by > default is a good idea, especially given that our users have > learned to expect the existing default. Enabling affinity by default > seems reasonable if we have evidence that it helps the majority of > applications, but we would first need to introduce an additional > flag to disable it. > Affinity is almost always a bad idea in my experience. > In general I think -N1 is a reasonable default as it acknowledges the > fact that deploying parallelism is not something that can be done > blindly in many (most?) applications. To make effective use of > parallelism the user needs to understand their hardware, their > application, and its interaction with the runtime system and configure > the RTS appropriately. > > Agree on keeping -N1. Related to this, I think it's about time we made -threaded the default. We could add a -single-threaded option to get back the old behaviour. There is a small overhead to using -threaded, but -threaded is also required to make a lot of things work (e.g. waitForProcess in a multithreaded program, not to mention parallelism). Anyone interested in doing this? Cheers Simon > Of course, this is just my two-cents. > > Cheers, > > - Ben > > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Default options for -threaded
Oh, thanks! I wasn't aware of either or these! Useful to know. That does cover the use case I could think of. Thanks, Tamar On Mon, Oct 10, 2016, 14:34 Ben Gamari wrote: > Phyx writes: > > > Oops, sorry, only just now seen this. It seems my overly aggressive > filters > > couldn't decide where to put the email :) > > > > I do agree to some extend with this. I'd prefer if I made a mistake for > my > > system not to hang. The one downside to this default though is that you > > can't just hand a program over to user and have it run at full > capabilities. > > > > If it possible to set this from inside a program? My guess is no, since > by > > the time you get to main the rts is already initialized? > > > > Would a useful alternative be to provide a compile flag that would change > > the default? e.g. opt-in? Since now there is a small burden on the end > user. > > > There exist two pretty good tools for accomplishing what you want, > > 1. Call Control.Concurrent.setNumCapabilities [1] from within your > application. > > 2. Use GHC's -with-rtsopts flag [2] to set the default RTS arguments > during compilation of your application. > > Cheers, > > - Ben > > > [1] > http://localhost:7000/file/opt/exp/ghc/roots/8.0.1/share/doc/ghc-8.0.1/html/libraries/base-4.9.0.0/Control-Concurrent.html#v:setNumCapabilities > [2] > http://downloads.haskell.org/~ghc/master/users-guide//phases.html?highlight=#ghc-flag--with-rtsopts > ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Default options for -threaded
Ah, I'm sorry, I believe I was thinking of -qm, which is supposed to prevent threads from being moved. I forgot these were separate options! And the latest version of the User's Guide includes a comment about -qm > This option is probably only of use for concurrent programs that explicitly > schedule threads onto CPUs with Control.Concurrent.forkOn. which is exactly what I had to do. On Mon, Oct 10, 2016, at 03:34, Phyx wrote: > Oh, this is surprising, I must admit I haven't tried forkIO, but with > forkOS is doesn't move the threads across capabilities. > > Do you know if this is by design or a bug? > > On Sat, Oct 8, 2016 at 6:13 PM, Eric Seidel wrote: > > > I would prefer keeping -N1 as a default, especially now that the number > > of capabilities can be set at runtime. Programs can then use the more > > common -j flag to enable parallelism. > > > > Regarding -qa, I was experimenting with it over the summer and found its > > behavior a bit surprising. It did prevent threads from being moved > > between capabilities, but it also forced all of the threads (created > > with forkIO) to be *spawned* on the same capability, which was > > unexpected. So -N -qa was, in my experience, equivalent to -N1! > > > > On Sat, Oct 8, 2016, at 09:55, Ben Gamari wrote: > > > loneti...@gmail.com writes: > > > > > > > Hi All, > > > > > > > > A user on https://ghc.haskell.org/trac/ghc/ticket/11054 has asked why > > > > -N -qa isn’t the default for -threaded. > > > > > > > I'm not sure that scheduling on all of the cores on the user's machine by > > > default is a good idea, especially given that our users have > > > learned to expect the existing default. Enabling affinity by default > > > seems reasonable if we have evidence that it helps the majority of > > > applications, but we would first need to introduce an additional > > > flag to disable it. > > > > > > In general I think -N1 is a reasonable default as it acknowledges the > > > fact that deploying parallelism is not something that can be done > > > blindly in many (most?) applications. To make effective use of > > > parallelism the user needs to understand their hardware, their > > > application, and its interaction with the runtime system and configure > > > the RTS appropriately. > > > > > > Of course, this is just my two-cents. > > > > > > Cheers, > > > > > > - Ben > > > ___ > > > ghc-devs mailing list > > > ghc-devs@haskell.org > > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > Email had 1 attachment: > > > + signature.asc > > > 1k (application/pgp-signature) > > ___ > > ghc-devs mailing list > > ghc-devs@haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Default options for -threaded
Phyx writes: > Oops, sorry, only just now seen this. It seems my overly aggressive filters > couldn't decide where to put the email :) > > I do agree to some extend with this. I'd prefer if I made a mistake for my > system not to hang. The one downside to this default though is that you > can't just hand a program over to user and have it run at full capabilities. > > If it possible to set this from inside a program? My guess is no, since by > the time you get to main the rts is already initialized? > > Would a useful alternative be to provide a compile flag that would change > the default? e.g. opt-in? Since now there is a small burden on the end user. > There exist two pretty good tools for accomplishing what you want, 1. Call Control.Concurrent.setNumCapabilities [1] from within your application. 2. Use GHC's -with-rtsopts flag [2] to set the default RTS arguments during compilation of your application. Cheers, - Ben [1] http://localhost:7000/file/opt/exp/ghc/roots/8.0.1/share/doc/ghc-8.0.1/html/libraries/base-4.9.0.0/Control-Concurrent.html#v:setNumCapabilities [2] http://downloads.haskell.org/~ghc/master/users-guide//phases.html?highlight=#ghc-flag--with-rtsopts signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Default options for -threaded
Oh, this is surprising, I must admit I haven't tried forkIO, but with forkOS is doesn't move the threads across capabilities. Do you know if this is by design or a bug? On Sat, Oct 8, 2016 at 6:13 PM, Eric Seidel wrote: > I would prefer keeping -N1 as a default, especially now that the number > of capabilities can be set at runtime. Programs can then use the more > common -j flag to enable parallelism. > > Regarding -qa, I was experimenting with it over the summer and found its > behavior a bit surprising. It did prevent threads from being moved > between capabilities, but it also forced all of the threads (created > with forkIO) to be *spawned* on the same capability, which was > unexpected. So -N -qa was, in my experience, equivalent to -N1! > > On Sat, Oct 8, 2016, at 09:55, Ben Gamari wrote: > > loneti...@gmail.com writes: > > > > > Hi All, > > > > > > A user on https://ghc.haskell.org/trac/ghc/ticket/11054 has asked why > > > -N -qa isn’t the default for -threaded. > > > > > I'm not sure that scheduling on all of the cores on the user's machine by > > default is a good idea, especially given that our users have > > learned to expect the existing default. Enabling affinity by default > > seems reasonable if we have evidence that it helps the majority of > > applications, but we would first need to introduce an additional > > flag to disable it. > > > > In general I think -N1 is a reasonable default as it acknowledges the > > fact that deploying parallelism is not something that can be done > > blindly in many (most?) applications. To make effective use of > > parallelism the user needs to understand their hardware, their > > application, and its interaction with the runtime system and configure > > the RTS appropriately. > > > > Of course, this is just my two-cents. > > > > Cheers, > > > > - Ben > > ___ > > ghc-devs mailing list > > ghc-devs@haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > Email had 1 attachment: > > + signature.asc > > 1k (application/pgp-signature) > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Default options for -threaded
Oops, sorry, only just now seen this. It seems my overly aggressive filters couldn't decide where to put the email :) I do agree to some extend with this. I'd prefer if I made a mistake for my system not to hang. The one downside to this default though is that you can't just hand a program over to user and have it run at full capabilities. If it possible to set this from inside a program? My guess is no, since by the time you get to main the rts is already initialized? Would a useful alternative be to provide a compile flag that would change the default? e.g. opt-in? Since now there is a small burden on the end user. Cheers, Tamar On Sat, Oct 8, 2016 at 5:55 PM, Ben Gamari wrote: > loneti...@gmail.com writes: > > > Hi All, > > > > A user on https://ghc.haskell.org/trac/ghc/ticket/11054 has asked why > > -N -qa isn’t the default for -threaded. > > > I'm not sure that scheduling on all of the cores on the user's machine by > default is a good idea, especially given that our users have > learned to expect the existing default. Enabling affinity by default > seems reasonable if we have evidence that it helps the majority of > applications, but we would first need to introduce an additional > flag to disable it. > > In general I think -N1 is a reasonable default as it acknowledges the > fact that deploying parallelism is not something that can be done > blindly in many (most?) applications. To make effective use of > parallelism the user needs to understand their hardware, their > application, and its interaction with the runtime system and configure > the RTS appropriately. > > Of course, this is just my two-cents. > > Cheers, > > - Ben > ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Default options for -threaded
I would prefer keeping -N1 as a default, especially now that the number of capabilities can be set at runtime. Programs can then use the more common -j flag to enable parallelism. Regarding -qa, I was experimenting with it over the summer and found its behavior a bit surprising. It did prevent threads from being moved between capabilities, but it also forced all of the threads (created with forkIO) to be *spawned* on the same capability, which was unexpected. So -N -qa was, in my experience, equivalent to -N1! On Sat, Oct 8, 2016, at 09:55, Ben Gamari wrote: > loneti...@gmail.com writes: > > > Hi All, > > > > A user on https://ghc.haskell.org/trac/ghc/ticket/11054 has asked why > > -N -qa isn’t the default for -threaded. > > > I'm not sure that scheduling on all of the cores on the user's machine by > default is a good idea, especially given that our users have > learned to expect the existing default. Enabling affinity by default > seems reasonable if we have evidence that it helps the majority of > applications, but we would first need to introduce an additional > flag to disable it. > > In general I think -N1 is a reasonable default as it acknowledges the > fact that deploying parallelism is not something that can be done > blindly in many (most?) applications. To make effective use of > parallelism the user needs to understand their hardware, their > application, and its interaction with the runtime system and configure > the RTS appropriately. > > Of course, this is just my two-cents. > > Cheers, > > - Ben > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > Email had 1 attachment: > + signature.asc > 1k (application/pgp-signature) ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Default options for -threaded
loneti...@gmail.com writes: > Hi All, > > A user on https://ghc.haskell.org/trac/ghc/ticket/11054 has asked why > -N -qa isn’t the default for -threaded. > I'm not sure that scheduling on all of the cores on the user's machine by default is a good idea, especially given that our users have learned to expect the existing default. Enabling affinity by default seems reasonable if we have evidence that it helps the majority of applications, but we would first need to introduce an additional flag to disable it. In general I think -N1 is a reasonable default as it acknowledges the fact that deploying parallelism is not something that can be done blindly in many (most?) applications. To make effective use of parallelism the user needs to understand their hardware, their application, and its interaction with the runtime system and configure the RTS appropriately. Of course, this is just my two-cents. Cheers, - Ben signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Default options for -threaded
Hi All, A user on https://ghc.haskell.org/trac/ghc/ticket/11054 has asked why -N -qa isn’t the default for -threaded. I’m afraid I don’t know a good reason why it’s not. Can anyone help shed some light on this? If there is no good reason, Would anyone object to making it the default? Thanks, Tamar ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
