Re: One-shot semantics in GHC event manager

2014-10-22 Thread 山本和彦
Q: Since registerFd uses OneShot and threadWait uses registerFd, basic IO functions use OneShot by default. No changes from GHC 7.8.3. Do I understand correctly? That is the idea. That being said adding another variant of registerFd (which as far as I know has three users) for

Re: One-shot semantics in GHC event manager

2014-10-21 Thread Ben Gamari
Kazu Yamamoto k...@iij.ad.jp writes: Hi, Andreas - want me to go ahead and get you some hardware to test Ben's patch in the mean time? This way we'll at least not leave it hanging until the last moment... I will also try this with two 20-core machines connected 10G on Monday. I

Re: One-shot semantics in GHC event manager

2014-10-21 Thread Ben Gamari
Kazu Yamamoto k...@iij.ad.jp writes: Hi, I measured the performace of GHC head, 7.8.3 and 7.8.3 + Ben's patch set. Server: witty 8080 -r -a -s +RTS -Nn *1 Measurement tool: weighttp -n 10 -c 1000 -k -t 19 http://192.168.0.1:8080/ Measurement env: two 20 core (w/o HT) machines

Re: One-shot semantics in GHC event manager

2014-10-21 Thread Ben Gamari
Kazu Yamamoto k...@iij.ad.jp writes: Hi, Hi Kazu, Andreas - want me to go ahead and get you some hardware to test Ben's patch in the mean time? This way we'll at least not leave it hanging until the last moment... I will also try this with two 20-core machines connected 10G on Monday.

Re: One-shot semantics in GHC event manager

2014-10-21 Thread 山本和彦
Ben, Hmm, uh oh. Thanks for testing this. I'll try to reproduce this on my end. It looks like it shouldn't be so hard as even the single-threaded performance regresses drastically. Just to confirm, you are using the latest revision of D347? I used the following as you suggested:

Re: One-shot semantics in GHC event manager

2014-10-21 Thread 山本和彦
Ben, This may be due to lacking INLINEs on definitions added in GHC.Event.Internal [1]. I'm currently in the middle of reproducing these results on an EC2 instance to confirm this. So far the results look much more consistent than my previous attempts at benchmarking on my own hardware. If

Re: One-shot semantics in GHC event manager

2014-10-21 Thread Ben Gamari
Kazu Yamamoto k...@iij.ad.jp writes: Ben, This may be due to lacking INLINEs on definitions added in GHC.Event.Internal [1]. I'm currently in the middle of reproducing these results on an EC2 instance to confirm this. So far the results look much more consistent than my previous attempts at

Re: One-shot semantics in GHC event manager

2014-10-21 Thread 山本和彦
I already pushed it. The commit in question is 5dce47eb8415eb31e1c6759b6f6a2ef5bfe32470. Thanks for the benchmarking! I believe this is in bgamari/ghc (for GHC 7.10?). I'm using bgamari/packages-base for GHC 7.8 and asking to push the same commit to this repo. Actually I compared the latest

Re: One-shot semantics in GHC event manager

2014-10-21 Thread Ben Gamari
Kazu Yamamoto k...@iij.ad.jp writes: I already pushed it. The commit in question is 5dce47eb8415eb31e1c6759b6f6a2ef5bfe32470. Thanks for the benchmarking! I believe this is in bgamari/ghc (for GHC 7.10?). I'm using bgamari/packages-base for GHC 7.8 and asking to push the same commit to this

Re: One-shot semantics in GHC event manager

2014-10-21 Thread 山本和彦
Ahh, yes. Sorry, I forgot you were on 7.8. Just pushed a new patch to the event-rework-squashed branch [1]. I believe that you are trying to merge your patches to GHC 7.8.4? If not, I will work on the GHC head branch. --Kazu ___ ghc-devs mailing

Re: One-shot semantics in GHC event manager

2014-10-21 Thread 山本和彦
Well, Bas was wondering whether this would be possible. At this point I'm a bit on the fence; on one hand it's not a crucial fix (we have a workaround in usb) and it may involve changes to exported interfaces (although not very high visibility). On the other hand, it's a pretty easy change to

Re: One-shot semantics in GHC event manager

2014-10-21 Thread Ben Gamari
Kazu Yamamoto k...@iij.ad.jp writes: Well, Bas was wondering whether this would be possible. At this point I'm a bit on the fence; on one hand it's not a crucial fix (we have a workaround in usb) and it may involve changes to exported interfaces (although not very high visibility). On the

Re: One-shot semantics in GHC event manager

2014-10-21 Thread 山本和彦
Out of curiosity are these numbers from single runs or do you average? Run three times and took the middle in this time. What are the uncertainties on these numbers? Even on the Rackspace machines I was finding very large variances in my benchmarks, largely due to far outliers. I didn't

Re: One-shot semantics in GHC event manager

2014-10-21 Thread 山本和彦
Hi Austin, You need to set the CPU into C0 using /dev/cpu_dma_latency. Here's a short paper with a program to show the way to do it[1]. This paper is what I'm looking for. Thanks! --Kazu ___ ghc-devs mailing list ghc-devs@haskell.org

Re: One-shot semantics in GHC event manager

2014-10-20 Thread 山本和彦
Hi, Andreas - want me to go ahead and get you some hardware to test Ben's patch in the mean time? This way we'll at least not leave it hanging until the last moment... I will also try this with two 20-core machines connected 10G on Monday. I measured the performace of GHC head, 7.8.3 and

Re: One-shot semantics in GHC event manager

2014-10-17 Thread Bas van Dijk
Hi Ben, Austin, Is there any chance of Ben's event manager patch landing in GHC-7.8.4? Bas On 13 October 2014 21:05, Ben Gamari bgamari.f...@gmail.com wrote: Ben Gamari bgamari.f...@gmail.com writes: Andreas Voellmy andreas.voel...@gmail.com writes: On Sat, Oct 11, 2014 at 12:17 PM, Ben

Re: One-shot semantics in GHC event manager

2014-10-17 Thread Andreas Voellmy
I haven't had a chance to dig into Ben's patch yet, but I expect it will accepted soon - I don't think the change will affect performance. Austin, would it be possible to get a relatively minor change to the event manager into 7.8.4? It may change a semi-public API under GHC.Event, but will not

Re: One-shot semantics in GHC event manager

2014-10-17 Thread Austin Seipp
The catch with such a change is that there is no macro to determine whether we're using 7.8.3 or 7.8.4, so it's harder for users to figure things out (they have to use `MIN_VERSION_base` from Cabal). But maybe that doesn'tm atter too much. So, yes, I think it's doable, but that's a sticky bit.

Re: One-shot semantics in GHC event manager

2014-10-17 Thread 山本和彦
Austin, Andreas - want me to go ahead and get you some hardware to test Ben's patch in the mean time? This way we'll at least not leave it hanging until the last moment... I will also try this with two 20-core machines connected 10G on Monday. --Kazu

Re: One-shot semantics in GHC event manager

2014-10-17 Thread Ben Gamari
Austin Seipp aus...@well-typed.com writes: The catch with such a change is that there is no macro to determine whether we're using 7.8.3 or 7.8.4, so it's harder for users to figure things out (they have to use `MIN_VERSION_base` from Cabal). But maybe that doesn'tm atter too much. So, yes, I

Re: One-shot semantics in GHC event manager

2014-10-17 Thread Mikhail Glushenkov
Hi, On 17 October 2014 16:27, Austin Seipp aus...@well-typed.com wrote: The catch with such a change is that there is no macro to determine whether we're using 7.8.3 or 7.8.4, so it's harder for users to figure things out (they have to use `MIN_VERSION_base` from Cabal). But maybe that

Re: One-shot semantics in GHC event manager

2014-10-17 Thread Ben Gamari
Austin Seipp aus...@well-typed.com writes: The catch with such a change is that there is no macro to determine whether we're using 7.8.3 or 7.8.4, so it's harder for users to figure things out (they have to use `MIN_VERSION_base` from Cabal). But maybe that doesn'tm atter too much. So, yes, I

Re: One-shot semantics in GHC event manager

2014-10-14 Thread Herbert Valerio Riedel
On 2014-10-13 at 23:33:13 +0200, Austin Seipp wrote: [...] Also, if any other developers (like Andreas, Johan, Bryan, etc) in this space want a big machine to test it on, I can probably equip you with one (or several). Since Rackspace is so gracious to us, we can immediately allocate

Re: One-shot semantics in GHC event manager

2014-10-14 Thread Andreas Voellmy
This is awesome. I'd like to try to recreate some of the evaluations for the multicore IO manager paper on that 40 core system at backspace. How can I get access to this? I'll jump on IRC - maybe it is easier to chat in realtime. On Mon, Oct 13, 2014 at 5:33 PM, Austin Seipp aus...@well-typed.com

Re: One-shot semantics in GHC event manager

2014-10-14 Thread Austin Seipp
Hey Andreas, The basic rundown is that if we equip you with an account, you can just do it yourself. Although we'd like to restrict access a bit more; I'll figure something out. Yeah, if you hop on IRC, we can chat quickly about it and work something out in the mean time. On Tue, Oct 14, 2014

Re: One-shot semantics in GHC event manager

2014-10-14 Thread Ben Gamari
Andreas Voellmy andreas.voel...@gmail.com writes: This is awesome. I'd like to try to recreate some of the evaluations for the multicore IO manager paper on that 40 core system at backspace. How can I get access to this? I'll jump on IRC - maybe it is easier to chat in realtime. Do you

Re: One-shot semantics in GHC event manager

2014-10-14 Thread Andreas Voellmy
Yes, I'll try to describe it and script it so that others can understand the benchmarks and run it easily as well. On Tue, Oct 14, 2014 at 1:23 PM, Ben Gamari bgamari.f...@gmail.com wrote: Andreas Voellmy andreas.voel...@gmail.com writes: This is awesome. I'd like to try to recreate some of

Re: One-shot semantics in GHC event manager

2014-10-13 Thread Ben Gamari
Ben Gamari bgamari.f...@gmail.com writes: Andreas Voellmy andreas.voel...@gmail.com writes: On Sat, Oct 11, 2014 at 12:17 PM, Ben Gamari bgamari.f...@gmail.com wrote: Ah... so this is not useful to you. I guess we could add `loop` to GHC.Event's export list. On the other hand, I like your

Re: One-shot semantics in GHC event manager

2014-10-13 Thread Austin Seipp
For the record, I talked to Ben earlier on IRC, and I can provide him with a machine to do intense benchmarks of the new I/O manager. Also, if any other developers (like Andreas, Johan, Bryan, etc) in this space want a big machine to test it on, I can probably equip you with one (or several).

Re: One-shot semantics in GHC event manager

2014-10-11 Thread Andreas Voellmy
On Sat, Oct 11, 2014 at 1:07 AM, Ben Gamari bgamari.f...@gmail.com wrote: Thanks for your quick reply! Andreas Voellmy andreas.voel...@gmail.com writes: On Sat, Oct 11, 2014 at 12:17 AM, Ben Gamari bgamari.f...@gmail.com wrote: I'm a bit perplexed as to why the change was made in the

Re: One-shot semantics in GHC event manager

2014-10-11 Thread Andreas Voellmy
Another way to fix usb would be to re-register the callback after a previously registered callback is fired. Of course it is cheaper not to have to re-register, but re-registration in the latest IO manager should be fairly cheap, so this may not be a performance problem for usb. Would this work

Re: One-shot semantics in GHC event manager

2014-10-11 Thread Ben Gamari
Andreas Voellmy andreas.voel...@gmail.com writes: Another way to fix usb would be to re-register the callback after a previously registered callback is fired. Of course it is cheaper not to have to re-register, but re-registration in the latest IO manager should be fairly cheap, so this may

Re: One-shot semantics in GHC event manager

2014-10-11 Thread Ben Gamari
Andreas Voellmy andreas.voel...@gmail.com writes: On Sat, Oct 11, 2014 at 1:07 AM, Ben Gamari bgamari.f...@gmail.com wrote: Thanks for your quick reply! What I'm wondering is what the extra complexity bought us. It seems like the same thing could have been achieved with less breakage by

Re: One-shot semantics in GHC event manager

2014-10-11 Thread Andreas Voellmy
On Sat, Oct 11, 2014 at 12:17 PM, Ben Gamari bgamari.f...@gmail.com wrote: Andreas Voellmy andreas.voel...@gmail.com writes: On Sat, Oct 11, 2014 at 1:07 AM, Ben Gamari bgamari.f...@gmail.com wrote: Thanks for your quick reply! What I'm wondering is what the extra complexity bought

Re: One-shot semantics in GHC event manager

2014-10-11 Thread Ben Gamari
Andreas Voellmy andreas.voel...@gmail.com writes: On Sat, Oct 11, 2014 at 12:17 PM, Ben Gamari bgamari.f...@gmail.com wrote: Yes, but it will be invoked by GHC.Thread and any other callers of it will simply block indefinitely waiting for the thread that is running loop to give it up - which

Re: One-shot semantics in GHC event manager

2014-10-10 Thread Andreas Voellmy
On Sat, Oct 11, 2014 at 12:17 AM, Ben Gamari bgamari.f...@gmail.com wrote: In ba2555ef and a6f52b19 one-shot semantics were added to event manager in `base`. If my understanding of this code is correct, in this mode the event manager will use only notify the user of the first event on a

Re: One-shot semantics in GHC event manager

2014-10-10 Thread Ben Gamari
Thanks for your quick reply! Andreas Voellmy andreas.voel...@gmail.com writes: On Sat, Oct 11, 2014 at 12:17 AM, Ben Gamari bgamari.f...@gmail.com wrote: I'm a bit perplexed as to why the change was made in the way that it was. Making one-shot a event-manager-wide attribute seems to add a