Re: [Haskell-cafe] Observer pattern in haskell FRP
On 11/27/2012 07:12 AM, Ertugrul Söylemez wrote: Nathan Hüsken nathan.hues...@posteo.de wrote: When writing games in other (imperative) languages, I like to separate the game logic from the rendering. For this I use something similar to the observer pattern. [...] So I am wondering: Is there (or can someone think of) a different pattern by which this could be archived? Or asked different: How would you do it? [...] As far as possible you should use a stateless monad, ideally simply Identity or a reader: type MyWire = WireM ((-) AppConfig) myApp :: MyWire a GameFrame This is only the first part of the story. The second part is the rendering itself. You certainly want a way to make use of various OpenGL extensions like vertex buffers, which are inherently stateful. One sensible way I see is not to output the game's state, but rather a state delta: myApp :: MyWire a GameDelta That way you can do the imperative stateful plumbing outside of the application's wire and get the full power of FRP without giving up efficient rendering. GameDelta itself would essentially be a type for game state commands. In fact it could be a (free) monad: myApp :: MyWire a (GameDelta ()) someDelta :: GameDelta () someDelta = do randomPos - liftA2 (,) getRandom getRandom replicateM_ 4 (addCreature randomPos) getPlayerPos = centerCamOver Then you could perform that monadic action as part of the rendering process. That sound like a good Idea. But I still have the problem of connection game logic objects with rendering objects, or am I missing something? Implementing addCreature is fine, but when I want a removeCreature, it has to remove the correct creature from a potentially very large list/set of creatures. How can I efficiently build this connections (which corresponds to a pointer in other languages, I guess)? Thanks! Nathan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] cabal configure cabal build cabal install
On Tue, 27 Nov 2012 02:20:35 -0500 Albert Y. C. Lai tre...@vex.net wrote: When cabal build succeeds, it always says: (older) registering name-version (newer) In-place registering name-version That's what it says. But use ghc-pkg and other tests to verify that no registration whatsoever has happened. It doesn't register in user package-db, it registers in it's own dist/package.conf.inplace. If it didn't you wouldn't be able to build an executable and a library in one package such that executable depends on the library. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Can a GC delay TCP connection formation?
GHC has a stop the world garbage collector, meaning that while major GC is happening, the entire process must be halted. In my experience GC pause times are typically low, but depending the heap residency profile of your application (and the quantity of garbage being produced by it), this may not be the case. If you have a hard real-time requirement then a garbage-collected language may not be appropriate for you. On Tue, Nov 27, 2012 at 5:19 AM, Jeff Shaw shawj...@gmail.com wrote: Hello, I've run into an issue that makes me think that when the GHC GC runs while a Snap or Warp HTTP server is serving connections, the GC prevents or delays TCP connections from forming. My application requires that TCP connections form within a few tens of milliseconds. I'm wondering if anyone else has run into this issue, and if there are some GC flags that could help. I've tried a few, such as -H and -c, and haven't found anything to help. I'm using GHC 7.4.1. Thanks, Jeff ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe -- Gregory Collins g...@gregorycollins.net ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Observer pattern in haskell FRP
Nathan Hüsken nathan.hues...@posteo.de wrote: In fact it could be a (free) monad: myApp :: MyWire a (GameDelta ()) someDelta :: GameDelta () someDelta = do randomPos - liftA2 (,) getRandom getRandom replicateM_ 4 (addCreature randomPos) getPlayerPos = centerCamOver Then you could perform that monadic action as part of the rendering process. That sound like a good Idea. But I still have the problem of connection game logic objects with rendering objects, or am I missing something? Implementing addCreature is fine, but when I want a removeCreature, it has to remove the correct creature from a potentially very large list/set of creatures. How can I efficiently build this connections (which corresponds to a pointer in other languages, I guess)? That was a simplified example. In the real world it depends on what generates your creatures. If they can be generated all over the code then you need some form of identifier generation. This can be done by the wire's underlying monad: type Identifier = Int type Game = WireM (StateT Identifier ((-) AppConfig)) A creature then may look something like this: creature :: Game World (Creature, GameDelta ()) The wire produces a creating action at the first instant, then switches to the creature's regular wire. The various GameDelta actions form at least a monoid under (), and depending on your design even a group: rec (creature1, gd1) - creature - w (creature2, gd2) - creature - w (creature3, gd3) - creature - w w - delay (World []) - World [c1, c2, c3] id - (gd1 gd2 gd3) That's the basic idea. Greets, Ertugrul -- Not to be or to be and (not to be or to be and (not to be or to be and (not to be or to be and ... that is the list monad. signature.asc Description: PGP signature ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Conduit and pipelined protocol processing using a threadpool
All, I've written a library to implement servers for some protocol using Conduit (I'll announce more details later). The protocol supports pipelining, i.e. a client can send a 'command' which contains some opaque 'handle' chosen by the client, the server processes this command, then returns some reply which contains this handle. The client is free to send other commands before receiving a reply for any previous request, and the server can process these commands in any order, sequential or concurrently. The library is based on network-conduit's Application style [1], as such now I write code like (OTOH) application :: AppData IO - IO () application client = appSource client $= handler $$ appSink client where handler = do negotiateResult - MyLib.negotiate liftIO $ validateNegotiateResult negotiateResult MyLib.sendInformation 123 loop loop = do command - MyLib.getCommand case command of CommandA handle arg - do result - liftIO $ doComplexProcessingA arg MyLib.sendReply handle result loop Disconnect - return () This approach handles commands in-order, sequentially. Since command processing can involve quite some IO operations to disk or network, I've been trying to support pipelining on the server-side, but as of now I was unable to get things working. The idea would be to have a pool of worker threads, which receive work items from some channel, then return any result on some other channel, which should then be returned to the client. This means inside loop I would have 2 sources: commands coming from the client (using 'MyLib.getCommand :: MonadIO m = Pipe ByteString ByteString o u m Command'), as well as command results coming from the worker threads through the result channel. Whenever the first source produces something, it should be pushed onto the work queue, and whenever the second on yields some result it should be sent to the client using 'MyLib.sendReply :: Monad m = Handle - Result - Pipe l i ByteString u m ()' I've been fighting this for a while and haven't managed to get something sensible working. Maybe the design of my library is flawed, or maybe I'm approaching the problem incorrectly, or ... Has this ever been done before, or would anyone have some pointers how to tackle this? Thanks, Nicolas [1] http://hackage.haskell.org/packages/archive/network-conduit/0.6.1.1/doc/html/Data-Conduit-Network.html#g:2 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Problem with benchmarking FFI calls with Criterion
I tested the same code on my second machine - Debian Squeeze (kernel 2.6.32)
with GHC 7.4.1 - and
the results are extremely surprising. At first I was unable to reproduce the
problem and got
consistent runtimes of about 107us:
benchmarking FFI/C binding
mean: 107.3837 us, lb 107.2013 us, ub 107.5862 us, ci 0.950
std dev: 983.6046 ns, lb 822.6750 ns, ub 1.292724 us, ci 0.950
benchmarking FFI/C binding
mean: 108.1152 us, lb 107.9457 us, ub 108.3052 us, ci 0.950
std dev: 916.2469 ns, lb 793.1004 ns, ub 1.122127 us, ci 0.950
I started experimenting with the vector size and after bumping its size to 32K
elements I started
getting this:
benchmarking FFI/C binding
mean: 38.50100 us, lb 36.71525 us, ub 46.87665 us, ci 0.950
std dev: 16.93131 us, lb 1.033678 us, ub 40.23900 us, ci 0.950
found 6 outliers among 100 samples (6.0%)
3 (3.0%) low mild
3 (3.0%) high severe
variance introduced by outliers: 98.921%
variance is severely inflated by outliers
benchmarking FFI/C binding
mean: 209.9733 us, lb 209.5316 us, ub 210.4680 us, ci 0.950
std dev: 2.401398 us, lb 2.052981 us, ub 2.889688 us, ci 0.950
First result is always about 39us (2,5 faster, despite longer signal!) while
the remaining
benchmarks take almost two times longer.
Janek
Dnia niedziela, 25 listopada 2012, Janek S. napisał:
Well, it seems that this only happens on my machine. I will try to test
this code on different computer and see if I can reproduce it.
I don't think using existing vector is a good idea - it would make the code
impure.
Janek
Dnia sobota, 24 listopada 2012, Branimir Maksimovic napisał:
I don't see such behavior neither.ubuntu 12.10, ghc 7.4.2.
Perhaps this has to do with how malloc allocates /cachebehavior. If you
try not to allocate array rather use existing one perhaps there would be
no inconsistency?It looks to me that's about CPU cache performance.
Branimir
I'm using GHC 7.4.2 on x86_64 openSUSE Linux, kernel 2.6.37.6.
Janek
Dnia piątek, 23 listopada 2012, Edward Z. Yang napisał:
Running the sample code on GHC 7.4.2, I don't see the one
fast, rest slow behavior. What version of GHC are you running?
Edward
Excerpts from Janek S.'s message of Fri Nov 23 13:42:03 -0500 2012:
What happens if you do the benchmark without unsafePerformIO
involved?
I removed unsafePerformIO, changed copy to have type Vector Double
- IO (Vector Double) and modified benchmarks like this:
bench C binding $ whnfIO (copy signal)
I see no difference - one benchmark runs fast, remaining ones run
slow.
Janek
Excerpts from Janek S.'s message of Fri Nov 23 10:44:15 -0500 2012:
I am using Criterion library to benchmark C code called via FFI
bindings and I've ran into a problem that looks like a bug.
The first benchmark that uses FFI runs correctly, but
subsequent benchmarks run much longer. I created demo code
(about 50 lines, available at github:
https://gist.github.com/4135698 ) in which C function copies a
vector of doubles. I benchmark that function a couple of times.
First run results in avarage time of about 17us, subsequent
runs take about 45us. In my real code additional time was about
15us and it seemed to be a constant factor, not relative to
correct run time. The surprising thing is that if my C
function only allocates memory and does no copying:
double* c_copy( double* inArr, int arrLen ) {
double* outArr = malloc( arrLen * sizeof( double ) );
return outArr;
}
then all is well - all runs take similar amount of time. I also
noticed that sometimes in my demo code all runs take about
45us, but this does not seem to happen in my real code - first
run is always shorter.
Does anyone have an idea what is going on?
Janek
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] 1st São Paulo Haskell Meeting [1° Encontro de Haskellers em São Paulo]
Hey! I'd like to invite you to the 1st São Paulo Haskell Meeting! It's going to be something simple, we just want to meet each other and talk about Haskell =). We already have 9 people confirmed on the Google+ event [1], so come join us already! Cheers, PS: We still didn't set the place yet. [1] https://plus.google.com/events/cng3rcv1tjl84g2juddk1i36icg -- Felipe. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Problem with benchmarking FFI calls with Criterion
I once had a problem like this. It turned out that my laptop was stepping
the cpu clock rate down whenever it got warm. Disabling that feature in my
BIOS fixed it. Your problem might be similar.
On Nov 27, 2012 7:23 AM, Janek S. fremenz...@poczta.onet.pl wrote:
I tested the same code on my second machine - Debian Squeeze (kernel
2.6.32) with GHC 7.4.1 - and
the results are extremely surprising. At first I was unable to reproduce
the problem and got
consistent runtimes of about 107us:
benchmarking FFI/C binding
mean: 107.3837 us, lb 107.2013 us, ub 107.5862 us, ci 0.950
std dev: 983.6046 ns, lb 822.6750 ns, ub 1.292724 us, ci 0.950
benchmarking FFI/C binding
mean: 108.1152 us, lb 107.9457 us, ub 108.3052 us, ci 0.950
std dev: 916.2469 ns, lb 793.1004 ns, ub 1.122127 us, ci 0.950
I started experimenting with the vector size and after bumping its size to
32K elements I started
getting this:
benchmarking FFI/C binding
mean: 38.50100 us, lb 36.71525 us, ub 46.87665 us, ci 0.950
std dev: 16.93131 us, lb 1.033678 us, ub 40.23900 us, ci 0.950
found 6 outliers among 100 samples (6.0%)
3 (3.0%) low mild
3 (3.0%) high severe
variance introduced by outliers: 98.921%
variance is severely inflated by outliers
benchmarking FFI/C binding
mean: 209.9733 us, lb 209.5316 us, ub 210.4680 us, ci 0.950
std dev: 2.401398 us, lb 2.052981 us, ub 2.889688 us, ci 0.950
First result is always about 39us (2,5 faster, despite longer signal!)
while the remaining
benchmarks take almost two times longer.
Janek
Dnia niedziela, 25 listopada 2012, Janek S. napisał:
Well, it seems that this only happens on my machine. I will try to test
this code on different computer and see if I can reproduce it.
I don't think using existing vector is a good idea - it would make the
code
impure.
Janek
Dnia sobota, 24 listopada 2012, Branimir Maksimovic napisał:
I don't see such behavior neither.ubuntu 12.10, ghc 7.4.2.
Perhaps this has to do with how malloc allocates /cachebehavior. If you
try not to allocate array rather use existing one perhaps there would
be
no inconsistency?It looks to me that's about CPU cache performance.
Branimir
I'm using GHC 7.4.2 on x86_64 openSUSE Linux, kernel 2.6.37.6.
Janek
Dnia piątek, 23 listopada 2012, Edward Z. Yang napisał:
Running the sample code on GHC 7.4.2, I don't see the one
fast, rest slow behavior. What version of GHC are you running?
Edward
Excerpts from Janek S.'s message of Fri Nov 23 13:42:03 -0500 2012:
What happens if you do the benchmark without unsafePerformIO
involved?
I removed unsafePerformIO, changed copy to have type Vector
Double
- IO (Vector Double) and modified benchmarks like this:
bench C binding $ whnfIO (copy signal)
I see no difference - one benchmark runs fast, remaining ones run
slow.
Janek
Excerpts from Janek S.'s message of Fri Nov 23 10:44:15 -0500
2012:
I am using Criterion library to benchmark C code called via
FFI
bindings and I've ran into a problem that looks like a bug.
The first benchmark that uses FFI runs correctly, but
subsequent benchmarks run much longer. I created demo code
(about 50 lines, available at github:
https://gist.github.com/4135698 ) in which C function
copies a
vector of doubles. I benchmark that function a couple of
times.
First run results in avarage time of about 17us, subsequent
runs take about 45us. In my real code additional time was
about
15us and it seemed to be a constant factor, not relative to
correct run time. The surprising thing is that if my C
function only allocates memory and does no copying:
double* c_copy( double* inArr, int arrLen ) {
double* outArr = malloc( arrLen * sizeof( double ) );
return outArr;
}
then all is well - all runs take similar amount of time. I
also
noticed that sometimes in my demo code all runs take about
45us, but this does not seem to happen in my real code -
first
run is always shorter.
Does anyone have an idea what is going on?
Janek
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Problem with benchmarking FFI calls with Criterion
Dnia wtorek, 27 listopada 2012, Jake McArthur napisał:
I once had a problem like this. It turned out that my laptop was stepping
the cpu clock rate down whenever it got warm. Disabling that feature in my
BIOS fixed it. Your problem might be similar.
I just check - I disabled frequency scaling and results are the same. Actually
I doubt that 39us
of benchmarking would cause CPU overheating with such repeatibility. Besides,
this wouldn't
explain why the first benchmark actually got faster.
Janek
On Nov 27, 2012 7:23 AM, Janek S. fremenz...@poczta.onet.pl wrote:
I tested the same code on my second machine - Debian Squeeze (kernel
2.6.32) with GHC 7.4.1 - and
the results are extremely surprising. At first I was unable to reproduce
the problem and got
consistent runtimes of about 107us:
benchmarking FFI/C binding
mean: 107.3837 us, lb 107.2013 us, ub 107.5862 us, ci 0.950
std dev: 983.6046 ns, lb 822.6750 ns, ub 1.292724 us, ci 0.950
benchmarking FFI/C binding
mean: 108.1152 us, lb 107.9457 us, ub 108.3052 us, ci 0.950
std dev: 916.2469 ns, lb 793.1004 ns, ub 1.122127 us, ci 0.950
I started experimenting with the vector size and after bumping its size
to 32K elements I started
getting this:
benchmarking FFI/C binding
mean: 38.50100 us, lb 36.71525 us, ub 46.87665 us, ci 0.950
std dev: 16.93131 us, lb 1.033678 us, ub 40.23900 us, ci 0.950
found 6 outliers among 100 samples (6.0%)
3 (3.0%) low mild
3 (3.0%) high severe
variance introduced by outliers: 98.921%
variance is severely inflated by outliers
benchmarking FFI/C binding
mean: 209.9733 us, lb 209.5316 us, ub 210.4680 us, ci 0.950
std dev: 2.401398 us, lb 2.052981 us, ub 2.889688 us, ci 0.950
First result is always about 39us (2,5 faster, despite longer signal!)
while the remaining
benchmarks take almost two times longer.
Janek
Dnia niedziela, 25 listopada 2012, Janek S. napisał:
Well, it seems that this only happens on my machine. I will try to test
this code on different computer and see if I can reproduce it.
I don't think using existing vector is a good idea - it would make the
code
impure.
Janek
Dnia sobota, 24 listopada 2012, Branimir Maksimovic napisał:
I don't see such behavior neither.ubuntu 12.10, ghc 7.4.2.
Perhaps this has to do with how malloc allocates /cachebehavior. If
you try not to allocate array rather use existing one perhaps there
would
be
no inconsistency?It looks to me that's about CPU cache performance.
Branimir
I'm using GHC 7.4.2 on x86_64 openSUSE Linux, kernel 2.6.37.6.
Janek
Dnia piątek, 23 listopada 2012, Edward Z. Yang napisał:
Running the sample code on GHC 7.4.2, I don't see the one
fast, rest slow behavior. What version of GHC are you running?
Edward
Excerpts from Janek S.'s message of Fri Nov 23 13:42:03 -0500 2012:
What happens if you do the benchmark without unsafePerformIO
involved?
I removed unsafePerformIO, changed copy to have type Vector
Double
- IO (Vector Double) and modified benchmarks like this:
bench C binding $ whnfIO (copy signal)
I see no difference - one benchmark runs fast, remaining ones
run slow.
Janek
Excerpts from Janek S.'s message of Fri Nov 23 10:44:15 -0500
2012:
I am using Criterion library to benchmark C code called via
FFI
bindings and I've ran into a problem that looks like a bug.
The first benchmark that uses FFI runs correctly, but
subsequent benchmarks run much longer. I created demo code
(about 50 lines, available at github:
https://gist.github.com/4135698 ) in which C function
copies a
vector of doubles. I benchmark that function a couple of
times.
First run results in avarage time of about 17us, subsequent
runs take about 45us. In my real code additional time was
about
15us and it seemed to be a constant factor, not relative to
correct run time. The surprising thing is that if my C
function only allocates memory and does no copying:
double* c_copy( double* inArr, int arrLen ) {
double* outArr = malloc( arrLen * sizeof( double ) );
return outArr;
}
then all is well - all runs take similar amount of time. I
also
noticed that sometimes in my demo code all runs take about
45us, but this does not seem to happen in my real code -
first
run is always shorter.
Does anyone have an idea what is going on?
Janek
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Problem with benchmarking FFI calls with Criterion
Did you pass the option to criterion asking it to do a GC between
trials? You might be measuring a GC pause.
On Tue, Nov 27, 2012 at 2:41 PM, Janek S. fremenz...@poczta.onet.pl wrote:
Dnia wtorek, 27 listopada 2012, Jake McArthur napisał:
I once had a problem like this. It turned out that my laptop was stepping
the cpu clock rate down whenever it got warm. Disabling that feature in my
BIOS fixed it. Your problem might be similar.
I just check - I disabled frequency scaling and results are the same.
Actually I doubt that 39us
of benchmarking would cause CPU overheating with such repeatibility. Besides,
this wouldn't
explain why the first benchmark actually got faster.
Janek
On Nov 27, 2012 7:23 AM, Janek S. fremenz...@poczta.onet.pl wrote:
I tested the same code on my second machine - Debian Squeeze (kernel
2.6.32) with GHC 7.4.1 - and
the results are extremely surprising. At first I was unable to reproduce
the problem and got
consistent runtimes of about 107us:
benchmarking FFI/C binding
mean: 107.3837 us, lb 107.2013 us, ub 107.5862 us, ci 0.950
std dev: 983.6046 ns, lb 822.6750 ns, ub 1.292724 us, ci 0.950
benchmarking FFI/C binding
mean: 108.1152 us, lb 107.9457 us, ub 108.3052 us, ci 0.950
std dev: 916.2469 ns, lb 793.1004 ns, ub 1.122127 us, ci 0.950
I started experimenting with the vector size and after bumping its size
to 32K elements I started
getting this:
benchmarking FFI/C binding
mean: 38.50100 us, lb 36.71525 us, ub 46.87665 us, ci 0.950
std dev: 16.93131 us, lb 1.033678 us, ub 40.23900 us, ci 0.950
found 6 outliers among 100 samples (6.0%)
3 (3.0%) low mild
3 (3.0%) high severe
variance introduced by outliers: 98.921%
variance is severely inflated by outliers
benchmarking FFI/C binding
mean: 209.9733 us, lb 209.5316 us, ub 210.4680 us, ci 0.950
std dev: 2.401398 us, lb 2.052981 us, ub 2.889688 us, ci 0.950
First result is always about 39us (2,5 faster, despite longer signal!)
while the remaining
benchmarks take almost two times longer.
Janek
Dnia niedziela, 25 listopada 2012, Janek S. napisał:
Well, it seems that this only happens on my machine. I will try to test
this code on different computer and see if I can reproduce it.
I don't think using existing vector is a good idea - it would make the
code
impure.
Janek
Dnia sobota, 24 listopada 2012, Branimir Maksimovic napisał:
I don't see such behavior neither.ubuntu 12.10, ghc 7.4.2.
Perhaps this has to do with how malloc allocates /cachebehavior. If
you try not to allocate array rather use existing one perhaps there
would
be
no inconsistency?It looks to me that's about CPU cache performance.
Branimir
I'm using GHC 7.4.2 on x86_64 openSUSE Linux, kernel 2.6.37.6.
Janek
Dnia piątek, 23 listopada 2012, Edward Z. Yang napisał:
Running the sample code on GHC 7.4.2, I don't see the one
fast, rest slow behavior. What version of GHC are you running?
Edward
Excerpts from Janek S.'s message of Fri Nov 23 13:42:03 -0500 2012:
What happens if you do the benchmark without unsafePerformIO
involved?
I removed unsafePerformIO, changed copy to have type Vector
Double
- IO (Vector Double) and modified benchmarks like this:
bench C binding $ whnfIO (copy signal)
I see no difference - one benchmark runs fast, remaining ones
run slow.
Janek
Excerpts from Janek S.'s message of Fri Nov 23 10:44:15 -0500
2012:
I am using Criterion library to benchmark C code called via
FFI
bindings and I've ran into a problem that looks like a bug.
The first benchmark that uses FFI runs correctly, but
subsequent benchmarks run much longer. I created demo code
(about 50 lines, available at github:
https://gist.github.com/4135698 ) in which C function
copies a
vector of doubles. I benchmark that function a couple of
times.
First run results in avarage time of about 17us, subsequent
runs take about 45us. In my real code additional time was
about
15us and it seemed to be a constant factor, not relative to
correct run time. The surprising thing is that if my C
function only allocates memory and does no copying:
double* c_copy( double* inArr, int arrLen ) {
double* outArr = malloc( arrLen * sizeof( double ) );
return outArr;
}
then all is well - all runs take similar amount of time. I
also
noticed that sometimes in my demo code all runs take about
45us, but this does not seem to happen in my real code -
first
run is always shorter.
Does anyone have an idea what is going on?
Re: [Haskell-cafe] Problem with benchmarking FFI calls with Criterion
Dnia wtorek, 27 listopada 2012, Gregory Collins napisał:
Did you pass the option to criterion asking it to do a GC between
trials?
Yes.
You might be measuring a GC pause.
On Tue, Nov 27, 2012 at 2:41 PM, Janek S. fremenz...@poczta.onet.pl wrote:
Dnia wtorek, 27 listopada 2012, Jake McArthur napisał:
I once had a problem like this. It turned out that my laptop was
stepping the cpu clock rate down whenever it got warm. Disabling that
feature in my BIOS fixed it. Your problem might be similar.
I just check - I disabled frequency scaling and results are the same.
Actually I doubt that 39us of benchmarking would cause CPU overheating
with such repeatibility. Besides, this wouldn't explain why the first
benchmark actually got faster.
Janek
On Nov 27, 2012 7:23 AM, Janek S. fremenz...@poczta.onet.pl wrote:
I tested the same code on my second machine - Debian Squeeze (kernel
2.6.32) with GHC 7.4.1 - and
the results are extremely surprising. At first I was unable to
reproduce the problem and got
consistent runtimes of about 107us:
benchmarking FFI/C binding
mean: 107.3837 us, lb 107.2013 us, ub 107.5862 us, ci 0.950
std dev: 983.6046 ns, lb 822.6750 ns, ub 1.292724 us, ci 0.950
benchmarking FFI/C binding
mean: 108.1152 us, lb 107.9457 us, ub 108.3052 us, ci 0.950
std dev: 916.2469 ns, lb 793.1004 ns, ub 1.122127 us, ci 0.950
I started experimenting with the vector size and after bumping its
size to 32K elements I started
getting this:
benchmarking FFI/C binding
mean: 38.50100 us, lb 36.71525 us, ub 46.87665 us, ci 0.950
std dev: 16.93131 us, lb 1.033678 us, ub 40.23900 us, ci 0.950
found 6 outliers among 100 samples (6.0%)
3 (3.0%) low mild
3 (3.0%) high severe
variance introduced by outliers: 98.921%
variance is severely inflated by outliers
benchmarking FFI/C binding
mean: 209.9733 us, lb 209.5316 us, ub 210.4680 us, ci 0.950
std dev: 2.401398 us, lb 2.052981 us, ub 2.889688 us, ci 0.950
First result is always about 39us (2,5 faster, despite longer signal!)
while the remaining
benchmarks take almost two times longer.
Janek
Dnia niedziela, 25 listopada 2012, Janek S. napisał:
Well, it seems that this only happens on my machine. I will try to
test this code on different computer and see if I can reproduce it.
I don't think using existing vector is a good idea - it would make
the
code
impure.
Janek
Dnia sobota, 24 listopada 2012, Branimir Maksimovic napisał:
I don't see such behavior neither.ubuntu 12.10, ghc 7.4.2.
Perhaps this has to do with how malloc allocates /cachebehavior.
If you try not to allocate array rather use existing one perhaps
there would
be
no inconsistency?It looks to me that's about CPU cache
performance. Branimir
I'm using GHC 7.4.2 on x86_64 openSUSE Linux, kernel 2.6.37.6.
Janek
Dnia piątek, 23 listopada 2012, Edward Z. Yang napisał:
Running the sample code on GHC 7.4.2, I don't see the one
fast, rest slow behavior. What version of GHC are you
running?
Edward
Excerpts from Janek S.'s message of Fri Nov 23 13:42:03 -0500
2012:
What happens if you do the benchmark without
unsafePerformIO involved?
I removed unsafePerformIO, changed copy to have type Vector
Double
- IO (Vector Double) and modified benchmarks like this:
bench C binding $ whnfIO (copy signal)
I see no difference - one benchmark runs fast, remaining
ones run slow.
Janek
Excerpts from Janek S.'s message of Fri Nov 23 10:44:15
-0500
2012:
I am using Criterion library to benchmark C code called
via
FFI
bindings and I've ran into a problem that looks like a
bug.
The first benchmark that uses FFI runs correctly, but
subsequent benchmarks run much longer. I created demo
code (about 50 lines, available at github:
https://gist.github.com/4135698 ) in which C function
copies a
vector of doubles. I benchmark that function a couple of
times.
First run results in avarage time of about 17us,
subsequent runs take about 45us. In my real code
additional time was
about
15us and it seemed to be a constant factor, not relative
to correct run time. The surprising thing is that if
my C function only allocates memory and does no copying:
double* c_copy( double* inArr, int arrLen ) {
double* outArr = malloc( arrLen * sizeof( double ) );
return outArr;
}
then all is well - all runs take similar amount of time.
Re: [Haskell-cafe] Can a GC delay TCP connection formation?
Kazu and Andreas, could this be IO manager related? On Monday, November 26, 2012, Jeff Shaw wrote: Hello, I've run into an issue that makes me think that when the GHC GC runs while a Snap or Warp HTTP server is serving connections, the GC prevents or delays TCP connections from forming. My application requires that TCP connections form within a few tens of milliseconds. I'm wondering if anyone else has run into this issue, and if there are some GC flags that could help. I've tried a few, such as -H and -c, and haven't found anything to help. I'm using GHC 7.4.1. Thanks, Jeff __**_ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/**mailman/listinfo/haskell-cafehttp://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Conduit and pipelined protocol processing using a threadpool
I think the stm-conduit package[1] may be helpful for this use case. Each time you get a new command, you can fork a thread and give it the TBMChan to write to, and you can use sourceTBMChan to get a source to send to the client. Michael [1] http://hackage.haskell.org/package/stm-conduit On Tue, Nov 27, 2012 at 12:57 PM, Nicolas Trangez nico...@incubaid.comwrote: All, I've written a library to implement servers for some protocol using Conduit (I'll announce more details later). The protocol supports pipelining, i.e. a client can send a 'command' which contains some opaque 'handle' chosen by the client, the server processes this command, then returns some reply which contains this handle. The client is free to send other commands before receiving a reply for any previous request, and the server can process these commands in any order, sequential or concurrently. The library is based on network-conduit's Application style [1], as such now I write code like (OTOH) application :: AppData IO - IO () application client = appSource client $= handler $$ appSink client where handler = do negotiateResult - MyLib.negotiate liftIO $ validateNegotiateResult negotiateResult MyLib.sendInformation 123 loop loop = do command - MyLib.getCommand case command of CommandA handle arg - do result - liftIO $ doComplexProcessingA arg MyLib.sendReply handle result loop Disconnect - return () This approach handles commands in-order, sequentially. Since command processing can involve quite some IO operations to disk or network, I've been trying to support pipelining on the server-side, but as of now I was unable to get things working. The idea would be to have a pool of worker threads, which receive work items from some channel, then return any result on some other channel, which should then be returned to the client. This means inside loop I would have 2 sources: commands coming from the client (using 'MyLib.getCommand :: MonadIO m = Pipe ByteString ByteString o u m Command'), as well as command results coming from the worker threads through the result channel. Whenever the first source produces something, it should be pushed onto the work queue, and whenever the second on yields some result it should be sent to the client using 'MyLib.sendReply :: Monad m = Handle - Result - Pipe l i ByteString u m ()' I've been fighting this for a while and haven't managed to get something sensible working. Maybe the design of my library is flawed, or maybe I'm approaching the problem incorrectly, or ... Has this ever been done before, or would anyone have some pointers how to tackle this? Thanks, Nicolas [1] http://hackage.haskell.org/packages/archive/network-conduit/0.6.1.1/doc/html/Data-Conduit-Network.html#g:2 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Observer pattern in haskell FRP
Nathan Hüsken wrote: Hey, When writing games in other (imperative) languages, I like to separate the game logic from the rendering. For this I use something similar to the observer pattern. With rendering I mean anything only related to how objects are drawn to the screen. Animation state for example. On my journey of exploring game programming with haskell (and FRP), I wonder what a good way of archiving something similar would be. [..] So I am wondering: Is there (or can someone think of) a different pattern by which this could be archived? Or asked different: How would you do it? Personally, I would recommend is a complete change in perspective. The main idea of FRP is that it is a method to describe the evolution of values in time. What is a game? It's just a picture that evolves in time. The user can exert influence on the evolution by clicking certain buttons on a mechanical device, but in the end, all he sees is a picture that moves. How to describe picture that moves? Your large picture is probably made from smaller pictures, for instance a small picture in the shape of something we often call a spaceship. So, you can implement a game by describing the evolution of smaller pictures, and then combine these into the description of a larger picture. Now, the smaller pictures tend to have hidden state, which means that their future evolution depends a lot on the past evolution of the other small pictures. In my experience with programming in FRP, it is very useful to describe the individual pictures in terms of tiny state machines and then connect these state machines via appropriate events and behaviors to each other. The essence here is to decouple the individual state machines from each other as much as possible and only then to use the FRP abstractions to connect and combine them into a large emergent state machine. (However, it is important to keep in mind that the fundamental abstraction is not a state machine, but a time evolution that remembers the past. This helps with embracing the new perspective and not accidentally fall back to previous ways of thinking. Whether that ends up with good code is up to you to find out, but if you decide to apply a new perspective, it's best to do it in an extremist way to gain the maximum benefit -- this benefit might certainly turn out to be zero, but you will never find out if you wet your feet only a little bit.) Best regards, Heinrich Apfelmus -- http://apfelmus.nfshost.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Can a GC delay TCP connection formation?
Could you give us more info on what your constraints are? Is it necessary that you have a certain number of connections per second, or is it necessary that the connection results very quickly after some other message is received? -- Původní zpráva -- Od: Johan Tibell johan.tib...@gmail.com Datum: 27. 11. 2012 Předmět: Re: [Haskell-cafe] Can a GC delay TCP connection formation? Kazu and Andreas, could this be IO manager related? On Monday, November 26, 2012, Jeff Shaw wrote: Hello, I've run into an issue that makes me think that when the GHC GC runs while a Snap or Warp HTTP server is serving connections, the GC prevents or delays TCP connections from forming. My application requires that TCP connections form within a few tens of milliseconds. I'm wondering if anyone else has run into this issue, and if there are some GC flags that could help. I've tried a few, such as -H and -c, and haven't found anything to help. I'm using GHC 7.4.1. Thanks, Jeff ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe (http://www.haskell.org/mailman/listinfo/haskell-cafe) ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Observer pattern in haskell FRP
On 11/27/2012 04:18 PM, Heinrich Apfelmus wrote: Nathan Hüsken wrote: Hey, When writing games in other (imperative) languages, I like to separate the game logic from the rendering. For this I use something similar to the observer pattern. With rendering I mean anything only related to how objects are drawn to the screen. Animation state for example. On my journey of exploring game programming with haskell (and FRP), I wonder what a good way of archiving something similar would be. [..] So I am wondering: Is there (or can someone think of) a different pattern by which this could be archived? Or asked different: How would you do it? Personally, I would recommend is a complete change in perspective. The main idea of FRP is that it is a method to describe the evolution of values in time. What is a game? It's just a picture that evolves in time. The user can exert influence on the evolution by clicking certain buttons on a mechanical device, but in the end, all he sees is a picture that moves. How to describe picture that moves? Your large picture is probably made from smaller pictures, for instance a small picture in the shape of something we often call a spaceship. So, you can implement a game by describing the evolution of smaller pictures, and then combine these into the description of a larger picture. Now, the smaller pictures tend to have hidden state, which means that their future evolution depends a lot on the past evolution of the other small pictures. In my experience with programming in FRP, it is very useful to describe the individual pictures in terms of tiny state machines and then connect these state machines via appropriate events and behaviors to each other. The essence here is to decouple the individual state machines from each other as much as possible and only then to use the FRP abstractions to connect and combine them into a large emergent state machine. That perspective certainly make sense. But couldn't one also describe a game as a set of entities (spaceships) that react to the clicking of buttons? If I take for example the breakout game from here [1]. It outputs an object scene of type Picture. But this picture is calculated from the objects ballPos and paddlePos. So first a game state (ballPos, paddlePos) is created and than transformed to something renderable. I believe all examples I have seen for games with FRP follow this pattern, and I would I want to do is seperate the steps of calculating the game state and calculating the renderable from it. (However, it is important to keep in mind that the fundamental abstraction is not a state machine, but a time evolution that remembers the past. This helps with embracing the new perspective and not accidentally fall back to previous ways of thinking. Whether that ends up with good code is up to you to find out, but if you decide to apply a new perspective, it's best to do it in an extremist way to gain the maximum benefit -- this benefit might certainly turn out to be zero, but you will never find out if you wet your feet only a little bit.) That certainly makes sense, and it is also very difficult for me to stick to the FRP perspective. But I do not see that seperating rendering and game logic code goes against the FRP perspective. Best Regards, Nathan [1] https://github.com/bernstein/breakout/blob/master/src/Main.hs ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Conduit and pipelined protocol processing using a threadpool
Michael,
On Tue, 2012-11-27 at 17:14 +0200, Michael Snoyman wrote:
I think the stm-conduit package[1] may be helpful for this use case.
Each time you get a new command, you can fork a thread and give it the
TBMChan to write to, and you can use sourceTBMChan to get a source to
send to the client.
That's +- what I had in mind. I did find stm-conduit before and did try
to get the thing working using it, but these attempts failed.
I attached an example which might clarify what I intend to do. I'm aware
it contains several potential bugs (leaking threads etc), but that's
beside the question ;-)
If only I could figure out what to put on the 3 lines of comment I left
in there...
Thanks for your help,
Nicolas
{-# LANGUAGE Rank2Types #-}
module Main where
import Data.Conduit
import qualified Data.Conduit.List as CL
import Data.Conduit.TMChan
import Control.Applicative
import Control.Concurrent (forkIO)
import Control.Concurrent.STM (atomically)
import Control.Monad (forM_)
import Control.Monad.IO.Class (MonadIO, liftIO)
data Command = Add Int Int
| Disconnect
deriving (Show)
data Reply = Result Int
deriving (Show)
application :: MonadIO m = GConduit Int m String
application = do
-- Create input and output channels to/from worker threads
(chanIn, chanOut) - liftIO $ (,) $ newTBMChanIO 10 * newTBMChanIO 10
-- Spawn some worker threads
liftIO $ forM_ [0..5] $ \i - forkIO $ processCommands i chanIn chanOut
-- How to make
-- sourceTBMChan chanOut
-- something of which all produced values are yield'ed by this Conduit?
loop chanIn
where
-- Loop retrieves one command from our source and pushes it to the
-- worker threads input channel, then loops
loop :: MonadIO m = TBMChan Command - GConduit Int m String
loop chan = do
liftIO $ putStrLn Enter loop
cmd - getCommand
liftIO $ do
putStrLn $ Got command: ++ show cmd
atomically $ writeTBMChan chan cmd
case cmd of
Disconnect - return ()
_ - loop chan
-- getCommand fetches and parses a single command from our source
getCommand :: Monad m = GSink Int m Command
getCommand = do
v - await
case v of
Nothing - return Disconnect
Just i - return $ Add i 1
-- processCommands reads commands from a given input channel, processes
-- them, and pushes the result to a given output channel
processCommands :: Int - TBMChan Command - TBMChan Reply - IO ()
processCommands i chanIn chanOut = do
putStrLn $ Enter processCommands ++ show i
cmd - atomically $ readTBMChan chanIn
putStrLn $ show i ++ read command: ++ show cmd
case cmd of
Nothing - return ()
Just (Add a b) - do
atomically $ writeTBMChan chanOut (Result (a + b))
putStrLn $ show i ++ pushed result
processCommands i chanIn chanOut
Just Disconnect - return ()
main :: IO ()
main = do
res - CL.sourceList [1..20] $= application $$ CL.consume
putStrLn $ Result: ++ show res
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Portability of Safe Haskell packages
FWIW, some very core libraries do this: http://hackage.haskell.org/packages/archive/bytestring/0.10.2.0/doc/html/src/Data-ByteString.html (see very top of linked source file) Perhaps a more general solution would be for GHC to take the internet explorer route and require a special javascript include in each source file to get compatibility :) -A On 11/23/2012 03:34 PM, Roman Cheplyaka wrote: * Herbert Valerio Riedel h...@gnu.org [2012-11-24 00:06:44+0100] Roman Cheplyaka r...@ro-che.info writes: It has been pointed out before that in order for Safe Haskell to be useful, libraries (especially core libraries) should be annotated properly with Safe Haskell LANGUAGE pragmas. However, that would make these libraries unusable with alternative Haskell implementations, even if otherwise they these libraries are Haskell2010. To quote the standard: If a Haskell implementation does not recognize or support a particular language feature that a source file requests (or cannot support the combination of language features requested), any attempt to compile or otherwise use that file with that Haskell implementation must fail with an error. Should it be advised to surround safe annotations with CPP #ifs? Or does anyone see a better way out of this contradiction? ...but IIRC CPP isn't part of Haskell2010, or is it? It isn't indeed. But: 1) it's a very basic extension which is supported by (almost?) all existing implementations; or 2) if you want to be 100% Haskell2010, you can name your file *.cpphs and let Cabal do preprocessing. 1) is a compromise and 2) is not very practical, so I'm eager to hear other alternatives. Roman ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] ANNOUNCE: language-java 0.2.0
Hi Cafe, with the approval of Niklas, the original author and maintainer, i'll be maintaining language-java for now. I've uploaded a new version on hackage [1] with some minor improvements and the repository is now hosted on github [2]. Thanks Niklas for language-java ! [1] http://hackage.haskell.org/package/language-java [2] http://github.com/vincenthz/language-java/ -- Vincent ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANNOUNCE: language-java 0.2.0
Thanks for the effort! Now, what about some documentation? :P Cheers, A. On 27 November 2012 18:26, Vincent Hanquez t...@snarc.org wrote: Hi Cafe, with the approval of Niklas, the original author and maintainer, i'll be maintaining language-java for now. I've uploaded a new version on hackage [1] with some minor improvements and the repository is now hosted on github [2]. Thanks Niklas for language-java ! [1] http://hackage.haskell.org/**package/language-javahttp://hackage.haskell.org/package/language-java [2] http://github.com/vincenthz/**language-java/http://github.com/vincenthz/language-java/ -- Vincent __**_ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/**mailman/listinfo/haskell-cafehttp://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Can a GC delay TCP connection formation?
Hello Timothy and others, One of my clients hosts their HTTP clients in an Amazon cloud, so even when they turn on persistent HTTP connections, they use many connections. Usually they only end up sending one HTTP request per TCP connection. My specific problem is that they want a response in 120 ms or so, and at times they are unable to complete a TCP connection in that amount of time. I'm looking at on the order of 100 TCP connections per second, and on the order of 1000 HTTP requests per second (other clients do benefit from persistent HTTP connections). Once each minute, a thread of my program updates a global state, stored in an IORef, and updated with atomicModifyIORef', based on query results via HDBC-obdc. The query results are strict, and atomicModifyIORef' should receive the updated state already evaluated. I reduced the amount of time that query took from tens of seconds to just a couple, and for some reason that reduced the proportion of TCP timeouts drastically. The approximate before and after TCP timeout proportions are 15% and 5%. I'm not sure why this reduction in timeouts resulted from the query time improving, but this discovery has me on the task of removing all database code from the main program and into a cron job. My best guess is that HDBC-odbc somehow disrupts other communications while it waits for the DB server to respond. To respond to Ertugrul, I'm compiling with -threaded, and running with +RTS -N. I hope this helps describe my problem. I c an probably come up with some hard information if requested, E.G. threadscope. Jeff On 11/27/2012 10:55 AM, timothyho...@seznam.cz wrote: Could you give us more info on what your constraints are? Is it necessary that you have a certain number of connections per second, or is it necessary that the connection results very quickly after some other message is received? ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Can a GC delay TCP connection formation?
On Tue, Nov 27, 2012 at 11:02 AM, Jeff Shaw shawj...@gmail.com wrote: Hello Timothy and others, One of my clients hosts their HTTP clients in an Amazon cloud, so even when they turn on persistent HTTP connections, they use many connections. Usually they only end up sending one HTTP request per TCP connection. My specific problem is that they want a response in 120 ms or so, and at times they are unable to complete a TCP connection in that amount of time. I'm looking at on the order of 100 TCP connections per second, and on the order of 1000 HTTP requests per second (other clients do benefit from persistent HTTP connections). Once each minute, a thread of my program updates a global state, stored in an IORef, and updated with atomicModifyIORef', based on query results via HDBC-obdc. The query results are strict, and atomicModifyIORef' should receive the updated state already evaluated. I reduced the amount of time that query took from tens of seconds to just a couple, and for some reason that reduced the proportion of TCP timeouts drastically. The approximate before and after TCP timeout proportions are 15% and 5%. I'm not sure why this reduction in timeouts resulted from the query time improving, but this discovery has me on the task of removing all database code from the main program and into a cron job. My best guess is that HDBC-odbc somehow disrupts other communications while it waits for the DB server to respond. Have you read section 8.4.2 of the ghc user guide? http://www.haskell.org/ghc/docs/7.4.1/html/users_guide/ffi-ghc.html Based on that I would check the FFI imports in your database library. In the best case (-threaded, 'safe', and thread-safe odbc), I think you'll find that N of these can run concurrently, but here your number of requests is likely to be much greater than N (where N is the number of threads the RTS created with +RTS -N). I'm not sure how to solve your problem, but perhaps this information can help you pinpoint the problem. Good luck, Jason ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Can a GC delay TCP connection formation?
On Tue, Nov 27, 2012 at 11:17 AM, Jason Dagit dag...@gmail.com wrote: On Tue, Nov 27, 2012 at 11:02 AM, Jeff Shaw shawj...@gmail.com wrote: Hello Timothy and others, One of my clients hosts their HTTP clients in an Amazon cloud, so even when they turn on persistent HTTP connections, they use many connections. Usually they only end up sending one HTTP request per TCP connection. My specific problem is that they want a response in 120 ms or so, and at times they are unable to complete a TCP connection in that amount of time. I'm looking at on the order of 100 TCP connections per second, and on the order of 1000 HTTP requests per second (other clients do benefit from persistent HTTP connections). Once each minute, a thread of my program updates a global state, stored in an IORef, and updated with atomicModifyIORef', based on query results via HDBC-obdc. The query results are strict, and atomicModifyIORef' should receive the updated state already evaluated. I reduced the amount of time that query took from tens of seconds to just a couple, and for some reason that reduced the proportion of TCP timeouts drastically. The approximate before and after TCP timeout proportions are 15% and 5%. I'm not sure why this reduction in timeouts resulted from the query time improving, but this discovery has me on the task of removing all database code from the main program and into a cron job. My best guess is that HDBC-odbc somehow disrupts other communications while it waits for the DB server to respond. Have you read section 8.4.2 of the ghc user guide? http://www.haskell.org/ghc/docs/7.4.1/html/users_guide/ffi-ghc.html Ahem, I meant *8.2.4*. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Real-time code in Haskell (Was: Can a GC delay TCP connection formation?)
On Tue, Nov 27, 2012 at 3:45 AM, Gregory Collins g...@gregorycollins.net wrote: If you have a hard real-time requirement then a garbage-collected language may not be appropriate for you. This is a common meme, but frankly, it isn't true. When writing real-time code, you just need to make sure that everything that happens takes a known maximum amount of time. Then, you can sum up the maximums and verify that you do indeed finish in the real-time window of the task. GC is a problem because it's not predictable, and may not have a maximum. However, it's no worse than a modern version of the C function malloc. Some of those even do garbage collection internally before doing an OS call if they're out of memory. The solution is the same in both cases - make sure you don't do GC (or call malloc) in the critical region. Both require knowing implementation details of everything you call, but it isn't impossible, or even particularly difficult. Lazyness, on the other hand ... I haven't thought about. I suspect you need to force the evaluation of everything you're going to need before you start the critical region, but I wonder if that's enough? Has anyone out there investigated this? Thanks, mike ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Can a GC delay TCP connection formation?
On 11/27/12 2:17 PM, Jason Dagit wrote: Based on that I would check the FFI imports in your database library. In the best case (-threaded, 'safe', and thread-safe odbc), I think you'll find that N of these can run concurrently, but here your number of requests is likely to be much greater than N (where N is the number of threads the RTS created with +RTS -N). HDBC-odbc has long used the wrong type of FFI imports, resulting in long-running database queries potentially blocking all other IO. I just checked, and apparently a patch was made to the repo in September that finally fixes this [1], but apparently a new release has yet to be uploaded to hackage. In any case, if you try to install it from the repo, this may at least solve some of your problems. [1] https://github.com/hdbc/hdbc-odbc/commit/7299d3441ce2e1d5a485fe79b37540c0a44a44d4 --Gershom ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Real-time code in Haskell (Was: Can a GC delay TCP connection formation?)
What triggers GC in haskell? We obviously aren't using Java's method of GC as needed(for good reasons, Java's method is terrible because you get slow downs when you need speed the most). But we should be able to learn something from Java and have a gc::IO() method that one could call BEFORE a critical region of code... -- Původní zpráva -- Od: Mike Meyer m...@mired.org Datum: 27. 11. 2012 Předmět: [Haskell-cafe] Real-time code in Haskell (Was: Can a GC delay TCP connection formation?) On Tue, Nov 27, 2012 at 3:45 AM, Gregory Collins g...@gregorycollins.net wrote: If you have a hard real-time requirement then a garbage-collected language may not be appropriate for you. This is a common meme, but frankly, it isn't true. When writing real-time code, you just need to make sure that everything that happens takes a known maximum amount of time. Then, you can sum up the maximums and verify that you do indeed finish in the real-time window of the task. GC is a problem because it's not predictable, and may not have a maximum. However, it's no worse than a modern version of the C function malloc. Some of those even do garbage collection internally before doing an OS call if they're out of memory. The solution is the same in both cases - make sure you don't do GC (or call malloc) in the critical region. Both require knowing implementation details of everything you call, but it isn't impossible, or even particularly difficult. Lazyness, on the other hand ... I haven't thought about. I suspect you need to force the evaluation of everything you're going to need before you start the critical region, but I wonder if that's enough? Has anyone out there investigated this? Thanks, mike ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe (http://www.haskell.org/mailman/listinfo/haskell-cafe)___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANNOUNCE: language-java 0.2.0
On 11/27/2012 06:46 PM, Alfredo Di Napoli wrote: Thanks for the effort! Now, what about some documentation? :P Sure ! Fork away, and send pull requests :-) -- Vincent ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Real-time code in Haskell (Was: Can a GC delay TCP connection formation?)
http://hackage.haskell.org/packages/archive/base/4.6.0.0/doc/html/System-Mem.html#v:performGC On Tue, Nov 27, 2012 at 5:52 PM, timothyho...@seznam.cz wrote: What triggers GC in haskell? We obviously aren't using Java's method of GC as needed(for good reasons, Java's method is terrible because you get slow downs when you need speed the most). But we should be able to learn something from Java and have a gc::IO() method that one could call BEFORE a critical region of code... -- Původní zpráva -- Od: Mike Meyer m...@mired.org Datum: 27. 11. 2012 Předmět: [Haskell-cafe] Real-time code in Haskell (Was: Can a GC delay TCP connection formation?) On Tue, Nov 27, 2012 at 3:45 AM, Gregory Collins g...@gregorycollins.net wrote: If you have a hard real-time requirement then a garbage-collected language may not be appropriate for you. This is a common meme, but frankly, it isn't true. When writing real-time code, you just need to make sure that everything that happens takes a known maximum amount of time. Then, you can sum up the maximums and verify that you do indeed finish in the real-time window of the task. GC is a problem because it's not predictable, and may not have a maximum. However, it's no worse than a modern version of the C function malloc. Some of those even do garbage collection internally before doing an OS call if they're out of memory. The solution is the same in both cases - make sure you don't do GC (or call malloc) in the critical region. Both require knowing implementation details of everything you call, but it isn't impossible, or even particularly difficult. Lazyness, on the other hand ... I haven't thought about. I suspect you need to force the evaluation of everything you're going to need before you start the critical region, but I wonder if that's enough? Has anyone out there investigated this? Thanks, mike ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe -- Felipe. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Real-time code in Haskell (Was: Can a GC delay TCP connection formation?)
* Mike Meyer m...@mired.org [2012-11-27 13:40:17-0600] Lazyness, on the other hand ... I haven't thought about. I suspect you need to force the evaluation of everything you're going to need before you start the critical region, but I wonder if that's enough? Has anyone out there investigated this? I don't know much about RT systems, but if GC is not a problem, then laziness should be even less so. It is much more predictable, especially when you look at the Core. Roman ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANNOUNCE: language-java 0.2.0
Great, thanks for this great work! One of the things I _love_ about haskell and it's community is that language-x packages. I really love playing with source code and compiling to other languages. And working with Haskell, thanks to that language-x packages, make this a joy. And recently I also wrote one, language-lua: http://hackage.haskell.org/package/language-lua . Maybe I should write an announcement mail too :-) . --- Ömer Sinan Ağacan 2012/11/27 Vincent Hanquez t...@snarc.org On 11/27/2012 06:46 PM, Alfredo Di Napoli wrote: Thanks for the effort! Now, what about some documentation? :P Sure ! Fork away, and send pull requests :-) -- Vincent __**_ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/**mailman/listinfo/haskell-cafehttp://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Can a GC delay TCP connection formation?
On 11/27/2012 2:45 PM, Gershom Bazerman wrote: HDBC-odbc has long used the wrong type of FFI imports, resulting in long-running database queries potentially blocking all other IO. I just checked, and apparently a patch was made to the repo in September that finally fixes this [1], but apparently a new release has yet to be uploaded to hackage. In any case, if you try to install it from the repo, this may at least solve some of your problems. [1] https://github.com/hdbc/hdbc-odbc/commit/7299d3441ce2e1d5a485fe79b37540c0a44a44d4 --Gershom Gershom, Thanks for pointing this out. I've checked out the latest hdbc-odbc code, and I'll see if there's an improvement. Jeff ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Can a GC delay TCP connection formation?
On Tue, Nov 27, 2012 at 9:45 PM, Jeff Shaw shawj...@gmail.com wrote: On 11/27/2012 2:45 PM, Gershom Bazerman wrote: HDBC-odbc has long used the wrong type of FFI imports, resulting in long-running database queries potentially blocking all other IO. I just checked, and apparently a patch was made to the repo in September that finally fixes this [1], but apparently a new release has yet to be uploaded to hackage. In any case, if you try to install it from the repo, this may at least solve some of your problems. [1] https://github.com/hdbc/hdbc-odbc/commit/7299d3441ce2e1d5a485fe79b37540c0a44a44d4 --Gershom Gershom, Thanks for pointing this out. I've checked out the latest hdbc-odbc code, and I'll see if there's an improvement. Hi, I'm the maintainer of HDBC. I haven't yet released this code since it hasn't yet been fully tested. However, if you're happy with it, I'll push the version with proper ffi bindings up to Hackage. Nick ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] cabal configure cabal build cabal install
On 12-11-27 04:40 AM, kudah wrote: On Tue, 27 Nov 2012 02:20:35 -0500 Albert Y. C. Lai tre...@vex.net wrote: When cabal build succeeds, it always says: (older) registering name-version (newer) In-place registering name-version That's what it says. But use ghc-pkg and other tests to verify that no registration whatsoever has happened. It doesn't register in user package-db, it registers in it's own dist/package.conf.inplace. If it didn't you wouldn't be able to build an executable and a library in one package such that executable depends on the library. That's fair. But it also means cabal configure cabal build is not equivalent to cabal configure cabal build cabal register --inplace which was the context when you said (newer) cabal build registers inplace automatically. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Recursive timezone-loading function
https://github.com/dlthomas/tzcache A small bit of code, but seems likely to be useful enough that I figured I should share. I've a few notes/questions: 1) Does this already exist somewhere I missed? 2) It seems silly to make this its own library - any suggestions where it could be added? 3) Is the traverse-a-directory-and-populate-a-map pattern one worth abstracting? If so, where should that go? 4) Presently, it's a static cache entirely pre-loaded. This seems fine, as it's not a terribly huge amount of data, but it's worth noting. 5) Any comments on the code generally? Improvements? Complaints? Thanks, - David ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Conduit and pipelined protocol processing using a threadpool
On Tue, Nov 27, 2012 at 7:25 PM, Nicolas Trangez nico...@incubaid.comwrote: Michael, On Tue, 2012-11-27 at 17:14 +0200, Michael Snoyman wrote: I think the stm-conduit package[1] may be helpful for this use case. Each time you get a new command, you can fork a thread and give it the TBMChan to write to, and you can use sourceTBMChan to get a source to send to the client. That's +- what I had in mind. I did find stm-conduit before and did try to get the thing working using it, but these attempts failed. I attached an example which might clarify what I intend to do. I'm aware it contains several potential bugs (leaking threads etc), but that's beside the question ;-) If only I could figure out what to put on the 3 lines of comment I left in there... Thanks for your help, Nicolas The issue is that you're trying to put everything into a single Conduit, which forces reading and writing to occur in a single thread of execution. Since you want your writing to be triggered by a separate event (data being available on the Chan), you're running into limitations. The reason network-conduit provides a Source for the incoming data and a Sink for outgoing data is specifically to address your use case. You want to take the data from the Source and put it into the Chan in one thread, and take the data from the other Chan and put it into the Sink in a separate thread. Something like: myApp appdata = do chan1 - ... chan2 - ... replicateM_ 5 $ forkIO $ worker chan1 chan2 forkIO $ appSource appdata $$ sinkTBMChan chan1 sourceTBMChan chan2 $$ appSink appdata You'll also want to make sure to close chan1 and chan2 to make sure that your threads stop running. Michael ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
