Joel Reymont wrote:
You are right in that I spent the first few weeks learning. By now I
know that pickling is the bottleneck, though. The timeleak code is very
simple. It forks X threads where each thread opens a file for reading.
Why on earth do you want each tread to open the file and unpickle?
Why not unpickle once and reuse it?
Or, if this is just a test and in the future they will all read
from different files (or sockets), then maybe you are hitting
on a different bottleneck than you think.
This discussion is a bit of a dead-end unless you are willing to take
the next step and apply your solution to my timeleak repro case. If you
or someone else is willing to do that than I can continue my set of
profiler reports and eventually get some closure. It will happen once
either 1) the last bit of performance is squeezed out of pickling and
it's determined that threading or the scheduler is the bottleneck or 2)
things work out nicely.
Instead of starting optimizing a particular pickling library perhaps you
should have tried different libraries and picked the best one.
Since this is your project I don't think your project I don't think
you can expect others to test things for you. Well, maybe if you post
your code as a challange. ;)
-- Lennart
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
