On 22/03/11 05:33, Daniel Fischer wrote:
On Wednesday 23 March 2011 03:32:16, Tim Docker wrote:
Is the slop number above likely to be a significant contribution to net
memory usage?
Yes, absolutely.
Are there any obvious reasons why the code below could be
generating so much?
I suspect packing a lot of presumably relatively short ByteStrings would
generate (the lion's share of) the slop. I'm not familiar with the
internals, though, so I don't know where GHC would put a
newPinnedByteArray# (which is where your ByteString contents is), what
alignement requirements those have.
Thanks, I'm aware that that the code could be optimised eg by sticking
to bytestrings and avoiding Strings and read - there were just to make
the example simple. I expected this would affect speed, though not
memory usage.
I'm a bit shocked at the amount of wasted memory here. The sample data
file has ~61k key/value pair. Hence ~122k ByteStrings - as you point out
many of these are very small (1500 of them are empty). Assuming it's the
bytestring that are generating slop, I am seeing ~500 bytes on average
per bytestring!
Tim
_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
