2011/11/15 Roman Cheplyaka <[email protected]>: > * Jason Dusek <[email protected]> [2011-11-15 20:08:48+0000] >> I'm having some trouble with memory usage in rebuilding a >> ByteString with some sequences escaped. I thought I'd try >> vectors. However, it seems that even a relatively simple >> function, one that places all the bytes of a ByteString in to a >> vector, uses a great deal of memory. > > I think what's happening here is ByteString's "strictness" makes things > actually lazy on your side. > > Namely, unpack function produces its result "strictly", whole list at > once. As a result, the resulting list cannot be consumed one-by-one, > so it takes memory. You see ST thunks because > > mapM_ f as = sequence_ (map f as) > > and that map probably gets fused with unpack. > > I guess the proper solution here is to use lazy bytestring and make sure > the chunks are not very big.
Hi Roman, Switching to the lazy ByteStrings API does, indeed, help; total memory usage is around 16M. I will have a look at the rules that are fired to see what I can learn. -- Jason Dusek () ascii ribbon campaign - against html e-mail /\ www.asciiribbon.org - against proprietary attachments _______________________________________________ Haskell-Cafe mailing list [email protected] http://www.haskell.org/mailman/listinfo/haskell-cafe
