On Tue, Dec 04, 2012 at 03:58:24PM +0100, Herbert Valerio Riedel wrote: > Iustin Pop <ius...@google.com> writes: > > [...] > > > Let's say we have a simple JSON message: an array of 5 million numbers. > > I would like to parse this in constant space, such that if I only need > > the last element, overall memory usage is low (yes, unrealistic use, but > > please bear with me for a moment). > > > > Using aeson, I thought the following program will be nicely-behaved: > > part of the problem is that aeson builds an intermediate JSON parse-tree > which has quite an overhead for representing a list of numbers on the > heap as each numbers requires multiple heap objects (see also [1]). This > is an area where e.g. Python has a significantly smaller footprint > (mostly due to a more efficient heap representation).
Ah, I see. Thanks for the link, so that's from where the 'S' constructor was coming from in the -hd output. And indeed, I was surprised as well that Python has a more efficient representation for this. > [...] > > > It seems that the Array constructor holds a vector, and this results in > > too much strictness? > > btw, using a list on the other hand would add an overhead of 2 words > (due to the ':' constructor) for representing each JSON array element in > the parse-tree, that's probably why aeson uses vectors instead of lists. Ack. thanks, iustin _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
