On 3/13/2013 12:15 AM, Scott Lawrence wrote:
Hey all,
All the object serialization/deserialization libraries I could find
(pretty much just binary and cereal) seem to be strict with respect to
the actual data being serialized. In particular, if I've serialized a
large [Int] to a file, and I want to get the first element, it seems I
have no choice but to deserialize the entire data structure. This is
obviously an issue for large data sets.
There are obvious workarounds (explicitly fetch elements from the
"database" instead of relying on unsafeInterleaveIO to deal with it
all magically), but it seems like it should be possible to build a
cereal-like library that allows proper lazy deserialization. Does it
exist, and I've just missed it?
Thanks,
I haven't tested this, but I suspect something like this could give you
lazy binary serialization and deserialization. It's not tail recursive,
though.
newtype LazyBinaryList a = LazyBinaryList [a]
instance Binary a => LazyBinaryList a where
put (LazyBinaryList []) = putWord8 0
put (LazyBinaryList (x:xs)) = putWord8 1 >> put x >> put
(LazyBinaryList xs)
get = do
t <- getWord8
case t of
0 -> return (LazyBinaryList [])
1 -> do
x <- get
(LazyBinaryList xs) <- get
return $ LazyBinaryList (x:xs)
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
