I had the same problem (stack overflow). The solution was to change the >>= operator in the Get monad. Currently it is:
m >>= k = Get (\s -> let (a, s') = unGet m s in unGet (k a) s') but I changed it to: m >>= k = Get (\s -> case unGet m s of (a, s') -> unGet (k a) s') It seems that the bind operator is lazy and this caused the stack overflow. I have also another problem. Every Int and Word is stored as 64-bit value and this expands the output file a lot. I have a lot of integers and most of them are < 128 but not all of them. I changed the serialization so that the Int and Word are serialized in a variable number of bytes. Without this change the binary serialization was even worse than the textual serialization that we had before. The file was almost full with zeros. I just haven't time to prepare a patch and to send it for review but if other people have the same problem I will do it. Best Regars, Krasimir On Wed, Aug 13, 2008 at 1:13 AM, Tim Newsham <[EMAIL PROTECTED]> wrote: > I have a program that read in and populated a large data structure and > then saved it out with Data.Binary and Data.ByteString.Lazy.Char8: > > saveState db = B.writeFile stateFile =<< > encode <$> atomically (readTVar db) > > when I go to read this in later I get a stack overflow: > > loadState db = do > d <- decode <$> B.readFile stateFile > atomically $ writeTVar db d > > Stack space overflow: current size 8388608 bytes. > Use `+RTS -Ksize' to increase it. > > or from ghci: > > d <- liftM decode > (Data.ByteString.Lazy.Char8.readFile > "savedState.bin") :: IO InstrsDb > > fromList *** Exception: stack overflow > > The data type I'm storing is a Map (of maps): > > type DailyDb = M.Map Date Daily > type InstrsDb = M.Map String DailyDb > > What's going on here? Why is the system capable of building and saving > the data but not in reading and umarhsalling it? What is the proper way > to track down where the exception is happening? Any debugging tips? > > I also noticed another issue while testing. If my program loads > the data at startup by calling loadState then all later calls to > saveState give an error: > > Log: savedState.bin: openFile: resource busy (file is locked) > > this does not occur if the program wasnt loaded. My best guess here > is that B.readFile isnt completing and closing the file for some > reason. Is there a good way to force this? > > Tim Newsham > http://www.thenewsh.com/~newsham/ > _______________________________________________ > Haskell-Cafe mailing list > Haskell-Cafe@haskell.org > http://www.haskell.org/mailman/listinfo/haskell-cafe > _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe