#7556: build/fold causes with ByteString unpack causes huge memory leak
Reporter:  glguy                    |          Owner:                  
    Type:  bug                      |         Status:  new             
Priority:  normal                   |      Component:  libraries/base  
 Version:  7.6.1                    |       Keywords:                  
      Os:  Unknown/Multiple         |   Architecture:  Unknown/Multiple
 Failure:  Runtime performance bug  |      Blockedby:                  
Blocking:                           |        Related:                  
Changes (by shachaf):

 * cc: shachaf@… (added)


 The issue is
 /Data-ByteString.html#unpackFoldr here]:

 unpack ps = build (unpackFoldr ps)
 {-# INLINE unpack #-}

 -- Have unpack fuse with good list consumers
 -- critical this isn't strict in the acc
 -- as it will break in the presence of list fusion. this is a known
 -- issue with seq and build/foldr rewrite rules, which rely on lazy
 -- demanding to avoid bottoms in the list.
 unpackFoldr :: ByteString -> (Word8 -> a -> a) -> a -> a
 unpackFoldr (PS fp off len) f ch = withPtr fp $ \p -> do
     let loop q n    _   | q `seq` n `seq` False = undefined -- n.b.
         loop _ (-1) acc = return acc
         loop q n    acc = do
            a <- peekByteOff q n
            loop q (n-1) (a `f` acc)
     loop (p `plusPtr` off) (len-1) ch
 {-# INLINE [0] unpackFoldr #-}

 {-# RULES
 "ByteString unpack-list" [1]  forall p  .
     unpackFoldr p (:) [] = unpackBytes p

 When we use `foldr`, `foldr/build` fusion turns the whole expression into
 an application of `unpackFoldr`, which is tail recursive and therefore not
 sufficiently lazy -- but also not strict in the accumulator, so it builds
 up a big thunk. In `example1`, fusion doesn't happen, so the fold happens
 over `unpackBytes` instead, which generates list in small chunks that can
 be processed lazily.

 This looks like a `bytestring` bug to me.

Ticket URL: <http://hackage.haskell.org/trac/ghc/ticket/7556#comment:2>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler

Glasgow-haskell-bugs mailing list

Reply via email to