What's a good minimal example of this resumption problem? Couldn't a functional approach be that running a pipe till it returns a value also returns a continuation? In the style of attoparsec and friends?
On Thursday, January 30, 2014, Jeremy Shaw <[email protected]> wrote: > Yes -- io-stream forces everything to be done in the IO monad and uses > hidden IORefs. conduit also uses hidden IORefs for resumable streams. > > But is that really the best choice? > > - jeremy > > On Thu, Jan 30, 2014 at 2:12 PM, Carter Schonwald > <[email protected]> wrote: > > I think this precise issue is why the snap server http parser tooling > uses > > the iostreams lib! > > > > > > On Thu, Jan 30, 2014 at 1:02 PM, Jeremy Shaw <[email protected]> > wrote: > >> > >> I have been thinking about what it means to run a 'Producer' > >> twice. Specifically -- whether the Producer resumes where it left of > >> or not. I think that in general the behavior is undefined. I feel like > >> this has not been explicitly stated much -- so I am going to say it > >> now. In some sense, it should be obvious -- but when peering through > >> the haze of Pipes, StateT, and IO, the simple things can get lost. > >> > >> Consider two different cases: > >> > >> 1. a producer that produces values from a pure list > >> > >> 2. a producer that produces values from a network connection > >> > >> > >> If we run the first producer twice we will get the same answer each > >> time. If we run the second producer twice -- we will likely get > >> different results -- depending on what data is available from the > >> network stream. > >> > >> Now -- that is not entirely surprising -- one value is pure and one is > >> based on IO. So that is no different than calling a normal pure > >> function versus a normal IO function. > >> > >> But -- I think it can be easy to forget that when writing pipes > >> code. Imagine we write some pipes code that processes a network stream > >> -- and it relies on the fact that the network Producer automatically > >> resumes from where it left off. > >> > >> Now, let's pretend we want to test our code. So we create a pure > >> Producer that produces the same bytestring that the network pipe was > >> producing. Alas, our code will not work because the pure Producer does > >> not automatically resume when called multiple times. > >> > >> I think this means that we must assume, by default, that the Producer > >> does not have resumable behavior. If we want to write code that relies > >> on the resumable behavior -- then we must explictly ensure that it > >> happens. > >> > >> In pipes-parse the resumability is handled by storing the 'Producer' > >> in 'StateT'. > >> > >> Another alternative is to use an 'IORef'. I have an example of the > >> 'IORef' solution below. > >> > >> > module Main where > >> > >> > import Data.IORef (IORef(..), newIORef, readIORef, > >> > writeIORef) > >> > import Pipes > >> > import qualified Pipes.Prelude as P > >> > >> Here is our pure Producer: > >> > >> > pure10 :: (Monad m) => Producer Int m () > >> > pure10 = mapM_ yield [1..10] > >> > >> And here is a function which uses a Producer twice. > >> > >> > take5_twice :: Show a => Producer a IO () -> IO () > >> > take5_twice p = > >> > do runEffect $ p >-> P.take 5 >-> P.print > >> > putStrLn "<<Intermission>>" > >> > runEffect $ p >-> P.take 5 >-> P.print > >> > >> Note that we have limited ability reason about the results since we do > >> not know if the 'Producer' is resumable or not. > >> > >> If we run 'take5_twice' using our pure Producer: > >> > >> > pure10_test :: IO () > >> > pure10_test = > >> > take5_twice pure10 > >> > >> it will restart from 1 each time: > >> > >> > pure10_test > >> 1 > >> 2 > >> 3 > >> 4 > >> 5 > >> <<Intermission>> > >> 1 > >> 2 > >> 3 > >> 4 > >> 5 > >> > >> Here is a (not very generalized) function that uses an 'IORef' to > >> store the current position in the 'Producer' -- similar to how > >> 'StateT' works: > >> > > -- You received this message because you are subscribed to the Google Groups "Haskell Pipes" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected].
