`streamN` just replicates the`streamN` in pipes-http, which presupposes that
the underlying monad for the byte stream is IO. But `Q.writeFile file` gives
you a stream with the underlying monad `ResourceT IO`.
It should be possible to write a `streamN` that takes a `ByteString
(ResourceT IO) ()`, though. See the duplication of functions in
`http-conduit`
- IO :
http://hackage.haskell.org/package/http-conduit-2.1.10.1/docs/Network-HTTP-Client-Conduit.html#v:requestBodySource
- ResourceT IO :
http://hackage.haskell.org/package/http-conduit-2.1.10.1/docs/Network-HTTP-Client-Conduit.html#v:requestBodySource
The first takes a `Source IO ByteString`, the second a `Source (ResourceT
IO) ByteString`. `streamN` here, replicating pipes-http, is like the first
of these.
It's always possible I'm missing something in the `ResourceT` + `Conduit`
wilderness of mirrors, but we just need an equivalent of `to` from
Pipes.HTTP that is more like something like `srcToPopper` inside
Network.HTTP.Conduit.
streamN' :: Int64 -> Q.ByteString (ResourceT IO) () -> SP.RequestBody
streamN' n str = SP.RequestBodyStream n (popperize str)
-- cp the helper functions `Pipes.HTTP.to` and
`Network.HTTP.srcToPopper`
popperize :: Q.ByteString (ResourceT IO) () -> (IO ByteString -> IO a)
-> IO a
popperize str0 with_reader = runResourceT $ do
ref <- liftIO (newIORef str0)
is <- getInternalState
let reader :: IO ByteString
reader = do
str <- readIORef ref
e <- runInternalState (Q.nextChunk str) is -- Q.nextChunk
str is in ResourceT IO but we
case e of -- here purport
to run it safely in IO
Left r -> do
writeIORef ref (return r)
return mempty -- the apparatus will take this null
chunk as eof, so ...
Right (chunk, rsrc') -> do
writeIORef ref rsrc'
if B.null chunk -- ... here we have to make sure not
to return one.
then reader
else return chunk
liftIO (with_reader reader)
I haven't tested this with anything real, but it works with this trivial
`NeedsPopper`
-- lengthy :: SP.NeedsPopper Int
lengthy :: IO ByteString -> IO Int
lengthy act = loop 0
where
loop n = do
chunk <- act
let len = B.length chunk
if len > 0
then loop (n+len)
else return n
-- >>> popperize (Q.readFile "/usr/share/dict/words") lengthy
-- 2493109
-- >>> runResourceT $ Q.length (Q.readFile "/usr/share/dict/words")
-- 2493109 :> ()
-- >>> :! wc -c /usr/share/dict/words
-- 2493109 /usr/share/dict/words
It would be interesting to know if it can survive the aws machinery.
--
You received this message because you are subscribed to the Google Groups
"Haskell Pipes" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
