Hey Gabriel,
Thanks for your response, but I'm not quite sure how to apply this in my
case? (I don't have any experience with conduit, so I'll ignore those
remarks for now). With your suggested type of:
handShake
:: SockAddr
-> Producer ByteString M r
-> Producer ByteString M ((Get Frame, Frame -> Put), Producer
ByteString M r)
I'm not quite sure how this will fit? The problem is that during handshake
I need to do things like "read/parse N bytes of the ByteString coming from
the socket, then based on that write M bytes to the socket". I'm not quite
sure how to do that with this type? Take for example "zmtpVersion", this
needs to read ByteString's from the input, write output to the socket and
return a Version. Does that mean that, just like handShake, zmtpVersion's
type becomes "Producer ByteString M r -> Producer ByteString M (Version,
Producer ByteString M r)"? How do I get the ByteString's to write to the
socket from there?
zmtpVersion
:: Producer ByteString M r
-> Producer ByteString M (Version, Producer ByteString M r)
zmtpVersion prod = do
??
I can't use any of the pipes-binary parsing utilities here, as they all
producer Parser's and you just said that using a Producer as a base monad
for Parser won't work (which is the only way I could run a Parser and end
up with a Producer like this type)?
I really don't know how to incrementally parse a value from the input
producer here, except: 1) manually thread leftovers everywhere like my
initial design or 2) manually step through the input producer using "next".
Both of these seem really painful and cumbersome, and the annoyance of the
former was the entire reason I was hoping pipes-parse would make things
better. But so far it seems that it can't?
Cheers,
Merijn
On Tuesday, July 29, 2014 1:57:06 AM UTC+2, Gabriel Gonzalez wrote:
> The `Parser` type is intended to be used with a base monad that is not a
> `Producer` or any other pipe type. The intuition you should have for
> `pipes-parse` is that they are almost exactly analogous to `conduit`
> abstractions (minus the finalization handling of `conduit`):
>
> * `Producer a m ()` is analogous to a `Source m a`
> * `Parser a m r` is analogous to a `conduit` `Sink a m r`
> * `Lens' (Producer a m r) (Producer b m r)` is analogous to a `Conduit a m
> b`that can propagate leftovers further upstream
> * `Getter' (Producer a m r) (Producer b m r)` (or just: `Producer a m r ->
> Producer b m r`) is analogous to a `Conduit a m b` that cannot propagate
> leftovers further upstream
>
> `conduit` does not make the distinction between transformations which can
> or cannot propagate leftovers further upstream, which leads to the leftover
> propagation bug described here:
> http://www.haskellforall.com/2014/02/pipes-parse-30-lens-based-parsing.html
>
> So any time you find yourself writing a `Parser` that is trying to `yield`
> things further downstream, that's your cue that you should switch it to
> become a `Lens'` or a `Getter` (i.e. function) instead.
>
> Back to your specific code: I believe the the type you want for
> `handShake` is either a leftover-passing function equivalent to the
> `Parser` you originally had:
>
> type M = ReaderT SockConfig (SafeT IO)
>
> handShake
>
:: SockAddr
> -> Producer ByteString M r
> -> Producer ByteString M ((Get Frame, Frame -> Put), Producer
> ByteString M r)
>
> ... or a lens if you included enough information in the return value to
> recover the original stream:
>
> type M = ReaderT SockConfig (SafeT IO)
>
> handShake
> :: SockAddr
> -> Lens' (Producer ByteString M r)
> (Producer ByteString M ((ExtraInformation, Get Frame,
> Frame -> Put), Producer ByteString M r)
>
> If you did that, then you'd be able to write:
>
> zoom (handShake . decoder) :: Parser Frame m r -> Parser ByteString m
> r
>
> ... and that would automatically handle all initial authentication and
> provide you with a stream of decoded frames. Or you could use it to
> transform a `Producer`:
>
> view (handShake . decoder) :: Producer ByteString M r -> Producer
> Frame M LargeReturnType
>
>
>
> On 7/26/14, 10:55 AM, Merijn Verstraaten wrote:
>
> Ok, so I have a network connection that has an interactive handshake to
> setup a connection (i.e. I need to write things to the network while
> parsing the handshake). Several months ago I wrote some code that manually
> plumbed a ByteString state throughout all my pipes to handle leftovers.
> Today I wanted to try and rewrite this code to use the new pipes-parse
> stuff, thinking the result would be much nicer to work with. However, I get
> stuck with no clue how to continue. My original code was as follows:
>
>
> readSocket :: MonadIO m => Handle -> Producer ByteString m r
> writeSocket :: MonadIO m => Handle -> Consumer ByteString m r
>
> handShake :: SockAddr ->
> StateT ByteString
> (Pipe ByteString
> ByteString
> (ReaderT SockConfig (SafeT IO)))
> ( Pipe ByteString Frame (SafeT IO) r,
> Pipe Frame ByteString (SafeT IO) r )
> handShake addr = do
> version <- zmtpVersion
> let (decoder, encoder) = case version of
> ZMTP1 {} -> (zmtp1Decoder, zmtp1Encoder)
> :: MonadSafe m => (Pipe ByteString Frame m r, Pipe Frame
> ByteString m r)
> ZMTP2 {} -> (zmtp2Decoder, zmtp2Encoder)
> ZMTP3 {} -> (zmtp3Decoder, zmtp3Encoder)
>
> -- authenticate version addr :: Pipe Frame Frame m r
> lift $ decoder >-> authenticate version addr >-> encoder
> return (decoder, encoder)
>
> session :: SockConfig
> -> Producer Frame (SafeT IO) ()
> -> Consumer Frame (SafeT IO) ()
> -> IO Handle
> -> SockAddr
> -> IO ()
> session cfg fromApp toApp newHandle addr =
> runSafeT . runEffect . bracket newHandle hClose $ \hnd -> do
>
> let fromNetwork = readSocket hnd
> toNetwork = writeSocket hnd
> handshake = runReaderP cfg (runStateT (handShake addr) empty)
>
> ((decoder, encoder), rest) <- fromNetwork >-> handshake >->
> toNetwork
> myTid <- liftIO myThreadId
>
> let appToNet = fromApp >-> encoder >-> toNetwork
> netToApp = (yield rest >> fromNetwork) >-> decoder >-> toApp
> -- code using appToNet/netToApp Effect's
>
> In other words, my handshake is just a pipe, reading and writing
> ByteStrings from and to the same socket. After version negotiation I do
> authentication, this uses a "Pipe Frame Frame" that I compose with the
> relevant decoders/encoders and lift into the handshake's pipe. So far, so
> good, the only real nastiness is manually threading the leftover state
> throughout the zmtpVersion, decoder and encoder.
>
> So on to my attempt to rewrite this using pipes-parse.
>
> readSocket :: MonadIO m => Handle -> Producer ByteString m r
> writeSocket :: MonadIO m => Handle -> Consumer ByteString m r
>
> handShake :: SockAddr
> -> Parser ByteString (Producer ByteString (ReaderT SockConfig
> (SafeT IO))) (Get Frame, Frame -> Put)
> handShake _addr = do
> version <- zmtpVersion
> let (getFrame, putFrame) = case version of
> ZMTP1 {} -> (ZMTP1.getFrame, ZMTP1.putFrame)
> ZMTP2 {} -> (ZMTP2.getFrame, ZMTP2.putFrame)
> ZMTP3 {} -> (ZMTP3.getFrame, ZMTP3.putFrame)
>
> --lift $ undefined >-> authenticate version addr >-> undefined
> return (getFrame, putFrame)
>
> session :: SockConfig
> -> Producer Frame (SafeT IO) ()
> -> Consumer Frame (SafeT IO) ()
> -> IO Handle
> -> SockAddr
> -> IO ()
> session cfg fromApp toApp newHandle addr =
> runSafeT . runEffect . bracket newHandle hClose $ \hnd -> do
>
> let fromNetwork = readSocket hnd
> toNetwork = writeSocket hnd
> handshake = runReaderP cfg . runStateT (handShake addr)
>
> ((decoder, encoder), rest) <- handshake fromNetwork >-> toNetwork
> myTid <- liftIO myThreadId
>
> let appToNet = for fromApp (encodePut . encoder) >-> toNetwork
>
> I immediately run into several problems. First off, my authentication
> code wants to deal with Frame's, not ByteString, but my initial has a
> ByteString producer. I can see that lenses are supposed to let me "zoom"
> "Parser a" to "Parser b", but unfortunately pipes-binary only has the
> following lens:
>
> decoded :: (Monad m, Binary a) => Lens' (Producer ByteString m r) (
> Producer a m (Either (DecodingError, Producer ByteString m r) r))
>
> This only works with Binary instances, unfortunate in my case where I
> need to use explicit getters/putters, but writing my own decoder that work
> using decodeGet and encodePut is not too hard. A bigger issue is, how to
> handle "authenticate", since it wants to deal with Frame's it would ideally
> have the type "Parser Frame (Producer Frame (ReaderT SockConfig (SafeT IO))
> r", but I have no way to run this within my "Parser ByteString (Producer
> ByteString (ReaderT SockConfig (SafeT IO))" as the types don't match.
>
> Even worse, after the "runState (handShake addr)" in session the
> remainder of fromNetwork's output is in the Producer returned as "rest",
> unfortunately the trip through the Parser's StateT has changed
> fromNetwork's type to "Producer ByteString (Producer ByteString (ReaderT
> SockConfig (SafeT IO))", I can't compose this with session "Consumer Frame
> (SafeT IO) ()", because the inner Producer and ReaderT are in the way, even
> though these are now useless as only the outer Producer will still do
> something. I could get rid of the ReaderT with a "runReaderP undefined",
> but I have no way to get rid of the inner Producer...
>
> Any suggestions on how to do this in the new pipes-parse world? Or
> should I just give up and stick with my old design of explicit passing
> around of leftovers?
>
> Cheers,
> Merijn
> --
> You received this message because you are subscribed to the Google Groups
> "Haskell Pipes" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected] <javascript:>.
> To post to this group, send email to [email protected]
> <javascript:>.
>
>
>
--
You received this message because you are subscribed to the Google Groups
"Haskell Pipes" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
