Hey Gabriel,
Thanks for your response, but I'm not quite sure how to apply
this in my case? (I don't have any experience with conduit, so
I'll ignore those remarks for now). With your suggested type of:
handShake
:: SockAddr
-> Producer ByteString M r
-> Producer ByteString M ((Get Frame, Frame -> Put), Producer
ByteString M r)
I'm not quite sure how this will fit? The problem is that during
handshake I need to do things like "read/parse N bytes of the
ByteString coming from the socket, then based on that write M
bytes to the socket". I'm not quite sure how to do that with this
type? Take for example "zmtpVersion", this needs to read
ByteString's from the input, write output to the socket and
return a Version. Does that mean that, just like handShake,
zmtpVersion's type becomes "Producer ByteString M r -> Producer
ByteString M (Version, Producer ByteString M r)"? How do I get
the ByteString's to write to the socket from there?
zmtpVersion
:: Producer ByteString M r
-> Producer ByteString M (Version, Producer ByteString M r)
zmtpVersion prod = do
??
I can't use any of the pipes-binary parsing utilities here, as
they all producer Parser's and you just said that using a
Producer as a base monad for Parser won't work (which is the only
way I could run a Parser and end up with a Producer like this type)?
I really don't know how to incrementally parse a value from the
input producer here, except: 1) manually thread leftovers
everywhere like my initial design or 2) manually step through the
input producer using "next".
Both of these seem really painful and cumbersome, and the
annoyance of the former was the entire reason I was hoping
pipes-parse would make things better. But so far it seems that it
can't?
Cheers,
Merijn
On Tuesday, July 29, 2014 1:57:06 AM UTC+2, Gabriel Gonzalez wrote:
The `Parser` type is intended to be used with a base monad
that is not a `Producer` or any other pipe type. The
intuition you should have for `pipes-parse` is that they are
almost exactly analogous to `conduit` abstractions (minus the
finalization handling of `conduit`):
* `Producer a m ()` is analogous to a `Source m a`
* `Parser a m r` is analogous to a `conduit` `Sink a m r`
* `Lens' (Producer a m r) (Producer b m r)` is analogous to a
`Conduit a m b`that can propagate leftovers further upstream
* `Getter' (Producer a m r) (Producer b m r)` (or just:
`Producer a m r -> Producer b m r`) is analogous to a
`Conduit a m b` that cannot propagate leftovers further upstream
`conduit` does not make the distinction between
transformations which can or cannot propagate leftovers
further upstream, which leads to the leftover propagation bug
described here:
http://www.haskellforall.com/2014/02/pipes-parse-30-lens-based-parsing.html
So any time you find yourself writing a `Parser` that is
trying to `yield` things further downstream, that's your cue
that you should switch it to become a `Lens'` or a `Getter`
(i.e. function) instead.
Back to your specific code: I believe the the type you want
for `handShake` is either a leftover-passing function
equivalent to the `Parser` you originally had:
type M = ReaderT SockConfig (SafeT IO)
handShake
:: SockAddr
-> Producer ByteString M r
-> Producer ByteString M ((Get Frame, Frame -> Put),
Producer ByteString M r)
... or a lens if you included enough information in the
return value to recover the original stream:
type M = ReaderT SockConfig (SafeT IO)
handShake
:: SockAddr
-> Lens' (Producer ByteString M r)
(Producer ByteString M ((ExtraInformation, Get Frame, Frame
-> Put), Producer ByteString M r)
If you did that, then you'd be able to write:
zoom (handShake . decoder):: Parser Frame m r -> Parser
ByteString m r
... and that would automatically handle all initial
authentication and provide you with a stream of decoded
frames. Or you could use it to transform a `Producer`:
view (handShake . decoder) :: Producer ByteString M r ->
Producer Frame M LargeReturnType
On 7/26/14, 10:55 AM, Merijn Verstraaten wrote:
Ok, so I have a network connection that has an interactive
handshake to setup a connection (i.e. I need to write things
to the network while parsing the handshake). Several months
ago I wrote some code that manually plumbed a ByteString
state throughout all my pipes to handle leftovers. Today I
wanted to try and rewrite this code to use the new
pipes-parse stuff, thinking the result would be much nicer
to work with. However, I get stuck with no clue how to
continue. My original code was as follows:
readSocket :: MonadIO m => Handle -> Producer ByteString m r
writeSocket :: MonadIO m => Handle -> Consumer ByteString m r
handShake :: SockAddr ->
StateT ByteString
(Pipe ByteString
ByteString
(ReaderT SockConfig (SafeT IO)))
( Pipe ByteString Frame (SafeT IO) r,
Pipe Frame ByteString (SafeT IO) r )
handShake addr = do
version <- zmtpVersion
let (decoder, encoder) = case version of
ZMTP1 {} -> (zmtp1Decoder, zmtp1Encoder)
:: MonadSafe m => (Pipe ByteString Frame m r,
Pipe Frame ByteString m r)
ZMTP2 {} -> (zmtp2Decoder, zmtp2Encoder)
ZMTP3 {} -> (zmtp3Decoder, zmtp3Encoder)
-- authenticate version addr :: Pipe Frame Frame m r
lift $ decoder >-> authenticate version addr >-> encoder
return (decoder, encoder)
session :: SockConfig
-> Producer Frame (SafeT IO) ()
-> Consumer Frame (SafeT IO) ()
-> IO Handle
-> SockAddr
-> IO ()
session cfg fromApp toApp newHandle addr =
runSafeT . runEffect . bracket newHandle hClose $ \hnd -> do
let fromNetwork = readSocket hnd
toNetwork = writeSocket hnd
handshake = runReaderP cfg (runStateT (handShake
addr) empty)
((decoder, encoder), rest) <- fromNetwork >-> handshake
>-> toNetwork
myTid <- liftIO myThreadId
let appToNet = fromApp >-> encoder >-> toNetwork
netToApp = (yield rest >> fromNetwork) >-> decoder
>-> toApp
-- code using appToNet/netToApp Effect's
In other words, my handshake is just a pipe, reading and
writing ByteStrings from and to the same socket. After
version negotiation I do authentication, this uses a "Pipe
Frame Frame" that I compose with the relevant
decoders/encoders and lift into the handshake's pipe. So
far, so good, the only real nastiness is manually threading
the leftover state throughout the zmtpVersion, decoder and
encoder.
So on to my attempt to rewrite this using pipes-parse.
readSocket :: MonadIO m => Handle -> Producer ByteString m r
writeSocket :: MonadIO m => Handle -> Consumer ByteString m r
handShake :: SockAddr
-> Parser ByteString (Producer ByteString (ReaderT
SockConfig (SafeT IO))) (Get Frame, Frame -> Put)
handShake _addr = do
version <- zmtpVersion
let (getFrame, putFrame) = case version of
ZMTP1 {} -> (ZMTP1.getFrame, ZMTP1.putFrame)
ZMTP2 {} -> (ZMTP2.getFrame, ZMTP2.putFrame)
ZMTP3 {} -> (ZMTP3.getFrame, ZMTP3.putFrame)
--lift $ undefined >-> authenticate version addr >-> undefined
return (getFrame, putFrame)
session :: SockConfig
-> Producer Frame (SafeT IO) ()
-> Consumer Frame (SafeT IO) ()
-> IO Handle
-> SockAddr
-> IO ()
session cfg fromApp toApp newHandle addr =
runSafeT . runEffect . bracket newHandle hClose $ \hnd -> do
let fromNetwork = readSocket hnd
toNetwork = writeSocket hnd
handshake = runReaderP cfg . runStateT (handShake addr)
((decoder, encoder), rest) <- handshake fromNetwork >->
toNetwork
myTid <- liftIO myThreadId
let appToNet = for fromApp (encodePut . encoder) >->
toNetwork
I immediately run into several problems. First off, my
authentication code wants to deal with Frame's, not
ByteString, but my initial has a ByteString producer. I can
see that lenses are supposed to let me "zoom" "Parser a" to
"Parser b", but unfortunately pipes-binary only has the
following lens:
decoded :: (Monad m, Binary a) => Lens'
(ProducerByteString m r) (Producer a m
(Either (DecodingError, ProducerByteString m r) r))
This only works with Binary instances, unfortunate in my
case where I need to use explicit getters/putters, but
writing my own decoder that work using decodeGet and
encodePut is not too hard. A bigger issue is, how to handle
"authenticate", since it wants to deal with Frame's it would
ideally have the type "Parser Frame (Producer Frame (ReaderT
SockConfig (SafeT IO)) r", but I have no way to run this
within my "Parser ByteString (Producer ByteString (ReaderT
SockConfig (SafeT IO))" as the types don't match.
Even worse, after the "runState (handShake addr)" in session
the remainder of fromNetwork's output is in the Producer
returned as "rest", unfortunately the trip through the
Parser's StateT has changed fromNetwork's type to "Producer
ByteString (Producer ByteString (ReaderT SockConfig (SafeT
IO))", I can't compose this with session "Consumer Frame
(SafeT IO) ()", because the inner Producer and ReaderT are
in the way, even though these are now useless as only the
outer Producer will still do something. I could get rid of
the ReaderT with a "runReaderP undefined", but I have no way
to get rid of the inner Producer...
Any suggestions on how to do this in the new pipes-parse
world? Or should I just give up and stick with my old design
of explicit passing around of leftovers?
Cheers,
Merijn
--
You received this message because you are subscribed to the
Google Groups "Haskell Pipes" group.
To unsubscribe from this group and stop receiving emails
from it, send an email to [email protected].
To post to this group, send email to [email protected].
--
You received this message because you are subscribed to the
Google Groups "Haskell Pipes" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to [email protected]
<mailto:[email protected]>.
To post to this group, send email to
[email protected]
<mailto:[email protected]>.
