Nicolas,
Somewhat related, I started this project some time ago: https://github.com/k0001/pipes-http-message/blob/master/src/Pipes/HTTP/Message.hs The idea, similar to yours, is was to parse HTTP messages (request and responses) in a streaming fashion. I kind of stopped working on it as I became busy with other things, but I'd like to spend some time on it again. Maybe you can get some ideas from there. Regards, Renzo Carbonara. On Mon, Jun 30, 2014 at 7:37 PM, Nicolas Trangez <[email protected]> wrote: > On Mon, 2014-06-30 at 22:46 +0300, Aycan iRiCAN wrote: >> Hi there, >> >> I'll be happy to add more features to `hweblib` if you want to use it. I >> think your use case is important and should be covered by the library. I >> was also thinking to implement it via ekmett's parsers library but failed >> since it doesn't supply any case-insensitive parsers. > > Thanks. I got the basics (request parsing of requests without a body) > working, using `pipes-parse`/`pipes-attoparsec` and `hweblib`. > > Maybe it could be interesting to add 'unparsers' to `hweblib`? Here's > e.g. what I use now: > > ```haskell > responseProducer :: Monad m => Response -> Producer ByteString m () > responseProducer Response{..} = fromLazy $ toLazyByteString builder > where > builder = status <> headers <> newline > version = byteString "HTTP/" <> intDec (httpMajor rpVersion) > <> char7 '.' > <> intDec (httpMinor rpVersion) > status = mconcat [ version > , space > , intDec rpCode > , space > , byteString rpMessage > , newline > ] > headers = mconcat $ map > (\(k, v) -> byteString k <> byteString ": " <> > byteString v <> newline) > rpHeaders > > newline = char7 '\n' > space = char7 ' ' > ``` > > where in `hweblib` the `pipes` stuff could be removed, returning > `builder` as-is instead. > > Nicolas > >> >> Kind Regards, >> >> >> On Mon, Jun 30, 2014 at 10:20 PM, Nicolas Trangez <[email protected]> >> wrote: >> >> > On Sun, 2014-06-29 at 18:05 -0700, Gabriel Gonzalez wrote: >> > > So if I understand correctly, you only need a way to: >> > > >> > > A) Parse requests from a raw byte stream (i.e. your `Producer`) >> > > B) Create responses >> > > C) Serialize responses to a byte stream >> > >> > That's about it indeed. >> > >> > > `pipes-attoparsec` is the most idiomatic way to do (A). You can get the >> > > necessary parsers from `hweblib`, which provides RFC2616-compliant >> > parsers. >> > >> > Aha, interesting. I didn't know that package, guess it could suit my >> > needs. >> > >> > Too bad the Warp parser can't be reused though. I'd expect it's more >> > battle-tested. On first sight, `hweblib` doesn't handle request bodies >> > (not to mention chunked encodings). I guess I could not support that for >> > now, maybe look into it later. >> > >> > > For (B), you can also use the `hweblib` library, which provides a >> > > `Response` type you can build. >> > > >> > > For (C), I'm not as sure. Ask Aycan (the maintainer of `hweblib`) if >> > > there is a way to serialize `Response`s to `ByteString`s. >> > >> > According to the package Haddocks, the `Response` constructor & >> > accessors are exported, so I can write `responseProducer` myself :-) >> > >> > > The main issue in general is that most web server libraries don't have a >> > > separate library just for parsing requests and serializing responses. >> > > `hweblib` is the closest library I've found for this purpose. >> > >> > Yeah. It's somewhat sad: HTTP seems deceivingly simple, but being able >> > to parse every valid HTTP request, including all possible quirks, is >> > harder than one might think, and writing a parser over and over again >> > seems rather useless. >> > >> > Maybe I should create a package, using `http-types`. >> > >> > On a side note: would it be hard to create a parser library (most likely >> > limited to an applicative interface, not monadic) which, given a parser >> > definition, can also create an 'unparser', up to an isomorphism (~ >> > ignored elements)? I'm aware of some research in this area, but given >> > Lens and its combination of getters & setters in one, there might be >> > some new opportunities. >> > >> > Nicolas >> > >> > > >> > > On 06/29/2014 05:00 PM, Nicolas Trangez wrote: >> > > > Gabriel, >> > > > >> > > > On Sun, 2014-06-29 at 12:37 -0700, Gabriel Gonzalez wrote: >> > > >> This is a simple "echo" server written using `pipes-wai`: >> > > >> >> > > >> {-# LANGUAGE OverloadedStrings #-} >> > > >> >> > > >> import Blaze.ByteString.Builder (fromByteString) >> > > >> import Pipes >> > > >> import qualified Pipes.Prelude as Pipes >> > > >> import Pipes.Wai >> > > >> import Network.Wai.Handler.Warp >> > > >> import Network.HTTP.Types (Status(..)) >> > > >> >> > > >> main = run 8000 $ \request onResponse -> do >> > > >> let p = do >> > > >> producerRequestBody request >-> Pipes.map (Chunk . >> > > >> fromByteString) >> > > >> yield Flush >> > > >> >> > > >> onResponse (responseProducer (Status 200 "") [] p) >> > > >> >> > > >> `producerRequestBody` turns the incoming client's request into a >> > > >> `Producer`. Then you can do whatever you want with that `Producer`, >> > > >> like read in its contents, discard it, or (in the above example) >> > stream >> > > >> it directly to the response. >> > > >> >> > > >> If you give a more specific example of what you want to do, I can >> > give a >> > > >> more specific example. >> > > > Thanks for taking time looking into this. It looks like I might have >> > > > been somewhat unclear before. >> > > > >> > > > Whilst your example above indeed turns a client request into a Producer >> > > > and turns a Producer into a response, this is all at another layer than >> > > > what I'm trying to accomplish. >> > > > >> > > > Here's the difference: in your example, the 'run' function (as provided >> > > > by Warp) takes care of setting up a listening socket, accepting client >> > > > connections, forking threads,... >> > > > >> > > > Whilst in my application, all this functionality is provided already, >> > > > all I'm left with are a ByteString Producer and ByteString Consumer >> > > > representing a client socket connection. Now a Request should be >> > > > read/parsed from the Producer, handled, and then a Response should be >> > > > yield'ed to the Consumer. All of this for this single client >> > connection, >> > > > no need to accept connections, spawn threads,... >> > > > >> > > > This could boil down to, e.g. >> > > > >> > > > runHTTPInteraction :: Monad m => Pipe ByteString ByteString m r >> > > > runHTTPInteraction = do >> > > > req <- readRequest >> > > > let resp = ... >> > > > renderResponse resp >> > > > >> > > > Then, when my client handler function is called with a `Producer >> > > > ByteString m r` and `Consumer ByteString m r`, I can use the above >> > using >> > > > something like >> > > > >> > > > -- Interact with a single client. This runs in its own thread etc. >> > > > myHandler :: Monad m => Producer ByteString IO () -> Consumer >> > ByteString >> > > > IO () -> () >> > > > myHandler prod cons = >> > > > runEffect $ >> > > > {- for HTTP pipelining / connection re-use... -} forever $ >> > > > prod >-> runHTTPInteractoin >-> cons >> > > > >> > > > (or something along those lines, this is no real code so might not even >> > > > type-check). >> > > > >> > > > Thanks, >> > > > >> > > > Nicolas >> > > > >> > > > >> > > >> On 06/28/2014 02:20 PM, Nicolas Trangez wrote: >> > > >>> On Sat, 2014-06-28 at 14:12 -0700, Gabriel Gonzalez wrote: >> > > >>>> Have you tried the `pipes-wai` library? >> > > >>> I passed by it during my search, but its operations seem >> > client-centric: >> > > >>> turning a `Request` into a `Producer ByteString m ()` (in order to >> > > >>> stream the data from that producer to some server?), or create a >> > > >>> `Response` out of a `Producer (Flush Builder) IO ()` (in order to >> > stream >> > > >>> data from the producer to a client on some server?). >> > > >>> >> > > >>> I could be missing something very obvious, but I don't think that's >> > what >> > > >>> I need. >> > > >>> >> > > >>> Thanks, >> > > >>> >> > > >>> Nicolas >> > > >>> >> > > >>>> On Jun 28, 2014 1:52 PM, "Nicolas Trangez" <[email protected]> >> > wrote: >> > > >>>> >> > > >>>>> All, >> > > >>>>> >> > > >>>>> Has anyone ever written anything HTTP-server-like using Pipes? I >> > looked >> > > >>>>> into Warp to check whether it could be integrated, but Warp seems >> > to do >> > > >>>>> connection management, thread management, timeout handling, >> > whatnot, >> > > >>>>> which is not what I need. >> > > >>>>> >> > > >>>>> I'm looking for something simpler: given a `Producer ByteString m >> > r` and >> > > >>>>> a `Consumer ByteString m r`, read a `Request` from the producer, >> > then do >> > > >>>>> some handling, and render a `Response` to the consumer. >> > > >>>>> >> > > >>>>> Are there any libraries I could look into, or is Warp easy to >> > integrate >> > > >>>>> yet I'm failing to see how? >> > > >>>>> >> > > >>>>> Thanks, >> > > >>>>> >> > > >>>>> Nicolas >> > > >>>>> >> > > >>>>> -- >> > > >>>>> You received this message because you are subscribed to the Google >> > Groups >> > > >>>>> "Haskell Pipes" group. >> > > >>>>> To unsubscribe from this group and stop receiving emails from it, >> > send an >> > > >>>>> email to [email protected]. >> > > >>>>> To post to this group, send email to >> > [email protected]. >> > > >>>>> >> > > > >> > > >> > >> > >> > -- >> > You received this message because you are subscribed to the Google Groups >> > "Haskell Pipes" group. >> > To unsubscribe from this group and stop receiving emails from it, send an >> > email to [email protected]. >> > To post to this group, send email to [email protected]. >> > >> >> >> >> -- >> http://www.google.com/profiles/iricanaycan >> > > > -- > You received this message because you are subscribed to the Google Groups > "Haskell Pipes" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. -- You received this message because you are subscribed to the Google Groups "Haskell Pipes" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected].
