Re: [Haskell-cafe] Web application interface
On Sat, 23 Jan 2010 18:52:01 +0200, Michael Snoyman mich...@snoyman.com wrote: Jeremy, What I meant is, if you use a sendfile system call to send raw files from the disk, how does this interact with gzip compression, which clearly cannot be used when using a sendfile call? I ask because you implied there were significant performance gains from using sendfile. I guess you can either gzip the file to another temporary one and sendfile the temporary gz one (even maybe cache it) or abandon sendfile and stream out a gzip of file contents. The former could be more performant. Regards, -- Nicolas Pouillard http://nicolaspouillard.fr ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Web application interface
On Jan 23, 2010, at 10:52 AM, Michael Snoyman wrote: Jeremy, What I meant is, if you use a sendfile system call to send raw files from the disk, how does this interact with gzip compression, which clearly cannot be used when using a sendfile call? I ask because you implied there were significant performance gains from using sendfile. At present there is no mechanism to use sendfile+gzip. The only solution would be to gzip the file in a cache of some sort, and the use sendfile on that. That is just a limitation of the way sendfile works. We do not have anything like that built-in yet.. but you could add in a 3rd party library... - jeremy ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Web application interface
Hello, In happstack, there is a Writer monad which holds a list of filters which will be applied to the Response before sending it out. One of these filters is the gzip filter. The compression filters are defined here: http://patch-tag.com/r/mae/happstack/snapshot/current/content/pretty/happstack-server/src/Happstack/Server/Parts.hs The filters are apply when runWebT is called: http://patch-tag.com/r/mae/happstack/snapshot/current/content/pretty/happstack-server/src/Happstack/Server/SimpleHTTP.hs runWebT is called automatically by the top-level function, simpleHTTP, that people actually call in their programs. We do not do anything fancy to cache gzip results to the disk. We don't even assume you *have* a disk. I believe that functionality could be added as a 3rd party library with out modifying core happstack. That is how we would prefer to see it done so that the core is simple, and so that people can implement their own caching system if their needs are different. - jeremy On Jan 21, 2010, at 10:37 PM, Michael Snoyman wrote: Hey Jeremy, I was just wondering: how does Happstack deal with gzip encoding when it uses sendfile? I can think of a few ways (cache gziped versions to the disk), but was wondering if you'd already come up with a good solution. I'm trying to keep all these things in mind when designing WAI. Thanks, Michael On Thu, Jan 14, 2010 at 5:42 PM, Jeremy Shaw jer...@n-heptane.com wrote: Hello, Happstack is currently bundled with it's own lazy I/O based HTTP backend. Ideally, we would like to split that out, and allow happstack to be used with that backend, hyena, or other options. A primary using for using hyena would be for the benefits of predictability and constant space usage that iterators bring. People do actually running into the issues that come with lazy I/O, such as running out of file descriptors, etc. So, I feel like I would want to stick with using iterators the whole way when using hyena, and not convert back to a lazy ByteString? Happstack now includes support for sendfile(). This is done by adding another constructor to the Response type: (line 94): http://patch-tag.com/r/mae/happstack/snapshot/current/content/pretty/happstack-server/src/Happstack/Server/HTTP/Types.hs Then here on line 197, we match on that case and use sendfile to send the data: http://patch-tag.com/r/mae/happstack/snapshot/current/content/pretty/happstack-server/src/Happstack/Server/HTTP/Handler.hs This makes it difficult for use to be compatible with WAI. We can write a wrapper that converts the sendfile case to use lazy bytestrings instead, but then we lose the advantages of using sendfile. I wonder if the 'Response' portion of WAI should support all three currently used methods: - lazy I/O - Enumerator - sendFile I haven't really thought about how that would work.. hyena currently includes a Network.WAI which uses ByteString: http://hackage.haskell.org/packages/archive/hyena/0.1/doc/html/Network-Wai.html gotta run, sorry about any typos! - jeremy On Jan 13, 2010, at 8:46 AM, Michael Snoyman wrote: Hi, I recently read (again) the wiki page on a web application interface[1] for Haskell. It seems like this basically works out to Hack[2], but using an enumerator instead of lazy bytestring in the response type. Is anyone working on implementing this? If not, I would like to create the package, though I wouldn't mind some community input on some design decisions: * Hack has been fairly well-tested in the past year and I think it provides the features that people want. Therefore, I would want to model the Environment variable for WAI from Hack. I *could* just import Hack in WAI and use the exact same Environment data type. Thoughts? * If using a different data type for Environment, should I replace the String parts with ByteStrings? On the one hand, ByteStrings are the correct data type since the HTTP protocol does not specify a character encoding; on the other hand, Strings are easier to deal with. * It's simple to write a function to convert between a lazy bytestring and an enumerator, meaning it would be very easy to write conversion functions between Hack and WAI applications. This would make it simpler for people to use either backend. If someone else is already working on WAI, please let me know, I don't want to have duplicate implementations. The idea here is to consolidate, not split the community. I have a few Hack handlers (simpleserver, cgi, fastcgi) that I would happily convert to WAI handlers as well. Michael [1] http://www.haskell.org/haskellwiki/WebApplicationInterface [2] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/hack ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___
Re: [Haskell-cafe] Web application interface
Hey Jeremy, I was just wondering: how does Happstack deal with gzip encoding when it uses sendfile? I can think of a few ways (cache gziped versions to the disk), but was wondering if you'd already come up with a good solution. I'm trying to keep all these things in mind when designing WAI. Thanks, Michael On Thu, Jan 14, 2010 at 5:42 PM, Jeremy Shaw jer...@n-heptane.com wrote: Hello, Happstack is currently bundled with it's own lazy I/O based HTTP backend. Ideally, we would like to split that out, and allow happstack to be used with that backend, hyena, or other options. A primary using for using hyena would be for the benefits of predictability and constant space usage that iterators bring. People do actually running into the issues that come with lazy I/O, such as running out of file descriptors, etc. So, I feel like I would want to stick with using iterators the whole way when using hyena, and not convert back to a lazy ByteString? Happstack now includes support for sendfile(). This is done by adding another constructor to the Response type: (line 94): http://patch-tag.com/r/mae/happstack/snapshot/current/content/pretty/happstack-server/src/Happstack/Server/HTTP/Types.hs Then here on line 197, we match on that case and use sendfile to send the data: http://patch-tag.com/r/mae/happstack/snapshot/current/content/pretty/happstack-server/src/Happstack/Server/HTTP/Handler.hs This makes it difficult for use to be compatible with WAI. We can write a wrapper that converts the sendfile case to use lazy bytestrings instead, but then we lose the advantages of using sendfile. I wonder if the 'Response' portion of WAI should support all three currently used methods: - lazy I/O - Enumerator - sendFile I haven't really thought about how that would work.. hyena currently includes a Network.WAI which uses ByteString: http://hackage.haskell.org/packages/archive/hyena/0.1/doc/html/Network-Wai.html gotta run, sorry about any typos! - jeremy On Jan 13, 2010, at 8:46 AM, Michael Snoyman wrote: Hi, I recently read (again) the wiki page on a web application interface[1] for Haskell. It seems like this basically works out to Hack[2], but using an enumerator instead of lazy bytestring in the response type. Is anyone working on implementing this? If not, I would like to create the package, though I wouldn't mind some community input on some design decisions: * Hack has been fairly well-tested in the past year and I think it provides the features that people want. Therefore, I would want to model the Environment variable for WAI from Hack. I *could* just import Hack in WAI and use the exact same Environment data type. Thoughts? * If using a different data type for Environment, should I replace the String parts with ByteStrings? On the one hand, ByteStrings are the correct data type since the HTTP protocol does not specify a character encoding; on the other hand, Strings are easier to deal with. * It's simple to write a function to convert between a lazy bytestring and an enumerator, meaning it would be very easy to write conversion functions between Hack and WAI applications. This would make it simpler for people to use either backend. If someone else is already working on WAI, please let me know, I don't want to have duplicate implementations. The idea here is to consolidate, not split the community. I have a few Hack handlers (simpleserver, cgi, fastcgi) that I would happily convert to WAI handlers as well. Michael [1] http://www.haskell.org/haskellwiki/WebApplicationInterface [2] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/hack ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Web application interface
Excerpts from Michael Snoyman's message of Wed Jan 13 15:46:12 +0100 2010: Hi, I recently read (again) the wiki page on a web application interface[1] for Haskell. It seems like this basically works out to Hack[2], but using an enumerator instead of lazy bytestring in the response type. Is anyone working on implementing this? If not, I would like to create the package, though I wouldn't mind some community input on some design decisions: Wai seems to lightly rely on ImpredicativeTypes which becomes deprecated in GHC 6.12 and could be removed in 6.14. This seems to be due to the response tuple. Is there any plan to change this to a more forward compatible solution? -- Nicolas Pouillard http://nicolaspouillard.fr ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Web application interface
Absolutely; the goals I have are minimal dependencies and no warnings for compilation ;). On Sat, Jan 16, 2010 at 9:35 PM, Nicolas Pouillard nicolas.pouill...@gmail.com wrote: Excerpts from Michael Snoyman's message of Wed Jan 13 15:46:12 +0100 2010: Hi, I recently read (again) the wiki page on a web application interface[1] for Haskell. It seems like this basically works out to Hack[2], but using an enumerator instead of lazy bytestring in the response type. Is anyone working on implementing this? If not, I would like to create the package, though I wouldn't mind some community input on some design decisions: Wai seems to lightly rely on ImpredicativeTypes which becomes deprecated in GHC 6.12 and could be removed in 6.14. This seems to be due to the response tuple. Is there any plan to change this to a more forward compatible solution? -- Nicolas Pouillard http://nicolaspouillard.fr ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Web application interface
Excerpts from Jinjing Wang's message of Thu Jan 14 01:28:31 +0100 2010: | The hyena backend is essentially just a translator between hack and | wai, i failed to finished it since I can't understand iteratee | (seriously) and eventually got distracted ... If I have well understood you miss a function to convert an enumerator to a list. What about this code? import qualified Data.ByteString.Lazy.Char8 as S import Control.Concurrent (forkIO) import Control.Concurrent.Chan (newChan,writeChan,getChanContents) type Enumerator = forall a. (a - S.ByteString - IO (Either a a)) - a - IO a enumToList :: Enumerator - IO [S.ByteString] enumToList e = do ch - newChan _ - forkIO $ e (writer ch) () getChanContents ch where writer ch () chunk = do writeChan ch chunk return (Right ()) Best regards, -- Nicolas Pouillard http://nicolaspouillard.fr ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Web application interface
Hi Michael, no, the message was not meant to be off-list, that was just me pressing the wrong button :-) Regarding happstack, I do not believe that there is a contrast with your effort, the core of happstack is in its persistency mechanism not in its http interface so I think it would be great to engage the happstack community in this effort. We have so many half-baked and dispersed attempts in the web area that any attempt at consolidation can only be welcome. Personally I have been using happstack for a few years though now I am running it behind a nginx server because of its known deficiencies (no HTTPS etc). titto P.S. I will be away for a few days and unable to answer my email 2010/1/13 Michael Snoyman mich...@snoyman.com: Not sure if you replied off-list on purpose or not, but I'll continue this off list for the moment. I think we have a bit of a problem in the Haskell web community: you've got the Happstack camp and then the rest of us. The rest of us need to rally around *something*, and it seems that Hack didn't get people's attention for some reason. I'm happy to write WAI, but I'd like more to make it a community effort. You have any thoughts on this? My first stab at the idea is to create a github repo, write the code, and then try to get people to comment on it. However, I also want to give it at least a day so I can get people's feedback on this e-mail. What have you been using for Haskell web development until now? It seems like each non-Happstack person has a totally different approach, and I'd like to try and consolidate this together somehow. Michael On Wed, Jan 13, 2010 at 11:12 PM, Pasqualino Titto Assini tittoass...@gmail.com wrote: A unified web app interface would be a God-sent, please please go ahead. Regarding point 1, I find hack interface nice and clean and would like to see something similar. Regarding point 2 I vote for correctness/performance vs convenience. titto 2010/1/13 Michael Snoyman mich...@snoyman.com: Hi, I recently read (again) the wiki page on a web application interface[1] for Haskell. It seems like this basically works out to Hack[2], but using an enumerator instead of lazy bytestring in the response type. Is anyone working on implementing this? If not, I would like to create the package, though I wouldn't mind some community input on some design decisions: * Hack has been fairly well-tested in the past year and I think it provides the features that people want. Therefore, I would want to model the Environment variable for WAI from Hack. I *could* just import Hack in WAI and use the exact same Environment data type. Thoughts? * If using a different data type for Environment, should I replace the String parts with ByteStrings? On the one hand, ByteStrings are the correct data type since the HTTP protocol does not specify a character encoding; on the other hand, Strings are easier to deal with. * It's simple to write a function to convert between a lazy bytestring and an enumerator, meaning it would be very easy to write conversion functions between Hack and WAI applications. This would make it simpler for people to use either backend. If someone else is already working on WAI, please let me know, I don't want to have duplicate implementations. The idea here is to consolidate, not split the community. I have a few Hack handlers (simpleserver, cgi, fastcgi) that I would happily convert to WAI handlers as well. Michael [1] http://www.haskell.org/haskellwiki/WebApplicationInterface [2] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/hack ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe -- Pasqualino Titto Assini, Ph.D. http://quicquid.org/ -- Pasqualino Titto Assini, Ph.D. http://quicquid.org/ ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Web application interface
2010/1/14 Jinjing Wang nfjinj...@gmail.com Hyena is especially tuned for streaming and that's exactly what hack can't do (in practice). Isn't possible to stream an (almost) infinite bytestring trough hack?. I ever trough that the laziness of haskell is a great advantage in Web applications. This is very important because the size of the block transferred vary widely. In my applications I don´t care whether I have to stream a hello world page or a video. The first block of my application goes trough the internet as soon as my procedure start without concern about if the processing is composed of a complicated chain of steps or not. And with no especial coding; Neither my web server interface nor my user responsiveness requirements force me to code iterations everywhere in my code. I know the chuncked mode in web server but I think that just this mode of web streaming is the right mode for serving lazy haskell applications. My question is why whatever performance advantage the iteratee may have, can not be coded under the clean interface of a lazy bytestring or whatever lazy stream. http://github.com/nfjinjing/hack-handler-hyena -- jinjing ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Web application interface
On Thu, Jan 14, 2010 at 12:50 PM, Alberto G. Corona agocor...@gmail.comwrote: 2010/1/14 Jinjing Wang nfjinj...@gmail.com Hyena is especially tuned for streaming and that's exactly what hack can't do (in practice). Isn't possible to stream an (almost) infinite bytestring trough hack?. I ever trough that the laziness of haskell is a great advantage in Web applications. This is very important because the size of the block transferred vary widely. In my applications I don´t care whether I have to stream a hello world page or a video. The first block of my application goes trough the internet as soon as my procedure start without concern about if the processing is composed of a complicated chain of steps or not. And with no especial coding; Neither my web server interface nor my user responsiveness requirements force me to code iterations everywhere in my code. I know the chuncked mode in web server but I think that just this mode of web streaming is the right mode for serving lazy haskell applications. My question is why whatever performance advantage the iteratee may have, can not be coded under the clean interface of a lazy bytestring or whatever lazy stream. Well, for one thing, you'd need to use lazy IO to achieve your goal, which has some safety issues. As things get more and more complex, the requirements of lazy IO will continue to grow. This also has implications for number of open file handles and deterministic space usage. Given the fact that a lazy bytestring and easily be converted to an enumerator, I think it makes sense to start a new package using this approach. As a side point, would anyone be interested in having a central location for web-specific Haskell development discussions? I know we have the mailing list, but it's never used. I'm thinking more of a place to post articles and links to packages. In particular, I think it would be great to have a site with multiple sections (model, view, controller, authentication, authorization, etc) and articles, forums and packages specific for each. Also a great place to post what the community is missing. Michael http://github.com/nfjinjing/hack-handler-hyena -- jinjing ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Web application interface
2010/1/14 Michael Snoyman mich...@snoyman.com Well, for one thing, you'd need to use lazy IO to achieve your goal, which has some safety issues. As things get more and more complex, the requirements of lazy IO will continue to grow. This also has implications for number of open file handles and deterministic space usage. Given the fact that a lazy bytestring and easily be converted to an enumerator, I think it makes sense to start a new package using this approach. These must be issues of base library developers, not application developpers. TIme ago a guy said me that using an standard library like malloc for memory allocation where not the optimum. And he was right. Fortunately things went in the non-optimum direction. You can make the application faster by using your own plumbing code instead of standard libraries provided that you have enough time and knowledge. Because I don´t have neither of the two :-) (nor have the people that read my code and maintain the application), I really like the laziness of haskell and the lazy bytestring interface. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Web application interface
On Thu, Jan 14, 2010 at 1:20 PM, Alberto G. Corona agocor...@gmail.comwrote: 2010/1/14 Michael Snoyman mich...@snoyman.com Well, for one thing, you'd need to use lazy IO to achieve your goal, which has some safety issues. As things get more and more complex, the requirements of lazy IO will continue to grow. This also has implications for number of open file handles and deterministic space usage. Given the fact that a lazy bytestring and easily be converted to an enumerator, I think it makes sense to start a new package using this approach. These must be issues of base library developers, not application developpers. TIme ago a guy said me that using an standard library like malloc for memory allocation where not the optimum. And he was right. Fortunately things went in the non-optimum direction. You can make the application faster by using your own plumbing code instead of standard libraries provided that you have enough time and knowledge. Because I don´t have neither of the two :-) (nor have the people that read my code and maintain the application), I really like the laziness of haskell and the lazy bytestring interface. Lazy bytestring interface is one thing; lazy IO is another. If you have pure code generating a lazy bytestring, Hack will work fine for you. Try this one however: take a 10MB YAML file, reformat it using something in the IO monad (for example, look up values from a database) and produce HTML output. Hack will *not* allow you to run in constant space without significant usageof unsafe functions. Michael ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Web application interface
On Thu, Jan 14, 2010 at 1:58 PM, Alberto G. Corona agocor...@gmail.comwrote: 2010/1/14 Michael Snoyman mich...@snoyman.com On Thu, Jan 14, 2010 at 1:20 PM, Alberto G. Corona agocor...@gmail.comwrote: 2010/1/14 Michael Snoyman mich...@snoyman.com Well, for one thing, you'd need to use lazy IO to achieve your goal, which has some safety issues. As things get more and more complex, the requirements of lazy IO will continue to grow. This also has implications for number of open file handles and deterministic space usage. Given the fact that a lazy bytestring and easily be converted to an enumerator, I think it makes sense to start a new package using this approach. These must be issues of base library developers, not application developpers. TIme ago a guy said me that using an standard library like malloc for memory allocation where not the optimum. And he was right. Fortunately things went in the non-optimum direction. You can make the application faster by using your own plumbing code instead of standard libraries provided that you have enough time and knowledge. Because I don´t have neither of the two :-) (nor have the people that read my code and maintain the application), I really like the laziness of haskell and the lazy bytestring interface. Lazy bytestring interface is one thing; lazy IO is another. If you have pure code generating a lazy bytestring, Hack will work fine for you. Try this one however: take a 10MB YAML file, reformat it using something in the IO monad (for example, look up values from a database) and produce HTML output. Hack will *not* allow you to run in constant space without significant usageof unsafe functions. Michael So there are memory leaks somewhere in the lazy bytestring IO libraries (not in hack neither is an inherent problem in the lazy bytestring design, the lazy IO concept or laziness as such). I did''t take a look, but surely a lazy bytestring IO read is composed of an iteration of strict block reads that present a lazy bytestring interface. It must be essentially the same than a iteratee IO, but with a higher level interface (at least higher from my point of view). I like haskell for Internet applications because streaming is the essence of communications and function call is the building block of programming. Haskell deals with both with zero impedance because its laziness. Don't break this!! No, that's not the way it works. Lazy IO requires the use of unsafeInterleaveIO, which is, well, unsafe. Pure functions in Haskell can safely be lazy, not so with IO. If you don't believe me, you can read more about it here: http://okmij.org/ftp/Streams.html#iteratee Michael ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Web application interface
Hello, Happstack is currently bundled with it's own lazy I/O based HTTP backend. Ideally, we would like to split that out, and allow happstack to be used with that backend, hyena, or other options. A primary using for using hyena would be for the benefits of predictability and constant space usage that iterators bring. People do actually running into the issues that come with lazy I/O, such as running out of file descriptors, etc. So, I feel like I would want to stick with using iterators the whole way when using hyena, and not convert back to a lazy ByteString? Happstack now includes support for sendfile(). This is done by adding another constructor to the Response type: (line 94): http://patch-tag.com/r/mae/happstack/snapshot/current/content/pretty/happstack-server/src/Happstack/Server/HTTP/Types.hs Then here on line 197, we match on that case and use sendfile to send the data: http://patch-tag.com/r/mae/happstack/snapshot/current/content/pretty/happstack-server/src/Happstack/Server/HTTP/Handler.hs This makes it difficult for use to be compatible with WAI. We can write a wrapper that converts the sendfile case to use lazy bytestrings instead, but then we lose the advantages of using sendfile. I wonder if the 'Response' portion of WAI should support all three currently used methods: - lazy I/O - Enumerator - sendFile I haven't really thought about how that would work.. hyena currently includes a Network.WAI which uses ByteString: http://hackage.haskell.org/packages/archive/hyena/0.1/doc/html/Network-Wai.html gotta run, sorry about any typos! - jeremy On Jan 13, 2010, at 8:46 AM, Michael Snoyman wrote: Hi, I recently read (again) the wiki page on a web application interface[1] for Haskell. It seems like this basically works out to Hack[2], but using an enumerator instead of lazy bytestring in the response type. Is anyone working on implementing this? If not, I would like to create the package, though I wouldn't mind some community input on some design decisions: * Hack has been fairly well-tested in the past year and I think it provides the features that people want. Therefore, I would want to model the Environment variable for WAI from Hack. I *could* just import Hack in WAI and use the exact same Environment data type. Thoughts? * If using a different data type for Environment, should I replace the String parts with ByteStrings? On the one hand, ByteStrings are the correct data type since the HTTP protocol does not specify a character encoding; on the other hand, Strings are easier to deal with. * It's simple to write a function to convert between a lazy bytestring and an enumerator, meaning it would be very easy to write conversion functions between Hack and WAI applications. This would make it simpler for people to use either backend. If someone else is already working on WAI, please let me know, I don't want to have duplicate implementations. The idea here is to consolidate, not split the community. I have a few Hack handlers (simpleserver, cgi, fastcgi) that I would happily convert to WAI handlers as well. Michael [1] http://www.haskell.org/haskellwiki/WebApplicationInterface [2] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/hack ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Web application interface
On Thu, Jan 14, 2010 at 5:42 PM, Jeremy Shaw jer...@n-heptane.com wrote: Hello, Happstack is currently bundled with it's own lazy I/O based HTTP backend. Ideally, we would like to split that out, and allow happstack to be used with that backend, hyena, or other options. A primary using for using hyena would be for the benefits of predictability and constant space usage that iterators bring. People do actually running into the issues that come with lazy I/O, such as running out of file descriptors, etc. So, I feel like I would want to stick with using iterators the whole way when using hyena, and not convert back to a lazy ByteString? Happstack now includes support for sendfile(). This is done by adding another constructor to the Response type: (line 94): http://patch-tag.com/r/mae/happstack/snapshot/current/content/pretty/happstack-server/src/Happstack/Server/HTTP/Types.hs Then here on line 197, we match on that case and use sendfile to send the data: http://patch-tag.com/r/mae/happstack/snapshot/current/content/pretty/happstack-server/src/Happstack/Server/HTTP/Handler.hs This makes it difficult for use to be compatible with WAI. We can write a wrapper that converts the sendfile case to use lazy bytestrings instead, but then we lose the advantages of using sendfile. I wonder if the 'Response' portion of WAI should support all three currently used methods: - lazy I/O - Enumerator - sendFile I haven't really thought about how that would work.. hyena currently includes a Network.WAI which uses ByteString: http://hackage.haskell.org/packages/archive/hyena/0.1/doc/html/Network-Wai.html gotta run, sorry about any typos! - jeremy Firstly, thanks for the Heyna Network.Wai link, I wasn't aware of it. Definitely something to take into consideration here. As for your proposal of three methods, I'm not sure if it's necesary. I understand that we would want sendfile for speedy serving of files straight from the filesystem, but it's fairly straight-forward to convert a lazy bytestring into an enumerator, and I don't think we get a performance penalty for doing so (if there's a benchmark otherwise, I'd be happy to see it). So if this were a changeset to Network.Wai in Hyena, I would see redefining Application as: type Application = Environment - IO (Int, ByteString, Headers, Either FilePath Enumerator) Implementations that wish to go for efficiency could use sendfile directly. We could even include a helper function for making this easy. We could also provide a lazy bytestring - enumerator function while we're at it (although those features might be more appropriate for a wai-helpers package, I'm not certain). I think it would be great if we could get Happstack involved in the WAI project. Michael On Jan 13, 2010, at 8:46 AM, Michael Snoyman wrote: Hi, I recently read (again) the wiki page on a web application interface[1] for Haskell. It seems like this basically works out to Hack[2], but using an enumerator instead of lazy bytestring in the response type. Is anyone working on implementing this? If not, I would like to create the package, though I wouldn't mind some community input on some design decisions: * Hack has been fairly well-tested in the past year and I think it provides the features that people want. Therefore, I would want to model the Environment variable for WAI from Hack. I *could* just import Hack in WAI and use the exact same Environment data type. Thoughts? * If using a different data type for Environment, should I replace the String parts with ByteStrings? On the one hand, ByteStrings are the correct data type since the HTTP protocol does not specify a character encoding; on the other hand, Strings are easier to deal with. * It's simple to write a function to convert between a lazy bytestring and an enumerator, meaning it would be very easy to write conversion functions between Hack and WAI applications. This would make it simpler for people to use either backend. If someone else is already working on WAI, please let me know, I don't want to have duplicate implementations. The idea here is to consolidate, not split the community. I have a few Hack handlers (simpleserver, cgi, fastcgi) that I would happily convert to WAI handlers as well. Michael [1] http://www.haskell.org/haskellwiki/WebApplicationInterface [2] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/hack ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Web application interface
Hi, I recently read (again) the wiki page on a web application interface[1] for Haskell. It seems like this basically works out to Hack[2], but using an enumerator instead of lazy bytestring in the response type. Is anyone working on implementing this? If not, I would like to create the package, though I wouldn't mind some community input on some design decisions: * Hack has been fairly well-tested in the past year and I think it provides the features that people want. Therefore, I would want to model the Environment variable for WAI from Hack. I *could* just import Hack in WAI and use the exact same Environment data type. Thoughts? * If using a different data type for Environment, should I replace the String parts with ByteStrings? On the one hand, ByteStrings are the correct data type since the HTTP protocol does not specify a character encoding; on the other hand, Strings are easier to deal with. * It's simple to write a function to convert between a lazy bytestring and an enumerator, meaning it would be very easy to write conversion functions between Hack and WAI applications. This would make it simpler for people to use either backend. If someone else is already working on WAI, please let me know, I don't want to have duplicate implementations. The idea here is to consolidate, not split the community. I have a few Hack handlers (simpleserver, cgi, fastcgi) that I would happily convert to WAI handlers as well. Michael [1] http://www.haskell.org/haskellwiki/WebApplicationInterface [2] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/hack ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Web application interface
The hyena backend is essentially just a translator between hack and wai, i failed to finished it since I can't understand iteratee (seriously) and eventually got distracted ... What hyena tries to solve can't be realized in hack, so there's not too much reason for a backend anyway. Hyena is especially tuned for streaming and that's exactly what hack can't do (in practice). http://github.com/nfjinjing/hack-handler-hyena -- jinjing ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
