Re: [Haskell-cafe] Conduit Best Practices for leftover data

2012-04-15 Thread Myles C. Maxfield
Thanks for responding to this. Some responses are inline. On Sat, Apr 14, 2012 at 8:30 PM, Michael Snoyman mich...@snoyman.com wrote: On Thu, Apr 12, 2012 at 9:25 AM, Myles C. Maxfield myles.maxfi...@gmail.com wrote: Hello, I am interested in the argument to Done, namely, leftover data. More

Re: [Haskell-cafe] Conduit Best Practices for leftover data

2012-04-15 Thread Myles C. Maxfield
2. If you use connect-and-resume ($$+), the leftovers are returned as part of the `Source`, and provided downstream. I'm trying to figure out how to use this, but I'm getting a little bit confused. In particular, here is a conduit that produces an output for every 'i' inputs. I'm returning

Re: [Haskell-cafe] Conduit Best Practices for leftover data

2012-04-15 Thread Myles C. Maxfield
Sorry for the spam. A similar matter is this following program, where something downstream reaches EOF right after a conduit outputs a HaveOutput. Because the type of the early-closed function is just 'r' or 'm r', there is no way for the conduit to return any partial output. This means that any

Re: [Haskell-cafe] Conduit Best Practices for leftover data

2012-04-15 Thread Michael Snoyman
On Mon, Apr 16, 2012 at 12:12 AM, Myles C. Maxfield myles.maxfi...@gmail.com wrote: Thanks for responding to this. Some responses are inline. On Sat, Apr 14, 2012 at 8:30 PM, Michael Snoyman mich...@snoyman.com wrote: [snip] No, nothing so complicated is intended. Most likely you'll never

Re: [Haskell-cafe] Conduit Best Practices for leftover data

2012-04-15 Thread Michael Snoyman
I'm not really certain of your previous example, but what you're describing here is data loss. AFAIK, this is inherent to any kind of streaming approach. Have a look at the chapter in the Yesod book on conduit, the description of data loss is still accurate. On Mon, Apr 16, 2012 at 3:46 AM, Myles

Re: [Haskell-cafe] Conduit Best Practices for leftover data

2012-04-14 Thread Michael Snoyman
On Thu, Apr 12, 2012 at 9:25 AM, Myles C. Maxfield myles.maxfi...@gmail.com wrote: Hello, I am interested in the argument to Done, namely, leftover data. More specifically, when implementing a conduit/sink, what should the conduit specify for the (Maybe i) argument to Done in the following

[Haskell-cafe] Conduit Best Practices for leftover data

2012-04-12 Thread Myles C. Maxfield
Hello, I am interested in the argument to Done, namely, leftover data. More specifically, when implementing a conduit/sink, what should the conduit specify for the (Maybe i) argument to Done in the following scenarios (Please note that these scenarios only make sense if the type of 'i' is