Re: [Haskell-cafe] How to write Source for TChan working with LC.take?
Thanks! I just read your article. I think your proposal is rational, useful and so brilliant! The new yield/await style would make writing conduits much easier. Thank you again for taking so much time for this problem! On 2012/05/29, at 22:14, Michael Snoyman wrote: OK, after thinking on this for the past week, I've come up with a proposal to make this kind of code easier to write (and more of an explanation on why the behavior was unintuitive in the first place). http://www.yesodweb.com/blog/2012/05/next-conduit-changes Do you think the modified yield/await would be a good solution to the problem? Michael On Mon, May 21, 2012 at 6:07 AM, Michael Snoyman mich...@snoyman.com wrote: I agree that this behavior is non-intuitive, but still believe it's the necessary approach. The short answer to why it's happening is that there's no exit path in the yield version of the function. To understand why, let's expand the code a little bit. Realizing that liftIO = lift . liftIO and lift mr = PipeM (Done Nothing `liftM` mr) (Finalize mr) we can expand the yield version into: sourceTChanYield2 ch = forever $ do let action = liftIO . atomically $ readTChan ch ans - PipeM (Done Nothing `liftM` action) (FinalizeM action) yield ans So the first hint that something is wrong is that the finalize function is calling the action. If you try to change that finalize action into a no-op, e.g.: sourceTChanYield3 :: MonadIO m = TChan a - Source m a sourceTChanYield3 ch = forever $ do let action = liftIO . atomically $ readTChan ch ans - PipeM (Done Nothing `liftM` action) (return ()) yield ans then you get an error message: test.hs:36:53: Could not deduce (a ~ ()) The problem is that, as the monadic binding is set up here, the code says after running the PipeM, I want you to continue by yielding, and then start over again. If you want to expand it further, you can change `forever` into a recursive call, expand `yield`, and then expand all the monadic binding. Every finalization call is forcing things to keep running. And remember: all of this is the desired behavior of conduit, since we want to guarantee finalizers are always called. Imagine that, instead of reading data from a TChan, you were reading from a Handle. In the code above, there was no way to call out to the finalizers. Not sure if all of that rambling was coherent, but here's my recommended solution. What we need is a helper function that allows you to branch based on whether or not it's time to clean up. `lift`, `liftIO`, and monadic bind all perform the same actions regardless of whether or not finalization is being called. The following code, however, works correctly: liftFinal :: Monad m = m a - Finalize m () - (a - Source m a) - Source m a liftFinal action final f = PipeM (liftM f action) final sourceTChanYield :: Show a = MonadIO m = TChan a - Source m a sourceTChanYield ch = liftFinal (liftIO . atomically $ readTChan ch) (return ()) $ \ans - do yield ans sourceTChanYield ch Michael On Sun, May 20, 2012 at 4:22 PM, Hiromi ISHII konn.ji...@gmail.com wrote: Oops, sorry. The last case's behaviour was not as I expected... A correct log is below: ghci sourceTChanRaw ch $$ LC.isolate 10 =$= LC.mapM_ print () () () () () () () () () () ghci sourceTChanState ch $$ LC.isolate 10 =$= LC.mapM_ print () () () () () () () () () () ghci sourceTChanYield ch $$ LC.isolate 10 =$= LC.mapM_ print () () () () () () () () () () *blocks* So again, sourceTChanYield blocks here even if it is already supplied with enough values! -- Hiromi ISHII konn.ji...@gmail.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe -- Hiromi ISHII konn.ji...@gmail.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] How to write Source for TChan working with LC.take?
OK, after thinking on this for the past week, I've come up with a proposal to make this kind of code easier to write (and more of an explanation on why the behavior was unintuitive in the first place). http://www.yesodweb.com/blog/2012/05/next-conduit-changes Do you think the modified yield/await would be a good solution to the problem? Michael On Mon, May 21, 2012 at 6:07 AM, Michael Snoyman mich...@snoyman.com wrote: I agree that this behavior is non-intuitive, but still believe it's the necessary approach. The short answer to why it's happening is that there's no exit path in the yield version of the function. To understand why, let's expand the code a little bit. Realizing that liftIO = lift . liftIO and lift mr = PipeM (Done Nothing `liftM` mr) (Finalize mr) we can expand the yield version into: sourceTChanYield2 ch = forever $ do let action = liftIO . atomically $ readTChan ch ans - PipeM (Done Nothing `liftM` action) (FinalizeM action) yield ans So the first hint that something is wrong is that the finalize function is calling the action. If you try to change that finalize action into a no-op, e.g.: sourceTChanYield3 :: MonadIO m = TChan a - Source m a sourceTChanYield3 ch = forever $ do let action = liftIO . atomically $ readTChan ch ans - PipeM (Done Nothing `liftM` action) (return ()) yield ans then you get an error message: test.hs:36:53: Could not deduce (a ~ ()) The problem is that, as the monadic binding is set up here, the code says after running the PipeM, I want you to continue by yielding, and then start over again. If you want to expand it further, you can change `forever` into a recursive call, expand `yield`, and then expand all the monadic binding. Every finalization call is forcing things to keep running. And remember: all of this is the desired behavior of conduit, since we want to guarantee finalizers are always called. Imagine that, instead of reading data from a TChan, you were reading from a Handle. In the code above, there was no way to call out to the finalizers. Not sure if all of that rambling was coherent, but here's my recommended solution. What we need is a helper function that allows you to branch based on whether or not it's time to clean up. `lift`, `liftIO`, and monadic bind all perform the same actions regardless of whether or not finalization is being called. The following code, however, works correctly: liftFinal :: Monad m = m a - Finalize m () - (a - Source m a) - Source m a liftFinal action final f = PipeM (liftM f action) final sourceTChanYield :: Show a = MonadIO m = TChan a - Source m a sourceTChanYield ch = liftFinal (liftIO . atomically $ readTChan ch) (return ()) $ \ans - do yield ans sourceTChanYield ch Michael On Sun, May 20, 2012 at 4:22 PM, Hiromi ISHII konn.ji...@gmail.com wrote: Oops, sorry. The last case's behaviour was not as I expected... A correct log is below: ghci sourceTChanRaw ch $$ LC.isolate 10 =$= LC.mapM_ print () () () () () () () () () () ghci sourceTChanState ch $$ LC.isolate 10 =$= LC.mapM_ print () () () () () () () () () () ghci sourceTChanYield ch $$ LC.isolate 10 =$= LC.mapM_ print () () () () () () () () () () *blocks* So again, sourceTChanYield blocks here even if it is already supplied with enough values! -- Hiromi ISHII konn.ji...@gmail.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] How to write Source for TChan working with LC.take?
Hello, there. I'm writing a Source to supply values from TChan. I wrote three implementations for that goal as follows: import Data.Conduit import qualified Data.Conduit.List as LC import Control.Monad.Trans import Control.Concurrent.STM import Control.Monad sourceTChanRaw :: MonadIO m = TChan a - Source m a sourceTChanRaw ch = pipe where pipe = PipeM next (return ()) next = do o - liftIO $ atomically $ readTChan ch return $ HaveOutput pipe (return ()) o sourceTChanState :: MonadIO m = TChan a - Source m a sourceTChanState ch = sourceState ch puller where puller ch = StateOpen ch `liftM` (liftIO . atomically $ readTChan ch) sourceTChanYield :: MonadIO m = TChan a - Source m a sourceTChanYield ch = forever $ do ans - liftIO . atomically $ readTChan ch yield ans Namely, one using raw Pipe constructors directly, using `sourceState` and `yield`. I tested these with GHCi. ghci ch - newTChanIO :: IO (TChan ()) ghci atomically $ replicateM_ 1500 $ writeTChan ch () ghci sourceTChanRaw ch $$ LC.take 10 [(),(),(),(),(),(),(),(),(),()] ghci sourceTChanState ch $$ LC.take 10 [(),(),(),(),(),(),(),(),(),()] ghci sourceTChanYield ch $$ LC.take 10 *thread blocks* First two versions' result is what I exactly expected but the last one not: the source written with `yield` never returns value even if there are much enough value. I also realized that following code runs perfectly as I expected: ghci ch - newTChanIO :: IO (TChan ()) ghci atomically $ replicateM_ 1500 $ writeTChan ch () ghci sourceTChanRaw ch $= LC.isolate 10 $$ LC.mapM_ print [(),(),(),(),(),(),(),(),(),()] ghci sourceTChanState ch $= LC.isolate 10 $$ LC.mapM_ print [(),(),(),(),(),(),(),(),(),()] ghci sourceTChanYield ch $= LC.isolate 10 $$ LC.mapM_ print [(),(),(),(),(),(),(),(),(),()] So, here is the question: Why the Source using `yield` doesn't work as expected with LC.take? Or, might be Semantically, what behaviour should be expected for LC.take? Thanks, -- Hiromi ISHII konn.ji...@gmail.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] How to write Source for TChan working with LC.take?
Oops, sorry. The last case's behaviour was not as I expected... A correct log is below: ghci sourceTChanRaw ch $$ LC.isolate 10 =$= LC.mapM_ print () () () () () () () () () () ghci sourceTChanState ch $$ LC.isolate 10 =$= LC.mapM_ print () () () () () () () () () () ghci sourceTChanYield ch $$ LC.isolate 10 =$= LC.mapM_ print () () () () () () () () () () *blocks* So again, sourceTChanYield blocks here even if it is already supplied with enough values! -- Hiromi ISHII konn.ji...@gmail.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] How to write Source for TChan working with LC.take?
I agree that this behavior is non-intuitive, but still believe it's the necessary approach. The short answer to why it's happening is that there's no exit path in the yield version of the function. To understand why, let's expand the code a little bit. Realizing that liftIO = lift . liftIO and lift mr = PipeM (Done Nothing `liftM` mr) (Finalize mr) we can expand the yield version into: sourceTChanYield2 ch = forever $ do let action = liftIO . atomically $ readTChan ch ans - PipeM (Done Nothing `liftM` action) (FinalizeM action) yield ans So the first hint that something is wrong is that the finalize function is calling the action. If you try to change that finalize action into a no-op, e.g.: sourceTChanYield3 :: MonadIO m = TChan a - Source m a sourceTChanYield3 ch = forever $ do let action = liftIO . atomically $ readTChan ch ans - PipeM (Done Nothing `liftM` action) (return ()) yield ans then you get an error message: test.hs:36:53: Could not deduce (a ~ ()) The problem is that, as the monadic binding is set up here, the code says after running the PipeM, I want you to continue by yielding, and then start over again. If you want to expand it further, you can change `forever` into a recursive call, expand `yield`, and then expand all the monadic binding. Every finalization call is forcing things to keep running. And remember: all of this is the desired behavior of conduit, since we want to guarantee finalizers are always called. Imagine that, instead of reading data from a TChan, you were reading from a Handle. In the code above, there was no way to call out to the finalizers. Not sure if all of that rambling was coherent, but here's my recommended solution. What we need is a helper function that allows you to branch based on whether or not it's time to clean up. `lift`, `liftIO`, and monadic bind all perform the same actions regardless of whether or not finalization is being called. The following code, however, works correctly: liftFinal :: Monad m = m a - Finalize m () - (a - Source m a) - Source m a liftFinal action final f = PipeM (liftM f action) final sourceTChanYield :: Show a = MonadIO m = TChan a - Source m a sourceTChanYield ch = liftFinal (liftIO . atomically $ readTChan ch) (return ()) $ \ans - do yield ans sourceTChanYield ch Michael On Sun, May 20, 2012 at 4:22 PM, Hiromi ISHII konn.ji...@gmail.com wrote: Oops, sorry. The last case's behaviour was not as I expected... A correct log is below: ghci sourceTChanRaw ch $$ LC.isolate 10 =$= LC.mapM_ print () () () () () () () () () () ghci sourceTChanState ch $$ LC.isolate 10 =$= LC.mapM_ print () () () () () () () () () () ghci sourceTChanYield ch $$ LC.isolate 10 =$= LC.mapM_ print () () () () () () () () () () *blocks* So again, sourceTChanYield blocks here even if it is already supplied with enough values! -- Hiromi ISHII konn.ji...@gmail.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
