Hello community, here is the log from the commit of package ghc-streaming for openSUSE:Factory checked in at 2017-02-11 01:42:06 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/ghc-streaming (Old) and /work/SRC/openSUSE:Factory/.ghc-streaming.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "ghc-streaming" Changes: -------- --- /work/SRC/openSUSE:Factory/ghc-streaming/ghc-streaming.changes 2016-11-02 12:40:08.000000000 +0100 +++ /work/SRC/openSUSE:Factory/.ghc-streaming.new/ghc-streaming.changes 2017-02-11 01:42:07.764998907 +0100 @@ -1,0 +2,5 @@ +Thu Jan 26 16:20:13 UTC 2017 - [email protected] + +- Update to version 0.1.4.5 with cabal2obs. + +------------------------------------------------------------------- Old: ---- streaming-0.1.4.3.tar.gz New: ---- streaming-0.1.4.5.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ ghc-streaming.spec ++++++ --- /var/tmp/diff_new_pack.N07H43/_old 2017-02-11 01:42:08.520892248 +0100 +++ /var/tmp/diff_new_pack.N07H43/_new 2017-02-11 01:42:08.524891683 +0100 @@ -1,7 +1,7 @@ # # spec file for package ghc-streaming # -# Copyright (c) 2016 SUSE LINUX GmbH, Nuernberg, Germany. +# Copyright (c) 2017 SUSE LINUX GmbH, Nuernberg, Germany. # # All modifications and additions to the file contributed by third parties # remain the property of their copyright owners, unless otherwise agreed @@ -18,15 +18,15 @@ %global pkg_name streaming Name: ghc-%{pkg_name} -Version: 0.1.4.3 +Version: 0.1.4.5 Release: 0 Summary: An elementary streaming prelude and general stream type License: BSD-3-Clause -Group: System/Libraries +Group: Development/Languages/Other Url: https://hackage.haskell.org/package/%{pkg_name} Source0: https://hackage.haskell.org/package/%{pkg_name}-%{version}/%{pkg_name}-%{version}.tar.gz BuildRequires: ghc-Cabal-devel -# Begin cabal-rpm deps: +BuildRequires: ghc-containers-devel BuildRequires: ghc-exceptions-devel BuildRequires: ghc-mmorph-devel BuildRequires: ghc-monad-control-devel @@ -37,57 +37,69 @@ BuildRequires: ghc-transformers-base-devel BuildRequires: ghc-transformers-devel BuildRoot: %{_tmppath}/%{name}-%{version}-build -# End cabal-rpm deps %description -'Streaming.Prelude' exports an elementary streaming prelude focused on a simple -"source" or "producer" type, namely 'Stream (Of a) m r'. 'Stream (Of a) m r' is -a sort of effectful version of '([a],r)' in which successive elements arise -from some sort of monadic action. Everything in the library is organized to -make programming with this type as simple as possible, by the simple expedient -of making it as close to 'Prelude' and 'Data.List' as possible. Thus for -example the trivial program +This package contains two modules, +<http://hackage.haskell.org/package/streaming/docs/Streaming.html Streaming> +and <http://hackage.haskell.org/package/streaming/docs/Streaming-Prelude.html +Streaming.Prelude>. The principal module, +<http://hackage.haskell.org/package/streaming-0.1.4.3/docs/Streaming-Prelude.html +Streaming.Prelude>, exports an elementary streaming prelude focused on a simple +"source" or "producer" type, namely 'Stream (Of a) m r'. This is a sort of +effectful version of '([a],r)' in which successive elements of type 'a' arise +from some sort of monadic action before the succession ends with a value of +type 'r'. Everything in the library is organized to make programming with this +type as simple as possible, by the simple expedient of making it as close to +'Prelude' and 'Data.List' as possible. Thus for example the trivial program -> S.sum (S.take 3 (S.readLn :: Stream (Of Integer) IO ())) +> >>> S.sum $ S.take 3 (S.readLn :: Stream (Of Int) IO ()) > 1<Enter> > +2<Enter> > 3<Enter> > 6 :> () sums the first three valid integers from user input. Similarly, -> S.stdoutLn (S.map reverse (S.take 3 S.stdinLn)) +> >>> S.stdoutLn $ S.map (map toUpper) $ S.take 2 S.stdinLn > hello<Enter> > +HELLO > world!<Enter> > WORLD! -reverses the first three lines from stdin as they arise, and sends them to -stdout. And so on, with filtering, mapping, breaking, chunking and so forth. -We program with streams of 'Int's or 'String's directly as if they constituted -something like a list. And we everywhere oppose "extracting a list from IO", -which is the origin of typical Haskell memory catastrophes. Basically any case -where you are tempted to use 'mapM', 'replicateM', 'traverse' or 'sequence' -with Haskell lists, you would do better to use something like 'Stream (Of a) m -r'. The type signatures are a little fancier, but the programs themselves are -mostly the same or simpler. Thus, consider the trivial demo program mentioned -in <http://stackoverflow.com/questions/24068399/haskell-performance-of-iorefs -this SO question> +upper-cases the first two lines from stdin as they arise, and sends them to +stdout. And so on, with filtering, mapping, breaking, chunking, zipping, +unzipping, replicating and so forth: we program with streams of 'Int's or +'String's directly as if they constituted something like a list. +That's because streams really do constitute something like a list, and the +associated operations can mostly have the same names. (A few, like 'reverse', +don't stream and thus disappear; others like 'unzip' are here given properly +streaming formulation for the first time.) And we everywhere oppose "extracting +a pure list from IO", which is the origin of typical Haskell memory +catastrophes. Basically any case where you are tempted to use 'mapM', +'replicateM', 'traverse' or 'sequence' with Haskell lists, you would do better +to use something like 'Stream (Of a) m r'. The type signatures are a little +fancier, but the programs themselves are mostly the same. /In fact, they are +mostly simpler./ Thus, consider the trivial demo program mentioned in +<http://stackoverflow.com/questions/24068399/haskell-performance-of-iorefs this +SO question> > main = mapM newIORef [1..10^8::Int] >>= mapM readIORef >>= mapM_ print -It quickly exhausts memory, of course, and this has nothing to do with the -efficiency of 'IORefs'. It is immediately cured by writing +The new user notices that this exhausts memory, and worries about the +efficiency of Haskell 'IORefs'. But of course it exhausts memory! Look what it +says! The problem is immediately cured by writing -> import qualified Streaming.Prelude as S > main = S.print (S.mapM readIORef -(S.mapM newIORef (S.each [1..10^8::Int]))) +> main = S.print $ S.mapM readIORef $ S.mapM newIORef $ S.each [1..10^8::Int] which really does what the other program was meant to do, uses no more memory -than 'hello-world', and is simpler anyway, since it doesn't involve "extracting -a list from IO". Almost every use of list 'mapM', 'replicateM', 'traverse' and -'sequence' produces this problem on a smaller scale. People get used to it, as -if it were characteristic of Haskell programs to use a lot of memory, when -"extracting a list or sequence from IO" is just bad practice pure and simple. -Of course, 'mapM', 'replicateM', 'traverse' and 'sequence' make sense for -lists, under certain conditions. Similarly, 'unsafePerformIO' makes sense under -certain conditions. - -The 'Streaming' module exports the general type, 'Stream f m r', which can be -used to stream successive distinct steps characterized by /any/ functor 'f', -though we are mostly interested in organizing computations of the form 'Stream -(Of a) m r'. The streaming-IO libraries have various devices for dealing with +than 'hello-world', /and is simpler anyway/, since it doesn't involve the +detour of "extracting a list from IO". Almost every use of list 'mapM', +'replicateM', 'traverse' and 'sequence' produces this problem on a smaller +scale. People get used to it, as if it were characteristic of Haskell programs +to use a lot of memory. But in truth "extracting a list or sequence from IO" is +mostly just bad practice pure and simple. Of course, 'mapM', 'replicateM', +'traverse' and 'sequence' make sense for lists, under certain conditions! But +'unsafePerformIO' also makes sense under certain conditions. + +The <http://hackage.haskell.org/package/streaming-0.1.4.3/docs/Streaming.html +Streaming> module exports the general type, 'Stream f m r', which can be used +to stream successive distinct steps characterized by /any/ functor 'f', though +we are mostly interested in organizing computations of the form 'Stream (Of a) +m r'. The streaming-IO libraries have various devices for dealing with effectful variants of '[a]' or '([a],r)' in which the emergence of successive elements somehow depends on IO. But it is only with the general type 'Stream f m r', or some equivalent, that one can envisage (for example) the connected @@ -96,22 +108,22 @@ properly streaming equivalents of e.g. > group :: Ord a => [a] -> [[a]] > chunksOf :: Int -> [a] -> [[a]] > lines :: -[Char] -> [[Char]] -- but similarly with bytestring, etc. +[Char] -> [[Char]] -- but similarly with byte streams, etc. to mention a few obviously desirable operations. (This is explained more elaborately in the <https://hackage.haskell.org/package/streaming#readme -readme> below.) One could throw something like 'Stream' on top of a prior -stream concept: this is how 'pipes' and 'pipes-group' (which are very much our -model here) use 'FreeT'. But once one grasps the iterable stream concept needed -to express those functions - the one here given a somewhat optimized -implementation as 'Stream f m r' (the specific optimization again follows the -model of the 'pipes' library) - then one will also see that, with it, one is -/already/ in possession of a complete elementary streaming library - since one -possesses 'Stream ((,) a) m r' or equivalently 'Stream (Of a) m r'. -This is the type of a 'generator' or 'producer' or whatever you call an -effectful stream of items. /The present Streaming.Prelude is thus the simplest -streaming library that can replicate anything like the API of the Prelude and -Data.List/. +readme> below.) + +One could throw of course throw something like the present 'Stream' type on top +of a prior stream concept: this is how 'pipes' and 'pipes-group' (which are +very much our model here) use 'FreeT'. But once one grasps the iterable stream +concept needed to express those functions then one will also see that, with it, +one is /already/ in possession of a complete elementary streaming library - +since one possesses 'Stream ((,) a) m r' or equivalently 'Stream (Of a) m r'. +This is the type of a 'generator' or 'producer' or 'source' or whatever you +call an effectful stream of items. /The present Streaming.Prelude is thus the +simplest streaming library that can replicate anything like the API of the +Prelude and Data.List/. The emphasis of the library is on interoperation; for the rest its advantages are: extreme simplicity, re-use of intuitions the user has gathered from @@ -124,9 +136,9 @@ is to express elementary streaming ideas without reliance on a complex framework, but in a way that integrates transparently with the rest of Haskell, using ideas - e.g. rank 2 types, which are here implicit or explicit in most -mapping - that the user can carry elsewhere, rather than binding her -intelligence to a so-called streaming IO framework (as necessary as that is for -certain purposes.) +mapping - that the user can carry elsewhere, rather than chaining her +understanding to the curiosities of a so-called streaming IO framework (as +necessary as that is for certain purposes.) See the <https://hackage.haskell.org/package/streaming#readme readme> below for further explanation, including the examples linked there. Elementary usage can @@ -199,15 +211,12 @@ %prep %setup -q -n %{pkg_name}-%{version} - %build %ghc_lib_build - %install %ghc_lib_install - %post devel %ghc_pkg_recache ++++++ streaming-0.1.4.3.tar.gz -> streaming-0.1.4.5.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/streaming-0.1.4.3/src/Streaming/Internal.hs new/streaming-0.1.4.5/src/Streaming/Internal.hs --- old/streaming-0.1.4.3/src/Streaming/Internal.hs 2016-06-12 13:01:30.000000000 +0200 +++ new/streaming-0.1.4.5/src/Streaming/Internal.hs 2017-01-17 09:10:24.000000000 +0100 @@ -738,7 +738,7 @@ {-# INLINABLE unexposed #-} -{- Wrap a new layer of a stream. So, e.g. +{-| Wrap a new layer of a stream. So, e.g. > S.cons :: Monad m => a -> Stream (Of a) m r -> Stream (Of a) m r > S.cons a str = wrap (a :> str) @@ -800,14 +800,12 @@ zipsWith phi s t = loop (s,t) where loop (s1, s2) = Effect (go s1 s2) go s1 s2 = do - e <- inspect s1 - case e of - Left r -> return (Return r) - Right fstr -> do - e <- inspect s2 - case e of - Left r -> return (Return r) - Right gstr -> return $ Step $ fmap loop (phi fstr gstr) + e <- inspect s1 + e' <- inspect s2 + case (e,e') of + (Left r, _) -> return (Return r) + (_, Left r) -> return (Return r) + (Right fstr, Right gstr) -> return $ Step $ fmap loop (phi fstr gstr) {-# INLINABLE zipsWith #-} zips :: (Monad m, Functor f, Functor g) @@ -839,13 +837,12 @@ {-| Swap the order of functors in a sum of functors. - ->>> S.toListM' $ S.print $ separate $ maps S.switch $ maps (S.distinguish (=='a')) $ S.each "banana" +>>> S.toList $ S.print $ separate $ maps S.switch $ maps (S.distinguish (=='a')) $ S.each "banana" 'a' 'a' 'a' "bnn" :> () ->>> S.toListM' $ S.print $ separate $ maps (S.distinguish (=='a')) $ S.each "banana" +>>> S.toList $ S.print $ separate $ maps (S.distinguish (=='a')) $ S.each "banana" 'b' 'n' 'n' @@ -859,8 +856,9 @@ {-| Given a stream on a sum of functors, make it a stream on the left functor, with the streaming on the other functor as the governing monad. This is - useful for acting on one or the other functor with a fold. It generalizes - 'Data.Either.partitionEithers' massively, but actually streams properly. + useful for acting on one or the other functor with a fold, leaving the + other material for another treatment. It generalizes + 'Data.Either.partitionEithers', but actually streams properly. >>> let odd_even = S.maps (S.distinguish even) $ S.each [1..10::Int] >>> :t separate odd_even @@ -908,6 +906,8 @@ return {-#INLINABLE separate #-} + + unseparate :: (Monad m, Functor f, Functor g) => Stream f (Stream g m) r -> Stream (Sum f g) m r unseparate str = destroyExposed str diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/streaming-0.1.4.3/src/Streaming/Prelude.hs new/streaming-0.1.4.5/src/Streaming/Prelude.hs --- old/streaming-0.1.4.3/src/Streaming/Prelude.hs 2016-06-12 13:01:30.000000000 +0200 +++ new/streaming-0.1.4.5/src/Streaming/Prelude.hs 2017-01-17 09:10:24.000000000 +0100 @@ -1,14 +1,15 @@ {-| This names exported by this module are closely modeled on those in @Prelude@ and @Data.List@, but also on - <http://hackage.haskell.org/package/pipes-4.1.9/docs/Pipes-Prelude.html @Pipes.Prelude@>, - <http://hackage.haskell.org/package/pipes-group-1.0.3/docs/Pipes-Group.html @Pipes.Group@> - and <http://hackage.haskell.org/package/pipes-parse-3.0.6/docs/Pipes-Parse.html @Pipes.Parse@>. + <http://hackage.haskell.org/package/pipes-4.1.9/docs/Pipes-Prelude.html Pipes.Prelude>, + <http://hackage.haskell.org/package/pipes-group-1.0.3/docs/Pipes-Group.html Pipes.Group> + and <http://hackage.haskell.org/package/pipes-parse-3.0.6/docs/Pipes-Parse.html Pipes.Parse>. The module may be said to give independent expression to the conception of - Producer / Source / Generator manipulation + Producer \/ Source \/ Generator manipulation articulated in the latter two modules. Because we dispense with piping and conduiting, the distinction between all of these modules collapses. Some things are - lost but much is gained in that everything comes much closer to ordinary - beginning Haskell programming. The leading type is chosen to permit an api + lost but much is gained: on the one hand, everything comes much closer to ordinary + beginning Haskell programming and, on the other, acquires the plasticity of programming + directly with a general free monad type. The leading type, @Stream (Of a) m r@ is chosen to permit an api that is as close as possible to that of @Data.List@ and the @Prelude@. Import qualified thus: @@ -57,8 +58,6 @@ -- $producers , yield , each - , each' - , unfoldr , stdinLn , readLn , fromHandle @@ -74,6 +73,8 @@ , enumFrom , enumFromThen , seconds + , unfoldr + -- * Consuming streams of elements @@ -99,7 +100,7 @@ , with , subst , copy - , copy' + , duplicate , store , chain , sequence @@ -122,8 +123,7 @@ , read , show , cons - , duplicate - , duplicate' + , slidingWindow -- * Splitting and inspecting streams of elements @@ -131,14 +131,13 @@ , uncons , splitAt , split --- , breaks + , breaks , break , breakWhen , span , group , groupBy -- , groupedBy - -- , split -- * Sum and Compose manipulation @@ -248,6 +247,7 @@ import Data.Foldable (Foldable) import Data.Traversable (Traversable) import qualified Data.Foldable as Foldable +import qualified Data.Sequence as Seq import Text.Read (readMaybe) import Prelude hiding (map, mapM, mapM_, filter, drop, dropWhile, take, mconcat , sum, product, iterate, repeat, cycle, replicate, splitAt @@ -542,19 +542,19 @@ -- [False] -- -- -} --- breaks --- :: Monad m => --- (a -> Bool) -> Stream (Of a) m r -> Stream (Stream (Of a) m) m r --- breaks thus = loop where --- loop stream = Effect $ do --- e <- next stream --- return $ case e of --- Left r -> Return r --- Right (a, p') -> --- if not (thus a) --- then Step $ fmap loop (yield a >> break thus p') --- else loop p' --- {-#INLINABLE breaks #-} +breaks + :: Monad m => + (a -> Bool) -> Stream (Of a) m r -> Stream (Stream (Of a) m) m r +breaks thus = loop where + loop stream = Effect $ do + e <- next stream + return $ case e of + Left r -> Return r + Right (a, p') -> + if not (thus a) + then Step $ fmap loop (yield a >> break thus p') + else loop p' +{-#INLINABLE breaks #-} {-| Apply an action to all values, re-yielding each @@ -627,7 +627,7 @@ > cycle = forever ->>> rest <- S.print $ S.splitAt 3 $ S.cycle (yield 0 >> yield 1) +>>> rest <- S.print $ S.splitAt 3 $ S.cycle (yield True >> yield False) True False True @@ -757,23 +757,13 @@ 1 2 3 ->>> S.print $ mapped S.toList $ chunksOf 3 $ S.replicateM 5 getLine -s<Enter> -t<Enter> -u<Enter> -["s","t","u"] -v<Enter> -w<Enter> -["v","w"] + -} each :: (Monad m, Foldable.Foldable f) => f a -> Stream (Of a) m () each = Foldable.foldr (\a p -> Step (a :> p)) (Return ()) {-# INLINABLE each #-} -each' :: (Monad m, Foldable.Foldable f) => f a -> Stream (Of a) m () -each' = Foldable.foldr (\a p -> Effect (return (Step (a :> p)))) (Return ()) -{-# INLINABLE each' #-} -- --------------- -- effects @@ -786,6 +776,7 @@ 3 4 5 + 'effects' should be understood together with 'copy' and is subject to the rules > S.effects . S.copy = id @@ -835,7 +826,7 @@ -- ------ {-| An infinite stream of enumerable values, starting from a given value. - It is the same as `S.iterate succ`. + It is the same as @S.iterate succ@. Because their return type is polymorphic, @enumFrom@ and @enumFromThen@ (and @iterate@ are useful for example with @zip@ and @zipWith@, which require the same return type in the zipped streams. @@ -897,12 +888,13 @@ filter pred = loop where loop str = case str of Return r -> Return r - Effect m -> Effect (liftM loop m) + Effect m -> Effect (liftM loop m) Step (a :> as) -> if pred a then Step (a :> loop as) else loop as -{-# INLINABLE filter #-} +{-# INLINE filter #-} -- ~ 10% faster than INLINABLE in simple bench + -- --------------- -- filterM -- --------------- @@ -912,14 +904,13 @@ filterM pred = loop where loop str = case str of Return r -> Return r - Effect m -> Effect $ liftM loop m + Effect m -> Effect $ liftM loop m Step (a :> as) -> Effect $ do bool <- pred a if bool then return $ Step (a :> loop as) else return $ loop as -{-# INLINABLE filterM #-} - +{-# INLINE filterM #-} -- ~ 10% faster than INLINABLE in simple bench -- -- --------------- -- -- first @@ -1355,12 +1346,12 @@ {-| Reduce a stream to its return value with a monadic action. ->>> S.mapM_ Prelude.print $ each [1..5] +>>> S.mapM_ Prelude.print $ each [1..3] 1 2 3 -4 -5 + + >>> rest <- S.mapM_ Prelude.print $ S.splitAt 3 $ each [1..10] 1 2 @@ -1613,6 +1604,7 @@ one<Enter> two<Enter> ["one","two"] + -} repeatM :: Monad m => m a -> Stream (Of a) m r @@ -1627,7 +1619,7 @@ -- replicate -- --------------- --- | Repeat an element several times +-- | Repeat an element several times. replicate :: Monad m => Int -> a -> Stream (Of a) m () replicate n a | n <= 0 = return () replicate n a = loop n where @@ -1635,7 +1627,7 @@ loop m = Effect (return (Step (a :> loop (m-1)))) {-# INLINABLE replicate #-} -{-| Repeat an action several times, streaming the results. +{-| Repeat an action several times, streaming its results. >>> S.print $ S.replicateM 2 getCurrentTime 2015-08-18 00:57:36.124508 UTC @@ -1648,7 +1640,7 @@ loop 0 = Return () loop n = Effect $ do a <- ma - return (Step $ a :> loop (n-1)) + return (Step (a :> loop (n-1))) {-# INLINABLE replicateM #-} {-| Read an @IORef (Maybe a)@ or a similar device until it reads @Nothing@. @@ -1666,7 +1658,8 @@ Just a -> return (Step (a :> loop)) {-# INLINABLE reread #-} -{-| Strict left scan, streaming, e.g. successive partial results. +{-| Strict left scan, streaming, e.g. successive partial results. The seed + is yielded first, before any action of finding the next element is performed. >>> S.print $ S.scan (++) "" id $ each (words "a b c d") @@ -1686,13 +1679,15 @@ -} scan :: Monad m => (x -> a -> x) -> x -> (x -> b) -> Stream (Of a) m r -> Stream (Of b) m r -scan step begin done = loop begin - where +scan step begin done str = Step (done begin :> loop begin str) + where loop !acc stream = do case stream of - Return r -> Step (done acc :> Return r) + Return r -> Return r Effect m -> Effect (liftM (loop acc) m) - Step (a :> rest) -> Step (done acc :> loop (step acc a) rest) + Step (a :> rest) -> + let !acc' = step acc a + in Step (done acc' :> loop acc' rest) {-#INLINABLE scan #-} {-| Strict left scan, accepting a monadic function. It can be used with @@ -1709,23 +1704,22 @@ -} scanM :: Monad m => (x -> a -> m x) -> m x -> (x -> m b) -> Stream (Of a) m r -> Stream (Of b) m r -scanM step begin done str = do - x <- lift begin - loop x str +scanM step begin done str = Effect $ do + x <- begin + b <- done x + return (Step (b :> loop x str)) where - loop !x stream = do - b <- lift (done x) - yield b - case stream of - Return r -> Return r - Effect m -> Effect (do - stream' <- m - return (loop x stream') - ) - Step (a :> rest) -> Effect (do - x' <- step x a - return (loop x' rest) - ) + loop !x stream = case stream of -- note we have already yielded from x + Return r -> Return r + Effect m -> Effect (do + stream' <- m + return (loop x stream') + ) + Step (a :> rest) -> Effect (do + x' <- step x a + b <- done x' + return (Step (b :> loop x' rest)) + ) {-# INLINABLE scanM #-} {- Label each element in a stream with a value accumulated according to a fold. @@ -1785,7 +1779,7 @@ five<Enter> ["one","two","three","four","five"] :> () - This is of course does not interrupt an action that has already begun. + This of course does not interrupt an action that has already begun. -} @@ -2025,14 +2019,20 @@ > mapped toList :: Stream (Stream (Of a)) m r -> Stream (Of [a]) m - Like 'toList_', it breaks streaming; unlike 'toList_' it preserves - the return value and thus is frequently useful with e.g. 'mapped' + Like 'toList_', 'toList' breaks streaming; unlike 'toList_' it /preserves the return value/ + and thus is frequently useful with e.g. 'mapped' >>> S.print $ mapped S.toList $ chunksOf 3 $ each [1..9] [1,2,3] [4,5,6] [7,8,9] - +>>> S.print $ mapped S.toList $ chunksOf 2 $ S.replicateM 4 getLine +s<Enter> +t<Enter> +["s","t"] +u<Enter> +v<Enter> +["u","v"] -} toList :: Monad m => Stream (Of a) m r -> m (Of [a] r) toList = fold (\diff a ls -> diff (a: ls)) id (\diff -> diff []) @@ -2055,29 +2055,32 @@ {-# INLINABLE uncons #-} -{-| Build a @Stream@ by unfolding steps starting from a seed. +{-| Build a @Stream@ by unfolding steps starting from a seed. In particular note + that @S.unfoldr S.next = id@. The seed can of course be anything, but this is one natural way to consume a @pipes@ 'Pipes.Producer'. Consider: ->>> S.stdoutLn $ S.take 2 $ S.unfoldr P.next P.stdinLn +>>> S.stdoutLn $ S.take 2 $ S.unfoldr Pipes.next Pipes.stdinLn hello<Enter> hello goodbye<Enter> goodbye ->>> S.stdoutLn $ S.unfoldr P.next (P.stdinLn P.>-> P.take 2) +>>> S.stdoutLn $ S.unfoldr Pipes.next (Pipes.stdinLn >-> Pipes.take 2) hello<Enter> hello goodbye<Enter> goodbye ->>> S.effects $ S.unfoldr P.next (P.stdinLn P.>-> P.take 2 P.>-> P.stdoutLn) +>>> S.effects $ S.unfoldr Pipes.next (Pipes.stdinLn >-> Pipes.take 2 >-> Pipes.stdoutLn) hello<Enter> hello goodbye<Enter> goodbye + @Pipes.unfoldr S.next@ similarly unfolds a @Pipes.Producer@ from a stream. + -} unfoldr :: Monad m => (s -> m (Either r (a, s))) -> s -> Stream (Of a) m r @@ -2137,18 +2140,17 @@ >>> stdoutLn $ yield "hello" hello ->>> S.sum $ do {yield 1; yield 2} -3 +>>> S.sum $ do {yield 1; yield 2; yield 3} +6 ->>> let prompt = putStrLn "Enter a number:" ->>> let number = lift (prompt >> readLn) >>= yield :: Stream (Of Int) IO () +>>> let number = lift (putStrLn "Enter a number:") >> lift readLn >>= yield :: Stream (Of Int) IO () >>> S.toList $ do {number; number; number} Enter a number: -1 +1<Enter> Enter a number: -2 +2<Enter> Enter a number: -3 +3<Enter> [1,2,3] :> () -} @@ -2222,8 +2224,7 @@ -- IO fripperies -- -------------- -{-| View standard input as a 'Stream (Of String) m r'. 'stdoutLn', by - contrast, renders a 'Stream (Of String) m r' to standard output. The names +{-| View standard input as a @Stream (Of String) m r@. By contrast, 'stdoutLn' renders a @Stream (Of String) m r@ to standard output. The names follow @Pipes.Prelude@ >>> stdoutLn stdinLn @@ -2246,26 +2247,28 @@ stdinLn = fromHandle IO.stdin {-# INLINABLE stdinLn #-} -{-| Read values from 'IO.stdin', ignoring failed parses +{-| Read values from 'IO.stdin', ignoring failed parses. ->>> S.sum_ $ S.take 2 S.readLn :: IO Int +>>> :set -XTypeApplications +>>> S.sum $ S.take 2 (S.readLn @IO @Int) 10<Enter> 12<Enter> -22 +22 :> () ->>> S.toList $ S.take 3 (S.readLn :: Stream (Of Int) IO ()) -1<Enter> -2<Enter> +>>> S.toList $ S.take 2 (S.readLn @IO @Int) +10<Enter> 1@#$%^&*\<Enter> -3<Enter> -[1,2,3] :> () +12<Enter> +[10,12] :> () -} readLn :: (MonadIO m, Read a) => Stream (Of a) m () -readLn = for stdinLn $ \str -> case readMaybe str of - Nothing -> return () - Just n -> yield n +readLn = do + str <- liftIO getLine + case readMaybe str of + Nothing -> readLn + Just n -> yield n >> readLn {-# INLINABLE readLn #-} @@ -2292,7 +2295,7 @@ {-| Write a succession of strings to a handle as separate lines. ->>> S.toHandle IO.stdout $ each $ words "one two three" +>>> S.toHandle IO.stdout $ each (words "one two three") one two three @@ -2310,9 +2313,9 @@ {-| Print the elements of a stream as they arise. >>> S.print $ S.take 2 S.stdinLn -hello +hello<Enter> "hello" -world +world<Enter> "world" >>> @@ -2321,14 +2324,14 @@ print = loop where loop stream = case stream of Return r -> return r - Effect m -> m >>= loop + Effect m -> m >>= loop Step (a :> rest) -> do liftIO (Prelude.print a) loop rest {-| Write 'String's to 'IO.stdout' using 'putStrLn'; terminates on a broken output pipe - (This operation is modelled on 'Pipes.Prelude.stdoutLn'). + (The name and implementation are modelled on the @Pipes.Prelude@ @stdoutLn@). >>> S.stdoutLn $ S.take 3 $ S.each $ words "one two three four five" one @@ -2354,29 +2357,22 @@ - {-| Write 'String's to 'IO.stdout' using 'putStrLn' - This does not handle a broken output pipe, but has a polymorphic return - value, which makes this possible: + Unlike @stdoutLn@, @stdoutLn'@ does not handle a broken output pipe. Thus it can have a polymorphic return + value, rather than @()@, and this kind of \"connect and resume\" is possible: >>> rest <- S.stdoutLn' $ S.show $ S.splitAt 3 (each [1..5]) 1 2 3 ->>> S.print rest -4 -5 +>>> S.toList rest +[4,5] :> () -} stdoutLn' :: MonadIO m => Stream (Of String) m r -> m r -stdoutLn' = loop where - loop stream = case stream of - Return r -> return r - Effect m -> m >>= loop - Step (s :> rest) -> liftIO (putStrLn s) >> loop rest -{-# INLINE stdoutLn' #-} +stdoutLn' = toHandle IO.stdout {-| Read the lines of a file as Haskell 'String's @@ -2398,15 +2394,15 @@ readFile :: MonadResource m => FilePath -> Stream (Of String) m () readFile f = bracketStream (IO.openFile f IO.ReadMode) (IO.hClose) fromHandle -{-| Write a series of strings as lines to a file. The handle is crudely - managed with 'ResourceT': +{-| Write a series of strings as lines to a file. The handle is + managed with 'ResourceT' (see the remarks on 'readFile'): >>> runResourceT $ S.writeFile "lines.txt" $ S.take 2 S.stdinLn hello<Enter> world<Enter> ->>> runResourceT $ S.print $ S.readFile "lines.txt" -"hello" -"world" +>>> runResourceT $ S.stdoutLn $ S.readFile "lines.txt" +hello +world -} writeFile :: MonadResource m => FilePath -> Stream (Of String) m r -> m r @@ -2577,9 +2573,9 @@ > instance (Functor f, MonadIO m) => MonadIO (Stream f m) We thus can't be touching the elements of the stream, or the final return value. - It it is the same with other constraints that @Stream (Of a)@ inherits, - like 'MonadResource'. Thus I can filter and write to one file, but - nub and write to another, or to a database or the like: + It is the same with other constraints that @Stream (Of a)@ inherits from the underlying monad, + like 'MonadResource'. Thus I can independently filter and write to one file, but + nub and write to another, or interact with a database and a logfile and the like: >>> runResourceT $ (S.writeFile "hello2.txt" . S.nub) $ store (S.writeFile >>> "hello.txt" . S.filter (/= "world")) $ each ["hello", "world", "goodbye", >>> "world"] >>> :! cat hello.txt @@ -2608,7 +2604,7 @@ one two - With copy, I can as well do: + With copy, I can do these simultaneously: >>> S.print $ S.stdoutLn $ S.copy $ each ["one","two"] one @@ -2675,7 +2671,6 @@ Return r -> Return r Effect m -> Effect (liftM loop (lift m)) Step (a :> rest) -> Effect (Step (a :> Return (Step (a :> loop rest)))) - {-#INLINABLE copy#-} duplicate @@ -2684,27 +2679,6 @@ duplicate = copy {-#INLINE duplicate #-} - -{-| @copy'@ is the same as @copy@ but reverses the order of interleaved effects. - The difference should not be observable at all for pure folds over the data. - --} -copy' - :: Monad m => - Stream (Of a) m r -> Stream (Of a) (Stream (Of a) m) r -copy' = Effect . return . loop where - loop str = case str of - Return r -> Return r - Effect m -> Effect (liftM loop (lift m)) - Step (a :> rest) -> Step (a :> Effect (Step (a :> Return (loop rest)))) -{-#INLINABLE copy' #-} - -duplicate' - :: Monad m => - Stream (Of a) m r -> Stream (Of a) (Stream (Of a) m) r -duplicate' = copy' -{-#INLINE duplicate' #-} - {-| The type > Data.List.unzip :: [(a,b)] -> ([a],[b]) @@ -2761,15 +2735,6 @@ {-#INLINABLE unzip #-} --- "fold/map" forall step begin done f str . --- fold step begin done (map f str) = fold (\x a -> step x $! f a) begin done str; --- --- "fold/filter" forall step begin done pred str . --- fold step begin done (filter pred str) = fold (\x a -> if pred a then step x a else x) begin done str; --- --- "scan/map" forall step begin done f str . --- scan step begin done (map f str) = scan (\x a -> step x $! f a) begin done str --- {- $maybes These functions discard the 'Nothing's that they encounter. They are analogous @@ -2777,7 +2742,8 @@ -} {-| The 'catMaybes' function takes a 'Stream' of 'Maybe's and returns - a 'Stream' of all of the 'Just' values. + a 'Stream' of all of the 'Just' values. 'concat' has the same behavior, + but is more general; it works for any foldable container type. -} catMaybes :: Monad m => Stream (Of (Maybe a)) m r -> Stream (Of a) m r catMaybes = loop where @@ -2792,6 +2758,7 @@ {-| The 'mapMaybe' function is a version of 'map' which can throw out elements. In particular, the functional argument returns something of type @'Maybe' b@. If this is 'Nothing', no element is added on to the result 'Stream'. If it is @'Just' b@, then @b@ is included in the result 'Stream'. + -} mapMaybe :: Monad m => (a -> Maybe b) -> Stream (Of a) m r -> Stream (Of b) m r mapMaybe phi = loop where @@ -2803,3 +2770,37 @@ Just b -> Step (b :> loop snext) {-#INLINABLE mapMaybe #-} +{-| 'slidingWindow' accumulates the first @n@ elements of a stream, + update thereafter to form a sliding window of length @n@. + It follows the behavior of the slidingWindow function in + <https://hackage.haskell.org/package/conduit-combinators-1.0.4/docs/Data-Conduit-Combinators.html#v:slidingWindow conduit-combinators>. + +>>> S.print $ slidingWindow 4 $ S.each "123456" +fromList "1234" +fromList "2345" +fromList "3456" + +-} + +slidingWindow :: Monad m + => Int + -> Stream (Of a) m b + -> Stream (Of (Seq.Seq a)) m b +slidingWindow n = setup (max 1 n :: Int) mempty + where + window !sequ str = do + e <- lift (next str) + case e of + Left r -> return r + Right (a,rest) -> do + yield (sequ Seq.|> a) + window (Seq.drop 1 sequ Seq.|> a) rest + setup 0 !sequ str = do + yield sequ + window (Seq.drop 1 sequ) str + setup n sequ str = do + e <- lift $ next str + case e of + Left r -> yield sequ >> return r + Right (x,rest) -> setup (n-1) (sequ Seq.|> x) rest +{-#INLINABLE slidingWindow #-} diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/streaming-0.1.4.3/streaming.cabal new/streaming-0.1.4.5/streaming.cabal --- old/streaming-0.1.4.3/streaming.cabal 2016-06-12 13:01:30.000000000 +0200 +++ new/streaming-0.1.4.5/streaming.cabal 2017-01-17 09:10:24.000000000 +0100 @@ -1,60 +1,76 @@ name: streaming -version: 0.1.4.3 +version: 0.1.4.5 cabal-version: >=1.10 build-type: Simple synopsis: an elementary streaming prelude and general stream type. -description: @Streaming.Prelude@ exports an elementary streaming prelude focused on +description: This package contains two modules, <http://hackage.haskell.org/package/streaming/docs/Streaming.html Streaming> + and <http://hackage.haskell.org/package/streaming/docs/Streaming-Prelude.html Streaming.Prelude>. + The principal module, <http://hackage.haskell.org/package/streaming-0.1.4.3/docs/Streaming-Prelude.html Streaming.Prelude>, exports an elementary streaming prelude focused on a simple \"source\" or \"producer\" type, namely @Stream (Of a) m r@. - @Stream (Of a) m r@ is a sort of effectful version of - @([a],r)@ in which successive elements arise from some sort of monadic - action. Everything in the library is organized to make + This is a sort of effectful version of + @([a],r)@ in which successive elements of type @a@ arise from some sort of monadic + action before the succession ends with a value of type @r@. + Everything in the library is organized to make programming with this type as simple as possible, by the simple expedient of making it as close to @Prelude@ and @Data.List@ as possible. Thus for example the trivial program . - > S.sum (S.take 3 (S.readLn :: Stream (Of Integer) IO ())) + > >>> S.sum $ S.take 3 (S.readLn :: Stream (Of Int) IO ()) + > 1<Enter> + > 2<Enter> + > 3<Enter> + > 6 :> () . sums the first three valid integers from user input. Similarly, . - > S.stdoutLn (S.map reverse (S.take 3 S.stdinLn)) + > >>> S.stdoutLn $ S.map (map toUpper) $ S.take 2 S.stdinLn + > hello<Enter> + > HELLO + > world!<Enter> + > WORLD! . - reverses the first three lines from stdin as they arise, + upper-cases the first two lines from stdin as they arise, and sends them to stdout. And so on, - with filtering, mapping, breaking, chunking and so forth. - We program with streams of @Int@s or @String@s directly as - if they constituted something like a list. And we everywhere - oppose \"extracting a list from IO\", + with filtering, mapping, breaking, chunking, zipping, unzipping, replicating + and so forth: + we program with streams of @Int@s or @String@s directly as + if they constituted something like a list. That's because streams really do constitute something + like a list, and the associated operations can mostly have the same names. + (A few, like @reverse@, don't stream and thus disappear; + others like @unzip@ are here given properly streaming formulation for the first time.) + And we everywhere + oppose \"extracting a pure list from IO\", which is the origin of typical Haskell memory catastrophes. Basically any case where you are tempted to use @mapM@, @replicateM@, @traverse@ or @sequence@ with Haskell lists, you would do better to use something like @Stream (Of a) m r@. The type signatures are a little fancier, but - the programs themselves are mostly the same or simpler. Thus, + the programs themselves are mostly the same. /In fact, they are mostly simpler./ Thus, consider the trivial demo program mentioned in <http://stackoverflow.com/questions/24068399/haskell-performance-of-iorefs this SO question> . > main = mapM newIORef [1..10^8::Int] >>= mapM readIORef >>= mapM_ print . - It quickly exhausts memory, of course, and this has nothing to do with - the efficiency of @IORefs@. It is immediately cured by writing + The new user notices that this exhausts memory, and worries about the efficiency of Haskell @IORefs@. + But of course it exhausts memory! Look what it says! + The problem is immediately cured by writing . - > import qualified Streaming.Prelude as S - > main = S.print (S.mapM readIORef (S.mapM newIORef (S.each [1..10^8::Int]))) + > main = S.print $ S.mapM readIORef $ S.mapM newIORef $ S.each [1..10^8::Int] . which really does what the other program was meant to do, - uses no more memory than @hello-world@, and is simpler anyway, since it - doesn't involve \"extracting a list from IO\". Almost + uses no more memory than @hello-world@, /and is simpler anyway/, since it + doesn't involve the detour of \"extracting a list from IO\". Almost every use of list @mapM@, @replicateM@, @traverse@ and @sequence@ produces this problem on a smaller scale. People get used to it, as if it were - characteristic of Haskell programs to use a lot of memory, when - \"extracting a list or sequence from IO\" is just bad practice pure and simple. + characteristic of Haskell programs to use a lot of memory. But in truth + \"extracting a list or sequence from IO\" is mostly just bad practice pure and simple. Of course, @mapM@, @replicateM@, @traverse@ and @sequence@ make sense for lists, - under certain conditions. Similarly, @unsafePerformIO@ makes sense under + under certain conditions! But @unsafePerformIO@ also makes sense under certain conditions. . - The @Streaming@ module exports the general type, + The <http://hackage.haskell.org/package/streaming-0.1.4.3/docs/Streaming.html Streaming> module exports the general type, @Stream f m r@, which can be used to stream successive distinct steps characterized by /any/ functor @f@, though we are mostly interested in organizing computations @@ -70,23 +86,21 @@ . > group :: Ord a => [a] -> [[a]] > chunksOf :: Int -> [a] -> [[a]] - > lines :: [Char] -> [[Char]] -- but similarly with bytestring, etc. + > lines :: [Char] -> [[Char]] -- but similarly with byte streams, etc. . to mention a few obviously desirable operations. (This is explained more elaborately in the <https://hackage.haskell.org/package/streaming#readme readme> below.) - One could throw something - like @Stream@ on top of a prior stream concept: this is how @pipes@ and + . + One could throw of course throw something + like the present @Stream@ type on top of a prior stream concept: this is how @pipes@ and @pipes-group@ (which are very much our model here) use @FreeT@. But once one grasps the iterable stream concept needed to express - those functions - - the one here given a somewhat optimized implementation as @Stream f m r@ - (the specific optimization again follows the model of the @pipes@ library) - - then one will also see that, + those functions then one will also see that, with it, one is /already/ in possession of a complete elementary streaming library - since one possesses @Stream ((,) a) m r@ or equivalently @Stream (Of a) m r@. This - is the type of a \'generator\' or \'producer\' or whatever - you call an effectful stream of items. + is the type of a \'generator\' or \'producer\' or \'source\' or whatever + you call an effectful stream of items. /The present Streaming.Prelude is thus the simplest streaming library that can replicate anything like the API of the Prelude and Data.List/. . The emphasis of the library is on interoperation; for @@ -103,8 +117,8 @@ a complex framework, but in a way that integrates transparently with the rest of Haskell, using ideas - e.g. rank 2 types, which are here implicit or explicit in most mapping - that the user can carry elsewhere, - rather than binding her intelligence to a so-called streaming IO framework (as - necessary as that is for certain purposes.) + rather than chaining her understanding to the curiosities of + a so-called streaming IO framework (as necessary as that is for certain purposes.) . See the <https://hackage.haskell.org/package/streaming#readme readme> @@ -206,6 +220,7 @@ , monad-control >=0.3.1 && <1.1 , time , ghc-prim + , containers hs-source-dirs: src default-language: Haskell2010
