John, This package looks very promising. I used iteratee for the yaml package, but I had many of the concerns that you have mentioned below. Version 0.2 of persistent is going to have some form of an enumerator interface for getting the results of a query, and I eventually decided that iteratee was introducing too much complexity to be a good candidate. However, I was able to port the package[1] over to enumerator in about half an hour; I especially benefited from your example applications.
The only concern that I had was the possible inefficiency of representing all chunks as a list. In the case of persistent, the enumerator will *always* generate a one-lengthed list, and the most common operation is selectList, which returns all results as a list. If I used your consume function, I believe there would be a *lot* of list traversals. Instead, selectList[2] uses ([a] -> [a]) for building up the result internally. I haven't really thought the issue through fully, so I can recommend anything better. Perhaps more importantly, the simplification introduced by just dealing with lists is well received. Keep up the good work, I look forward to seeing more about enumerator. Michael [1] http://github.com/snoyberg/persistent/tree/enumerator [2] http://github.com/snoyberg/persistent/blob/enumerator/Database/Persist/Base.hs#L322 <http://github.com/snoyberg/persistent/tree/enumerator> On Thu, Aug 19, 2010 at 7:31 AM, John Millikin <jmilli...@gmail.com> wrote: > Most of you have probably read Oleg's essays on using left-fold > enumerators for incremental IO. In short, by encapsulating monadic > left-folds in an "Iteratee" type, incremental pure processing is > possible without using lazy IO. Sources to read: > > Oleg: Streams and Iteratees < http://okmij.org/ftp/Streams.html > > Magnus Therning: Trying to work out iteratees < > http://therning.org/magnus/archives/735 > > cdsmith: Iteratees Step By Step (Part 1) < > http://cdsmith.wordpress.com/2010/05/23/iteratees-step-by-step-part-1/ > > > John Millikin (me): Understanding Iteratees < > http://ianen.org/articles/understanding-iteratees/ > > > Currently, the primary package for left-fold enumerators is John > Lato's "iteratee". It is based on Oleg's original code, extended to > support various forms of containers, platform-specific IO, and codecs > for the WAV and TIFF formats. > > While I appreciate Mr. Lato's development of the package, I find it > far too large, and its documentation too sparse, to effectively use. > To correct this, I've written the "enumerator" package. It is also > derived from Oleg's IterateeM.hs , but with a simplified API and > significantly reduced dependency list. > > Hackage entry: http://hackage.haskell.org/package/enumerator > Haddock docs: http://ianen.org/haskell/enumerator/api-docs/ > Source code (literate PDF): > http://ianen.org/haskell/enumerator/enumerator.pdf > > darcs get http://ianen.org/haskell/enumerator/ > > Additionally, I've included examples of using enumerators to implement > simplified versions of the "cat" and "wc" utilities. These should > serve as a useful starting point for anybody who wants to use > enumerators in their own code: > > > http://patch-tag.com/r/jmillikin/enumerator/snapshot/current/content/pretty/Examples/cat.hs > > http://patch-tag.com/r/jmillikin/enumerator/snapshot/current/content/pretty/Examples/wc.hs > > There are already a few libraries using the existing "iteratee" > package (snap, attoparsec-iteratee, hexpat-iteratee); I am very > interested in advice from the authors of these libraries. In > particular, are any of the removed features (ListLike, > WrappedByteString, seeking) something your libraries depend on? Are > there any useful combinators you'd like to see included? > _______________________________________________ > Haskell-Cafe mailing list > Haskell-Cafe@haskell.org > http://www.haskell.org/mailman/listinfo/haskell-cafe >
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
