I'm not sure how true this is now, but it used to be true that iterators were much more efficient than tasks/coroutines for roughly equivalent algorithms (at least for, e.g., most of the combinatorics routines).
Cheers, Kevin On Sat, Mar 8, 2014 at 5:36 AM, Mike Innes <[email protected]> wrote: > Ok, fair enough - I think the confusion for me lies in the fact that I > wouldn't have said that Julia has "lazy lists, tasks and iterators", in the > same way that I wouldn't say it has "floats, integers and numbers", because > the former two are just types of the latter. But now I think I understand > that by "iterator" you mean "iterator implementation via a custom type" - > like the Take and Repeat types that Iterators.jl uses. Right? Also, I want > to separate the idea of "tasks" and "generators", because "tasks" are just > coroutines - they can be used to make generators, as you have, but it's not > their only purpose. > > I think I'm in agreement with you that iterators, in that sense, are best > reserved for when they have a specific purpose (like Ranges, for example). > I'm not convinced that the Iterators.jl style is the best idea myself, so > lets leave that alone for now. Then it comes down to generators and lazy > sequences, which as you've pointed out are two different ways to solve the > same problem. > > As I've mentioned, these are both reflections of two very different styles > of programming, procedural vs. functional. In my view, the fact that > different people have different tastes is *exactly *the reason to support > both paradigms, as opposed to deciding on "one true way" for everyone. That > article, while it doesn't apply 1:1 to our discussion, also looks at the > idea that in many cases one style is objectively preferable to another - in > which case, it only make sense for Julia to support both. > > I'd be interested to see the tree-walking iterator mentioned in the > article implemented via a task. I could be wrong, but I imagine it would be > reasonably difficult compared to the lazy sequence version. Equally, I > don't know of anything that's harder with sequences than with generators, > so if you can think of anything I'd be interested in having a go at it. > > > On 8 March 2014 11:44, andrew cooke <[email protected]> wrote: > >> >> i realise that in julia iterators are a protocol (that they rely on >> start, done and next, and that the underlying type used to "do" the >> iteration depends on what is being iterated over). but that's not true in >> python, for example, where all iterators are implemented as coroutines. >> the only reason i can see for julia adding a separate mechanism for >> iterators separate from tasks is efficiency - it's less work to use the >> iterator protocol to effectively manage an integer than to have a task. or >> maybe it's that consume is explicit in julia while it's not in python, so >> tasks look uglier in julia? >> >> to me this seems confusing. for example, it would be nice to have >> something that takes a task and generates a new task than is the contents >> of the old task repeated? but the repeat() function in Iterators.jl >> doesn't do that. instead it gives you an iterator. i don't know if this >> matters in practice - i haven't use tasks and iterators enough - but it >> seems like a mess. why two different things? >> >> similarly, i understand, i think, that both lazy streams and tasks are >> implemented differently. but a task that produces tasks doesn't give me a >> headache any more than lazy streams of lazy streams. in fact tasks >> generally seem simpler (to me) because you don't have to worry about making >> the flow work nicely - you can just bail out with a produce. but maybe >> it's just that i am more used to python than to scheme. again, why two >> different things? just because you are used to programming in scheme and i >> am used to python? that's not a great answer in my book. >> >> (and the task version of take doesn't require 20 lines, for example - >> https://github.com/andrewcooke/BlockCipherSelfStudy.jl/blob/master/src/Tasks.jl#L5) >> >> someone else has pointed me to >> http://journal.stuffwithstuff.com/2013/01/13/iteration-inside-and-out/which >> i haven't read yet, saying that it explains the difference between >> iterators and tasks. maybe that will help me. >> >> thinking more about this last night i did realise that my instinctive >> aversion to having lots of ways to do the same thing isn't necessarily >> reasonable in julia. in a sense, what does it matter if julia has lazy >> streams, tasks and iterators, if they all use the same names for >> functions? because then you can swap types out and code will still work. >> so i guess the cost to have take defined for iterators, and for tasks and >> for lazy streams is less than i imagined. >> >> andrew >> >> >> On Saturday, 8 March 2014 08:06:33 UTC-3, Mike Innes wrote: >>> >>> So, to clarify, Iterators aren't a thing in themselves. Iteration is an >>> interface, and to call something an iterator just means that you can put it >>> in a for loop. Tasks and Lazy Lists are both iterators; so are arrays, >>> sets, dictionaries, and a whole bunch of other things. But although you can >>> use them in a similar way if you want to, they are all designed to solve >>> very different problems. >>> >>> Now, Tasks and Lazy Lists do look similar in that you can produce and >>> consume a stream of values with both, but conceptually they are quite >>> different - Tasks are a mechanism for control flow, whereas Lazy Lists are >>> a data structure. Perhaps you could call them the procedural and functional >>> analogies of each other. I can't tell you what's best for you, but if >>> you're thinking of Tasks as representing a sequence of data, then there's a >>> good chance you'll find Lazy Lists easier to reason about. >>> >>> For example, consider the partition() function. In Lazy.jl terms this >>> splits a single list into a list of lists - it's fairly easy to visualise >>> this: >>> >>> > partition(3, seq(1:9)) >>> List: >>> (1 2 3) >>> (4 5 6) >>> (7 8 9) >>> >>> If you wanted to write partition() for Tasks, you'd end up with tasks >>> that produce tasks. I don't know about you, but that gives me a headache. >>> >>> You'll also notice that working with general iterators takes a lot of >>> work; consider the Iterators.jl version of take(), which takes about twenty >>> lines, versus the two-liner in Lazy.jl. Some things are simply impossible >>> to do generically, like flatten(). >>> >>> That's not to say that Tasks aren't useful - they're better if you want >>> to do more things in terms of control flow and less in terms of >>> manipulating the data itself, for example. Both Tasks and Lazy Lists are >>> extremely powerful, but each within their own scope - hence it's useful to >>> have both. >>> >>> Is this roughly what you were looking for? Let me know if I've missed >>> anything. >>> >> >
