Hindsight, I probably sent that last message prematurely. -----
Here's some of the things I learned while making that, among dabbling with other things: The problem domain of working with collections is hard. Generically modifying them is also hard. It wasn't until we got OO until we learned how to iterate an immediate collection (sets, arrays, maps). With interfaces, synchronous iteration\* becomes easy, since you can only iterate a collection all at once, and there's a predetermined order that's easy to configure. - Arrays can be easily iterated in either direction, by simply moving a pointer where you want it. - Sets and maps are a bit more complex, but they still have all the values immediately ready. - Generators are also a little more complex, but this is where the interface makes things easy. Really, all you need for sync iteration is: - A `.next()` method that returns either "done" (with optional value) or "not done" (with required value). - A method to create a collection of that type from a list (if you plan to allow mapping). Now, with async stuff, it's all nice to support iteration more generally, but there's a few other things that need to be included: - A way to initialize lazy values (lazy properties), collections (Lodash wrappers, observables), etc. - A way to break iteration. Both of these are required for sync iterators already (think: `.next()` + `.return()`, where `.next()` also forcibly initializes the collection), but with async collections, it's not always as simple as that. With traditional `.next()`-based iterators, you can just stop requesting subsequent values (`.return()` is really just best-effort notification of "not needed" for resource cleanup), but with observables, you have to have an explicit notification mechanism in place, so it knows to stop sending you things and so you know to start ignoring what you receive. There's another issue, too: sync iterators can't do things between iterations, but async ones *could*. The obvious result would be to allow the user to specify whether to handle the results in parallel, but not everything supports that (like async coroutines). Also, the user can't always handle results as quickly as the "iterator" produces them. This frequently is the case with observables listening to spammy event emitters, and is why RxJS has `.debounce(ms)`. Now that we've gotten that out of the way, what now? Well...what about mapping? How do we go from a bunch of As to a bunch of Bs? This is where procedural OO starts to falter: - We find ourselves building a set of Bs as we iterate the As, and we return that. That's a bunch of boilerplate. - We can't generically create that set of Bs without knowing the type of As we have. Interfaces only type *values*, not *types*. We instead need to look to the functional and type-level stuff, where this once again becomes easy. - A `.map(f)` method, which for each `foo` in it, calls `f` with it, and returns a new collection with all the results. That is magical. It simplifies a lot. We don't have syntax to capture it yet in JS, but it's a natural extension of what we already have for promises. In fact, promise `.then` isn't far off, except there's a catch: we also need a way to flatten a structure. This is where we need a new method, to go from a collection of As to a collection of Bs while transforming a single A to a collection of Bs. - A `.chain(f)` method, which for each `foo` in it, calls `f` with it, and returns a new collection with all the values within the results. Now, this seems very convenient, and it's very well typed. But this isn't very incredibly convenient: we might only want to flatten it sometimes, and promises are a good example of this (it's why `.then` implicitly unwraps promises). A common solution in the functional world is to always flatten (giving rise to a `.constructor.of`), and it simplifies it on paper, but it doesn't simplify it for the user. Instead, we could use two methods to do this, that are a little looser in their constraints: - A `.lift(f)` method, which for each `foo` in it, calls `f` with it, and returns a new collection with all the results, optionally flattened. - A `.chain(f)` method, which does the same, but `f` instead returns either a wrapped instance or an array of 0 or more values to be wrapped. This allows types to more easily be mapped over, but there's a critical thing this version of `.chain` enables us to do: it allows us to filter and otherwise manipulate the collection, without being dependent on the caller's type (it doesn't even need to have a `.constructor` property). Filtering would equate to `.chain(x => f(x) ? [x] : [])`. If you read the proposal, I wrote a basic implementation of the `.distinctBy(func)` operator commonly found in observable implementations. This does not address the issue of executing lazily allocated pipelines, but that's required for iteration. If there's nothing to lift over, there's conceptually no need to iterate to fulfill the required interface contract. This necessitates a new operation: - An `.each(f)` method, which initializes the pipeline and calls `f` with each result, breaks if `f` returns falsy, and optionally returns a promise resolved when done. This enables iteration as well as breaking early. This doesn't cover the scenario when you want to break from outside the loop (think: external consumer closed abruptly. `.return()`/`.throw()` on iterators), but it does cover the scenario of breaking within the loop. Or to put it another way, iterables are hard, mapping is harder, and chaining, it's complicated. There's just so many edge cases I could write a book about it (and I'm pretty sure there already *has* been one written already, just from the common nature of the topic). Or to put it another way, pipelines are hard. Looping is hard. Language design is hard. Or to put it another way, I have no clue what the hell I'm doing, and I'm just trying to figure it out as I go along. Diving head-first into fully opaque water is always fun. :-) On Fri, Mar 23, 2018 at 7:55 PM, Isiah Meadows <[email protected]> wrote: > I've already looked into this kind of thing myself privately. I came > up with this last year [1], and I more recently came up with this [2] > (the second is much better IMHO). Both of those offer solutions to > this problem, and they do in fact offer ways for 1. iteration, and 2. > mapping/filtering/etc. > > [1]: https://github.com/isiahmeadows/non-linear-proposal > [2]: https://github.com/isiahmeadows/lifted-pipeline-strawman > ----- > > Isiah Meadows > [email protected] > > Looking for web consulting? Or a new website? > Send me an email and we can get started. > www.isiahmeadows.com > > > On Fri, Mar 23, 2018 at 11:06 AM, Thomas Grainger <[email protected]> wrote: >> You can convert an observable into an async iterator. You have to choose >> between discarding or buffering uniterated items >> >> On 23 Mar 2018 14:39, "Bob Myers" <[email protected]> wrote: >>> >>> Could someone jog my memory about proposals for better syntax for >>> observable mapping and subscribing, if any? >>> >>> I'm getting really tired of writing >>> >>> ``` >>> foo$.pipe(map(bar => mapper(bar))) >>> ``` >>> >>> I would much prefer to write something along the lines of >>> >>> ``` >>> stream function fooMapper(foo$) { >>> while async (const bar = foo$()) { >>> emit mapper(bar); >>> } >>> } >>> ``` >>> >>> Yes, I'm aware of all the potential issues here and this is just an >>> example, not an actual syntax proposal. I'm just wondering about any prior >>> art. >>> >>> Bob >>> >>> >>> _______________________________________________ >>> es-discuss mailing list >>> [email protected] >>> https://mail.mozilla.org/listinfo/es-discuss >>> >> >> _______________________________________________ >> es-discuss mailing list >> [email protected] >> https://mail.mozilla.org/listinfo/es-discuss >> ----- Isiah Meadows [email protected] Looking for web consulting? Or a new website? Send me an email and we can get started. www.isiahmeadows.com _______________________________________________ es-discuss mailing list [email protected] https://mail.mozilla.org/listinfo/es-discuss
