Hi all, We've created a reference implementation. It's been done in such a way that implementation of new features (stages) is quite straight forward, there is an abstraction that does the hard work of handling concurrency, the reactive streams spec conformance, and managing demand and buffering, and error handling, and then individual stages (eg, map, filter, flatMap) are implemented using a very easy to use API (note, this API/abstraction is all private, internal to the reference implementation). Rudimentary tests on performance show that it's not terrible compared to other reactive streams implementations, with a number of clear optimization paths identified should we decide that's necessary. I believe this proposal is now close to complete - the remaining work is:
* Decide what scope beyond JDK8 streams it should support - while this decision is not trivial, the amount of work required to actually add these to the API and implement in the reference implementation is trivial. * Fill out the TCK with more rigorous verification. * Create some rigorous benchmarks. I'm not sure what should be done next. I have talked to a number of people who are either involved in, or are writing APIs that use Reactive Streams in private, and interest seems high. Also, there is general consensus in public discussions in the Jakarta EE/MicroProfile communities that an API like this would be very valuable in the JDK. The API of course could be done in Jakarta EE instead, but given that Reactive Streams is part of and used by the JDK, and given that the JDK8 Streams API is part of the JDK, Jakarta EE would seem an odd place to put this library - it essentially would mean that to make effective use of JDK libraries that use Reactive Streams (eg HTTP client, possibly java.sql2 aka ADBA), you need to use Jakarta EE (or some third party streaming library). So unless there's any major feedback coming here on this list, I'd like to put this forward as a JEP. Regards, James On 15 March 2018 at 12:24, James Roper <ja...@lightbend.com> wrote: > Hi all, > > An update on this. We've now filled out the API with feature parity with > the JDK8 Streams API - for operators that make sense in Reactive Streams. > We've provided example implementations of the API backed by both Akka > Streams and rxjava, showing that it can be widely implemented. The TCK > still needs some work, but covers the major features, and comprehensively > covers all publishers/subscribers with the Reactive Streams TCK (so > comprehensive that we actually found two Reactive Streams TCK violations in > Akka Streams with this, and a couple in rxjava too). > > There are two major areas of work left to get something that would be > ready to be a candidate for a final API. The first is to produce a zero > dependency reference implementation of the API. This is what I plan on > starting on next. > > The second is to decide what additional operators and generators the API > should provide. So far, the scope has been mostly limited to a subset of > the JDK8 Streams scope, only a few additional API pieces have been created, > such as a few variations on flatMap (one that supports CompletionStage, and > one that supports Iterable). There are a number of other features for > consideration to provide basic necessary functionality for this API, here's > some examples off the top of my head (some of these can already be > implemented in terms of other stages already provided): > > * A cancelled subscriber (useful for cleaning up hot publishers) > * An ignoring subscriber (useful when you do the actual work in a previous > stage of the graph, such as mapping to completion stages) > * Error handling subscribers and/or processors > * Termination listening subscribers and/or processors > * A processor that wraps a subscriber and publisher > * The ability to merge streams - so far only concat is provided, and all > flatMaps are essentially concatenations, merge variants may be useful > (though introduce a lot of complexity, such as specifying breadth) > * The ability to split streams into sub streams - a use case for this is > in parsing a stream that contains potentially large sub streams, like > parsing a multipart/form-data body > * Batching of elements in a stream, based on predicate, influenced by > backpressure or based on a scheduled tick > * Scheduled features such as emitting ticks, rate limiting, etc > * The ability to control buffering and asynchronous boundaries within a > graph > * Naming of stages for debug/error reporting/monitoring purposes > > Not all of the above may be absolutely necessary, but should be > considered, and there may be other features as well that would be useful to > consider. > > Please visit the repo and any feedback would be much appreciated: > > https://github.com/lightbend/reactive-streams-utils > > Regards, > > James > > On 8 March 2018 at 03:59, Brian Goetz <brian.go...@oracle.com> wrote: > >> To answer the questions at the bottom: the next step is to start working >> on this and get folks excited about contributing. There's plenty of time >> for process later, but filing a JEP or creating a project shouldn't be a >> barrier to innovating. >> >> >> On 2/28/2018 10:33 PM, James Roper wrote: >> >>> Hi all, >>> >>> We've put together a simple proposal for this. Please read the README for >>> an introduction to this proposal. >>> >>> https://github.com/lightbend/reactive-streams-utils >>> >>> Regards, >>> >>> James >>> >>> On 22 February 2018 at 11:47, James Roper <ja...@lightbend.com> wrote: >>> >>> Hi all, >>>> >>>> This is an email to give people a heads up that we'd like to look at >>>> creating an API, in the same vein as the JDK8 Streams API, for building >>>> reactive streams (a la JDK9 juc.Flow). Our goals for this are: >>>> >>>> * To fill a gap in the JDK where if a developer wants to do even the >>>> simplest of things with a JDK9 juc.Flow, such as map or filter, they >>>> need >>>> to bring in a third party library that implements that. >>>> * To produce an API that can build Publishers, Subscribers, Processors, >>>> and complete graphs, for the purposes of consuming APIs that use >>>> reactive >>>> streams (for example, JDK9 Http Client). >>>> * To produce an API that aligns closely with ju.stream.Stream, using it >>>> for inspiration for naming, scope, general API shape, and other aspects. >>>> The purpose of this goal is to ensure familiarity of Java developers >>>> with >>>> the new API, and to limit the number of concepts Java developers need to >>>> understand to do the different types of streaming offered by the JDK. >>>> * To produce an API that can be implemented by multiple providers >>>> (including an RI in the JDK itself), using the ServiceLoader mechanism >>>> to >>>> provide and load a default implementation (while allowing custom >>>> implementations to be manually provided). There are a lot of concerns >>>> that >>>> each different streams implementation provides and implements, beyond >>>> streaming, for example monitoring/tracing, concurrency modelling, >>>> buffering >>>> strategies, performance aspects of the streams handling including >>>> fusing, >>>> and context (eg thread local) propagation. This will allow libraries to >>>> use >>>> and provide contracts based on this API without depending on a >>>> particular >>>> implementation, and allows developers to select the implementation that >>>> meets their needs. >>>> >>>> Non goals: >>>> >>>> * To produce a kitchen sink of utilities for working with reactive >>>> streams. There already exist a number of reactive streams >>>> implementations >>>> that seek to meet this goal (eg, Akka Streams, Reactor, RxJava), and >>>> once >>>> you go past the basics (map, filter, collect), and start dealing with >>>> things like fan in/out, cycles, restarting, etc, the different >>>> approaches >>>> to solving this start to vary greatly. The JDK should provide enough to >>>> be >>>> useful for typical every day streaming use cases, with developers being >>>> able to select a third party library for anything more advanced. >>>> >>>> We will update this list when we have something ready for public review. >>>> This probably won't be far off. Our hope is that we can propose this as >>>> a >>>> JEP. >>>> >>>> Regards, >>>> >>>> James >>>> >>>> -- >>>> *James Roper* >>>> *Senior Octonaut* >>>> >>>> Lightbend <https://www.lightbend.com/> – Build reactive apps! >>>> Twitter: @jroper <https://twitter.com/jroper> >>>> >>>> >>> >>> >> > > > -- > *James Roper* > *Senior Octonaut* > > Lightbend <https://www.lightbend.com/> – Build reactive apps! > Twitter: @jroper <https://twitter.com/jroper> > -- *James Roper* *Senior Octonaut* Lightbend <https://www.lightbend.com/> – Build reactive apps! Twitter: @jroper <https://twitter.com/jroper>
