One of the initial things we are focusing on as we move things to Beam is cleaning up the API that new runners will implement. (The current version is neither particularly stable nor clean.) But as long as you are comfortable with a little churn, feel free to start poking around in the code that will be seeding the repository (after some in-progress refactoring):
* Dataflow Java SDK: https://github.com/GoogleCloudPlatform/DataflowJavaSDK (includes the user-level API, DirectRunner, and DataflowRunner) * Flink runner: https://github.com/dataArtisans/flink-dataflow * Spark runner: https://github.com/cloudera/spark-dataflow Gathering together folks that are interested in coordinating on various runners is a great idea. As we get the repo structured -- we'll make sure that that's easy to do. Frances On Wed, Feb 10, 2016 at 4:50 PM, Yang, Connie <[email protected]> wrote: > Hi Taylor, > > Good news indeed that the upcoming Storm 1.0 release will include sliding > and tumble windowing. > > Subash and I would definitely interested in joining Storm running design > and implementation. Is there an ongoing design discussion that we can be > part of? Is there a development branch we can take a look? > > Thanks > Connie > > On 2/10/16, 2:55 PM, "P. Taylor Goetz" <[email protected]> wrote: > > >Hi Connie, > > > >I've definitely considered it (no implementation yet, just thoughts), and > >would like to start coordinating such an effort. That's one of the > >reasons I wanted to get involved with Beam. > > > >A Storm runner would likely/definitely depend on features in the upcoming > >1.0 release (namely MillWheel style windowing). That release is currently > >stabilizing, but should be out soon. > > > >The question I have for the Beam community would be where/how to host the > >collaboration effort: Storm, Beam, or a combination? > > > >I'm happy to facilitate with the Storm community/PMC. > > > >-Taylor > > > >> On Feb 9, 2016, at 2:07 PM, Yang, Connie <[email protected]> wrote: > >> > >> Hi Beamers, > >> > >> We¹re considering adding an Apache Storm Runner to Apache Beam. > >> > >> * Has anyone in the community considered this and started working on > >>this? > >> * If so, what¹s the current state? > >> > >> We would like to join the design discussion and development of it if > >>it¹s in flight. > >> > >> Thanks > >> Connie > >
