@Kenn: I am going to add the build.gradle. Is there anything else? @Ahmet, @Robert: here are more details about the samza runner right now:
- Missing pieces: timer support in ParDo is not there yet and I plan to add it soon. SplittableParDo is missing but we don't have a use case so far. We are on par with the other runners for the rest of the Java features. - Work in Progress: implement the portable pipeline runner logic. - Future plans: support Python is our next goal. Hopefully we will get a prototype working sometime next quarter :). Btw, thanks everyone for the comments! Thanks, Xinyu On Mon, Jun 18, 2018 at 4:59 PM, Robert Burke <rob...@frantil.com> wrote: > This is exciting! Is it implemented as a portability framework runner too? > > > On Mon, Jun 18, 2018, 4:36 PM Pablo Estrada <pabl...@google.com> wrote: > >> It's very exciting to see a new runner making it into master. : ) >> >> Best >> -P. >> >> On Mon, Jun 18, 2018 at 3:38 PM Rafael Fernandez <rfern...@google.com> >> wrote: >> >>> I've just read this and wanted to share my excitement :D >>> >>> >>> >>> On Mon, Jun 18, 2018 at 3:10 PM Kenneth Knowles <k...@google.com> wrote: >>> >>>> One thing that will be necessary is porting the build to Gradle. >>>> >>>> Kenn >>>> >>>> On Mon, Jun 18, 2018 at 11:57 AM Xinyu Liu <xinyuliu...@gmail.com> >>>> wrote: >>>> >>>>> Hi, Folks, >>>>> >>>>> On behalf of the Samza team, I would like to propose to merge the >>>>> samza-runner branch into master. The branch was created on Jan when we >>>>> first introduced the Samza Runner [1], and we've been adding features and >>>>> refining it afterwards. Now the runner satisfies the criteria outlined in >>>>> [2], and merging it to master will give more visibility to other >>>>> contributors and users. >>>>> >>>>> 1. Have at least 2 contributors interested in maintaining it, and 1 >>>>> committer interested in supporting it: *Both Chris and me have been making >>>>> contributions and I am going to sign up for the support. There are more >>>>> folks in the Samza team interested in contributing to it. Thanks Kenn for >>>>> all the help and reviews for the runner!* >>>>> 2. Provide both end-user and developer-facing documentation: *The PR >>>>> for the samza-runner doc has runner user guide, capability matrix, and >>>>> tutorial using WordCount examples.* >>>>> 3. Have at least a basic level of unit test coverage: *Unit tests are >>>>> here [3].* >>>>> 4. Run all existing applicable integration tests with other Beam >>>>> components >>>>> and create additional tests as appropriate: Enabled ValidatesRunner >>>>> tests.* >>>>> 5. Be able to handle a subset of the model that addresses a >>>>> significant set of use cases, such as ‘traditional batch’ or ‘processing >>>>> time streaming’: *We have test Beam jobs running in Yarn using event-time >>>>> processing of Kafka streams.* >>>>> 6. Update the capability matrix with the current status. *Same as #2.* >>>>> 7. Add a webpage under documentation/runners. *Same as #2.* >>>>> >>>>> The PR for the samza-runner merge: https://github.com/ >>>>> apache/beam/pull/5668 >>>>> The PR for the samza-runner doc: https://github.com/ >>>>> apache/beam-site/pull/471 >>>>> >>>>> Thanks, >>>>> Xinyu >>>>> >>>>> [1] https://issues.apache.org/jira/browse/BEAM-3079 >>>>> [2] https://beam.apache.org/contribute/ >>>>> [3] https://github.com/apache/beam/tree/samza-runner/ >>>>> runners/samza/src/test >>>>> >>>> -- >> Got feedback? go/pabloem-feedback >> >
