I updated the merge PR with the gradle integration (there was some Jenkins Java tests failure with google cloud quota issues. It seems not related to this patch). Please feel free to ping me if anything else is needed.
Thanks, Xinyu On Mon, Jun 18, 2018 at 5:44 PM, Xinyu Liu <[email protected]> wrote: > @Kenn: I am going to add the build.gradle. Is there anything else? > > @Ahmet, @Robert: here are more details about the samza runner right now: > > - Missing pieces: timer support in ParDo is not there yet and I plan to > add it soon. SplittableParDo is missing but we don't have a use case so > far. We are on par with the other runners for the rest of the Java features. > - Work in Progress: implement the portable pipeline runner logic. > - Future plans: support Python is our next goal. Hopefully we will get a > prototype working sometime next quarter :). > > Btw, thanks everyone for the comments! > > Thanks, > Xinyu > > On Mon, Jun 18, 2018 at 4:59 PM, Robert Burke <[email protected]> wrote: > >> This is exciting! Is it implemented as a portability framework runner too? >> >> >> On Mon, Jun 18, 2018, 4:36 PM Pablo Estrada <[email protected]> wrote: >> >>> It's very exciting to see a new runner making it into master. : ) >>> >>> Best >>> -P. >>> >>> On Mon, Jun 18, 2018 at 3:38 PM Rafael Fernandez <[email protected]> >>> wrote: >>> >>>> I've just read this and wanted to share my excitement :D >>>> >>>> >>>> >>>> On Mon, Jun 18, 2018 at 3:10 PM Kenneth Knowles <[email protected]> wrote: >>>> >>>>> One thing that will be necessary is porting the build to Gradle. >>>>> >>>>> Kenn >>>>> >>>>> On Mon, Jun 18, 2018 at 11:57 AM Xinyu Liu <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi, Folks, >>>>>> >>>>>> On behalf of the Samza team, I would like to propose to merge the >>>>>> samza-runner branch into master. The branch was created on Jan when we >>>>>> first introduced the Samza Runner [1], and we've been adding features and >>>>>> refining it afterwards. Now the runner satisfies the criteria outlined in >>>>>> [2], and merging it to master will give more visibility to other >>>>>> contributors and users. >>>>>> >>>>>> 1. Have at least 2 contributors interested in maintaining it, and 1 >>>>>> committer interested in supporting it: *Both Chris and me have been >>>>>> making >>>>>> contributions and I am going to sign up for the support. There are more >>>>>> folks in the Samza team interested in contributing to it. Thanks Kenn for >>>>>> all the help and reviews for the runner!* >>>>>> 2. Provide both end-user and developer-facing documentation: *The PR >>>>>> for the samza-runner doc has runner user guide, capability matrix, and >>>>>> tutorial using WordCount examples.* >>>>>> 3. Have at least a basic level of unit test coverage: *Unit tests are >>>>>> here [3].* >>>>>> 4. Run all existing applicable integration tests with other Beam >>>>>> components >>>>>> and create additional tests as appropriate: Enabled ValidatesRunner >>>>>> tests.* >>>>>> 5. Be able to handle a subset of the model that addresses a >>>>>> significant set of use cases, such as ‘traditional batch’ or ‘processing >>>>>> time streaming’: *We have test Beam jobs running in Yarn using event-time >>>>>> processing of Kafka streams.* >>>>>> 6. Update the capability matrix with the current status. *Same as #2.* >>>>>> 7. Add a webpage under documentation/runners. *Same as #2.* >>>>>> >>>>>> The PR for the samza-runner merge: https://github.com/apac >>>>>> he/beam/pull/5668 >>>>>> The PR for the samza-runner doc: https://github.com/apache >>>>>> /beam-site/pull/471 >>>>>> >>>>>> Thanks, >>>>>> Xinyu >>>>>> >>>>>> [1] https://issues.apache.org/jira/browse/BEAM-3079 >>>>>> [2] https://beam.apache.org/contribute/ >>>>>> [3] https://github.com/apache/beam/tree/samza-runner/runners >>>>>> /samza/src/test >>>>>> >>>>> -- >>> Got feedback? go/pabloem-feedback >>> >> >
