Cheers team. I have found a compilation issue on master (in CreateGearpumpPCollectionView), attached is a small patch
Kind regards, Paul On Tue, Aug 8, 2017 at 4:10 PM, Manu Zhang <owenzhang1...@gmail.com> wrote: > Thanks Kenn!!! Thanks everyone!!! It's a great achievement for us. > > On Tue, Aug 8, 2017 at 7:54 AM Kenneth Knowles <k...@google.com.invalid> > wrote: > > > Done! > > > > On Fri, Jul 21, 2017 at 11:08 PM, Jean-Baptiste Onofré <j...@nanthrax.net> > > wrote: > > > > > +1 > > > > > > Regards > > > JB > > > > > > On Jul 22, 2017, 05:06, at 05:06, Kenneth Knowles > <k...@google.com.INVALID > > > > > > wrote: > > > >+1 to this! > > > > > > > >I really want to call out the longevity of contribution behind this, > > > >following many changes in both Beam and Gearpump for over a year. > > > >Here's > > > >the first commit on the branch: > > > > > > > > commit 9478f4117de3a2d0ea40614ed4cb801918610724 (github/pr/323) > > > > Author: manuzhang <owenzhang1...@gmail.com> > > > > Date: Tue Mar 15 16:15:16 2016 +0800 > > > > > > > >And here are some numbers, FWIW: 163 non-merge commits, 203 total. So > > > >that's a PR and review every couple of weeks. > > > > > > > >The ValidatesRunner capability coverage is very good. The only skipped > > > >tests are state/timers, metrics, and TestStream, which many runners > > > >have > > > >partial or no support for. > > > > > > > >I'll save practical TODOs like moving ValidatesRunner execution to > > > >postcommit, etc. Pending the results of this discussion, of course. > > > > > > > >Kenn > > > > > > > > > > > >On Fri, Jul 21, 2017 at 12:02 AM, Manu Zhang <owenzhang1...@gmail.com > > > > > >wrote: > > > > > > > >> Guys, > > > >> > > > >> On behalf of the gearpump team, I'd like to propose to merge the > > > >> gearpump-runner branch into master, which will give it more > > > >visibility to > > > >> other contributors and users. The runner satisfies the following > > > >criteria > > > >> outlined in contribution guide [1]. > > > >> > > > >> > > > >> 1. Have at least 2 contributors interested in maintaining it, and > > > >1 > > > >> committer interested in supporting it: *Both Huafeng and me have > > > >been > > > >> making contributions[2] and we will continue to maintain it. Kenn > > > >and JB > > > >> have been supporting the runner (Thank you, guys!)* > > > >> 2. Provide both end-user and developer-facing documentation*: > They > > > >are > > > >> already on the website ([3] and [4]).* > > > >> 3. Have at least a basic level of unit test coverage: *We do.* > > > >*[5]* > > > >> 4. Run all existing applicable integration tests with other Beam > > > >> components and create additional tests as appropriate: > > > >*gearpump-runner > > > >> passes ValidatesRunner tests.* > > > >> > > > >> > > > >> Additionally, as a runner, > > > >> > > > >> > > > >> 1. Be able to handle a subset of the model that address a > > > >significant > > > >> set of use cases (aka. ‘traditional batch’ or ‘processing time > > > >> streaming’): *gearpump > > > >> runner is able to handle event time streaming * > > > >> 2. Update the capability matrix with the current status: *[4]* > > > >> 3. Add a webpage under documentation/runners: *[3]* > > > >> > > > >> > > > >> The PR for the merge: https://github.com/apache/beam/pull/3611 > > > >> > > > >> Thanks, > > > >> Manu > > > >> > > > >> > > > >> [1] > > > >http://beam.apache.org/contribute/contribution-guide/ > #feature-branches > > > >> [2] https://issues.apache.org/jira/browse/BEAM-79 > > > >> [3] https://beam.apache.org/documentation/runners/gearpump/ > > > >> [4] https://beam.apache.org/documentation/runners/ > capability-matrix/ > > > >> [5] > > > >> https://github.com/apache/beam/tree/gearpump-runner/ > > > >> runners/gearpump/src/test/java/org/apache/beam/runners/gearpump > > > >> > > > > > >
Index: runners/gearpump/src/main/java/org/apache/beam/runners/gearpump/translators/CreateStreamingGearpumpView.java IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- runners/gearpump/src/main/java/org/apache/beam/runners/gearpump/translators/CreateStreamingGearpumpView.java (revision 89236e3b588cd8ab5092667b1fe155738c69dbcf) +++ runners/gearpump/src/main/java/org/apache/beam/runners/gearpump/translators/CreateStreamingGearpumpView.java (revision ) @@ -18,11 +18,6 @@ package org.apache.beam.runners.gearpump.translators; import com.google.common.collect.Iterables; - -import java.util.ArrayList; -import java.util.List; -import java.util.Map; - import org.apache.beam.runners.core.construction.ReplacementOutputs; import org.apache.beam.sdk.coders.Coder; import org.apache.beam.sdk.coders.CoderRegistry; @@ -37,6 +32,10 @@ import org.apache.beam.sdk.values.PValue; import org.apache.beam.sdk.values.TupleTag; +import java.util.ArrayList; +import java.util.List; +import java.util.Map; + /** Gearpump streaming overrides for various view (side input) transforms. */ class CreateStreamingGearpumpView<ElemT, ViewT> extends PTransform<PCollection<ElemT>, PCollection<ElemT>> { @@ -123,9 +122,8 @@ @Override public PCollection<List<ElemT>> expand(PCollection<List<ElemT>> input) { - return PCollection.<List<ElemT>>createPrimitiveOutputInternal( - input.getPipeline(), input.getWindowingStrategy(), input.isBounded()) - .setCoder(input.getCoder()); + return PCollection.createPrimitiveOutputInternal( + input.getPipeline(), input.getWindowingStrategy(), input.isBounded(), input.getCoder()); } public PCollectionView<ViewT> getView() {