Re: examples-java8 tests running slow

s. Is this > happening when you run them locally or in Jenkins? > > On Mon, Dec 12, 2016 at 5:06 PM, Manu Zhang <owenzhang1...@gmail.com> > wrote: > > > Sorry, they are tests under *maven-archetypes/examples-java8 *and cannot > > be > > skipped with &qu

Re: examples-java8 tests running slow

Sorry, they are tests under *maven-archetypes/examples-java8 *and cannot be skipped with "-DskipTests" On Tue, Dec 13, 2016 at 9:01 AM Manu Zhang <owenzhang1...@gmail.com> wrote: > Hi, > > I find tests under example-java8 are running slow each time. Looking

Re: [DISCUSS] [BEAM-438] Rename one of PTransform.apply or PInput.apply

+1 Manu On Thu, Dec 8, 2016 at 2:40 PM Tyler Akidau wrote: > +1 > > On Thu, Dec 8, 2016 at 1:10 PM Jean-Baptiste Onofré > wrote: > > > +1 > > > > Regards > > JB > > > > On 12/07/2016 10:37 PM, Kenneth Knowles wrote: > > > Hi all, > > > > > > I

Test failure on beam-sdks-java-maven-archetypes-examples-java8

Guys, Has anyone seen the following failure on the latest master ? [INFO] java.lang.IllegalStateException: Failed to validate gs://apache-beam-samples/shakespeare/* [INFO] at it.pkg.MinimalWordCountJava8Test.testMinimalWordCountJava8(MinimalWordCountJava8Test.java:63) [INFO] Caused by:

Re: SBT/ivy dependency issues

ry adding explicit compile scope to beam-parent's > dependencyManagement ? > > On Thu, Nov 10, 2016 at 2:27 PM Manu Zhang <owenzhang1...@gmail.com> > wrote: > > > No, it's not even looking. > > > > Very interesting ivy file generated by sbt where neither

Re: SBT/ivy dependency issues

beam-sdks-java-core *& > *beam-runners-core-java* but fails to find them ? > > On Thu, Nov 10, 2016 at 3:43 AM Manu Zhang <owenzhang1...@gmail.com> > wrote: > > > Hi all, > > > > I tried and reproduced the issue. "sbt-dependency-graph" doesn't show

Re: PCollection to PCollection Conversion

I would love to see a lean core and abundant Transforms at the same time. Maybe we can look at what Confluent does for kafka-connect. They have official extensions support for JDBC, HDFS and ElasticSearch under https://github.com/confluentinc. They put them along

Re: Why does `Combine.perKey(SerializableFunction)` require same input and output type

s > from, had an explicit init rather than addInput and that turned out to > be a drawback when implementing CombineFns. > > > - With `compact` we allow multiple physical representations of the same > > semantic accumulator, and a hook to switch between them > > - A

Re: Why does `Combine.perKey(SerializableFunction)` require same input and output type

and then discarding the key and extracting the single > output. > PerKeyCombineFn is shorthand for doing accumulation where the key doesn't > modify the accumulation in anyway. > > On Fri, Oct 28, 2016 at 6:09 PM, Manu Zhang <owenzhang1...@gmail.com> > wrote: > >

Re: migrating gearpump-runner to new DoFn fails with NotSerializableException

request (where tests will fail, > of course) so I can see and comment on the code itself. > > Kenn > > On Sun, Oct 30, 2016 at 8:52 PM Manu Zhang <owenzhang1...@gmail.com> > wrote: > > > Hi all, > > > > I'm migrating `OldDoFn` to `DoFn` in gearpump-runner. We ser

migrating gearpump-runner to new DoFn fails with NotSerializableException

Hi all, I'm migrating `OldDoFn` to `DoFn` in gearpump-runner. We serialize all the functions locally and ship to remote cluster. Hence, I try to make sure the functions are serializable. Unluckily, the integration-tests fail with `NotSerializableException` as follows. Anyone knows what that

Can we have more quick start examples ?

e more quick start examples that can be run anywhere ? Thanks, Manu Zhang

Why does `Combine.perKey(SerializableFunction)` require same input and output type

Hi all, I'm wondering why `Combine.perKey(SerializableFunction)` requires input and output to be of the same type while `Combine.PerKey` doesn't have this restriction. Thanks, Manu

Re: build failed with dependency problems

Thanks, it succeeded after `maven clean`. On Wed, Oct 26, 2016 at 2:58 PM Amit Sela <amitsel...@gmail.com> wrote: I just fetched and pulled latest master and build succeeded, maybe try again ? On Wed, Oct 26, 2016 at 9:19 AM Manu Zhang <owenzhang1...@gmail.com> wrote: > Hi A

Re: [DISCUSS] Current ongoing work on runners

We usually have docs sitting together with the source codes such that each release has its own versioned docs. If the capability matrix is like other codes, we can update it as we add new features. It also applies to other docs like new IO. We could make it a requirement in the PR template.

Re: [DISCUSS] Merging master -> feature branch

I like this idea as a maintainer for gearpump-runner branch. Usually the merge is to keep the feature branch updated to the latest API changes on master. Sorry that I haven't made merge PR as frequent as possible which would bring in a lot of changes each time and make it harder for others to

how to work on gearpump-runner branch

nflict with the branch. What should I do in this case ? Any references or documentations would be helpful as well. Thanks, Manu Zhang <https://github.com/apache/incubator-beam/tree/gearpump-runner>

Re: [KUDOS] Contributed runner: Gearpump!

so probably you should rename > GearpumpPipelineRunner -> GearpumpRunner. > > Congratulations again, > Ismael > > On Thu, Jul 21, 2016 at 3:10 PM, Bobby Evans <ev...@yahoo-inc.com.invalid> > wrote: > > > Great work. - Bobby > > > > On Thursda

add component tag to pull request title / commit comment

whether it's API change or Runner specific change, or something else, like, [BEAM-XXX] [Direct-Java] Comments... ? Thanks, Manu Zhang

Re: Apache Beam Web Docs Contribution

Yes it will be great to have. By the way, a few documentations have not been put into web pages but on Google Docs and Apache Beam Documentation link on README is not found. Thanks,