Jenkins precommit worker affinity

It appears that the new job beam_PreCommit_Java_MavenInstall has an affinity for Jenkins worker beam3 while workers beam1 and beam2 sit idle. Is this intentional? There seems to be a backlog of half a dozen builds.

FYI: Jenkins Job Changes

Hi all, Building off of Davor's recent changes to the website Jenkins jobs, I'm in process in converting the main builds to the Groovy DSL. This is being tracked in https://issues.apache.org/jira/browse/BEAM-1064, and work is ongoing in https://github.com/apache/incubator-beam/pull/1390. Some

Re: Questions about coders

Thanks Kenn! This is very helpful. On Wed, Nov 30, 2016 at 4:13 PM Kenneth Knowles wrote: > On Wed, Nov 30, 2016 at 3:52 PM, Eugene Kirpichov < > kirpic...@google.com.invalid> wrote: > > > Hello, > > > > Do we have anywhere a set of recommendations for developing new

Re: Questions about coders

On Wed, Nov 30, 2016 at 3:52 PM, Eugene Kirpichov < kirpic...@google.com.invalid> wrote: > Hello, > > Do we have anywhere a set of recommendations for developing new coders? I'm > confused by a couple of things: > > - Why are coders serialized by JSON serialization instead of by regular > Java

Re: Jenkins build became unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #1690

Same issue Kenn is investigating with MavenVerify. On Wed, Nov 30, 2016 at 3:28 PM, Apache Jenkins Server < jenk...@builds.apache.org> wrote: > See GoogleCloudDataflow/1690/changes> > > -- --- Jason Kuster Apache Beam

Re: Jenkins build is still unstable: beam_PostCommit_MavenVerify #1948

This is a Dataflow-specific linking error. I am investigating and proceeding with a temporary rollback. On Wed, Nov 30, 2016 at 3:06 PM, Apache Jenkins Server < jenk...@builds.apache.org> wrote: > See > >

Re: [PROPOSAL] "IOChannelFactory" Redesign and Make it Configurable

Thanks JB for the feedback. Yes, we should provide a hadoop.fs.FileSystem adaptor. As you said, it will make a range of file system available in Beam. And, people can choose to implement BeamFileSystem directly to get the best performance (For example, providing bulk operations.) -- Pei On

Re: Jenkins build became unstable: beam_Release_NightlySnapshot #249

This looks like it might have been the sort of thing that #1189 (just merged) will fix. On Tue, Nov 29, 2016 at 11:29 PM, Apache Jenkins Server < jenk...@builds.apache.org> wrote: > See

Jenkins build became unstable: beam_Release_NightlySnapshot #249

See

Re: How to create a Pipeline with Cycles

Don't worry, we're all constantly learning. :-) On Wed, 30 Nov 2016 at 12:00 Ismaël Mejía wrote: > Hello, thanks for the clarifying Aljoscha, you are absolutely right, sorry > for my imprecision. I should do my homework and work more with flink :). > > For ref, the iteration

Re: Add DistributedLog IO

I just sent a pull request for adding a bounded source to Beam for reading distributedlog streams - https://github.com/apache/incubator-beam/pull/1464 Appreciate any review comments. - KN On Wed, Aug 31, 2016 at 2:10 AM, Jean-Baptiste Onofré wrote: > Hi Khurrum, > > I

Re: How to create a Pipeline with Cycles

Hi, there is support for cycles in Flink but the Yahoo benchmark is not making use of that feature, if I'm not completely mistaken. Cheers, Aljoscha On Wed, 30 Nov 2016 at 09:57 Ismaël Mejía wrote: > Hello, > > Shen you should probably first check the benchmark

Re: Meet up at Strata+Hadoop World in Singapore

Lucky you guys ;) Unfortunately, I won't be there. See you soon. Regards JB On 11/30/2016 08:07 AM, Aljoscha Krettek wrote: Hi, I'll also be there to give a talk (and also at the Beam tutorial). Cheers, Aljoscha On Wed, Nov 30, 2016, 00:51 Dan Halperin wrote: Hey