Re: Jenkins build is still unstable: beam_PostCommit_Java_RunnableOnService_Dataflow #1805

Jira filed: https://issues.apache.org/jira/browse/BEAM-1153 PR of a forward fix: https://github.com/apache/incubator-beam/pull/1608 PR of a partially rollback fix: https://github.com/apache/incubator-beam/pull/1611 On Tue, Dec 13, 2016 at 4:13 PM, Pei He <pe...@google.com> wrote: > T

Re: [PROPOSAL] "IOChannelFactory" Redesign and Make it Configurable

Thanks for the thorough answers. It all sounds good to me. > > On Tue, Dec 6, 2016 at 12:57 PM, Pei He <pe...@google.com.invalid> wrote: > > > Thanks Kenn for the feedback and questions. > > > > I responded inline. > > > > On Mon, Dec 5, 2016 at 7:49

Re: [PROPOSAL] "IOChannelFactory" Redesign and Make it Configurable

for that. However, I am happy to hear thoughts and get help from people working on the python sdk. > On Mon, Dec 5, 2016 at 4:41 PM, Pei He <pe...@google.com.invalid> wrote: > > > I have received a lot of comments in "Part 1: IOChannelFactory > > Redesign" [1]. And,

Re: [PROPOSAL] "IOChannelFactory" Redesign and Make it Configurable

gree that a Beam filesystem abstract is fine. > > My point is that we should provide a HadoopFilesystem extension/plugin for > Beam filesystem asap: that would help us to support a good range of > filesystems quickly. > > Just my $0.01 ;) > > Regards > JB > > > On

Re: [DISCUSS] Graduation to a top-level project

+1, very exciting and looking forward. -- Pei On Tue, Nov 22, 2016 at 11:07 AM, Aljoscha Krettek wrote: > +1 > > I'm quite enthusiastic about the growth of the community and the open > discussions! > > On Tue, 22 Nov 2016 at 19:51 Jason Kuster

Re: [PROPOSAL] "IOChannelFactory" Redesign and Make it Configurable

he record: why introducing BeamFileSystem and not > using the Hadoop FileSystem interface ? > > Thanks > Regards > JB > > On 11/17/2016 01:09 AM, Pei He wrote: > >> Hi, >> >> I am working on BEAM-59 >> <https://issues.apache.org/jira/browse/BEA

Re: [Proposal] Add waitToFinish(), cancel(), waitToRunning() to PipelineResult.

> > On Mon, Jul 25, 2016 at 3:28 PM, Lukasz Cwik <lc...@google.com.invalid> > > wrote: > > > +1 for your proposal Pei > > > > > > On Mon, Jul 25, 2016 at 5:54 PM, Pei He <pe...@google.com.invalid> > > wrote: > > > > > >> Looks to me tha

Re: [Proposal] Add waitToFinish(), cancel(), waitToRunning() to PipelineResult.

ing - wait(Duration) > 2. blocking - waitForInterruption() - some signal that terminates the > job. > 3. non-blocking. > > My 2¢, > Amit > > On Thu, Jul 21, 2016 at 1:39 AM Pei He <pe...@google.com.invalid> wrote: > >> Hi everyone, >> Here

[Proposal] Add waitToFinish(), cancel(), waitToRunning() to PipelineResult.

Hi everyone, Here is a proposal to address the following issue: JIRA issue https://issues.apache.org/jira/browse/BEAM-443 Currently, users doesn’t have a consistent way to wait for the pipeline to finish. Different runners have different implementations. For example: 1. DirectRunner have a