Re: Possible to use GlobalWindows for writing unbounded input to files?

2019-01-09 Thread Kenneth Knowles
Definitely sounds like a bug but also I want to caution you (or anyone reading this archived) that there are known problems with continuation triggers. A spec on continuation triggers that we missed was that they really must be "compatible" (this is an arbitrary concept, having only to do with Flat

Re: Enforce javadoc comments in public methods?

2019-01-09 Thread Ruoyun Huang
created a PR: https://github.com/apache/beam/pull/7454 Note instead of having separated checkstyle specs for Main versus Test, this PR simply uses suppression to turn off JavaDocComment for test files. If this PR draft looks good, then next step I will commit another change that: 1) throw error o

Re: [DISCUSS] (Forked thread) Beam issue triage & assignees

2019-01-09 Thread Ahmet Altay
I agree with the proposals here. Initial state of "Needs Review" and blocking releases on untriaged issues will ensure that we will at least look at every new issue once. On Wed, Jan 9, 2019 at 10:30 AM Maximilian Michels wrote: > Hi Kenn, > > As your data shows, default-assigning issues to a si

Re: Add all tests to release validation

2019-01-09 Thread Sam Rohde
Cool, thanks for all of the replies. Does this summary sound reasonable? *Problem:* there are a number of failing tests (including flaky) that don't get looked at, and aren't necessarily green upon cutting a new Beam release. *Proposed Solution:* - Add all tests to the release validation -

Re: error with DirectRunner

2019-01-09 Thread Allie Chen
Greetings! May I ask whether there is any plan to work on this issue? Or if I just use `BundleBasedDirectRunner` instead of `DirectRunner`, will there be any performance issues/caveats I should worry about? Thanks! Allie On Tue, Oct 30, 2018 at 8:13 PM Udi Meiri wrote: > +Robert Bradshaw I wo

Dev contact for KafkaIO

2019-01-09 Thread Raghu Angadi
Hi Everyone, Last Friday was my last day at Google and Dataflow team. I had a great time and learnt lot from working on both Apache Beam and Dataflow. My new job at a startup is not directly related to Apache Beam and I will not be able to spend a lot of time on support or development of KafkaIO.

Re: [DISCUSS] (Forked thread) Beam issue triage & assignees

2019-01-09 Thread Maximilian Michels
Hi Kenn, As your data shows, default-assigning issues to a single person does not automatically solve triaging issues. Quite the contrary, it hides the triage status of an issue. From the perspective of the Flink Runner, we used to auto-assign but we got rid of this. Instead, we monitor the

Re: [Go SDK] User Defined Coders

2019-01-09 Thread Robert Bradshaw
On Tue, Jan 8, 2019 at 9:15 PM Reuven Lax wrote: > > I wonder if we could do this _only_ over the FnApi. The FnApi already does > batching I believe. What if we made schemas a fundamental part of our protos, > and had no SchemaCoder. One advantage of SchemaCoders is that they allow nesting insi

Re: Python 3: final step

2019-01-09 Thread Robbe Sneyders
Hi all, We've been making quite some progress these last weeks. I'll give a short status update on where we are right now. Our current goal is to make all unittests succeed in Python 3. We are currently at: 1842 tests: (SKIP=350, errors=100, failures=9) All of these remaining errors and failures

Re: Enforce javadoc comments in public methods?

2019-01-09 Thread Robert Bradshaw
On Tue, Jan 8, 2019 at 11:15 PM Kenneth Knowles wrote: > > I think @Internal would be a reasonable annotation to exempt from > documentation, as that means it is explicitly *not* part of the actual public > API, as Ismaƫl alluded to. We'll probably want a distinct annotation from that. Forced c