Re: [VOTE] Vendored Dependencies Release gRPC 1.36.0 v0.1 RC1

2021-03-16 Thread Pablo Estrada
+1! On Tue, Mar 16, 2021 at 4:22 PM Ahmet Altay wrote: > +1 > > On Tue, Mar 16, 2021 at 4:21 PM Robert Bradshaw > wrote: > >> +1 >> >> On Tue, Mar 16, 2021 at 4:00 PM Kenneth Knowles wrote: >> >>> Please review the release of the following artifacts that we vendor: >>> *

Re: Adding flaky postcommit test suite

2021-03-16 Thread Tyson Hamilton
The Apache Airflow project has some interesting automation around flaky tests. They annotate such flaky tests as 'quarantined', those quarantined tests still run (maybe even with retries?) but won't fail a test suite. Quarantined tests are run in a separate scheduled job, when they start passing,

Re: SDK improvements (BEAM-3304)

2021-03-16 Thread Fernando Morales Martinez
Thanks again Robert! We will start with the implementation of AfterPane and the WindowingStrategy (core/graph/window/strategy.go) - Fernando Morales On Tue, Mar 16, 2021 at 1:02 PM Robert Burke wrote: > Hello Fernando! > > That should be correct. > > The protos are used to represent the user

BEAM-11023: tests failing on Spark Structured Streaming runner

2021-03-16 Thread Fernando Morales Martinez
Hi team, it is mentioned in this WI that the tests (GroupByKeyTest testLargeKeys100MB and testGroupByKeyWithBadEqualsHashCode) stopped working around five months ago. I took a look at the PRs prior to that date and couldn't find a report stating that they were working. Is there a way to get

Re: [VOTE] Vendored Dependencies Release gRPC 1.36.0 v0.1 RC1

2021-03-16 Thread Ahmet Altay
+1 On Tue, Mar 16, 2021 at 4:21 PM Robert Bradshaw wrote: > +1 > > On Tue, Mar 16, 2021 at 4:00 PM Kenneth Knowles wrote: > >> Please review the release of the following artifacts that we vendor: >> * beam-vendor-grpc-1_36_0 >> >> Hi everyone, >> >> Please review and vote on the release

Re: [VOTE] Vendored Dependencies Release gRPC 1.36.0 v0.1 RC1

2021-03-16 Thread Robert Bradshaw
+1 On Tue, Mar 16, 2021 at 4:00 PM Kenneth Knowles wrote: > Please review the release of the following artifacts that we vendor: > * beam-vendor-grpc-1_36_0 > > Hi everyone, > > Please review and vote on the release candidate #1 for the version 0.1, as > follows: > [ ] +1, Approve the release

Re: Adding flaky postcommit test suite

2021-03-16 Thread Kenneth Knowles
I expect the suite to be permared, right? Because of some thing or another flaking at all times. Kenn On Tue, Mar 16, 2021 at 2:13 PM Alex Amato wrote: > Is it possible to make the presubmit auto retry all failed tests a few > times? (and maybe generate a report of a list of flakey tests). >

[VOTE] Vendored Dependencies Release gRPC 1.36.0 v0.1 RC1

2021-03-16 Thread Kenneth Knowles
Please review the release of the following artifacts that we vendor: * beam-vendor-grpc-1_36_0 Hi everyone, Please review and vote on the release candidate #1 for the version 0.1, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The

Re: Adding flaky postcommit test suite

2021-03-16 Thread Alex Amato
Is it possible to make the presubmit auto retry all failed tests a few times? (and maybe generate a report of a list of flakey tests). Then you don't need to disable/isolate the flakey tests. If this is not possible, or hard to setup, then manually moving them to a different suite sounds like a

Adding flaky postcommit test suite

2021-03-16 Thread Pablo Estrada
Hi all, In Beam, we sometimes hit the issue of having one or two test cases that are particularly flaky, and we deactivate them. This is completely reasonable to me, because we need to keep good testing signal on our primary suites. The danger of deactivating these tests is that, although we have

Re: SDK improvements (BEAM-3304)

2021-03-16 Thread Robert Burke
Hello Fernando! That should be correct. The protos are used to represent the user pipeline in a portable way, and handle the serialization of such. However the only time the SDKs actually need deal with the protos is the "last mile", but this is a per SDK choice. The Go SDK is mostly set up to

SDK improvements (BEAM-3304)

2021-03-16 Thread Fernando Morales Martinez
*Hi team,* We’re working on WI https://issues.apache.org/jira/browse/BEAM-3304 and have a couple of questions. As we understand it, the Proto compiler based on file beam_runner_api.proto creates the whole RunnerApi for Java (RunnerApi.class) and for Go (beam_runner_api.pb.go)

Re: [PROPOSAL] Preparing for Beam 2.29.0 release

2021-03-16 Thread Kenneth Knowles
Update: There are 5 issues tagged to 2.29.0 release: https://issues.apache.org/jira/issues/?jql=statusCategory%20!%3D%20done%20AND%20project%20%3D%2012319527%20AND%20fixVersion%20%3D%2012349629%20ORDER%20BY%20priority%20DESC%2C%20key%20ASC There are 4 issues in Needs Triage state with priority

Re: Upgrading vendored gRPC from 1.26.0 to 1.36.0

2021-03-16 Thread Kenneth Knowles
Update on this: there are some minor issues and then I'll send out the RC. I think this is worth blocking 2.29.0 release on, so I will do this first. We are still eliminating other blockers from 2.29.0 anyhow. Kenn On Mon, Mar 15, 2021 at 7:17 AM Tomo Suzuki wrote: > Hi Beam developers, > >

Access to Beam contributor list

2021-03-16 Thread César Cueva Orozco
Hello Team, Could you please add my user name CesarCueva19 to the contributor list? Thank you -- *This email and its contents (including any attachments) are being sent to you on the condition of confidentiality and may be protected by legal privilege. Access to this email by anyone other than

Re: Null checking in Beam

2021-03-16 Thread Kenneth Knowles
I've asked on checkerframework users mailing list if they or any users have recommendations for the IntelliJ integration issue. It is worth noting that the default annotations inserted into the bytecode do add value: the @NonNull type annotations are default for checkerframework but not default

Re: Contributor permission for Beam Jira tickets

2021-03-16 Thread Ismaël Mejía
Hello Vitaly, What is your jira id? On Tue, Mar 16, 2021 at 5:54 PM Vitaly Terentyev wrote: > > This is Vitaly from Akvelon. > Could you please add me as a contributor to Beam's Jira issue tracker? > I would like to assign some tickets for my work. > > Best regards, > > Vitaly

Re: Null checking in Beam

2021-03-16 Thread Kenneth Knowles
Seems it is an FAQ with no solution: https://checkerframework.org/manual/#faq-classfile-annotations On Tue, Mar 16, 2021 at 10:01 AM Kenneth Knowles wrote: > Adding -PskipCheckerframework when releasing will solve it for users, but > not for dev@. > > Making it off by default and a separate CI

Re: Null checking in Beam

2021-03-16 Thread Kenneth Knowles
Adding -PskipCheckerframework when releasing will solve it for users, but not for dev@. Making it off by default and a separate CI check that turns it on would solve it overall but has the downsides mentioned before. It would be very nice to simply have a flag to not insert default annotations.

Contributor permission for Beam Jira tickets

2021-03-16 Thread Vitaly Terentyev
This is Vitaly from Akvelon. Could you please add me as a contributor to Beam's Jira issue tracker? I would like to assign some tickets for my work. Best regards, Vitaly

Re: Null checking in Beam

2021-03-16 Thread Kenneth Knowles
I agree with all of your points. I have good news and bad news. Reordering your points to put some together On Tue, Mar 16, 2021 at 2:14 AM Jan Lukavský wrote: > a) if a check does not modify the bytecode, it is fine and we can use it > - we are absolutely free to use any tooling we agree on,

Re: Null checking in Beam

2021-03-16 Thread Jan Lukavský
I believe it is not a problem of Idea. At least I didn't notice behavior like that with Guava, although Guava uses the framework as well. Maybe there is a way to tune which annotations should be generated? Or Guava handles that somehow differently? On 3/16/21 5:22 PM, Reuven Lax wrote: I've

Re: Null checking in Beam

2021-03-16 Thread Reuven Lax
I've also been annoyed at IntelliJ autogenenerating all these annotations. I believe Kenn said that this was not the intention - maybe there's an IntelliJ setting that would stop this from happening? On Tue, Mar 16, 2021 at 2:14 AM Jan Lukavský wrote: > I don't know the details of the

Re: Null checking in Beam

2021-03-16 Thread Jan Lukavský
I don't know the details of the checkerframework, but there seems a contradiction between what code is (currently) generated and what we therefore release and what actually the checkerframework states [1]: @UnknownKeyFor: Used internally by the type system; should never be written by a

Re: Null checking in Beam

2021-03-16 Thread Reuven Lax
On Mon, Mar 15, 2021 at 11:42 PM Reuven Lax wrote: > > > On Mon, Mar 15, 2021 at 9:12 PM Kenneth Knowles wrote: > >> I will be blunt about my opinions about the general issue: >> >> - NullPointerExceptions (and similar) are a solved problem. >>* They have been since 2003 at the latest [1]

Re: Null checking in Beam

2021-03-16 Thread Reuven Lax
On Mon, Mar 15, 2021 at 9:12 PM Kenneth Knowles wrote: > I will be blunt about my opinions about the general issue: > > - NullPointerExceptions (and similar) are a solved problem. >* They have been since 2003 at the latest [1] (this is when the types > were hacked into Java - the foundation