Re: Possible 2.36.0 regression with CoGbkResult

2022-03-24 Thread Steve Niemitz
That fixed it. I assume the issue is that the `assert` isn't run in dataflow v1 (because asserts are disabled?), so maybeAdvance() is never called, but should be? Interestingly enough, IntelliJ actually flags this with a warning: "'assert' has side effects: call to 'maybeAdvance()' mutates field

Re: Weird AutoValue bug when compiling GenerateSequence.Builder

2022-03-24 Thread Steve Niemitz
So the first time I run with --rerun-tasks correctly generates the builder. If I run gradle again after making any change to sdks-java-core, the incorrect builder is generated and persists until I run with --rerun-tasks again. I think another target is overwriting it after it's generated, if I

Re: Weird AutoValue bug when compiling GenerateSequence.Builder

2022-03-24 Thread Steve Niemitz
./gradlew --rerun-tasks fixed it. I had tried a clean many times (and deleted the generated file even), who knows what was actually cached that was wrong... On Thu, Mar 24, 2022 at 7:57 PM Steve Niemitz wrote: > 2.37 is on 1.8.2, I also tried 1.9.0 (which is what's on master) and ran > into

Re: Weird AutoValue bug when compiling GenerateSequence.Builder

2022-03-24 Thread Steve Niemitz
2.37 is on 1.8.2, I also tried 1.9.0 (which is what's on master) and ran into the same problem. I feel like there must be something weird with my setup, but I can't for the life of me think what it'd be. I tried gradle with --no-parallel as well which didn't help. Right now every time I build I

Weird AutoValue bug when compiling GenerateSequence.Builder

2022-03-24 Thread Steve Niemitz
Seemingly randomly when I build beam (I'm building 2.37 right now), the AutoValue builder for GenerateSequence seems to ignore the @Nullable attribute on many of the fields, resulting in an AutoValue builder that enforces all fields are set. This breaks building eg

Re: Possible 2.36.0 regression with CoGbkResult

2022-03-24 Thread Steve Niemitz
We were able to reproduce the issue on our end as well in some of our jobs as well. We're going to revert that commit and see if that fixes it for us too. On Wed, Mar 23, 2022 at 3:47 PM Robert Bradshaw wrote: > If it hadn't been specifically requested by some customers, and been > out for

Flaky test issue report (53)

2022-03-24 Thread Beam Jira Bot
This is your daily summary of Beam's current flaky tests (https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20statusCategory%20!%3D%20Done%20AND%20labels%20%3D%20flake) These are P1 issues because they have a major negative impact on the community and make it hard to

P1 issues report (70)

2022-03-24 Thread Beam Jira Bot
This is your daily summary of Beam's current P1 issues, not including flaky tests (https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20statusCategory%20!%3D%20Done%20AND%20priority%20%3D%20P1%20AND%20(labels%20is%20EMPTY%20OR%20labels%20!%3D%20flake). See

P0 (outage) report

2022-03-24 Thread Beam Jira Bot
This is your daily summary of Beam's current outages. See https://beam.apache.org/contribute/jira-priorities/#p0-outage for the meaning and expectations around P0 issues. BEAM-14163: Performance Regressions in streaming python ParDo and GBK Load Tests

Beam VertexAI feature store IO

2022-03-24 Thread Fu Yi (BT - AI, Analytics & Data Domain)
Hi there: As far as I know, there is no built-in IO for Vertex AI feature store yet? Is there any plan to add it by the community or is there any work been doing right now? Best regards, Yi The information contained in this e-mail message may be privileged, confidential, and protected from

Re: Updating output watermark on bundle boundaries

2022-03-24 Thread Evan Galpin
Thanks for starting this thread Jan, I'm keen to hear thoughts and outcomes! I thought I would mention that answers to the questions posed here will help to unblock a 2.38.0 release blocker[1]. [1] https://issues.apache.org/jira/browse/BEAM-14064 On Thu, Mar 24, 2022 at 5:28 AM Jan Lukavský

Re: [ANNOUNCE] New committer: Evan Galpin

2022-03-24 Thread Jan Lukavský
Congrats Evan! On 3/23/22 18:42, Ahmet Altay wrote: Congratulations, Evan! On Wed, Mar 23, 2022 at 10:12 AM Kenneth Knowles wrote: Thanks for all your contributions! On Mon, Mar 14, 2022 at 1:56 PM Aizhamal Nurmamat kyzy wrote: Congratulations, Evan! And thank you for

Updating output watermark on bundle boundaries

2022-03-24 Thread Jan Lukavský
Hi, this is follow-up thread started from [1]. In the thread there is mentioned multiple times that (in stateless ParDo), the output watermark is allowed to advance only on bundle boundaries [2]. Essentially that would mean that anything in between calls to @StartBundle and @FinishBundle