Re: Flake trends - better?

2021-05-06 Thread Kenneth Knowles
I spoke too soon? https://issues.apache.org/jira/secure/ConfigureReport.jspa?projectOrFilterId=project-12319527=daily=300=12319527=com.atlassian.jira.jira-core-reports-plugin%3Aaverageage-report_token=A5KQ-2QAV-T4JA-FDED_ea6ac783c727523cf6bfed04ba94ce91bb62da91_lin=Next On Thu, May 6, 2021 at

Flake trends - better?

2021-05-06 Thread Kenneth Knowles
I made a quick* Jira chart to see how we are doing at flakes: https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=464=BEAM=reporting=cumulativeFlowDiagram=1174=1175=2038=2039=2040 Looking a lot better recently at resolving them! (whether these are new fixes or just resolving stale

Flaky test issue report

2021-05-06 Thread Beam Jira Bot
This is your daily summary of Beam's current flaky tests. These are P1 issues because they have a major negative impact on the community and make it hard to determine the quality of the software. BEAM-12303: Flake in PubSubIntegrationTest.test_streaming_with_attributes

P1 issues report

2021-05-06 Thread Beam Jira Bot
This is your daily summary of Beam's current P1 issues, not including flaky tests. See https://beam.apache.org/contribute/jira-priorities/#p1-critical for the meaning and expectations around P1 issues. BEAM-12290: TestPubsub.assertThatSubscriptionEventuallyCreated timeout does not work

Re: [Proposal] Support State Batching and Prefetching over FnApi

2021-05-06 Thread Rui Wang
At this moment, the third approach in the doc is preferred. To recap, the third approach is the one that only changes FnApi by adding a repeated field in the state request to support batching over FnApi. This approach has the following benefits: 1. Avoid double requests problem introduced by

Re: BEAM-11838: An x-lang wrapper for DebeziumIO

2021-05-06 Thread Pablo Estrada
Hi Benjamin! That's strange if the Java test is working well that the Python/xlang test would run into issues. Let me take a look at your PR and see if I can see anything suspicious. Best -P. On Thu, May 6, 2021 at 10:49 AM Benjamin Gonzalez Delgado < benjamin.gonza...@wizeline.com> wrote: > Hi

Re: Please help with the 2.29.0 release announcement! (curate the bugs)

2021-05-06 Thread Kenneth Knowles
Yes. Done! On Thu, May 6, 2021 at 7:06 AM Alexey Romanenko wrote: > Don’t we need to make 2.29.0 “released" in Jira to make it possible to > specify it in “Affects Version/s” since the artefacts are already > available? > > On 28 Apr 2021, at 19:23, Brian Hulette wrote: > > Here's a query to

BEAM-11838: An x-lang wrapper for DebeziumIO

2021-05-06 Thread Benjamin Gonzalez Delgado
Hi team, I am working on BEAM-11838[1], but I am a little stuck right now[2], my issue is that when I set param maxNumberOfRecords through ExternalTransformBuilder(L117) its sets the value correctly to the constructor on KafkaSourceConsumerFn (L111), but when it arrives at tryClaim method(L307)

Re: Potential bug with Kafka (or other external) IO in Python Portable Runner

2021-05-06 Thread Chamikara Jayalath
On Thu, May 6, 2021 at 4:58 AM Nir Gazit wrote: > Hey, > Not sure I follow - from the code you sent me, it seems that the > environment is chosen according to the pipeline options > (JAVA_SDK_HARNESS_ENVIRONMENT is used by createOrGetDefaultEnvironment only > AFAIU). So if I'm passing

Re: Please help with the 2.29.0 release announcement! (curate the bugs)

2021-05-06 Thread Alexey Romanenko
Don’t we need to make 2.29.0 “released" in Jira to make it possible to specify it in “Affects Version/s” since the artefacts are already available? > On 28 Apr 2021, at 19:23, Brian Hulette wrote: > > Here's a query to find 2.29.0 issues where you are the author or reporter [1]. > > [1] >

Re: Potential bug with Kafka (or other external) IO in Python Portable Runner

2021-05-06 Thread Nir Gazit
Hey, Not sure I follow - from the code you sent me, it seems that the environment is chosen according to the pipeline options (JAVA_SDK_HARNESS_ENVIRONMENT is used by createOrGetDefaultEnvironment only AFAIU). So if I'm passing `--environment_type=EXTERNAL` why wouldn't I get to using an external

Re: Window Assignment Across SplittableDoFn

2021-05-06 Thread Evan Galpin
Thanks! On Wed, May 5, 2021 at 23:14 Boyuan Zhang wrote: > Hi, > Yes, just like normal DoFn, Splittable DoFn preserves the window > information as well. > > On Wed, May 5, 2021 at 8:04 PM Evan Galpin wrote: > >> Hi folks, >> >> I’d just like to confirm what happens to window assignments