Re: Java post commit: HL7v2IOReadWriteIT failures

2021-09-10 Thread Tomo Suzuki
Here is a PR to fix: https://github.com/apache/beam/pull/15498 I confirmed "Run Java Postcommit" passes. On Fri, Sep 10, 2021 at 4:32 PM Tomo Suzuki wrote: > I'm troubleshooting Java postcommit build failure on HL7v2IOReadWriteIT. > > https://issues.apache.org/jira/browse/BEAM-12873 > > > org.ap

Re: [PROPOSAL] Preparing for Beam 2.33.0 Release

2021-09-10 Thread Udi Meiri
The release branch is now passing Jenkins tests: https://github.com/apache/beam/pull/15431 Thank you all so far for help with release blockers. There are 5 left . On Thu, Aug 26, 2021 at 7:54 AM Alexey Romanenko wrote: > See my com

DRAFT - Beam board report September 2021

2021-09-10 Thread Kenneth Knowles
Hi all, The September report is due today. Sorry for the late notice! Through this link anyone should be able to suggest changes. Please help me out by adding highlights. Please help populate it! https://s.apache.org/beam-draft-report-2021-09 You can read past reports at https://whimsy.apache.o

Flaky test issue report (34)

2021-09-10 Thread Beam Jira Bot
This is your daily summary of Beam's current flaky tests (https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20statusCategory%20!%3D%20Done%20AND%20labels%20%3D%20flake) These are P1 issues because they have a major negative impact on the community and make it hard to determin

P1 issues report (44)

2021-09-10 Thread Beam Jira Bot
This is your daily summary of Beam's current P1 issues, not including flaky tests (https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20statusCategory%20!%3D%20Done%20AND%20priority%20%3D%20P1%20AND%20(labels%20is%20EMPTY%20OR%20labels%20!%3D%20flake). See https://beam.apache.

P0 (outage) report

2021-09-10 Thread Beam Jira Bot
This is your daily summary of Beam's current outages. See https://beam.apache.org/contribute/jira-priorities/#p0-outage for the meaning and expectations around P0 issues. BEAM-12871: Support for Confluent Schema Registry (https://issues.apache.org/jira/browse/BEAM-12871) BEAM-12805: Mul

Re: MapState API

2021-09-10 Thread Ke Wu
Another question on this topic, is any work planned to add the map state support in portable mode [1], same for set state, list state? [1] https://github.com/apache/beam/blob/master/sdks/java/harness/src/main/java/org/apache/beam/fn/harness/state/FnApiStateAccessor.java#L337

Does python DirectRunner correctly set watermark hold for timers?

2021-09-10 Thread Jan Lukavský
Hi, I didn't dig into the code yet, just wanted to ask if someone could give me some pointers. I'm experiencing weird behavior in python SDK, when emitting data from stateful DoFn with event-time timer set on GlobalWindow().max_timestamp(). The data is correctly emitted, but is probably delay

Re: [Proposal] Default tempDirectory setting in file-based IOs

2021-09-10 Thread Claire McGinty
Niel, that's a good point -- I don't think there's any restriction on the filesystem of PipelineOptions#tempLocation, I was able to run a job on DirectRunner with PipelineOptions#tempLocation set to a local path & my TextIO.Write#outputDirectory set to a remote filesystem. But currently there's al