Re: [RESULT] [VOTE] Vendored Dependency guava 32.1.2-jre Release

[Yi Hu via dev]

Hi everyone,

The vendored guava 32.1.2-jre is now released, and beam is switched to use
this version [1]. If you have working pull requests that have called vendor
guava directly (Java SDK), it may need to rebase the PR branch onto the
latest master branch, and switch `guava.v26_0_jre` namespace to
`guava.v32_1_2_jre`. Please do not hesitate to reply to this thread or pin
me (github username Abacn) in your PR if there is any question.

Best,
Yi

[1] https://github.com/apache/beam/pull/27895

On Tue, Aug 8, 2023 at 9:22 AM Yi Hu  wrote:

> Hi,
>
> Could a PMC member please help me finalize the source release? That is
> copy https://dist.apache.org/repos/dist/dev/beam/vendor/guava-32_1_2-jre/
> into
> https://dist.apache.org/repos/dist/release/beam/vendor/guava-32_1_2-jre
> using svn. Thanks!
>
> Regards,
> Yi
>
> On Tue, Aug 8, 2023 at 9:01 AM Yi Hu  wrote:
>
>> I'm happy to announce that we have unanimously approved this release.
>>
>> There are 5 approving votes, 3 of which are binding:
>>
>> * Chamikara Jayalath
>> * Ahmet Altay
>> * Robert Bradshaw
>>
>> There are no disapproving votes.
>>
>> Thanks everyone!
>>
>> --
>>
>> Yi Hu, (he/him/his)
>>
>> Software Engineer
>>
>>
>>

Re: [Discuss] Get rid of OWNERS files

[Danny McCormick via dev]

> I’m generally agree with this (initially that was a good intention imho)
but what could be an alternative for this? Review bot also may assign
reviewers that are no longer active on the project.

I propose that the reviewer bot be the full replacement, and I agree with
Robert's comments about it being better at staying up to date. I don't
think it is perfect, but I do think it's an upgrade over OWNERS.

> IMO folks who aren't active but are still getting emails and review
requests should be incentivised to redirect requests to new owners or at
least active members.

+1 - annecdotally I've seen this happen several times
, but
I've never seen it happen with an OWNERS file.

On Tue, Aug 8, 2023 at 12:50 PM Robert Burke  wrote:

> Either we keep OWNERS and have the review bot use them, or we remove them
> and use the reviews bot config as the single source of truth.
>
> The bot is less likely to go out of date since it's at least active in how
> it behaves. I agree it doesn't necessarily solve the problem of things
> getting out of date, but other than inactive folks officially, actively
> bowing out of the project, I don't know there's anything we can do.
>
> IMO folks who aren't active but are still getting emails and review
> requests should be incentivised to redirect requests to new owners or at
> least active members.
>
>
> On Tue, Aug 8, 2023, 9:13 AM Alexey Romanenko 
> wrote:
>
>> I’m generally agree with this (initially that was a good intention imho)
>> but what could be an alternative for this? Review bot also may assign
>> reviewers that are no longer active on the project.
>>
>> —
>> Alexey
>>
>>
>> On 8 Aug 2023, at 16:55, Danny McCormick via dev 
>> wrote:
>>
>> Hey everyone, I'd like to propose getting rid of OWNERS files from the
>> Beam repo. Right now, I don't think they are serving a meaningful purpose:
>>
>> - Many OWNERS files are outdated and point to people who are no longer
>> actively involved in the project (examples: 1
>> , 2
>> , 3
>> ,
>> there are many more)
>> - Many dependencies don't have owners assigned
>> - Many major directories function fine without OWNERS files
>> - We lack sufficient documentation of what OWNERS files mean (
>> https://s.apache.org/beam-owners is not helpful and I couldn't find
>> other resources)
>> - We now have the review bot to automatically assign reviewers based on
>> areas of ownership. That has proven more likely to stay up to date.
>>
>> Given all of these, I don't see any obvious usefulness for OWNERS files.
>> Please chime in if you disagree (or agree). If there are no objections I'll
>> assume silent consensus and remove them next week.
>>
>> Thanks,
>> Danny
>>
>>
>>

Re: [Discuss] Get rid of OWNERS files

[Robert Burke]

Either we keep OWNERS and have the review bot use them, or we remove them
and use the reviews bot config as the single source of truth.

The bot is less likely to go out of date since it's at least active in how
it behaves. I agree it doesn't necessarily solve the problem of things
getting out of date, but other than inactive folks officially, actively
bowing out of the project, I don't know there's anything we can do.

IMO folks who aren't active but are still getting emails and review
requests should be incentivised to redirect requests to new owners or at
least active members.


On Tue, Aug 8, 2023, 9:13 AM Alexey Romanenko 
wrote:

> I’m generally agree with this (initially that was a good intention imho)
> but what could be an alternative for this? Review bot also may assign
> reviewers that are no longer active on the project.
>
> —
> Alexey
>
>
> On 8 Aug 2023, at 16:55, Danny McCormick via dev 
> wrote:
>
> Hey everyone, I'd like to propose getting rid of OWNERS files from the
> Beam repo. Right now, I don't think they are serving a meaningful purpose:
>
> - Many OWNERS files are outdated and point to people who are no longer
> actively involved in the project (examples: 1
> , 2
> , 3
> ,
> there are many more)
> - Many dependencies don't have owners assigned
> - Many major directories function fine without OWNERS files
> - We lack sufficient documentation of what OWNERS files mean (
> https://s.apache.org/beam-owners is not helpful and I couldn't find other
> resources)
> - We now have the review bot to automatically assign reviewers based on
> areas of ownership. That has proven more likely to stay up to date.
>
> Given all of these, I don't see any obvious usefulness for OWNERS files.
> Please chime in if you disagree (or agree). If there are no objections I'll
> assume silent consensus and remove them next week.
>
> Thanks,
> Danny
>
>
>

Re: [Discuss] Get rid of OWNERS files

[Alexey Romanenko]

I’m generally agree with this (initially that was a good intention imho) but 
what could be an alternative for this? Review bot also may assign reviewers 
that are no longer active on the project.

—
Alexey


> On 8 Aug 2023, at 16:55, Danny McCormick via dev  wrote:
> 
> Hey everyone, I'd like to propose getting rid of OWNERS files from the Beam 
> repo. Right now, I don't think they are serving a meaningful purpose:
> 
> - Many OWNERS files are outdated and point to people who are no longer 
> actively involved in the project (examples: 1 
> , 2 
> , 3 
> , 
> there are many more)
> - Many dependencies don't have owners assigned
> - Many major directories function fine without OWNERS files
> - We lack sufficient documentation of what OWNERS files mean 
> (https://s.apache.org/beam-owners is not helpful and I couldn't find other 
> resources) 
> - We now have the review bot to automatically assign reviewers based on areas 
> of ownership. That has proven more likely to stay up to date.
> 
> Given all of these, I don't see any obvious usefulness for OWNERS files. 
> Please chime in if you disagree (or agree). If there are no objections I'll 
> assume silent consensus and remove them next week.
> 
> Thanks,
> Danny

Re: [Discuss] Get rid of OWNERS files

[Kerry Donny-Clark via dev]

Thanks Danny!
I agree. OWNERS causes unnecessary friction, and doesn't provide value.
Kerry

On Tue, Aug 8, 2023 at 10:55 AM Danny McCormick via dev 
wrote:

> Hey everyone, I'd like to propose getting rid of OWNERS files from the
> Beam repo. Right now, I don't think they are serving a meaningful purpose:
>
> - Many OWNERS files are outdated and point to people who are no longer
> actively involved in the project (examples: 1
> , 2
> , 3
> ,
> there are many more)
> - Many dependencies don't have owners assigned
> - Many major directories function fine without OWNERS files
> - We lack sufficient documentation of what OWNERS files mean (
> https://s.apache.org/beam-owners is not helpful and I couldn't find other
> resources)
> - We now have the review bot to automatically assign reviewers based on
> areas of ownership. That has proven more likely to stay up to date.
>
> Given all of these, I don't see any obvious usefulness for OWNERS files.
> Please chime in if you disagree (or agree). If there are no objections I'll
> assume silent consensus and remove them next week.
>
> Thanks,
> Danny
>

Re: [Discuss] Get rid of OWNERS files

[XQ Hu via dev]

+1.

On Tue, Aug 8, 2023 at 10:55 AM Danny McCormick via dev 
wrote:

> Hey everyone, I'd like to propose getting rid of OWNERS files from the
> Beam repo. Right now, I don't think they are serving a meaningful purpose:
>
> - Many OWNERS files are outdated and point to people who are no longer
> actively involved in the project (examples: 1
> , 2
> , 3
> ,
> there are many more)
> - Many dependencies don't have owners assigned
> - Many major directories function fine without OWNERS files
> - We lack sufficient documentation of what OWNERS files mean (
> https://s.apache.org/beam-owners is not helpful and I couldn't find other
> resources)
> - We now have the review bot to automatically assign reviewers based on
> areas of ownership. That has proven more likely to stay up to date.
>
> Given all of these, I don't see any obvious usefulness for OWNERS files.
> Please chime in if you disagree (or agree). If there are no objections I'll
> assume silent consensus and remove them next week.
>
> Thanks,
> Danny
>

Re: [RESULT] [VOTE] Vendored Dependency guava 32.1.2-jre Release

[Yi Hu via dev]

Hi,

Could a PMC member please help me finalize the source release? That is copy
https://dist.apache.org/repos/dist/dev/beam/vendor/guava-32_1_2-jre/ into
https://dist.apache.org/repos/dist/release/beam/vendor/guava-32_1_2-jre
using svn. Thanks!

Regards,
Yi

On Tue, Aug 8, 2023 at 9:01 AM Yi Hu  wrote:

> I'm happy to announce that we have unanimously approved this release.
>
> There are 5 approving votes, 3 of which are binding:
>
> * Chamikara Jayalath
> * Ahmet Altay
> * Robert Bradshaw
>
> There are no disapproving votes.
>
> Thanks everyone!
>
> --
>
> Yi Hu, (he/him/his)
>
> Software Engineer
>
>
>

[RESULT] [VOTE] Vendored Dependency guava 32.1.2-jre Release

[Yi Hu via dev]

I'm happy to announce that we have unanimously approved this release.

There are 5 approving votes, 3 of which are binding:

* Chamikara Jayalath
* Ahmet Altay
* Robert Bradshaw

There are no disapproving votes.

Thanks everyone!

-- 

Yi Hu, (he/him/his)

Software Engineer

Beam Metrics Report (2023-08-08)

[Apache Jenkins Server]

ERROR: File 'src/.test-infra/jenkins/metrics_report/beam-metrics_report.html' does not exist

Re: [VOTE] Vendored Dependency guava 32.1.2-jre Release

[Hong Liang]

+1 (non-binding)

- Verified 512 checksum of released artifacts
- Reviewed test PR and found that the two failing CI tests are due to
unrelated issues

Thanks for driving this, Yi Hu!

Regards,
Hong

On Thu, Aug 3, 2023 at 8:02 PM Yi Hu via dev  wrote:

> Hi everyone,
>
>
> Please review the release of the following artifacts that we vendor:
>
>
> * beam-vendor-guava-32_1_2-jre
>
>
> Please review and vote on the release candidate 1 for the version 0.1, as
> follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
>
> The complete staging area is available for your review, which includes:
> * the official Apache source release to be deployed to dist.apache.org
> [1], which is signed with the key with fingerprint 170405CB [2],
> * all artifacts to be deployed to the Maven Central Repository [3],
> * commit hash "ef2ca7a" [4],
>
> * Testing PR on the vendored dependency [5]
>
> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> Release Manager
>
> [1] https://dist.apache.org/repos/dist/dev/beam/vendor/
> [2] https://dist.apache.org/repos/dist/release/beam/KEYS
> [3] https://repository.apache.org/content/repositories/orgapachebeam-1350/
> [4]
> https://github.com/apache/beam/commit/ef2ca7aa49ec75c3ab4e3a94b7ad8162e1c81c1e
> [5] https://github.com/apache/beam/pull/27825
>
> Regards,
> Yi
>
> --
>
> Yi Hu, (he/him/his)
>
> Software Engineer
>
>
>

Beam High Priority Issue Report (39)

[beamactions]

This is your daily summary of Beam's current high priority issues that may need 
attention.

See https://beam.apache.org/contribute/issue-priorities for the meaning and 
expectations around issue priorities.

Unassigned P1 Issues:

https://github.com/apache/beam/issues/27892 [Bug]: ignoreUnknownValues not 
working when using CreateDisposition.CREATE_IF_NEEDED 
https://github.com/apache/beam/issues/27648 [Bug]: Python SDFs (e.g. 
PeriodicImpulse) running in Flink and polling using tracker.defer_remainder 
have checkpoint size growing indefinitely 
https://github.com/apache/beam/issues/27616 [Bug]: Unable to use 
applyRowMutations() in bigquery IO apache beam java
https://github.com/apache/beam/issues/27486 [Bug]: Read from datastore with 
inequality filters
https://github.com/apache/beam/issues/27314 [Failing Test]: 
bigquery.StorageApiSinkCreateIfNeededIT.testCreateManyTables[1]
https://github.com/apache/beam/issues/27238 [Bug]: Window trigger has lag when 
using Kafka and GroupByKey on Dataflow Runner
https://github.com/apache/beam/issues/26981 [Bug]: Getting an error related to 
SchemaCoder after upgrading to 2.48
https://github.com/apache/beam/issues/26969 [Failing Test]: Python PostCommit 
is failing due to exceeded rate limits
https://github.com/apache/beam/issues/26911 [Bug]: UNNEST ARRAY with a nested 
ROW (described below)
https://github.com/apache/beam/issues/26354 [Bug]: BigQueryIO direct read not 
reading all rows when set --setEnableBundling=true
https://github.com/apache/beam/issues/26343 [Bug]: 
apache_beam.io.gcp.bigquery_read_it_test.ReadAllBQTests.test_read_queries is 
flaky
https://github.com/apache/beam/issues/26329 [Bug]: BigQuerySourceBase does not 
propagate a Coder to AvroSource
https://github.com/apache/beam/issues/26041 [Bug]: Unable to create 
exactly-once Flink pipeline with stream source and file sink
https://github.com/apache/beam/issues/25975 [Bug]: Reducing parallelism in 
FlinkRunner leads to a data loss
https://github.com/apache/beam/issues/24776 [Bug]: Race condition in Python SDK 
Harness ProcessBundleProgress
https://github.com/apache/beam/issues/24389 [Failing Test]: 
HadoopFormatIOElasticTest.classMethod ExceptionInInitializerError 
ContainerFetchException
https://github.com/apache/beam/issues/24313 [Flaky]: 
apache_beam/runners/portability/portable_runner_test.py::PortableRunnerTestWithSubprocesses::test_pardo_state_with_custom_key_coder
https://github.com/apache/beam/issues/23944  beam_PreCommit_Python_Cron 
regularily failing - test_pardo_large_input flaky
https://github.com/apache/beam/issues/23709 [Flake]: Spark batch flakes in 
ParDoLifecycleTest.testTeardownCalledAfterExceptionInProcessElement and 
ParDoLifecycleTest.testTeardownCalledAfterExceptionInStartBundle
https://github.com/apache/beam/issues/23525 [Bug]: Default PubsubMessage coder 
will drop message id and orderingKey
https://github.com/apache/beam/issues/22913 [Bug]: 
beam_PostCommit_Java_ValidatesRunner_Flink is flakes in 
org.apache.beam.sdk.transforms.GroupByKeyTest$BasicTests.testAfterProcessingTimeContinuationTriggerUsingState
https://github.com/apache/beam/issues/22605 [Bug]: Beam Python failure for 
dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest.test_metrics_it
https://github.com/apache/beam/issues/21714 
PulsarIOTest.testReadFromSimpleTopic is very flaky
https://github.com/apache/beam/issues/21708 beam_PostCommit_Java_DataflowV2, 
testBigQueryStorageWrite30MProto failing consistently
https://github.com/apache/beam/issues/21706 Flaky timeout in github Python unit 
test action 
StatefulDoFnOnDirectRunnerTest.test_dynamic_timer_clear_then_set_timer
https://github.com/apache/beam/issues/21643 FnRunnerTest with non-trivial 
(order 1000 elements) numpy input flakes in non-cython environment
https://github.com/apache/beam/issues/21476 WriteToBigQuery Dynamic table 
destinations returns wrong tableId
https://github.com/apache/beam/issues/21469 beam_PostCommit_XVR_Flink flaky: 
Connection refused
https://github.com/apache/beam/issues/21424 Java VR (Dataflow, V2, Streaming) 
failing: ParDoTest$TimestampTests/OnWindowExpirationTests
https://github.com/apache/beam/issues/21262 Python AfterAny, AfterAll do not 
follow spec
https://github.com/apache/beam/issues/21260 Python DirectRunner does not emit 
data at GC time
https://github.com/apache/beam/issues/21121 
apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT.test_streaming_wordcount_it
 flakey
https://github.com/apache/beam/issues/21104 Flaky: 
apache_beam.runners.portability.fn_api_runner.fn_runner_test.FnApiRunnerTestWithGrpcAndMultiWorkers
https://github.com/apache/beam/issues/20976 
apache_beam.runners.portability.flink_runner_test.FlinkRunnerTestOptimized.test_flink_metrics
 is flaky
https://github.com/apache/beam/issues/20108 Python direct runner doesn't emit 
empty pane when it should
https://github.com/apache/beam/issues/19814 Flink streaming flakes in