Beam Dependency Check Report (2023-06-01)

[Apache Jenkins Server]

<<< text/html; charset=UTF-8: Unrecognized >>>

Re: [Proposal] Remove of Fix the Beam Dependency Check Report Job

[Jack McCluskey via dev]

No one has spoken in defense of the email, so I'll start advancing a PR to
remove the job.

Thanks,

Jack McCluskey

On Tue, May 30, 2023 at 1:32 PM Kenneth Knowles  wrote:

> +1 to just stopping the automated email. I don't find it valuable. It was
> never finely-tuned enough in terms of actionability vs spam volume.
>
> On Tue, May 30, 2023 at 7:24 AM Jack McCluskey via dev <
> dev@beam.apache.org> wrote:
>
>> Hi everyone,
>>
>> Just bumping this again now that the long weekend is behind us. If no one
>> advocates for fixing the job in the next few days I'll assume a lazy
>> consensus and remove it.
>>
>> I also want to point out a typo in the subject, it should be "Remove *or*
>>  Fix."
>>
>> Thanks,
>>
>> Jack McCluskey
>>
>> On Thu, May 25, 2023 at 3:16 PM Jack McCluskey 
>> wrote:
>>
>>> Hey everyone,
>>>
>>> The Beam Dependency Check Report email (like
>>> https://lists.apache.org/thread/tc9v1d66rx77wzvrjnkcf0jo3rxtmrhn) has
>>> not had a successful incarnation since July 21st, 2022. I've done a little
>>> bit of digging into the problem and have found that the issue lies in a
>>> query
>>> 
>>> to a "Python Compatibility Checking Service" that is just an IP address
>>> also taking the package name, version, and then specifying that it wants
>>> Python 2 packages specifically. I made a few brief attempts to figure out
>>> what that IP address was supposed to lead to and didn't turn up anything;
>>> however, that doesn't seem to matter since the root of the problem is that
>>> the job cannot connect to anything at that address, so the build fails and
>>> the email is sent out without a body.
>>>
>>> I started a bit of work this afternoon trying to update the job to
>>> direct its Python-related queries to PyPi's JSON API (
>>> https://github.com/apache/beam/pull/26897); however, I question the
>>> need for this automated email at all given that we added Dependabot to the
>>> repository around 6 weeks before the Jenkins job started failing. If
>>> there's a good reason to fix it I'll keep digging, otherwise I'm in favor
>>> of removing the job altogether.
>>>
>>> Thanks,
>>>
>>> Jack McCluskey
>>>
>>> --
>>>
>>>
>>> Jack McCluskey
>>> SWE - DataPLS PLAT/ Dataflow ML
>>> RDU
>>> jrmcclus...@google.com
>>>
>>>
>>>

Re: 2.48.0 Release PMC Finalization

[Ritesh Ghorse via dev]

Sure, let me know once that is done, I'll send out the announcement email.

Thank you!

On Thu, Jun 1, 2023 at 1:55 AM Jean-Baptiste Onofré  wrote:

> Hi
>
> I can do that today if there's no other PMC available.
>
> Regards
> JB
>
> On Thu, Jun 1, 2023 at 2:02 AM Ritesh Ghorse via dev
>  wrote:
> >
> > Could a PMC member help with the PMC-only finalization steps for 2.48.0
> [1]? Specifically:
> >
> > - Deploy source release to dist.apache.org
> > - Recordkeeping with ASF
> >
> > Once those steps are done all that's left is to promote the release [2].
> >
> > Thank you!
> >
> > [1]
> https://beam.apache.org/contribute/release-guide/#pmc-only-finalization
> > [2]
> https://beam.apache.org/contribute/release-guide/#12-promote-the-release
>


-- 
Regards,
Ritesh Ghorse

Beam High Priority Issue Report (31)

[beamactions]

This is your daily summary of Beam's current high priority issues that may need 
attention.

See https://beam.apache.org/contribute/issue-priorities for the meaning and 
expectations around issue priorities.

Unassigned P1 Issues:

https://github.com/apache/beam/issues/26911 [Bug]: UNNEST ARRAY with a nested 
ROW (described below)
https://github.com/apache/beam/issues/26547 [Failing Test]: 
beam_PostCommit_Java_DataflowV2
https://github.com/apache/beam/issues/26354 [Bug]: BigQueryIO direct read not 
reading all rows when set --setEnableBundling=true
https://github.com/apache/beam/issues/26343 [Bug]: 
apache_beam.io.gcp.bigquery_read_it_test.ReadAllBQTests.test_read_queries is 
flaky
https://github.com/apache/beam/issues/26329 [Bug]: BigQuerySourceBase does not 
propagate a Coder to AvroSource
https://github.com/apache/beam/issues/26041 [Bug]: Unable to create 
exactly-once Flink pipeline with stream source and file sink
https://github.com/apache/beam/issues/25975 [Bug]: Reducing parallelism in 
FlinkRunner leads to a data loss
https://github.com/apache/beam/issues/24776 [Bug]: Race condition in Python SDK 
Harness ProcessBundleProgress
https://github.com/apache/beam/issues/24389 [Failing Test]: 
HadoopFormatIOElasticTest.classMethod ExceptionInInitializerError 
ContainerFetchException
https://github.com/apache/beam/issues/24313 [Flaky]: 
apache_beam/runners/portability/portable_runner_test.py::PortableRunnerTestWithSubprocesses::test_pardo_state_with_custom_key_coder
https://github.com/apache/beam/issues/23944  beam_PreCommit_Python_Cron 
regularily failing - test_pardo_large_input flaky
https://github.com/apache/beam/issues/23709 [Flake]: Spark batch flakes in 
ParDoLifecycleTest.testTeardownCalledAfterExceptionInProcessElement and 
ParDoLifecycleTest.testTeardownCalledAfterExceptionInStartBundle
https://github.com/apache/beam/issues/22913 [Bug]: 
beam_PostCommit_Java_ValidatesRunner_Flink is flakes in 
org.apache.beam.sdk.transforms.GroupByKeyTest$BasicTests.testAfterProcessingTimeContinuationTriggerUsingState
https://github.com/apache/beam/issues/22605 [Bug]: Beam Python failure for 
dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest.test_metrics_it
https://github.com/apache/beam/issues/21714 
PulsarIOTest.testReadFromSimpleTopic is very flaky
https://github.com/apache/beam/issues/21708 beam_PostCommit_Java_DataflowV2, 
testBigQueryStorageWrite30MProto failing consistently
https://github.com/apache/beam/issues/21706 Flaky timeout in github Python unit 
test action 
StatefulDoFnOnDirectRunnerTest.test_dynamic_timer_clear_then_set_timer
https://github.com/apache/beam/issues/21643 FnRunnerTest with non-trivial 
(order 1000 elements) numpy input flakes in non-cython environment
https://github.com/apache/beam/issues/21476 WriteToBigQuery Dynamic table 
destinations returns wrong tableId
https://github.com/apache/beam/issues/21469 beam_PostCommit_XVR_Flink flaky: 
Connection refused
https://github.com/apache/beam/issues/21424 Java VR (Dataflow, V2, Streaming) 
failing: ParDoTest$TimestampTests/OnWindowExpirationTests
https://github.com/apache/beam/issues/21262 Python AfterAny, AfterAll do not 
follow spec
https://github.com/apache/beam/issues/21260 Python DirectRunner does not emit 
data at GC time
https://github.com/apache/beam/issues/21121 
apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT.test_streaming_wordcount_it
 flakey
https://github.com/apache/beam/issues/21104 Flaky: 
apache_beam.runners.portability.fn_api_runner.fn_runner_test.FnApiRunnerTestWithGrpcAndMultiWorkers
https://github.com/apache/beam/issues/20976 
apache_beam.runners.portability.flink_runner_test.FlinkRunnerTestOptimized.test_flink_metrics
 is flaky
https://github.com/apache/beam/issues/20108 Python direct runner doesn't emit 
empty pane when it should
https://github.com/apache/beam/issues/19814 Flink streaming flakes in 
ParDoLifecycleTest.testTeardownCalledAfterExceptionInStartBundleStateful and 
ParDoLifecycleTest.testTeardownCalledAfterExceptionInProcessElementStateful
https://github.com/apache/beam/issues/19465 Explore possibilities to lower 
in-use IP address quota footprint.


P1 Issues with no update in the last week:

https://github.com/apache/beam/issues/23525 [Bug]: Default PubsubMessage coder 
will drop message id and orderingKey
https://github.com/apache/beam/issues/21645 
beam_PostCommit_XVR_GoUsingJava_Dataflow fails on some test transforms