Re: [DISCUSS] What to do about P0/P1/flake automation Was: P1 issues report (70)

2022-06-23 Thread Manu Zhang
Sounds good! It’s like our internal reports of JIRA tickets exceeding SLA
time and having no response from engineers.  We either resolve them or
downgrade the priority to extend time window.

Besides,
1. P2 and P3 issues should be noticed and resolved as well. Shall we have a
longer time window for the rest of not triaged or stagnate issues and
include them?
2. The links in this report start with api.github.* and don’t take us
directly to the issues.


Danny McCormick 于2022年6月24日 周五04:48写道:

> That generally sounds right to me - I also would vote that we consolidate
> to 1 email and stop distinguishing between flaky P1s and normal P1s.
>
> So the single daily report would be:
>
> - Unassigned P0s
> - P0s with no update in the last 36 hours
> - Unassigned P1s
> - P1s with no update in the last 7 days
>
> I think that will generate a pretty good list of issues that require some
> kind of action.
>
> On Thu, Jun 23, 2022 at 4:43 PM Kenneth Knowles  wrote:
>
>> Sounds good to me. Perhaps P0s > 36 hours ago (presumably they are more
>> like ~hours for true outages of CI/website/etc) and P1s > 7 days?
>>
>> On Thu, Jun 23, 2022 at 1:27 PM Brian Hulette 
>> wrote:
>>
>>> I think that Danny's alternate proposal (a daily email that show only
>>> issues last updated >7 days ago, and those with no assignee) fits well with
>>> the two goals you describe, if we include "triage needed" issues in the
>>> latter category. Maybe we also explicitly separate these two concerns in
>>> the report?
>>>
>>>
>>> On Thu, Jun 23, 2022 at 1:14 PM Kenneth Knowles  wrote:
>>>
 Forking thread because lots of people may just ignore this topic, per
 the discussion :-)

 (sometimes gmail doesn't fork thread properly, but here's hoping...)

 I'll add some other outcomes of these emails:

  - people file P0s that are not outages and P1s that are not data loss
 and I downgrade them
  - I randomly open up a few flaky test bugs and see if I can fix them
 really quick
  - people file legit P0s and P1s and I subscribe and follow them

 Of these, only the last one seems important (not just that *I* follow
 them, but that new P0s and P1s get immediate attention from many eyes)

 So maybe one take on the goal is to:

  - have new P0s and P1s evaluated quickly: P0s are an outage or
 outage-like occurrence that needs immediate remedy, and P1s need to be
 evaluated for release blocking, etc.
  - make sure P0s and P1s get attention appropriate to their priority

 It can also be helpful to just state the failure modes which would
 happen by default if we don't have a good process or automation:

  - Real P0 gets filed and not noticed or fixed in a timely manner,
 blocking users and/or community in real time
  - Real P1 gets filed and not noticed, so release goes out with known
 data loss bug or other total loss of functionality
  - Non-real P0s and P1s accumulate, throwing off our data and making it
 hard to find the real problems
  - Flakes are never fixed

 WDYT?

 If we have P0s and P1s in the "awaiting triage" state, those are the
 ones we need to notice. Then for a P0 or P1 outside of that state, we just
 need some way of making sure it doesn't stagnate. Or if it does stagnate,
 that empirically demonstrates it isn't really P1 (just like our P2 to P3
 downgrade automation). If everything is P1, nothing is, as they say.

 Kenn

 On Thu, Jun 23, 2022 at 10:01 AM Danny McCormick <
 dannymccorm...@google.com> wrote:

> > Maybe it would be helpful to sort these by last update time (and
> potentially include that information in the email). Then we can at least
> prioritize them instead of looking at a big wall of issues.
>
> I agree that this is a good idea (and pretty trivial to do). I'll
> update the automation to do that once we get consensus on an approach.
>
> > I think the motivation for daily emails is that per the priorities
> guide [1] P1 issues should be getting "continuous status updates". If 
> these
> issues aren't actually that important, I think the noise is good as it
> should motivate us to prioritize them correctly. In practice that hasn't
> been happening though...
>
> I guess the questions here are:
>
> 1) What is the goal of this email?
> 2) Is it effective at accomplishing that goal.
>
> I think you're saying that the goal (or a goal) is to highlight issues
> that aren't getting the attention they need; if that's our goal, then I
> don't think this is a particularly effective mechanism for it because (a)
> its very unclear which issues fall into that category and (b) there are 
> too
> many to manually go through on a daily basis. From the email alone, it's
> not clear to me that any of the issues above "shouldn't" be P1s (though 
> I'd

Re: [DISCUSS] What to do about P0/P1/flake automation Was: P1 issues report (70)

2022-06-23 Thread Kenneth Knowles
Sounds good to me. Perhaps P0s > 36 hours ago (presumably they are more
like ~hours for true outages of CI/website/etc) and P1s > 7 days?

On Thu, Jun 23, 2022 at 1:27 PM Brian Hulette  wrote:

> I think that Danny's alternate proposal (a daily email that show only
> issues last updated >7 days ago, and those with no assignee) fits well with
> the two goals you describe, if we include "triage needed" issues in the
> latter category. Maybe we also explicitly separate these two concerns in
> the report?
>
>
> On Thu, Jun 23, 2022 at 1:14 PM Kenneth Knowles  wrote:
>
>> Forking thread because lots of people may just ignore this topic, per the
>> discussion :-)
>>
>> (sometimes gmail doesn't fork thread properly, but here's hoping...)
>>
>> I'll add some other outcomes of these emails:
>>
>>  - people file P0s that are not outages and P1s that are not data loss
>> and I downgrade them
>>  - I randomly open up a few flaky test bugs and see if I can fix them
>> really quick
>>  - people file legit P0s and P1s and I subscribe and follow them
>>
>> Of these, only the last one seems important (not just that *I* follow
>> them, but that new P0s and P1s get immediate attention from many eyes)
>>
>> So maybe one take on the goal is to:
>>
>>  - have new P0s and P1s evaluated quickly: P0s are an outage or
>> outage-like occurrence that needs immediate remedy, and P1s need to be
>> evaluated for release blocking, etc.
>>  - make sure P0s and P1s get attention appropriate to their priority
>>
>> It can also be helpful to just state the failure modes which would happen
>> by default if we don't have a good process or automation:
>>
>>  - Real P0 gets filed and not noticed or fixed in a timely manner,
>> blocking users and/or community in real time
>>  - Real P1 gets filed and not noticed, so release goes out with known
>> data loss bug or other total loss of functionality
>>  - Non-real P0s and P1s accumulate, throwing off our data and making it
>> hard to find the real problems
>>  - Flakes are never fixed
>>
>> WDYT?
>>
>> If we have P0s and P1s in the "awaiting triage" state, those are the ones
>> we need to notice. Then for a P0 or P1 outside of that state, we just need
>> some way of making sure it doesn't stagnate. Or if it does stagnate, that
>> empirically demonstrates it isn't really P1 (just like our P2 to P3
>> downgrade automation). If everything is P1, nothing is, as they say.
>>
>> Kenn
>>
>> On Thu, Jun 23, 2022 at 10:01 AM Danny McCormick <
>> dannymccorm...@google.com> wrote:
>>
>>> > Maybe it would be helpful to sort these by last update time (and
>>> potentially include that information in the email). Then we can at least
>>> prioritize them instead of looking at a big wall of issues.
>>>
>>> I agree that this is a good idea (and pretty trivial to do). I'll update
>>> the automation to do that once we get consensus on an approach.
>>>
>>> > I think the motivation for daily emails is that per the priorities
>>> guide [1] P1 issues should be getting "continuous status updates". If these
>>> issues aren't actually that important, I think the noise is good as it
>>> should motivate us to prioritize them correctly. In practice that hasn't
>>> been happening though...
>>>
>>> I guess the questions here are:
>>>
>>> 1) What is the goal of this email?
>>> 2) Is it effective at accomplishing that goal.
>>>
>>> I think you're saying that the goal (or a goal) is to highlight issues
>>> that aren't getting the attention they need; if that's our goal, then I
>>> don't think this is a particularly effective mechanism for it because (a)
>>> its very unclear which issues fall into that category and (b) there are too
>>> many to manually go through on a daily basis. From the email alone, it's
>>> not clear to me that any of the issues above "shouldn't" be P1s (though I'd
>>> guess you're right that some/many of them don't belong since most were
>>> created before the Jira -> GH migration based on the titles). I'd also
>>> argue that a daily email just desensitizes us to them since there almost
>>> always will be *some *valid P1s that don't need extra attention.
>>>
>>> I do still think this could have value as a weekly email, with the goal
>>> being "it's probably a good idea for someone to take a look at each of
>>> these". Another option would be to only include issues with no action in
>>> the last 7 days and/or no assignees and keep it daily.
>>>
>>> A couple side notes:
>>> - No matter what we do, if we keep the current automation in any form we
>>> should fix the url from
>>> https://api.github.com/repos/apache/beam/issues/# to
>>> https://github.com/apache/beam/issues/# - the current links are very
>>> annoying.
>>> - After I send this, I will do a pass of the current P1s since it does
>>> indeed seem like too many are P1s and many should actually be P2s (or
>>> lower).
>>>
>>> Thanks,
>>> Danny
>>>
>>> On Thu, Jun 23, 2022 at 12:21 PM Brian Hulette 
>>> wrote:
>>>
 I think the motivation for daily emails is 

Some Beam Pulsar Connector Improvements

2022-06-23 Thread Neng Lu
Hi folks,

I'm trying to provide some improvements for the beam pulsar connector.
Here are the very first two issues:

- async produce: https://github.com/apache/beam/issues/22025
- take auth params: https://github.com/apache/beam/issues/22027

PRs links are added into the issue also. Please take a look and let me know
what you think.

-- 
Best Regards,
Neng Lu


[DISCUSS] What to do about P0/P1/flake automation Was: P1 issues report (70)

2022-06-23 Thread Kenneth Knowles
Forking thread because lots of people may just ignore this topic, per the
discussion :-)

(sometimes gmail doesn't fork thread properly, but here's hoping...)

I'll add some other outcomes of these emails:

 - people file P0s that are not outages and P1s that are not data loss and
I downgrade them
 - I randomly open up a few flaky test bugs and see if I can fix them
really quick
 - people file legit P0s and P1s and I subscribe and follow them

Of these, only the last one seems important (not just that *I* follow them,
but that new P0s and P1s get immediate attention from many eyes)

So maybe one take on the goal is to:

 - have new P0s and P1s evaluated quickly: P0s are an outage or outage-like
occurrence that needs immediate remedy, and P1s need to be evaluated for
release blocking, etc.
 - make sure P0s and P1s get attention appropriate to their priority

It can also be helpful to just state the failure modes which would happen
by default if we don't have a good process or automation:

 - Real P0 gets filed and not noticed or fixed in a timely manner, blocking
users and/or community in real time
 - Real P1 gets filed and not noticed, so release goes out with known data
loss bug or other total loss of functionality
 - Non-real P0s and P1s accumulate, throwing off our data and making it
hard to find the real problems
 - Flakes are never fixed

WDYT?

If we have P0s and P1s in the "awaiting triage" state, those are the ones
we need to notice. Then for a P0 or P1 outside of that state, we just need
some way of making sure it doesn't stagnate. Or if it does stagnate, that
empirically demonstrates it isn't really P1 (just like our P2 to P3
downgrade automation). If everything is P1, nothing is, as they say.

Kenn

On Thu, Jun 23, 2022 at 10:01 AM Danny McCormick 
wrote:

> > Maybe it would be helpful to sort these by last update time (and
> potentially include that information in the email). Then we can at least
> prioritize them instead of looking at a big wall of issues.
>
> I agree that this is a good idea (and pretty trivial to do). I'll update
> the automation to do that once we get consensus on an approach.
>
> > I think the motivation for daily emails is that per the priorities guide
> [1] P1 issues should be getting "continuous status updates". If these
> issues aren't actually that important, I think the noise is good as it
> should motivate us to prioritize them correctly. In practice that hasn't
> been happening though...
>
> I guess the questions here are:
>
> 1) What is the goal of this email?
> 2) Is it effective at accomplishing that goal.
>
> I think you're saying that the goal (or a goal) is to highlight issues
> that aren't getting the attention they need; if that's our goal, then I
> don't think this is a particularly effective mechanism for it because (a)
> its very unclear which issues fall into that category and (b) there are too
> many to manually go through on a daily basis. From the email alone, it's
> not clear to me that any of the issues above "shouldn't" be P1s (though I'd
> guess you're right that some/many of them don't belong since most were
> created before the Jira -> GH migration based on the titles). I'd also
> argue that a daily email just desensitizes us to them since there almost
> always will be *some *valid P1s that don't need extra attention.
>
> I do still think this could have value as a weekly email, with the goal
> being "it's probably a good idea for someone to take a look at each of
> these". Another option would be to only include issues with no action in
> the last 7 days and/or no assignees and keep it daily.
>
> A couple side notes:
> - No matter what we do, if we keep the current automation in any form we
> should fix the url from https://api.github.com/repos/apache/beam/issues/#
> to https://github.com/apache/beam/issues/# - the current links are very
> annoying.
> - After I send this, I will do a pass of the current P1s since it does
> indeed seem like too many are P1s and many should actually be P2s (or
> lower).
>
> Thanks,
> Danny
>
> On Thu, Jun 23, 2022 at 12:21 PM Brian Hulette 
> wrote:
>
>> I think the motivation for daily emails is that per the priorities guide
>> [1] P1 issues should be getting "continuous status updates". If these
>> issues aren't actually that important, I think the noise is good as it
>> should motivate us to prioritize them correctly. In practice that hasn't
>> been happening though...
>>
>> Maybe it would be helpful to sort these by last update time (and
>> potentially include that information in the email). Then we can at least
>> prioritize them instead of looking at a big wall of issues.
>>
>> Brian
>>
>> [1] https://beam.apache.org/contribute/issue-priorities/
>>
>> On Thu, Jun 23, 2022 at 6:07 AM Danny McCormick <
>> dannymccorm...@google.com> wrote:
>>
>>> I think a weekly summary seems like a good idea for the P1 issues and
>>> flaky tests, though daily still seems appropriate for P0 issues. I put up
>>> 

Re: [CdapIO] Dereference of possibly-null reference warning

2022-06-23 Thread Kenneth Knowles
Hi Elizaveta,

Very good question.

For this, I created
org.apache.beam.sdk.util.Preconditions.checkStateNotNull (
https://github.com/apache/beam/blob/1851eef15c5f455c95402ced50d757ea167d33d9/sdks/java/core/src/main/java/org/apache/beam/sdk/util/Preconditions.java#L450
)

Multiple reasons:

   - The annotations on this method are more useful than the ones in the
   equivalent Guava method.
   - The exception thrown is an IllegalStateException rather than NPE.

Hope this helps!

Kenn

On Fri, Jun 17, 2022 at 11:30 AM Elizaveta Lomteva <
elizaveta.lomt...@akvelon.com> wrote:

> Hi community,
>
> I have a question about the compileJava check, null dereference warning.
>
> We use the checkState()/checkNotNull() methods instead of the if-else
> statement to check that the variable is not null to follow the convention,
> but this leads to a dereferencing warning for a possibly null reference.
>
> This is an example
>
>
>
> Could you please suggest what we should do in this case:
> 1. leave the checkState()/checkNotNull() methods and suppress the
> dereference of possibly-null reference warning;
> 2. leave the if-else clause?
>
> What would you recommend?
>
> Thanks in advance,
> Elizaveta
>
>


Code review for P1 CassandraIO fix

2022-06-23 Thread Vincent Marquez
Hello.  I submitted a PR to fix a P1 issue but haven't gotten anyone to
review it yet.  Would anyone be able to give some feedback or help out with
this issue?  Thanks in advance.

https://github.com/apache/beam/pull/21786

*~Vincent*


P1 issues report (76)

2022-06-23 Thread beamactions
This is your daily summary of Beam's current P1 issues, not including flaky 
tests.

See https://beam.apache.org/contribute/issue-priorities/#p1-critical for 
the meaning and expectations around P1 issues.



https://api.github.com/repos/apache/beam/issues/22011: [Bug]: 
org.apache.beam.sdk.io.aws2.kinesis.KinesisIOWriteTest.testWriteFailure flaky
https://api.github.com/repos/apache/beam/issues/22010: [Bug]: 
org.apache.beam.runners.flink.FlinkRunnerTest.testEnsureStdoutStdErrIsRestored 
flaky
https://api.github.com/repos/apache/beam/issues/22009: [Bug]: 
org.apache.beam.examples.subprocess.ExampleEchoPipelineTest.testExampleEchoPipeline
 flaky
https://api.github.com/repos/apache/beam/issues/22008: [Bug]: 
org.apache.beam.sdk.io.gcp.spanner.SpannerIOWriteExceptionHandlingTest.testExceptionHandlingForWriteGrouped
 flaky
https://api.github.com/repos/apache/beam/issues/21999: [Bug]: 
org.apache.beam.sdk.io.gcp.spanner.SpannerIOReadTest.runBatchQueryTestWithFailures
 flaky
https://api.github.com/repos/apache/beam/issues/21978: [Playground] Implement 
Share Any Code feature on the frontend
https://api.github.com/repos/apache/beam/issues/21948: [Bug]: KinesisIO javadoc 
is no longer up-to-date
https://api.github.com/repos/apache/beam/issues/21946: [Bug]: No way to read or 
write to file when running Beam in Flink
https://api.github.com/repos/apache/beam/issues/21935: [Bug]: Reject illformed 
GBK Coders
https://api.github.com/repos/apache/beam/issues/21897: [Feature Request]: Flink 
runner savepoint backward compatibility 
https://api.github.com/repos/apache/beam/issues/21893: [Bug]: BigQuery Storage 
Write API implementation does not support table partitioning
https://api.github.com/repos/apache/beam/issues/21794: Dataflow runner creates 
a new timer whenever the output timestamp is change
https://api.github.com/repos/apache/beam/issues/21763: [Playground Task]: 
Migrate from Google Analytics to Matomo Cloud
https://api.github.com/repos/apache/beam/issues/21715: Data missing when using 
CassandraIO.Read
https://api.github.com/repos/apache/beam/issues/21713: 404s in BigQueryIO don't 
get output to Failed Inserts PCollection
https://api.github.com/repos/apache/beam/issues/21711: Python Streaming job 
failing to drain with BigQueryIO write errors
https://api.github.com/repos/apache/beam/issues/21703: pubsublite.ReadWriteIT 
failing in beam_PostCommit_Java_DataflowV1 and V2
https://api.github.com/repos/apache/beam/issues/21702: SpannerWriteIT failing 
in beam PostCommit Java V1
https://api.github.com/repos/apache/beam/issues/21700: 
--dataflowServiceOptions=use_runner_v2 is broken
https://api.github.com/repos/apache/beam/issues/21695: DataflowPipelineResult 
does not raise exception for unsuccessful states.
https://api.github.com/repos/apache/beam/issues/21694: BigQuery Storage API 
insert with writeResult retry and write to error table
https://api.github.com/repos/apache/beam/issues/21479: Install Python wheel and 
dependencies to local venv in SDK harness
https://api.github.com/repos/apache/beam/issues/21478: 
KafkaIO.read.withDynamicRead() doesn't pick up new TopicPartitions
https://api.github.com/repos/apache/beam/issues/21477: Add integration testing 
for BQ Storage API  write modes
https://api.github.com/repos/apache/beam/issues/21476: WriteToBigQuery Dynamic 
table destinations returns wrong tableId
https://api.github.com/repos/apache/beam/issues/21475: Beam x-lang Dataflow 
tests failing due to _InactiveRpcError
https://api.github.com/repos/apache/beam/issues/21473: PVR_Spark2_Streaming 
perma-red
https://api.github.com/repos/apache/beam/issues/21466: Simplify version 
override for Dev versions of the Go SDK.
https://api.github.com/repos/apache/beam/issues/21465: Kafka commit offset drop 
data on failure for runners that have non-checkpointing shuffle
https://api.github.com/repos/apache/beam/issues/21269: Delete orphaned files
https://api.github.com/repos/apache/beam/issues/21268: Race between member 
variable being accessed due to leaking uninitialized state via 
OutboundObserverFactory
https://api.github.com/repos/apache/beam/issues/21267: WriteToBigQuery submits 
a duplicate BQ load job if a 503 error code is returned from googleapi
https://api.github.com/repos/apache/beam/issues/21265: 
apache_beam.runners.portability.fn_api_runner.translations_test.TranslationsTest.test_run_packable_combine_globally
 'apache_beam.coders.coder_impl._AbstractIterable' object is not reversible
https://api.github.com/repos/apache/beam/issues/21263: (Broken Pipe induced) 
Bricked Dataflow Pipeline 
https://api.github.com/repos/apache/beam/issues/21262: Python AfterAny, 
AfterAll do not follow spec
https://api.github.com/repos/apache/beam/issues/21260: Python DirectRunner does 
not emit data at GC time
https://api.github.com/repos/apache/beam/issues/21259: Consumer group with 
random prefix
https://api.github.com/repos/apache/beam/issues/21258: Dataflow error in 
CombinePerKey operation

Flaky test issue report (56)

2022-06-23 Thread beamactions
This is your daily summary of Beam's current flaky tests.

These are P1 issues because they have a major negative impact on the 
community and make it hard to determine the quality of the software.



https://api.github.com/repos/apache/beam/issues/21714: 
PulsarIOTest.testReadFromSimpleTopic is very flaky
https://api.github.com/repos/apache/beam/issues/21709: 
beam_PostCommit_Java_ValidatesRunner_Samza Failing
https://api.github.com/repos/apache/beam/issues/21708: 
beam_PostCommit_Java_DataflowV2, testBigQueryStorageWrite30MProto failing 
consistently
https://api.github.com/repos/apache/beam/issues/21707: GroupByKeyTest 
BasicTests testLargeKeys100MB flake (on ULR)
https://api.github.com/repos/apache/beam/issues/21706: Flaky timeout in github 
Python unit test action 
StatefulDoFnOnDirectRunnerTest.test_dynamic_timer_clear_then_set_timer
https://api.github.com/repos/apache/beam/issues/21704: 
beam_PostCommit_Java_DataflowV2 failures parent bug
https://api.github.com/repos/apache/beam/issues/21701: 
beam_PostCommit_Java_DataflowV1 failing with a variety of flakes and errors
https://api.github.com/repos/apache/beam/issues/21698: Docker Snapshots failing 
to be published since April 14th
https://api.github.com/repos/apache/beam/issues/21696: Flink Tests failure :  
java.lang.NoClassDefFoundError: Could not initialize class 
org.apache.beam.runners.core.construction.SerializablePipelineOptions 
https://api.github.com/repos/apache/beam/issues/21643: FnRunnerTest with 
non-trivial (order 1000 elements) numpy input flakes in non-cython environment
https://api.github.com/repos/apache/beam/issues/21629: Multiple XVR Suites 
having similar flakes simultaneously
https://api.github.com/repos/apache/beam/issues/21587: 
beam_PreCommit_PythonDocs failing (jinja2)
https://api.github.com/repos/apache/beam/issues/21540: Jenkins worker sometimes 
crashes while running Python Flink pipeline
https://api.github.com/repos/apache/beam/issues/21480: flake: 
FlinkRunnerTest.testEnsureStdoutStdErrIsRestored
https://api.github.com/repos/apache/beam/issues/21474: Flaky tests: Gradle 
build daemon disappeared unexpectedly
https://api.github.com/repos/apache/beam/issues/21472: Dataflow streaming tests 
failing new AfterSynchronizedProcessingTime test
https://api.github.com/repos/apache/beam/issues/21471: Flakes: Failed to load 
cache entry
https://api.github.com/repos/apache/beam/issues/21470: Test flake: 
test_split_half_sdf
https://api.github.com/repos/apache/beam/issues/21469: 
beam_PostCommit_XVR_Flink flaky: Connection refused
https://api.github.com/repos/apache/beam/issues/21468: 
beam_PostCommit_Python_Examples_Dataflow failing
https://api.github.com/repos/apache/beam/issues/21467: GBK and CoGBK streaming 
Java load tests failing
https://api.github.com/repos/apache/beam/issues/21464: GroupIntoBatchesTest is 
failing
https://api.github.com/repos/apache/beam/issues/21463: NPE in Flink Portable 
ValidatesRunner streaming suite
https://api.github.com/repos/apache/beam/issues/21462: Flake in 
org.apache.beam.sdk.io.mqtt.MqttIOTest.testReadObject: Address already in use
https://api.github.com/repos/apache/beam/issues/21333: Flink 
testParDoRequiresStableInput flaky
https://api.github.com/repos/apache/beam/issues/21271: pubsublite.ReadWriteIT 
flaky in beam_PostCommit_Java_DataflowV2  
https://api.github.com/repos/apache/beam/issues/21270: 
org.apache.beam.sdk.transforms.CombineTest$WindowingTests.testWindowedCombineGloballyAsSingletonView
 flaky on Dataflow Runner V2
https://api.github.com/repos/apache/beam/issues/21266: 
org.apache.beam.sdk.transforms.ParDoLifecycleTest.testTeardownCalledAfterExceptionInProcessElementStateful
 is flaky in Java ValidatesRunner Flink suite.
https://api.github.com/repos/apache/beam/issues/21264: beam_PostCommit_Python36 
- CrossLanguageSpannerIOTest - flakey failing
https://api.github.com/repos/apache/beam/issues/21261: 
org.apache.beam.runners.dataflow.worker.fn.logging.BeamFnLoggingServiceTest.testMultipleClientsFailingIsHandledGracefullyByServer
 is flaky
https://api.github.com/repos/apache/beam/issues/21242: 
org.apache.beam.sdk.transforms.ParDoLifecycleTest.testTeardownCalledAfterExceptionInStartBundle
 is flaky in Java Spark ValidatesRunner suite 
https://api.github.com/repos/apache/beam/issues/21121: 
apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT.test_streaming_wordcount_it
 flakey
https://api.github.com/repos/apache/beam/issues/21120: 
beam_PostRelease_NightlySnapshot failed
https://api.github.com/repos/apache/beam/issues/21118: 
PortableRunnerTestWithExternalEnv.test_pardo_timers flaky
https://api.github.com/repos/apache/beam/issues/21116: Python PreCommit flaking 
in PipelineOptionsTest.test_display_data
https://api.github.com/repos/apache/beam/issues/21114: Already Exists: Dataset 
apache-beam-testing:python_bq_file_loads_NNN
https://api.github.com/repos/apache/beam/issues/21113: 
testTwoTimersSettingEachOtherWithCreateAsInputBounded flaky

P0 issues report (1)

2022-06-23 Thread beamactions
This is your daily summary of Beam's current P0 issues, not including flaky 
tests.

See https://beam.apache.org/contribute/issue-priorities/#p0-outage for the 
meaning and expectations around P0 issues.



https://api.github.com/repos/apache/beam/issues/21998: [Bug]: Loop last element 
when write to file using textio in Go SDK


Beam Dependency Check Report (2022-06-23)

2022-06-23 Thread Apache Jenkins Server

High Priority Dependency Updates Of Beam Python SDK:


  Dependency Name
  Current Version
  Latest Version
  Release Date Of the Current Used Version
  Release Date Of The Latest Release
  
cachetools
4.2.4
5.2.0
2021-12-27
2022-06-02
chromedriver-binary
100.0.4896.60.0
103.0.5060.53.0
2022-05-05
2022-06-23
dill
0.3.1.1
0.3.5.1
2019-10-07
2022-05-26
google-api-core
1.31.6
2.8.2
2022-06-02
2022-06-16
google-auth
1.35.0
2.8.0
2021-08-23
2022-06-16
google-cloud-bigquery
2.34.4
3.2.0
2022-06-09
2022-06-09
google-cloud-bigtable
1.7.2
2.10.1
2022-06-09
2022-06-16
google-cloud-core
1.7.2
2.3.1
2021-08-23
2022-06-09
google-cloud-dataproc
3.1.1
4.0.3
2022-02-21
2022-06-09
google-cloud-datastore
1.15.5
2.7.1
2022-06-16
2022-06-23
google-cloud-language
1.3.2
2.4.3
2022-06-16
2022-06-09
google-cloud-recommendations-ai
0.2.0
0.6.2
2021-07-05
2022-06-09
google-cloud-spanner
1.19.3
3.15.1
2022-06-16
2022-06-23
google-cloud-videointelligence
1.16.3
2.7.1
2022-06-16
2022-06-09
google-cloud-vision
1.0.2
2.7.3
2022-06-16
2022-06-09
grpcio-tools
1.37.0
1.47.0
2021-04-12
2022-06-23
jupyter-client
6.1.12
7.3.4
2021-04-12
2022-06-09
mistune
0.8.4
2.0.2
2021-12-06
2022-01-17
mock
2.0.0
4.0.3
2022-06-02
2020-12-14
mypy-protobuf
1.18
3.2.0
2020-03-24
2022-01-24
Pillow
7.2.0
9.1.1
2020-10-19
2022-05-19
protobuf
3.20.1
4.21.1
2022-06-02
2022-06-02
pyarrow
7.0.0
8.0.0
2022-02-07
2022-05-12
PyHamcrest
1.10.1
2.0.3
2020-01-20
2021-12-13
pymongo
3.12.3
4.1.1
2021-12-13
2022-04-14
selenium
3.141.0
4.2.0
2021-11-18
2022-06-02
setuptools
62.3.4
62.6.0
None
2022-06-23
tenacity
5.1.5
8.0.1
2019-11-11
2021-07-19
High Priority Dependency Updates Of Beam Java SDK:


  Dependency Name
  Current Version
  Latest Version
  Release Date Of the Current Used Version
  Release Date Of The Latest Release
  
biz.aQute:bndlib
1.50.0
2.0.0.20130123-133441
2011-11-04
2013-02-27
com.alibaba:fastjson
1.2.69
2.0.7
2020-05-31
2022-06-11
com.amazonaws:amazon-kinesis-client
1.14.2
1.14.8
2021-02-24
2022-02-24
com.amazonaws:amazon-kinesis-producer
0.14.1
0.14.12
2020-07-31
2022-03-16
com.azure:azure-core
1.9.0
1.29.1
2020-10-02
2022-06-04
com.azure:azure-identity
1.0.8
1.5.2
2020-07-07
2022-06-06
com.azure:azure-storage-blob
12.10.0
12.18.0-beta.1
2021-01-15
2022-06-15
com.azure:azure-storage-common
12.10.0
12.17.0-beta.1
2021-01-14
2022-06-15
com.carrotsearch.randomizedtesting:randomizedtesting-runner
2.7.8
2.7.9
2020-07-07
2021-10-22
com.clearspring.analytics:stream
2.9.5
2.9.8
2016-08-01
2019-08-27
com.datastax.cassandra:cassandra-driver-core
3.10.2
4.0.0
2020-08-26
2019-03-18
com.datastax.cassandra:cassandra-driver-mapping
3.10.2
3.11.2
2020-08-26
2022-04-28
com.esotericsoftware:kryo
4.0.2
5.3.0
2018-03-20
2022-02-11
com.esotericsoftware.kryo:kryo
2.21
2.24.0
2013-02-27
2014-05-04
com.github.ben-manes.versions:com.github.ben-manes.versions.gradle.plugin
0.33.0
0.42.0
2020-09-14
2022-02-07
com.github.jbellis:jamm
0.3.0
0.3.3
2014-11-19
2018-11-16
com.github.jk1.dependency-license-report:com.github.jk1.dependency-license-report.gradle.plugin
1.16
2.1
2020-10-26
2022-01-24
com.github.spotbugs:spotbugs
4.0.6
4.7.0