Re: Welcome Aisulu for this Summer's GSoC!

[Ais K]

Hello everyone!

My name is Aisulu from Kyrgyzstan, currently working in Seoul(South Korea).

I am so grateful for giving me such a great opportunity to be able to
contribute.
This is going to be my first open-source project, so I'm ready to do my
best working on BEAM-2888[1],
and other tickets.

Looking forward to working with my mentor and other team members.

Best Regards,
Aisulu Karimbaeva ㅣ 이수아
Email: alive.k...@gmail.com


On Wed, Jun 1, 2022 at 1:45 AM Pablo Estrada  wrote:

> Hello dev@!
> I'd like us to welcome Aisulu. She will be working on BEAM-2888[1], which
> is a project to automate our capability matrix to source capability results
> from our ValidatesRunner tests.
>
> We have a CI/CD job implemented that generates a file with this
> information (the job is deactivated at the moment because it sort of
> DDoS-es the ci-beam server, so we also need to figure that out : )). Aisulu
> will work on taking this job and generating a UI based on its output.
>
> If time permits, I may also work with Aisulu on fixes / examples for the
> Typescript SDK.
>
> Please welcome Aisulu and help her if she asks you any questions : )
> Best
> -P.
>
> [1] https://issues.apache.org/jira/browse/BEAM-2888
>

Re: [CDAP IO] Failing R JavaPrecommit checks

[Alexey Romanenko]

I guess it’s related to added a hard version of scala-library dependency. Try 
to make it “compileOnly” (as we do for Spark runner for example). Also, there 
is no need to escalate it to BeamModulePlugin.groovy - please, keep it local 
for your module.

—
Alexey


> On 31 May 2022, at 23:17, Elizaveta Lomteva  
> wrote:
> 
> Hi everyone,
> 
> In Add ReceiverBuilder for SparkReceiverIO PR [1] for the CDAP IO connector 
> project, the Java Precommit checks failed with errors unrelated to changes in 
> the PR:
> 
> 18:13:23 1: Task failed with an exception.
> 18:13:23 ---
> 18:13:23 * What went wrong:
> 18:13:23 Execution failed for task ':sdks:java:io:amazon-web-services:test'.
> 
> 18:13:23 2: Task failed with an exception.
> 18:13:23 ---
> 18:13:23 * What went wrong:
> 18:13:23 Execution failed for task ':sdks:java:io:amazon-web-services2:test'.
> 
> 18:13:23 3: Task failed with an exception.
> 18:13:23 ---
> 18:13:23 * What went wrong:
> 18:13:23 Execution failed for task ':runners:spark:3:test'.
> 
> 18:13:23 4: Task failed with an exception.
> 18:13:23 ---
> 18:13:23 * What went wrong:
> 18:13:23 Execution failed for task ':runners:flink:1.12:test'.
> 
> 18:13:23 5: Task failed with an exception.
> 18:13:23 ---
> 18:13:23 * What went wrong:
> 18:13:23 Execution failed for task ':runners:flink:1.13:test'.
> 
> 
> Does someone know why it's so and how to fix it?
> 
> Regards,
> Elizaveta
> 
> [1] [BEAM-14101] [CdapIO] Add ReceiverBuilder for SparkReceiverIO PR 
> 

[CDAP IO] Failing R JavaPrecommit checks

[Elizaveta Lomteva]

Hi everyone,


In Add ReceiverBuilder for SparkReceiverIO PR [1] for the CDAP IO connector 
project, the Java Precommit checks failed with errors unrelated to changes in 
the PR:


18:13:23 1: Task failed with an exception.
18:13:23 ---
18:13:23 * What went wrong:
18:13:23 Execution failed for task ':sdks:java:io:amazon-web-services:test'.


18:13:23 2: Task failed with an exception.
18:13:23 ---
18:13:23 * What went wrong:
18:13:23 Execution failed for task ':sdks:java:io:amazon-web-services2:test'.


18:13:23 3: Task failed with an exception.
18:13:23 ---
18:13:23 * What went wrong:
18:13:23 Execution failed for task ':runners:spark:3:test'.


18:13:23 4: Task failed with an exception.
18:13:23 ---
18:13:23 * What went wrong:
18:13:23 Execution failed for task ':runners:flink:1.12:test'.


18:13:23 5: Task failed with an exception.
18:13:23 ---
18:13:23 * What went wrong:
18:13:23 Execution failed for task ':runners:flink:1.13:test'.


Does someone know why it's so and how to fix it?


Regards,
Elizaveta


[1] [BEAM-14101] [CdapIO] Add ReceiverBuilder for SparkReceiverIO 
PR

[GitHub] [beam-site] y1chi opened a new pull request, #631: Update beam-site for release 2.39.0.

[GitBox]

y1chi opened a new pull request, #631:
URL: https://github.com/apache/beam-site/pull/631

   Content generated from commit 23bb12f665148b0203b2d0b4acbab26f0b1e341c.
   
   Apache Beam website sources have been moved to the
   apache/beam repository at: https://github.com/apache/beam/blob/master/website
   Please open pull requests for website contributions there.
   
   This repository hosts generated HTML release documentation (Javadocs, pydocs)
   on the release-docs branch: 
https://github.com/apache/beam-site/tree/release-docs
   branch. Please see the Apache Beam Release guide for details on how to 
publish
   documentation for a new release: 
https://beam.apache.org/contribute/release-guide
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@beam.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [beam-site] y1chi closed pull request #630: Publish 2.39.0 release

[GitBox]

y1chi closed pull request #630: Publish 2.39.0 release
URL: https://github.com/apache/beam-site/pull/630


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@beam.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

P1 issues report (78)

[Beam Jira Bot]

This is your daily summary of Beam's current P1 issues, not including flaky 
tests 
(https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20statusCategory%20!%3D%20Done%20AND%20priority%20%3D%20P1%20AND%20(labels%20is%20EMPTY%20OR%20labels%20!%3D%20flake).

See https://beam.apache.org/contribute/jira-priorities/#p1-critical for the 
meaning and expectations around P1 issues.

https://issues.apache.org/jira/browse/BEAM-14483: Add Java cross-language 
transforms for invoking Python Map and FlatMap (created 2022-05-17)
https://issues.apache.org/jira/browse/BEAM-14481: ScopedState can not be 
re-entered (regression caused by Batched DoFn worker changes) (created 
2022-05-17)
https://issues.apache.org/jira/browse/BEAM-14459: Docker Snapshots failing 
to be published since April 14th (created 2022-05-11)
https://issues.apache.org/jira/browse/BEAM-14434: 
beam_LoadTests_Python_GBK_reiterate_Dataflow_Streaming failure (created 
2022-05-06)
https://issues.apache.org/jira/browse/BEAM-14431: Handle nulls using 
SnowflakeIO (created 2022-05-06)
https://issues.apache.org/jira/browse/BEAM-14421: 
--dataflowServiceOptions=use_runner_v2 is broken (created 2022-05-05)
https://issues.apache.org/jira/browse/BEAM-14411: TypeCodersTest is never 
executed (created 2022-05-04)
https://issues.apache.org/jira/browse/BEAM-14390: Java license check is 
broken (created 2022-05-02)
https://issues.apache.org/jira/browse/BEAM-14364: 404s in BigQueryIO don't 
get output to Failed Inserts PCollection (created 2022-04-25)
https://issues.apache.org/jira/browse/BEAM-14291: DataflowPipelineResult 
does not raise exception for unsuccessful states. (created 2022-04-11)
https://issues.apache.org/jira/browse/BEAM-14276: 
beam_PostCommit_Java_DataflowV2 failures parent bug (created 2022-04-07)
https://issues.apache.org/jira/browse/BEAM-14275: SpannerWriteIT failing in 
beam PostCommit Java V1 (created 2022-04-07)
https://issues.apache.org/jira/browse/BEAM-14265: Flink should hold the 
watermark at the output timestamp for processing time timers (created 
2022-04-06)
https://issues.apache.org/jira/browse/BEAM-14263: 
beam_PostCommit_Java_DataflowV2, testBigQueryStorageWrite30MProto failing 
consistently (created 2022-04-05)
https://issues.apache.org/jira/browse/BEAM-14253: pubsublite.ReadWriteIT 
failing in beam_PostCommit_Java_DataflowV1 and V2 (created 2022-04-05)
https://issues.apache.org/jira/browse/BEAM-14239: Changing the output 
timestamp of a timer does not clear the previously set timer (created 
2022-04-04)
https://issues.apache.org/jira/browse/BEAM-14174: Flink Tests failure :  
java.lang.NoClassDefFoundError: Could not initialize class 
org.apache.beam.runners.core.construction.SerializablePipelineOptions  (created 
2022-03-24)
https://issues.apache.org/jira/browse/BEAM-14135: BigQuery Storage API 
insert with writeResult retry and write to error table (created 2022-03-20)
https://issues.apache.org/jira/browse/BEAM-13952: Dataflow streaming tests 
failing new AfterSynchronizedProcessingTime test (created 2022-02-15)
https://issues.apache.org/jira/browse/BEAM-13950: PVR_Spark2_Streaming 
perma-red (created 2022-02-15)
https://issues.apache.org/jira/browse/BEAM-13920: Beam x-lang Dataflow 
tests failing due to _InactiveRpcError (created 2022-02-10)
https://issues.apache.org/jira/browse/BEAM-13852: 
KafkaIO.read.withDynamicRead() doesn't pick up new TopicPartitions (created 
2022-02-08)
https://issues.apache.org/jira/browse/BEAM-13850: 
beam_PostCommit_Python_Examples_Dataflow failing (created 2022-02-08)
https://issues.apache.org/jira/browse/BEAM-13822: GBK and CoGBK streaming 
Java load tests failing (created 2022-02-03)
https://issues.apache.org/jira/browse/BEAM-13805: Simplify version override 
for Dev versions of the Go SDK. (created 2022-02-02)
https://issues.apache.org/jira/browse/BEAM-13747: Add integration testing 
for BQ Storage API  write modes (created 2022-01-26)
https://issues.apache.org/jira/browse/BEAM-13715: Kafka commit offset drop 
data on failure for runners that have non-checkpointing shuffle (created 
2022-01-21)
https://issues.apache.org/jira/browse/BEAM-13669: Install Python wheel and 
dependencies to local venv in SDK harness (created 2022-01-17)
https://issues.apache.org/jira/browse/BEAM-13487: WriteToBigQuery Dynamic 
table destinations returns wrong tableId (created 2021-12-17)
https://issues.apache.org/jira/browse/BEAM-13393: GroupIntoBatchesTest is 
failing (created 2021-12-07)
https://issues.apache.org/jira/browse/BEAM-13164: Race between member 
variable being accessed due to leaking uninitialized state via 
OutboundObserverFactory (created 2021-11-01)
https://issues.apache.org/jira/browse/BEAM-13132: WriteToBigQuery submits a 
duplicate BQ load job if a 503 error code is returned from googleapi (created 
2021-10-27)

Flaky test issue report (58)

[Beam Jira Bot]

This is your daily summary of Beam's current flaky tests 
(https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20statusCategory%20!%3D%20Done%20AND%20labels%20%3D%20flake)

These are P1 issues because they have a major negative impact on the community 
and make it hard to determine the quality of the software.

https://issues.apache.org/jira/browse/BEAM-14459: Docker Snapshots failing 
to be published since April 14th (created 2022-05-11)
https://issues.apache.org/jira/browse/BEAM-14410: FnRunnerTest with 
non-trivial (order 1000 elements) numpy input flakes in non-cython environment 
(created 2022-05-04)
https://issues.apache.org/jira/browse/BEAM-14407: Jenkins worker sometimes 
crashes while running Python Flink pipeline (created 2022-05-04)
https://issues.apache.org/jira/browse/BEAM-14367: Flaky timeout in github 
Python unit test action 
StatefulDoFnOnDirectRunnerTest.test_dynamic_timer_clear_then_set_timer (created 
2022-04-26)
https://issues.apache.org/jira/browse/BEAM-14349: GroupByKeyTest BasicTests 
testLargeKeys100MB flake (on ULR) (created 2022-04-21)
https://issues.apache.org/jira/browse/BEAM-14276: 
beam_PostCommit_Java_DataflowV2 failures parent bug (created 2022-04-07)
https://issues.apache.org/jira/browse/BEAM-14269: 
PulsarIOTest.testReadFromSimpleTopic is very flaky (created 2022-04-06)
https://issues.apache.org/jira/browse/BEAM-14263: 
beam_PostCommit_Java_DataflowV2, testBigQueryStorageWrite30MProto failing 
consistently (created 2022-04-05)
https://issues.apache.org/jira/browse/BEAM-14252: 
beam_PostCommit_Java_DataflowV1 failing with a variety of flakes and errors 
(created 2022-04-05)
https://issues.apache.org/jira/browse/BEAM-14216: Multiple XVR Suites 
having similar flakes simultaneously (created 2022-03-31)
https://issues.apache.org/jira/browse/BEAM-14174: Flink Tests failure :  
java.lang.NoClassDefFoundError: Could not initialize class 
org.apache.beam.runners.core.construction.SerializablePipelineOptions  (created 
2022-03-24)
https://issues.apache.org/jira/browse/BEAM-14172: beam_PreCommit_PythonDocs 
failing (jinja2) (created 2022-03-24)
https://issues.apache.org/jira/browse/BEAM-13952: Dataflow streaming tests 
failing new AfterSynchronizedProcessingTime test (created 2022-02-15)
https://issues.apache.org/jira/browse/BEAM-13859: Test flake: 
test_split_half_sdf (created 2022-02-09)
https://issues.apache.org/jira/browse/BEAM-13850: 
beam_PostCommit_Python_Examples_Dataflow failing (created 2022-02-08)
https://issues.apache.org/jira/browse/BEAM-13822: GBK and CoGBK streaming 
Java load tests failing (created 2022-02-03)
https://issues.apache.org/jira/browse/BEAM-13810: Flaky tests: Gradle build 
daemon disappeared unexpectedly (created 2022-02-03)
https://issues.apache.org/jira/browse/BEAM-13809: beam_PostCommit_XVR_Flink 
flaky: Connection refused (created 2022-02-03)
https://issues.apache.org/jira/browse/BEAM-13797: Flakes: Failed to load 
cache entry (created 2022-02-01)
https://issues.apache.org/jira/browse/BEAM-13708: flake: 
FlinkRunnerTest.testEnsureStdoutStdErrIsRestored (created 2022-01-20)
https://issues.apache.org/jira/browse/BEAM-13575: Flink 
testParDoRequiresStableInput flaky (created 2021-12-28)
https://issues.apache.org/jira/browse/BEAM-13500: NPE in Flink Portable 
ValidatesRunner streaming suite (created 2021-12-21)
https://issues.apache.org/jira/browse/BEAM-13453: Flake in 
org.apache.beam.sdk.io.mqtt.MqttIOTest.testReadObject: Address already in use 
(created 2021-12-13)
https://issues.apache.org/jira/browse/BEAM-13393: GroupIntoBatchesTest is 
failing (created 2021-12-07)
https://issues.apache.org/jira/browse/BEAM-13367: 
[beam_PostCommit_Python36] [ 
apache_beam.io.gcp.experimental.spannerio_read_it_test] Failure summary 
(created 2021-12-01)
https://issues.apache.org/jira/browse/BEAM-13312: 
org.apache.beam.sdk.transforms.ParDoLifecycleTest.testTeardownCalledAfterExceptionInStartBundle
 is flaky in Java Spark ValidatesRunner suite  (created 2021-11-23)
https://issues.apache.org/jira/browse/BEAM-13311: 
org.apache.beam.sdk.transforms.ParDoLifecycleTest.testTeardownCalledAfterExceptionInProcessElementStateful
 is flaky in Java ValidatesRunner Flink suite. (created 2021-11-23)
https://issues.apache.org/jira/browse/BEAM-13237: 
org.apache.beam.sdk.transforms.CombineTest$WindowingTests.testWindowedCombineGloballyAsSingletonView
 flaky on Dataflow Runner V2 (created 2021-11-12)
https://issues.apache.org/jira/browse/BEAM-13025: pubsublite.ReadWriteIT 
flaky in beam_PostCommit_Java_DataflowV2   (created 2021-10-08)
https://issues.apache.org/jira/browse/BEAM-12928: beam_PostCommit_Python36 
- CrossLanguageSpannerIOTest - flakey failing (created 2021-09-21)
https://issues.apache.org/jira/browse/BEAM-12859: 

Re: [ANNOUNCE] New committer: Ke Wu

[Alexey Romanenko]

Congrats, Ke!

—
Alexey

> On 31 May 2022, at 18:09, Xinyu Liu  wrote:
> 
> Congrats!
> 
> Xinyu
> 
> On Mon, May 30, 2022 at 7:46 AM Evan Galpin  > wrote:
> Congrats Ke!
> 
> - Evan
> 
> 
> On Mon, May 30, 2022 at 4:11 AM Jan Lukavský  > wrote:
> Congrats Ke!
> 
>  Jan
> 
> On 5/29/22 04:12, Yi Pan wrote:
>> Congrats, Ke!
>> 
>> -Yi
>> 
>> On Sat, May 28, 2022 at 6:57 PM Robert Burke > > wrote:
>> Congratulations!
>> Another place that runs the Go SDK ;)
>> 
>> On Fri, May 27, 2022, 3:49 PM Ahmet Altay > > wrote:
>> Hi all,
>> 
>> Please join me and the rest of the Beam PMC in welcoming a new committer: Ke 
>> Wu (kw2542@)
>> 
>> Ke has been contributing to Beam since 2020. Ke's contributions are mostly 
>> focused on the SamzaRunner, as a result of Ke's efforts Beam has a fully 
>> featured, portable, supported SamzaRunner with happy users!
>> 
>> Considering these contributions, the Beam PMC trusts Ke with the 
>> responsibilities of a Beam committer.[1]
>> 
>> Thank you Ke!
>> 
>> Ahmet
>> 
>> [1] 
>> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>>  
>> 

Re: [ANNOUNCE] New committer: Ke Wu

[Xinyu Liu]

Congrats!

Xinyu

On Mon, May 30, 2022 at 7:46 AM Evan Galpin  wrote:

> Congrats Ke!
>
> - Evan
>
>
> On Mon, May 30, 2022 at 4:11 AM Jan Lukavský  wrote:
>
>> Congrats Ke!
>>
>>  Jan
>> On 5/29/22 04:12, Yi Pan wrote:
>>
>> Congrats, Ke!
>>
>> -Yi
>>
>> On Sat, May 28, 2022 at 6:57 PM Robert Burke  wrote:
>>
>>> Congratulations!
>>> Another place that runs the Go SDK ;)
>>>
>>> On Fri, May 27, 2022, 3:49 PM Ahmet Altay  wrote:
>>>
 Hi all,

 Please join me and the rest of the Beam PMC in welcoming a new
 committer: Ke Wu (kw2542@)

 Ke has been contributing to Beam since 2020. Ke's contributions are
 mostly focused on the SamzaRunner, as a result of Ke's efforts Beam has a
 fully featured, portable, supported SamzaRunner with happy users!

 Considering these contributions, the Beam PMC trusts Ke with the
 responsibilities of a Beam committer.[1]

 Thank you Ke!

 Ahmet

 [1]
 https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer

>>>

Re: Failing beam_SeedJob

[Alexey Romanenko]

The first failed job is https://ci-beam.apache.org/job/beam_SeedJob/9696/ 

It fails with this error (that says not so much):

Processing DSL script .test-infra/jenkins/job_00_seed.groovy
Processing DSL script .test-infra/jenkins/job_CancelStaleDataflowJobs.groovy
Processing DSL script .test-infra/jenkins/job_CleanUpPrebuiltSDKImages.groovy
Processing DSL script .test-infra/jenkins/job_Dependency_Check.groovy
ERROR: java.io.IOException: Failed to persist config.xml

 and I don’t see any recent changes for these files.

> On 31 May 2022, at 16:24, Alexey Romanenko  wrote:
> 
> Hi everyone,
> 
> Jenkins job `beam_SeedJob` keeps failing starting from May 19th. The last 
> successful build [2] was 12 days ago.
> Does anyone know the reasons of this?
> 
> —
> Alexey
> 
> 
> [1] https://ci-beam.apache.org/job/beam_SeedJob/ 
> 
> [2] https://ci-beam.apache.org/job/beam_SeedJob/lastSuccessfulBuild/ 
> 
>   
> 

Failing beam_SeedJob

[Alexey Romanenko]

Hi everyone,

Jenkins job `beam_SeedJob` keeps failing starting from May 19th. The last 
successful build [2] was 12 days ago.
Does anyone know the reasons of this?

—
Alexey


[1] https://ci-beam.apache.org/job/beam_SeedJob/ 

[2] https://ci-beam.apache.org/job/beam_SeedJob/lastSuccessfulBuild/