Re: [VOTE] Release 2.56.0, release candidate #2

+1 (non-binding) Validated Dataflow Template [1] (except Yaml and Python xlang templates, which will be validated separately), and GCPIO load tests on Dataflow runner [2] [1] https://github.com/GoogleCloudPlatform/DataflowTemplates/pull/1487 [2]

Re: Patch release proposal

>>> I can take that on if needed. >>> >>> > Besides, there should be a Beam YAML validation workflow and added in >>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1368030253 >>> >>> &g

Re: Patch release proposal

her known > issue for which fix is available: > https://github.com/apache/beam/blob/master/CHANGES.md#known-issues-1 > > On Thu, Mar 28, 2024 at 8:01 AM Yi Hu via dev wrote: > >> 2.55.0 release manager here >> >> The patch itself [1] is trivial, however, t

Re: Patch release proposal

2.55.0 release manager here The patch itself [1] is trivial, however, the release process is not trivial. There is little documentation nor practice for a patch release process. I could imagine two options 1. Do a full "2.55.1" release 2. Do a patch release only for Python SDK, that is a.

[Announce] Beam 2.55.0 Release

We are happy to present the new 2.55.0 release of Beam. This release includes both improvements and new functionality. See https://beam.apache.org/get-started/downloads/ for this release. For more information on changes in 2.55.0, check out the detailed release notes at

Re: [VOTE] Release 2.55.0, release candidate #3

; >> > * GitHub Release notes [1] >> >> Is the link correct? It points to the milestone. >> >> On Fri, Mar 22, 2024 at 1:10 PM Yi Hu via dev >> wrote: >> >>> +1 (non-binding) >>> >>> 1. Checked published Java artifacts &

Re: [VOTE] Release 2.55.0, release candidate #3

points to the milestone. > > On Fri, Mar 22, 2024 at 1:10 PM Yi Hu via dev wrote: > >> +1 (non-binding) >> >> 1. Checked published Java artifacts >> >> 2. Tested with GCP IO performance benchmark >> >> 3. Tested with Java PostRelease workflow (inc

Re: [VOTE] Release 2.55.0, release candidate #3

>> wrote: >> >>> +1 (non-binding). Ran the simple ML pipeline without any issue: >>> https://github.com/google/dataflow-ml-starter/actions/runs/8349158153 >>> >>> On Tue, Mar 19, 2024 at 11:55 AM Ritesh Ghorse via dev < >>> dev@beam.apache

[VOTE] Release 2.55.0, release candidate #3

Hi everyone, Please review and vote on the release candidate #3 for the version 2.55.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) Reviewers are encouraged to test their own use cases with the release candidate, and vote +1 if

Re: [VOTE] Release 2.55.0, release candidate #2

Abualsaud > wrote: > >> >> >> ------ Forwarded message - >> From: Yi Hu via dev >> Date: Fri, Mar 15, 2024 at 10:54 AM >> Subject: Re: [VOTE] Release 2.55.0, release candidate #2 >> To: dev >> CC: Yi Hu >> >> >> +1 (non-binding).

Re: Update confluent dependencies version in kafka io

+1 to move forward with avro. Users can still stick with old avro version if chosen to do so, and we have Avro compatibility tests also passed in https://github.com/apache/beam/pull/30638 Thanks for doing this! On Fri, Mar 15, 2024 at 3:30 PM Maciej Szwaja via dev wrote: > Hi team, thanks for

Re: [VOTE] Release 2.55.0, release candidate #2

istory > > On Fri, Mar 15, 2024 at 10:54 AM Yi Hu via dev > wrote: > >> +1 (non-binding). >> >> Tested with https://github.com/GoogleCloudPlatform/DataflowTemplates >> (Java SDK 11, Dataflow runner, details: >> https://github.com/GoogleCloudPlatform/Dataflow

Re: [VOTE] Release 2.55.0, release candidate #2

+1 (non-binding). Tested with https://github.com/GoogleCloudPlatform/DataflowTemplates (Java SDK 11, Dataflow runner, details: https://github.com/GoogleCloudPlatform/DataflowTemplates/pull/1366). On Fri, Mar 15, 2024 at 10:37 AM Yi Hu wrote: > Hi everyone, > Please review and vote on the

[VOTE] Release 2.55.0, release candidate #2

Hi everyone, Please review and vote on the release candidate #1 for the version 2.55.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) Reviewers are encouraged to test their own use cases with the release candidate, and vote +1 if no

Re: [VOTE] Release 2.55.0, release candidate #1

examples with Python 3.9. > > Thanks! > > On Tue, Mar 12, 2024 at 10:21 AM Yi Hu via dev > wrote: > >> Hi everyone, >> Please review and vote on the release candidate #1 for the version >> 2.55.0, as follows: >> [ ] +1, Approve the release >> [ ] -1, Do

[VOTE] Release 2.55.0, release candidate #1

Hi everyone, Please review and vote on the release candidate #1 for the version 2.55.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) Reviewers are encouraged to test their own use cases with the release candidate, and vote +1 if no

[Release 2.55.0] Release Branch has been Cut

Hi Beam Developers! The 2.55.0 release branch has been cut [1]. There are 2 outstanding issues to be triaged for 2.55.0 in the release milestones [2] - https://github.com/apache/beam/issues/30459 - https://github.com/apache/beam/issues/30534 Both issues are triaged and waiting for PR and

[PROPOSAL] Preparing for 2.55.0 Release

Hey Beam community, The next release (2.55.0) branch cut is scheduled on Mar 6th, 2024, according to the release calendar [1]. I volunteer to perform this release. My plan is to cut the branch on that date, and cherrypick release-blocking fixes afterwards, if any. Please help me make sure the

Re: [VOTE] Vendored Dependencies Release

gt; On Mon, Feb 12, 2024 at 8:32 PM Yi Hu via dev wrote: > >> Hi, >> >> I am trying to open " >> https://repository.apache.org/content/repositories/orgapachebeam-1369/; >> but get "[id=orgapachebeam-1369] exists but is not exposed." It seems th

Re: [ANNOUNCE] New Committer: Svetak Sundhar

Congrats, Svetak! On Wed, Feb 14, 2024 at 9:50 AM John Casey via dev wrote: > Congrats Svetak! > > On Wed, Feb 14, 2024 at 9:00 AM Ahmed Abualsaud > wrote: > >> Congrats Svetak! >> >> On 2024/02/14 02:05:02 Priyans Desai via dev wrote: >> > Congratulations Svetak!! >> > >> > On Tue, Feb 13,

Re: [VOTE] Vendored Dependencies Release

Hi, I am trying to open " https://repository.apache.org/content/repositories/orgapachebeam-1369/; but get "[id=orgapachebeam-1369] exists but is not exposed." It seems the staging repository needs to be closed to have it available to public: [1] [1]

Re: [VOTE] Release 2.54.0, release candidate #2

> from 2.53.0 to 2.54.0 using the Transform Service. >>> >>> Thanks, >>> Cham >>> >>> On Wed, Feb 7, 2024 at 5:52 PM XQ Hu via dev >>> wrote: >>> >>>> +1 (non-binding) >>>> >>>> Validated with

Re: [VOTE] Release 2.54.0, release candidate #2

+1 (non-binding) Validated with Dataflow Template: https://github.com/GoogleCloudPlatform/DataflowTemplates/pull/1317 Regards, On Wed, Feb 7, 2024 at 11:18 AM Ritesh Ghorse via dev wrote: > +1 (non-binding) > > Ran a few batch and streaming examples for Python SDK on Dataflow Runner > >

Re: [VOTE] Release 2.54.0, release candidate #1

Thanks for taking care of the release process! After validation two breaking change was found (1) Python Xlang Gcp Direct and Python Xlang Gcp Dataflow PostCommit tests [1, 2]. It affects Python xlang BigQueryIO write (STORAGE_WRITE_API mode) configuration. Filed [3] and pull request for cherry

[Result][VOTE] Vendored Dependencies Release beam-vendor-grpc-1_60_1

Hi everyone, I'm happy to announce that we have unanimously approved this release. There are three approving votes, three of which are binding: * Kenneth Knowles * Robert Bradshaw * Chamikara Jayalath There are no disapproving votes. I will begin finalizing the release. Thanks everyone! --

Re: [VOTE] Vendored Dependencies Release

56 PM Robert Bradshaw via dev < >>> dev@beam.apache.org> wrote: >>> >>>> Could you explain the process you used to produce these artifacts? >>>> >>>> On Thu, Jan 18, 2024 at 11:23 AM Kenneth Knowles >>>> wrote: >>>&g

Re: [VOTE] Vendored Dependencies Release

; On Wed, Jan 17, 2024 at 6:03 PM Yi Hu via dev >> wrote: >> >>> Hi everyone, >>> >>> >>> Please review the release of the following artifacts that we vendor: >>> >>> * beam-vendor-grpc-1_60_1 >>> >>> >>>

[VOTE] Vendored Dependencies Release

Hi everyone, Please review the release of the following artifacts that we vendor: * beam-vendor-grpc-1_60_1 Please review and vote on the release candidate #1 for the version 0.1, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments)

Re: [PROPOSAL] Upgrade vendor grpc

t 11:28 AM Kenneth Knowles wrote: > Yes, thank you! > > On Thu, Jan 11, 2024 at 8:21 PM Chamikara Jayalath via dev < > dev@beam.apache.org> wrote: > >> Sounds good and thanks for doing this :) >> >> - Cham >> >> On Thu, Jan 11, 2024 at 8:06 AM Yi H

[PROPOSAL] Upgrade vendor grpc

Hi everyone, I would like to volunteer to upgrade the Beam vendored grpc, as requested by the GitHub Issue [1]. The last update was in Apr 2023 [2]. There have been vulnerabilities in its dependencies as well as potential oom issues found since then (see [1]), and also to include grpc-alts [2].

Re: [VOTE] Release 2.53.0, release candidate #2

+1 (non-binding) Tested with Beam GCP IOs benchmarking ( https://github.com/GoogleCloudPlatform/DataflowTemplates/tree/main/it/google-cloud-platform ) On Thu, Dec 28, 2023 at 11:36 AM Svetak Sundhar via dev wrote: > +1 (non binding) > > Tested with Healthcare notebooks. > > > Svetak Sundhar >

Re: Disabling Jenkins Jobs

Hi all, Just another update that we have shutdown all precommit tests on Jenkins. For now, the remaining tests running on Jenkins are PostCommit suites. They are also exercised on GitHub Action, but cannot be triggered from a pull request due to [1]. That said, once the blocker [1] is resolved,

Re: [VOTE] Release 2.52.0, release candidate #2

+1 (non-binding) Tested on Java IO load tests ( https://github.com/bvolpato/DataflowTemplates/tree/56d18a31c1c95e58543d7a1656bd83d7e859b482/it) BigQueryIO, TextIO, BigtableIO, SpannerIO on Dataflow legacy runner and runner v2 While it was announced there will be an RC3, the RC2 validation for IO

Re: Disabling Jenkins Jobs

t;> Is it known problem? >> >> — >> Alexey >> >> On 24 Oct 2023, at 21:50, Yi Hu via dev wrote: >> >> Hi all, >> >> We have shut down most tests in the Jenkins Load Tests and Performance >> Tests categories [1, 2], as they have been mig

Re: Disabling Jenkins Jobs

Hi all, We have shut down most tests in the Jenkins Load Tests and Performance Tests categories [1, 2], as they have been migrated to GitHub Actions for a while and are continuously publishing the metrics as expected. Please refer to https://github.com/apache/beam/pull/29092 for these tests. Note

Re: [VOTE] Release 2.51.0, release candidate #1

> Would it impact all python users including breaking the new user, quick > start experience? Or would it impact users of a specific IO or > configuration? > It is the latter. It will impact users of Specific IO (BigQueryIO read) specific configuration (Direct_Read). Note that the default

Re: [PROPOSAL] [Nice-to-have] CI job names and commands that match

Thanks for raising this. This generally works, though some jobs run more than one gradle task (e.g. some IO_Direct_PreCommit run both :build (which executes unit tests) and :integrationTest). Another option is to normalize the naming of every job, saying the job name is X, then workflow name is

Re: [VOTE] Release 2.51.0, release candidate #1

e errors in the instructions >>> but they don't have to do with this release. >>> >>> Re-ran mass_comment.py on https://github.com/apache/beam/pull/28663. >>> There are enough red signals there that some triage is needed. Any help >>> triaging would be appre

Re: [VOTE] Release 2.51.0, release candidate #1

+1 (non-binding) Tested on Java IO load tests ( https://github.com/bvolpato/DataflowTemplates/tree/56d18a31c1c95e58543d7a1656bd83d7e859b482/it) BigQueryIO, TextIO, BigtableIO, SpannerIO on Dataflow legacy runner and runner v2 On Fri, Oct 6, 2023 at 3:23 PM Kenneth Knowles wrote: > Additionally

Re: [Proposal] Enable EnricoMi/publish-unit-test-result-action

Yi, I'm definitely +1 on adding this, this is definitely a gap in >>> our GitHub Actions infra today. >>> >>> Thanks, >>> Danny >>> >>> On Tue, Sep 5, 2023 at 10:35 AM Yi Hu via dev >>> wrote: >>> >>>> Hi eve

[Proposal] Enable EnricoMi/publish-unit-test-result-action

Hi everyone, As you may have noticed, GitHub Action for test suites are setting up in the Beam repository. A current gap is that Jenkins has a pretty convenient test result page showing all tests / failed tests and stack trace, while these are not available in github workflow logs. Here we

Re: [VOTE] Release 2.50.0, release candidate #2

+1 (non-binding) Verified Java IO load tests (TextIO, BigQuery, Bigtable) on Dataflow runner (legacy and V2) using https://github.com/apache/beam/tree/master/it On Mon, Aug 28, 2023 at 1:13 PM Ahmet Altay via dev wrote: > +1 (binding). > > I validated python quick starts on direct and dataflow

Re: [ANNOUNCE] New committer: Ahmed Abualsaud

Congrats Ahmed! On Fri, Aug 25, 2023 at 10:11 AM Ritesh Ghorse via dev wrote: > Congrats Ahmed! > > On Fri, Aug 25, 2023 at 9:53 AM Kerry Donny-Clark via dev < > dev@beam.apache.org> wrote: > >> Well done Ahmed! >> >> On Fri, Aug 25, 2023 at 9:17 AM Danny McCormick via dev < >>

Re: [RESULT] [VOTE] Vendored Dependency guava 32.1.2-jre Release

Hi everyone, The vendored guava 32.1.2-jre is now released, and beam is switched to use this version [1]. If you have working pull requests that have called vendor guava directly (Java SDK), it may need to rebase the PR branch onto the latest master branch, and switch `guava.v26_0_jre` namespace

Re: [RESULT] [VOTE] Vendored Dependency guava 32.1.2-jre Release

Hi, Could a PMC member please help me finalize the source release? That is copy https://dist.apache.org/repos/dist/dev/beam/vendor/guava-32_1_2-jre/ into https://dist.apache.org/repos/dist/release/beam/vendor/guava-32_1_2-jre using svn. Thanks! Regards, Yi On Tue, Aug 8, 2023 at 9:01 AM Yi Hu

[RESULT] [VOTE] Vendored Dependency guava 32.1.2-jre Release

I'm happy to announce that we have unanimously approved this release. There are 5 approving votes, 3 of which are binding: * Chamikara Jayalath * Ahmet Altay * Robert Bradshaw There are no disapproving votes. Thanks everyone! -- Yi Hu, (he/him/his) Software Engineer

Re: [VOTE] Vendored Dependency guava 32.1.2-jre Release

+1 (non-binding) verified signature of source release, artifacts (jar, pom); also tested the staged artifact locally with beam java core On Mon, Aug 7, 2023 at 1:22 PM Ahmet Altay wrote: > +1 - I checked the tests on the linked testing PR. Thank you! > > On Thu, Aug 3, 2023 at 12:02

[VOTE] Vendored Dependency guava 32.1.2-jre Release

Hi everyone, Please review the release of the following artifacts that we vendor: * beam-vendor-guava-32_1_2-jre Please review and vote on the release candidate 1 for the version 0.1, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific

Re: [DISCUSS] Upgrade vended guava version

Hi Hong, Thanks for bringing this up. Sure, I would like to volunteer to work as release manager [1] to vendor guava 32.1.2-jre. Created GitHub Issue for tracking [2]. Regards, Yi [1] https://s.apache.org/beam-release-vendored-artifacts [2] https://github.com/apache/beam/issues/27801 On Mon,

Re: [VOTE] Release 2.49.0, release candidate #2

Sorry for bothering, It appears the email delivery issue still exists. In case the last emails are not received, check the complete thread here: https://lists.apache.org/thread/r7r5q5mq7rqjrfbf8nj90smrdkss0sbf Still waiting for PMC finalization, mainly deploy the source release from staging

[ANNOUNCE] Beam 2.49.0 Released

The Apache Beam Team is pleased to announce the release of version 2.49.0. You can download the release here: https://beam.apache.org/get-started/downloads/ (website daily update pending) This release includes bug fixes, features, and improvements detailed on the Beam Blog:

Re: [VOTE] Release 2.49.0, release candidate #2

Could a PMC member please help finalizing the release ( https://beam.apache.org/contribute/release-guide/#pmc-only-finalization), mainly deploy the source release from staging ( https://dist.apache.org/repos/dist/dev/beam/2.49.0/) to release (will be

Re: [VOTE] Release 2.49.0, release candidate #2

>> >>> Jan >>> On 7/13/23 02:30, Bruno Volpato via dev wrote: >>> >>> +1 (non-binding). >>> >>> Tested with https://github.com/GoogleCloudPlatform/DataflowTemplates >>> (Java SDK 11, Dataflow runner). >>> >>> Thanks Yi!

Re: [VOTE] Release 2.49.0, release candidate #2

Bruno Volpato via dev wrote: > > +1 (non-binding). > > Tested with https://github.com/GoogleCloudPlatform/DataflowTemplates > (Java SDK 11, Dataflow runner). > > Thanks Yi! > > On Tue, Jul 11, 2023 at 4:23 PM Yi Hu via dev wrote: > >> Hi everyone, >> Please

[VOTE] Release 2.49.0, release candidate #2

Hi everyone, Please review and vote on the release candidate #2 for the version 2.49.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) Reviewers are encouraged to test their own use cases with the release candidate, and vote +1 if no

Re: [VOTE] Release 2.49.0, release candidate #1

>> (Java SDK 11, Dataflow runner). >> >> Thanks Yi! >> >> On Fri, Jul 7, 2023 at 10:51 AM Alexey Romanenko < >> aromanenko@gmail.com> wrote: >> >>> +1 (binding) >>> >>> Tested with https://github.com/Talend/beam-sample

Re: [VOTE] Release 2.49.0, release candidate #1

ctions >> (Java SDK v8/v11/v17, Spark 3.x runner). >> >> — >> Alexey >> >> On 6 Jul 2023, at 17:34, Yi Hu via dev wrote: >> >> Hi everyone, >> Please review and vote on the release candidate #1 for the version >> 2.49.0, as follows: >> [ ] +

Re: [VOTE] Release 2.49.0, release candidate #1

wrote: > +1 (binding) > > Tested with https://github.com/Talend/beam-samples/actions > (Java SDK v8/v11/v17, Spark 3.x runner). > > — > Alexey > > On 6 Jul 2023, at 17:34, Yi Hu via dev wrote: > > Hi everyone, > Please review and vote on the release candidate #1

[VOTE] Release 2.49.0, release candidate #1

Hi everyone, Please review and vote on the release candidate #1 for the version 2.49.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) Reviewers are encouraged to test their own use cases with the release candidate, and vote +1 if no

Re: [PROPOSAL] Preparing for 2.49.0 Release

Hi everyone, The release branch for 2.49.0 has been cut. There is currently no outstanding change that needs to be cherry-picked, and no open milestone [1]. The verification on the release branch is done on [2]. There were a few postcommit failures and flaky tests, all known issues identified as

Re: [PROPOSAL] Preparing for 2.49.0 Release

Hi, As for preparing the release, could one of the owner of apache-beam project in PyPi please add me as a maintainer? Username: abacn email: y...@apache.org Best, Yi On Thu, Jun 15, 2023 at 10:44 AM Yi Hu wrote: > Hey Beam community, > > The next release (2.49.0) branch cut is scheduled on

Re: Jenkins down

Hi, Jenkins is now back and tests run successfully. Thanks for your patience! Best, Yi On Thu, Jun 15, 2023 at 3:13 PM Yi Hu wrote: > Dear Beam developers, > > There is currently an outage for Beam repo's Jenkins test server ( > https://github.com/apache/beam/issues/27142 and >

Jenkins down

Dear Beam developers, There is currently an outage for Beam repo's Jenkins test server ( https://github.com/apache/beam/issues/27142 and https://issues.apache.org/jira/browse/INFRA-24703) some tests shows success status but actually not run anything. Until it get resolved I propose a code freeze

[PROPOSAL] Preparing for 2.49.0 Release

Hey Beam community, The next release (2.49.0) branch cut is scheduled on June 28th, 2023, according to the release calendar [1]. I volunteer to perform this release. My plan is to cut the branch on that date, and cherrypick release-blocking fixes afterwards, if any. Please help me make sure the

[Notice] Jenkins seed job comment trigger no longer working, and possible solutions

Dear Beam Developers, tl;dr For PRs involving Jenkins task changes authored by Beam committers, "Run seed job" no longer working due to apache infra change. It is noted that due to recent Apache Infra change on LDAP server, Beam Jenkins CI/CD no longer has access to the GitHub username list, and

Re: [ANNOUNCE] New committer: Damon Douglas

Congrats Damon! On Tue, Apr 25, 2023 at 8:55 AM Ritesh Ghorse via dev wrote: > Congratulations Damon! > > On Tue, Apr 25, 2023 at 12:03 AM Byron Ellis via dev > wrote: > >> Congrats Damon! >> >> On Mon, Apr 24, 2023 at 8:57 PM Austin Bennett wrote: >> >>> thanks for all you do @Damon Douglas

Re: [ANNOUNCE] New committer: Anand Inguva

Congrats Anand! On Fri, Apr 21, 2023 at 3:54 PM Danielle Syse via dev wrote: > Congratulations! > > On Fri, Apr 21, 2023 at 3:53 PM Damon Douglas via dev > wrote: > >> Congratulations Anand! >> >> On Fri, Apr 21, 2023 at 12:28 PM Ritesh Ghorse via dev < >> dev@beam.apache.org> wrote: >> >>>

Re: [VOTE] Vendored Dependencies Release

t;> +1 >> >> On Mon, Apr 17, 2023 at 11:20 AM Chamikara Jayalath via dev < >> dev@beam.apache.org> wrote: >> >>> +1 >>> >>> Thanks, >>> Cham >>> >>> On Mon, Apr 17, 2023 at 11:04 AM Kenneth Knowles >>>

Re: [VOTE] Vendored Dependencies Release

+1 > > On Mon, Apr 17, 2023 at 11:20 AM Chamikara Jayalath via dev < > dev@beam.apache.org> wrote: > >> +1 >> >> Thanks, >> Cham >> >> On Mon, Apr 17, 2023 at 11:04 AM Kenneth Knowles wrote: >> >>> +1 >>> >>> On

[VOTE] Vendored Dependencies Release

Please review the release of the following artifacts that we vendor: * beam-vendor-grpc-1_54_0 Hi everyone, Please review and vote on the release candidate #1 for the version 0.1, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments)

Re: [Python SDK] Use pre-released dependencies for Beam python unit testing

/issues/23734 -- affected MacOS suites >> - https://github.com/apache/beam/issues/22159 -- (not detected by us, >> but potentially could have affected a performance test). >> >> I'm afraid a dedicated suite may not give us desired test coverage to >> catch regression at RC s

Re: [Python SDK] Use pre-released dependencies for Beam python unit testing

Thanks Anand, This would be very helpful to avoid experiencing multiple time ( https://s.apache.org/beam-python-dependencies-pm). One thing to note is that Beam Jenkins CI is experiencing many issues recently, mostly due to that multiple Jenkins plugins does not scale (draining GitHub API call

Re: Jenkins Flakes

+1 Thanks Danny for figuring out a solution. Best, Yi On Tue, Apr 11, 2023 at 10:56 AM Svetak Sundhar via dev wrote: > +1 to the proposal. > > Regarding the "(and not guaranteed to work)" part, is the resolution that > the memory issues may still persist and we restore the normal retention >

[PROPOSAL] Upgrade vendor grpc

Hi all, I would like to volunteer to upgrade the Beam vendored grpc, as requested by the GitHub Issue [1]. I checked the project history that we did four upgrades in the last 2 years (1.26->1.36->1.43->1.48) and the last time was in Aug 2022 [2]. There have been vulnerabilities in its

Re: Broken Jenkins jobs

Hi Alexy, Thanks for raising this. The breaking change is found: https://github.com/apache/beam/pull/25566 Best, Yi On Mon, Feb 20, 2023 at 12:44 PM Alexey Romanenko wrote: > Hi all, > > Jenkins jobs “beam_PreCommit_SQL_Java11_Commit” [1] and > “beam_PreCommit_SQL_Java17_Commit” [2] seems are

Re: Performance and Cost benchmarking

Hi everyone, Thanks for your attention. Since this last thread there are works implementing the utilities for the performance and cost benchmarking framework ongoing, currently under the DataflowTemplate repository ( https://github.com/GoogleCloudPlatform/DataflowTemplates). In order to use these

Re: Refactor Kubernetes Kafka External load balancer dependency in tests

Hi Damon, Thanks for the proposal! Our k8s infrastructure has long been under maintained. I agree Public IP exposure is not necessary and it has triggered security alerts quite often... Would like to help if needed. Best, Yi On Mon, Jan 23, 2023 at 1:25 PM Damon Douglas via dev wrote: > Hello

Re: Credentials Rotation Failure on IO-Datastores cluster

Thanks for reporting. I have bumped the pool size of io-datastore as we have more tests being added and the default-pool frequently becomes unschedulable due to memory constraints. A simple fix is just rename the 'pool1' back to 'default-pool'. On Thu, Dec 1, 2022 at 1:26 PM Danny McCormick

Re: [ANNOUNCE] New committer: Yi Hu

Thank you for the help of you all over the time, and I am glad to contribute and help with the community. Best, Yi On Thu, Nov 10, 2022 at 11:29 AM Alexey Romanenko wrote: > Congratulations! Well deserved! > > — > Alexey > > On 9 Nov 2022, at 21:01, Tomo Suzuki via dev wrote: > >

Re: [ANNOUNCE] New committer: Ritesh Ghorse

Congratulations Ritesh! On Thu, Nov 3, 2022 at 7:23 PM Byron Ellis via dev wrote: > Congratulations! > > On Thu, Nov 3, 2022 at 4:21 PM Austin Bennett > wrote: > >> Congratulations, and Thanks @riteshgho...@apache.org! >> >> On Thu, Nov 3, 2022 at 4:17 PM Sachin Agarwal via dev < >>

Re: [ACTION REQUESTED] Triage 3-10 issues (get the labels right, basically)

up to 120, my bad On Mon, Oct 31, 2022 at 4:29 PM Yi Hu wrote: > Thanks for raising this. Also closed a couple of issues. Now the number is > up to 12. > > On Mon, Oct 31, 2022 at 9:24 AM Manu Zhang > wrote: > >> I closed several issues that have already been fixed. It looks we didn’t >>

Re: [ACTION REQUESTED] Triage 3-10 issues (get the labels right, basically)

Thanks for raising this. Also closed a couple of issues. Now the number is up to 12. On Mon, Oct 31, 2022 at 9:24 AM Manu Zhang wrote: > I closed several issues that have already been fixed. It looks we didn’t > reference the issue number in commit message which can auto-close the issue. > >

Re: Inclusive terminology: "Sickbay" ==> "Disabled test"

(From someone who received education not in English and from another field). I first assumed this is some terminology just like "flaky", "smock test" because I came from another world and did not know most of the terms in this field. That said, I just assumed it was another term used in the field

Re: Performance and Cost benchmarking

Hi Andrew and Pranav, Thanks for pointing out the current infrastructure we have in Beam. I have investigated the current Performance tests of Beam IOs in the Beam repo and summarized the current tools and infrastructure we have in this document:

Re: Representation of logical type beam:logical_type:datetime:v1

Hi, It now appears that if we want a clean solution then we have to add a fixed size primitive type to Beam atomic types. Or, we then have a millis_instant logical type that does not have to_language_type and to_representation_type implementations. Any suggestions are welcome! Best, Yi On Thu,

Re: Representation of logical type beam:logical_type:datetime:v1

On Wed, Aug 17, 2022 at 5:14 PM Chamikara Jayalath wrote: > > I think this is fine (even though it would add a small perf hit to > JdbcIO.Read). We also probably should make this conversion a utility method > that can be used elsewhere when we need to encode datetime fields. > We should also

Re: Representation of logical type beam:logical_type:datetime:v1

w to resolve the conflicting definitions of > beam:logical_type:datetime:v1. I'm not quite sure how/if we can do that > without breaking pipeline update. > > Brian > > > On Fri, Aug 12, 2022 at 7:50 AM Yi Hu via dev wrote: > >> Hi Cham, >> >> Thanks for the comment

Re: Representation of logical type beam:logical_type:datetime:v1

Hi Cham, Thanks for the comments. > >> >> ii. "beam:logical_type:instant:v1" is still backed by INT64, but in >> implementation it will use BigEndianLongCoder to encode/decode the stream. >> >> > Is this to be compatible with the current Java implementation ? And we > have to update other SDKs

Representation of logical type beam:logical_type:datetime:v1

Hi, tl;dr There are two (or three) different representations used for the URN "beam:logical_type:datetime:v1" in Beam Java SDK. Clarification or clean up is needed. I recently try to resolve a long-time issue https://github.com/apache/beam/issues/19817 for the datetime logical type

Re: Update PR description template

Hi all, I have drafted a document summarizing the ideas suggested in this thread. https://docs.google.com/document/d/10FlXOo_hL2QYTPhwS8uHSyJbQCzwC3K3C12tFccANA8/edit# where I extracted the following action items: short-term (easy) items - Split the linting task (e.g. java checkStyle) out from

Re: [Proposal] Path to Re-enabling Build Comment Triggers

Thanks for the investigation done and disabling the "admins verify patch" comment. Wondering if there is a way to early return the bot comments for ghprb (like blacklist)? Besides asf-ci there are also comments from github-actions (e.g. assign reviewer or "stopping review notifications"). In the