[VOTE] Release 2.56.0, release candidate #2

Hi everyone, Please review and vote on the release candidate #2 for the version 2.56.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) Reviewers are encouraged to test their own use cases with the release candidate, and vote +1 if no

Re: [VOTE] Release 2.56.0, release candidate #1

ing). Tested the simple Dataflow ML job: > https://github.com/google/dataflow-ml-starter/actions/runs/8824985423/job/24228468173 > > On Wed, Apr 24, 2024 at 2:01 PM Danny McCormick via dev < > dev@beam.apache.org> wrote: > >> Hi everyone, >> Please review and vote on the

[VOTE] Release 2.56.0, release candidate #1

Hi everyone, Please review and vote on the release candidate #1 for the version 2.56.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) Reviewers are encouraged to test their own use cases with the release candidate, and vote +1 if no

RunInference Model Loading

Hey folks, while I've been waiting on cherry-picks I put together a proposal for allowing users to load more than one copy of a model in RunInference. Basically, today, RunInference allows loading models in 3 different ways: 1. (Default): load a model per worker process 2. Load a single copy of

Re: 2.56.0 release

a dev >> wrote: >> >>> Hi, >>> >>> a nice round version number :) >>> >>> Cheers, >>> Maciej >>> >>> On Thu, Apr 4, 2024 at 3:56 PM Danny McCormick via dev < >>> dev@beam.apache.org> wrote: >>&g

Re: 2.56.0 release

i, >> >> a nice round version number :) >> >> Cheers, >> Maciej >> >> On Thu, Apr 4, 2024 at 3:56 PM Danny McCormick via dev < >> dev@beam.apache.org> wrote: >> >>> Hey everyone, >>> >>> On top of running

Re: release process: typescript SDK?

Probably the easiest way for this to happen is for @Robert Bradshaw to get the token set up as a secret (should be quick) and then Austin to take the workflow forward. In the past to get secrets added, Infra has asked that I (a) email r...@apache.org with the secret name and secret contents, and

Re: release process: typescript SDK?

I've never published npm artifacts before, but I imagine the hardest part is getting the credentials set up, then it is probably very easy to set up a GitHub Actions workflow to publish

Re: [RESULT] [VOTE] Release 2.55.1, release candidate #2

Hi everyone, all steps have been completed, so the release is finalized! Please let me know if you encounter any issues Thanks, Danny On Mon, Apr 8, 2024 at 9:06 AM Danny McCormick wrote: > I'm happy to announce that we have unanimously approved this release. > > There are 5 approving votes, 3

[RESULT] [VOTE] Release 2.55.1, release candidate #2

I'm happy to announce that we have unanimously approved this release. There are 5 approving votes, 3 of which are binding: * Valentyn Tymofieiev (binding) * Kenneth Knowles (binding) * Robert Bradshaw (binding) * Jeff Kinard * XQ Hu There are no disapproving votes. I will now work on finalizing

2.56.0 release

Hey everyone, On top of running the current 2.55.1 patch release, I'd like to volunteer as release manager for the 2.56.0 release, which will be cut on April 17. Please add any release blockers to the 2.56.0 milestone [1], I will be trying to push on those as the cut date approaches. Thanks,

Re: [VOTE] Patch Release 2.55.1, release candidate #2

breakage, but in general we should > include that for patch releases as well. > > +1. Spot-checked some Python SDK artifacts and containers. > > On Wed, Apr 3, 2024 at 8:08 AM Danny McCormick via dev < > dev@beam.apache.org> wrote: > >> Hi everyone, >> >>

[VOTE] Patch Release 2.55.1, release candidate #2

Hi everyone, I put together a patch release per the conversation in https://lists.apache.org/thread/kvq1wsj505pvopkq186dnvc0l6ryyfh0. Please review and vote on the release candidate #2 (I messed up rc1) for the version 2.55.1, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the

Re: Patch release proposal

cycle, as the >>> release process itself could take ~10 days minimum if there is single RC. >>> >>> Besides, there should be a Beam YAML validation workflow and added in >>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=136803

Re: Patch release proposal

+1 on a patch release - we've done a fair amount of work to make releasing easier, and one of my hopes is that it will enable quick patches like this. I'd vote we try to fix the underlying Java piece as well, though, doing a patch release for one language shouldn't be significantly cheaper than

Re: [VOTE] Release 2.55.0, release candidate #3

+1 - validated some ML examples with the interactive runner Thanks, Danny On Thu, Mar 21, 2024 at 9:21 AM Jan Lukavský wrote: > +1 (binding) > > Tested Java SDK with FlinkRunner. > > Jan > On 3/20/24 22:40, Chamikara Jayalath via dev wrote: > > +1 (binding) > > Tested multi-lang Java/Python

Re: Credentials Rotation Failure on Metrics cluster (2024-03-01)

FYI, this succeeded on retry. This is the second time in a row it failed on the first try, both times because it ran into a conflicting operation. My guess is a fork is running this job as well - added https://github.com/apache/beam/pull/30479 to try to fix the problem. On Thu, Feb 29, 2024 at

Re: [GSOC] Build out Beam Use Cases

Hey Nivaldo, thanks for reaching out and for your interest in contributing to Beam in this way! I'll do my best to answer your questions below: > 1. Would I actually be eligible to apply to GSOC for this project, or do I not count as an open source beginner anymore in this case? The total number

Re: GsoC Apache Beam ManagerIO project communication

Hey Frank, thanks for reaching out, I'm glad to hear you're interested in this project! I'll try to answer your questions below. > As I understand it, ManageIO serves as an abstraction layer between the Runner (pipeline) and the Connector, facilitating dynamic configuration, data access, and

Re: Credentials Rotation Failure on Metrics cluster (2024-02-01)

This was caused by a race with another cluster update operation that was happening when this job ran. I reran the job and it succeeded, so it should be safe to ignore for now (though we may want to make this more robust in the future). On Wed, Jan 31, 2024 at 9:10 PM gacti...@beam.apache.org <

Re: [Question] Eager to Contribute to Apache Beam - Hiranmaye Pakshirajan

Hey Hiranmaye, welcome to Beam, I appreciate your enthusiasm for the project! If you're interested in getting involved in the project generally, I'd recommend starting by reading through the Beam contribution guide , and taking a look to see if there are any

Re: Beam Jenkins nodes offline

Hey Gavin, thanks for looking into this. I think there's a separate thread going for this, but we've now fully migrated all of our CI to GitHub Actions, so we can probably just safely turn down Beam's Jenkins instance. Valentyn opened a Jira ticket yesterday for this purpose -

Re: [VOTE] Release 2.53.0, release candidate #2

+1 (non-binding) Tested with some example ML notebooks. Thanks, Danny On Tue, Dec 26, 2023 at 6:41 PM XQ Hu via dev wrote: > +1 (non-binding) > > Tested with the simple RunInference pipeline: > https://github.com/google/dataflow-ml-starter/actions/runs/7332832875/job/19967521369 > > On Tue,

Re: [VOTE] Release 2.53.0, release candidate #1

+1 (non-binding) Tested against a few Beam ML notebooks using the direct runner and Dataflow runner. Thanks, Danny On Thu, Dec 21, 2023 at 3:50 PM Jack McCluskey via dev wrote: > Happy holidays everyone, > > Please review and vote on the release candidate #1 for the version 2.53.0, > as

Beam 2.52.0 Release

I am happy to announce that the 2.52.0 release of Beam has been finalized. This release includes both improvements and new functionality. For more information on changes in 2.52.0, check out the detailed release notes - https://github.com/apache/beam/milestone/16. Here is an overview of the

[RESULT] [VOTE] Release 2.52.0, release candidate #5

I'm happy to announce that we have unanimously approved this release. There are 7 approving votes, 4 of which are binding: * Bruno Volpato * Valentyn Tymofieiev (binding) * Jean-Baptiste Onofré (binding) * Jan Lukavský (binding) * Svetak Sundhar * Robert Bradshaw (binding) * XQ Hu There are no

[VOTE] Release 2.52.0, release candidate #5

Hi everyone, Please review and vote on the release candidate #5 for the version 2.52.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) Reviewers are encouraged to test their own use cases with the release candidate, and vote +1 if no

Re: [VOTE] Release 2.52.0, release candidate #4

e this issue, but would like to hear opinions. > > > Best, > Bruno > > > On Mon, Nov 13, 2023 at 5:14 AM Jan Lukavský wrote: > >> +1 (binding) >> >> Validated Java SDK with Flink runner on own use cases. >> Jan >> >> On 11/12/23 00:4

Re: [VOTE] Release 2.52.0, release candidate #4

ume this is a VOTE to match the subject and is for RC #4 > , but wanted to call that out. > > Cheers, > Austin > > > > On Sat, Nov 11, 2023, 3:44 PM Danny McCormick via dev > wrote: > >> Hi everyone, >> Please review and vote on the release candidate

[VOTE] Release 2.52.0, release candidate #4

Hi everyone, Please review and vote on the release candidate #3 for the version 2.52.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) Reviewers are encouraged to test their own use cases with the release candidate, and vote +1 if no

Re: [VOTE] Release 2.52.0, release candidate #3

gt; >> +1 (binding). >> >> Tested multi-lang Java/Python jobs. >> >> Thanks, >> Cham >> >> On Fri, Nov 10, 2023, 12:28 PM Svetak Sundhar via dev < >> dev@beam.apache.org> wrote: >> >>> +1 Non Binding -- tested Python SDK batch. >

Re: [VOTE] Release 2.52.0, release candidate #3

t;> wrote: >> > >> > +1 (binding) >> > >> > Java SDK with Spark runner >> > >> > — >> > Alexey >> > >> > On 9 Nov 2023, at 16:44, Ritesh Ghorse via dev >> wrote: >> > >> > +1 (non-binding) >>

Re: Adding Dead Letter Queues to Beam IOs

Thanks - the general ideas seem solid, I added some questions/comments as well. On Fri, Nov 10, 2023 at 1:32 PM Robert Bradshaw via dev wrote: > Thanks. I added some comments to the doc and open PR. > > On Wed, Nov 8, 2023 at 12:44 PM John Casey via dev > wrote: > > > > Hi All, > > > > I've

[VOTE] Release 2.52.0, release candidate #3

Hi everyone, Please review and vote on the release candidate #3 for the version 2.52.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) Reviewers are encouraged to test their own use cases with the release candidate, and vote +1 if no

Re: [VOTE] Release 2.52.0, release candidate #2

t; > > Svetak Sundhar > > Data Engineer > s vetaksund...@google.com > > > > On Wed, Nov 8, 2023 at 9:04 AM Jean-Baptiste Onofré > wrote: > >> +1 (binding) >> >> Regards >> JB >> >> On Wed, Nov 8, 2023 at 12:24 AM Danny McCormick

[VOTE] Release 2.52.0, release candidate #2

Hi everyone, Please review and vote on the release candidate #2 for the version 2.52.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) Reviewers are encouraged to test their own use cases with the release candidate, and vote +1 if no

Re: Disabling Jenkins Jobs

I don't think it's related. I noticed the problem half an hour ago; it seems there's an expired cert on the Jenkins machines. I'm hoping https://github.com/apache/beam/actions/runs/6786537134/job/18447281366 will fix this since the IO-Datastores cert is the problematic piece I think (and that has

Re: Beam 2.52.0 Release

Update: all issues mentioned in my last email are resolved and I was able to get all artifacts for RC published except for the PyPi artifacts (which I was about to do). However, during my validation before sending out the voting email I discovered a breaking change in the Datastore IO caused by

Re: Beam 2.52.0 Release

Update before the weekend - I likely won't have an RC out until early next week. I'm currently running down the following issues with RC creation: 1) Publishing java artifacts is failing because of an illegal implicit gradle dependency [1]. I have a PR [2] and cherrypick PR [3] prepared which

Re: Credentials Rotation Failure on Metrics cluster (2023-11-01)

Yep, it is safe to ignore On Wed, Nov 1, 2023 at 2:43 PM Kenneth Knowles wrote: > +Danny McCormick is this the converse of the > other failure? (I didn't click through I just read the other thread) > > On Tue, Oct 31, 2023 at 10:10 PM gacti...@beam.apache.org < > beamacti...@gmail.com> wrote:

Re: Beam 2.52.0 Release

I just cut the 2.52.0 release branch . All commits after https://github.com/apache/beam/commit/ea8596f2df0e3e4b9da9f215ae6745c2ddfb6612 will be targeted for a later release. There is currently still one open release blocker

Re: Credentials Rotation Failure on IO-Datastores cluster

My guess is that this is due to running this both on GitHub Actions and Jenkins. The Actions run succeeded, so I don't think we need to worry about this - https://github.com/apache/beam/actions/runs/6714783844 It seems like for the metrics job the opposite happened - the Actions run failing

Re: [Discuss] Idea to increase RC voting participation

, >>> https://github.com/apache/beam/issues/28635 seems like it was closed as >>> completed but not tagged. >>> >>> I'm happy to see that the bot reads the version from the repo to find >>> the appropriate milestone, rather than using the nearest open one. Ju

Re: [Discuss] Idea to increase RC voting participation

he version from the repo to find the > appropriate milestone, rather than using the nearest open one. Just > recording that for the thread since I first read the description as the > latter. > > Kenn > > On Tue, Oct 24, 2023 at 2:34 PM Danny McCormick via dev < > dev@bea

Re: [Discuss] Idea to increase RC voting participation

on, Oct 23, 2023 at 7:26 AM Danny McCormick via dev < >> dev@beam.apache.org> wrote: >> >>> So to summarize, I think there's broad consensus (or at least lazy >>> consensus) around the following: >>> >>> - (1) Updating our release email/guid

Re: [Discuss] Idea to increase RC voting participation

So to summarize, I think there's broad consensus (or at least lazy consensus) around the following: - (1) Updating our release email/guidelines to be more specific about what we mean by release validation/how to be helpful during this process. This includes both encouraging validation within each

Re: [PYTHON] partitioner utilities?

I'm interested adding something like this, I could see these being generally useful for a number of cases (one that immediately comes to mind is partitioning datasets into train/test/validation sets and writing each to a different place). I'm assuming Top (or FixedSample) needs to be small enough

Beam 2.52.0 Release

Hey everyone, the next release (2.52.0) branch cut is scheduled for Nov 1, 2023, 2 weeks from today, according to the release calendar [1]. I'd like to perform this release; I will cut the branch on that date, and cherrypick release-blocking fixes afterwards, if any. Please help with the release

Re: Disabling Jenkins Jobs

es. The full list is >> tracked in https://github.com/apache/beam/issues/28426 . In the Issue >> there is also information about the status of migration as we proceed. >> >> Regards, >> Yi >> >> On Thu, Sep 14, 2023 at 12:44 PM Danny McCormick via dev < >>

Re: [GitHub Actions] Requiring a test but not running it until requested

I actually don't think GitHub supports path filters for required checks, so you can't say something like "check X is only required if path Y or Z are modified", you can only say "check X is required". I'm not 100% sure on this, but it matches my memory and I could neither find docs with that

Re: [PROPOSAL] [Nice-to-have] CI job names and commands that match

t; Also agree that this is a post Jenkins exit task. One migration in an area > at a time please. > > On Tue, Oct 10, 2023, 8:07 AM Kenneth Knowles wrote: > >> On Tue, Oct 10, 2023 at 10:21 AM Danny McCormick via dev < >> dev@beam.apache.org> wrote: >> >>&

Re: [PROPOSAL] [Nice-to-have] CI job names and commands that match

I'm +1 on: - standardizing our naming - making job names match their commands exactly (though I'd still like the `Run` prefix so that you can do things like say "Suite XYZ failed" without triggering the automation) - removing pre/postcommit from the naming (we actually already run many precommits

Re: [ANNOUNCE] New PMC Member: Alex Van Boxel

Congrats Alex, this is well deserved! On Tue, Oct 3, 2023 at 2:50 PM Jack McCluskey via dev wrote: > Congrats, Alex! > > On Tue, Oct 3, 2023 at 2:49 PM XQ Hu via dev wrote: > >> Configurations, Alex! >> >> On Tue, Oct 3, 2023 at 2:40 PM Kenneth Knowles wrote: >> >>> Hi all, >>> >>> Please

Re: [ANNOUNCE] New PMC Member: Valentyn Tymofieiev

Congrats Valentyn, this is well deserved! On Tue, Oct 3, 2023 at 2:50 PM Anand Inguva via dev wrote: > Congratulations!! > > On Tue, Oct 3, 2023 at 2:49 PM XQ Hu via dev wrote: > >> Congratulations, Valentyn! >> >> On Tue, Oct 3, 2023 at 2:40 PM Kenneth Knowles wrote: >> >>> Hi all, >>> >>>

Re: [ANNOUNCE] New PMC Member: Robert Burke

Congrats Robert, this is very well deserved! On Tue, Oct 3, 2023 at 2:50 PM Anand Inguva via dev wrote: > Congratulations!! > > On Tue, Oct 3, 2023 at 2:49 PM XQ Hu via dev wrote: > >> Congratulations, Robert! >> >> On Tue, Oct 3, 2023 at 2:40 PM Kenneth Knowles wrote: >> >>> Hi all, >>> >>>

Re: [VOTE] Release 2.51.0, release candidate #1

v1 for this release? > > Thanks in advance for the clarification, > > > > Svetak Sundhar > > Data Engineer > s vetaksund...@google.com > > > > On Tue, Oct 3, 2023 at 2:14 PM Danny McCormick via dev < > dev@beam.apache.org> wrote: > >>

Re: [VOTE] Release 2.51.0, release candidate #1

+1 (non-binding) Tested python/ML execution with https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_huggingface.ipynb (interactive runner) and https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/automatic_model_refresh.ipynb (Dataflow runner).

Re: User-facing website vs. contributor-facing website

gt;>>> >>>>> > Wiki is yet another place to go, hard to navigate, not discoverable. >>>>> >>>>> Agree. >>>>> >>>>> So the "neverending argument" is so far unanimous on this particular >>>>&

Re: User-facing website vs. contributor-facing website

a quick search to see if there was a standard answer to having top > and side nav for a docs/ folder of markdown in your github repo. I guess > that is GitHub Pages? TBH I have used them happily in the distant past but > somehow I thought they had been deprecated or something.

Re: User-facing website vs. contributor-facing website

, > Cham > > On Thu, Sep 21, 2023 at 9:14 AM Danny McCormick via dev < > dev@beam.apache.org> wrote: > >> +1 on moving the release guide. I'd argue that everything under the >> `contribute` tag other than the main page ( >> https://beam.apache.org/contribute/) and t

Re: User-facing website vs. contributor-facing website

+1 on moving the release guide. I'd argue that everything under the `contribute` tag other than the main page ( https://beam.apache.org/contribute/) and the link to CONTRIBUTING.md makes more sense on the wiki (we can keep the section

Re: Beam ML Use Cases - Google Summer of Code 2023

Thanks for all your hard work this summer Reeba! I've really enjoyed getting to work closely with you, and I know that Beam and its users are better off because of your contributions. Thanks, Danny On Wed, Sep 13, 2023 at 1:01 PM XQ Hu via dev wrote: > The blog looks great! Thanks for doing

Disabling Jenkins Jobs

Hey everyone, I wanted to let you know that as part of the migration from Jenkins to GitHub Actions we are going to start disabling Jenkins jobs if they have a corresponding GitHub Actions job that has been running successfully for a while. We are starting with Yi's PR here -

Re: [Proposal] Enable EnricoMi/publish-unit-test-result-action

Thanks Yi, I'm definitely +1 on adding this, this is definitely a gap in our GitHub Actions infra today. Thanks, Danny On Tue, Sep 5, 2023 at 10:35 AM Yi Hu via dev wrote: > Hi everyone, > > As you may have noticed, GitHub Action for test suites are setting up in > the Beam repository. A

Re: Contribution of Asgarde: Error Handling for Beam?

I think my only concerns here are around the toil we'll be taking on, and will we be leaving the asgarde project in a better or worse place. >From a release standpoint, we would need to release it with the same cadence as Beam. Adding asgarde into our standard release process seems fairly

Re: Proposal for pyproject.toml Support in Apache Beam Python

Thanks Anand! This generally sounds good to me. I left a few questions before giving a full +1, but it definitely seems like we need some migration effort here and this seems like a good route. Thanks, Danny On Mon, Aug 28, 2023 at 10:14 AM Anand Inguva via dev wrote: > Hello Beam Dev Team, >

Re: [ANNOUNCE] New committer: Ahmed Abualsaud

Congrats Ahmed! On Fri, Aug 25, 2023 at 3:16 AM Jan Lukavský wrote: > Congrats Ahmed! > On 8/25/23 07:56, Anand Inguva via dev wrote: > > Congratulations Ahmed :) > > On Fri, Aug 25, 2023 at 1:17 AM Damon Douglas > wrote: > >> Well deserved! Congratulations, Ahmed! I'm so happy for you. >> >>

Re: Automatic signing of releases

Hey Ismael, We've actually already been doing this since May! I started a thread for this here https://lists.apache.org/thread/mw9dbbdjtkqlvs0mmrh452z3jsf68sct and its in an Actions workflow here -

Re: [Discuss] Get rid of OWNERS files

ially that was a good intention > imho) but what could be an alternative for this? Review bot also may assign > reviewers that are no longer active on the project. > >> > >> — > >> Alexey > >> > >> > >> On 8 Aug 2023, at 16:55, Danny M

Re: Propose to add the new security section to the Beam releases

Thanks XQ! This should help us be cleaner with what versions we're recommending to security conscious users. On Fri, Aug 11, 2023 at 2:17 PM XQ Hu via dev wrote: > Hi All, > > We are proposing to explicitly add the security fixes to the Beam release > notes.

Re: [Discuss] Get rid of OWNERS files

9:13 AM Alexey Romanenko > wrote: > >> I’m generally agree with this (initially that was a good intention imho) >> but what could be an alternative for this? Review bot also may assign >> reviewers that are no longer active on the project. >> >> — >> Alexey &

[RFC] Model Per Key RunInference

Hey everyone! Today, many users have pipelines that choose a single model for inference from 100s or 1000s of models based on properties of the data. Unfortunately, RunInference does not support this use case. I put together a proposal for RunInference that allows a single keyed RunInference

Re: [Feature Proposal] Add ARM Support to Beam SDK Container Images

Thanks Celeste! I left a few comments. Overall I like the proposal, but I think that the open question "If Beam SDK containers are still released by the release manager, how should we integrate the multiarch containers into the current Beam container release process?" needs to be answered before I

Re: [DISCUSS] Enable Github Discussions?

Thanks for starting this discussion! I'm a weak -1 for this proposal. While I think that GH Discussions can be a good forum, I think most of the things that Discussions do are covered by some combination of the dev/user lists and GitHub issues, and the net outcome of this will be creating one

Re: [VOTE] Release 2.48.0 release candidate #2

It looks like some Dataflow containers didn't get published, so some jobs using the legacy runner (runner v2 disabled) will fail. I kicked off the container release, so that should hopefully be available later today. Thanks, Danny On Thu, May 25, 2023 at 11:19 PM Ritesh Ghorse via dev wrote: >

Re: [Release-2.48.0] Uploading images to dockerhub

I pushed the missing containers as well, so we should be good to go. I also created a cherry pick PR, that only needs to be merged if we make it to RC2 though (hopefully not!) - https://github.com/apache/beam/pull/26871 Thanks, Danny On Wed, May 24, 2023 at 11:48 AM Danny McCormick wrote: >

Re: [Release-2.48.0] Uploading images to dockerhub

https://github.com/apache/beam/pull/26869 should fix On Wed, May 24, 2023 at 11:45 AM Chamikara Jayalath wrote: > > > On Wed, May 24, 2023 at 8:41 AM Danny McCormick > wrote: > >> This has finished. One thing I did notice: we requested that Infra create >> repos for some transform service

Re: [Release-2.48.0] Uploading images to dockerhub

This has finished. One thing I did notice: we requested that Infra create repos for some transform service docker images (context ), but those didn't seem to get published. @Chamikara Jayalath should they be added to

Re: [Release-2.48.0] Uploading images to dockerhub

I'm currently in the process of publishing the containers, I will let you know when it completes. Thanks, Danny On Tue, May 23, 2023 at 8:09 PM Ritesh Ghorse via dev wrote: > Hey everyone, > > I'm at the stage of pushing docker containers to the apache repository of >

[PMC Request] Add gpg key to release keys file

Hey everyone, as part of automating our release process (see thread here - https://lists.apache.org/thread/mw9dbbdjtkqlvs0mmrh452z3jsf68sct), could a PMC member please add the infra supplied gpg public key to our release KEYS file? I added it to our dev KEYS file

[Benchmarks/Proposal] Loading models with multi_process_shared.py for RunInference

Hey everyone, Right now, using RunInference with large models and on GPUs has several performance gaps. I put together a document focusing on one: when running inference with large models, we often OOM because we load several copies of the model at once. My document explores using the

Re: [Notice] Jenkins seed job comment trigger no longer working, and possible solutions

I'm +1 on just adding committers to a list manually. Having the ability to run seed jobs from a PR is nice, but adding a new committer is a rare enough event that automating is not worth the time IMO (as opposed to documenting this as something to do when you're a new committer). Plus this problem

Re: Introducing beam.MLTransform

Thanks Anand! I left a few comments, but overall I think this is a great/well constructed proposal - this is a good way to quickly bring a suite of preprocessing operations to Beam. Thanks, Danny On Tue, May 9, 2023 at 12:52 PM Anand Inguva via dev wrote: > Hi all, > > In Apache Beam, we plan

Re: [Proposal] Automate Release Signing

gt;>>>> +1 to automating release signing. As it stands now, this step >>>>>> requires a PMC member to add a new release manager's GPG key which can >>>>>> add >>>>>> time to getting a release started. This also results in the public key

[Proposal] Automate Release Signing

Hey everyone, I'm currently working on improving our release process so that it's easier and faster for us to release. As part of this work, I'd like to propose automating our release signing step (the push java artifacts step of build_release_candidate.sh

Re: [ANNOUNCE] New committer: Damon Douglas

Congrats Damon! On Mon, Apr 24, 2023 at 4:03 PM Ahmet Altay via dev wrote: > Congratulations Damon! > > On Mon, Apr 24, 2023 at 1:00 PM Robert Burke wrote: > >> Congratulations Damon!!! >> >> On Mon, Apr 24, 2023, 12:52 PM Kenneth Knowles wrote: >> >>> Hi all, >>> >>> Please join me and the

Re: [ANNOUNCE] New committer: Anand Inguva

Woohoo, congrats Anand! This is very well deserved! On Fri, Apr 21, 2023 at 1:54 PM Chamikara Jayalath wrote: > Hi all, > > Please join me and the rest of the Beam PMC in welcoming a new committer: > Anand > Inguva (ananding...@apache.org) > > Anand has been contributing to Apache Beam for

[RFC] RunInference Pre/Postprocessing and DLQ UX

Hey everyone, I put up a small design doc proposing a user experience for adding DLQ and pre/post processing map operation support to RunInference. I'd appreciate any thoughts or comments. https://docs.google.com/document/d/1hr1SaWraneB9dYSFyGA99JT44oKgGNhT70wz99lmdEU/edit?usp=sharing Thanks,

Re: Beam Release DockerHub Group

). Thanks, Danny On Mon, Apr 17, 2023 at 12:34 PM Robert Bradshaw wrote: > I think it'd be good if the intersection between this list and the PMC had > cardinality greater than 1. Ahmet might be a good person to keep there. > > On Mon, Apr 17, 2023 at 9:25 AM Danny McCormick via

Re: Beam Release DockerHub Group

ify >> the better. >> >> On Mon, Apr 17, 2023 at 11:18 AM Danny McCormick via dev < >> dev@beam.apache.org> wrote: >> >>> Hey everyone, in an effort to reduce the burden of running a Beam >>> release, a few committers (self include

Beam Release DockerHub Group

Hey everyone, in an effort to reduce the burden of running a Beam release, a few committers (self included) have volunteered to try to take a larger role in releases (including both running them and contributing to making them better going forward). To aid in that process, I would like to request

Re: [Python SDK] Use pre-released dependencies for Beam python unit testing

Thanks for doing this Anand, I'm +1 on option 1 as well - I think having the clear signal of the normal suite succeeding and the prerelease one failing would be helpful and there shouldn't be too much additional code necessary. That makes it really easy to treat the prerelease suite as a (at least

Re: Jenkins Flakes

ed friction and the >>>> proposed changes sound legitimate. >>>> >>>> >>>> >>>> Also, considering the number of flaky tests in general [1], code >>>> coverage might not be the pressing issue. Should it be disabled everywhere >

Re: Jenkins Flakes

ng issue. Should it be disabled everywhere >>> in favor of more reliable / faster builds? Unless Devs here are willing to >>> commit on taking actions, it doesn’t seem to provide too much value >>> recording these numbers as part of the normal pre commit jobs? >>&g

Re: Jenkins Flakes

>> >> >> Kind regards, >> >> Moritz >> >> >> >> [1] >> https://github.com/apache/beam/issues?q=is%3Aissue+is%3Aopen+label%3Aflake >> <https://urldefense.com/v3/__https:/github.com/apache/beam/issues?q=is*3Aissue*is*3Aopen*label*3Af

Jenkins Flakes

*;tldr - I want to temporarily reduce the number of builds that we retain to reduce pressure on Jenkins* Hey everyone, over the past few days our Jenkins runs have been particularly flaky across the board, with errors like the following showing up all over the place [1]:

Re: [DISCUSS] @Experimental, @Internal, @Stable, etc annotations

*;tldr - I'd like "evolving" to be further defined, specifically around how we will make decisions about breaking behavior and API changes* I don't particularly care what tags we use as long as they're well documented. With that said, I think the following framing needs to be documented with more

Re: Regarding Project proposal review and feedback

Thanks Siddharth! I left some comments on the sentiment analysis proposal, I am probably not the best person to comment on the flink datastream api one though. Thanks, Danny On Fri, Mar 24, 2023 at 11:53 PM Siddharth Aryan < siddhartharyan...@gmail.com> wrote: > Hello , > I am Siddharth Aryan a

Re: [RESULT] [VOTE] Release 2.46.0, release candidate #1

perly because it depends on versioning. I will >> update this thread with next steps when we know more. >> >> Thanks, >> Danny >> >> On Wed, Mar 8, 2023 at 1:02 PM Anand Inguva >> wrote: >> >>> Thanks Danny!! >>> >>> O

Re: [RESULT] [VOTE] Release 2.46.0, release candidate #1

; wrote: > >> Thanks Danny!! >> >> On Wed, Mar 8, 2023 at 12:14 PM Danny McCormick via dev < >> dev@beam.apache.org> wrote: >> >>> I'm happy to announce that we have unanimously approved release 2.46.0 >>> There are 8 approving votes, 5 of

  1   2   >