[Request for Feedback] Mechanism for "Beam Website Feedback"

Hi all, Following what Ahmet proposed here ( https://lists.apache.org/thread/9l18gs7vgqz6g1cfc2m66rbkrcnr9qmj), we changed the "SEED FEEDBACK" button to open the Google form (

Re: Beam ML Use Cases - Google Summer of Code 2023

The blog looks great! Thanks for doing this and I hope you have learned a lot! Thanks a lot to Danny for your support! On Wed, Sep 13, 2023 at 12:58 PM Reeba Qureshi wrote: > Hi everyone > > I have completed Google Summer of Code 2023 with Apache Beam, where I > worked on developing real-world

Re: [ANNOUNCE] New Committer: Sam Whittle

Congratulations! On Mon, Oct 16, 2023 at 1:58 PM Ahmet Altay via dev wrote: > Congratulations Sam! > > On Mon, Oct 16, 2023 at 10:42 AM Byron Ellis via dev > wrote: > >> Congrats Sam! >> >> On Mon, Oct 16, 2023 at 10:32 AM Chamikara Jayalath via dev < >> dev@beam.apache.org> wrote: >> >>>

Re: [ANNOUNCE] New Committer: Byron Ellis

Congratulations! On Mon, Oct 16, 2023 at 1:58 PM Ahmet Altay via dev wrote: > Congratulations Byron! > > On Mon, Oct 16, 2023 at 10:35 AM Tomo Suzuki via dev > wrote: > >> Congratulations! >> >> >> On Mon, Oct 16, 2023 at 1:33 PM Chamikara Jayalath via dev < >> dev@beam.apache.org> wrote: >>

Re: [VOTE] Release 2.51.0, release candidate #1

+1 (non-binding). Tested the simple dataflow ML starter job with https://github.com/google/dataflow-ml-starter/actions/runs/6397130175/job/17364408813 . On Tue, Oct 3, 2023 at 2:29 PM Danny McCormick via dev wrote: > All Beam Python versions 2.50 and greater run exclusively on Dataflow > runner

Re: [ANNOUNCE] New PMC Member: Valentyn Tymofieiev

Congratulations, Valentyn! On Tue, Oct 3, 2023 at 2:40 PM Kenneth Knowles wrote: > Hi all, > > Please join me and the rest of the Beam PMC in welcoming Valentyn > Tymofieiev as our newest PMC member. > > Valentyn has been contributing to Beam since 2017. Notable highlights > include his work

Re: [ANNOUNCE] New PMC Member: Robert Burke

Congratulations, Robert! On Tue, Oct 3, 2023 at 2:40 PM Kenneth Knowles wrote: > Hi all, > > Please join me and the rest of the Beam PMC in welcoming Robert Burke < > lostl...@apache.org> as our newest PMC member. > > Robert has been a part of the Beam community since 2017. He is our > resident

Re: [ANNOUNCE] New PMC Member: Alex Van Boxel

Configurations, Alex! On Tue, Oct 3, 2023 at 2:40 PM Kenneth Knowles wrote: > Hi all, > > Please join me and the rest of the Beam PMC in welcoming Alex Van Boxel < > alexvanbo...@apache.org> as our newest PMC member. > > Alex has been with Beam since 2016, very early in the life of the project.

Re: [YAML] Aggregations

+1 on your proposal. On Fri, Oct 20, 2023 at 4:59 PM Robert Bradshaw via dev wrote: > On Fri, Oct 20, 2023 at 11:35 AM Kenneth Knowles wrote: > > > > A couple other bits on having an expression language: > > > > - You already have Python lambdas at places, right? so that's quite a > lot more

Re: [Discuss] Idea to increase RC voting participation

+1. This is a great idea to try. @Danny McCormick FYI as our next release manager. On Wed, Oct 18, 2023 at 2:30 PM Johanna Öjeling via dev wrote: > When I have contributed to Apache Airflow, they have tagged all > contributors concerned in a GitHub issue when the RC is available and asked > us

Re: [PYTHON] partitioner utilities?

+1 on this idea. Thanks! On Thu, Oct 19, 2023 at 3:40 PM Joey Tran wrote: > Yeah, I already implemented these partitioners for my use case (I just > pasted the classnames/docstrings for them) and I used both combiners.Top > and combiners.Sample. > > In fact, before writing these partitioners I

Re: [VOTE] Release 2.50.0, release candidate #2

+1 Ran the simple Dataflow ML GPU batch job using https://github.com/google/dataflow-ml-starter with Python 2.50.0rc2 to validate the RC works well. On Sat, Aug 26, 2023 at 12:16 AM Valentyn Tymofieiev via dev < dev@beam.apache.org> wrote: > +1 > > Verified that the issue detected in RC0 has

Re: [ANNOUNCE] New committer: Ahmed Abualsaud

Congratulations, Ahmed! On Thu, Aug 24, 2023, 5:49 PM Ahmet Altay via dev wrote: > Hi all, > > Please join me and the rest of the Beam PMC in welcoming a new > committer: Ahmed Abualsaud (ahmedabuals...@apache.org). > > Ahmed has been part of the Beam community since January 2022, working >

Re: [ANNOUNCE] New committer: Anand Inguva

Congratulations, Anand!!! On Fri, Apr 21, 2023 at 2:31 PM Jack McCluskey via dev wrote: > Congratulations, Anand! > > On Fri, Apr 21, 2023 at 2:28 PM Valentyn Tymofieiev via dev < > dev@beam.apache.org> wrote: > >> Congratulations! >> >> On Fri, Apr 21, 2023 at 8:19 PM Jan Lukavský wrote: >>

Re: [ANNOUNCE] New committer: Damon Douglas

Congrats Damon!!! On Mon, Apr 24, 2023 at 4:34 PM Danny McCormick via dev wrote: > Congrats Damon! > > On Mon, Apr 24, 2023 at 4:03 PM Ahmet Altay via dev > wrote: > >> Congratulations Damon! >> >> On Mon, Apr 24, 2023 at 1:00 PM Robert Burke wrote: >> >>> Congratulations Damon!!! >>> >>> On

Re: [beam-starter-typescript]: Missing place to create issue

I believe Robert is the owner for that project. On Mon, Jun 12, 2023 at 11:30 PM david-kh...@hotmail.com < david-kh...@hotmail.com> wrote: > Hi Beam community, > > > > I am David and new to the community. After tried to tweak some code from > beam-starter-ts, I have found some issues and want to

Re: [DISCUSS] Enable Github Discussions?

+1 with GH discussion. If Airflow can do this https://github.com/apache/airflow/discussions, I think we can do this as well. On Mon, Jul 3, 2023 at 9:51 AM Alexey Romanenko wrote: > -1 > I understand that for some people, who maybe are not very familiar with > ASF and its “Apache Way” [1], it

Re: KafkaIO Parameter Issue | Runtime PipelineOptions | Apche Beam

I am not entirely sure what you try to do with PipelineOptions. Assuming you use Beam Java, https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/options/PipelineOptions.html documents how to construct PipelineOptions is via command-line argument parsing.

Propose to add the new security section to the Beam releases

Hi All, We are proposing to explicitly add the security fixes to the Beam release notes. https://github.com/apache/beam/pull/27976 modified the template in CHANGES.md by adding this new section. Please let us know if you have any questions or feel free to comment PR:

Re: [Discuss] Get rid of OWNERS files

+1. On Tue, Aug 8, 2023 at 10:55 AM Danny McCormick via dev wrote: > Hey everyone, I'd like to propose getting rid of OWNERS files from the > Beam repo. Right now, I don't think they are serving a meaningful purpose: > > - Many OWNERS files are outdated and point to people who are no longer >

Re: Introducing beam.MLTransform

Agree with Danny. Thanks for writing this! On Wed, May 10, 2023 at 10:35 AM Danny McCormick via dev < dev@beam.apache.org> wrote: > Thanks Anand! I left a few comments, but overall I think this is a > great/well constructed proposal - this is a good way to quickly bring a > suite of

Re: [PROPOSAL] Preparing for 2.50.0 Release

Welcome to the Beam community! Our release managers usually follow this https://beam.apache.org/contribute/release-guide/#10-vote-and-validate-release-candidate to send the votes out and ask for any feedback regarding the release candidate. If you could help run any validation on your side and

Re: Google Artifact Registry detects critical vuln CVE-2023-45853 in beam dataflow

FYI. The ongoing PR: https://github.com/apache/beam/pull/30011 will switch to the distroless images, which will have less vulnerabilities in the future. On Wed, Jan 24, 2024 at 12:32 PM Valentyn Tymofieiev wrote: > > Does the beam project generally attempt to address as many of these >

Re: Playground: File Explorer?

Can you provide which example you are referring to? I checked a few examples and usually we use beam.Map(print) to display some output values. On Wed, Feb 7, 2024 at 8:55 PM Joey Tran wrote: > Hey all, > > I've been really trying to use Playground for educating new Beam users but > it feels

Re: [ANNOUNCE] New Committer: Svetak Sundhar

Great job, Svetak! Thanks for all your contributions to Beam!!! On Mon, Feb 12, 2024 at 4:44 PM Valentyn Tymofieiev via dev < dev@beam.apache.org> wrote: > Congrats, Svetak! > > On Mon, Feb 12, 2024 at 11:20 AM Kenneth Knowles wrote: > >> Hi all, >> >> Please join me and the rest of the Beam

Re: [VOTE] Release 2.54.0, release candidate #2

+1 (non-binding) Validated with a simple RunInference Python pipeline: https://github.com/google/dataflow-ml-starter/actions/runs/7821639833/job/21339032997 On Wed, Feb 7, 2024 at 7:10 PM Yi Hu via dev wrote: > +1 (non-binding) > > Validated with Dataflow Template: >

Re: [VOTE] Release 2.53.0, release candidate #1

+1 (non-binding) Tested apache-beam[gcp]==2.53.0rc1 with a simple RunInference pipeline using Dataflow Runner: https://github.com/google/dataflow-ml-starter/actions/runs/7296067726/job/19883348692 On Thu, Dec 21, 2023 at 4:30 PM Danny McCormick via dev wrote: > +1 (non-binding) > > Tested

Re: Unable to run - execute java examples

Welcome to Beam! To test your Java environment, you can just run this gradle task: ./gradlew :examples:java:wordCount On Sat, Dec 16, 2023 at 6:30 PM Vineet wrote: > Hi, > > I want to run the beam examples locally , building from source (for > contributing). > Hence after setting all

Re: [VOTE] Release 2.53.0, release candidate #2

+1 (non-binding) Tested with the simple RunInference pipeline: https://github.com/google/dataflow-ml-starter/actions/runs/7332832875/job/19967521369 On Tue, Dec 26, 2023 at 3:29 PM Jack McCluskey via dev wrote: > Happy holidays everyone, > > Please review and vote on the release candidate #2

Re: Setting up beam locally

Assuming you run this under beam/sdks/python, since I use zsh, I have to do this `pip install -e ".[gcp,test]"`. On Fri, Jan 5, 2024 at 1:33 AM G Gautam wrote: > Hi everyone, > > Need help in setting beam locally. > > When trying to setup locally > on entering this command: pip install -e

Re: [VOTE] Release 2.54.0, release candidate #1

+1 validated by running the simple RunInference ML pipeline: https://github.com/google/dataflow-ml-starter/actions/runs/7761835540/job/21171080332 On Fri, Feb 2, 2024 at 4:10 PM Robert Burke wrote: > Hi everyone, > Please review and vote on the release candidate #1 for the version 2.54.0, > as

Re: Beam 2.52.0 Release

Thanks a lot! Great job, Team! On Fri, Nov 17, 2023 at 7:21 PM Danny McCormick via user < u...@beam.apache.org> wrote: > I am happy to announce that the 2.52.0 release of Beam has been finalized. > This release includes both improvements and new functionality. > > For more information on changes

Re: [VOTE] Release 2.52.0, release candidate #5

+1 (non binding) Tested the Python SDK RC5 using the ML pipeline under https://github.com/google/dataflow-ml-starter https://github.com/google/dataflow-ml-starter/actions/runs/6898545809/job/18768732434 ran well. On Thu, Nov 16, 2023 at 7:46 PM Robert Bradshaw via dev wrote: > +1 (binding) > >

Re: Issue building python SDK with M2 Mac

I do not have the problem to do this on my M1 by cloning the repo and using conda to create venv with python 3.11 and installing it with pip install -e ".[gcp,test]". It installs numpy with 1.26.4. On Thu, Mar 7, 2024 at 7:48 AM Joey Tran wrote: > Hey all, > > I'm trying to get a beam python

Re: Update confluent dependencies version in kafka io

This sounds great! Feel free to create an issue to track this work! On Mon, Mar 11, 2024 at 9:36 AM Maciej Szwaja via dev wrote: > Hi, > > This is my first email to this list, hello everyone! > > I have a question regarding the confluent version dependency that beam, or > more specifically

Re: [ACTION REQUESTED] Help me draft the Beam Board Report for March 2024

Thanks for the ping! I added several notes and feel free to make more changes. On Mon, Mar 11, 2024 at 2:49 PM Kenneth Knowles wrote: > Ping! > > Would really love help from folks building stuff to report out on what > they've built, especially! > > Kenn > > On Tue, Mar 5, 2024 at 12:15 PM

Re: [VOTE] Release 2.55.0, release candidate #2

OK. +1 (non-bindg). Tested the simple Dataflow ML pipeline: https://github.com/google/dataflow-ml-starter/actions/runs/8308559154 On Mon, Mar 18, 2024 at 10:26 AM Yi Hu wrote: > Yes, there is only one cherry-pick compared to RC1 (#30637) regarding a > dependency change, not all artifacts are

Re: [VOTE] Release 2.55.0, release candidate #2

I do not see RC2 for Python SDK here: https://pypi.org/project/apache-beam/#history On Fri, Mar 15, 2024 at 10:54 AM Yi Hu via dev wrote: > +1 (non-binding). > > Tested with https://github.com/GoogleCloudPlatform/DataflowTemplates > (Java SDK 11, Dataflow runner, details: >

Re: [VOTE] Release 2.55.0, release candidate #3

+1 (non-binding). Ran the simple ML pipeline without any issue: https://github.com/google/dataflow-ml-starter/actions/runs/8349158153 On Tue, Mar 19, 2024 at 11:55 AM Ritesh Ghorse via dev wrote: > +1 (non-binding) - Ran a few python batch examples on Direct and Dataflow > runner. > > Thanks! >

Re: [VOTE] Patch Release 2.55.1, release candidate #2

+1 (non-binding). Tested this using a simple Dataflow ML pipeline: https://github.com/google/dataflow-ml-starter/actions/runs/8541848483. On Wed, Apr 3, 2024 at 2:35 PM Jeff Kinard wrote: > +1. Validated running from local gradle JAR and staged maven JAR for > expansion-service. > > On Wed, Apr

Re: tox issues in dev container

> wrote: > >> Yeah that was the tox command I was running >> >> On Fri, Apr 5, 2024, 4:37 PM XQ Hu via dev wrote: >> >>> >>> https://cwiki.apache.org/confluence/display/BEAM/Python+Tips#PythonTips-LintandFormattingChecks >>> >>> This gene

Re: tox issues in dev container

https://cwiki.apache.org/confluence/display/BEAM/Python+Tips#PythonTips-LintandFormattingChecks This generally works well. Have you checked this? On Fri, Apr 5, 2024 at 4:07 PM Joey Tran wrote: > I think I might be doing something silly with my environment. > > I'm trying to lint using tox in

Re: [VOTE] Release 2.56.0, release candidate #1

+1 (non binding). Tested the simple Dataflow ML job: https://github.com/google/dataflow-ml-starter/actions/runs/8824985423/job/24228468173 On Wed, Apr 24, 2024 at 2:01 PM Danny McCormick via dev wrote: > Hi everyone, > Please review and vote on the release candidate #1 for the version 2.56.0, >

Re: [VOTE] Release 2.56.0, release candidate #2

+1 (non-binding). Tested it using the dataflow ML pipeline: https://github.com/google/dataflow-ml-starter/actions/runs/8862170843/job/24334816481 On Sat, Apr 27, 2024 at 7:42 AM Danny McCormick via dev wrote: > Hi everyone, > Please review and vote on the release candidate #2 for the version

Re: Beam + Google Summer of Code 2024

Welcome to Beam! On Wed, May 1, 2024 at 4:13 PM Danny McCormick via dev wrote: > Hey everyone, > > It's my pleasure to announce 2 contributors have been accepted as GSoC > students for Beam this year! > > Ayush Pandey will be working on a project to implement RAG example > pipelines using Beam

Re: Chronon + Beam

Welcome to Beam! Let us know if you have any questions when contributing to Beam! On Tue, May 7, 2024 at 7:05 PM Varant Zanoyan wrote: > Hi Beam community, > > Nikhil and I are two of the authors of the Chronon > project, maintained by Airbnb and Stripe. We're >