Re: [VOTE] Release 2.57.0, release candidate #1

+1 (binding) Regards JB On Fri, Jun 21, 2024 at 10:17 PM Kenneth Knowles wrote: > > Hi everyone, > > Please review and vote on the release candidate #1 for the version 2.57.0, as > follows: > > [ ] +1, Approve the release > [ ] -1, Do not approve the release (please provide specific comments)

Re: [VOTE] Release 2.57.0, release candidate #1

+1 (non-binding). Tested this with a simple Dataflow ML pipeline with Python: https://github.com/google/dataflow-ml-starter/actions/runs/9619553472/job/26536415002 On Fri, Jun 21, 2024 at 4:18 PM Kenneth Knowles wrote: > Hi everyone, > > Please review and vote on the release candidate #1 for

[VOTE] Release 2.57.0, release candidate #1

Hi everyone, Please review and vote on the release candidate #1 for the version 2.57.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) Reviewers are encouraged to test their own use cases with the release candidate, and vote +1 if

Re: Getting started with Apache Beam Quest

Hi Ronobir, The quest is free now; could you navigate to https://www.cloudskillsboost.google/catalog?qlcampaign=3l-event-90, and search "Getting Started with Apache Beam" and try? Let me know if you run into any issues, Svetak Sundhar Data Engineer s vetaksund...@google.com On Tue, Jun

Beam High Priority Issue Report (50)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31432 The PreCommit Java

Beam High Priority Issue Report (51)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31432 The PreCommit Java

Beam High Priority Issue Report (51)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31432 The PreCommit Java

Beam High Priority Issue Report (52)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31432 The PreCommit Java

Beam High Priority Issue Report (51)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31432 The PreCommit Java

Beam High Priority Issue Report (50)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31432 The PreCommit Java

Last day to submit your CFP for the upcoming Beam Summit! 

We have exciting news for the Apache Beam Community!  The CFP is Extended for 1 More Day! We listened to your feedback and extended the Call for Proposal submission deadline for the upcoming Beam Summit to end-of-day tomorrow! Please fine-tune your ideas and contribute to our vibrant community.

Re: Beam Infrastructure - Highlights of Recent Changes

Thanks for the update. Lots of great stuff here! On Thu, Jun 13, 2024 at 8:27 AM Andrey Devyatkin via dev wrote: > > Hey Beam community, > > We are glad to announce some new changes that our team has been working on > for a while to implement new solutions and enhance the existing ones. The >

Beam Infrastructure - Highlights of Recent Changes

Hey Beam community, We are glad to announce some new changes that our team has been working on for a while to implement new solutions and enhance the existing ones. The main focus was to improve the Beam infrastructure by increasing test coverage, adding a reporting mechanism, and enhancing

Beam High Priority Issue Report (50)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31432 The PreCommit Java

Beam High Priority Issue Report (50)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31432 The PreCommit Java

Re: Getting started with Apache Beam Quest

Hi Ronobir, We're working on enabling the free code again. I'll follow up again here once it's enabled. Thanks for your patience! Svetak Sundhar Data Engineer s vetaksund...@google.com On Thu, May 30, 2024 at 4:57 PM Ronobir Das wrote: > Hey All, > > Are you guys still running the

Re: [QUESTION] Add Company logo

Hi, Thank you for the message and consideration. There is no requirement beyond using Beam. Feel free to add it there. To state the obvious: please keep the description neutral (not advertisement like) and link to the main company page (without ads, special trackers etc.) In the future, we

Beam High Priority Issue Report (54)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31432 The PreCommit Java

[QUESTION] Add Company logo

Hello, Beam users! I've just started working with Apache Beam at my company and would love to add a company logo, like the one found at https://github.com/apache/beam/blob/master/website/ADD_LOGO.md beam/website/ADD_LOGO.md at master · apache/beam github.com Does anyone know if there are any

Beam High Priority Issue Report (54)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31432 The PreCommit Java

Beam High Priority Issue Report (57)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31432 The PreCommit Java

Re: University of Innsbruck - Apache Beam options

Hi Stefan, the settings in the runner works differently than how you might expect. According to the Beam model, a "bundle" is a unit of work, that either completes successfully as a whole or is restarted from scratch (i.e. it is perfectly fine for the bundle to be processed to the middle,

Re: University of Innsbruck - Apache Beam options

I sent you an invite for Slack. Welcome to the Beam community. On Thu, Jun 6, 2024 at 10:47 AM Pedratscher Stefan < stefan.pedratsc...@student.uibk.ac.at> wrote: > Dear Apache Beam Team, > > I am a student at the University of Innsbruck, currently using Apache > Beam for my research. > > I am

University of Innsbruck - Apache Beam options

Dear Apache Beam Team, I am a student at the University of Innsbruck, currently using Apache Beam for my research. I am using Flink v1.19 and the Flink runner (https://beam.apache.org/documentation/runners/flink/) to execute pipelines. I managed to run the pipeline with several options:

Beam High Priority Issue Report (58)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31432 The PreCommit Java

Re: [PROPOSAL] Preparing for 2.57.0 Release

Hi all, I delayed a week but have now cut the 2.57.0 branch. Kenn On Thu, May 16, 2024 at 8:41 PM Ahmet Altay via dev wrote: > Thank you Kenn! > > On Wed, May 15, 2024 at 8:00 AM Kenneth Knowles wrote: > >> Hi everyone, >> >> The next release (2.57.0) branch cut is scheduled on May 29th,

Beam High Priority Issue Report (57)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31475 The PostCommit Go

Beam High Priority Issue Report (58)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31475 The PostCommit Go

Re: Joining Apache Beam Slack

Thanks! On Jun 3, 2024 at 10:38:05 AM, Ahmet Altay wrote: > Sent an invite. Welcome to the Beam community! > > On Fri, May 31, 2024 at 6:00 PM Ben Warren wrote: > >> Hello Apache Beam team, >> >> I’m a startup founder using Apache Beam, and I’d like to stay up-to-date >> with Apache Beam

Re: Joining Apache Beam Slack

Sent an invite. Welcome to the Beam community! On Fri, May 31, 2024 at 6:00 PM Ben Warren wrote: > Hello Apache Beam team, > > I’m a startup founder using Apache Beam, and I’d like to stay up-to-date > with Apache Beam developments. Can I join the ASF Slack #beam channel? I > don’t have an

Re: design docs that get deleted, etc

I like the idea of a "final" step of exporting the Google Doc to markdown and checking it in, if there is any tool that does an OK job (my experience with such tools over my lifetime has been underwhelming, but I'm always open to being pleasantly surprised). PDF would be a close second. Kenn On

Beam High Priority Issue Report (60)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31475 The PostCommit Go

Joining Apache Beam Slack

Hello Apache Beam team, I’m a startup founder using Apache Beam, and I’d like to stay up-to-date with Apache Beam developments. Can I join the ASF Slack #beam channel? I don’t have an @apache email, so I’m not able to add myself. My email is b...@snowpilot.com Thanks, Ben

Beam High Priority Issue Report (59)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31432 The PreCommit Java

Getting started with Apache Beam Quest

Hey All, Are you guys still running the Apache Beam Quest promotion for a free access code? I'm just starting to learn about Dataflow and Beam and while taking the Coursera course series on Dataflow I was looking at Youtube materials on Apache Beam. I saw the Apache Beam youtube channel and the

Re: Question: Java Apache Beam, mock external Clients initialized in Setup

A recurring pattern, although not uniformly named, for I/O connectors is to provide a method on the PTransform named something along the lines of withClient, withClientFactory, withClientBuilderFactory, withClientProvider, etc. which stores the configuration the client is to be instantiated with

Beam High Priority Issue Report (59)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31432 The PreCommit Java

Re: design docs that get deleted, etc

Yes, I think that the discoverability and durability would be part of some "process". There could be "informal" part, where the design can take place virtually anywhere and the document can be owned by anyone (as we do today), then it would be copied to a durable store (asf wiki?) and a

Re: design docs that get deleted, etc

On Wed, May 29, 2024 at 1:01 PM Robert Burke wrote: > > Honestly, it's less about how we propose things and more to do with actually > making "completing" the documentation work to some permanent place. > > That can be as simple as "migrate the final implemented design to a markdown > file in

Re: design docs that get deleted, etc

Honestly, it's less about how we propose things and more to do with actually making "completing" the documentation work to some permanent place. That can be as simple as "migrate the final implemented design to a markdown file in the GitHub repo at *foo* or published on the beam website at *bar*.

APACHE BEAM SUPPORT

Dear team, I am currently using Apache Beam to read messages from Kafka with a Protobuf schema and a schema registry URL, but I am still encountering errors. Could you please help check my use of the Beam SDK for this case? Below attachment is my code. package org.example; import

A native Beam connector to read from Solace message queue

Hi all, I've been recently working on a Java connector that reads messages from Solace . Please take a look at the design doc and feel free to leave comments and suggestions: https://docs.google.com/document/d/1Gvq67VrcHCnlO8f_NzMM1Y4c7wCNSdvo6qqLWg8upfw/ Related Feature

Re: design docs that get deleted, etc

While I'm not opposed to having a more formal process for proposals to go from idea to consensus to implementation, I'm not sure how much this would solve the primary issues we face (discoverability and durability). But maybe that could be built into the process? At the very least we could have an

Re: [ACTION REQUESTED] Help me draft the Beam Board Report for June 2024

Thanks a lot for doing this for Beam. I added a few notes and feel free to change them. On Thu, May 23, 2024 at 2:57 PM Kenneth Knowles wrote: > The next Beam board report is due next Wednesday, June 12. Please draft it > together at https://s.apache.org/beam-draft-report-2024-06 > > The doc is

Re: Image Embedding Support for MLTransform

It would help if I actually included the link to the new doc! https://docs.google.com/document/d/1n0ZyXcRjxqIb5f-wZcMBXdrkgjEfZHYu-zf8bpzE0dA/edit?usp=sharing On Wed, May 29, 2024 at 10:26 AM Jack McCluskey wrote: > Hey everyone, > > I've put together a very small overview doc on what steps

Image Embedding Support for MLTransform

Hey everyone, I've put together a very small overview doc on what steps need to be taken to add support for generating image embeddings to the Python SDK's MLTransform suite. It's mostly organized as a reference for me while I get familiar with the framework and is relatively general, but it may

Re: design docs that get deleted, etc

Hi, regarding changing the way we document past (and more importantly future) changes, I've always been a big fan of the FLIP analogy [1]. I would love if we could make this work for Beam as well, while preserving the 'informal' part that I believe all of us want to keep. On the other hand,

design docs that get deleted, etc

Hi all, Yesterday someone asked me about the design doc linked from https://github.com/apache/beam/issues/18297 because it is now a 404. There are plenty of reasons a Google Doc might no longer be accessible. They exist outside the project's control. This is part of why ASF projects emphasize

Beam High Priority Issue Report (60)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31432 The PreCommit Java

Re: Question: Java Apache Beam, mock external Clients initialized in Setup

You could let your @Setup method initialize your serviceAPI if (and only if) it's null. For your tests, pre-initialize it with a Fake, and otherwise the real thing will get used. On Sat, May 25, 2024 at 1:27 PM Ritwik Dutta via dev wrote: > > Hi Hu, > > Thank you, > I am testing the serviceAPI

Beam High Priority Issue Report (56)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31418 The PostCommit XVR

Beam High Priority Issue Report (56)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31346 The PreCommit Java

Re: Question: Java Apache Beam, mock external Clients initialized in Setup

Hi Hu, Thank you,I am testing the serviceAPI  call in separate unit tests already. Let me rephrase  (serviceAPI should be renamed to serviceLayer,  it doesn't necessarily have to be a restful API with an http endpoint)The question is how, do we create Fakes for items with difficult

Re: default_sdk_harness_log_level multi-language support

Feel free to open a github issue to track this idea. For Dataflow, the potential workaround is to filter the logs: https://cloud.google.com/dataflow/docs/guides/filter-logs On Fri, May 24, 2024 at 5:01 AM Ferran Fernández Garrido < ffernandez@gmail.com> wrote: > Hi! > > I am trying to adjust

Re: Question: Java Apache Beam, mock external Clients initialized in Setup

I am not sure which part you want to test. If the processData part should be tested, you could refactor the code without use any Beam specific code and test the processing data logic. >From your example, it seems that you are calling some APIs, we recently added a new Web API IO:

Beam High Priority Issue Report (53)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31346 The PreCommit Java

default_sdk_harness_log_level multi-language support

Hi! I am trying to adjust the log level for a Beam YAML pipeline that uses Kafka. Since Kafka is a multi-language transform, default_sdk_harness_log_level is not currently supported, which means I can't limit the logs (Kafka can be a bit too verbose sometimes). This could lead to increased

[ACTION REQUESTED] Help me draft the Beam Board Report for June 2024

The next Beam board report is due next Wednesday, June 12. Please draft it together at https://s.apache.org/beam-draft-report-2024-06 The doc is open for anyone to edit. Ideas: - highlights from CHANGES.md - interesting technical discussions - integrations with other projects - community

Beam Summit 2024 - Registration is open!

[image: Beam Summit 2024] *Dear Beam Community,* Join the global Apache Beam community in Sunnyvale, USA from September 4-5th. Don't miss the opportunity to submit a talk proposal and connect with data processing experts! *CFP is closing soon!*  We're eager to hear

Beam High Priority Issue Report (53)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31346 The PreCommit Java

Re: Question: Java Apache Beam, mock external Clients initialized in Setup

any response yet? No one has answers? I left a stackoverflow bounty on the question Using external methods is pretty important On Sunday, May 12, 2024 at 11:52:25 AM PDT, Ritwik Dutta wrote: Hi, I wrote the following question here.It would be really helpful also, if you can also

Flink 1.18 quickstart issues

Hi, currently our documentation for running quickstart examples uses maven and `exec:java` mojo. Unfortunately, there is a (not yet confirmed, but likely, see [1], [2]) Flink bug which causes Flink runner 1.18 to fail. The bug causes MiniCluster (which is used in local testing) use wrong

Beam High Priority Issue Report (52)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31346 The PreCommit Java

Beam High Priority Issue Report (56)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31346 The PreCommit Java

Beam High Priority Issue Report (56)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31297 [Failing Test]:

Re: Python apache beam bigquery temporary tables

I think you hit this issue: https://stackoverflow.com/questions/64526500/using-writetobigquery-file-loads-in-a-streaming-pipeline-just-creates-a-lot-of-t I created https://github.com/apache/beam/issues/31329 to track this bug. Some workarounds: 1.

Beam High Priority Issue Report (58)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31297 [Failing Test]:

Re: [PROPOSAL] Preparing for 2.57.0 Release

Thank you Kenn! On Wed, May 15, 2024 at 8:00 AM Kenneth Knowles wrote: > Hi everyone, > > The next release (2.57.0) branch cut is scheduled on May 29th, 2024, > according to the release calendar [1]. > > I volunteer to perform this release. My plan is to cut the branch on that > date, and

Python apache beam bigquery temporary tables

Hi, I'm working on a pipeline that gets data from pub/sub and splits the data into a few big query tables. I have a problem with temporary tables because the pipeline creates the temp tables (I have to compute table id and schema based on data) but it does not delete all of them. Is there any

KafkaIO/FixedWindow changes 2.56?

Hi all, I have a project running with Beam 2.51, using Flink runner. In one of my pipelines i have a FixedWindow and had a problem upgrading until now, with a timers issue now resolved, and hopefully allowing me to upgrade to version 2.56 However, I encounter another problem now which I believe is

Beam High Priority Issue Report (59)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31297 [Failing Test]:

[PROPOSAL] Preparing for 2.57.0 Release

Hi everyone, The next release (2.57.0) branch cut is scheduled on May 29th, 2024, according to the release calendar [1]. I volunteer to perform this release. My plan is to cut the branch on that date, and cherrypick release-blocking fixes afterwards, if any. Please help me make sure the release

Beam High Priority Issue Report (57)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31297 [Failing Test]:

Community over Code EU 2024: The countdown has started!

[Note: You're receiving this email because you are subscribed to one or more project dev@ mailing lists at the Apache Software Foundation.] We are very close to Community Over Code EU -- check out the amazing program and the special discounts that we have for you. Special discounts You still

Beam High Priority Issue Report (55)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31254 [Failing Test]:

Re: Beam + Google Summer of Code 2024

Welcome! It is great to have new and returning contributors working on exciting projects! On Thu, May 2, 2024 at 9:35 AM Svetak Sundhar via dev wrote: > Welcome and welcome (back)! > > > Svetak Sundhar > > Data Engineer > s vetaksund...@google.com > > > > On Thu, May 2, 2024 at 2:06 AM Reeba

Beam High Priority Issue Report (55)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31254 [Failing Test]:

Beam High Priority Issue Report (54)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31245 The IcebergIO Unit

Beam High Priority Issue Report (53)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31122 The PostCommit Go

Re: Chronon + Beam

Welcome to Beam! Let us know if you have any questions when contributing to Beam! On Tue, May 7, 2024 at 7:05 PM Varant Zanoyan wrote: > Hi Beam community, > > Nikhil and I are two of the authors of the Chronon > project, maintained by Airbnb and Stripe. We're >

Beam High Priority Issue Report (55)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31122 The PostCommit Go

Chronon + Beam

Hi Beam community, Nikhil and I are two of the authors of the Chronon project, maintained by Airbnb and Stripe. We're planning on adding an Enrichment Handler for Beam to Chronon soon, so we wanted to reach out and see if anyone in this community would be interested in

Re: Job execution starts very slowly on Flink server.

This sounds about right to me. When I was working on getting flink running this was one of the biggest pain points, the upload/download process is painfully slow. We had implemented a jar cache in flink to improve this (and stopped using fat jars)

Job execution starts very slowly on Flink server.

Hi, We are switching from embedded flink runner to filink server. The embedded flink runner works fine, jobs start instantly. However when we use the flink server we are experiencing about 30 seconds of idle time (no significant cpu, io load) before we see the job in the flink. We are targeting at

Beam High Priority Issue Report (54)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31122 The PostCommit Go

Beam High Priority Issue Report (54)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31122 The PostCommit Go

Re: PCollection#applyWindowingStrategyInternal

Right. Generally speaking, merging of state would require some modifications to the model to support at least some hooks. +1 On 4/25/24 18:03, Reuven Lax via dev wrote: I think there is more to it than that. You'll probably want retractions integrated into Beam's core triggering. One example

SpannerIO Bounded Connector

Hello, I want to implement a SpannerIO connector using bounded PCollections, to optimize execution cost for high latency tolerated scenarios, do you know, if anyone has already tried or is actively working on something similar? Thank you, Dmytro

Beam High Priority Issue Report (55)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31161 [Failing Test]:

Re: Beam + Google Summer of Code 2024

Welcome and welcome (back)! Svetak Sundhar Data Engineer s vetaksund...@google.com On Thu, May 2, 2024 at 2:06 AM Reeba Qureshi wrote: > Hello everyone! > > I'm really excited to be working with Apache Beam again! Looking forward > to it! > > Thanks, > Reeba > > On Thu, 2 May, 2024,

Beam High Priority Issue Report (54)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31122 The PostCommit Go

Re: Beam + Google Summer of Code 2024

Hello everyone! I'm really excited to be working with Apache Beam again! Looking forward to it! Thanks, Reeba On Thu, 2 May, 2024, 10:04 Ayush Pandey, wrote: > Hi Danny, > > Thank you for the kind introduction. I really look forward to > collaborating with and learning from this amazing

Re: Beam + Google Summer of Code 2024

Hi Danny, Thank you for the kind introduction. I really look forward to collaborating with and learning from this amazing community. Best Regards, Ayush On Wed, 1 May 2024 at 14:40, XQ Hu wrote: > Welcome to Beam! > > On Wed, May 1, 2024 at 4:13 PM Danny McCormick via dev < >

Re: [PR] Publish docs for 2.56.0 release [beam-site]

damccorm merged PR #665: URL: https://github.com/apache/beam-site/pull/665 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[RESULT] [VOTE] Release 2.56.0, release candidate #2

I'm happy to announce that we have unanimously approved this release. There are 7 approving votes, 3 of which are binding: * Jan Lukavský (binding) * Chamikara Jayalath (binding) * Valentyn Tymofieiev (binding) * XQ Hu * Yi Hu * Jeff Kinard * Ritesh Ghorse There are no disapproving votes. I will

Re: Beam + Google Summer of Code 2024

Welcome to Beam! On Wed, May 1, 2024 at 4:13 PM Danny McCormick via dev wrote: > Hey everyone, > > It's my pleasure to announce 2 contributors have been accepted as GSoC > students for Beam this year! > > Ayush Pandey will be working on a project to implement RAG example > pipelines using Beam

Beam + Google Summer of Code 2024

Hey everyone, It's my pleasure to announce 2 contributors have been accepted as GSoC students for Beam this year! Ayush Pandey will be working on a project to implement RAG example pipelines using Beam [1]. This will be a really valuable addition to Beam's ML offering, showing how users can

Re: [VOTE] Release 2.56.0, release candidate #2

Thanks, Danny. +1 Verified Python pipelines on Dataflow, checked that released containers include fixes for a regression in GRPC IO that affected last release. On Wed, May 1, 2024 at 11:30 AM Danny McCormick wrote: > I added https://github.com/apache/beam/pull/31038#issuecomment-2088879104 >

Re: [VOTE] Release 2.56.0, release candidate #2

I added https://github.com/apache/beam/pull/31038#issuecomment-2088879104 triaging the issues. The JPMS ones do seem to have been from the Dataflow containers not being published, they succeeded on retry. The remaining issues generally seem to be flakes or infrastructure issues. Thanks, Danny On

Re: [VOTE] Release 2.56.0, release candidate #2

I'll go through and triage/rekick off some Dataflow suites and summarize the results in the PR. The JPMS ones (and possibly others) might be because I hadn't published Dataflow containers yet when I ran them, the log isn't helpful so I'll try to capture the actual job which failed. Thanks, Danny

  1   2   3   4   5   6   7   8   9   10   >