Re: [Proposal] Enable EnricoMi/publish-unit-test-result-action

+1, this is helpful. We had a similar situation with DataflowTemplates . Even though we used a different repository (mikepenz/action-junit-report ), this strategy was invaluable to reduce

Re: [VOTE] Release 2.51.0, release candidate #1

+1 (non-binding). Tested with https://github.com/GoogleCloudPlatform/DataflowTemplates (Java SDK 11, Dataflow Runner using both legacy and v2). Thanks Kenn! On Wed, Oct 4, 2023 at 3:03 PM Robert Bradshaw via dev wrote: > +1 (binding) > > Verified artifacts and signatures and tested a simple

Re: Apache Beam 2.50.0 - org.apache.beam.sdk.options.MemoryMonitorOptions ClassDef Not found

Hi Jaymik, This class (MemoryMonitorOptions) was indeed introduced in Beam 2.50.0. My guess is that somehow dependencies were upgraded only partially (for example, just the Runner module, but not Core). A good way to make sure that all dependencies are upgraded correctly is to use a BOM (Bill of

Re: [VOTE] Release 2.50.0, release candidate #1

+1 (non-binding). Tested with https://github.com/GoogleCloudPlatform/DataflowTemplates (Java SDK 11, Dataflow runner). Thanks Robert! On Thu, Aug 17, 2023 at 8:58 PM Robert Burke wrote: > Hi everyone, > Please review and vote on the release candidate #1 for the version 2.50.0, > as follows: >

Re: [ANNOUNCE] New committer: Ahmed Abualsaud

Congratulations, Ahmed! Very well deserved! On Thu, Aug 24, 2023 at 6:09 PM XQ Hu via dev wrote: > Congratulations, Ahmed! > > On Thu, Aug 24, 2023, 5:49 PM Ahmet Altay via dev > wrote: > >> Hi all, >> >> Please join me and the rest of the Beam PMC in welcoming a new >> committer: Ahmed

Re: [VOTE] Release 2.50.0, release candidate #2

+1 (non-binding). Tested with https://github.com/GoogleCloudPlatform/DataflowTemplates (Java SDK 11, Dataflow runner). Thanks Robert! On Thu, Aug 24, 2023 at 7:12 PM Robert Burke wrote: > Two minor erata from the previous email: > > The validation spreadsheet link should be: > >

Re: [VOTE] Release 2.44.0, release candidate #1

+1 (non-binding) Tested with https://github.com/GoogleCloudPlatform/DataflowTemplates (Java SDK 11, Dataflow runner). Thanks! On Wed, Jan 11, 2023 at 11:08 AM Alexey Romanenko wrote: > +1 (binding) > > Tested with https://github.com/Talend/beam-samples/ > (Java SDK v8/v11/v17, Spark 3

Re: [VOTE] Release 2.46.0, release candidate #1

+1 (non-binding) Tested with https://github.com/GoogleCloudPlatform/DataflowTemplates (Java SDK 11, Dataflow runner). Thanks Danny! On Thu, Mar 2, 2023 at 5:16 PM Danny McCormick via dev wrote: > Hi everyone, > Please review and vote on release candidate #1 for the version 2.46.0, as >

Re: Beam Website Feedback

Hi Shlomi, Unfortunately, those cited references are about as much as we have available. I acknowledge that they are not very comprehensive -- so I'll try to share some insight. Related to your sample, I believe there are relevant missing pieces, as I am note sure what the input looks like

Re: [VOTE] Release 2.45.0, Release Candidate #1

+1 (non-binding) Tested with https://github.com/GoogleCloudPlatform/DataflowTemplates (Java SDK 11, Dataflow runner). Thanks! On Mon, Feb 13, 2023 at 1:13 AM Chamikara Jayalath via dev < dev@beam.apache.org> wrote: > +1 (binding) > > Tried several Java and Python multi-language pipelines. > >

Re: Launch Dataflow Flex Templates from Go

Hello, The most similar to the mentioned Python APIs is through https://github.com/googleapis/google-api-go-client. There are some docs that include Go examples: - Flex Templates:

Re: Beam Website Feedback

Hi Julian, That's a good point. I think the focus of https://beam.apache.org/documentation/pipelines/test-your-pipeline/ is showing how to test Beam, but probably linking to working examples would be helpful to reduce friction. Related to your code, I believe you are missing 2 imports: import

Re: [VOTE] Release 2.49.0, release candidate #2

+1 (non-binding). Tested with https://github.com/GoogleCloudPlatform/DataflowTemplates (Java SDK 11, Dataflow runner). Thanks Yi! On Tue, Jul 11, 2023 at 4:23 PM Yi Hu via dev wrote: > Hi everyone, > Please review and vote on the release candidate #2 for the version 2.49.0, > as follows: > [

Re: [VOTE] Release 2.49.0, release candidate #1

+1 (non-binding). Tested with https://github.com/GoogleCloudPlatform/DataflowTemplates (Java SDK 11, Dataflow runner). Thanks Yi! On Fri, Jul 7, 2023 at 10:51 AM Alexey Romanenko wrote: > +1 (binding) > > Tested with https://github.com/Talend/beam-samples/actions > (Java SDK v8/v11/v17, Spark

Re: [VOTE] Release 2.48.0 release candidate #2

I was able to check that containers are all there and complete my validation. +1 (non-binding). Tested with https://github.com/GoogleCloudPlatform/DataflowTemplates (Java SDK 11, Dataflow runner). Thanks Ritesh and Danny! On Fri, May 26, 2023 at 10:09 AM Danny McCormick via dev <

Re: KafkaIO Parameter Issue | Runtime PipelineOptions | Apche Beam

Hi Himanshu, I see that Ferran Fernández Garrido provided a very good suggestion. For some reason/glitch, I guess you haven't received it. Please see it in the mail list archive https://lists.apache.org/thread/96vz0pdbc6t159vt82ns90qpft1w9d2w On Thu, Aug 10, 2023 at 11:37 AM himanshu singhal <

Re: Beam IO Connector

Hi Jeremy, Apparently you are trying to use Beam's DirectRunner , which is mostly focused on small pipelines / testing purposes. Even if it runs in the JVM, there are protections in place to make sure your pipeline will be able to be

Re: Ensuring a task does not get executed concurrently

Hi Stephan, I am not sure if this is the best way to achieve this, but I've seen parallelism being limited by using state / KV and limiting the number of keys. In your case, you could have the same key for both non concurrency-safe operations and when using state, the Beam model will guarantee

Re: [VOTE] Release 2.47.0, release candidate #3

A little bit late to this thread since I've seen the voting closed email, but +1 (non-binding) from me as well. Tested with https://github.com/GoogleCloudPlatform/DataflowTemplates (Java SDK 11, Dataflow runner). Thanks Jack! On Wed, May 10, 2023 at 10:38 AM Valentyn Tymofieiev via dev <

Re: [VOTE] Release 2.53.0, release candidate #1

-1 Had issues using 2.53.0 rc1 on https://github.com/GoogleCloudPlatform/DataflowTemplates: DataflowTemplates/pull/1233/checks?check_run_id=19897484522 This seems to be the root cause:

Re: [VOTE] Release 2.53.0, release candidate #2

+1 (non-binding). Tested with https://github.com/GoogleCloudPlatform/DataflowTemplates (Java SDK 11, Dataflow runner). Thanks Jack! On Wed, Dec 27, 2023 at 8:13 AM Danny McCormick via dev wrote: > +1 (non-binding) > > Tested with some example ML notebooks. > > Thanks, > Danny > > On Tue, Dec

Re: [VOTE] Release 2.52.0, release candidate #5

+1 (non-binding). Tested with https://github.com/GoogleCloudPlatform/DataflowTemplates (Java SDK 11, Dataflow runner). Thanks Danny! On Mon, Nov 13, 2023 at 6:07 PM Danny McCormick via dev wrote: > Hi everyone, > Please review and vote on the release candidate #5 for the version 2.52.0, > as

Re: [VOTE] Release 2.52.0, release candidate #4

I'm having some problems validating the RCs proposed here (both 3 and 4). User code that depends on versions newer than Avro 1.8.2 are having problems running on Dataflow. > Caused by: java.io.InvalidClassException: org.apache.avro.specific.SpecificRecordBase; local class incompatible: stream