[no subject]

2023-02-01 Thread Martin Chi
Hi, I want to subscribe it.

Re: [DISCUSS] Migrate Flink runner to run batch jobs in DataStream API

2023-02-01 Thread Becket Qin
Thanks Robert. I am going to create the GitHub issues and PRs. If there are further questions/concerns raised later, we can address them here. Cheers, Jiangjie (Becket) Qin On Thu, Feb 2, 2023 at 8:39 AM Robert Bradshaw wrote: > Thanks. In that case keeping both in parallel, and tying the

Re: [fyi] Updating ip address for Playground staging

2023-02-01 Thread Austin Bennett
[ Happy New Year ] Pablo, I imagine there shouldn't be an issue with changing regions, esp. in a testing environment ( verifying this is actually testing, which is distinct from production as served from an account with a 'testing' name :-) ). Though, it seems likely that you'll also want to

Beam CassandraIO

2023-02-01 Thread Enzo Bonggio
I have a spark standalone installed in two machines but once I send spark-submit, it will only execute in one executer. Is that the way that it suppose to work? I thought that I could read from Cassandra with multiple machines

Re: [DISCUSS] Migrate Flink runner to run batch jobs in DataStream API

2023-02-01 Thread Becket Qin
Hi Robert, Thanks for the feedback. This change will be transparent to the user applications in most cases. However, there are still a few differences visible to the users. 1. Configurations. DataStream and DataSet take different configurations. 2. Metrics. DataStream operators and DataSet

Re:

2023-02-01 Thread Anand Inguva via dev
Hi, You can send an email to dev-subscr...@beam.apache.org instead to subscribe to the dev list. Thanks, Anand On Wed, Feb 1, 2023 at 6:52 PM Anand Inguva wrote: > Hi, > > You can send an email to dev-subscr...@beam.apache.org instead to > subscribe the dev list. > > Thanks, > Anand > > On

Re:

2023-02-01 Thread Anand Inguva via dev
Hi, You can send an email to dev-subscr...@beam.apache.org instead to subscribe the dev list. Thanks, Anand On Wed, Feb 1, 2023 at 6:41 PM Martin Chi wrote: > Hi, > > I want to subscribe it. >

[fyi] Updating ip address for Playground staging

2023-02-01 Thread Pablo Estrada via dev
Hi all, this email is just to inform contributors that due to resource constraints in the apache-beam-testing project, we want to update the IP address for the staging environment of the Beam playground from a us-west one to a us-east one. If you have any concerns about this, please reach out to

Re: [DISCUSS] Migrate Flink runner to run batch jobs in DataStream API

2023-02-01 Thread Robert Bradshaw via dev
Thanks. In that case keeping both in parallel, and tying the switch in the default to a (possibly overridable) choice of Flink version, makes a lot of sense. On Wed, Feb 1, 2023 at 3:33 PM Becket Qin wrote: > > Hi Robert, > > Thanks for the feedback. This change will be transparent to the user

[GitHub] [beam-site] robertwb opened a new pull request, #639: Update beam-site for release 2.44.0.

2023-02-01 Thread via GitHub
robertwb opened a new pull request, #639: URL: https://github.com/apache/beam-site/pull/639 Content generated from commit 8f7d2cc8eb95286654cd31e01d32bb91e1165848. A separate updated as typescript was not yet part of the release scripts. -- This is an automated message from the

[GitHub] [beam-site] robertwb commented on pull request #639: Update beam-site for release 2.44.0.

2023-02-01 Thread via GitHub
robertwb commented on PR #639: URL: https://github.com/apache/beam-site/pull/639#issuecomment-1412401291 R: @pabloem -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: Performance and Cost benchmarking

2023-02-01 Thread Yi Hu via dev
Hi everyone, Thanks for your attention. Since this last thread there are works implementing the utilities for the performance and cost benchmarking framework ongoing, currently under the DataflowTemplate repository ( https://github.com/GoogleCloudPlatform/DataflowTemplates). In order to use these

Re: [DISCUSS] Migrate Flink runner to run batch jobs in DataStream API

2023-02-01 Thread Robert Bradshaw via dev
This sounds reasonable to me. One question I have is why a user would prefer to stick with the DataSet API if the DataStream API is available. Would there be any user-visible difference? On Wed, Feb 1, 2023 at 1:11 AM Becket Qin wrote: > > Hi Beam devs, > > I'd like to start a discussion about

[DISCUSS] Migrate Flink runner to run batch jobs in DataStream API

2023-02-01 Thread Becket Qin
Hi Beam devs, I'd like to start a discussion about migrating the Flink runner to execute the batch jobs in DataStream API instead of DataSet API. Today Flink runner executes batch jobs with DataSet API which is semi-deprecated and will be removed sometime in future Flink releases. Flink

Beam High Priority Issue Report (40)

2023-02-01 Thread beamactions
This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/25140 [Bug]: