Re: [PROPOSAL] Preparing for Beam 2.23.0 release

Hi, I've just created PR introducing usage of GH-Actions to release process https://github.com/apache/beam/pull/12150 Let me know what you think, maybe you have some suggestions on what may be improved. BR Tobiasz On Wed, Jul 1, 2020 at 6:08 AM Ahmet Altay wrote: > Valentyn, > >

[Ask] Beam-6832

Hi all, I am irchan, I want to add labels in dataflow VM. I found this issue https://issues.apache.org/jira/browse/BEAM-6832. Is there an estimated time where it will be featured in the beam release? Thank you -- Irchan Bani Hasym Data Science Engineer [image: Logo] T:

[Ask] Beam-6832

Hi all, I am irchan, I want to add labels in dataflow VM. I found this issue https://issues.apache.org/jira/browse/BEAM-6832. Is there an estimated time where it will be featured in the beam release? Thank you Best regards, Irchan Bani H Kurio | Data Science Engineer

Re: [PROPOSAL] Preparing for Beam 2.23.0 release

Great, thank you Tobiasz, I will take a look. On Wed, Jul 1, 2020 at 7:27 AM Tobiasz Kędzierski < tobiasz.kedzier...@polidea.com> wrote: > Hi, > > I've just created PR introducing usage of GH-Actions to release process > https://github.com/apache/beam/pull/12150 > > Let me know what you think,

Re: [Ask] Beam-6832

Hi Irchan, are you saying you'd like to include user-defined labels for Dataflow VMs? On Wed, Jul 1, 2020 at 8:46 AM Irchan Bani Hasym wrote: > Hi all, > > I am irchan, I want to add labels in dataflow VM. I found this issue > https://issues.apache.org/jira/browse/BEAM-6832. > Is there an

Season of Docs Query

Hello, I am an open source enthusiast and have a passion for writing as well. I do not have a computer science degree but i am well equipped with modern software technologies. Can you please give me your top two priority projects in which i can apply. I may have been a little late but i assure you

Re: [jsoc] Season of Docs Query

Hey Anchal, Please join our community Slack (https://slackinvite.julialang.org) and the #gsod channel. With respect to priorities, we don’t have any specific order defined. We are enthusiastic about all of the projects we presented. We are looking forward to seeing you online! Best,

Re: On Auto-creating GCS buckets on behalf of users

IMHO, we're erroring a bit to far on making it hard to get started. I would lean towards automatically creating (and using) a bucket, provided it had a name that was unlikely to conflict with others and very obvious when one saw it. (Logging is important too, but also very often ignored and not

Re: [DISCUSS] ReadAll pattern and consistent use in IO connectors

>From the Go side of the table, the Go language doesn't provide a mechanism to serialize or access closure data, which means DoFns can't be functional closures.This combined with the move to have the "Structural DoFns" be serialized using Beam Schemas, has the net result that if Go transforms are

XLang sub-graph representation within the SDKs pipeline types

It seems like we keep running into translation issues with XLang due to how it is represented in the SDK. (e.g. Brian's work on context map due to loss of coder ids, Heejong's work related to missing environment ids on windowing strategies). I understand that there is an effort that is Dataflow

Beam Summit Status Report - 7/1

Hi folks, I wanted to provide you with the Beam Summit Status report from today's meeting. If you would like to join the next public meeting on Wednesday, July 8th at 11:30 AM PST please let me know and I will send a calendar invite over to you! Also don't forget to register for the Summit

Re: [PROPOSAL] Preparing for Beam 2.23.0 release

Release branch has been cut. As a reminder, please do not merge commits into the release branch directly, instead, loop in the release manager if any cherry-picks are required. Thank you. On Wed, Jul 1, 2020 at 9:25 AM Valentyn Tymofieiev wrote: > Great, thank you Tobiasz, I will take a look.

Re: [PROPOSAL] Preparing for Beam 2.23.0 release

Can somebody please add my pypi username (tvalentyn) to the list of apache-beam maintainers on PyPi: https://pypi.org/project/apache-beam/ ? Thank you! On Wed, Jul 1, 2020 at 4:51 PM Valentyn Tymofieiev wrote: > Release branch has been cut. > > As a reminder, please do not merge commits into

Re: Composable DoFn IOs Connection Reuse

I see, the splitting of state shards is related to splitting of splittable DoFns. On Tue, Jun 30, 2020 at 3:36 PM Kenneth Knowles wrote: > I agree at the level of GroupIntoBatches. The part that is similar is if > you implement GroupIntoBatches with a new primitive supporting > runner-scoped

Re: XLang sub-graph representation within the SDKs pipeline types

>From the Go SDK side, it was built that way nearly from the start. Historically there was a direct SDK rep -> Dataflow rep conversion, but that's been replaced with a SDK rep -> Beam Proto -> Dataflow rep conversion. In particular, this approach had a few benefits: easier to access local context

Re: [DISCUSS] ReadAll pattern and consistent use in IO connectors

Kenn - I don't mean an enum of common closures, I mean expressing closures in a restricted sub-language such as the language of SQL expressions. That would only work if there is a portable way to interpret SQL expressions, but if there isn't, maybe there should be - for the sake of, well,

Re: [Ask] Beam-6832

There is no estimated time as it is a feature request and no one on the dev@ mailing list has mentioned that they are working on it. Are you interested in implementing the feature? On Wed, Jul 1, 2020 at 8:47 AM irchan bani wrote: > Hi all, > > I am irchan, I want to add labels in dataflow VM.

RequiresStableInput on Spark runner

We have a component which we use in streaming and batch jobs. Streaming we run on FlinkRunner and batch on SparkRunner. Recently we needed to add @RequiresStableInput to taht component because of streaming use-case. But now batch case crash on SparkRunner with Caused by:

subscribe

-- *Software Engineer* *Salzburg Research Forschungsgesellschaft * *Salzburg, Austria*

Re: subscribe

I think you wanted to send the mail to dev-subscr...@beam.apache.org Have a good day! Michal On Wed, Jul 1, 2020 at 10:43 AM Ayeshmantha Perera wrote: > > > -- > *Software Engineer* > *Salzburg Research Forschungsgesellschaft * > *Salzburg, Austria* > -- Michał Walenia Polidea