Re: [ANNOUNCE] New committer: Reza Ardeshir Rokni

Excellent- well done Reza!!! On Thu, Sep 10, 2020 at 7:35 PM Austin Bennett wrote: > Thanks and congrats, Reza! > > On Thu, Sep 10, 2020 at 5:48 PM Heejong Lee wrote: > >> Congratulations! >> >> On Thu, Sep 10, 2020 at 4:42 PM Robert Bradshaw >> wrote: >> >>> Thank you and welcome, Reza! >>>

Re: [ANNOUNCE] New committer: Reza Ardeshir Rokni

Thanks and congrats, Reza! On Thu, Sep 10, 2020 at 5:48 PM Heejong Lee wrote: > Congratulations! > > On Thu, Sep 10, 2020 at 4:42 PM Robert Bradshaw > wrote: > >> Thank you and welcome, Reza! >> >> On Thu, Sep 10, 2020 at 4:00 PM Ahmet Altay wrote: >> >>> Congratulations Reza! And thank you

Contributor permission for Beam Jira tickets

Hi all-- I'm Monica, a SWE at Google! Can someone add me as a contributor for Beam's Jira issue tracker? My username is monicadsong. I would like to create/assign tickets for my work. I have 3 so far: [BEAM-10824] [BEAM-7654] [BEAM-10855]. Thanks, Monica

Re: [ANNOUNCE] New committer: Reza Ardeshir Rokni

Congratulations! On Thu, Sep 10, 2020 at 4:42 PM Robert Bradshaw wrote: > Thank you and welcome, Reza! > > On Thu, Sep 10, 2020 at 4:00 PM Ahmet Altay wrote: > >> Congratulations Reza! And thank you for your contributions! >> >> On Thu, Sep 10, 2020 at 3:59 PM Chamikara Jayalath >> wrote: >>

Re: Go SDK & Dataflow

This is a default limit on the receive Buffer on the SDK side. We set that it's maximum on July 30th. It's likely your copy doesn't have that change yet. https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/util/grpcx/dial.go#L39 The Go SDK is experimental and not meaningfully versioned

Go SDK & Dataflow

Beam team, I’m currently making use of the Beam Go SDK to construct a pipeline for creating and inserting large text data files into GCS. The pipeline is relatively simple; prepare a bracket of source data, transform it into fleshed out data sets, write to GCS with the textio package. func

Re: [VOTE] Release 2.24.0, release candidate #3

+1 (binding) Rebased fork and run internal performance tests. While doing so, I run into the unit test issue below with the fn_runner (Python direct runner), which did not occur with 2.21 [1]. That processing time timers are not supported wasn't an issue previously, because the timer, though

Re: [ANNOUNCE] New committer: Reza Ardeshir Rokni

Thank you and welcome, Reza! On Thu, Sep 10, 2020 at 4:00 PM Ahmet Altay wrote: > Congratulations Reza! And thank you for your contributions! > > On Thu, Sep 10, 2020 at 3:59 PM Chamikara Jayalath > wrote: > >> Congrats Reza! >> >> On Thu, Sep 10, 2020 at 10:35 AM Kenneth Knowles wrote: >>

Re: [VOTE] Release 2.24.0, release candidate #3

Given the additional information, I am upgrading my vote to +1 (binding) based on my prior analysis. On Thu, Sep 10, 2020 at 4:14 PM Kyle Weaver wrote: > +1 (non-binding) > > Validated wordcount with Python 3.7.8 and Flink 1.10.0 (both loopback and > using the Docker image). Also Python 3.7.8

Re: [VOTE] Release 2.24.0, release candidate #3

+1 (non-binding) Validated wordcount with Python 3.7.8 and Flink 1.10.0 (both loopback and using the Docker image). Also Python 3.7.8 loopback with an embedded Spark cluster. On Thu, Sep 10, 2020 at 2:32 PM Daniel Oliveira wrote: > By the way, most of the validation so far has covered Direct

Support for file version/generation in GcsPath

Hello, I added an issue and PR for adding support for file generation to GcsPath: https://issues.apache.org/jira/browse/BEAM-10867 https://github.com/apache/beam/pull/12805 This is already supported by GoogleCloudStorageReadChannel.java (I added that support in 2019), so I think we should

Re: [DISCUSS] Move Avro dependency out of core Beam

On Tue, Sep 8, 2020 at 9:18 AM Robert Bradshaw wrote: > IIRC Dataflow (and perhaps others) implicitly depend on Avro to write > out intermediate files (e.g. for non-shuffle Fusion breaks). Would > this break if we just removed it? > I think Dataflow would just need to declare a dependency on

Re: [VOTE] Release 2.24.0, release candidate #3

By the way, most of the validation so far has covered Direct runner and Dataflow, but Flink and Spark still have little validation, so if anyone can help with those it will help speed up the release. On Thu, Sep 10, 2020 at 2:12 PM Daniel Oliveira wrote: > So I tracked the --temp_location issue

Re: [VOTE] Release 2.24.0, release candidate #3

So I tracked the --temp_location issue down to https://github.com/apache/beam/pull/12203 and asked @Pablo Estrada and @Chamikara Jayalath about it. It's not exactly a bug, but an intended change in requirements for WriteToBigQuery, so the only fix I'll need to do is update the test script with

Re: [VOTE] Release 2.24.0, release candidate #3

+1 (non-binding) Thanks, Cham On Thu, Sep 10, 2020 at 11:26 AM Ahmet Altay wrote: > +1 - validated py3 quickstarts. The problem I mentioned earlier is > resolved. > > On Wed, Sep 9, 2020 at 7:46 PM Daniel Oliveira > wrote: > >> Good news: According to >>

Re: Ability to link to "latest" of python docs

Hey Brian - 100% this is what I was looking for! It works with intersphinx mapping just fine (looks like it handles redirects). Thanks so much! On Thu, Sep 10, 2020 at 1:50 PM Brian Hulette wrote: > There's https://beam.apache.org/releases/pydoc/current (and >

Re: Adding Twister2 documentation

@Ismaël Mejía would you have some cycles to validates these changes? Best Regards, Pulasthi On Mon, Aug 31, 2020 at 10:49 PM Pulasthi Supun Wickramasinghe < pulasthi...@gmail.com> wrote: > Hi Devs. > > I created two PR's to add the missing Twister2 runner documentation[1] > [2], however,

[ANNOUNCE] New committer: Reza Ardeshir Rokni

Hi all, Please join me and the rest of the Beam PMC in welcoming a new committer: Reza Ardeshir Rokni. Reza has been part of the Beam community since 2017! Reza has spearheaded advanced Beam examples [1], blogged and presented at multiple Beam Summits. Reza helps out users on the mailing lists

Notification of analysis on publicly available project data

Dear PMC, I’m contacting you because your project has been selected by the ASF D committee which is leading a research project to evaluate and understand the current state of diversity in our community [1]. As part of this research, we will analyze publicly available data about your project such

Re: Suggestion to let KafkaIO support the deserializer API with headers

Thanks Luke, I took a stab at this in https://issues.apache.org/jira/browse/BEAM-10865 - also outlined extensively avenues explored with the range of Kafka APIs to support and how Record headers and then eventually the (Extended)Deserializer APIs evolved. Tested with kafka-clients 1.0.0 through

Re: [ANNOUNCE] New committer: Heejong Lee

Congratulations! On Thu, Sep 10, 2020 at 4:41 PM Reza Rokni wrote: > Congratulations! > > On Thu, Sep 10, 2020 at 12:20 AM Alexey Romanenko < > aromanenko@gmail.com> wrote: > >> Congratulations! >> >> On 9 Sep 2020, at 09:50, Jan Lukavský wrote: >> >> Congratulations! >> On 9/9/20 1:00 AM,

Re: [ANNOUNCE] New committer: Heejong Lee

Congratulations! On Thu, Sep 10, 2020 at 12:20 AM Alexey Romanenko wrote: > Congratulations! > > On 9 Sep 2020, at 09:50, Jan Lukavský wrote: > > Congratulations! > On 9/9/20 1:00 AM, Chamikara Jayalath wrote: > > Congrats Heejong! > > On Tue, Sep 8, 2020 at 1:55 PM Yichi Zhang wrote: > >>

Re: [PROPOSAL] Preparing for Beam 2.25.0 release

Thanx Robin! On Thu, Sep 10, 2020 at 2:30 AM Ahmet Altay wrote: > Thank you Robin! > > On Wed, Sep 9, 2020 at 10:23 AM Rui Wang wrote: > >> Thanks Robin for working on this! >> >> >> -Rui >> >> On Wed, Sep 9, 2020 at 10:11 AM Robin Qiu wrote: >> >>> Hello everyone, >>> >>> The next Beam