Re: Introduction to the mailing list

Hello, Thank you for reaching out. I will take a look at all the links sent and get back to you in case of any queries. Regards, Manuela. On Sat, Oct 5, 2019, 2:51 AM Kenneth Knowles wrote: > Welcome! > > On Fri, Oct 4, 2019 at 1:29 PM Ahmet Altay wrote: > >> Welcome Manuela. >> >> You can lo

Re: [VOTE] Release 2.16.0, release candidate #1

Don't worry about the template. It is not required. Your first email contained enough information and the archives support the details. Your second one contained even more, and that is certainly nice :-) However, please do edit the subject when a vote is done, to "[RESULT] [VOTE] ". This makes it

Re: Plan for dropping python 2 support

On Fri, Oct 4, 2019 at 11:02 AM Robert Bradshaw wrote: > Thanks for holding this vote. Note that this is a pledge to remove > support sometime in 2020, but no promises as to whether that will be > January or December (though I hope sooner rather than later) Right. > Valentyn, did you want to

Re: Introduction to the mailing list

Welcome! On Fri, Oct 4, 2019 at 1:29 PM Ahmet Altay wrote: > Welcome Manuela. > > You can look at (https://issues.apache.org/jira/browse/BEAM-2855) as the > starting point. It has links to 2 previously merged PRs that can serve as > starting points for writing new nexmark queries. There are nexm

[portability] Removing the old portable metrics API...

Hello devs, I recently took a look at how Dataflow is retrieving metrics from the Beam SDK harnesses, and noticed something. As you may (or may not) remember, the portability API currently has two ways of reporting metrics. Namely, the newer MonitoringInfo API[1], and the older Metrics one[2]. Thi

Re: [VOTE] Release 2.16.0, release candidate #1

(Sorry for the informal note. I just realized there is a template I need to follow for the announcement:) I'm happy to announce that we have unanimously approved this release. There are 7 approving votes, 4 of which are binding: * Ahmet (al...@google.com) * Pablo (pabl...@google.com) * Robert (ro

Re: [VOTE] Release 2.16.0, release candidate #1

Thank you all for rc validation and voting! We collected 7 votes including 4 from PMC and all 2.16 JIRA issues are resolved. This meets release finalization criteria and I'll go ahead with the next steps. Thanks, Mark On Fri, Oct

Re: [VOTE] Release 2.16.0, release candidate #1

+1 Verified the new module sdks/java/extensions/zetasketch works (on direct runner) On Fri, Oct 4, 2019 at 12:41 PM Kenneth Knowles wrote: > +1 (binding) > > - Reviewed what verifications had been done. Nice. > - Also did a gradle build of some targets in the archival source release > > The s

Re: [VOTE] Release 2.16.0, release candidate #1

I also ran portable wordcount on Spark and Flink runners using docker container images that we are releasing with 2.16.0. I was using the SDK code from RC1 tag in github, and pulled the container image from Docker repo as follows: git checkout tags/v2.16.0-RC1 ./gradlew :runners:spark:job-server:

Re: Introduction to the mailing list

Welcome Manuela. You can look at (https://issues.apache.org/jira/browse/BEAM-2855) as the starting point. It has links to 2 previously merged PRs that can serve as starting points for writing new nexmark queries. There are nexmark queries described in (https://cwiki.apache.org/confluence/display/B

Re: outreachy intern

Welcome! On Fri, Oct 4, 2019 at 10:28 AM Rui Wang wrote: > Welcome Diksha! > > > Can you share your username of [1]? > > [1]: https://jira.apache.org/jira/secure/Dashboard.jspa > > -Rui > > On Fri, Oct 4, 2019 at 9:44 AM Thomas Weise wrote: > >> Welcome, Diksha! >> >> >> On Fri, Oct 4, 2019 at

Re: [VOTE] Release 2.16.0, release candidate #1

+1 (binding) - Reviewed what verifications had been done. Nice. - Also did a gradle build of some targets in the archival source release The source release still does not build as a whole, as it has not since 2.9.0 it seems. It is not as simple as excluding website from the build, because it fa

Spring with Apache Beam

Hi, I want to add Spring framework in my apache beam project. Somehow i am unable to inject the Spring Application context to executing ParDo functions. I couldn't find the way to do so? Can you please let me know how to integrate Spring runtime application context with Apache Beam pipeline. Tha

Re: [VOTE] Release 2.16.0, release candidate #1

OK, this appears to have been a weird config issue on my system (though the error certainly could have been better). As BEAM-8303 has a workaround and all else is looking good, I don't think that's worth another RC. +1 (binding) to this release. On Fri, Oct 4, 2019 at 10:56 AM Robert Bradshaw wr

Re: using avro instead of json for BigQueryIO.Write

Thanks Steve! I'll take a look next week. Sorry about the delay so far. Best -P. On Fri, Sep 27, 2019 at 10:37 AM Steve Niemitz wrote: > I put up a semi-WIP pull request https://github.com/apache/beam/pull/9665 for > this. The initial results look good. I'll spend some time soon adding > unit

Re: Feature addition to java CassandraIO connector

Hi Vincent! Do you think you could add some code snippets / pseudocode as to what this looks like? Feel free to do it on email, gist, google doc, etc? Best -P. On Thu, Oct 3, 2019 at 4:16 PM Vincent Marquez wrote: > Currently the CassandraIO connector allows a user to specify a table, and > the

Re: Plan for dropping python 2 support

Thanks for holding this vote. Note that this is a pledge to remove support sometime in 2020, but no promises as to whether that will be January or December (though I hope sooner rather than later). Valentyn, did you want to go ahead and make a PR adding Apache Beam to the python3statement page? O

Re: Multiple iterations after GroupByKey with SparkRunner

The DoFnSignature is where the information "this ParDo only needs a oneshot" would be recorded. This is what enables a runner to use the GBKOneShot in place of a full GBK. Kenn On Fri, Oct 4, 2019 at 1:13 AM Reuven Lax wrote: > Yes - this approach puts compatibility checking on the user. Howeve

Re: [VOTE] Release 2.16.0, release candidate #1

The artifact signatures and contents all look good to me. I've also verify the wheels work for the direct runner. However, I'm having an issue with trying to run on dataflow with Python 3.6: python -m apache_beam.examples.wordcount --input gs://clouddfe-robertwb/chicago_taxi_data/eval/data.csv

Re: NOTICE: New Python PreCommit jobs

> I have a WiP PR to convert Beam to use pytest, but it's been stalled. > What would it take to get it back on track? > Another nice thing about pytest is that you'll be able to tell which suite > a test belongs to. > pytest has a lot of quality of life improvements over nose. The biggest and

Re: [VOTE] Sign a pledge to discontinue support of Python 2 in 2020.

I also vote +1 and conclude the vote. There are 23 approving votes, 6 of which come from Apache Beam PMC, and there are no disapproving votes. Thanks everyone. On Wed, Oct 2, 2019 at 1:09 AM Mikhail Gryzykhin < gryzykhin.mikh...@gmail.com> wrote: > +1 > > On Tue, Oct 1, 2019 at 6:24 PM Ankur Go

Re: outreachy intern

Welcome Diksha! Can you share your username of [1]? [1]: https://jira.apache.org/jira/secure/Dashboard.jspa -Rui On Fri, Oct 4, 2019 at 9:44 AM Thomas Weise wrote: > Welcome, Diksha! > > > On Fri, Oct 4, 2019 at 8:47 AM diksha gupta > wrote: > >> Hi, I am Diksha Gupta, outreachy intern. >>

Re: [VOTE] Release 2.16.0, release candidate #1

Hi all, I looked at https://issues.apache.org/jira/browse/BEAM-8303, and it seems like the user has a workaround - is that correct? If that's the case, then I vote +1. @Max - lmk if you'd like to discuss further, but for now my vote is on +1. Best -P. On Fri, Oct 4, 2019 at 9:29 AM Mark Liu wrot

ApacheCon Europe 2019 talks which are relevant to Apache Beam

Dear Apache Beam committers, In a little over 2 weeks time, ApacheCon Europe is taking place in Berlin. Join us from October 22 to 24 for an exciting program and lovely get-together of the Apache Community. We are also planning a hackathon.  If your project is interested in participating, pl

Re: outreachy intern

Welcome, Diksha! On Fri, Oct 4, 2019 at 8:47 AM diksha gupta wrote: > Hi, I am Diksha Gupta, outreachy intern. > I will work with your host on beamSQL. >

Re: NOTICE: New Python PreCommit jobs

I have a WiP PR to convert Beam to use pytest, but it's been stalled. The nice thing about pytest-xdist is that it runs tests in a multi-process, single-thread-per-process fashion, so one test isn't affected by another changing some global setting. The not-so-nice thing is that xdist adds some glob

Re: Introduction to the mailing list

Welcome, Manuela! For getting familiar with the Beam development environment in general, I would recommend to take a look at: https://beam.apache.org/get-started/quickstart-py/ https://cwiki.apache.org/confluence/display/BEAM/Nexmark For contributing and collaborating in general, please take a

Re: [VOTE] Release 2.16.0, release candidate #1

+1 (forgot to vote) I also triggered Java Nexmark on direct, dataflow, spark and flink runner. Didn't saw performance regression from the dashboard ( https://apache-beam-testing.appspot.com/dashboard-admin) On Fri, Oct 4, 2019 at 8:23 AM Mark Liu wrote: > Thanks for the validation work! I valid

outreachy intern

Hi, I am Diksha Gupta, outreachy intern. I will work with your host on beamSQL.

Re: [VOTE] Release 2.16.0, release candidate #1

Thanks for the validation work! I validated following: - Java Quickstart on direct, dataflow,spark local, flink local runner - Java mobile gaming on direct and dataflow runner - Python Quickstart in batch and streaming in py2/3.5/3.6/3.7 using wheals/zip - Python Mobile Game in batch/streaming in

Introduction to the mailing list

Hello. My name is Chamda Manuela from the University of Buea, Cameroon. I am new to open source and comfortable with Python programming language. I will like to contribute to the outreachy project "Extend the Nextmark Benchmarking suite in Apache Beam to include python and portable runners". I w