Re: HDFS Support for Python SDK

Thanks Udi. It's a very good analysis and I agree with your conclusion. - Cham On Mon, Nov 20, 2017 at 9:26 PM Ahmet Altay wrote: > Thank you Udi, this is a great comparison of available options. > > On Mon, Nov 20, 2017 at 5:26 PM, Udi Meiri

Re: Python SDK DirectRunner (new feature)

Thank you María. On Mon, Nov 20, 2017 at 5:31 PM, María García Herrero < mari...@google.com.invalid> wrote: > Hello, > > I recently worked on adding a bundle retry for the Python SDK DirectRunner > ( > https://issues.apache.org/jira/browse/BEAM-2718). > > The goal was to have a more reliable

Re: HDFS Support for Python SDK

Thank you Udi, this is a great comparison of available options. On Mon, Nov 20, 2017 at 5:26 PM, Udi Meiri wrote: > Hi, > > I've done some research into implementing HDFS support for Python SDK and > I'd like your input. This work is regarding BEAM-3099 >

Re: [VOTE] Release 2.2.0, release candidate #4

We now have 3 PMC votes for yes. I'm just waiting for Flink runner validation (still missing on the spreadsheet) before finalizing the release. On Tue, Nov 21, 2017 at 9:39 AM, Robert Bradshaw < rober...@google.com.invalid> wrote: > +1 (binding) > > On Mon, Nov 20, 2017 at 5:01 PM, Eugene

Re: [VOTE] Release 2.2.0, release candidate #4

+1 (binding) On Mon, Nov 20, 2017 at 5:01 PM, Eugene Kirpichov < kirpic...@google.com.invalid> wrote: > I've additionally verified the game examples against Dataflow runner, and > verified UserScore and Leaderboard against Spark on Dataproc (YARN). All of > these worked* > So my +1 stands. > >

Python SDK DirectRunner (new feature)

Hello, I recently worked on adding a bundle retry for the Python SDK DirectRunner ( https://issues.apache.org/jira/browse/BEAM-2718). The goal was to have a more reliable processing of bundles. The change included having any bundle retry be processed up to 4 times and making sure GroupByKey

HDFS Support for Python SDK

Hi, I've done some research into implementing HDFS support for Python SDK and I'd like your input. This work is regarding BEAM-3099 . This doc lists several options for implementing HDFS support and attempts to weigh the differences.

Re: [VOTE] Release 2.2.0, release candidate #4

I've additionally verified the game examples against Dataflow runner, and verified UserScore and Leaderboard against Spark on Dataproc (YARN). All of these worked* So my +1 stands. In the verification spreadsheet, I'm not sure I understand the difference between the "YARN" and "Standalone

Re: [VOTE] Release 2.2.0, release candidate #4

Done On Tue, Nov 21, 2017 at 3:08 AM, Robert Bradshaw < rober...@google.com.invalid> wrote: > Thanks. You need to re-sign as well. > > On Mon, Nov 20, 2017 at 12:14 AM, Reuven Lax > wrote: > > FYI these generated files have been removed from the source distribution. >

Re: [VOTE] Release 2.2.0, release candidate #4

+1 I verified the python quick start on Windows. I could not verify the documentation changes because the staged version expired. On Mon, Nov 20, 2017 at 12:08 PM, Eugene Kirpichov < kirpic...@google.com.invalid> wrote: > Thanks Luke. I was able to validate quickstart on Dataflow and on Spark

Re: [VOTE] Release 2.2.0, release candidate #4

Thanks Luke. I was able to validate quickstart on Dataflow and on Spark cluster (using Cloud Dataproc). So +1 from me so far. On Sun, Nov 19, 2017 at 4:28 PM Lukasz Cwik wrote: > Eugene, you can setup your ~/.m2/settings.xml to point to the repository > containing the

Jenkins build is back to normal : beam_SeedJob #654

See

Re: [VOTE] Release 2.2.0, release candidate #4

Thanks. You need to re-sign as well. On Mon, Nov 20, 2017 at 12:14 AM, Reuven Lax wrote: > FYI these generated files have been removed from the source distribution. > > On Sat, Nov 18, 2017 at 9:09 AM, Reuven Lax wrote: > >> hmmm, I thought I removed

Build failed in Jenkins: beam_SeedJob #653

See -- GitHub pull request #4146 of commit b565925f487a703b9190774ca8d17ce068076c11, no merge conflicts. [EnvInject] - Loading node environment variables. Building remotely on beam2 (beam)

Re: [VOTE] Choose the "new" Spark runner

[ ] Use Spark 1 & Spark 2 Support Branch [ X ] Use Spark 2 Only Branch non-binding On Mon, Nov 20, 2017 at 1:00 AM, Etienne Chauchot wrote: > [ ] Use Spark 1 & Spark 2 Support Branch > [X] Use Spark 2 Only Branch > > Best > Etienne > > > > Le 19/11/2017 à 13:56,

Re: [VOTE] Choose the "new" Spark runner

[ ] Use Spark 1 & Spark 2 Support Branch [X] Use Spark 2 Only Branch Best Etienne Le 19/11/2017 à 13:56, Tyler Akidau a écrit : [ ] Use Spark 1 & Spark 2 Support Branch [X] Use Spark 2 Only Branch

Re: [VOTE] Release 2.2.0, release candidate #4

Yeah, I have a Jira about that. You just have to update the existing symlink to point on the new release. I will update the release guide asap. Thanks ! Regards JB On 11/20/2017 09:39 AM, Reuven Lax wrote: Good point. How do I update that link? I think this step might be missing from our

Re: [VOTE] Release 2.2.0, release candidate #4

Or is this simply a matter of copying from the dev to the release repository? On Mon, Nov 20, 2017 at 4:39 PM, Reuven Lax wrote: > Good point. How do I update that link? I think this step might be missing > from our release guide (or I simply can't find it). > > Reuven > > On

Re: [VOTE] Release 2.2.0, release candidate #4

Good point. How do I update that link? I think this step might be missing from our release guide (or I simply can't find it). Reuven On Mon, Nov 20, 2017 at 4:24 PM, Jean-Baptiste Onofré wrote: > +1 (binding) > > Tested on Beam samples, it looks good. > > Checked the

Re: [VOTE] Choose the "new" Spark runner

Moving my vote from previous threads: [ ] Use Spark 1 & Spark 2 Support Branch [X] Use Spark 2 Only Branch Ismaël On Thu, Nov 16, 2017 at 2:08 PM, Jean-Baptiste Onofré wrote: > Hi guys, > > To illustrate the current discussion about Spark versions support, you can > take a

Re: [VOTE] Release 2.2.0, release candidate #4

+1 (binding) Tested on Beam samples, it looks good. Checked the legal/license, content. OK for me. Just a side note: don't forget to update the latest link on dist.apache.org. Thanks Regards JB On 11/17/2017 07:08 AM, Reuven Lax wrote: Hi everyone, Please review and vote on the release

Re: [VOTE] Release 2.2.0, release candidate #4

Tested on some custom transforms and integrations, +1 (non-binding), thanks for the perseverance Reuven! Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github | LinkedIn 2017-11-20 9:14 GMT+01:00 Reuven Lax : > FYI these generated files have been removed from the

Re: [VOTE] Release 2.2.0, release candidate #4

FYI these generated files have been removed from the source distribution. On Sat, Nov 18, 2017 at 9:09 AM, Reuven Lax wrote: > hmmm, I thought I removed those generated files from the zip file before > sending this email. Let me check again. > > Reuven > > On Sat, Nov 18, 2017