Re: How to use "PortableRunner" in Python SDK?

Hi Ruoyun, I just ran the wordcount locally using the instructions on the page. I've tried the local file system and GCS. Both times it ran successfully and produced valid output. I'm assuming there is some problem with your setup. Which platform are you using? I'm on MacOS. Could you

Re: contributor permission for Beam Jira tickets

Resending. On Wed, Nov 14, 2018 at 1:06 PM Craig Chambers wrote: > Hi, I'm working on the Dataflow runner and the portable APIs. I'd like to > be granted contributor permission for Beam JIRA tickets. My user id is > CraigChambersG. Thanks. > > -- Craig > >

Re: contributor permission for Beam Jira tickets

Welcome, I have granted you contributor access to Beam's JIRA project. On Wed, Nov 14, 2018 at 3:09 PM Craig Chambers wrote: > Resending. > > On Wed, Nov 14, 2018 at 1:06 PM Craig Chambers > wrote: > >> Hi, I'm working on the Dataflow runner and the portable APIs. I'd like >> to be granted

Please do not merge Python PRs

It seems that Gradle is not getting the correct exit status from test runs. Possible culprit: https://github.com/apache/beam/pull/6903 smime.p7s Description: S/MIME Cryptographic Signature

Re: Please do not merge Python PRs

https://github.com/apache/beam/pull/7048 is the rollback PR On Wed, Nov 14, 2018 at 5:28 PM Ahmet Altay wrote: > Thank you Udi. Could you send a rollback PR? > > I believe this is https://issues.apache.org/jira/browse/BEAM-6048 > > On Wed, Nov 14, 2018 at 5:16 PM, Udi Meiri wrote: > >> It

Re: Please do not merge Python PRs

Thank you Udi. Could you send a rollback PR? I believe this is https://issues.apache.org/jira/browse/BEAM-6048 On Wed, Nov 14, 2018 at 5:16 PM, Udi Meiri wrote: > It seems that Gradle is not getting the correct exit status from test runs. > Possible culprit:

Re: Please do not merge Python PRs

Recreated locally: https://gradle.com/s/psqgcywnc3h2m On Wed, Nov 14, 2018 at 5:16 PM Udi Meiri wrote: > It seems that Gradle is not getting the correct exit status from test runs. > Possible culprit: https://github.com/apache/beam/pull/6903 > smime.p7s Description: S/MIME Cryptographic

Re: Design review for supporting AutoValue Coders and conversions to Row

We already have a framework for ByteBuddy codegen for JavaBean Row interfaces, which should hopefully be easy to extend AutoValue (and more efficient than using reflection). I'm working on adding constructor support to this right now. On Wed, Nov 14, 2018 at 12:29 AM Jeff Klukas wrote: >

Re: Bigquery streaming TableRow size limit

Generally I would agree, but the consequences here of a mistake are severe. Not only will the beam pipeline get stuck for 24 hours, _anything_ else in the user's GCP project that tries to load data into BigQuery will also fail for the next 24 hours. Given the severity, I think it's best to make

Re: [Call for items] November Beam Newsletter

Thank you, thank you! :) On Wed, Nov 14, 2018 at 10:34 AM Maximilian Michels wrote: > Updated the Flink Runner section. Just in time for the deadline :) > > On 14.11.18 00:37, Rui Wang wrote: > > Hi, > > > > I just added some thing related to BeamSQL. > > > > -Rui > > > > On Tue, Nov 13, 2018

Re: How to use "PortableRunner" in Python SDK?

Works for me on macOS as well. In case you don't launch the pipeline through Gradle, this would be the command: python -m apache_beam.examples.wordcount \ --input=/etc/profile \ --output=/tmp/py-wordcount-direct \ --runner=PortableRunner \ --job_endpoint=localhost:8099 \

Re: [DISCUSS] Publish vendored dependencies independently

Its a small hassle but could be checked in with some changes, my example commit was so that people could try it out as is. I'll work towards getting it checked in and then start a release for gRPC and guava. On Wed, Nov 14, 2018 at 11:45 AM Scott Wegner wrote: > Thanks for pushing this forward

Re: [DISCUSS] Publish vendored dependencies independently

Thanks for pushing this forward Luke. My understanding is that these vendored grpc artifacts will only be consumed directly by Beam internal components (as opposed to Beam user projects). So there should be a fairly low bar for publishing them. But perhaps we should have some short checklist for

Re: How to use "PortableRunner" in Python SDK?

Thanks Thomas! My desktop runs Linux. I was using gradle to run wordcount, and that was how I got the job hanging. Since both of you get it working, I guess more likely sth is wrong with my setup. By using Thmoas's python command line exactly as is, I am able to see the job run succeeds,

Re: How to use "PortableRunner" in Python SDK?

To answer Maximilian's question. I am using Linux, debian distribution. It probably sounded too much when I used the word 'planned merge'. What I really meant entails less change than it sounds. More specifically: 1) The default behavior, where PortableRunner starts a flink server. It is

Re: [Call for items] November Beam Newsletter

Updated the Flink Runner section. Just in time for the deadline :) On 14.11.18 00:37, Rui Wang wrote: Hi, I just added some thing related to BeamSQL. -Rui On Tue, Nov 13, 2018 at 3:26 AM Etienne Chauchot > wrote: Hi, I just added some things that were