Re: Pipeline Execution

2017-01-23 Thread Stephen Sisk
No problem - I understood what you meant and didn't read it as offensive. I definitely have times where I need to talk to someone else to point out something obvious when I've been staring at my code too long. :) S On Mon, Jan 23, 2017 at 3:40 PM Wyatt Frelot wrote: Stephen,

Re: Pipeline Execution

2017-01-23 Thread Stephen Sisk
hi! Thanks for following up with questions! I see you're using BatchWriter - based on the docs [1] I see that there are flush() and close() calls. I presume you'd want to be calling one of those. I'm not familiar with Accumulo, but usually batching writers wait to do anything until methods like

Unsubscribe

2017-01-23 Thread amir bahmanyari
Unsubscribe

Re: Beam Spark/Flink runner with DC/OS

2017-01-23 Thread Chaoran Yu
Sorry for the spam. But to clarify, I didn’t write the code. I’m using the code described here https://beam.apache.org/get-started/wordcount-example/ So the file already exists in GS. > On Jan 23, 2017, at 4:55 PM, Chaoran Yu

Re: Beam Spark/Flink runner with DC/OS

2017-01-23 Thread Chaoran Yu
Well, my file is not in my local filesystem. It’s in GS. This is the line of code that reads the input file: p.apply(TextIO.Read.from("gs://apache-beam-samples/shakespeare/*")) And this page https://beam.apache.org/get-started/quickstart/ says

Re: Beam Spark/Flink runner with DC/OS

2017-01-23 Thread Chaoran Yu
No I’m not using Dataproc.I’m simply running on my local machine. I started a local Spark cluster with sbin/start-master.sh and sbin/start-slave.sh. Then I submitted my Beam job to that cluster.The gs file is the kinglear.txt from Beam’s example code and it should be public. My full stack trace is

Re: Regarding Beam Slack Channel

2017-01-23 Thread Amit Sela
Done. Welcome! On Mon, Jan 23, 2017 at 11:29 PM Kai Jiang wrote: > Hi Amit, > > I am a new contributor. Could you add me as well? jiang...@gmail.com > > Thanks, > Kai > > On Mon, Jan 23, 2017 at 1:04 PM, Davor Bonaci wrote: > > I wouldn't be in favor of

Re: Regarding Beam Slack Channel

2017-01-23 Thread Davor Bonaci
I wouldn't be in favor of such "solutions" on top of Slack, unless specifically endorsed by Slack itself. It sounds like something intended to override specific design decisions, and we may not be acting in the best faith if we were to deploy it. If Slack doesn't support features we think we

Re: Regarding Beam Slack Channel

2017-01-23 Thread Xu Mingmin
Thank you @Amit Sent from my iPhone > On Jan 23, 2017, at 12:08 PM, Amit Sela wrote: > > Done. Welcome! > >> On Mon, Jan 23, 2017 at 10:03 PM Xu Mingmin wrote: >> can anyone add me to the channel as well? Thanks a lot! >> >> mmxu1...@gmail.com >>

Re: Regarding Beam Slack Channel

2017-01-23 Thread Amit Sela
Done. Welcome! On Mon, Jan 23, 2017 at 10:26 PM Wyatt Frelot wrote: > Can I be added as well? > > wjfr...@gmail.com > > Wyatt > > On Jan 23, 2017 3:20 PM, "Amit Sela" wrote: > > Done. Welcome! > > On Mon, Jan 23, 2017 at 10:17 PM Brackman, Levi < >

Re: Regarding Beam Slack Channel

2017-01-23 Thread Amit Sela
Done. Welcome! On Mon, Jan 23, 2017 at 10:10 PM Chaoran Yu wrote: > Can anyone add me to the channel as well? Thank you! > chaoran...@lightbend.com > > Chaoran > > On Jan 23, 2017, at 1:32 AM, Ritesh Kasat wrote: > > Hello, > > Can someone add

Re: Regarding Beam Slack Channel

2017-01-23 Thread Chaoran Yu
Can anyone add me to the channel as well? Thank you! chaoran...@lightbend.com Chaoran > On Jan 23, 2017, at 1:32 AM, Ritesh Kasat wrote: > > Hello, > > Can someone add me to the Beam slack channel. > > Thanks > Ritesh

Re: KafkaIO: reset topic for reading from the start with every run

2017-01-23 Thread Thomas Groh
You should also generally expect the DirectRunner to be slower than production runners - the goals of the DirectRunner are primarily ensuring that Pipelines are portable to other production runners and enforcing the Beam Programming Model while enabling local iteration and development, and

Re: KafkaIO: reset topic for reading from the start with every run

2017-01-23 Thread Gareth Western
Actually I made a silly mistake in my configuration and had the wrong topic name. Tired eyes. Thanks for confirming the config! On 23. jan. 2017 19:17, Raghu Angadi wrote: On Mon, Jan 23, 2017 at 10:10 AM, Gareth Western > wrote:

Re: Regarding Beam Slack Channel

2017-01-23 Thread Jean-Baptiste Onofré
Thanks Jason, that's a good alternative if open channel is not possible. I'm checking DNS open channel. I keep you posted. Regards JB On 01/23/2017 07:41 PM, Jason Kuster wrote: I've set up https://github.com/thesandlord/SlackEngine for a different slack group I'm a part of before; would