Re: Two example pipelines built by Yahoo intern

2017-08-08 Thread Eugene Kirpichov
+Aljoscha Krettek for comments on Flink runner +Thomas Weise likewise for Apex runner On Tue, Aug 8, 2017 at 4:52 PM Eugene Kirpichov wrote: > Hi Claire, > > Thank you - happy to see a paper with such a detailed description of

Re: Two example pipelines built by Yahoo intern

2017-08-08 Thread Eugene Kirpichov
Hi Claire, Thank you - happy to see a paper with such a detailed description of your experience with both usability of Beam per se and the execution on the Flink runner! The paper looks well-written, and, from a quick look at the code, it seems to be using the Beam API properly without obvious

Re: Two example pipelines built by Yahoo intern

2017-08-08 Thread Jesse Anderson
Claire, Interesting work. In section 5, you talk about the Java language being difficult. Was there a reason you didn't use Java lambdas for your work? Thanks, Jesse On Tue, Aug 8, 2017 at 3:40 PM Claire Yuan wrote: > Hi folks, > We are a two-members team

Re: Slack invite request

2017-08-08 Thread Jason Kuster
Done! On Tue, Aug 8, 2017 at 1:56 PM, Steve Anderson wrote: > Hi there, can i please get an invite to the beam slack channel? > > Thanks! > - Steve > > -- > > Steven Anderson > Software Developer > Mobile: 650.455.6530 > Email: st...@maestro.io > Website: http://www.maestro.io

Slack invite request

2017-08-08 Thread Steve Anderson
Hi there, can i please get an invite to the beam slack channel? Thanks! - Steve -- Steven Anderson Software Developer Mobile: 650.455.6530 Email: st...@maestro.io Website: http://www.maestro.io

streaming output in just one files

2017-08-08 Thread Claire Yuan
Hi all,  I am currently running some jobs coded in Beam in streaming mode on Yarn session by Flink. My data sink was CSV files like the one in examples of TfIdf. And I noticed that the output format for Beam is to produce one file for every record, and also temp files for them. That would