Re: Callbacks/other functions run after a PDone/output transform

2017-12-19 Thread Eugene Kirpichov
I figured out the Never.ever() approach and it seems to work. Will finish this up and send a PR at some point. Woohoo, thanks Kenn! Seems like this will be quite a useful transform. On Mon, Dec 18, 2017 at 1:23 PM Eugene Kirpichov wrote: > I'm a bit confused by all of

FlinkRunner restore from save point, CoGroupByKey holds onto state

2017-12-19 Thread Seth Albanese
I’m running Beam 2.2.0 on Flink 1.3 using KafkaIO. Reading from two topics, applying a fixed window, joining via a CoGrouByKey, and outputting to another topic. Example code that reproduces the issue can be seen here: https://gist.github.com/salbanese/c46df2718c09a897e04d498c3f59d9d7 When I

Re: Reading from ORC Files in HDFS

2017-12-19 Thread Allan Wilson
Had a feeling that would be the answer, but being new to Beam I wanted to make sure I wasn’t missing something. :) Thanks Ismael On 12/18/17, 3:07 AM, "Ismaël Mejía" wrote: >Hello, > >There is not support yet to read ORC files directly on Beam, You can >track the

Re: Regarding Beam Slack Channel

2017-12-19 Thread Jean-Baptiste Onofré
Done, you should have received an invite. Regards JB On 12/19/2017 03:20 PM, Unais T wrote: Hello Can someone please add me to the Beam slack channel? Thanks. -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Regarding Beam Slack Channel

2017-12-19 Thread Unais T
Hello Can someone please add me to the Beam slack channel? Thanks.