Re: Update of Pei in Alibaba

Regards > > Jian Liu(Basti) > > > > -Original Message- > From: Ismaël Mejía [mailto:ieme...@gmail.com] > Sent: Sunday, April 02, 2017 3:18 AM > To: dev@beam.apache.org > Subject: Re: Update of Pei in Alibaba > > > > Excellent news, >

Re: Update of Pei in Alibaba

Nice to hear from you again, Pei! This is awesome news. I'd love to help when you are ready to get it in the repo and hooked up to our testing infrastructure. Kenn On Fri, Mar 31, 2017 at 6:24 PM, Pei HE wrote: > Hi all, > On February, I moved from Seattle to Hangzhou,

Re: Update of Pei in Alibaba

Thanks Jingsong for answering, and the Streamscope ref, I am going to check the paper, the concept of non-global-checkpointing sounds super interesting. It is nice that you guys are also trying to promote the move to a unified model. Regards, Ismaël On Sun, Apr 2, 2017 at 3:40 PM, JingsongLee

Re: Update of Pei in Alibaba

Hi Ismaël, We have a streaming computing platform in Alibaba. Galaxy is an internal system, so you can't find some information from Google. It is becoming more like StreamScope (you can search it for the paper). Non-global-checkpoint makes failure recovery quickly and makes streaming 

Re: Update of Pei in Alibaba

Excellent news, Pei it would be great to have a new runner. I am curious about how different are the implementations of storm among them considering that there are already three 'versions': Storm, Jstorm and Heron, I wonder if one runner could traduce to an API that would cover all of them (of

Re: Update of Pei in Alibaba

Exciting times, looking forward to try it out! I shall mention that Taylor Goetz also started creating a BEAM runner using Storm. His work is available in the storm repo: https://github.com/apache/storm/commits/beam-runner Maybe it's worth while to take a peek and see if something is reusable