Hi Ismaël,
Sorry for the late response. I am the developer of JStorm, and currently work with Pei on JStorm runner. We have went through current Storm runner( <https://github.com/apache/storm/commits/beam-runner> https://github.com/apache/storm/commits/beam-runner). But it is a very draft version, several PTransforms are not supported or not fully supported, especial for window, trigger and state. Generally, JStorm is compatible with the basic API of Storm, while providing improvements or new features on topology master, state manager, exactly once, message transfer mechanism, stage-by-stage backpressure flow control, metrics, etc. For the basic “at most once” job, JStorm runner can be reused on Storm. But for “window”, “state” and “exactly once” job, unfortunately, JStorm runner can’t be reused. Anyway, we will figure out if the propagation is possible for Storm in the future. Regards Jian Liu(Basti) -----Original Message----- From: Ismaël Mejía [mailto:[email protected]] Sent: Sunday, April 02, 2017 3:18 AM To: [email protected] Subject: Re: Update of Pei in Alibaba Excellent news, Pei it would be great to have a new runner. I am curious about how different are the implementations of storm among them considering that there are already three 'versions': Storm, Jstorm and Heron, I wonder if one runner could traduce to an API that would cover all of them (of course maybe I am super naive I really don't know much about JStorm or Heron and how much they differ from the original storm). Jingson, I am super curious about this Galaxy project, it is there any public information about it? is this related to the previous blink ali baba project? I already looked a bit but searching "Ali baba galaxy" is a recipe for a myriad of telephone sellers :) Nice to see that you are going to keep contributing to the project Pei. Regards, Ismaël On Sat, Apr 1, 2017 at 4:59 PM, Tibor Kiss < <mailto:[email protected]> [email protected]> wrote: > Exciting times, looking forward to try it out! > > I shall mention that Taylor Goetz also started creating a BEAM runner > using Storm. > His work is available in the storm repo: > <https://github.com/apache/storm/commits/beam-runner> > https://github.com/apache/storm/commits/beam-runner > Maybe it's worth while to take a peek and see if something is reusable > from there. > > - Tibor > > On Sat, Apr 1, 2017 at 4:37 AM, JingsongLee < > <mailto:[email protected]> [email protected]> wrote: > >> Wow, very glad to see JStorm also started building BeamRunner. >> I am working in Galaxy (Another streaming process engine in Alibaba). >> I hope that we can work together to promote the use of Apache Beam in >> Alibaba and China. >> >> best, >> JingsongLee >> ------------------------------------------------------------------Fro >> m:Pei HE < <mailto:[email protected]> [email protected]>Time:2017 Apr 1 (Sat) >> 09:24To:dev < >> <mailto:[email protected]%3eSubject:Update> >> [email protected]>Subject:Update of Pei in Alibaba Hi all, On >> February, I moved from Seattle to Hangzhou, China, and joined Alibaba. >> And, I want to give an update of things in here. >> >> A colleague and I have been working on JStorm >> < <https://github.com/alibaba/jstorm> https://github.com/alibaba/jstorm> >> runner. We have a prototype that >> works with WordCount and PAssert. (I am going to start a separate >> email thread about how to get it reviewed and merged in Apache Beam.) >> We also have Spark clusters, and are very interested in using Spark >> runner. >> >> Last Saturday, I went to China Hadoop Summit, and gave a talk about >> Apache Beam model. While many companies gave talks of their in-house >> solutions for unified batch&streaming and unified SQL, there are also >> lots of interests and enthusiasts of Beam. >> >> Looking forward to chat more. >> -- >> Pei >> >> > > > -- > Kiss Tibor > > +36 70 275 9863 > <mailto:[email protected]> [email protected]
