Dear all, This seems an interesting discussion and I want to just give my 2 cents here.
1. I disagree with this statement 'According to the current development situation in China, java core is indeed to bring more contributors here'. In recent QCon in Shanghai, I saw folks from China local internet companies did quite a lot contribution to Storm project without requiring it to have a Java core. Also, technically, with the minimum core code in Clojure and right interfaces in place, it's easy to extend Storm features and functions with Java. 2. If the proposal of using JStrom replacing Storm is accepted, then the community will spend a lot time migrating and the global users of Storm needs to do a lot regression testing to verify no large issues, no matter how good the unit testing will be done. The impact is too large for Storm user community to afford. 3. I firm believe the combination of some Java together with uprising JVM compatible language like Clojure and Scala, is the future for complex distributed application development. There are quite some successful example of new open source projects like Kafka and Spark, both developed using both Scala and expose APIs and extensions in Java. If we are going to replace all non-Java projects with a Java version, will it be a worth our precious time? 4. I understand Alibaba has certain special requirements and some implementation to satisfy the specific needs of shared cluster resource management, etc. As JStorm already has implementation of these kind of requirements, I suggest that folks from Alibaba to start some good communication with Storm committers and ask them to update just the extension points of Storm, to enable specific extension, but without replacing the core entirely. Lex eBay -----Original Message----- From: 刘键(夏�G) [mailto:[email protected]] Sent: 2014年10月24日 11:20 To: [email protected]; [email protected]; 徐明明(护城); [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; 封仲淹(纪君祥) Cc: aloha-dev; [email protected]; [email protected]; [email protected]; [email protected]; 陈昱（叔至）; 姬风; 周小帆(承嗣); [email protected] Subject: Re: Could JStorm project collaborate with Storm? Hi Andy, Thanks for your quick response. What you concern is reasonable and understandable. But maybe there is some misunderstanding on our proposal. According to the current development situation in China, java core is indeed to bring more contributors here. That is reason why we begin to think if it is also a way for us to provide our contribution to Storm project. It is true that it is a risk to switch the core in a short period. Actually, if possible, maybe we can consider to take JStorm as a subproject of Storm(A branch or some other ways…). We will be responsible for the maintain of it and try to propagate the features from Storm to JStorm, and continue to develop our features and improvements on this. You guys can try the new core and take a long time to estimate if it is worth doing the switch in the future or taking JStorm as official optional core of Storm for user. Regards Basti 发件人: Andrew Feng [mailto:[email protected]] 发送时间: 2014年10月24日 0:24 收件人: 封仲淹(纪君祥); [email protected]; 徐明明(护城); [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected] 抄送: aloha-dev; [email protected]; [email protected]; [email protected]; [email protected] 主题: Re: 答复: Could JStorm project collaborate with Storm? Zhongyan: Yahoo almost took your path 2 years ago. After some discussion with Nathan, we decided to work with community on Storm, and some of us learned Clojure quickly. We are very glad that we did that. Under the assumption that Storm provides all features of JStorm, my personal vote will be NO for your proposal. * Storm has enjoyed its current success before Nathan built a very solid core in Clojure. We should not replace that core until we are 100% sure that alternative implementation is at least as good as our Clojure implementation. * Clojure has not prevent Storm attract contributors (currently 108). We have contributors who write code in Clojure and Java. It doesn’t take much time for one to be able to understand Clojure code. * The convergence of streaming processing and batch processing will occur at higher level of abstractions. Trident API, for example, is very much similar to batch API such as Pig or Cascading. * Including Jstorm under Storm will only create confusion to our user community at this stage. Why don’t you have discussion with 徐明明 to figure out an alternative path for JStorm? Thanks, Andy From: "封仲淹 (纪君祥)" <[email protected]> Reply-To: "封仲淹(纪君祥)" <[email protected]> Date: Wednesday, October 22, 2014 at 11:37 PM To: Andy Feng <[email protected]>, "[email protected]" <[email protected]>, "徐明明(护城)" <[email protected]>, "[email protected]" <[email protected]>, "[email protected]" <[email protected]>, "[email protected]" <[email protected]>, "[email protected]" <[email protected]>, Doug Cutting <[email protected]>, "[email protected]" <[email protected]>, "[email protected]" <[email protected]>, "[email protected]" <[email protected]>, "[email protected]" <[email protected]>, "[email protected]" <[email protected]> Cc: aloha-dev <[email protected]>, "[email protected]" <[email protected]>, "[email protected]" <[email protected]>, "[email protected]" <[email protected]>, "[email protected]" <zeus@tencent. com> Subject: 答复: Could JStorm project collaborate with Storm? Andrew, Thanks for suggest. The problem is that the core of Storm is implemented by Clojure, if it is java, we are glad to merge all our commit into the Strom Trunk. If the core of Storm is implemented by Java, I think the contributor of Storm would be double. May I put forward 2 proposal: 1. Could Jstorm be the son project of Storm? 2. Task a long time to change Storm-core as Jstorm-core. a) If the core of Storm is implemented by Java, I think the contributor of Storm would be double. Once one user found one bug, maybe he is able to fix it by himself, in this method, Storm resolve problem will be double b) In the next one or two years, batch handling and stream handling will merge into one solution, Spark/Flink are doing this job. If we still use clojure, I am a little afraid that we can’t follow the steps of other community and this direction. Thanks Longda 发件人: Andrew Feng [mailto:[email protected]] 发送时间: 2014年10月23日 13:31 收件人: 封仲淹(纪君祥); [email protected]; 徐明明(护城); [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected] 抄送: aloha-dev; [email protected]; [email protected]; [email protected]; [email protected] 主题: Re: Could JStorm project collaborate with Storm? Zhongyan: I would suggest you identify a collection of key features in Jstorm, and then propose pull requests to Storm. Once these pull requests are accepted, you could then simply adopt Storm. Andy From: "封仲淹 (纪君祥)" <[email protected]> Reply-To: "封仲淹(纪君祥)" <[email protected]> Date: Wednesday, October 22, 2014 at 8:23 PM To: "[email protected]" <[email protected]>, "徐明明(护城)" <[email protected]>, "[email protected]" <[email protected]>, Andy Feng <[email protected]>, "[email protected]" <[email protected]>, "[email protected]" <[email protected]>, "[email protected]" <[email protected]>, Doug Cutting <[email protected]>, "[email protected]" <[email protected]>, "[email protected]" <[email protected]>, "[email protected]" <[email protected]>, "[email protected]" <[email protected]>, "[email protected]" <[email protected]> Cc: aloha-dev <[email protected]>, "[email protected]" <[email protected]>, "[email protected]" <[email protected]>, "[email protected]" <[email protected]>, "[email protected]" <zeus@tencent. com> Subject: Could JStorm project collaborate with Storm? Hi all, Firstly, congratulate Storm becoming the top project of Apache. Since the first day of Storm opened source, it has become one of hottest technologies in the BigData field and brought a new perspective to the world, which has helped many companies resolve hundreds of requirements. Since 2012/4, we have forked a project named "JStorm". At the beginning time, we just wanted to implement Storm with Java. As time goes on, more and more features have been added to JStorm. In 2013/9, JStorm changed to be an open source project; from then on, more and more people and companies join this project. As JStorm is derived from Storm, and born in a great company named Alibaba, which just successfully listed on the New York Stock Exchange on Sept. 19, is a very open company. We also want to share our technology with more people, and favor rapid evolution of streaming technology. So, could JStorm project collaborate with Storm? A little introduction for JStorm as follows: JStorm is a distributed and fault-tolerant real-time computation system. Inspired by Apache Storm, JStorm has been completely rewritten in Java and provides more enhanced features. JStorm has been widely used in many enterprise environments and proved robust and stable. Begun from 2012/2, we have released 11 versions till 2014/09/28. More than 10 companies are using it, including several leading companies such as Alibaba, Tencent, Baidu, Huawei, ZTE in China. Please refer to https://github.com/alibaba/jstorm for more details Regards Longda
