Hi Cody, I wasn’t able to find your username. Are you sure you have an account on cwiki.apache.org?
-Taylor > On Nov 22, 2015, at 8:46 AM, Cody Innowhere <e.neve...@gmail.com> wrote: > > Hi Taylor, > I'd like to help too, could you add me in? my id is: Cody > > On Sat, Nov 21, 2015 at 11:51 AM, 刘键(Basti Liu) <basti...@alibaba-inc.com> > wrote: > >> Hi Taylor， >> >> Sorry for the late response. >> I'd like to help on this. Could you please help to give me the permission? >> Thanks. >> UserName: basti.lj >> >> Regards >> Basti >> >> -----Original Message----- >> From: P. Taylor Goetz [mailto:ptgo...@gmail.com] >> Sent: Thursday, November 19, 2015 6:24 AM >> To: dev@storm.apache.org; Bobby Evans >> Subject: Re: [DISCUSS] Plan for Merging JStorm Code >> >> All I have at this point is a placeholder wiki entry [1], and a lot of >> local notes that likely would only make sense to me. >> >> Let me know your wiki username and I’ll give you permissions. The same >> goes for anyone else who wants to help. >> >> -Taylor >> >> [1] >> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61328109 >> >>> On Nov 18, 2015, at 2:08 PM, Bobby Evans <ev...@yahoo-inc.com.INVALID> >> wrote: >>> >>> Taylor and others I was hoping to get started filing JIRA and planning >>> on how we are going to do the java migration + JStorm merger. Is >>> anyone else starting to do this? If not would anyone object to me >>> starting on it? - Bobby >>> >>> >>> On Thursday, November 12, 2015 12:04 PM, P. Taylor Goetz < >> ptgo...@gmail.com> wrote: >>> >>> >>> Thanks for putting this together Basti, that comparison helps a lot. >>> >>> And thanks Bobby for converting it into markdown. I was going to just >> attach the spreadsheet to JIRA, but markdown is a much better solution. >>> >>> -Taylor >>> >>>> On Nov 12, 2015, at 12:03 PM, Bobby Evans <ev...@yahoo-inc.com.INVALID> >> wrote: >>>> >>>> I translated the excel spreadsheet into a markdown file and put up a >> pull request for it. >>>> https://github.com/apache/storm/pull/877 >>>> I did a few edits to it to make it work with Markdown, and to add in a >> few of my own comments. I also put in a field for JIRAs to be able to >> track the migration. >>>> Overall I think your evaluation was very good. We have a fair amount >> of work ahead of us to decide what version of various features we want to >> go forward with. >>>> - Bobby >>>> >>>> >>>> On Thursday, November 12, 2015 9:37 AM, 刘键(Basti Liu) < >> basti...@alibaba-inc.com> wrote: >>>> >>>> >>>> Hi Bobby & Jungtaek, >>>> >>>> Thanks for your replay. >>>> I totally agree that compatibility is the most important thing. >> Actually, JStorm has been compatible with the user API of Storm. >>>> As you mentioned below, we indeed still have some features different >> between Storm and JStorm. I have tried to list them (minor update or >> improvements are not included). >>>> Please refer to attachment for details. If any missing, please help >>>> to point out. (The current working features are probably missing here.) >> Just have a look at these differences. For the missing features in JStorm, >> I did not see any obstacle which will block the merge to JStorm. >>>> For the features which has different solution between Storm and JStorm, >> we can evaluate the solution one by one to decision which one is >> appropriate. >>>> After the finalization of evaluation, I think JStorm team can take the >> merging job and publish a stable release in 2 months. >>>> But anyway, the detailed implementation for these features with >> different solution is transparent to user. So, from user's point of view, >> there is not any compatibility problem. >>>> >>>> Besides compatibility, by our experience, stability is also important >> and is not an easy job. 4 people in JStorm team took almost one year to >> finish the porting from "clojure core" >>>> to "java core", and to make it stable. Of course, we have many devs in >> community to make the porting job faster. But it still needs a long time to >> run many online complex topologys to find bugs and fix them. So, that is >> the reason why I proposed to do merging and build on a stable "java core". >>>> >>>> -----Original Message----- >>>> From: Bobby Evans [mailto:ev...@yahoo-inc.com.INVALID] >>>> Sent: Wednesday, November 11, 2015 10:51 PM >>>> To: dev@storm.apache.org >>>> Subject: Re: [DISCUSS] Plan for Merging JStorm Code >>>> >>>> +1 for doing a 1.0 release based off of the clojure 0.11.x code. >> Migrating the APIs to org.apache.storm is a big non-backwards compatible >> move, and a major version bump to 2.x seems like a good move there. >>>> +1 for the release plan >>>> >>>> I would like the move for user facing APIs to org.apache to be one of >> the last things we do. Translating clojure code into java and moving it to >> org.apache I am not too concerned about. >>>> >>>> Basti, >>>> We have two code bases that have diverged significantly from one >> another in terms of functionality. The storm code now or soon will have A >> Heartbeat Server, Nimbus HA (Different Implementation), Resource Aware >> Scheduling, a distributed cache like API, log searching, security, massive >> performance improvements, shaded almost all of our dependencies, a REST API >> for programtically accessing everything on the UI, and I am sure I am >> missing a few other things. JStorm also has many changes including cgroup >> isolation, restructured zookeeper layout, classpath isolation, and more too. >>>> No matter what we do it will be a large effort to port changes from >>>> one code base to another, and from clojure to java. I proposed this >>>> initially because it can be broken up into incremental changes. It >>>> may take a little longer, but we will always have a working codebase >>>> that is testable and compatible with the current storm release, at >>>> least until we move the user facing APIs to be under org.apache. >>>> This lets the community continue to build and test the master branch >>>> and report problems that they find, which is incredibly valuable. I >>>> personally don't think it will be much easier, especially if we are >>>> intent on always maintaining compatibility with storm. - Bobby >>>> >>>> >>>> On Wednesday, November 11, 2015 5:42 AM, 刘键(Basti Liu) < >> basti...@alibaba-inc.com> wrote: >>>> >>>> >>>> Hi Taylor, >>>> >>>> >>>> >>>> Thanks for the merge plan. I have a question about “Phase 2.2”. >>>> >>>> Do you mean community plan to create a fresh new “java core” based on >> current “clojure core” firstly, and then migrate the features from JStorm? >>>> >>>> If so, it confused me. It is really a huge job which might require a >> long developing time to make it stable, while JStorm is already a stable >> version. >>>> >>>> The release planned to be release after Nov 11th has already run online >> stably several month in Alibaba. >>>> >>>> Besides this, there are many valuable internal requirements in Alibaba, >> the fast evolution of JStorm is forseeable in next few months. >>>> >>>> If the “java core” is totally fresh new, it might bring many problems >> for the coming merge. >>>> >>>> So, from the point of this view, I think it is much better and easier >> to migrate the features of “clojure core” basing on JStorm for the “java >> core”. >>>> >>>> Please correct me, if any misunderstanding. >>>> >>>> >>>> >>>> Regards >>>> >>>> Basti >>>> >>>> >>>> >>>> 发件人: P. Taylor Goetz [mailto:ptgo...@gmail.com] >>>> 发送时间: 2015年11月11日 5:32 >>>> 收件人: dev@storm.apache.org >>>> 主题: [DISCUSS] Plan for Merging JStorm Code >>>> >>>> >>>> >>>> Based on a number of discussions regarding merging the JStorm code, >> I’ve tried to distill the ideas presented and inserted some of my own. The >> result is below. >>>> >>>> >>>> >>>> I’ve divided the plan into three phases, though they are not >> necessarily sequential — obviously some tasks can take place in parallel. >>>> >>>> >>>> >>>> None of this is set in stone, just presented for discussion. Any and >> all comments are welcome. >>>> >>>> >>>> >>>> ------- >>>> >>>> >>>> >>>> Phase 1 - Plan for 0.11.x Release >>>> >>>> 1. Determine feature set for 0.11.x and publish to wiki [1]. >>>> >>>> 2. Announce feature-freeze for 0.11.x >>>> >>>> 3. Create 0.11.x branch from master (Phase 2.4 can begin.) >>>> >>>> 4. Release 0.11.0 (or whatever version # we want to use) >>>> >>>> 5. Bug fixes and subsequent releases from 0.11.x-branch >>>> >>>> >>>> >>>> >>>> >>>> Phase 2 - Prepare for Merge ("master" and "jstorm-import" branches) >>>> >>>> 1. Determine/document unique features in JStorm (e.g. classpath >> isolation, cgroups, etc.) and create JIRA for migrating the feature. >>>> >>>> 2. Create JIRA for migrating each clojure component (or logical group >> of components) to Java. Assumes tests will be ported as well. >>>> >>>> 3. Discuss/establish style guide for Java coding conventions. Consider >> using Oracle’s or Google’s Java conventions as a base — they are both >> pretty solid. >>>> >>>> 4. align package names (e.g backtype.storm --> org.apache.storm / >>>> com.alibaba.jstorm --> org.apache.storm) (Phase 3 can begin) >>>> >>>> >>>> >>>> >>>> >>>> Phase 3 - Migrate Clojure --> Java >>>> >>>> 1. Port code/tests to Java, leveraging existing JStorm code wherever >> possible (core functionality only, features distinct to JStorm migrated >> separately). >>>> >>>> 2. Port JStorm-specific features. >>>> >>>> 3. Begin releasing preview/beta versions. >>>> >>>> 4. Code cleanup (across the board) and refactoring using established >>>> coding conventions, and leveraging PMD/Checkstyle reports for >>>> reference. (Note: good oportunity for new contributors.) >>>> >>>> 5. Release 0.12.0 (or whatever version # we want to use) and lift >> feature freeze. >>>> >>>> >>>> >>>> >>>> >>>> Notes: >>>> >>>> We should consider bumping up to version 1.0 sometime soon and then >> switching to semantic versioning [3] from then on. >>>> >>>> >>>> >>>> >>>> >>>> With the exception of package name alignment, the "jstorm-import" >> branch will largely be read-only throughout the process. >>>> >>>> >>>> >>>> During migration, it's probably easiest to operate with two local >> clones of the Apache Storm repo: one for working (i.e. checked out to >> working branch) and one for reference/copying (i.e. checked out to >> "jstorm-import"). >>>> >>>> >>>> >>>> Feature-freeze probably only needs to be enforced against core >> functionality. Components under "external" can likely be exempt, but we >> should figure out a process for accepting and releasing new features during >> the migration. >>>> >>>> >>>> >>>> Performance testing should be continuous throughout the process. Since >> we don't really have ASF infrastructure for performance testing, we will >> need a volunteer(s) to host and run the performance tests. Performance test >> results can be posted to the wiki [2]. It would probably be a good idea to >> establish a baseline with the 0.10.0 release. >>>> >>>> >>>> >>>> I’ve attached an analysis document Sean Zhong put together a while back >> to the JStorm merge JIRA [4]. The analysis was against the 0.9.3 release >> but is still relevant and has a lot of good information. >>>> >>>> >>>> >>>> >>>> >>>> [1] >>>> https://cwiki.apache.org/confluence/display/STORM/Release+0.11.0+Feat >>>> ure+Set >>>> >>>> [2] https://cwiki.apache.org/confluence/display/STORM/Storm+Home >>>> >>>> [3] http://semver.org >>>> >>>> [4] https://issues.apache.org/jira/browse/STORM-717 >>>> >>>> >>>> >>>> >>>> >>>> -Taylor >>>> >>>> >>>> >>>> >>>> >>>> >>> >>> >> >> >>
signature.asc
Description: Message signed with OpenPGP using GPGMail
