Taylor,
Can you add me to the wiki as well my id is "harsha".
Thanks,
Harsha
On Fri, Nov 20, 2015, at 01:19 AM, Longda Feng wrote:
> @Taylor, I want to work with Bobby, my wiki name is "longda".
> ThanksLongda------------------------------------------------------------------From:Bobby
> Evans <[email protected]>Send Time:2015年11月20日(星期五) 06:54To:P.
> Taylor Goetz <[email protected]>，[email protected]
> <[email protected]>Subject:Re: [DISCUSS] Plan for Merging JStorm Code
> Sorry about this taking so long. I am revans2 on the cwiki. - Bobby
>
>
> On Wednesday, November 18, 2015 4:24 PM, P. Taylor Goetz
> <[email protected]> wrote:
>
>
> All I have at this point is a placeholder wiki entry [1], and a lot of local
> notes that likely would only make sense to me.
>
> Let me know your wiki username and I’ll give you permissions. The same
> goes for anyone else who wants to help.
>
> -Taylor
>
> [1]
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61328109
>
> > On Nov 18, 2015, at 2:08 PM, Bobby Evans <[email protected]>
> >wrote:
> >
> > Taylor and others I was hoping to get started filing JIRA and planning on
> >how we are going to do the java migration + JStorm merger. Is anyone else
> >starting to do this? If not would anyone object to me starting on it? -
> >Bobby
> >
> >
> > On Thursday, November 12, 2015 12:04 PM, P. Taylor Goetz
> ><[email protected]> wrote:
> >
> >
> > Thanks for putting this together Basti, that comparison helps a lot.
> >
> > And thanks Bobby for converting it into markdown. I was going to just
> >attach the spreadsheet to JIRA, but markdown is a much better solution.
> >
> > -Taylor
> >
> >> On Nov 12, 2015, at 12:03 PM, Bobby Evans <[email protected]>
> >>wrote:
> >>
> >> I translated the excel spreadsheet into a markdown file and put up a pull
> >>request for it.
> >> https://github.com/apache/storm/pull/877
> >> I did a few edits to it to make it work with Markdown, and to add in a few
> >>of my own comments. I also put in a field for JIRAs to be able to track
> >>the migration.
> >> Overall I think your evaluation was very good. We have a fair amount of
> >>work ahead of us to decide what version of various features we want to go
> >>forward with.
> >> - Bobby
> >>
> >>
> >> On Thursday, November 12, 2015 9:37 AM, 刘键(Basti Liu)
> >><[email protected]> wrote:
> >>
> >>
> >> Hi Bobby & Jungtaek,
> >>
> >> Thanks for your replay.
> >> I totally agree that compatibility is the most important thing. Actually,
> >>JStorm has been compatible with the user API of Storm.
> >> As you mentioned below, we indeed still have some features different
> >>between Storm and JStorm. I have tried to list them (minor update or
> >>improvements are not included).
> >> Please refer to attachment for details. If any missing, please help to
> >>point out. (The current working features are probably missing here.)
> >> Just have a look at these differences. For the missing features in JStorm,
> >>I did not see any obstacle which will block the merge to JStorm.
> >> For the features which has different solution between Storm and JStorm, we
> >>can evaluate the solution one by one to decision which one is appropriate.
> >> After the finalization of evaluation, I think JStorm team can take the
> >>merging job and publish a stable release in 2 months.
> >> But anyway, the detailed implementation for these features with different
> >>solution is transparent to user. So, from user's point of view, there is
> >>not any compatibility problem.
> >>
> >> Besides compatibility, by our experience, stability is also important and
> >>is not an easy job. 4 people in JStorm team took almost one year to finish
> >>the porting from "clojure core"
> >> to "java core", and to make it stable. Of course, we have many devs in
> >>community to make the porting job faster. But it still needs a long time to
> >>run many online complex topologys to find bugs and fix them. So, that is
> >>the reason why I proposed to do merging and build on a stable "java core".
> >>
> >> -----Original Message-----
> >> From: Bobby Evans [mailto:[email protected]]
> >> Sent: Wednesday, November 11, 2015 10:51 PM
> >> To: [email protected]
> >> Subject: Re: [DISCUSS] Plan for Merging JStorm Code
> >>
> >> +1 for doing a 1.0 release based off of the clojure 0.11.x code.
> >>Migrating the APIs to org.apache.storm is a big non-backwards compatible
> >>move, and a major version bump to 2.x seems like a good move there.
> >> +1 for the release plan
> >>
> >> I would like the move for user facing APIs to org.apache to be one of the
> >>last things we do. Translating clojure code into java and moving it to
> >>org.apache I am not too concerned about.
> >>
> >> Basti,
> >> We have two code bases that have diverged significantly from one another
> >>in terms of functionality. The storm code now or soon will have A
> >>Heartbeat Server, Nimbus HA (Different Implementation), Resource Aware
> >>Scheduling, a distributed cache like API, log searching, security, massive
> >>performance improvements, shaded almost all of our dependencies, a REST API
> >>for programtically accessing everything on the UI, and I am sure I am
> >>missing a few other things. JStorm also has many changes including cgroup
> >>isolation, restructured zookeeper layout, classpath isolation, and more too.
> >> No matter what we do it will be a large effort to port changes from one
> >>code base to another, and from clojure to java. I proposed this initially
> >>because it can be broken up into incremental changes. It may take a little
> >>longer, but we will always have a working codebase that is testable and
> >>compatible with the current storm release, at least until we move the user
> >>facing APIs to be under org.apache. This lets the community continue to
> >>build and test the master branch and report problems that they find, which
> >>is incredibly valuable. I personally don't think it will be much easier,
> >>especially if we are intent on always maintaining compatibility with storm.
> >>- Bobby
> >>
> >>
> >> On Wednesday, November 11, 2015 5:42 AM, 刘键(Basti Liu)
> >><[email protected]> wrote:
> >>
> >>
> >> Hi Taylor,
> >>
> >>
> >>
> >> Thanks for the merge plan. I have a question about “Phase 2.2”.
> >>
> >> Do you mean community plan to create a fresh new “java core” based on
> >>current “clojure core” firstly, and then migrate the features from JStorm?
> >>
> >> If so, it confused me. It is really a huge job which might require a long
> >>developing time to make it stable, while JStorm is already a stable version.
> >>
> >> The release planned to be release after Nov 11th has already run online
> >>stably several month in Alibaba.
> >>
> >> Besides this, there are many valuable internal requirements in Alibaba,
> >>the fast evolution of JStorm is forseeable in next few months.
> >>
> >> If the “java core” is totally fresh new, it might bring many problems for
> >>the coming merge.
> >>
> >> So, from the point of this view, I think it is much better and easier to
> >>migrate the features of “clojure core” basing on JStorm for the “java core”.
> >>
> >> Please correct me, if any misunderstanding.
> >>
> >>
> >>
> >> Regards
> >>
> >> Basti
> >>
> >>
> >>
> >> 发件人: P. Taylor Goetz [mailto:[email protected]]
> >> 发送时间: 2015年11月11日 5:32
> >> 收件人: [email protected]
> >> 主题: [DISCUSS] Plan for Merging JStorm Code
> >>
> >>
> >>
> >> Based on a number of discussions regarding merging the JStorm code, I’ve
> >>tried to distill the ideas presented and inserted some of my own. The
> >>result is below.
> >>
> >>
> >>
> >> I’ve divided the plan into three phases, though they are not necessarily
> >>sequential — obviously some tasks can take place in parallel.
> >>
> >>
> >>
> >> None of this is set in stone, just presented for discussion. Any and all
> >>comments are welcome.
> >>
> >>
> >>
> >> -------
> >>
> >>
> >>
> >> Phase 1 - Plan for 0.11.x Release
> >>
> >> 1. Determine feature set for 0.11.x and publish to wiki [1].
> >>
> >> 2. Announce feature-freeze for 0.11.x
> >>
> >> 3. Create 0.11.x branch from master (Phase 2.4 can begin.)
> >>
> >> 4. Release 0.11.0 (or whatever version # we want to use)
> >>
> >> 5. Bug fixes and subsequent releases from 0.11.x-branch
> >>
> >>
> >>
> >>
> >>
> >> Phase 2 - Prepare for Merge ("master" and "jstorm-import" branches)
> >>
> >> 1. Determine/document unique features in JStorm (e.g. classpath isolation,
> >>cgroups, etc.) and create JIRA for migrating the feature.
> >>
> >> 2. Create JIRA for migrating each clojure component (or logical group of
> >>components) to Java. Assumes tests will be ported as well.
> >>
> >> 3. Discuss/establish style guide for Java coding conventions. Consider
> >>using Oracle’s or Google’s Java conventions as a base — they are both
> >>pretty solid.
> >>
> >> 4. align package names (e.g backtype.storm --> org.apache.storm /
> >>com.alibaba.jstorm --> org.apache.storm) (Phase 3 can begin)
> >>
> >>
> >>
> >>
> >>
> >> Phase 3 - Migrate Clojure --> Java
> >>
> >> 1. Port code/tests to Java, leveraging existing JStorm code wherever
> >>possible (core functionality only, features distinct to JStorm migrated
> >>separately).
> >>
> >> 2. Port JStorm-specific features.
> >>
> >> 3. Begin releasing preview/beta versions.
> >>
> >> 4. Code cleanup (across the board) and refactoring using established
> >>coding conventions, and leveraging PMD/Checkstyle reports for reference.
> >>(Note: good oportunity for new contributors.)
> >>
> >> 5. Release 0.12.0 (or whatever version # we want to use) and lift feature
> >>freeze.
> >>
> >>
> >>
> >>
> >>
> >> Notes:
> >>
> >> We should consider bumping up to version 1.0 sometime soon and then
> >>switching to semantic versioning [3] from then on.
> >>
> >>
> >>
> >>
> >>
> >> With the exception of package name alignment, the "jstorm-import" branch
> >>will largely be read-only throughout the process.
> >>
> >>
> >>
> >> During migration, it's probably easiest to operate with two local clones
> >>of the Apache Storm repo: one for working (i.e. checked out to working
> >>branch) and one for reference/copying (i.e. checked out to "jstorm-import").
> >>
> >>
> >>
> >> Feature-freeze probably only needs to be enforced against core
> >>functionality. Components under "external" can likely be exempt, but we
> >>should figure out a process for accepting and releasing new features during
> >>the migration.
> >>
> >>
> >>
> >> Performance testing should be continuous throughout the process. Since we
> >>don't really have ASF infrastructure for performance testing, we will need
> >>a volunteer(s) to host and run the performance tests. Performance test
> >>results can be posted to the wiki [2]. It would probably be a good idea to
> >>establish a baseline with the 0.10.0 release.
> >>
> >>
> >>
> >> I’ve attached an analysis document Sean Zhong put together a while back to
> >>the JStorm merge JIRA [4]. The analysis was against the 0.9.3 release but
> >>is still relevant and has a lot of good information.
> >>
> >>
> >>
> >>
> >>
> >> [1]
> >>https://cwiki.apache.org/confluence/display/STORM/Release+0.11.0+Feature+Set
> >>
> >> [2] https://cwiki.apache.org/confluence/display/STORM/Storm+Home
> >>
> >> [3] http://semver.org
> >>
> >> [4] https://issues.apache.org/jira/browse/STORM-717
> >>
> >>
> >>
> >>
> >>
> >> -Taylor
> >>
> >>
> >>
> >>
> >>
> >>
> >
> >
>
>
>
