Re: [DISCUSS] Storm 2.0 Roadmap

Do we have a clear date to release Storm 2.0 beta version? I saw some users expecting to use Java8. BTW. Could somebody help to update the Storm site? The 2.0.0-SNAPSHOT documents should be updated. Thanks. - Xin 2017-06-29 23:27 GMT+08:00 Jungtaek Lim : > FYI I just gave

Re: [DISCUSS] Storm 2.0 Roadmap

FYI I just gave it a try with separating 1.x branch and 1.1.x branch (sure experimental in forked repo) https://github.com/heartsavior/storm/tree/1.1.x-branch-experimental I've updated CHANGELOG only once in that branch so you can see full of the changelog which contains the issues ported back.

Re: [DISCUSS] Storm 2.0 Roadmap

Hi Hugo, Thanks for your concerns about our troubles with the new storm-kafka-client. Our "bench" is based on our live production data of our cloud supervision system, collecting at least 1million metrics/min in our Kafka Brokers cluster (currently based on Kafka 0.10.1.0, with "compatibility

Re: [DISCUSS] Storm 2.0 Roadmap

Hi Alexandre, In my benchmarks the storm-kafka-client spout improves throughput by 70% and latency by 40% vs the storm-kafka implementation. I am surprised by your findings substantiating the opposite. Can you share your benchmark where you compare the performances of both implementations? As

Re: [DISCUSS] Storm 2.0 Roadmap

> On Jun 28, 2017, at 4:01 PM, Jungtaek Lim wrote: > > If my memory is right, when releasing 1.1.0 we postponed resolving some > critical issues for storm-kafka-client, and seems like it still haven't > been sorted out. I even think these issues can be effectively blocker for

Re: [DISCUSS] Storm 2.0 Roadmap

-1 to deprecate the old storm-kafka. I don't feel storm-kafka-client is stable given that it has some critical issues, and the module doesn't have enough volunteer committers to be stabilized faster as it should be. If my memory is right, when releasing 1.1.0 we postponed resolving some critical

Re: [DISCUSS] Storm 2.0 Roadmap

Hi Alexandre, Thanks for your input. I think we’re very much on the same page. The new Kafka spout needs to on par with the old one in terms of performance and stability before we even think about deprecation, let alone removal. I’ve heard a lot of complaints both publicly and privately about

Re: [DISCUSS] Storm 2.0 Roadmap

Hello, If that matters, our current experiences with StormKafkaClient isdisappointing (see my recent posts "Lag issues using Storm 1.1.1 latest build with StormKafkaClient 1.1.1 vs old StormKafka spouts" in this mailing list). Our current experience is that the old StormKafka spout always beats

Re: [DISCUSS] Storm 2.0 Roadmap

> On Jun 28, 2017, at 1:16 PM, Hugo Da Cruz Louro > wrote: > > I still need to go over the entire discussion thread in more detail, but one > thing I would like to bring up right way is the proposal to DEPRECATE, and > eventually remove, the KafkaSpout with the old

Re: [DISCUSS] Storm 2.0 Roadmap

> On Jun 28, 2017, at 10:48 AM, Bobby Evans wrote: > > +1. > If the 1.1 and 1.2 lines start to become difficult to maintain we can look at > putting them in maintenance mode too once we have a 2.x release. > I am a little nervous about merging a new feature into

Re: [DISCUSS] Storm 2.0 Roadmap

I still need to go over the entire discussion thread in more detail, but one thing I would like to bring up right way is the proposal to DEPRECATE, and eventually remove, the KafkaSpout with the old Kafka Consumer APIs. The storm-kafka-client KafkaSpout is getting stabilized, and I think we are

Re: [DISCUSS] Storm 2.0 Roadmap

+1. If the 1.1 and 1.2 lines start to become difficult to maintain we can look at putting them in maintenance mode too once we have a 2.x release. I am a little nervous about merging a new feature into 1.x branch without first going to master, but I hope that it will not be too much work to port

Re: [DISCUSS] Storm 2.0 Roadmap

That's great news that metrics work is ready! I'm +1 to Taylor's proposal, but in order to respect semantic versioning, I propose some modifications from Taylor's proposal: - create 1.1.x-branch with target version 1.1.1-SNAPSHOT and port back only bug fixes to the 1.1.x-branch - change the

Re: [DISCUSS] Storm 2.0 Roadmap

+1 for above stated approach on releasing 1.2.0 with metrics -Harsha On Tue, Jun 27, 2017, at 12:17 PM, P. Taylor Goetz wrote: > The work on metrics is ready for a pull request to 1.x-branch from the > feature branch. I’ve held off because we haven’t reached consensus on a > path forward with

Re: [DISCUSS] Storm 2.0 Roadmap

The work on metrics is ready for a pull request to 1.x-branch from the feature branch. I’ve held off because we haven’t reached consensus on a path forward with the 1.x release lines . I’d like to propose the following for the 1.x line: 1. Create a branch for 1.2 so we have a branch to review

Re: [DISCUSS] Storm 2.0 Roadmap

The new metrics APIs are not a breaking change. The only user-facing API change is the addition of methods to TopologyContext that allow for the registration of user-defined metrics [1]. -Taylor [1]

Re: [DISCUSS] Storm 2.0 Roadmap

Yes I would prefer to see us get a 2.x alpha release out.  Are the new metrics APIs going to be a breaking change?  I assume not, because we wouldn't want to put them in 1.x otherwise.  If that is the case then we don't need to wait for them before doing a 2.x release. - Bobby On Friday,

Re: [DISCUSS] Storm 2.0 Roadmap

Yes I prefer option 1, but it might depend on the progress of metrics V2. If it can be done within predictable near future I'm OK to pick option 2, but if not, we may be better to focus releasing 2.0.0 and make it really happen. Whichever we go, I feel it's time to track remaining work on Storm

Re: [DISCUSS] Storm 2.0 Roadmap

Bobby/Jungtaek, Are you saying you want to forego the 1.2 “metrics_v2” release and include it only in 2.0? (I ask because that work is already based on 1.x-branch, and forward-porting it to master is relatively simple.) I’d kind of like that work go out soon. If we go with option 1, I would

Re: [DISCUSS] Storm 2.0 Roadmap

I see 2 ways to address this. 1) We put the 1.x line into maintenance mode like with 0.10.  We don't backport anything except bug fixes.2) We backport a lot of the backwards compatible changes from 2.x to 1.x. My personal preference is 1.  It makes it clear the direction we want to go in.  The

Re: [DISCUSS] Storm 2.0 Roadmap

I'd like to bump this again instead of initiating new discussion thread. I had having hard time to create and apply pull requests for both master and 1.x-branch and that's really painful and sometimes blocker for me to do merge step. Two branches are heavily diverged more than between 0.10 and

Re: [DISCUSS] Storm 2.0 Roadmap

+1 for Roshan's suggestion : in our Storm 1.x based supervision system, we're very interested anything that can provide better throughput. 2017-06-03 18:12 GMT+02:00 Roshan Naik : > For 2.0 beta … it would be good to incorporate some of the Worker > improvements

Re: [DISCUSS] Storm 2.0 Roadmap

For 2.0 beta … it would be good to incorporate some of the Worker improvements (STORM-2284) IMO. Changes to messaging subsystem can be delivered sooner and my in-progress implementation suggests that it will yield substantial latency improvements. The 2.0 beta phase would really help kick the

Re: [DISCUSS] Storm 2.0 Roadmap

I also would love to see metrics V2 code sooner or later too. If we can get it before releasing 2.0.0 that will be great, and then maybe we could just move toward to 2.0.0, not adding any improvements to 1.x version line. (And that's what I would want to.) If we would really want to have 1.2.0, I

Re: [DISCUSS] Storm 2.0 Roadmap

I would love to see the metrics V2 code come out sooner rather than later.  +1. My biggest blocker for a 1.x release is https://github.com/apache/storm/pull/2142  Even though the pull request says it is minor it showed that we messed up pushing back some changes for pacemaker to open source

Re: [DISCUSS] Storm 2.0 Roadmap

I’d like to bump this thread and start a discussion around our next release. Here are my thoughts. There are a number of important fixes in 1.x-branch so I’d like to consider releasing 1.1.1 soon. I’d appreciate input on any open issues that should be resolved for that release. I’d like us to

Re: [DISCUSS] Storm 2.0 Roadmap

Maybe related to or not, but would we want to create a new branch "1.1.x-branch", and make "1.x-branch" target for 1.2? I'm not clear we don't release 1.2 for moving toward to 2.0.0, so hence the question. - Jungtaek Lim (HeartSaVioR) 2017년 3월 29일 (수) 오전 1:56, Hugo Da Cruz Louro

Re: [DISCUSS] Storm 2.0 Roadmap

+1 for finishing the porting to Java ahead of anything else - it will be a significant milestone. I have a JIRA assigned concerning to the porting. I will work on it for the 2.0 release. it’s a priority to guarantee no performance regressions. As part of this endeavor, explore an automated (or

Re: [DISCUSS] Storm 2.0 Roadmap

+1 to release with the porting completed. I think its mainly the UI server and log viewer that’s pending. We can start doing the regression and performance tests for whatever is already ported. If anyone is running the master branch in their pre-prod / prod environments, it will be good to

Re: [DISCUSS] Storm 2.0 Roadmap

+1 to have 2.0 with porting and performance(it should be at least as good as 1.x release) issues addressed We can target other tasks(mentioned by Taylor and Jungtaek) for 2.x-branch. Exactly-once support: While thinking through the exactlyonce support design, it is realized better to avoid

Re: [DISCUSS] Storm 2.0 Roadmap

Storm 2.0 migration to java in itself is a big win and would attract wider community and adoption. So my vote would be to resolve the first 3 items to get a release out. All the other featured mentioned are great to have but shouldn't be blockers for 2.0 release. -Harsha On Thu, Mar 23, 2017 at

[DISCUSS] Storm 2.0 Roadmap

With the 1.1.0 release nearing completion, I’d like to turn our attention to 2.0 and develop a plan for what features, etc. to include. The following 3 are what I feel are the minimum for a 2.0 release. These could likely be resolved relatively quickly: * Performance — I’ve not benchmarked the