UPDATE: Submitted a pull request https://github.com/apache/storm/pull/2549 for STORM-2936 (against 1.1.x-branch)
Erik, please change the status to "IN PROGRESS" if someone is working on. I would find the free time and just do it if there's no one working in progress. 2018년 2월 6일 (화) 오전 10:39, Jungtaek Lim <[email protected]>님이 작성: > Thanks for quick response Erik! > > Just filed two issues : > https://issues.apache.org/jira/browse/STORM-2936 (for 1.1.x-branch) > https://issues.apache.org/jira/browse/STORM-2937 (for 1.0.x-branch) > > We have another discussion around making storm-kafka-client be experiment > of managing separately (independent of Storm release). So the three > versions which are in release phase might be the last releases of > "battery-included" of storm-kafka-client if our experiment works well. > > If we would want to make the change for storm-kafka-client, it might be > better to put the change and release before start experimenting, but that's > just a thought on my own. In opposite way, we could even start experiment > and make change of storm-core of 1.0.x-branch to be compatible with that > version of storm-kafka-client. We could even do it for 1.1.x-branch, but > the change is almost done so it doesn't look like needed to postpone it. > > Would like to here everyone's voice on this. > > -Jungtaek Lim (HeartSaVioR) > > 2018년 2월 6일 (화) 오전 10:23, Erik Weathers <[email protected]>님이 > 작성: > >> Thanks for the quick response Jungtaek! >> >> Yes, my teammates and myself would like to help on this. Is there an >> existing JIRA for the work you've been doing on the other branches? >> >> I propose we don't make this block 1.0.6 -- we can just release 1.0.7 >> quickly when the backport is done, if that is amenable. >> That strategy also might be cleaner since it would avoid other changes in >> 1.0.6 being lumped together with this. >> >> - Erik >> >> On Mon, Feb 5, 2018 at 5:16 PM, Jungtaek Lim <[email protected]> wrote: >> >> > UPDATE: I've finished working on overwriting storm-kafka-client >> 1.x-branch >> > to 1.1.x-branch. Not yet pushed to ASF git, but pushed to my fork first >> to >> > trigger Travis CI to see how the build goes well. >> > >> > https://github.com/HeartSaVioR/storm/commit/76b8a7d3a6f91e66 >> > 612e87da8589f5723f05218a >> > https://travis-ci.org/HeartSaVioR/storm/builds/337819430 >> > >> > Thanks for the input regarding 1.0.x version, Erik. I guess then we >> have no >> > alternative here: someone has to fix storm-kafka-client as well as >> > storm-core, since including shaded storm-core doesn't make sense for >> > official Storm release. >> > >> > I guess it doesn't take many hour(s), hence may not worth to sync and >> talk >> > offline. I just wanted to judge whether we are OK to make change of >> > storm-core in bugfix version lines, but maybe the judgement itself can >> be >> > possible after finishing the change, so I'll just go ahead making the >> > change. >> > Since this is blocking release candidate, we should get it ASAP. That's >> why >> > I'm eager to go ahead making the change. If you could spend time now >> > helping with making the change ASAP, please leave short notice (maybe >> with >> > JIRA issue?) and go ahead. >> > >> > Thanks, >> > Jungtaek Lim (HeartSaVioR) >> > >> > 2018년 2월 6일 (화) 오전 9:41, Erik Weathers <[email protected] >> >님이 >> > 작성: >> > >> > > hey Jungtaek, >> > > >> > > Thanks for continuing to pursue this! >> > > >> > > The issue for Storm not working on Mesos is due to a fundamental >> change >> > to >> > > the core scheduling logic in Storm: >> > > >> > > - >> > > >> > > https://issues.apache.org/jira/browse/STORM-2126?focusedComm >> > entId=16136150&page=com.atlassian.jira.plugin.system. >> > issuetabpanels%3Acomment-tabpanel#comment-16136150 >> > > >> > > The yet-to-be-ironed-out solution that Bobby was brainstorming about >> > isn't >> > > a short term fix as far as I understand it. I believe it to be many >> many >> > > months (years?) out for it to actually be workable. Per my naive >> > > understanding of the proposal, we'd probably have to completely >> rewrite >> > the >> > > Storm-on-Mesos framework. So it's probably the right long-term >> solution, >> > > but it isn't anything that should impact this discussion. >> > > >> > > > The thing is, even users pick storm-kafka-client 1.1.x/1.2.0 and >> > include >> > > it into their topology jar, it will also not work with Storm 1.0.x. It >> > > even can't >> > > compile. >> > > >> > > FWIW, I'm pretty sure that I was able to successfully run >> > > storm-kafka-client-1.1.x on a 1.0.5 storm cluster, but only after >> shading >> > > in storm-core-1.1.x to the topology uber jar. There was *at least* a >> > > change to some timer-related class in storm-core in 1.1.x (something >> > about >> > > milliseconds IIRC -- it's been 1.5 months since I did it, need to >> revisit >> > > the process I followed). >> > > >> > > I'm happy to help with backporting / stomping storm-kafka-client in >> > 1.0.x. >> > > Maybe we can talk offline about it? >> > > >> > > - Erik >> > > >> > > On Mon, Feb 5, 2018 at 4:20 PM, Jungtaek Lim <[email protected]> >> wrote: >> > > >> > > > UPDATE: Looks like we changed some parts of storm-core while fixing >> > > > storm-kafka-client issues (especially went in 1.1.0), hence >> overwriting >> > > > also incurs changes of storm-core. It doesn't look like a big deal >> for >> > > > 1.1.x-branch, but there looks like needed many changes for >> > 1.0.x-branch. >> > > > >> > > > The thing is, even users pick storm-kafka-client 1.1.x/1.2.0 and >> > include >> > > it >> > > > into their topology jar, it will also not work with Storm 1.0.x. It >> > even >> > > > can't compile. >> > > > >> > > > 1.0.x version line was long lived (22 months) even we released Storm >> > > 1.1.0 >> > > > at 11 months ago. Instead of struggling 1.0.x-branch to up to date, >> I'd >> > > > like to suggest that we define 1.0.x-branch as deprecated with >> guiding >> > to >> > > > update to latest 1.1.x version or 1.2.0 (after release), and try to >> > > resolve >> > > > storm-mesos issue with Storm 1.1.0 ASAP to resolve Erik's concern. >> > > > >> > > > Makes sense? I'll continue working on 1.1.x-branch and update >> anyway. >> > > > >> > > > -Jungtaek Lim (HeartSaVioR) >> > > > >> > > > 2018년 2월 6일 (화) 오전 7:53, Jungtaek Lim <[email protected]>님이 작성: >> > > > >> > > > > OK. No more opinion/vote in 5 days. I'll treat consensus was made, >> > and >> > > go >> > > > > ahead making change: overwrite storm-kafka-client 1.2.0 to two >> > branches >> > > > > 1.1.x/1.0.x. >> > > > > >> > > > > -Jungtaek Lim (HeartSaVioR) >> > > > > >> > > > > 2018년 2월 1일 (목) 오전 10:48, Jungtaek Lim <[email protected]>님이 작성: >> > > > > >> > > > >> This discussion got 4 +1 (binding) and no -1. Moreover two active >> > > > >> maintainers for storm-kafka-client (Hugo and Stig) voted +1. >> > > > >> >> > > > >> Do we want to hold on for hearing more voices, or treating above >> > > > opinions >> > > > >> as consensus and reflect the change? >> > > > >> >> > > > >> Btw, I think we need to sort out the sequences between two >> topics: >> > > > >> separating storm-kafka-client as independent release cycle, and >> > this. >> > > I >> > > > >> guess some of us agreed former topic doesn't related to current >> RC, >> > > but >> > > > I >> > > > >> think this topic can be (should be) reflected to current RC >> ongoing. >> > > > >> >> > > > >> -Jungtaek Lim (HeartSaVioR) >> > > > >> >> > > > >> 2018년 2월 1일 (목) 오전 4:08, Hugo Da Cruz Louro < >> [email protected] >> > > >님이 >> > > > >> 작성: >> > > > >> >> > > > >>> +1 to replace storm-kafka-client in 1.0.x branch. >> > > > >>> Hugo >> > > > >>> >> > > > >>> > On Jan 31, 2018, at 11:03 AM, Stig Rohde Døssing < >> > > > >>> [email protected]> wrote: >> > > > >>> > >> > > > >>> > +1 to replace storm-kafka-client in 1.0.x branch. Breaking >> > semantic >> > > > >>> > versioning is really nasty, but I think it is the lesser evil >> in >> > > this >> > > > >>> case. >> > > > >>> > >> > > > >>> > 2018-01-31 5:14 GMT+01:00 Harsha <[email protected]>: >> > > > >>> > >> > > > >>> >> +1 to replace storm-kafka-client in 1.0.x branch >> > > > >>> >> -Harsha >> > > > >>> >> On Tue, Jan 30, 2018, at 7:04 PM, Jungtaek Lim wrote: >> > > > >>> >>> Bump up this thread so that we could reach consensus >> earlier. >> > > Given >> > > > >>> that >> > > > >>> >> we >> > > > >>> >>> got concern related to this, I think it is ideal to release >> > > > >>> 1.1.x/1.0.x >> > > > >>> >>> with making decision and applying the change if we want. >> > > > >>> >>> >> > > > >>> >>> 2018년 1월 30일 (화) 오전 9:25, Jungtaek Lim <[email protected] >> >님이 >> > 작성: >> > > > >>> >>> >> > > > >>> >>>> Erik's concern brought from 1.0.6 RC1, because they can't >> use >> > > > Storm >> > > > >>> >> 1.1.0 >> > > > >>> >>>> or higher (Storm 1.1.0 broke storm-mesos.). While he could >> > take >> > > an >> > > > >>> >>>> workaround to use storm-kafka-client 1.2.0 or 1.1.2 (if we >> > > decide >> > > > to >> > > > >>> >>>> replace) with Storm 1.0.6, it would be better if we don't >> > allow >> > > > >>> leaving >> > > > >>> >>>> storm-kafka-client in 1.0.x in inconsistent state. >> > > > >>> >>>> >> > > > >>> >>>> IMHO, breaking backward compatibility is worse, but leaving >> > > broken >> > > > >>> >> thing >> > > > >>> >>>> is worst. Hence I'm +1 to replace all, with noticing that >> it >> > may >> > > > >>> bring >> > > > >>> >>>> backward incompatibility in release announce. >> > > > >>> >>>> >> > > > >>> >>>> -Jungtaek Lim (HeartSaVioR) >> > > > >>> >>>> >> > > > >>> >>>> 2018년 1월 30일 (화) 오전 4:49, P. Taylor Goetz < >> [email protected] >> > >님이 >> > > > 작성: >> > > > >>> >>>> >> > > > >>> >>>>> As I mentioned else thread I’m open to this but would >> defer >> > to >> > > > >>> >> community >> > > > >>> >>>>> consensus. >> > > > >>> >>>>> >> > > > >>> >>>>> If there’s concern about doing this for 1.0.x, one option >> > would >> > > > be >> > > > >>> >> skip >> > > > >>> >>>>> that version line and only apply it to 1.2.0 and 1.1.x. >> > > > >>> >>>>> >> > > > >>> >>>>> -Taylor >> > > > >>> >>>>> >> > > > >>> >>>>>> On Jan 29, 2018, at 12:12 AM, Jungtaek Lim < >> > [email protected] >> > > > >> > > > >>> >> wrote: >> > > > >>> >>>>>> >> > > > >>> >>>>>> Hi devs, >> > > > >>> >>>>>> >> > > > >>> >>>>>> This is initial post to separate out discussion topic >> from >> > > vote >> > > > >>> >> thread, >> > > > >>> >>>>> and >> > > > >>> >>>>>> continue discussing. >> > > > >>> >>>>>> >> > > > >>> >>>>>> Background of the topic: >> > > > >>> >>>>>> 1. Only 1.x-branch of storm-kafka-client got stabilized. >> > > > >>> >> (relatively) >> > > > >>> >>>>>> 2. We would avoid to port back patches to 1.1.x and 1.0.x >> > > > because >> > > > >>> >>>>> they're >> > > > >>> >>>>>> diverged too much. >> > > > >>> >>>>>> >> > > > >>> >>>>>> Downside: >> > > > >>> >>>>>> Backward compatibility might be broken for 1.1.x and >> 1.0.x. >> > > Not >> > > > >>> >> sure for >> > > > >>> >>>>>> 1.1.x, but at least 1.0.x, since supported Kafka client >> > > version >> > > > is >> > > > >>> >>>>>> different, and if my memory is right, we already applied >> > > > backward >> > > > >>> >>>>>> incompatible change into storm-kafka-client 1.1.0. >> > > > >>> >>>>>> >> > > > >>> >>>>>> Please put your opinion regarding topic. You're >> encouraged >> > to >> > > > copy >> > > > >>> >> your >> > > > >>> >>>>>> previous post in vote thread which helps to centralize >> > > opinions >> > > > in >> > > > >>> >>>>> current >> > > > >>> >>>>>> thread. >> > > > >>> >>>>>> >> > > > >>> >>>>>> Thanks, >> > > > >>> >>>>>> Jungtaek Lim (HeartSaVioR) >> > > > >>> >>>>> >> > > > >>> >>>>> >> > > > >>> >> >> > > > >>> >> > > > >>> >> > > > >> > > >> > >> >
