Thanks for quick response Erik! Just filed two issues : https://issues.apache.org/jira/browse/STORM-2936 (for 1.1.x-branch) https://issues.apache.org/jira/browse/STORM-2937 (for 1.0.x-branch)
We have another discussion around making storm-kafka-client be experiment of managing separately (independent of Storm release). So the three versions which are in release phase might be the last releases of "battery-included" of storm-kafka-client if our experiment works well. If we would want to make the change for storm-kafka-client, it might be better to put the change and release before start experimenting, but that's just a thought on my own. In opposite way, we could even start experiment and make change of storm-core of 1.0.x-branch to be compatible with that version of storm-kafka-client. We could even do it for 1.1.x-branch, but the change is almost done so it doesn't look like needed to postpone it. Would like to here everyone's voice on this. -Jungtaek Lim (HeartSaVioR) 2018년 2월 6일 (화) 오전 10:23, Erik Weathers <[email protected]>님이 작성: > Thanks for the quick response Jungtaek! > > Yes, my teammates and myself would like to help on this. Is there an > existing JIRA for the work you've been doing on the other branches? > > I propose we don't make this block 1.0.6 -- we can just release 1.0.7 > quickly when the backport is done, if that is amenable. > That strategy also might be cleaner since it would avoid other changes in > 1.0.6 being lumped together with this. > > - Erik > > On Mon, Feb 5, 2018 at 5:16 PM, Jungtaek Lim <[email protected]> wrote: > > > UPDATE: I've finished working on overwriting storm-kafka-client > 1.x-branch > > to 1.1.x-branch. Not yet pushed to ASF git, but pushed to my fork first > to > > trigger Travis CI to see how the build goes well. > > > > https://github.com/HeartSaVioR/storm/commit/76b8a7d3a6f91e66 > > 612e87da8589f5723f05218a > > https://travis-ci.org/HeartSaVioR/storm/builds/337819430 > > > > Thanks for the input regarding 1.0.x version, Erik. I guess then we have > no > > alternative here: someone has to fix storm-kafka-client as well as > > storm-core, since including shaded storm-core doesn't make sense for > > official Storm release. > > > > I guess it doesn't take many hour(s), hence may not worth to sync and > talk > > offline. I just wanted to judge whether we are OK to make change of > > storm-core in bugfix version lines, but maybe the judgement itself can be > > possible after finishing the change, so I'll just go ahead making the > > change. > > Since this is blocking release candidate, we should get it ASAP. That's > why > > I'm eager to go ahead making the change. If you could spend time now > > helping with making the change ASAP, please leave short notice (maybe > with > > JIRA issue?) and go ahead. > > > > Thanks, > > Jungtaek Lim (HeartSaVioR) > > > > 2018년 2월 6일 (화) 오전 9:41, Erik Weathers <[email protected]>님이 > > 작성: > > > > > hey Jungtaek, > > > > > > Thanks for continuing to pursue this! > > > > > > The issue for Storm not working on Mesos is due to a fundamental change > > to > > > the core scheduling logic in Storm: > > > > > > - > > > > > > https://issues.apache.org/jira/browse/STORM-2126?focusedComm > > entId=16136150&page=com.atlassian.jira.plugin.system. > > issuetabpanels%3Acomment-tabpanel#comment-16136150 > > > > > > The yet-to-be-ironed-out solution that Bobby was brainstorming about > > isn't > > > a short term fix as far as I understand it. I believe it to be many > many > > > months (years?) out for it to actually be workable. Per my naive > > > understanding of the proposal, we'd probably have to completely rewrite > > the > > > Storm-on-Mesos framework. So it's probably the right long-term > solution, > > > but it isn't anything that should impact this discussion. > > > > > > > The thing is, even users pick storm-kafka-client 1.1.x/1.2.0 and > > include > > > it into their topology jar, it will also not work with Storm 1.0.x. It > > > even can't > > > compile. > > > > > > FWIW, I'm pretty sure that I was able to successfully run > > > storm-kafka-client-1.1.x on a 1.0.5 storm cluster, but only after > shading > > > in storm-core-1.1.x to the topology uber jar. There was *at least* a > > > change to some timer-related class in storm-core in 1.1.x (something > > about > > > milliseconds IIRC -- it's been 1.5 months since I did it, need to > revisit > > > the process I followed). > > > > > > I'm happy to help with backporting / stomping storm-kafka-client in > > 1.0.x. > > > Maybe we can talk offline about it? > > > > > > - Erik > > > > > > On Mon, Feb 5, 2018 at 4:20 PM, Jungtaek Lim <[email protected]> > wrote: > > > > > > > UPDATE: Looks like we changed some parts of storm-core while fixing > > > > storm-kafka-client issues (especially went in 1.1.0), hence > overwriting > > > > also incurs changes of storm-core. It doesn't look like a big deal > for > > > > 1.1.x-branch, but there looks like needed many changes for > > 1.0.x-branch. > > > > > > > > The thing is, even users pick storm-kafka-client 1.1.x/1.2.0 and > > include > > > it > > > > into their topology jar, it will also not work with Storm 1.0.x. It > > even > > > > can't compile. > > > > > > > > 1.0.x version line was long lived (22 months) even we released Storm > > > 1.1.0 > > > > at 11 months ago. Instead of struggling 1.0.x-branch to up to date, > I'd > > > > like to suggest that we define 1.0.x-branch as deprecated with > guiding > > to > > > > update to latest 1.1.x version or 1.2.0 (after release), and try to > > > resolve > > > > storm-mesos issue with Storm 1.1.0 ASAP to resolve Erik's concern. > > > > > > > > Makes sense? I'll continue working on 1.1.x-branch and update anyway. > > > > > > > > -Jungtaek Lim (HeartSaVioR) > > > > > > > > 2018년 2월 6일 (화) 오전 7:53, Jungtaek Lim <[email protected]>님이 작성: > > > > > > > > > OK. No more opinion/vote in 5 days. I'll treat consensus was made, > > and > > > go > > > > > ahead making change: overwrite storm-kafka-client 1.2.0 to two > > branches > > > > > 1.1.x/1.0.x. > > > > > > > > > > -Jungtaek Lim (HeartSaVioR) > > > > > > > > > > 2018년 2월 1일 (목) 오전 10:48, Jungtaek Lim <[email protected]>님이 작성: > > > > > > > > > >> This discussion got 4 +1 (binding) and no -1. Moreover two active > > > > >> maintainers for storm-kafka-client (Hugo and Stig) voted +1. > > > > >> > > > > >> Do we want to hold on for hearing more voices, or treating above > > > > opinions > > > > >> as consensus and reflect the change? > > > > >> > > > > >> Btw, I think we need to sort out the sequences between two topics: > > > > >> separating storm-kafka-client as independent release cycle, and > > this. > > > I > > > > >> guess some of us agreed former topic doesn't related to current > RC, > > > but > > > > I > > > > >> think this topic can be (should be) reflected to current RC > ongoing. > > > > >> > > > > >> -Jungtaek Lim (HeartSaVioR) > > > > >> > > > > >> 2018년 2월 1일 (목) 오전 4:08, Hugo Da Cruz Louro < > [email protected] > > > >님이 > > > > >> 작성: > > > > >> > > > > >>> +1 to replace storm-kafka-client in 1.0.x branch. > > > > >>> Hugo > > > > >>> > > > > >>> > On Jan 31, 2018, at 11:03 AM, Stig Rohde Døssing < > > > > >>> [email protected]> wrote: > > > > >>> > > > > > >>> > +1 to replace storm-kafka-client in 1.0.x branch. Breaking > > semantic > > > > >>> > versioning is really nasty, but I think it is the lesser evil > in > > > this > > > > >>> case. > > > > >>> > > > > > >>> > 2018-01-31 5:14 GMT+01:00 Harsha <[email protected]>: > > > > >>> > > > > > >>> >> +1 to replace storm-kafka-client in 1.0.x branch > > > > >>> >> -Harsha > > > > >>> >> On Tue, Jan 30, 2018, at 7:04 PM, Jungtaek Lim wrote: > > > > >>> >>> Bump up this thread so that we could reach consensus earlier. > > > Given > > > > >>> that > > > > >>> >> we > > > > >>> >>> got concern related to this, I think it is ideal to release > > > > >>> 1.1.x/1.0.x > > > > >>> >>> with making decision and applying the change if we want. > > > > >>> >>> > > > > >>> >>> 2018년 1월 30일 (화) 오전 9:25, Jungtaek Lim <[email protected]>님이 > > 작성: > > > > >>> >>> > > > > >>> >>>> Erik's concern brought from 1.0.6 RC1, because they can't > use > > > > Storm > > > > >>> >> 1.1.0 > > > > >>> >>>> or higher (Storm 1.1.0 broke storm-mesos.). While he could > > take > > > an > > > > >>> >>>> workaround to use storm-kafka-client 1.2.0 or 1.1.2 (if we > > > decide > > > > to > > > > >>> >>>> replace) with Storm 1.0.6, it would be better if we don't > > allow > > > > >>> leaving > > > > >>> >>>> storm-kafka-client in 1.0.x in inconsistent state. > > > > >>> >>>> > > > > >>> >>>> IMHO, breaking backward compatibility is worse, but leaving > > > broken > > > > >>> >> thing > > > > >>> >>>> is worst. Hence I'm +1 to replace all, with noticing that it > > may > > > > >>> bring > > > > >>> >>>> backward incompatibility in release announce. > > > > >>> >>>> > > > > >>> >>>> -Jungtaek Lim (HeartSaVioR) > > > > >>> >>>> > > > > >>> >>>> 2018년 1월 30일 (화) 오전 4:49, P. Taylor Goetz < > [email protected] > > >님이 > > > > 작성: > > > > >>> >>>> > > > > >>> >>>>> As I mentioned else thread I’m open to this but would defer > > to > > > > >>> >> community > > > > >>> >>>>> consensus. > > > > >>> >>>>> > > > > >>> >>>>> If there’s concern about doing this for 1.0.x, one option > > would > > > > be > > > > >>> >> skip > > > > >>> >>>>> that version line and only apply it to 1.2.0 and 1.1.x. > > > > >>> >>>>> > > > > >>> >>>>> -Taylor > > > > >>> >>>>> > > > > >>> >>>>>> On Jan 29, 2018, at 12:12 AM, Jungtaek Lim < > > [email protected] > > > > > > > > >>> >> wrote: > > > > >>> >>>>>> > > > > >>> >>>>>> Hi devs, > > > > >>> >>>>>> > > > > >>> >>>>>> This is initial post to separate out discussion topic from > > > vote > > > > >>> >> thread, > > > > >>> >>>>> and > > > > >>> >>>>>> continue discussing. > > > > >>> >>>>>> > > > > >>> >>>>>> Background of the topic: > > > > >>> >>>>>> 1. Only 1.x-branch of storm-kafka-client got stabilized. > > > > >>> >> (relatively) > > > > >>> >>>>>> 2. We would avoid to port back patches to 1.1.x and 1.0.x > > > > because > > > > >>> >>>>> they're > > > > >>> >>>>>> diverged too much. > > > > >>> >>>>>> > > > > >>> >>>>>> Downside: > > > > >>> >>>>>> Backward compatibility might be broken for 1.1.x and > 1.0.x. > > > Not > > > > >>> >> sure for > > > > >>> >>>>>> 1.1.x, but at least 1.0.x, since supported Kafka client > > > version > > > > is > > > > >>> >>>>>> different, and if my memory is right, we already applied > > > > backward > > > > >>> >>>>>> incompatible change into storm-kafka-client 1.1.0. > > > > >>> >>>>>> > > > > >>> >>>>>> Please put your opinion regarding topic. You're encouraged > > to > > > > copy > > > > >>> >> your > > > > >>> >>>>>> previous post in vote thread which helps to centralize > > > opinions > > > > in > > > > >>> >>>>> current > > > > >>> >>>>>> thread. > > > > >>> >>>>>> > > > > >>> >>>>>> Thanks, > > > > >>> >>>>>> Jungtaek Lim (HeartSaVioR) > > > > >>> >>>>> > > > > >>> >>>>> > > > > >>> >> > > > > >>> > > > > >>> > > > > > > > > > >
