Thanks for the quick response Jungtaek! Yes, my teammates and myself would like to help on this. Is there an existing JIRA for the work you've been doing on the other branches?
I propose we don't make this block 1.0.6 -- we can just release 1.0.7 quickly when the backport is done, if that is amenable. That strategy also might be cleaner since it would avoid other changes in 1.0.6 being lumped together with this. - Erik On Mon, Feb 5, 2018 at 5:16 PM, Jungtaek Lim <kabh...@gmail.com> wrote: > UPDATE: I've finished working on overwriting storm-kafka-client 1.x-branch > to 1.1.x-branch. Not yet pushed to ASF git, but pushed to my fork first to > trigger Travis CI to see how the build goes well. > > https://github.com/HeartSaVioR/storm/commit/76b8a7d3a6f91e66 > 612e87da8589f5723f05218a > https://travis-ci.org/HeartSaVioR/storm/builds/337819430 > > Thanks for the input regarding 1.0.x version, Erik. I guess then we have no > alternative here: someone has to fix storm-kafka-client as well as > storm-core, since including shaded storm-core doesn't make sense for > official Storm release. > > I guess it doesn't take many hour(s), hence may not worth to sync and talk > offline. I just wanted to judge whether we are OK to make change of > storm-core in bugfix version lines, but maybe the judgement itself can be > possible after finishing the change, so I'll just go ahead making the > change. > Since this is blocking release candidate, we should get it ASAP. That's why > I'm eager to go ahead making the change. If you could spend time now > helping with making the change ASAP, please leave short notice (maybe with > JIRA issue?) and go ahead. > > Thanks, > Jungtaek Lim (HeartSaVioR) > > 2018년 2월 6일 (화) 오전 9:41, Erik Weathers <eweath...@groupon.com.invalid>님이 > 작성: > > > hey Jungtaek, > > > > Thanks for continuing to pursue this! > > > > The issue for Storm not working on Mesos is due to a fundamental change > to > > the core scheduling logic in Storm: > > > > - > > > > https://issues.apache.org/jira/browse/STORM-2126?focusedComm > entId=16136150&page=com.atlassian.jira.plugin.system. > issuetabpanels%3Acomment-tabpanel#comment-16136150 > > > > The yet-to-be-ironed-out solution that Bobby was brainstorming about > isn't > > a short term fix as far as I understand it. I believe it to be many many > > months (years?) out for it to actually be workable. Per my naive > > understanding of the proposal, we'd probably have to completely rewrite > the > > Storm-on-Mesos framework. So it's probably the right long-term solution, > > but it isn't anything that should impact this discussion. > > > > > The thing is, even users pick storm-kafka-client 1.1.x/1.2.0 and > include > > it into their topology jar, it will also not work with Storm 1.0.x. It > > even can't > > compile. > > > > FWIW, I'm pretty sure that I was able to successfully run > > storm-kafka-client-1.1.x on a 1.0.5 storm cluster, but only after shading > > in storm-core-1.1.x to the topology uber jar. There was *at least* a > > change to some timer-related class in storm-core in 1.1.x (something > about > > milliseconds IIRC -- it's been 1.5 months since I did it, need to revisit > > the process I followed). > > > > I'm happy to help with backporting / stomping storm-kafka-client in > 1.0.x. > > Maybe we can talk offline about it? > > > > - Erik > > > > On Mon, Feb 5, 2018 at 4:20 PM, Jungtaek Lim <kabh...@gmail.com> wrote: > > > > > UPDATE: Looks like we changed some parts of storm-core while fixing > > > storm-kafka-client issues (especially went in 1.1.0), hence overwriting > > > also incurs changes of storm-core. It doesn't look like a big deal for > > > 1.1.x-branch, but there looks like needed many changes for > 1.0.x-branch. > > > > > > The thing is, even users pick storm-kafka-client 1.1.x/1.2.0 and > include > > it > > > into their topology jar, it will also not work with Storm 1.0.x. It > even > > > can't compile. > > > > > > 1.0.x version line was long lived (22 months) even we released Storm > > 1.1.0 > > > at 11 months ago. Instead of struggling 1.0.x-branch to up to date, I'd > > > like to suggest that we define 1.0.x-branch as deprecated with guiding > to > > > update to latest 1.1.x version or 1.2.0 (after release), and try to > > resolve > > > storm-mesos issue with Storm 1.1.0 ASAP to resolve Erik's concern. > > > > > > Makes sense? I'll continue working on 1.1.x-branch and update anyway. > > > > > > -Jungtaek Lim (HeartSaVioR) > > > > > > 2018년 2월 6일 (화) 오전 7:53, Jungtaek Lim <kabh...@gmail.com>님이 작성: > > > > > > > OK. No more opinion/vote in 5 days. I'll treat consensus was made, > and > > go > > > > ahead making change: overwrite storm-kafka-client 1.2.0 to two > branches > > > > 1.1.x/1.0.x. > > > > > > > > -Jungtaek Lim (HeartSaVioR) > > > > > > > > 2018년 2월 1일 (목) 오전 10:48, Jungtaek Lim <kabh...@gmail.com>님이 작성: > > > > > > > >> This discussion got 4 +1 (binding) and no -1. Moreover two active > > > >> maintainers for storm-kafka-client (Hugo and Stig) voted +1. > > > >> > > > >> Do we want to hold on for hearing more voices, or treating above > > > opinions > > > >> as consensus and reflect the change? > > > >> > > > >> Btw, I think we need to sort out the sequences between two topics: > > > >> separating storm-kafka-client as independent release cycle, and > this. > > I > > > >> guess some of us agreed former topic doesn't related to current RC, > > but > > > I > > > >> think this topic can be (should be) reflected to current RC ongoing. > > > >> > > > >> -Jungtaek Lim (HeartSaVioR) > > > >> > > > >> 2018년 2월 1일 (목) 오전 4:08, Hugo Da Cruz Louro <hlo...@hortonworks.com > > >님이 > > > >> 작성: > > > >> > > > >>> +1 to replace storm-kafka-client in 1.0.x branch. > > > >>> Hugo > > > >>> > > > >>> > On Jan 31, 2018, at 11:03 AM, Stig Rohde Døssing < > > > >>> stigdoess...@gmail.com> wrote: > > > >>> > > > > >>> > +1 to replace storm-kafka-client in 1.0.x branch. Breaking > semantic > > > >>> > versioning is really nasty, but I think it is the lesser evil in > > this > > > >>> case. > > > >>> > > > > >>> > 2018-01-31 5:14 GMT+01:00 Harsha <st...@harsha.io>: > > > >>> > > > > >>> >> +1 to replace storm-kafka-client in 1.0.x branch > > > >>> >> -Harsha > > > >>> >> On Tue, Jan 30, 2018, at 7:04 PM, Jungtaek Lim wrote: > > > >>> >>> Bump up this thread so that we could reach consensus earlier. > > Given > > > >>> that > > > >>> >> we > > > >>> >>> got concern related to this, I think it is ideal to release > > > >>> 1.1.x/1.0.x > > > >>> >>> with making decision and applying the change if we want. > > > >>> >>> > > > >>> >>> 2018년 1월 30일 (화) 오전 9:25, Jungtaek Lim <kabh...@gmail.com>님이 > 작성: > > > >>> >>> > > > >>> >>>> Erik's concern brought from 1.0.6 RC1, because they can't use > > > Storm > > > >>> >> 1.1.0 > > > >>> >>>> or higher (Storm 1.1.0 broke storm-mesos.). While he could > take > > an > > > >>> >>>> workaround to use storm-kafka-client 1.2.0 or 1.1.2 (if we > > decide > > > to > > > >>> >>>> replace) with Storm 1.0.6, it would be better if we don't > allow > > > >>> leaving > > > >>> >>>> storm-kafka-client in 1.0.x in inconsistent state. > > > >>> >>>> > > > >>> >>>> IMHO, breaking backward compatibility is worse, but leaving > > broken > > > >>> >> thing > > > >>> >>>> is worst. Hence I'm +1 to replace all, with noticing that it > may > > > >>> bring > > > >>> >>>> backward incompatibility in release announce. > > > >>> >>>> > > > >>> >>>> -Jungtaek Lim (HeartSaVioR) > > > >>> >>>> > > > >>> >>>> 2018년 1월 30일 (화) 오전 4:49, P. Taylor Goetz <ptgo...@gmail.com > >님이 > > > 작성: > > > >>> >>>> > > > >>> >>>>> As I mentioned else thread I’m open to this but would defer > to > > > >>> >> community > > > >>> >>>>> consensus. > > > >>> >>>>> > > > >>> >>>>> If there’s concern about doing this for 1.0.x, one option > would > > > be > > > >>> >> skip > > > >>> >>>>> that version line and only apply it to 1.2.0 and 1.1.x. > > > >>> >>>>> > > > >>> >>>>> -Taylor > > > >>> >>>>> > > > >>> >>>>>> On Jan 29, 2018, at 12:12 AM, Jungtaek Lim < > kabh...@gmail.com > > > > > > >>> >> wrote: > > > >>> >>>>>> > > > >>> >>>>>> Hi devs, > > > >>> >>>>>> > > > >>> >>>>>> This is initial post to separate out discussion topic from > > vote > > > >>> >> thread, > > > >>> >>>>> and > > > >>> >>>>>> continue discussing. > > > >>> >>>>>> > > > >>> >>>>>> Background of the topic: > > > >>> >>>>>> 1. Only 1.x-branch of storm-kafka-client got stabilized. > > > >>> >> (relatively) > > > >>> >>>>>> 2. We would avoid to port back patches to 1.1.x and 1.0.x > > > because > > > >>> >>>>> they're > > > >>> >>>>>> diverged too much. > > > >>> >>>>>> > > > >>> >>>>>> Downside: > > > >>> >>>>>> Backward compatibility might be broken for 1.1.x and 1.0.x. > > Not > > > >>> >> sure for > > > >>> >>>>>> 1.1.x, but at least 1.0.x, since supported Kafka client > > version > > > is > > > >>> >>>>>> different, and if my memory is right, we already applied > > > backward > > > >>> >>>>>> incompatible change into storm-kafka-client 1.1.0. > > > >>> >>>>>> > > > >>> >>>>>> Please put your opinion regarding topic. You're encouraged > to > > > copy > > > >>> >> your > > > >>> >>>>>> previous post in vote thread which helps to centralize > > opinions > > > in > > > >>> >>>>> current > > > >>> >>>>>> thread. > > > >>> >>>>>> > > > >>> >>>>>> Thanks, > > > >>> >>>>>> Jungtaek Lim (HeartSaVioR) > > > >>> >>>>> > > > >>> >>>>> > > > >>> >> > > > >>> > > > >>> > > > > > >
