Just a heads up: While this gets sorted out I’m going to proceed with a 1.2.0 RC.
-Taylor > On Feb 5, 2018, at 10:46 PM, Jungtaek Lim <[email protected]> wrote: > > UPDATE: Submitted a pull request https://github.com/apache/storm/pull/2549 for > STORM-2936 (against 1.1.x-branch) > > Erik, please change the status to "IN PROGRESS" if someone is working on. I > would find the free time and just do it if there's no one working in > progress. > > 2018년 2월 6일 (화) 오전 10:39, Jungtaek Lim <[email protected]>님이 작성: > >> Thanks for quick response Erik! >> >> Just filed two issues : >> https://issues.apache.org/jira/browse/STORM-2936 (for 1.1.x-branch) >> https://issues.apache.org/jira/browse/STORM-2937 (for 1.0.x-branch) >> >> We have another discussion around making storm-kafka-client be experiment >> of managing separately (independent of Storm release). So the three >> versions which are in release phase might be the last releases of >> "battery-included" of storm-kafka-client if our experiment works well. >> >> If we would want to make the change for storm-kafka-client, it might be >> better to put the change and release before start experimenting, but that's >> just a thought on my own. In opposite way, we could even start experiment >> and make change of storm-core of 1.0.x-branch to be compatible with that >> version of storm-kafka-client. We could even do it for 1.1.x-branch, but >> the change is almost done so it doesn't look like needed to postpone it. >> >> Would like to here everyone's voice on this. >> >> -Jungtaek Lim (HeartSaVioR) >> >> 2018년 2월 6일 (화) 오전 10:23, Erik Weathers <[email protected]>님이 >> 작성: >> >>> Thanks for the quick response Jungtaek! >>> >>> Yes, my teammates and myself would like to help on this. Is there an >>> existing JIRA for the work you've been doing on the other branches? >>> >>> I propose we don't make this block 1.0.6 -- we can just release 1.0.7 >>> quickly when the backport is done, if that is amenable. >>> That strategy also might be cleaner since it would avoid other changes in >>> 1.0.6 being lumped together with this. >>> >>> - Erik >>> >>> On Mon, Feb 5, 2018 at 5:16 PM, Jungtaek Lim <[email protected]> wrote: >>> >>>> UPDATE: I've finished working on overwriting storm-kafka-client >>> 1.x-branch >>>> to 1.1.x-branch. Not yet pushed to ASF git, but pushed to my fork first >>> to >>>> trigger Travis CI to see how the build goes well. >>>> >>>> https://github.com/HeartSaVioR/storm/commit/76b8a7d3a6f91e66 >>>> 612e87da8589f5723f05218a >>>> https://travis-ci.org/HeartSaVioR/storm/builds/337819430 >>>> >>>> Thanks for the input regarding 1.0.x version, Erik. I guess then we >>> have no >>>> alternative here: someone has to fix storm-kafka-client as well as >>>> storm-core, since including shaded storm-core doesn't make sense for >>>> official Storm release. >>>> >>>> I guess it doesn't take many hour(s), hence may not worth to sync and >>> talk >>>> offline. I just wanted to judge whether we are OK to make change of >>>> storm-core in bugfix version lines, but maybe the judgement itself can >>> be >>>> possible after finishing the change, so I'll just go ahead making the >>>> change. >>>> Since this is blocking release candidate, we should get it ASAP. That's >>> why >>>> I'm eager to go ahead making the change. If you could spend time now >>>> helping with making the change ASAP, please leave short notice (maybe >>> with >>>> JIRA issue?) and go ahead. >>>> >>>> Thanks, >>>> Jungtaek Lim (HeartSaVioR) >>>> >>>> 2018년 2월 6일 (화) 오전 9:41, Erik Weathers <[email protected] >>>> 님이 >>>> 작성: >>>> >>>>> hey Jungtaek, >>>>> >>>>> Thanks for continuing to pursue this! >>>>> >>>>> The issue for Storm not working on Mesos is due to a fundamental >>> change >>>> to >>>>> the core scheduling logic in Storm: >>>>> >>>>> - >>>>> >>>>> https://issues.apache.org/jira/browse/STORM-2126?focusedComm >>>> entId=16136150&page=com.atlassian.jira.plugin.system. >>>> issuetabpanels%3Acomment-tabpanel#comment-16136150 >>>>> >>>>> The yet-to-be-ironed-out solution that Bobby was brainstorming about >>>> isn't >>>>> a short term fix as far as I understand it. I believe it to be many >>> many >>>>> months (years?) out for it to actually be workable. Per my naive >>>>> understanding of the proposal, we'd probably have to completely >>> rewrite >>>> the >>>>> Storm-on-Mesos framework. So it's probably the right long-term >>> solution, >>>>> but it isn't anything that should impact this discussion. >>>>> >>>>>> The thing is, even users pick storm-kafka-client 1.1.x/1.2.0 and >>>> include >>>>> it into their topology jar, it will also not work with Storm 1.0.x. It >>>>> even can't >>>>> compile. >>>>> >>>>> FWIW, I'm pretty sure that I was able to successfully run >>>>> storm-kafka-client-1.1.x on a 1.0.5 storm cluster, but only after >>> shading >>>>> in storm-core-1.1.x to the topology uber jar. There was *at least* a >>>>> change to some timer-related class in storm-core in 1.1.x (something >>>> about >>>>> milliseconds IIRC -- it's been 1.5 months since I did it, need to >>> revisit >>>>> the process I followed). >>>>> >>>>> I'm happy to help with backporting / stomping storm-kafka-client in >>>> 1.0.x. >>>>> Maybe we can talk offline about it? >>>>> >>>>> - Erik >>>>> >>>>> On Mon, Feb 5, 2018 at 4:20 PM, Jungtaek Lim <[email protected]> >>> wrote: >>>>> >>>>>> UPDATE: Looks like we changed some parts of storm-core while fixing >>>>>> storm-kafka-client issues (especially went in 1.1.0), hence >>> overwriting >>>>>> also incurs changes of storm-core. It doesn't look like a big deal >>> for >>>>>> 1.1.x-branch, but there looks like needed many changes for >>>> 1.0.x-branch. >>>>>> >>>>>> The thing is, even users pick storm-kafka-client 1.1.x/1.2.0 and >>>> include >>>>> it >>>>>> into their topology jar, it will also not work with Storm 1.0.x. It >>>> even >>>>>> can't compile. >>>>>> >>>>>> 1.0.x version line was long lived (22 months) even we released Storm >>>>> 1.1.0 >>>>>> at 11 months ago. Instead of struggling 1.0.x-branch to up to date, >>> I'd >>>>>> like to suggest that we define 1.0.x-branch as deprecated with >>> guiding >>>> to >>>>>> update to latest 1.1.x version or 1.2.0 (after release), and try to >>>>> resolve >>>>>> storm-mesos issue with Storm 1.1.0 ASAP to resolve Erik's concern. >>>>>> >>>>>> Makes sense? I'll continue working on 1.1.x-branch and update >>> anyway. >>>>>> >>>>>> -Jungtaek Lim (HeartSaVioR) >>>>>> >>>>>> 2018년 2월 6일 (화) 오전 7:53, Jungtaek Lim <[email protected]>님이 작성: >>>>>> >>>>>>> OK. No more opinion/vote in 5 days. I'll treat consensus was made, >>>> and >>>>> go >>>>>>> ahead making change: overwrite storm-kafka-client 1.2.0 to two >>>> branches >>>>>>> 1.1.x/1.0.x. >>>>>>> >>>>>>> -Jungtaek Lim (HeartSaVioR) >>>>>>> >>>>>>> 2018년 2월 1일 (목) 오전 10:48, Jungtaek Lim <[email protected]>님이 작성: >>>>>>> >>>>>>>> This discussion got 4 +1 (binding) and no -1. Moreover two active >>>>>>>> maintainers for storm-kafka-client (Hugo and Stig) voted +1. >>>>>>>> >>>>>>>> Do we want to hold on for hearing more voices, or treating above >>>>>> opinions >>>>>>>> as consensus and reflect the change? >>>>>>>> >>>>>>>> Btw, I think we need to sort out the sequences between two >>> topics: >>>>>>>> separating storm-kafka-client as independent release cycle, and >>>> this. >>>>> I >>>>>>>> guess some of us agreed former topic doesn't related to current >>> RC, >>>>> but >>>>>> I >>>>>>>> think this topic can be (should be) reflected to current RC >>> ongoing. >>>>>>>> >>>>>>>> -Jungtaek Lim (HeartSaVioR) >>>>>>>> >>>>>>>> 2018년 2월 1일 (목) 오전 4:08, Hugo Da Cruz Louro < >>> [email protected] >>>>>> 님이 >>>>>>>> 작성: >>>>>>>> >>>>>>>>> +1 to replace storm-kafka-client in 1.0.x branch. >>>>>>>>> Hugo >>>>>>>>> >>>>>>>>>> On Jan 31, 2018, at 11:03 AM, Stig Rohde Døssing < >>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>> +1 to replace storm-kafka-client in 1.0.x branch. Breaking >>>> semantic >>>>>>>>>> versioning is really nasty, but I think it is the lesser evil >>> in >>>>> this >>>>>>>>> case. >>>>>>>>>> >>>>>>>>>> 2018-01-31 5:14 GMT+01:00 Harsha <[email protected]>: >>>>>>>>>> >>>>>>>>>>> +1 to replace storm-kafka-client in 1.0.x branch >>>>>>>>>>> -Harsha >>>>>>>>>>> On Tue, Jan 30, 2018, at 7:04 PM, Jungtaek Lim wrote: >>>>>>>>>>>> Bump up this thread so that we could reach consensus >>> earlier. >>>>> Given >>>>>>>>> that >>>>>>>>>>> we >>>>>>>>>>>> got concern related to this, I think it is ideal to release >>>>>>>>> 1.1.x/1.0.x >>>>>>>>>>>> with making decision and applying the change if we want. >>>>>>>>>>>> >>>>>>>>>>>> 2018년 1월 30일 (화) 오전 9:25, Jungtaek Lim <[email protected] >>>> 님이 >>>> 작성: >>>>>>>>>>>> >>>>>>>>>>>>> Erik's concern brought from 1.0.6 RC1, because they can't >>> use >>>>>> Storm >>>>>>>>>>> 1.1.0 >>>>>>>>>>>>> or higher (Storm 1.1.0 broke storm-mesos.). While he could >>>> take >>>>> an >>>>>>>>>>>>> workaround to use storm-kafka-client 1.2.0 or 1.1.2 (if we >>>>> decide >>>>>> to >>>>>>>>>>>>> replace) with Storm 1.0.6, it would be better if we don't >>>> allow >>>>>>>>> leaving >>>>>>>>>>>>> storm-kafka-client in 1.0.x in inconsistent state. >>>>>>>>>>>>> >>>>>>>>>>>>> IMHO, breaking backward compatibility is worse, but leaving >>>>> broken >>>>>>>>>>> thing >>>>>>>>>>>>> is worst. Hence I'm +1 to replace all, with noticing that >>> it >>>> may >>>>>>>>> bring >>>>>>>>>>>>> backward incompatibility in release announce. >>>>>>>>>>>>> >>>>>>>>>>>>> -Jungtaek Lim (HeartSaVioR) >>>>>>>>>>>>> >>>>>>>>>>>>> 2018년 1월 30일 (화) 오전 4:49, P. Taylor Goetz < >>> [email protected] >>>>> 님이 >>>>>> 작성: >>>>>>>>>>>>> >>>>>>>>>>>>>> As I mentioned else thread I’m open to this but would >>> defer >>>> to >>>>>>>>>>> community >>>>>>>>>>>>>> consensus. >>>>>>>>>>>>>> >>>>>>>>>>>>>> If there’s concern about doing this for 1.0.x, one option >>>> would >>>>>> be >>>>>>>>>>> skip >>>>>>>>>>>>>> that version line and only apply it to 1.2.0 and 1.1.x. >>>>>>>>>>>>>> >>>>>>>>>>>>>> -Taylor >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Jan 29, 2018, at 12:12 AM, Jungtaek Lim < >>>> [email protected] >>>>>> >>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi devs, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> This is initial post to separate out discussion topic >>> from >>>>> vote >>>>>>>>>>> thread, >>>>>>>>>>>>>> and >>>>>>>>>>>>>>> continue discussing. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Background of the topic: >>>>>>>>>>>>>>> 1. Only 1.x-branch of storm-kafka-client got stabilized. >>>>>>>>>>> (relatively) >>>>>>>>>>>>>>> 2. We would avoid to port back patches to 1.1.x and 1.0.x >>>>>> because >>>>>>>>>>>>>> they're >>>>>>>>>>>>>>> diverged too much. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Downside: >>>>>>>>>>>>>>> Backward compatibility might be broken for 1.1.x and >>> 1.0.x. >>>>> Not >>>>>>>>>>> sure for >>>>>>>>>>>>>>> 1.1.x, but at least 1.0.x, since supported Kafka client >>>>> version >>>>>> is >>>>>>>>>>>>>>> different, and if my memory is right, we already applied >>>>>> backward >>>>>>>>>>>>>>> incompatible change into storm-kafka-client 1.1.0. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Please put your opinion regarding topic. You're >>> encouraged >>>> to >>>>>> copy >>>>>>>>>>> your >>>>>>>>>>>>>>> previous post in vote thread which helps to centralize >>>>> opinions >>>>>> in >>>>>>>>>>>>>> current >>>>>>>>>>>>>>> thread. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Jungtaek Lim (HeartSaVioR) >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>> >>>>> >>>> >>> >>
signature.asc
Description: Message signed with OpenPGP
