Update: I also merged the patch for 1.0.x-branch. Thanks all for taking valuable efforts and time to participate this topic and making it possible so quickly. Now release phase of both 1.1.2 and 1.0.6 can be restart.
2018년 2월 8일 (목) 오전 6:30, Jungtaek Lim <[email protected]>님이 작성: > Update: I just merged the patch for 1.1.x-branch so release phase of Storm > 1.1.2 can be restarted. Patch for 1.0.x-branch from Erik is available and > got some +1s but waiting for 24hrs rule. > > 2018년 2월 7일 (수) 오전 5:03, Stig Rohde Døssing <[email protected]>님이 작성: > >> Took a look at backporting to 1.0.x. We'll have to update the time >> simulation code (Time.java in storm-core) to support nanoseconds, as Erik >> noted, but this isn't a breaking change and only affects tests. >> >> This PR https://github.com/apache/storm/pull/1995/files#diff- >> 72647db30ffd6005dc01c4d1f75d2c68 made a breaking change to >> IOpaquePartitionedTridentSpoutExecutor, so we'll have to do the same on >> 1.0.x. >> >> 2018-02-06 19:13 GMT+01:00 P. Taylor Goetz <[email protected]>: >> >> > Just a heads up: While this gets sorted out I’m going to proceed with a >> > 1.2.0 RC. >> > >> > -Taylor >> > >> > > On Feb 5, 2018, at 10:46 PM, Jungtaek Lim <[email protected]> wrote: >> > > >> > > UPDATE: Submitted a pull request https://github.com/apache/ >> > storm/pull/2549 for >> > > STORM-2936 (against 1.1.x-branch) >> > > >> > > Erik, please change the status to "IN PROGRESS" if someone is working >> > on. I >> > > would find the free time and just do it if there's no one working in >> > > progress. >> > > >> > > 2018년 2월 6일 (화) 오전 10:39, Jungtaek Lim <[email protected]>님이 작성: >> > > >> > >> Thanks for quick response Erik! >> > >> >> > >> Just filed two issues : >> > >> https://issues.apache.org/jira/browse/STORM-2936 (for 1.1.x-branch) >> > >> https://issues.apache.org/jira/browse/STORM-2937 (for 1.0.x-branch) >> > >> >> > >> We have another discussion around making storm-kafka-client be >> > experiment >> > >> of managing separately (independent of Storm release). So the three >> > >> versions which are in release phase might be the last releases of >> > >> "battery-included" of storm-kafka-client if our experiment works >> well. >> > >> >> > >> If we would want to make the change for storm-kafka-client, it might >> be >> > >> better to put the change and release before start experimenting, but >> > that's >> > >> just a thought on my own. In opposite way, we could even start >> > experiment >> > >> and make change of storm-core of 1.0.x-branch to be compatible with >> that >> > >> version of storm-kafka-client. We could even do it for 1.1.x-branch, >> but >> > >> the change is almost done so it doesn't look like needed to postpone >> it. >> > >> >> > >> Would like to here everyone's voice on this. >> > >> >> > >> -Jungtaek Lim (HeartSaVioR) >> > >> >> > >> 2018년 2월 6일 (화) 오전 10:23, Erik Weathers >> <[email protected] >> > >님이 >> > >> 작성: >> > >> >> > >>> Thanks for the quick response Jungtaek! >> > >>> >> > >>> Yes, my teammates and myself would like to help on this. Is there >> an >> > >>> existing JIRA for the work you've been doing on the other branches? >> > >>> >> > >>> I propose we don't make this block 1.0.6 -- we can just release >> 1.0.7 >> > >>> quickly when the backport is done, if that is amenable. >> > >>> That strategy also might be cleaner since it would avoid other >> changes >> > in >> > >>> 1.0.6 being lumped together with this. >> > >>> >> > >>> - Erik >> > >>> >> > >>> On Mon, Feb 5, 2018 at 5:16 PM, Jungtaek Lim <[email protected]> >> > wrote: >> > >>> >> > >>>> UPDATE: I've finished working on overwriting storm-kafka-client >> > >>> 1.x-branch >> > >>>> to 1.1.x-branch. Not yet pushed to ASF git, but pushed to my fork >> > first >> > >>> to >> > >>>> trigger Travis CI to see how the build goes well. >> > >>>> >> > >>>> https://github.com/HeartSaVioR/storm/commit/76b8a7d3a6f91e66 >> > >>>> 612e87da8589f5723f05218a >> > >>>> https://travis-ci.org/HeartSaVioR/storm/builds/337819430 >> > >>>> >> > >>>> Thanks for the input regarding 1.0.x version, Erik. I guess then we >> > >>> have no >> > >>>> alternative here: someone has to fix storm-kafka-client as well as >> > >>>> storm-core, since including shaded storm-core doesn't make sense >> for >> > >>>> official Storm release. >> > >>>> >> > >>>> I guess it doesn't take many hour(s), hence may not worth to sync >> and >> > >>> talk >> > >>>> offline. I just wanted to judge whether we are OK to make change of >> > >>>> storm-core in bugfix version lines, but maybe the judgement itself >> can >> > >>> be >> > >>>> possible after finishing the change, so I'll just go ahead making >> the >> > >>>> change. >> > >>>> Since this is blocking release candidate, we should get it ASAP. >> > That's >> > >>> why >> > >>>> I'm eager to go ahead making the change. If you could spend time >> now >> > >>>> helping with making the change ASAP, please leave short notice >> (maybe >> > >>> with >> > >>>> JIRA issue?) and go ahead. >> > >>>> >> > >>>> Thanks, >> > >>>> Jungtaek Lim (HeartSaVioR) >> > >>>> >> > >>>> 2018년 2월 6일 (화) 오전 9:41, Erik Weathers >> <[email protected] >> > >>>> 님이 >> > >>>> 작성: >> > >>>> >> > >>>>> hey Jungtaek, >> > >>>>> >> > >>>>> Thanks for continuing to pursue this! >> > >>>>> >> > >>>>> The issue for Storm not working on Mesos is due to a fundamental >> > >>> change >> > >>>> to >> > >>>>> the core scheduling logic in Storm: >> > >>>>> >> > >>>>> - >> > >>>>> >> > >>>>> https://issues.apache.org/jira/browse/STORM-2126?focusedComm >> > >>>> entId=16136150&page=com.atlassian.jira.plugin.system. >> > >>>> issuetabpanels%3Acomment-tabpanel#comment-16136150 >> > >>>>> >> > >>>>> The yet-to-be-ironed-out solution that Bobby was brainstorming >> about >> > >>>> isn't >> > >>>>> a short term fix as far as I understand it. I believe it to be >> many >> > >>> many >> > >>>>> months (years?) out for it to actually be workable. Per my naive >> > >>>>> understanding of the proposal, we'd probably have to completely >> > >>> rewrite >> > >>>> the >> > >>>>> Storm-on-Mesos framework. So it's probably the right long-term >> > >>> solution, >> > >>>>> but it isn't anything that should impact this discussion. >> > >>>>> >> > >>>>>> The thing is, even users pick storm-kafka-client 1.1.x/1.2.0 and >> > >>>> include >> > >>>>> it into their topology jar, it will also not work with Storm >> 1.0.x. >> > It >> > >>>>> even can't >> > >>>>> compile. >> > >>>>> >> > >>>>> FWIW, I'm pretty sure that I was able to successfully run >> > >>>>> storm-kafka-client-1.1.x on a 1.0.5 storm cluster, but only after >> > >>> shading >> > >>>>> in storm-core-1.1.x to the topology uber jar. There was *at >> least* >> > a >> > >>>>> change to some timer-related class in storm-core in 1.1.x >> (something >> > >>>> about >> > >>>>> milliseconds IIRC -- it's been 1.5 months since I did it, need to >> > >>> revisit >> > >>>>> the process I followed). >> > >>>>> >> > >>>>> I'm happy to help with backporting / stomping storm-kafka-client >> in >> > >>>> 1.0.x. >> > >>>>> Maybe we can talk offline about it? >> > >>>>> >> > >>>>> - Erik >> > >>>>> >> > >>>>> On Mon, Feb 5, 2018 at 4:20 PM, Jungtaek Lim <[email protected]> >> > >>> wrote: >> > >>>>> >> > >>>>>> UPDATE: Looks like we changed some parts of storm-core while >> fixing >> > >>>>>> storm-kafka-client issues (especially went in 1.1.0), hence >> > >>> overwriting >> > >>>>>> also incurs changes of storm-core. It doesn't look like a big >> deal >> > >>> for >> > >>>>>> 1.1.x-branch, but there looks like needed many changes for >> > >>>> 1.0.x-branch. >> > >>>>>> >> > >>>>>> The thing is, even users pick storm-kafka-client 1.1.x/1.2.0 and >> > >>>> include >> > >>>>> it >> > >>>>>> into their topology jar, it will also not work with Storm 1.0.x. >> It >> > >>>> even >> > >>>>>> can't compile. >> > >>>>>> >> > >>>>>> 1.0.x version line was long lived (22 months) even we released >> Storm >> > >>>>> 1.1.0 >> > >>>>>> at 11 months ago. Instead of struggling 1.0.x-branch to up to >> date, >> > >>> I'd >> > >>>>>> like to suggest that we define 1.0.x-branch as deprecated with >> > >>> guiding >> > >>>> to >> > >>>>>> update to latest 1.1.x version or 1.2.0 (after release), and try >> to >> > >>>>> resolve >> > >>>>>> storm-mesos issue with Storm 1.1.0 ASAP to resolve Erik's >> concern. >> > >>>>>> >> > >>>>>> Makes sense? I'll continue working on 1.1.x-branch and update >> > >>> anyway. >> > >>>>>> >> > >>>>>> -Jungtaek Lim (HeartSaVioR) >> > >>>>>> >> > >>>>>> 2018년 2월 6일 (화) 오전 7:53, Jungtaek Lim <[email protected]>님이 작성: >> > >>>>>> >> > >>>>>>> OK. No more opinion/vote in 5 days. I'll treat consensus was >> made, >> > >>>> and >> > >>>>> go >> > >>>>>>> ahead making change: overwrite storm-kafka-client 1.2.0 to two >> > >>>> branches >> > >>>>>>> 1.1.x/1.0.x. >> > >>>>>>> >> > >>>>>>> -Jungtaek Lim (HeartSaVioR) >> > >>>>>>> >> > >>>>>>> 2018년 2월 1일 (목) 오전 10:48, Jungtaek Lim <[email protected]>님이 >> 작성: >> > >>>>>>> >> > >>>>>>>> This discussion got 4 +1 (binding) and no -1. Moreover two >> active >> > >>>>>>>> maintainers for storm-kafka-client (Hugo and Stig) voted +1. >> > >>>>>>>> >> > >>>>>>>> Do we want to hold on for hearing more voices, or treating >> above >> > >>>>>> opinions >> > >>>>>>>> as consensus and reflect the change? >> > >>>>>>>> >> > >>>>>>>> Btw, I think we need to sort out the sequences between two >> > >>> topics: >> > >>>>>>>> separating storm-kafka-client as independent release cycle, and >> > >>>> this. >> > >>>>> I >> > >>>>>>>> guess some of us agreed former topic doesn't related to current >> > >>> RC, >> > >>>>> but >> > >>>>>> I >> > >>>>>>>> think this topic can be (should be) reflected to current RC >> > >>> ongoing. >> > >>>>>>>> >> > >>>>>>>> -Jungtaek Lim (HeartSaVioR) >> > >>>>>>>> >> > >>>>>>>> 2018년 2월 1일 (목) 오전 4:08, Hugo Da Cruz Louro < >> > >>> [email protected] >> > >>>>>> 님이 >> > >>>>>>>> 작성: >> > >>>>>>>> >> > >>>>>>>>> +1 to replace storm-kafka-client in 1.0.x branch. >> > >>>>>>>>> Hugo >> > >>>>>>>>> >> > >>>>>>>>>> On Jan 31, 2018, at 11:03 AM, Stig Rohde Døssing < >> > >>>>>>>>> [email protected]> wrote: >> > >>>>>>>>>> >> > >>>>>>>>>> +1 to replace storm-kafka-client in 1.0.x branch. Breaking >> > >>>> semantic >> > >>>>>>>>>> versioning is really nasty, but I think it is the lesser evil >> > >>> in >> > >>>>> this >> > >>>>>>>>> case. >> > >>>>>>>>>> >> > >>>>>>>>>> 2018-01-31 5:14 GMT+01:00 Harsha <[email protected]>: >> > >>>>>>>>>> >> > >>>>>>>>>>> +1 to replace storm-kafka-client in 1.0.x branch >> > >>>>>>>>>>> -Harsha >> > >>>>>>>>>>> On Tue, Jan 30, 2018, at 7:04 PM, Jungtaek Lim wrote: >> > >>>>>>>>>>>> Bump up this thread so that we could reach consensus >> > >>> earlier. >> > >>>>> Given >> > >>>>>>>>> that >> > >>>>>>>>>>> we >> > >>>>>>>>>>>> got concern related to this, I think it is ideal to release >> > >>>>>>>>> 1.1.x/1.0.x >> > >>>>>>>>>>>> with making decision and applying the change if we want. >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> 2018년 1월 30일 (화) 오전 9:25, Jungtaek Lim <[email protected] >> > >>>> 님이 >> > >>>> 작성: >> > >>>>>>>>>>>> >> > >>>>>>>>>>>>> Erik's concern brought from 1.0.6 RC1, because they can't >> > >>> use >> > >>>>>> Storm >> > >>>>>>>>>>> 1.1.0 >> > >>>>>>>>>>>>> or higher (Storm 1.1.0 broke storm-mesos.). While he could >> > >>>> take >> > >>>>> an >> > >>>>>>>>>>>>> workaround to use storm-kafka-client 1.2.0 or 1.1.2 (if we >> > >>>>> decide >> > >>>>>> to >> > >>>>>>>>>>>>> replace) with Storm 1.0.6, it would be better if we don't >> > >>>> allow >> > >>>>>>>>> leaving >> > >>>>>>>>>>>>> storm-kafka-client in 1.0.x in inconsistent state. >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> IMHO, breaking backward compatibility is worse, but >> leaving >> > >>>>> broken >> > >>>>>>>>>>> thing >> > >>>>>>>>>>>>> is worst. Hence I'm +1 to replace all, with noticing that >> > >>> it >> > >>>> may >> > >>>>>>>>> bring >> > >>>>>>>>>>>>> backward incompatibility in release announce. >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> -Jungtaek Lim (HeartSaVioR) >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> 2018년 1월 30일 (화) 오전 4:49, P. Taylor Goetz < >> > >>> [email protected] >> > >>>>> 님이 >> > >>>>>> 작성: >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>>> As I mentioned else thread I’m open to this but would >> > >>> defer >> > >>>> to >> > >>>>>>>>>>> community >> > >>>>>>>>>>>>>> consensus. >> > >>>>>>>>>>>>>> >> > >>>>>>>>>>>>>> If there’s concern about doing this for 1.0.x, one option >> > >>>> would >> > >>>>>> be >> > >>>>>>>>>>> skip >> > >>>>>>>>>>>>>> that version line and only apply it to 1.2.0 and 1.1.x. >> > >>>>>>>>>>>>>> >> > >>>>>>>>>>>>>> -Taylor >> > >>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>> On Jan 29, 2018, at 12:12 AM, Jungtaek Lim < >> > >>>> [email protected] >> > >>>>>> >> > >>>>>>>>>>> wrote: >> > >>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>> Hi devs, >> > >>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>> This is initial post to separate out discussion topic >> > >>> from >> > >>>>> vote >> > >>>>>>>>>>> thread, >> > >>>>>>>>>>>>>> and >> > >>>>>>>>>>>>>>> continue discussing. >> > >>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>> Background of the topic: >> > >>>>>>>>>>>>>>> 1. Only 1.x-branch of storm-kafka-client got stabilized. >> > >>>>>>>>>>> (relatively) >> > >>>>>>>>>>>>>>> 2. We would avoid to port back patches to 1.1.x and >> 1.0.x >> > >>>>>> because >> > >>>>>>>>>>>>>> they're >> > >>>>>>>>>>>>>>> diverged too much. >> > >>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>> Downside: >> > >>>>>>>>>>>>>>> Backward compatibility might be broken for 1.1.x and >> > >>> 1.0.x. >> > >>>>> Not >> > >>>>>>>>>>> sure for >> > >>>>>>>>>>>>>>> 1.1.x, but at least 1.0.x, since supported Kafka client >> > >>>>> version >> > >>>>>> is >> > >>>>>>>>>>>>>>> different, and if my memory is right, we already applied >> > >>>>>> backward >> > >>>>>>>>>>>>>>> incompatible change into storm-kafka-client 1.1.0. >> > >>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>> Please put your opinion regarding topic. You're >> > >>> encouraged >> > >>>> to >> > >>>>>> copy >> > >>>>>>>>>>> your >> > >>>>>>>>>>>>>>> previous post in vote thread which helps to centralize >> > >>>>> opinions >> > >>>>>> in >> > >>>>>>>>>>>>>> current >> > >>>>>>>>>>>>>>> thread. >> > >>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>> Thanks, >> > >>>>>>>>>>>>>>> Jungtaek Lim (HeartSaVioR) >> > >>>>>>>>>>>>>> >> > >>>>>>>>>>>>>> >> > >>>>>>>>>>> >> > >>>>>>>>> >> > >>>>>>>>> >> > >>>>>> >> > >>>>> >> > >>>> >> > >>> >> > >> >> > >> > >> >
