Took a look at backporting to 1.0.x. We'll have to update the time simulation code (Time.java in storm-core) to support nanoseconds, as Erik noted, but this isn't a breaking change and only affects tests.
This PR https://github.com/apache/storm/pull/1995/files#diff- 72647db30ffd6005dc01c4d1f75d2c68 made a breaking change to IOpaquePartitionedTridentSpoutExecutor, so we'll have to do the same on 1.0.x. 2018-02-06 19:13 GMT+01:00 P. Taylor Goetz <[email protected]>: > Just a heads up: While this gets sorted out I’m going to proceed with a > 1.2.0 RC. > > -Taylor > > > On Feb 5, 2018, at 10:46 PM, Jungtaek Lim <[email protected]> wrote: > > > > UPDATE: Submitted a pull request https://github.com/apache/ > storm/pull/2549 for > > STORM-2936 (against 1.1.x-branch) > > > > Erik, please change the status to "IN PROGRESS" if someone is working > on. I > > would find the free time and just do it if there's no one working in > > progress. > > > > 2018년 2월 6일 (화) 오전 10:39, Jungtaek Lim <[email protected]>님이 작성: > > > >> Thanks for quick response Erik! > >> > >> Just filed two issues : > >> https://issues.apache.org/jira/browse/STORM-2936 (for 1.1.x-branch) > >> https://issues.apache.org/jira/browse/STORM-2937 (for 1.0.x-branch) > >> > >> We have another discussion around making storm-kafka-client be > experiment > >> of managing separately (independent of Storm release). So the three > >> versions which are in release phase might be the last releases of > >> "battery-included" of storm-kafka-client if our experiment works well. > >> > >> If we would want to make the change for storm-kafka-client, it might be > >> better to put the change and release before start experimenting, but > that's > >> just a thought on my own. In opposite way, we could even start > experiment > >> and make change of storm-core of 1.0.x-branch to be compatible with that > >> version of storm-kafka-client. We could even do it for 1.1.x-branch, but > >> the change is almost done so it doesn't look like needed to postpone it. > >> > >> Would like to here everyone's voice on this. > >> > >> -Jungtaek Lim (HeartSaVioR) > >> > >> 2018년 2월 6일 (화) 오전 10:23, Erik Weathers <[email protected] > >님이 > >> 작성: > >> > >>> Thanks for the quick response Jungtaek! > >>> > >>> Yes, my teammates and myself would like to help on this. Is there an > >>> existing JIRA for the work you've been doing on the other branches? > >>> > >>> I propose we don't make this block 1.0.6 -- we can just release 1.0.7 > >>> quickly when the backport is done, if that is amenable. > >>> That strategy also might be cleaner since it would avoid other changes > in > >>> 1.0.6 being lumped together with this. > >>> > >>> - Erik > >>> > >>> On Mon, Feb 5, 2018 at 5:16 PM, Jungtaek Lim <[email protected]> > wrote: > >>> > >>>> UPDATE: I've finished working on overwriting storm-kafka-client > >>> 1.x-branch > >>>> to 1.1.x-branch. Not yet pushed to ASF git, but pushed to my fork > first > >>> to > >>>> trigger Travis CI to see how the build goes well. > >>>> > >>>> https://github.com/HeartSaVioR/storm/commit/76b8a7d3a6f91e66 > >>>> 612e87da8589f5723f05218a > >>>> https://travis-ci.org/HeartSaVioR/storm/builds/337819430 > >>>> > >>>> Thanks for the input regarding 1.0.x version, Erik. I guess then we > >>> have no > >>>> alternative here: someone has to fix storm-kafka-client as well as > >>>> storm-core, since including shaded storm-core doesn't make sense for > >>>> official Storm release. > >>>> > >>>> I guess it doesn't take many hour(s), hence may not worth to sync and > >>> talk > >>>> offline. I just wanted to judge whether we are OK to make change of > >>>> storm-core in bugfix version lines, but maybe the judgement itself can > >>> be > >>>> possible after finishing the change, so I'll just go ahead making the > >>>> change. > >>>> Since this is blocking release candidate, we should get it ASAP. > That's > >>> why > >>>> I'm eager to go ahead making the change. If you could spend time now > >>>> helping with making the change ASAP, please leave short notice (maybe > >>> with > >>>> JIRA issue?) and go ahead. > >>>> > >>>> Thanks, > >>>> Jungtaek Lim (HeartSaVioR) > >>>> > >>>> 2018년 2월 6일 (화) 오전 9:41, Erik Weathers <[email protected] > >>>> 님이 > >>>> 작성: > >>>> > >>>>> hey Jungtaek, > >>>>> > >>>>> Thanks for continuing to pursue this! > >>>>> > >>>>> The issue for Storm not working on Mesos is due to a fundamental > >>> change > >>>> to > >>>>> the core scheduling logic in Storm: > >>>>> > >>>>> - > >>>>> > >>>>> https://issues.apache.org/jira/browse/STORM-2126?focusedComm > >>>> entId=16136150&page=com.atlassian.jira.plugin.system. > >>>> issuetabpanels%3Acomment-tabpanel#comment-16136150 > >>>>> > >>>>> The yet-to-be-ironed-out solution that Bobby was brainstorming about > >>>> isn't > >>>>> a short term fix as far as I understand it. I believe it to be many > >>> many > >>>>> months (years?) out for it to actually be workable. Per my naive > >>>>> understanding of the proposal, we'd probably have to completely > >>> rewrite > >>>> the > >>>>> Storm-on-Mesos framework. So it's probably the right long-term > >>> solution, > >>>>> but it isn't anything that should impact this discussion. > >>>>> > >>>>>> The thing is, even users pick storm-kafka-client 1.1.x/1.2.0 and > >>>> include > >>>>> it into their topology jar, it will also not work with Storm 1.0.x. > It > >>>>> even can't > >>>>> compile. > >>>>> > >>>>> FWIW, I'm pretty sure that I was able to successfully run > >>>>> storm-kafka-client-1.1.x on a 1.0.5 storm cluster, but only after > >>> shading > >>>>> in storm-core-1.1.x to the topology uber jar. There was *at least* > a > >>>>> change to some timer-related class in storm-core in 1.1.x (something > >>>> about > >>>>> milliseconds IIRC -- it's been 1.5 months since I did it, need to > >>> revisit > >>>>> the process I followed). > >>>>> > >>>>> I'm happy to help with backporting / stomping storm-kafka-client in > >>>> 1.0.x. > >>>>> Maybe we can talk offline about it? > >>>>> > >>>>> - Erik > >>>>> > >>>>> On Mon, Feb 5, 2018 at 4:20 PM, Jungtaek Lim <[email protected]> > >>> wrote: > >>>>> > >>>>>> UPDATE: Looks like we changed some parts of storm-core while fixing > >>>>>> storm-kafka-client issues (especially went in 1.1.0), hence > >>> overwriting > >>>>>> also incurs changes of storm-core. It doesn't look like a big deal > >>> for > >>>>>> 1.1.x-branch, but there looks like needed many changes for > >>>> 1.0.x-branch. > >>>>>> > >>>>>> The thing is, even users pick storm-kafka-client 1.1.x/1.2.0 and > >>>> include > >>>>> it > >>>>>> into their topology jar, it will also not work with Storm 1.0.x. It > >>>> even > >>>>>> can't compile. > >>>>>> > >>>>>> 1.0.x version line was long lived (22 months) even we released Storm > >>>>> 1.1.0 > >>>>>> at 11 months ago. Instead of struggling 1.0.x-branch to up to date, > >>> I'd > >>>>>> like to suggest that we define 1.0.x-branch as deprecated with > >>> guiding > >>>> to > >>>>>> update to latest 1.1.x version or 1.2.0 (after release), and try to > >>>>> resolve > >>>>>> storm-mesos issue with Storm 1.1.0 ASAP to resolve Erik's concern. > >>>>>> > >>>>>> Makes sense? I'll continue working on 1.1.x-branch and update > >>> anyway. > >>>>>> > >>>>>> -Jungtaek Lim (HeartSaVioR) > >>>>>> > >>>>>> 2018년 2월 6일 (화) 오전 7:53, Jungtaek Lim <[email protected]>님이 작성: > >>>>>> > >>>>>>> OK. No more opinion/vote in 5 days. I'll treat consensus was made, > >>>> and > >>>>> go > >>>>>>> ahead making change: overwrite storm-kafka-client 1.2.0 to two > >>>> branches > >>>>>>> 1.1.x/1.0.x. > >>>>>>> > >>>>>>> -Jungtaek Lim (HeartSaVioR) > >>>>>>> > >>>>>>> 2018년 2월 1일 (목) 오전 10:48, Jungtaek Lim <[email protected]>님이 작성: > >>>>>>> > >>>>>>>> This discussion got 4 +1 (binding) and no -1. Moreover two active > >>>>>>>> maintainers for storm-kafka-client (Hugo and Stig) voted +1. > >>>>>>>> > >>>>>>>> Do we want to hold on for hearing more voices, or treating above > >>>>>> opinions > >>>>>>>> as consensus and reflect the change? > >>>>>>>> > >>>>>>>> Btw, I think we need to sort out the sequences between two > >>> topics: > >>>>>>>> separating storm-kafka-client as independent release cycle, and > >>>> this. > >>>>> I > >>>>>>>> guess some of us agreed former topic doesn't related to current > >>> RC, > >>>>> but > >>>>>> I > >>>>>>>> think this topic can be (should be) reflected to current RC > >>> ongoing. > >>>>>>>> > >>>>>>>> -Jungtaek Lim (HeartSaVioR) > >>>>>>>> > >>>>>>>> 2018년 2월 1일 (목) 오전 4:08, Hugo Da Cruz Louro < > >>> [email protected] > >>>>>> 님이 > >>>>>>>> 작성: > >>>>>>>> > >>>>>>>>> +1 to replace storm-kafka-client in 1.0.x branch. > >>>>>>>>> Hugo > >>>>>>>>> > >>>>>>>>>> On Jan 31, 2018, at 11:03 AM, Stig Rohde Døssing < > >>>>>>>>> [email protected]> wrote: > >>>>>>>>>> > >>>>>>>>>> +1 to replace storm-kafka-client in 1.0.x branch. Breaking > >>>> semantic > >>>>>>>>>> versioning is really nasty, but I think it is the lesser evil > >>> in > >>>>> this > >>>>>>>>> case. > >>>>>>>>>> > >>>>>>>>>> 2018-01-31 5:14 GMT+01:00 Harsha <[email protected]>: > >>>>>>>>>> > >>>>>>>>>>> +1 to replace storm-kafka-client in 1.0.x branch > >>>>>>>>>>> -Harsha > >>>>>>>>>>> On Tue, Jan 30, 2018, at 7:04 PM, Jungtaek Lim wrote: > >>>>>>>>>>>> Bump up this thread so that we could reach consensus > >>> earlier. > >>>>> Given > >>>>>>>>> that > >>>>>>>>>>> we > >>>>>>>>>>>> got concern related to this, I think it is ideal to release > >>>>>>>>> 1.1.x/1.0.x > >>>>>>>>>>>> with making decision and applying the change if we want. > >>>>>>>>>>>> > >>>>>>>>>>>> 2018년 1월 30일 (화) 오전 9:25, Jungtaek Lim <[email protected] > >>>> 님이 > >>>> 작성: > >>>>>>>>>>>> > >>>>>>>>>>>>> Erik's concern brought from 1.0.6 RC1, because they can't > >>> use > >>>>>> Storm > >>>>>>>>>>> 1.1.0 > >>>>>>>>>>>>> or higher (Storm 1.1.0 broke storm-mesos.). While he could > >>>> take > >>>>> an > >>>>>>>>>>>>> workaround to use storm-kafka-client 1.2.0 or 1.1.2 (if we > >>>>> decide > >>>>>> to > >>>>>>>>>>>>> replace) with Storm 1.0.6, it would be better if we don't > >>>> allow > >>>>>>>>> leaving > >>>>>>>>>>>>> storm-kafka-client in 1.0.x in inconsistent state. > >>>>>>>>>>>>> > >>>>>>>>>>>>> IMHO, breaking backward compatibility is worse, but leaving > >>>>> broken > >>>>>>>>>>> thing > >>>>>>>>>>>>> is worst. Hence I'm +1 to replace all, with noticing that > >>> it > >>>> may > >>>>>>>>> bring > >>>>>>>>>>>>> backward incompatibility in release announce. > >>>>>>>>>>>>> > >>>>>>>>>>>>> -Jungtaek Lim (HeartSaVioR) > >>>>>>>>>>>>> > >>>>>>>>>>>>> 2018년 1월 30일 (화) 오전 4:49, P. Taylor Goetz < > >>> [email protected] > >>>>> 님이 > >>>>>> 작성: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> As I mentioned else thread I’m open to this but would > >>> defer > >>>> to > >>>>>>>>>>> community > >>>>>>>>>>>>>> consensus. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> If there’s concern about doing this for 1.0.x, one option > >>>> would > >>>>>> be > >>>>>>>>>>> skip > >>>>>>>>>>>>>> that version line and only apply it to 1.2.0 and 1.1.x. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> -Taylor > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On Jan 29, 2018, at 12:12 AM, Jungtaek Lim < > >>>> [email protected] > >>>>>> > >>>>>>>>>>> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Hi devs, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> This is initial post to separate out discussion topic > >>> from > >>>>> vote > >>>>>>>>>>> thread, > >>>>>>>>>>>>>> and > >>>>>>>>>>>>>>> continue discussing. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Background of the topic: > >>>>>>>>>>>>>>> 1. Only 1.x-branch of storm-kafka-client got stabilized. > >>>>>>>>>>> (relatively) > >>>>>>>>>>>>>>> 2. We would avoid to port back patches to 1.1.x and 1.0.x > >>>>>> because > >>>>>>>>>>>>>> they're > >>>>>>>>>>>>>>> diverged too much. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Downside: > >>>>>>>>>>>>>>> Backward compatibility might be broken for 1.1.x and > >>> 1.0.x. > >>>>> Not > >>>>>>>>>>> sure for > >>>>>>>>>>>>>>> 1.1.x, but at least 1.0.x, since supported Kafka client > >>>>> version > >>>>>> is > >>>>>>>>>>>>>>> different, and if my memory is right, we already applied > >>>>>> backward > >>>>>>>>>>>>>>> incompatible change into storm-kafka-client 1.1.0. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Please put your opinion regarding topic. You're > >>> encouraged > >>>> to > >>>>>> copy > >>>>>>>>>>> your > >>>>>>>>>>>>>>> previous post in vote thread which helps to centralize > >>>>> opinions > >>>>>> in > >>>>>>>>>>>>>> current > >>>>>>>>>>>>>>> thread. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>> Jungtaek Lim (HeartSaVioR) > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>> > >>>>> > >>>> > >>> > >> > >
