@Alexandre if you want to build the minimum possible, from the repo root directory run:
mvn clean install -DskipTests -f external/storm-kafka-client/pom.xml The jars will be under external/storm-kafka-client/target. You can replace the existing storm-kafka-client Jar in your installation or put it under extlib and remove any other storm-kafka-client jars that you may have. Thanks for helping out. Hugo > On Nov 18, 2017, at 5:54 AM, Stig Rohde Døssing <[email protected]> > wrote: > > Alexandre, > > I've uploaded the 1.2.0-SNAPSHOT storm-kafka-client jar here > https://drive.google.com/file/d/1DgJWjhWwczYgZS82YGd63V3GT2G_v9fd/view?usp=sharing. > You can probably keep the 1.1.0 versions of storm-hbase and storm-hdfs, > since the core Storm API hasn't changed. > > If you want to build the jars yourself, clone the repository at > https://github.com/apache/storm and check out the 1.x-branch branch. You > can build the entire project by running "mvn clean install -DskipTests" in > the root. The individual module jars will be available in the target > directories for each module, e.g. storm-kafka-client.jar will be in > external/storm-kafka-client/target. > > If you need to build the regular Storm distribution (the tarball Jungtaek > sent you), you can go to storm-dist/binary and run "mvn clean install > -Dgpg.skip". The tar/zip will be in storm-dist/binary/target. > > Further reference in case you need it > https://github.com/apache/storm/blob/master/DEVELOPER.md#build-the-code-and-run-the-tests > . > > 2017-11-18 12:57 GMT+01:00 Alexandre Vermeerbergen <[email protected] >> : > >> Hello Jungtaek, >> >> Thanks for the link to 1.2.0 preview binaries. >> However, since we heavily depend on Storm Kafka Client, would you please >> either add it or remind me how to build the jars of this external lib, at >> same "snapshot" version as the rest ? >> On a side note, some of our topologies also rely on storm hbase & storm >> hdfs : does it matters if these later ones stay in 1.1.0 version for this >> test? >> Indeed, I want to focus on Storm 1.2.0 "core" + Storm Kafka Client 1.2.0 >> >> Best regards, >> Alexandre Vermeerbergen >> >> >> 2017-11-18 7:41 GMT+01:00 Jungtaek Lim <[email protected]>: >> >>> Alexandre, >>> >>> https://www.dropbox.com/s/mg2gnunk24oesyc/apache-storm- >>> 1.2.0-SNAPSHOT.tar.gz?dl=0 >>> >>> Above link is custom binary distribution of current 1.x-branch (SNAPSHOT >> of >>> 1.2.0). Could you run the test in your environment first and let us know >>> about the result? Regardless of including metrics V2 to 1.2.0 or not, >> your >>> test report should be valuable for us. >>> >>> And please refer my analysis of current metrics: >>> https://cwiki.apache.org/confluence/display/STORM/ >> Limitations+of+current+ >>> metrics+feature >>> to >>> see why we want to move toward to Metrics V2. Current PR on Metrics V2 is >>> for initial state and it could cover only some of issues in list, but I >>> expect we will address another issues as well based on the PR. >>> >>> Thanks in advance! >>> Jungtaek Lim (HeartSaVioR) >>> >>> 2017년 11월 18일 (토) 오전 6:40, Alexandre Vermeerbergen < >>> [email protected]>님이 >>> 작성: >>> >>>> Hello Hugo, >>>> >>>> As I already posted, I was getting ready to test the upcoming RC, so >>> yeah : >>>> as soon as I have some binaries for testing, I can run any Storm 1.2.0 >>>> preview on my pre-production (well stressed) environment. Is there some >>> URL >>>> from where I can download such "snapshot" binaries? >>>> >>>> >>>> Best regards, >>>> Alexandre Vermeerbergen >>>> >>>> PS: By the way, I have no interest in the metrics-related enhancements >>> you >>>> mentionned - I can understand it is of interest for people aware of it >>> and >>>> who are expecting "something" related to it - but I would be sad to >> see a >>>> postponing of the imminent Storm 1.2.0 release for something we do not >>> use >>>> at all. >>>> PPS: I would love to learn more about these "metrics", maybe it's >>> something >>>> worth which never quite understood... any link with clear explanation >> of >>>> this feature? >>>> >>>> >>>> 2017-11-17 17:44 GMT+01:00 Hugo Da Cruz Louro <[email protected] >>> : >>>> >>>>> I am also in agreement that we should not delay the 1.2.0 release for >>> too >>>>> long, but in order to release it a few things need to be kept in >> mind: >>>>> >>>>> - If we want to avoid releasing 1.3.0 soon or at all, that means >> that >>> we >>>>> should include in 1.2.0 as many important features (e.g. metricsV2) >> as >>>>> possible within a reasonable time frame. Also to avoid as much back >>>> porting >>>>> as possible one should really get all features that we foresee we >> want >>> to >>>>> maintain in 1.2.0 right away, and then simply maintain them there. >>>> However >>>>> it’s a fact of life that if a bug is found and it is a blocker, it >> will >>>>> have to be back ported. >>>>> >>>>> - STORM-2153 details that the new metrics requirements were driven by >>> the >>>>> users. Therefore if this feature is highly important and sought >> after, >>> it >>>>> may not make much sense to release 1.2.0 without it. If we do release >>>> 1.2.0 >>>>> without the metrics, does it mean that metrics will go only in 2.0? I >>>> would >>>>> conjecture that most production deployments will take a while to >>> upgrade >>>> to >>>>> 2.0 even after it is released. That means that they will still be >>> running >>>>> Storm without the benefit of the new metrics. >>>>> >>>>> - Several fixes have got in storm-kafka-client. There have been a >> lot >>> of >>>>> changes and I wonder to which degree they have been system tested in >>>>> addition the existing unit tests >>>>> >>>>> @Alexandre, since you are using storm-kafka-client and have filed >> some >>>>> bugs and driven some feature requests, I would like to ask if you >> could >>>>> help us within what is reasonably possible for you with the >> following: >>>>> >>>>> - system test the latest storm-kafka-client changes in your >>>>> test/pre-production environment >>>>> - provide some information on about your setup, >>>>> - how you are using storm-kafka-client >>>>> - how it is performing >>>>> - Kafka brokers, number of topics/partitions, storm >> parallelism, >>>> and >>>>> some info about your network >>>>> - if it is reasonable to do so share some of your tests such that we >>> can >>>>> also test it at our end. >>>>> - Tell us specific things that you would like us to test >>>>> >>>>> Thanks, >>>>> Hugo >>>>> >>>>>> On Nov 16, 2017, at 11:49 PM, Stig Rohde Døssing < >>>> [email protected]> >>>>> wrote: >>>>>> >>>>>> I agree with Jungtaek, if metrics v2 can go in very soon it should >> go >>>> in, >>>>>> otherwise I'd rather release 1.2.0 now and work on getting 2.0.0 >>> ready >>>>> for >>>>>> release. >>>>>> >>>>>> 1.x and master have drifted pretty far apart, and it's causing a >> lot >>> of >>>>>> porting work at this point. 1.x still has a lot of Clojure code, >>>>> storm-core >>>>>> hasn't been split yet in that branch and it's also targeting JDK >> 1.7. >>>>> It's >>>>>> seems rare at this point that a PR cherry picks cleanly onto 1.x >> from >>>>>> master. >>>>>> >>>>>> I've linked the list of issues that are only fixed in 2.0.0 and not >>>> 1.x, >>>>> to >>>>>> illustrate how far ahead 2.0.0 is. >>>>>> >>>>>> https://pste.eu/p/9CJT.html >>>>>> >>>>>> 2017-11-17 5:41 GMT+01:00 Arun Iyer <[email protected]>: >>>>>> >>>>>>> Hi Taylor, >>>>>>> >>>>>>> Is it https://github.com/apache/storm/pull/2203 ? >>>>>>> >>>>>>> >>>>>>> I think it would be great to get this in 1.2 release. Can we try >> to >>>>>>> address the issues in a week or so and get this in? >>>>>>> >>>>>>> Thanks, >>>>>>> Arun >>>>>>> >>>>>>> On 11/17/17, 7:13 AM, "P. Taylor Goetz" <[email protected]> >> wrote: >>>>>>> >>>>>>>> The original idea for the 1.2 release was that would introduce >> the >>>>>>> metrics v2 work (and there is additional work required there, so I >>>>>>> understand a desire not to delay the release). Do we want to stick >>>> with >>>>>>> that, or deviate? If the latter, would we do a 1.3 release for >>>> metrics? >>>>>>>> >>>>>>>> As far as a 1.1.2 release I’m fine with releasing that at any >> time. >>>>>>>> >>>>>>>> -Taylor >>>>>>>> >>>>>>>>> On Nov 15, 2017, at 6:24 PM, Jungtaek Lim <[email protected]> >>>> wrote: >>>>>>>>> >>>>>>>>> I think we could start release phase for both 1.1.2 and 1.2.0 >> when >>>>>>>>> https://github.com/apache/storm/pull/2423 will be merged. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Jungtaek Lim (HeartSaVioR) >>>>>>>>> >>>>>>>>> 2017년 11월 16일 (목) 오전 7:00, Alexandre Vermeerbergen < >>>>>>> [email protected]>님이 >>>>>>>>> 작성: >>>>>>>>> >>>>>>>>>> Hello, >>>>>>>>>> >>>>>>>>>> I'd love to see a Storm 1.2.0 released: it's a perfect schedule >>> for >>>>> me >>>>>>> for >>>>>>>>>> test any Release Candidate that might be available if it >> happens >>>>> soon. >>>>>>>>>> >>>>>>>>>> Best regards, >>>>>>>>>> Alexandre >>>>>>>>>> >>>>>>>>>> 2017-11-15 7:29 GMT+01:00 Arun Mahadevan <[email protected]>: >>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> Looks like we are only waiting on https://issues.apache.org/ >>>>>>>>>>> jira/browse/STORM-2546 . >>>>>>>>>>> >>>>>>>>>>> Are there any other issues which are blockers for Storm 1.2.0? >>>> Would >>>>>>> be >>>>>>>>>>> great to see the 1.2.0 release out soon as it has a lot of >>>> critical >>>>>>>>>> fixes. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Arun >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>>> >>>> >>> >>
