As promised, binaries can be found here: http://people.apache.org/~ptgoetz/storm/storm-0.9.2-incubating/dep-frankenbuild/
This is not a release. It is merely a convenience build provided to enable further testing among the community. -Taylor On Mar 26, 2014, at 9:54 PM, P. Taylor Goetz <[email protected]> wrote: > I've submitted pull requests for most of these changes (I think I still need > to submit one for http-client), where there wasn't already one. > > The only problem I saw was with upgrading to clojure 1.6.0. IIRC it had to do > with ns conflicts with "some.*". Falling back to 1.5.1 solved it. I didn't > investigate it. But it seems upgrading to 1.6.x will require some code change. > > Anyway I created a snapshot build with all these updates pulled in. All unit > tests pass, and my basic fully distributed mode smoke tests pass, so I'm > leaning toward a +1 for all. > > However, if anyone who is interested in seeing these upgrades merged has the > time and resources to try it out, that would be a huge help to us. > > I'll post the binaries soon, when I have access to a real computer. > > -Taylor > >> On Mar 26, 2014, at 10:49 AM, Brian O'Neill <[email protected]> wrote: >> >> >> I’m +1 on all of those (including the clojure version bump). >> >> We’ve been hit with the maven-clojure-plugin issue as well, with no real >> way to diagnose. >> >> -brian >> >> --- >> Brian O'Neill >> Chief Technology Officer >> >> Health Market Science >> The Science of Better Results >> 2700 Horizon Drive · King of Prussia, PA · 19406 >> M: 215.588.6024 · @boneill42 <http://www.twitter.com/boneill42> · >> healthmarketscience.com >> >> This information transmitted in this email message is for the intended >> recipient only and may contain confidential and/or privileged material. If >> you received this email in error and are not the intended recipient, or >> the person responsible to deliver it to the intended recipient, please >> contact the sender at the email above and delete this email and any >> attachments and destroy any copies thereof. Any review, retransmission, >> dissemination, copying or other use of, or taking any action in reliance >> upon, this information by persons or entities other than the intended >> recipient is strictly prohibited. >> >> >> >> >> >> >> >>> On 3/26/14, 10:09 AM, "P. Taylor Goetz" <[email protected]> wrote: >>> >>> There are a number of email threads and JIRA issues regarding upgrading >>> various storm dependencies, so I’d like to enumerate them and discuss >>> them in one thread. >>> >>> Here’s the list so far: >>> >>> 1. Kryo/Carbonite (STORM-263)[1] >>> 2. Clojure (STORM-265) [2] >>> 3. commons-io (STORM-258) [3] >>> 4. curator (STORM-252) [4] >>> 5. http-client [5] >>> >>> I am +1 for all of the above, with the exception of #2, which I am +0 >>> only because I personally haven’t had a chance to do any testing with >>> newer versions of clojure. I’d be interested to hear if anyone has done >>> any testing with newer versions of clojure. >>> >>> I think we should at least consider a bump to clojure 1.5 since it >>> includes Bobby Evan’s patch that fixes error output getting swallowed >>> (this manifests itself as the maven-clojure-plugin failing without any >>> useful information when there are certain AOT compilation issues — very >>> annoying). >>> >>> - Taylor >>> >>> [1] https://issues.apache.org/jira/browse/STORM-263 >>> [2] https://issues.apache.org/jira/browse/STORM-265 >>> [3] https://issues.apache.org/jira/browse/STORM-258 >>> [4] https://issues.apache.org/jira/browse/STORM-252 >>> [5] >>> http://mail-archives.apache.org/mod_mbox/storm-user/201402.mbox/%3CA732B22 >>> [email protected]%3E >>> >>>> On Mar 26, 2014, at 7:13 AM, Brian O'Neill <[email protected]> wrote: >>>> >>>> Agreed. One of our guys got hung up on that just yesterday. >>>> It isn’t hard to track down the assembly, but it might be worth making >>>> it a bit easier. >>>> >>>> Also, I’d love to see STORM-263 included in the next release. It >>>> seems like a quick win. >>>> >>>> -brian >>>> >>>> --- >>>> Brian O'Neill >>>> Chief Technology Officer >>>> >>>> Health Market Science >>>> The Science of Better Results >>>> 2700 Horizon Drive · King of Prussia, PA · 19406 >>>> M: 215.588.6024 · @boneill42 · healthmarketscience.com >>>> >>>> This information transmitted in this email message is for the intended >>>> recipient only and may contain confidential and/or privileged material. >>>> If you received this email in error and are not the intended recipient, >>>> or the person responsible to deliver it to the intended recipient, >>>> please contact the sender at the email above and delete this email and >>>> any attachments and destroy any copies thereof. Any review, >>>> retransmission, dissemination, copying or other use of, or taking any >>>> action in reliance upon, this information by persons or entities other >>>> than the intended recipient is strictly prohibited. >>>> >>>>> On Mar 26, 2014, at 2:19 AM, Kang Xiao <[email protected]> wrote: >>>>> >>>>> hi guys >>>>> >>>>> How about adding a bin/build_release.sh (http://build_release.sh) >>>>> script in this 0.9.2-incubating release? >>>>> >>>>> Since some guys asked the question about building storm release >>>>> package more than once in the mail list. >>>>> >>>>> -- >>>>> Best Regards! >>>>> >>>>> 肖康(Kang Xiao,<[email protected] (mailto:[email protected])>) >>>>> Distributed Software Engineer >>>>> >>>>>> 在 2014年3月25日 星期二，1:38，Suresh Srinivas 写道： >>>>>> Taylor, I am not very clear on "Lazy consensus +2". By this >>>>>> definition code >>>>>> can committed with no +1 from a committer, right? >>>>>> >>>>>> You may want to look at Hadoop bylaws - >>>>>> https://hadoop.apache.org/bylaws.html >>>>>> >>>>>> The code commits in Hadoop are consensus approval with minimum +1 >>>>>> from an >>>>>> active committer and no veto. This has worked well in my experience. >>>>>> It may >>>>>> be a good idea to also adapt minimum 3 +1s from active committers for >>>>>> merging feature branches. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Fri, Mar 21, 2014 at 6:22 PM, P. Taylor Goetz <[email protected] >>>>>> (mailto:[email protected])> wrote: >>>>>> >>>>>>> That brings up a good point that probably deserves a separate thread. >>>>>>> >>>>>>> We should establish by-laws soon. Specifically a commit/merge policy. >>>>>>> >>>>>>> For code changes, I've been operating under a "lazy consensus +2" >>>>>>> model: 2 >>>>>>> committer +1 votes and no vetoes (-1). If a committer submits the >>>>>>> patch, >>>>>>> that's an implicit +1. Unless it's a somewhat urgent fix, I've been >>>>>>> waiting >>>>>>> for 3 binding votes and no vetoes. >>>>>>> >>>>>>> That's kind of a middle ground between the traditional code >>>>>>> modification >>>>>>> rule and lazy consensus [1]. >>>>>>> >>>>>>> When wearing my "release manager" hat, I've also interpreted "code >>>>>>> change" >>>>>>> to mean "anything that alters the behavior of the software we >>>>>>> produce." In >>>>>>> terms of the build/packaging I've been a little looser. For large >>>>>>> changes >>>>>>> (e.g. The switch to maven), I've waited for 3 binding votes. For some >>>>>>> changes I've committed directly -- I don't think we need to have a >>>>>>> 3-day >>>>>>> vote on updating the CHANGELOG, for example. >>>>>>> >>>>>>> Anyway, it's something to think about. Sorry for hijacking the >>>>>>> thread. >>>>>>> >>>>>>> +1 (again ;) ) >>>>>>> >>>>>>> Taylor >>>>>>> >>>>>>> [1] https://www.apache.org/foundation/voting.html >>>>>>> >>>>>>>> On Mar 21, 2014, at 7:48 PM, Nathan Marz <[email protected] >>>>>>>> (mailto:[email protected])> wrote: >>>>>>>> >>>>>>>> Let's get https://issues.apache.org/jira/browse/STORM-262 in there. >>>>>>>> Just >>>>>>>> one more vote needed by a committer. >>>>>>>> >>>>>>>> >>>>>>>>> On Fri, Mar 21, 2014 at 3:21 PM, Patrick Lucas <[email protected] >>>>>>>>> (mailto:[email protected])> wrote: >>>>>>>>> >>>>>>>>> A fix STORM-120 would be greatly appreciated. It's making it >>>>>>>>> impossible >>>>>>> to >>>>>>>>> increase tasks/executors > 1 when there is a downstream shuffle >>>>>>>> >>>>>>> >>>>>>> grouping. >>>>>>>>> >>>>>>>>> I'm not sure why there haven't been more reports of problems with >>>>>>>>> it. >>>>>>> Two >>>>>>>>> possibilities I can think of are that we are using exclusively >>>>>>>>> shell >>>>>>>>> components--perhaps there's a root-cause bug in those component >>>>>>>>> classes--and >>>>>>>>> that we are dealing with a high volume stream of large tuples. >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> (thousands / >>>>>>>>> sec, KB in size) >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thu, Mar 20, 2014 at 2:14 PM, P. Taylor Goetz >>>>>>>>> <[email protected] (mailto:[email protected])> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Never mind... just found it. >>>>>>>>>> >>>>>>>>>>> On Mar 20, 2014, at 5:09 PM, P. Taylor Goetz <[email protected] >>>>>>>>>>> (mailto:[email protected])> >>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> Derek do you have an idea for a fix? >>>>>>>>>>> >>>>>>>>>>> On Mar 20, 2014, at 3:43 PM, Derek Dagit <[email protected] >>>>>>>>>>> (mailto:[email protected])> >>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>>> As I said above, this fix is the most important in my opinion. >>>>>>>>>>>>> STORM-259 (Random#nextInt) is new to me -- can't say whether >>>>>>>>>>>>> it's as >>>>>>>>>>>>> important as STORM-187 or not. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Yeah, we found it recently, and I created it this morning after >>>>>>>>> reading >>>>>>>>>> Taylor's mail. >>>>>>>>>>>> >>>>>>>>>>>> STORM-187 can be a problem with fewer than 30 retries >>>>>>>>>>>> (likelihood >>>>>>>>>> depends on configuration), but we will hit STORM-259 when retries >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> exceeds >>>>>>>>>> 30. >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Derek >>>>>>>>>>>> >>>>>>>>>>>>> On 3/20/14, 14:18, Michael G. Noll wrote: >>>>>>>>>>>>> On my side the most important change is, as you point out, >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> STORM-187. >>>>>>>>>>>>> The primary reason is like Adam Lewis is pointing out because >>>>>>>>>>>>> it's a >>>>>>>>>>>>> stability problem. The secondary aspect is that this issue >>>>>>>>>>>>> taints >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> the >>>>>>>>>>>>> new Netty backend, and at least IMHO the faster Storm could >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> confidently >>>>>>>>>>>>> bury ZeroMQ the better. :-) >>>>>>>>>>>>> >>>>>>>>>>>>> As I said above, this fix is the most important in my opinion. >>>>>>>>>>>>> STORM-259 (Random#nextInt) is new to me -- can't say whether >>>>>>>>>>>>> it's as >>>>>>>>>>>>> important as STORM-187 or not. >>>>>>>>>>>>> >>>>>>>>>>>>> Switching to my non-essential wishlist I'd also +1 STORM-252 >>>>>>> (Upgrade >>>>>>>>>>>>> Curator and thus ZooKeeper to 3.4.5). We have been running ZK >>>>>>>>>>>>> 3.4.5 >>>>>>>>>>>>> anyway for a couple of reasons, and it would be nice to have >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> official >>>>>>>>>>>>> Storm support for the latest ZK version (ok, the recently >>>>>>>>>>>>> released >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> ZK >>>>>>>>>>>>> 3.4.6 is actually the latest but hey). Although I don't know >>>>>>>>>>>>> how >>>>>>>>>>>>> confident we are that the code in STORM-252 actually works, >>>>>>>>>>>>> i.e. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> whether >>>>>>>>>>>>> integrating STORM-252 into 0.9.2 on such short notice would be >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> jumping >>>>>>>>>>>>> the gun or a safe move. >>>>>>>>>>>>> >>>>>>>>>>>>> Btw, in terms of Storm/Kafka integration Kafka is in the same >>>>>>>>>>>>> boat: >>>>>>>>>>>>> it's built against ZK 3.3.x, and LinkedIn recommends the use >>>>>>>>>>>>> of ZK >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 3.3.4 >>>>>>>>>>>>> in the docs. There's an open ticket KAFKA-854 [1] that's >>>>>>>>>>>>> basically >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> the >>>>>>>>>>>>> equivalent of STORM-252, but I'm not sure how actively the >>>>>>>>>>>>> Kafka >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> team >>>>>>>>>> is >>>>>>>>>>>>> working on that. >>>>>>>>>>>>> >>>>>>>>>>>>> Best, >>>>>>>>>>>>> Michael >>>>>>>>>>>>> >>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/KAFKA-854 >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> On 03/20/2014 02:33 AM, P. Taylor Goetz wrote: >>>>>>>>>>>>>> I'd like to get this discussion started, largely because the >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> "negative timeout" bug (STORM-187) really bothers me. I've not >>>>>>>>>> seen it >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> in >>>>>>>>>> the wild, but I've heard of a few cases where it was enough to >>>>>>>>>> hinder >>>>>>>>>> upgrading. >>>>>>>>>>>>>> >>>>>>>>>>>>>> HEAD looks good to me at the moment, with the major difference >>>>>>> being >>>>>>>>>> the zookeeper update and the patch mentioned above. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Any thoughts on other PRs or patches to include? >>>>>>>>>>>>>> >>>>>>>>>>>>>> -Taylor >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Patrick Lucas >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Twitter: @nathanmarz >>>>>>>> http://nathanmarz.com >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> http://hortonworks.com/download/ >>>>>> >>>>>> -- >>>>>> CONFIDENTIALITY NOTICE >>>>>> NOTICE: This message is intended for the use of the individual or >>>>>> entity to >>>>>> which it is addressed and may contain information that is >>>>>> confidential, >>>>>> privileged and exempt from disclosure under applicable law. If the >>>>>> reader >>>>>> of this message is not the intended recipient, you are hereby >>>>>> notified that >>>>>> any printing, copying, dissemination, distribution, disclosure or >>>>>> forwarding of this communication is strictly prohibited. If you have >>>>>> received this communication in error, please contact the sender >>>>>> immediately >>>>>> and delete it from your system. Thank You. >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>> >>> >> >>
signature.asc
Description: Message signed with OpenPGP using GPGMail
