I’m +1 on all of those (including the clojure version bump). We’ve been hit with the maven-clojure-plugin issue as well, with no real way to diagnose.
-brian --- Brian O'Neill Chief Technology Officer Health Market Science The Science of Better Results 2700 Horizon Drive ・ King of Prussia, PA ・ 19406 M: 215.588.6024 ・ @boneill42 <http://www.twitter.com/boneill42> ・ healthmarketscience.com This information transmitted in this email message is for the intended recipient only and may contain confidential and/or privileged material. If you received this email in error and are not the intended recipient, or the person responsible to deliver it to the intended recipient, please contact the sender at the email above and delete this email and any attachments and destroy any copies thereof. Any review, retransmission, dissemination, copying or other use of, or taking any action in reliance upon, this information by persons or entities other than the intended recipient is strictly prohibited. On 3/26/14, 10:09 AM, "P. Taylor Goetz" <[email protected]> wrote: >There are a number of email threads and JIRA issues regarding upgrading >various storm dependencies, so I’d like to enumerate them and discuss >them in one thread. > >Here’s the list so far: > >1. Kryo/Carbonite (STORM-263)[1] >2. Clojure (STORM-265) [2] >3. commons-io (STORM-258) [3] >4. curator (STORM-252) [4] >5. http-client [5] > >I am +1 for all of the above, with the exception of #2, which I am +0 >only because I personally haven’t had a chance to do any testing with >newer versions of clojure. I’d be interested to hear if anyone has done >any testing with newer versions of clojure. > >I think we should at least consider a bump to clojure 1.5 since it >includes Bobby Evan’s patch that fixes error output getting swallowed >(this manifests itself as the maven-clojure-plugin failing without any >useful information when there are certain AOT compilation issues ― very >annoying). > >- Taylor > >[1] https://issues.apache.org/jira/browse/STORM-263 >[2] https://issues.apache.org/jira/browse/STORM-265 >[3] https://issues.apache.org/jira/browse/STORM-258 >[4] https://issues.apache.org/jira/browse/STORM-252 >[5] >http://mail-archives.apache.org/mod_mbox/storm-user/201402.mbox/%3CA732B22 >[email protected]%3E > >On Mar 26, 2014, at 7:13 AM, Brian O'Neill <[email protected]> wrote: > >> Agreed. One of our guys got hung up on that just yesterday. >> It isn’t hard to track down the assembly, but it might be worth making >>it a bit easier. >> >> Also, I’d love to see STORM-263 included in the next release. It >>seems like a quick win. >> >> -brian >> >> --- >> Brian O'Neill >> Chief Technology Officer >> >> Health Market Science >> The Science of Better Results >> 2700 Horizon Drive ・ King of Prussia, PA ・ 19406 >> M: 215.588.6024 ・ @boneill42 ・ healthmarketscience.com >> >> This information transmitted in this email message is for the intended >>recipient only and may contain confidential and/or privileged material. >>If you received this email in error and are not the intended recipient, >>or the person responsible to deliver it to the intended recipient, >>please contact the sender at the email above and delete this email and >>any attachments and destroy any copies thereof. Any review, >>retransmission, dissemination, copying or other use of, or taking any >>action in reliance upon, this information by persons or entities other >>than the intended recipient is strictly prohibited. >> >> On Mar 26, 2014, at 2:19 AM, Kang Xiao <[email protected]> wrote: >> >>> hi guys >>> >>> How about adding a bin/build_release.sh (http://build_release.sh) >>>script in this 0.9.2-incubating release? >>> >>> Since some guys asked the question about building storm release >>>package more than once in the mail list. >>> >>> -- >>> Best Regards! >>> >>> 肖康(Kang Xiao,<[email protected] (mailto:[email protected])>) >>> Distributed Software Engineer >>> >>> 在 2014年3月25日 星期二，1:38，Suresh Srinivas 写道： >>>> Taylor, I am not very clear on "Lazy consensus +2". By this >>>>definition code >>>> can committed with no +1 from a committer, right? >>>> >>>> You may want to look at Hadoop bylaws - >>>> https://hadoop.apache.org/bylaws.html >>>> >>>> The code commits in Hadoop are consensus approval with minimum +1 >>>>from an >>>> active committer and no veto. This has worked well in my experience. >>>>It may >>>> be a good idea to also adapt minimum 3 +1s from active committers for >>>> merging feature branches. >>>> >>>> >>>> >>>> >>>> On Fri, Mar 21, 2014 at 6:22 PM, P. Taylor Goetz <[email protected] >>>>(mailto:[email protected])> wrote: >>>> >>>>> That brings up a good point that probably deserves a separate thread. >>>>> >>>>> We should establish by-laws soon. Specifically a commit/merge policy. >>>>> >>>>> For code changes, I've been operating under a "lazy consensus +2" >>>>>model: 2 >>>>> committer +1 votes and no vetoes (-1). If a committer submits the >>>>>patch, >>>>> that's an implicit +1. Unless it's a somewhat urgent fix, I've been >>>>>waiting >>>>> for 3 binding votes and no vetoes. >>>>> >>>>> That's kind of a middle ground between the traditional code >>>>>modification >>>>> rule and lazy consensus [1]. >>>>> >>>>> When wearing my "release manager" hat, I've also interpreted "code >>>>>change" >>>>> to mean "anything that alters the behavior of the software we >>>>>produce." In >>>>> terms of the build/packaging I've been a little looser. For large >>>>>changes >>>>> (e.g. The switch to maven), I've waited for 3 binding votes. For some >>>>> changes I've committed directly -- I don't think we need to have a >>>>>3-day >>>>> vote on updating the CHANGELOG, for example. >>>>> >>>>> Anyway, it's something to think about. Sorry for hijacking the >>>>>thread. >>>>> >>>>> +1 (again ;) ) >>>>> >>>>> Taylor >>>>> >>>>> [1] https://www.apache.org/foundation/voting.html >>>>> >>>>>> On Mar 21, 2014, at 7:48 PM, Nathan Marz <[email protected] >>>>>>(mailto:[email protected])> wrote: >>>>>> >>>>>> Let's get https://issues.apache.org/jira/browse/STORM-262 in there. >>>>>>Just >>>>>> one more vote needed by a committer. >>>>>> >>>>>> >>>>>>> On Fri, Mar 21, 2014 at 3:21 PM, Patrick Lucas <[email protected] >>>>>>>(mailto:[email protected])> wrote: >>>>>>> >>>>>>> A fix STORM-120 would be greatly appreciated. It's making it >>>>>>>impossible >>>>> to >>>>>>> increase tasks/executors > 1 when there is a downstream shuffle >>>>>> >>>>> >>>>> grouping. >>>>>>> >>>>>>> I'm not sure why there haven't been more reports of problems with >>>>>>>it. >>>>> Two >>>>>>> possibilities I can think of are that we are using exclusively >>>>>>>shell >>>>>>> components--perhaps there's a root-cause bug in those component >>>>>>> classes--and >>>>>>> that we are dealing with a high volume stream of large tuples. >>>>>>> >>>>>> >>>>> >>>>> (thousands / >>>>>>> sec, KB in size) >>>>>>> >>>>>>> >>>>>>> On Thu, Mar 20, 2014 at 2:14 PM, P. Taylor Goetz >>>>>>><[email protected] (mailto:[email protected])> >>>>>>> wrote: >>>>>>> >>>>>>>> Never mind... just found it. >>>>>>>> >>>>>>>>> On Mar 20, 2014, at 5:09 PM, P. Taylor Goetz <[email protected] >>>>>>>>>(mailto:[email protected])> >>>>> wrote: >>>>>>>>> >>>>>>>>> Derek do you have an idea for a fix? >>>>>>>>> >>>>>>>>> On Mar 20, 2014, at 3:43 PM, Derek Dagit <[email protected] >>>>>>>>>(mailto:[email protected])> >>>>> wrote: >>>>>>>>> >>>>>>>>>>> As I said above, this fix is the most important in my opinion. >>>>>>>>>>> STORM-259 (Random#nextInt) is new to me -- can't say whether >>>>>>>>>>>it's as >>>>>>>>>>> important as STORM-187 or not. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Yeah, we found it recently, and I created it this morning after >>>>>>> reading >>>>>>>> Taylor's mail. >>>>>>>>>> >>>>>>>>>> STORM-187 can be a problem with fewer than 30 retries >>>>>>>>>>(likelihood >>>>>>>> depends on configuration), but we will hit STORM-259 when retries >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> exceeds >>>>>>>> 30. >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Derek >>>>>>>>>> >>>>>>>>>>> On 3/20/14, 14:18, Michael G. Noll wrote: >>>>>>>>>>> On my side the most important change is, as you point out, >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>>> STORM-187. >>>>>>>>>>> The primary reason is like Adam Lewis is pointing out because >>>>>>>>>>>it's a >>>>>>>>>>> stability problem. The secondary aspect is that this issue >>>>>>>>>>>taints >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> the >>>>>>>>>>> new Netty backend, and at least IMHO the faster Storm could >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> confidently >>>>>>>>>>> bury ZeroMQ the better. :-) >>>>>>>>>>> >>>>>>>>>>> As I said above, this fix is the most important in my opinion. >>>>>>>>>>> STORM-259 (Random#nextInt) is new to me -- can't say whether >>>>>>>>>>>it's as >>>>>>>>>>> important as STORM-187 or not. >>>>>>>>>>> >>>>>>>>>>> Switching to my non-essential wishlist I'd also +1 STORM-252 >>>>> (Upgrade >>>>>>>>>>> Curator and thus ZooKeeper to 3.4.5). We have been running ZK >>>>>>>>>>>3.4.5 >>>>>>>>>>> anyway for a couple of reasons, and it would be nice to have >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>>> official >>>>>>>>>>> Storm support for the latest ZK version (ok, the recently >>>>>>>>>>>released >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>>> ZK >>>>>>>>>>> 3.4.6 is actually the latest but hey). Although I don't know >>>>>>>>>>>how >>>>>>>>>>> confident we are that the code in STORM-252 actually works, >>>>>>>>>>>i.e. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> whether >>>>>>>>>>> integrating STORM-252 into 0.9.2 on such short notice would be >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> jumping >>>>>>>>>>> the gun or a safe move. >>>>>>>>>>> >>>>>>>>>>> Btw, in terms of Storm/Kafka integration Kafka is in the same >>>>>>>>>>>boat: >>>>>>>>>>> it's built against ZK 3.3.x, and LinkedIn recommends the use >>>>>>>>>>>of ZK >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> 3.3.4 >>>>>>>>>>> in the docs. There's an open ticket KAFKA-854 [1] that's >>>>>>>>>>>basically >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> the >>>>>>>>>>> equivalent of STORM-252, but I'm not sure how actively the >>>>>>>>>>>Kafka >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> team >>>>>>>> is >>>>>>>>>>> working on that. >>>>>>>>>>> >>>>>>>>>>> Best, >>>>>>>>>>> Michael >>>>>>>>>>> >>>>>>>>>>> [1] https://issues.apache.org/jira/browse/KAFKA-854 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> On 03/20/2014 02:33 AM, P. Taylor Goetz wrote: >>>>>>>>>>>> I'd like to get this discussion started, largely because the >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> "negative timeout" bug (STORM-187) really bothers me. I've not >>>>>>>>seen it >>>>>>>> >>>>>>> >>>>>> >>>>> >>>>> in >>>>>>>> the wild, but I've heard of a few cases where it was enough to >>>>>>>>hinder >>>>>>>> upgrading. >>>>>>>>>>>> >>>>>>>>>>>> HEAD looks good to me at the moment, with the major difference >>>>> being >>>>>>>> the zookeeper update and the patch mentioned above. >>>>>>>>>>>> >>>>>>>>>>>> Any thoughts on other PRs or patches to include? >>>>>>>>>>>> >>>>>>>>>>>> -Taylor >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Patrick Lucas >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Twitter: @nathanmarz >>>>>> http://nathanmarz.com >>>>>> >>>>> >>>>> >>>> >>>> >>>> >>>> >>>> -- >>>> http://hortonworks.com/download/ >>>> >>>> -- >>>> CONFIDENTIALITY NOTICE >>>> NOTICE: This message is intended for the use of the individual or >>>>entity to >>>> which it is addressed and may contain information that is >>>>confidential, >>>> privileged and exempt from disclosure under applicable law. If the >>>>reader >>>> of this message is not the intended recipient, you are hereby >>>>notified that >>>> any printing, copying, dissemination, distribution, disclosure or >>>> forwarding of this communication is strictly prohibited. If you have >>>> received this communication in error, please contact the sender >>>>immediately >>>> and delete it from your system. Thank You. >>>> >>>> >>>> >>> >>> >> >
