There are a number of email threads and JIRA issues regarding upgrading various storm dependencies, so I’d like to enumerate them and discuss them in one thread.
Here’s the list so far: 1. Kryo/Carbonite (STORM-263)[1] 2. Clojure (STORM-265) [2] 3. commons-io (STORM-258) [3] 4. curator (STORM-252) [4] 5. http-client [5] I am +1 for all of the above, with the exception of #2, which I am +0 only because I personally haven’t had a chance to do any testing with newer versions of clojure. I’d be interested to hear if anyone has done any testing with newer versions of clojure. I think we should at least consider a bump to clojure 1.5 since it includes Bobby Evan’s patch that fixes error output getting swallowed (this manifests itself as the maven-clojure-plugin failing without any useful information when there are certain AOT compilation issues — very annoying). - Taylor [1] https://issues.apache.org/jira/browse/STORM-263 [2] https://issues.apache.org/jira/browse/STORM-265 [3] https://issues.apache.org/jira/browse/STORM-258 [4] https://issues.apache.org/jira/browse/STORM-252 [5] http://mail-archives.apache.org/mod_mbox/storm-user/201402.mbox/%[email protected]%3E On Mar 26, 2014, at 7:13 AM, Brian O'Neill <[email protected]> wrote: > Agreed. One of our guys got hung up on that just yesterday. > It isn’t hard to track down the assembly, but it might be worth making it a > bit easier. > > Also, I’d love to see STORM-263 included in the next release. It seems like > a quick win. > > -brian > > --- > Brian O'Neill > Chief Technology Officer > > Health Market Science > The Science of Better Results > 2700 Horizon Drive • King of Prussia, PA • 19406 > M: 215.588.6024 • @boneill42 • healthmarketscience.com > > This information transmitted in this email message is for the intended > recipient only and may contain confidential and/or privileged material. If > you received this email in error and are not the intended recipient, or the > person responsible to deliver it to the intended recipient, please contact > the sender at the email above and delete this email and any attachments and > destroy any copies thereof. Any review, retransmission, dissemination, > copying or other use of, or taking any action in reliance upon, this > information by persons or entities other than the intended recipient is > strictly prohibited. > > On Mar 26, 2014, at 2:19 AM, Kang Xiao <[email protected]> wrote: > >> hi guys >> >> How about adding a bin/build_release.sh (http://build_release.sh) script in >> this 0.9.2-incubating release? >> >> Since some guys asked the question about building storm release package more >> than once in the mail list. >> >> -- >> Best Regards! >> >> 肖康(Kang Xiao,<[email protected] (mailto:[email protected])>) >> Distributed Software Engineer >> >> 在 2014年3月25日 星期二，1:38，Suresh Srinivas 写道： >>> Taylor, I am not very clear on "Lazy consensus +2". By this definition code >>> can committed with no +1 from a committer, right? >>> >>> You may want to look at Hadoop bylaws - >>> https://hadoop.apache.org/bylaws.html >>> >>> The code commits in Hadoop are consensus approval with minimum +1 from an >>> active committer and no veto. This has worked well in my experience. It may >>> be a good idea to also adapt minimum 3 +1s from active committers for >>> merging feature branches. >>> >>> >>> >>> >>> On Fri, Mar 21, 2014 at 6:22 PM, P. Taylor Goetz <[email protected] >>> (mailto:[email protected])> wrote: >>> >>>> That brings up a good point that probably deserves a separate thread. >>>> >>>> We should establish by-laws soon. Specifically a commit/merge policy. >>>> >>>> For code changes, I've been operating under a "lazy consensus +2" model: 2 >>>> committer +1 votes and no vetoes (-1). If a committer submits the patch, >>>> that's an implicit +1. Unless it's a somewhat urgent fix, I've been waiting >>>> for 3 binding votes and no vetoes. >>>> >>>> That's kind of a middle ground between the traditional code modification >>>> rule and lazy consensus [1]. >>>> >>>> When wearing my "release manager" hat, I've also interpreted "code change" >>>> to mean "anything that alters the behavior of the software we produce." In >>>> terms of the build/packaging I've been a little looser. For large changes >>>> (e.g. The switch to maven), I've waited for 3 binding votes. For some >>>> changes I've committed directly -- I don't think we need to have a 3-day >>>> vote on updating the CHANGELOG, for example. >>>> >>>> Anyway, it's something to think about. Sorry for hijacking the thread. >>>> >>>> +1 (again ;) ) >>>> >>>> Taylor >>>> >>>> [1] https://www.apache.org/foundation/voting.html >>>> >>>>> On Mar 21, 2014, at 7:48 PM, Nathan Marz <[email protected] >>>>> (mailto:[email protected])> wrote: >>>>> >>>>> Let's get https://issues.apache.org/jira/browse/STORM-262 in there. Just >>>>> one more vote needed by a committer. >>>>> >>>>> >>>>>> On Fri, Mar 21, 2014 at 3:21 PM, Patrick Lucas <[email protected] >>>>>> (mailto:[email protected])> wrote: >>>>>> >>>>>> A fix STORM-120 would be greatly appreciated. It's making it impossible >>>> to >>>>>> increase tasks/executors > 1 when there is a downstream shuffle >>>>> >>>> >>>> grouping. >>>>>> >>>>>> I'm not sure why there haven't been more reports of problems with it. >>>> Two >>>>>> possibilities I can think of are that we are using exclusively shell >>>>>> components--perhaps there's a root-cause bug in those component >>>>>> classes--and >>>>>> that we are dealing with a high volume stream of large tuples. >>>>>> >>>>> >>>> >>>> (thousands / >>>>>> sec, KB in size) >>>>>> >>>>>> >>>>>> On Thu, Mar 20, 2014 at 2:14 PM, P. Taylor Goetz <[email protected] >>>>>> (mailto:[email protected])> >>>>>> wrote: >>>>>> >>>>>>> Never mind... just found it. >>>>>>> >>>>>>>> On Mar 20, 2014, at 5:09 PM, P. Taylor Goetz <[email protected] >>>>>>>> (mailto:[email protected])> >>>> wrote: >>>>>>>> >>>>>>>> Derek do you have an idea for a fix? >>>>>>>> >>>>>>>> On Mar 20, 2014, at 3:43 PM, Derek Dagit <[email protected] >>>>>>>> (mailto:[email protected])> >>>> wrote: >>>>>>>> >>>>>>>>>> As I said above, this fix is the most important in my opinion. >>>>>>>>>> STORM-259 (Random#nextInt) is new to me -- can't say whether it's as >>>>>>>>>> important as STORM-187 or not. >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Yeah, we found it recently, and I created it this morning after >>>>>> reading >>>>>>> Taylor's mail. >>>>>>>>> >>>>>>>>> STORM-187 can be a problem with fewer than 30 retries (likelihood >>>>>>> depends on configuration), but we will hit STORM-259 when retries >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> exceeds >>>>>>> 30. >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Derek >>>>>>>>> >>>>>>>>>> On 3/20/14, 14:18, Michael G. Noll wrote: >>>>>>>>>> On my side the most important change is, as you point out, >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> STORM-187. >>>>>>>>>> The primary reason is like Adam Lewis is pointing out because it's a >>>>>>>>>> stability problem. The secondary aspect is that this issue taints >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> the >>>>>>>>>> new Netty backend, and at least IMHO the faster Storm could >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> confidently >>>>>>>>>> bury ZeroMQ the better. :-) >>>>>>>>>> >>>>>>>>>> As I said above, this fix is the most important in my opinion. >>>>>>>>>> STORM-259 (Random#nextInt) is new to me -- can't say whether it's as >>>>>>>>>> important as STORM-187 or not. >>>>>>>>>> >>>>>>>>>> Switching to my non-essential wishlist I'd also +1 STORM-252 >>>> (Upgrade >>>>>>>>>> Curator and thus ZooKeeper to 3.4.5). We have been running ZK 3.4.5 >>>>>>>>>> anyway for a couple of reasons, and it would be nice to have >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> official >>>>>>>>>> Storm support for the latest ZK version (ok, the recently released >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> ZK >>>>>>>>>> 3.4.6 is actually the latest but hey). Although I don't know how >>>>>>>>>> confident we are that the code in STORM-252 actually works, i.e. >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> whether >>>>>>>>>> integrating STORM-252 into 0.9.2 on such short notice would be >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> jumping >>>>>>>>>> the gun or a safe move. >>>>>>>>>> >>>>>>>>>> Btw, in terms of Storm/Kafka integration Kafka is in the same boat: >>>>>>>>>> it's built against ZK 3.3.x, and LinkedIn recommends the use of ZK >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> 3.3.4 >>>>>>>>>> in the docs. There's an open ticket KAFKA-854 [1] that's basically >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> the >>>>>>>>>> equivalent of STORM-252, but I'm not sure how actively the Kafka >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> team >>>>>>> is >>>>>>>>>> working on that. >>>>>>>>>> >>>>>>>>>> Best, >>>>>>>>>> Michael >>>>>>>>>> >>>>>>>>>> [1] https://issues.apache.org/jira/browse/KAFKA-854 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On 03/20/2014 02:33 AM, P. Taylor Goetz wrote: >>>>>>>>>>> I'd like to get this discussion started, largely because the >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> "negative timeout" bug (STORM-187) really bothers me. I've not seen it >>>>>>> >>>>>> >>>>> >>>> >>>> in >>>>>>> the wild, but I've heard of a few cases where it was enough to hinder >>>>>>> upgrading. >>>>>>>>>>> >>>>>>>>>>> HEAD looks good to me at the moment, with the major difference >>>> being >>>>>>> the zookeeper update and the patch mentioned above. >>>>>>>>>>> >>>>>>>>>>> Any thoughts on other PRs or patches to include? >>>>>>>>>>> >>>>>>>>>>> -Taylor >>>>>> >>>>>> >>>>>> -- >>>>>> Patrick Lucas >>>>>> >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Twitter: @nathanmarz >>>>> http://nathanmarz.com >>>>> >>>> >>>> >>> >>> >>> >>> >>> -- >>> http://hortonworks.com/download/ >>> >>> -- >>> CONFIDENTIALITY NOTICE >>> NOTICE: This message is intended for the use of the individual or entity to >>> >>> which it is addressed and may contain information that is confidential, >>> privileged and exempt from disclosure under applicable law. If the reader >>> of this message is not the intended recipient, you are hereby notified that >>> >>> any printing, copying, dissemination, distribution, disclosure or >>> forwarding of this communication is strictly prohibited. If you have >>> received this communication in error, please contact the sender immediately >>> >>> and delete it from your system. Thank You. >>> >>> >>> >> >> >
signature.asc
Description: Message signed with OpenPGP using GPGMail
