This release has been baking for a while and a number of important improvements and bug fixes have been merged into the master branch.
There are a few dependency updates that are still pending: - STORM-265 (clojure) - STORM-252 (curator) - STORM-291 (http-client) Would committers be able to review the patches above and +1/-1 as appropriate? Any other patches that we should include in this release? - Taylor On Mar 26, 2014, at 10:09 AM, P. Taylor Goetz <[email protected]> wrote: > There are a number of email threads and JIRA issues regarding upgrading > various storm dependencies, so I’d like to enumerate them and discuss them in > one thread. > > Here’s the list so far: > > 1. Kryo/Carbonite (STORM-263)[1] > 2. Clojure (STORM-265) [2] > 3. commons-io (STORM-258) [3] > 4. curator (STORM-252) [4] > 5. http-client [5] > > I am +1 for all of the above, with the exception of #2, which I am +0 only > because I personally haven’t had a chance to do any testing with newer > versions of clojure. I’d be interested to hear if anyone has done any testing > with newer versions of clojure. > > I think we should at least consider a bump to clojure 1.5 since it includes > Bobby Evan’s patch that fixes error output getting swallowed (this manifests > itself as the maven-clojure-plugin failing without any useful information > when there are certain AOT compilation issues — very annoying). > > - Taylor > > [1] https://issues.apache.org/jira/browse/STORM-263 > [2] https://issues.apache.org/jira/browse/STORM-265 > [3] https://issues.apache.org/jira/browse/STORM-258 > [4] https://issues.apache.org/jira/browse/STORM-252 > [5] > http://mail-archives.apache.org/mod_mbox/storm-user/201402.mbox/%[email protected]%3E > > On Mar 26, 2014, at 7:13 AM, Brian O'Neill <[email protected]> wrote: > >> Agreed. One of our guys got hung up on that just yesterday. >> It isn’t hard to track down the assembly, but it might be worth making it a >> bit easier. >> >> Also, I’d love to see STORM-263 included in the next release. It seems >> like a quick win. >> >> -brian >> >> --- >> Brian O'Neill >> Chief Technology Officer >> >> Health Market Science >> The Science of Better Results >> 2700 Horizon Drive • King of Prussia, PA • 19406 >> M: 215.588.6024 • @boneill42 • healthmarketscience.com >> >> This information transmitted in this email message is for the intended >> recipient only and may contain confidential and/or privileged material. If >> you received this email in error and are not the intended recipient, or the >> person responsible to deliver it to the intended recipient, please contact >> the sender at the email above and delete this email and any attachments and >> destroy any copies thereof. Any review, retransmission, dissemination, >> copying or other use of, or taking any action in reliance upon, this >> information by persons or entities other than the intended recipient is >> strictly prohibited. >> >> On Mar 26, 2014, at 2:19 AM, Kang Xiao <[email protected]> wrote: >> >>> hi guys >>> >>> How about adding a bin/build_release.sh (http://build_release.sh) script in >>> this 0.9.2-incubating release? >>> >>> Since some guys asked the question about building storm release package >>> more than once in the mail list. >>> >>> -- >>> Best Regards! >>> >>> 肖康(Kang Xiao,<[email protected] (mailto:[email protected])>) >>> Distributed Software Engineer >>> >>> 在 2014年3月25日 星期二，1:38，Suresh Srinivas 写道： >>>> Taylor, I am not very clear on "Lazy consensus +2". By this definition code >>>> can committed with no +1 from a committer, right? >>>> >>>> You may want to look at Hadoop bylaws - >>>> https://hadoop.apache.org/bylaws.html >>>> >>>> The code commits in Hadoop are consensus approval with minimum +1 from an >>>> active committer and no veto. This has worked well in my experience. It may >>>> be a good idea to also adapt minimum 3 +1s from active committers for >>>> merging feature branches. >>>> >>>> >>>> >>>> >>>> On Fri, Mar 21, 2014 at 6:22 PM, P. Taylor Goetz <[email protected] >>>> (mailto:[email protected])> wrote: >>>> >>>>> That brings up a good point that probably deserves a separate thread. >>>>> >>>>> We should establish by-laws soon. Specifically a commit/merge policy. >>>>> >>>>> For code changes, I've been operating under a "lazy consensus +2" model: 2 >>>>> committer +1 votes and no vetoes (-1). If a committer submits the patch, >>>>> that's an implicit +1. Unless it's a somewhat urgent fix, I've been >>>>> waiting >>>>> for 3 binding votes and no vetoes. >>>>> >>>>> That's kind of a middle ground between the traditional code modification >>>>> rule and lazy consensus [1]. >>>>> >>>>> When wearing my "release manager" hat, I've also interpreted "code change" >>>>> to mean "anything that alters the behavior of the software we produce." In >>>>> terms of the build/packaging I've been a little looser. For large changes >>>>> (e.g. The switch to maven), I've waited for 3 binding votes. For some >>>>> changes I've committed directly -- I don't think we need to have a 3-day >>>>> vote on updating the CHANGELOG, for example. >>>>> >>>>> Anyway, it's something to think about. Sorry for hijacking the thread. >>>>> >>>>> +1 (again ;) ) >>>>> >>>>> Taylor >>>>> >>>>> [1] https://www.apache.org/foundation/voting.html >>>>> >>>>>> On Mar 21, 2014, at 7:48 PM, Nathan Marz <[email protected] >>>>>> (mailto:[email protected])> wrote: >>>>>> >>>>>> Let's get https://issues.apache.org/jira/browse/STORM-262 in there. Just >>>>>> one more vote needed by a committer. >>>>>> >>>>>> >>>>>>> On Fri, Mar 21, 2014 at 3:21 PM, Patrick Lucas <[email protected] >>>>>>> (mailto:[email protected])> wrote: >>>>>>> >>>>>>> A fix STORM-120 would be greatly appreciated. It's making it impossible >>>>> to >>>>>>> increase tasks/executors > 1 when there is a downstream shuffle >>>>>> >>>>> >>>>> grouping. >>>>>>> >>>>>>> I'm not sure why there haven't been more reports of problems with it. >>>>> Two >>>>>>> possibilities I can think of are that we are using exclusively shell >>>>>>> components--perhaps there's a root-cause bug in those component >>>>>>> classes--and >>>>>>> that we are dealing with a high volume stream of large tuples. >>>>>>> >>>>>> >>>>> >>>>> (thousands / >>>>>>> sec, KB in size) >>>>>>> >>>>>>> >>>>>>> On Thu, Mar 20, 2014 at 2:14 PM, P. Taylor Goetz <[email protected] >>>>>>> (mailto:[email protected])> >>>>>>> wrote: >>>>>>> >>>>>>>> Never mind... just found it. >>>>>>>> >>>>>>>>> On Mar 20, 2014, at 5:09 PM, P. Taylor Goetz <[email protected] >>>>>>>>> (mailto:[email protected])> >>>>> wrote: >>>>>>>>> >>>>>>>>> Derek do you have an idea for a fix? >>>>>>>>> >>>>>>>>> On Mar 20, 2014, at 3:43 PM, Derek Dagit <[email protected] >>>>>>>>> (mailto:[email protected])> >>>>> wrote: >>>>>>>>> >>>>>>>>>>> As I said above, this fix is the most important in my opinion. >>>>>>>>>>> STORM-259 (Random#nextInt) is new to me -- can't say whether it's as >>>>>>>>>>> important as STORM-187 or not. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Yeah, we found it recently, and I created it this morning after >>>>>>> reading >>>>>>>> Taylor's mail. >>>>>>>>>> >>>>>>>>>> STORM-187 can be a problem with fewer than 30 retries (likelihood >>>>>>>> depends on configuration), but we will hit STORM-259 when retries >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> exceeds >>>>>>>> 30. >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Derek >>>>>>>>>> >>>>>>>>>>> On 3/20/14, 14:18, Michael G. Noll wrote: >>>>>>>>>>> On my side the most important change is, as you point out, >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>>> STORM-187. >>>>>>>>>>> The primary reason is like Adam Lewis is pointing out because it's a >>>>>>>>>>> stability problem. The secondary aspect is that this issue taints >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> the >>>>>>>>>>> new Netty backend, and at least IMHO the faster Storm could >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> confidently >>>>>>>>>>> bury ZeroMQ the better. :-) >>>>>>>>>>> >>>>>>>>>>> As I said above, this fix is the most important in my opinion. >>>>>>>>>>> STORM-259 (Random#nextInt) is new to me -- can't say whether it's as >>>>>>>>>>> important as STORM-187 or not. >>>>>>>>>>> >>>>>>>>>>> Switching to my non-essential wishlist I'd also +1 STORM-252 >>>>> (Upgrade >>>>>>>>>>> Curator and thus ZooKeeper to 3.4.5). We have been running ZK 3.4.5 >>>>>>>>>>> anyway for a couple of reasons, and it would be nice to have >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>>> official >>>>>>>>>>> Storm support for the latest ZK version (ok, the recently released >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>>> ZK >>>>>>>>>>> 3.4.6 is actually the latest but hey). Although I don't know how >>>>>>>>>>> confident we are that the code in STORM-252 actually works, i.e. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> whether >>>>>>>>>>> integrating STORM-252 into 0.9.2 on such short notice would be >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> jumping >>>>>>>>>>> the gun or a safe move. >>>>>>>>>>> >>>>>>>>>>> Btw, in terms of Storm/Kafka integration Kafka is in the same boat: >>>>>>>>>>> it's built against ZK 3.3.x, and LinkedIn recommends the use of ZK >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> 3.3.4 >>>>>>>>>>> in the docs. There's an open ticket KAFKA-854 [1] that's basically >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> the >>>>>>>>>>> equivalent of STORM-252, but I'm not sure how actively the Kafka >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> team >>>>>>>> is >>>>>>>>>>> working on that. >>>>>>>>>>> >>>>>>>>>>> Best, >>>>>>>>>>> Michael >>>>>>>>>>> >>>>>>>>>>> [1] https://issues.apache.org/jira/browse/KAFKA-854 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> On 03/20/2014 02:33 AM, P. Taylor Goetz wrote: >>>>>>>>>>>> I'd like to get this discussion started, largely because the >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> "negative timeout" bug (STORM-187) really bothers me. I've not seen it >>>>>>>> >>>>>>> >>>>>> >>>>> >>>>> in >>>>>>>> the wild, but I've heard of a few cases where it was enough to hinder >>>>>>>> upgrading. >>>>>>>>>>>> >>>>>>>>>>>> HEAD looks good to me at the moment, with the major difference >>>>> being >>>>>>>> the zookeeper update and the patch mentioned above. >>>>>>>>>>>> >>>>>>>>>>>> Any thoughts on other PRs or patches to include? >>>>>>>>>>>> >>>>>>>>>>>> -Taylor >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Patrick Lucas >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Twitter: @nathanmarz >>>>>> http://nathanmarz.com >>>>>> >>>>> >>>>> >>>> >>>> >>>> >>>> >>>> -- >>>> http://hortonworks.com/download/ >>>> >>>> -- >>>> CONFIDENTIALITY NOTICE >>>> NOTICE: This message is intended for the use of the individual or entity >>>> to >>>> which it is addressed and may contain information that is confidential, >>>> privileged and exempt from disclosure under applicable law. If the reader >>>> of this message is not the intended recipient, you are hereby notified >>>> that >>>> any printing, copying, dissemination, distribution, disclosure or >>>> forwarding of this communication is strictly prohibited. If you have >>>> received this communication in error, please contact the sender >>>> immediately >>>> and delete it from your system. Thank You. >>>> >>>> >>>> >>> >>> >> >
signature.asc
Description: Message signed with OpenPGP using GPGMail
