Hi Taylor, Is there a fork or branch with Clojure 1.5 support? Or is this just a change of Clojure version in master branch?
Thanks Milinda On Thu, Mar 27, 2014 at 1:05 AM, P. Taylor Goetz <[email protected]> wrote: > As promised, binaries can be found here: > > http://people.apache.org/~ptgoetz/storm/storm-0.9.2-incubating/dep-frankenbuild/ > > This is not a release. It is merely a convenience build provided to enable > further testing among the community. > > -Taylor > > On Mar 26, 2014, at 9:54 PM, P. Taylor Goetz <[email protected]> wrote: > >> I've submitted pull requests for most of these changes (I think I still need >> to submit one for http-client), where there wasn't already one. >> >> The only problem I saw was with upgrading to clojure 1.6.0. IIRC it had to >> do with ns conflicts with "some.*". Falling back to 1.5.1 solved it. I >> didn't investigate it. But it seems upgrading to 1.6.x will require some >> code change. >> >> Anyway I created a snapshot build with all these updates pulled in. All unit >> tests pass, and my basic fully distributed mode smoke tests pass, so I'm >> leaning toward a +1 for all. >> >> However, if anyone who is interested in seeing these upgrades merged has the >> time and resources to try it out, that would be a huge help to us. >> >> I'll post the binaries soon, when I have access to a real computer. >> >> -Taylor >> >>> On Mar 26, 2014, at 10:49 AM, Brian O'Neill <[email protected]> wrote: >>> >>> >>> I’m +1 on all of those (including the clojure version bump). >>> >>> We’ve been hit with the maven-clojure-plugin issue as well, with no real >>> way to diagnose. >>> >>> -brian >>> >>> --- >>> Brian O'Neill >>> Chief Technology Officer >>> >>> Health Market Science >>> The Science of Better Results >>> 2700 Horizon Drive · King of Prussia, PA · 19406 >>> M: 215.588.6024 · @boneill42 <http://www.twitter.com/boneill42> · >>> healthmarketscience.com >>> >>> This information transmitted in this email message is for the intended >>> recipient only and may contain confidential and/or privileged material. If >>> you received this email in error and are not the intended recipient, or >>> the person responsible to deliver it to the intended recipient, please >>> contact the sender at the email above and delete this email and any >>> attachments and destroy any copies thereof. Any review, retransmission, >>> dissemination, copying or other use of, or taking any action in reliance >>> upon, this information by persons or entities other than the intended >>> recipient is strictly prohibited. >>> >>> >>> >>> >>> >>> >>> >>>> On 3/26/14, 10:09 AM, "P. Taylor Goetz" <[email protected]> wrote: >>>> >>>> There are a number of email threads and JIRA issues regarding upgrading >>>> various storm dependencies, so I’d like to enumerate them and discuss >>>> them in one thread. >>>> >>>> Here’s the list so far: >>>> >>>> 1. Kryo/Carbonite (STORM-263)[1] >>>> 2. Clojure (STORM-265) [2] >>>> 3. commons-io (STORM-258) [3] >>>> 4. curator (STORM-252) [4] >>>> 5. http-client [5] >>>> >>>> I am +1 for all of the above, with the exception of #2, which I am +0 >>>> only because I personally haven’t had a chance to do any testing with >>>> newer versions of clojure. I’d be interested to hear if anyone has done >>>> any testing with newer versions of clojure. >>>> >>>> I think we should at least consider a bump to clojure 1.5 since it >>>> includes Bobby Evan’s patch that fixes error output getting swallowed >>>> (this manifests itself as the maven-clojure-plugin failing without any >>>> useful information when there are certain AOT compilation issues — very >>>> annoying). >>>> >>>> - Taylor >>>> >>>> [1] https://issues.apache.org/jira/browse/STORM-263 >>>> [2] https://issues.apache.org/jira/browse/STORM-265 >>>> [3] https://issues.apache.org/jira/browse/STORM-258 >>>> [4] https://issues.apache.org/jira/browse/STORM-252 >>>> [5] >>>> http://mail-archives.apache.org/mod_mbox/storm-user/201402.mbox/%3CA732B22 >>>> [email protected]%3E >>>> >>>>> On Mar 26, 2014, at 7:13 AM, Brian O'Neill <[email protected]> wrote: >>>>> >>>>> Agreed. One of our guys got hung up on that just yesterday. >>>>> It isn’t hard to track down the assembly, but it might be worth making >>>>> it a bit easier. >>>>> >>>>> Also, I’d love to see STORM-263 included in the next release. It >>>>> seems like a quick win. >>>>> >>>>> -brian >>>>> >>>>> --- >>>>> Brian O'Neill >>>>> Chief Technology Officer >>>>> >>>>> Health Market Science >>>>> The Science of Better Results >>>>> 2700 Horizon Drive · King of Prussia, PA · 19406 >>>>> M: 215.588.6024 · @boneill42 · healthmarketscience.com >>>>> >>>>> This information transmitted in this email message is for the intended >>>>> recipient only and may contain confidential and/or privileged material. >>>>> If you received this email in error and are not the intended recipient, >>>>> or the person responsible to deliver it to the intended recipient, >>>>> please contact the sender at the email above and delete this email and >>>>> any attachments and destroy any copies thereof. Any review, >>>>> retransmission, dissemination, copying or other use of, or taking any >>>>> action in reliance upon, this information by persons or entities other >>>>> than the intended recipient is strictly prohibited. >>>>> >>>>>> On Mar 26, 2014, at 2:19 AM, Kang Xiao <[email protected]> wrote: >>>>>> >>>>>> hi guys >>>>>> >>>>>> How about adding a bin/build_release.sh (http://build_release.sh) >>>>>> script in this 0.9.2-incubating release? >>>>>> >>>>>> Since some guys asked the question about building storm release >>>>>> package more than once in the mail list. >>>>>> >>>>>> -- >>>>>> Best Regards! >>>>>> >>>>>> 肖康(Kang Xiao,<[email protected] (mailto:[email protected])>) >>>>>> Distributed Software Engineer >>>>>> >>>>>>> 在 2014年3月25日 星期二，1:38，Suresh Srinivas 写道： >>>>>>> Taylor, I am not very clear on "Lazy consensus +2". By this >>>>>>> definition code >>>>>>> can committed with no +1 from a committer, right? >>>>>>> >>>>>>> You may want to look at Hadoop bylaws - >>>>>>> https://hadoop.apache.org/bylaws.html >>>>>>> >>>>>>> The code commits in Hadoop are consensus approval with minimum +1 >>>>>>> from an >>>>>>> active committer and no veto. This has worked well in my experience. >>>>>>> It may >>>>>>> be a good idea to also adapt minimum 3 +1s from active committers for >>>>>>> merging feature branches. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Fri, Mar 21, 2014 at 6:22 PM, P. Taylor Goetz <[email protected] >>>>>>> (mailto:[email protected])> wrote: >>>>>>> >>>>>>>> That brings up a good point that probably deserves a separate thread. >>>>>>>> >>>>>>>> We should establish by-laws soon. Specifically a commit/merge policy. >>>>>>>> >>>>>>>> For code changes, I've been operating under a "lazy consensus +2" >>>>>>>> model: 2 >>>>>>>> committer +1 votes and no vetoes (-1). If a committer submits the >>>>>>>> patch, >>>>>>>> that's an implicit +1. Unless it's a somewhat urgent fix, I've been >>>>>>>> waiting >>>>>>>> for 3 binding votes and no vetoes. >>>>>>>> >>>>>>>> That's kind of a middle ground between the traditional code >>>>>>>> modification >>>>>>>> rule and lazy consensus [1]. >>>>>>>> >>>>>>>> When wearing my "release manager" hat, I've also interpreted "code >>>>>>>> change" >>>>>>>> to mean "anything that alters the behavior of the software we >>>>>>>> produce." In >>>>>>>> terms of the build/packaging I've been a little looser. For large >>>>>>>> changes >>>>>>>> (e.g. The switch to maven), I've waited for 3 binding votes. For some >>>>>>>> changes I've committed directly -- I don't think we need to have a >>>>>>>> 3-day >>>>>>>> vote on updating the CHANGELOG, for example. >>>>>>>> >>>>>>>> Anyway, it's something to think about. Sorry for hijacking the >>>>>>>> thread. >>>>>>>> >>>>>>>> +1 (again ;) ) >>>>>>>> >>>>>>>> Taylor >>>>>>>> >>>>>>>> [1] https://www.apache.org/foundation/voting.html >>>>>>>> >>>>>>>>> On Mar 21, 2014, at 7:48 PM, Nathan Marz <[email protected] >>>>>>>>> (mailto:[email protected])> wrote: >>>>>>>>> >>>>>>>>> Let's get https://issues.apache.org/jira/browse/STORM-262 in there. >>>>>>>>> Just >>>>>>>>> one more vote needed by a committer. >>>>>>>>> >>>>>>>>> >>>>>>>>>> On Fri, Mar 21, 2014 at 3:21 PM, Patrick Lucas <[email protected] >>>>>>>>>> (mailto:[email protected])> wrote: >>>>>>>>>> >>>>>>>>>> A fix STORM-120 would be greatly appreciated. It's making it >>>>>>>>>> impossible >>>>>>>> to >>>>>>>>>> increase tasks/executors > 1 when there is a downstream shuffle >>>>>>>>> >>>>>>>> >>>>>>>> grouping. >>>>>>>>>> >>>>>>>>>> I'm not sure why there haven't been more reports of problems with >>>>>>>>>> it. >>>>>>>> Two >>>>>>>>>> possibilities I can think of are that we are using exclusively >>>>>>>>>> shell >>>>>>>>>> components--perhaps there's a root-cause bug in those component >>>>>>>>>> classes--and >>>>>>>>>> that we are dealing with a high volume stream of large tuples. >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> (thousands / >>>>>>>>>> sec, KB in size) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Thu, Mar 20, 2014 at 2:14 PM, P. Taylor Goetz >>>>>>>>>> <[email protected] (mailto:[email protected])> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Never mind... just found it. >>>>>>>>>>> >>>>>>>>>>>> On Mar 20, 2014, at 5:09 PM, P. Taylor Goetz <[email protected] >>>>>>>>>>>> (mailto:[email protected])> >>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Derek do you have an idea for a fix? >>>>>>>>>>>> >>>>>>>>>>>> On Mar 20, 2014, at 3:43 PM, Derek Dagit <[email protected] >>>>>>>>>>>> (mailto:[email protected])> >>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>>> As I said above, this fix is the most important in my opinion. >>>>>>>>>>>>>> STORM-259 (Random#nextInt) is new to me -- can't say whether >>>>>>>>>>>>>> it's as >>>>>>>>>>>>>> important as STORM-187 or not. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Yeah, we found it recently, and I created it this morning after >>>>>>>>>> reading >>>>>>>>>>> Taylor's mail. >>>>>>>>>>>>> >>>>>>>>>>>>> STORM-187 can be a problem with fewer than 30 retries >>>>>>>>>>>>> (likelihood >>>>>>>>>>> depends on configuration), but we will hit STORM-259 when retries >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> exceeds >>>>>>>>>>> 30. >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Derek >>>>>>>>>>>>> >>>>>>>>>>>>>> On 3/20/14, 14:18, Michael G. Noll wrote: >>>>>>>>>>>>>> On my side the most important change is, as you point out, >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> STORM-187. >>>>>>>>>>>>>> The primary reason is like Adam Lewis is pointing out because >>>>>>>>>>>>>> it's a >>>>>>>>>>>>>> stability problem. The secondary aspect is that this issue >>>>>>>>>>>>>> taints >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> the >>>>>>>>>>>>>> new Netty backend, and at least IMHO the faster Storm could >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> confidently >>>>>>>>>>>>>> bury ZeroMQ the better. :-) >>>>>>>>>>>>>> >>>>>>>>>>>>>> As I said above, this fix is the most important in my opinion. >>>>>>>>>>>>>> STORM-259 (Random#nextInt) is new to me -- can't say whether >>>>>>>>>>>>>> it's as >>>>>>>>>>>>>> important as STORM-187 or not. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Switching to my non-essential wishlist I'd also +1 STORM-252 >>>>>>>> (Upgrade >>>>>>>>>>>>>> Curator and thus ZooKeeper to 3.4.5). We have been running ZK >>>>>>>>>>>>>> 3.4.5 >>>>>>>>>>>>>> anyway for a couple of reasons, and it would be nice to have >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> official >>>>>>>>>>>>>> Storm support for the latest ZK version (ok, the recently >>>>>>>>>>>>>> released >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> ZK >>>>>>>>>>>>>> 3.4.6 is actually the latest but hey). Although I don't know >>>>>>>>>>>>>> how >>>>>>>>>>>>>> confident we are that the code in STORM-252 actually works, >>>>>>>>>>>>>> i.e. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> whether >>>>>>>>>>>>>> integrating STORM-252 into 0.9.2 on such short notice would be >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> jumping >>>>>>>>>>>>>> the gun or a safe move. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Btw, in terms of Storm/Kafka integration Kafka is in the same >>>>>>>>>>>>>> boat: >>>>>>>>>>>>>> it's built against ZK 3.3.x, and LinkedIn recommends the use >>>>>>>>>>>>>> of ZK >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> 3.3.4 >>>>>>>>>>>>>> in the docs. There's an open ticket KAFKA-854 [1] that's >>>>>>>>>>>>>> basically >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> the >>>>>>>>>>>>>> equivalent of STORM-252, but I'm not sure how actively the >>>>>>>>>>>>>> Kafka >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> team >>>>>>>>>>> is >>>>>>>>>>>>>> working on that. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Best, >>>>>>>>>>>>>> Michael >>>>>>>>>>>>>> >>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/KAFKA-854 >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 03/20/2014 02:33 AM, P. Taylor Goetz wrote: >>>>>>>>>>>>>>> I'd like to get this discussion started, largely because the >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> "negative timeout" bug (STORM-187) really bothers me. I've not >>>>>>>>>>> seen it >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> in >>>>>>>>>>> the wild, but I've heard of a few cases where it was enough to >>>>>>>>>>> hinder >>>>>>>>>>> upgrading. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> HEAD looks good to me at the moment, with the major difference >>>>>>>> being >>>>>>>>>>> the zookeeper update and the patch mentioned above. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Any thoughts on other PRs or patches to include? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -Taylor >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Patrick Lucas >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Twitter: @nathanmarz >>>>>>>>> http://nathanmarz.com >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> http://hortonworks.com/download/ >>>>>>> >>>>>>> -- >>>>>>> CONFIDENTIALITY NOTICE >>>>>>> NOTICE: This message is intended for the use of the individual or >>>>>>> entity to >>>>>>> which it is addressed and may contain information that is >>>>>>> confidential, >>>>>>> privileged and exempt from disclosure under applicable law. If the >>>>>>> reader >>>>>>> of this message is not the intended recipient, you are hereby >>>>>>> notified that >>>>>>> any printing, copying, dissemination, distribution, disclosure or >>>>>>> forwarding of this communication is strictly prohibited. If you have >>>>>>> received this communication in error, please contact the sender >>>>>>> immediately >>>>>>> and delete it from your system. Thank You. >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >>> > -- Milinda Pathirage PhD Student | Research Assistant School of Informatics and Computing | Data to Insight Center Indiana University twitter: milindalakmal skype: milinda.pathirage blog: http://milinda.pathirage.org
