Apache Zeppelin provides 700 MB of full set and 272 MB of netinst via Apache download page so there should be ways to do it, but for me 200MB is already too huge.
I have been wondering why we package external connectors, which other projects don't. When we didn't make uber jar for connectors from previous versions, only adding jar shouldn't work which makes packaged jars useless. Since we start to make uber jar for some connectors, and 1.1.0 will introduce "adding jars at the topology submission step" so it's not useless, but still inconvenient way to use. Moreover, in this case we can still use new feature "adding maven artifacts at the topology submission step". It's only useful for who has the node for submission which blocks Maven central or even internet. If we want to have a discussion about this, I'll initiate separate discussion thread. - Jungtaek Lim (HeartSaVioR) 2017년 2월 17일 (금) 오전 6:00, P. Taylor Goetz <[email protected]>님이 작성: > I have a release candidate for 1.1.0 built, but I can’t stage the > convenience binaries because they are now over the ASF subversion size > limit of 200MB. > > I’ll work with infra to figure out a solution. > > -Taylor > > > On Feb 16, 2017, at 9:46 AM, Jungtaek Lim <[email protected]> wrote: > > > > Now all issues on the Storm 1.1.0 epic got resolved. :) > > > > There're still some bugfix pull requests for storm-kafka-client, but > > releasing Storm 1.1.0 has been tremendously dragged (I initiated this at > > Sep. 2016) so unless they're critical, I'm +1 on starting release phase > on > > Storm 1.1.0 ASAP. > > > > - Jungtaek Lim > > > > 2017년 2월 14일 (화) 오전 11:41, Harsha Chintalapani <[email protected]>님이 작성: > > > > STORM-2340 is more of a feature . Auto-commit mode in storm-kafka used > > rarely and most users > > run the kafka spout with ackers and get at-least once guarantee. If its > > going to longer to address the PR reviews > > I am +1 on moving this out of Storm 1.1.0. We already quite a few patches > > storm-kafka-client and 1.1.0 release brings in lot of improvements > > and bug-fixes. > > -Harsha > > > > On Wed, Feb 8, 2017 at 6:15 PM Jungtaek Lim <[email protected]> wrote: > > > >> There seems some pull requests for bugfix/improvement on > >> storm-kafka-client, and some authors in PRs are not availble for now. > >> (waiting 7 days) > >> > >> If we plan to get 1.1.1 out soon (say 1 month later or even closer) we > can > >> postpone, but if not, it might be better to coordinate these things ASAP > >> and include to 1.1.0. > >> > >> There seems to be other small PRs, but nothing seems critical so it > would > >> be OK to not wait for merging. > >> > >> - Jungtaek Lim > >> > >> 2017년 2월 9일 (목) 오전 6:48, P. Taylor Goetz <[email protected]>님이 작성: > >> > >> Right now we’re down to 1 open issue on the 1.1.0 release epic: > STORM-2250 > >> which is under active review/discussion. > >> > >> Assuming that is mergeable in the near future, are there any other open > >> issues that should be considered for this release? > >> > >> -Taylor > >> > >> > >>> On Feb 2, 2017, at 4:48 PM, P. Taylor Goetz <[email protected]> wrote: > >>> > >>> Thanks for putting this list together Jungtaek. I added a few to the > 1.1 > >> release epic that I think are important. Feel free to do the same. > >>> > >>> Looks like we have a few to go, but there are pull requests for them. > >> It’s mostly just a matter of reviews and review responses, so I think we > >> are close. > >>> > >>> -Taylor > >>> > >>>> On Feb 2, 2017, at 1:41 AM, Jungtaek Lim <[email protected]> wrote: > >>>> > >>>> Seems like there're not blockers for 1.1.0, but some pull requests are > >>>> worth to check. > >>>> There're pending pull requests for storm-kafka-client waited on > >> STORM-2225. > >>>> Given that STORM-2225 is now merged, we might need to take a look at. > >>>> > >>>> *- reviewing* > >>>> > >>>> [storm-core] > >>>> > >>>>> STORM-2324 : Fix deployment failure if resources directory is missing > >> in > >>>> topology jar > >>>> (master) https://github.com/apache/storm/pull/1908 > >>>> (1.x) https://github.com/apache/storm/pull/1898 > >>>> > >>>>> STORM-2321 Handle blobstore zk key deletion in KeySequenceNumber > >>>> (master) https://github.com/apache/storm/pull/1904 > >>>> (1.x) https://github.com/apache/storm/pull/1905 > >>>> > >>>> [storm-kafka] > >>>> > >>>>> STORM-2270 Kafka spout should consume from latest when ZK partition > >>>> commit offset bigger than the latest offset > >>>> (1.x) https://github.com/apache/storm/pull/1851 > >>>> > >>>> [storm-kafka-client] > >>>> > >>>>> STORM-2281: Running Multiple Kafka Spouts (Trident) Throws Illegal > >> State > >>>> Exception > >>>> (1.x) https://github.com/apache/storm/pull/1902 > >>>> > >>>>> STORM-2315 Storm kafka client does not commit offsets when ack is > >> disabled > >>>> (1.x) https://github.com/apache/storm/pull/1891 > >>>> > >>>>> fix: KafkaSpout is blocked in AutoCommitMode > >>>> (master) https://github.com/apache/storm/pull/1863 > >>>> > >>>>> STORM-2250: Kafka Spout Refactoring to Increase Modularity and > >> Testability > >>>> (master) https://github.com/apache/storm/pull/1832 > >>>> > >>>>> STORM-2014: Put logic around dropping messages into RetryService, > >> remove > >>>> maxRetry setting from new KafkaSpout > >>>> (master) https://github.com/apache/storm/pull/1605 > >>>> > >>>>> fix NullPointException with acked.get(rtp) > >>>> (master) https://github.com/apache/storm/pull/1807 > >>>> > >>>> [storm-sql] > >>>> > >>>>> STORM-1443 [Storm SQL] Support customizing parallelism in StormSQL > >>>> https://github.com/apache/storm/pull/1739 > >>>> > >>>> *- pending* > >>>> > >>>> [storm-kafka-client] > >>>> > >>>>> STORM-2296 Kafka spout no dup on leader changes > >>>> (1.0.x) https://github.com/apache/storm/pull/1873 > >>>> (1.x) https://github.com/apache/storm/pull/1888 > >>>> > >>>> [storm-sql] > >>>> > >>>>> STORM-2148 [Storm SQL] Trident mode: back to code generate and > compile > >>>> Trident topology > >>>> https://github.com/apache/storm/pull/1743 > >>>> > >>>> Thanks, > >>>> Jungtaek Lim (HeartSaVioR) > >>>> > >>>> 2017년 2월 2일 (목) 오전 8:14, Harsha Chintalapani <[email protected]>님이 작성: > >>>> > >>>>> Trying to check the status on this release of 1.1.0. Are we going to > > do > >>>>> this release anytime soon? > >>>>> > >>>>> > >>>>> On Fri, Jan 13, 2017 at 7:50 PM S G <[email protected]> > wrote: > >>>>> > >>>>>> Not sure if its a little late to include for the 1.1.0 and 1.0.3 > >> releases > >>>>>> now, but can we consider using zookeeper 3.4.9 for the future > > versions > >> as > >>>>>> 3.4.9 brings in a lot of stability improvements ( > >>>>>> http://zookeeper.apache.org/releases.html) and storm is still using > >>>>> 3.4.6 > >>>>>> ( > >>>>>> https://github.com/apache/storm/blob/master/pom.xml) > >>>>>> > >>>>>> On Fri, Jan 6, 2017 at 11:54 AM, P. Taylor Goetz <[email protected] > > > >>>>>> wrote: > >>>>>> > >>>>>>> Thanks for the update Jungtaek. > >>>>>>> > >>>>>>> I’m verifying the patches now. And they should be mergeable > shortly. > >>>>>>> > >>>>>>> I think we’ll likely be ready for 1.1.0 and 1.0.3 releases next > > week. > >>>>>>> > >>>>>>> -Taylor > >>>>>>> > >>>>>>>> On Jan 6, 2017, at 3:40 AM, Jungtaek Lim <[email protected]> > wrote: > >>>>>>>> > >>>>>>>> I just submitted a patch for STORM-2176 > >>>>>>>> <https://issues.apache.org/jira/browse/STORM-2176>. Since it's a > >>>>> small > >>>>>>> fix, > >>>>>>>> we just need to handle STORM-2228 > >>>>>>>> <https://issues.apache.org/jira/browse/STORM-2228> to release. > >>>>>>>> > >>>>>>>> - Jungtaek Lim (HeartSaVioR) > >>>>>>>> > >>>>>>>> 2017년 1월 5일 (목) 오후 1:43, Jungtaek Lim <[email protected]>님이 작성: > >>>>>>>> > >>>>>>>>> Recently we receive some requests regarding release Storm 1.0.3, > > so > >>>>>>> would > >>>>>>>>> like to bump this again. > >>>>>>>>> > >>>>>>>>> Given that blocker issues for Storm 1.1.0 are also blocker for > >>>>> 1.0.3, > >>>>>>> I'd > >>>>>>>>> like to ask a favor of taking care of 'open' / 'in progress' > > issues > >>>>> on > >>>>>>>>> 1.1.0 epic. > >>>>>>>>> https://issues.apache.org/jira/browse/STORM-1856 > >>>>>>>>> > >>>>>>>>> There're one 'open' issue and three 'in progress' issues. Two of > >>>>> three > >>>>>>> 'in > >>>>>>>>> progress' issues are tiny fix so easy to be handled, so actual > >>>>>> blockers > >>>>>>> are > >>>>>>>>> STORM-2176 <https://issues.apache.org/jira/browse/STORM-2176> > and > >>>>>>>>> STORM-2228 <https://issues.apache.org/jira/browse/STORM-2228>. > >>>>>>>>> > >>>>>>>>> Thanks, > >>>>>>>>> Jungtaek Lim (HeartSaVioR) > >>>>>>>>> > >>>>>>>>> 2016년 11월 17일 (목) 오후 11:41, Satish Duggana < > >>>>> [email protected] > >>>>>>> 님이 > >>>>>>>>> 작성: > >>>>>>>>> > >>>>>>>>> STORM-2205: Race condition in getting nimbus summaries while ZK > >>>>>>>>> connections are reconnected. > >>>>>>>>> > >>>>>>>>> This issue seems to occur in our environments and I would like > > this > >>>>> to > >>>>>>> be > >>>>>>>>> part of 1.1.0. > >>>>>>>>> > >>>>>>>>> Thanks, > >>>>>>>>> Satish. > >>>>>>>>> > >>>>>>>>> On Thu, Nov 17, 2016 at 9:36 AM, Jungtaek Lim <[email protected] > > > >>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> I have no idea on storm-kafka-client, but some bugfix issues for > >>>>>>>>>> storm-kafka-client are waiting for reviewing / merging. > >>>>>>>>>> > >>>>>>>>>> STORM-2014 <https://issues.apache.org/jira/browse/STORM-2014> > >>>>>>>>>> STORM-2087 <https://issues.apache.org/jira/browse/STORM-2087> > >>>>>>>>>> STORM-2104 <https://issues.apache.org/jira/browse/STORM-2104> > >>>>>>>>>> > >>>>>>>>>> If someone can review them in several days it would be great. > >>>>>>>>>> > >>>>>>>>>> I hope that we include currently opened pull requests for Storm > >> SQL > >>>>>> so > >>>>>>>>> that > >>>>>>>>>> we can release 'usable Storm SQL' more usable, but I'm also OK > to > >>>>>>>>> postpone > >>>>>>>>>> them to be included to next release if they drag the release. > >>>>>>>>>> > >>>>>>>>>> STORM-1446 <http://issues.apache.org/jira/browse/STORM-1446> > >>>>>>>>>> STORM-1443 <https://issues.apache.org/jira/browse/STORM-1443> > >>>>>>>>>> STORM-2148 <https://issues.apache.org/jira/browse/STORM-2148> > >>>>>>>>>> STORM-2170 <https://issues.apache.org/jira/browse/STORM-2170> > >>>>>>>>>> > >>>>>>>>>> I can see some pull requests which address Trident > > implementations > >>>>>> for > >>>>>>>>>> storm-kafka-client, storm-mongodb, storm-cassandra. > >>>>>>>>>> > >>>>>>>>>> storm-kafka-client: STORM-1694 > >>>>>>>>>> <https://issues.apache.org/jira/browse/STORM-1694> (patch for > 2.0 > >>>>> is > >>>>>>>>>> merged, patch for 1.x is ready for reviewing) > >>>>>>>>>> storm-cassandra: STORM-1369 > >>>>>>>>>> <https://issues.apache.org/jira/browse/STORM-1369> > >>>>>>>>>> storm-mongodb: STORM-1607 <https://issues.apache.org/ > >>>>>>>>>> jira/browse/STORM-1607> > >>>>>>>>>> > >>>>>>>>>> If we want to cut the release now, we could include only bugfix > >>>>>> issues > >>>>>>>>> and > >>>>>>>>>> postpone others. Otherwise we could discuss and include some or > >> all > >>>>>> of > >>>>>>>>> the > >>>>>>>>>> above. > >>>>>>>>>> > >>>>>>>>>> What do you think? When we want to start the release process for > >>>>>> 1.1.0? > >>>>>>>>>> > >>>>>>>>>> - Jungtaek Lim (HeartSaVioR) > >>>>>>>>>> > >>>>>>>>>> 2016년 11월 16일 (수) 오전 4:11, P. Taylor Goetz <[email protected] > >님이 > >>>>> 작성: > >>>>>>>>>> > >>>>>>>>>> Thanks Xin, I added it to the 1.1.0 epic. > >>>>>>>>>> > >>>>>>>>>> -Taylor > >>>>>>>>>> > >>>>>>>>>>> On Nov 15, 2016, at 9:01 AM, Xin Wang <[email protected]> > >>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>> STORM-2198 ( PR: https://github.com/apache/storm/pull/1773 ) > >>>>> fixes > >>>>>> a > >>>>>>>>> bug > >>>>>>>>>> of > >>>>>>>>>>> storm-hdfs. Do we have a consideration to include this? > >>>>>>>>>>> > >>>>>>>>>>> Thanks, > >>>>>>>>>>> Xin Wang (vesense) > >>>>>>>>>>> > >>>>>>>>>>> 2016-11-15 10:03 GMT+08:00 Jungtaek Lim <[email protected]>: > >>>>>>>>>>> > >>>>>>>>>>>> Some issues on Storm SQL are resolved but not documented yet. > >>>>> I'll > >>>>>>>>> file > >>>>>>>>>> an > >>>>>>>>>>>> issue and assign to 1.1.0 release epic. > >>>>>>>>>>>> And also I want to address dropping aggregation and join on > >> Storm > >>>>>> SQL > >>>>>>>>>>>> Trident mode before releasing. I'll assign it too. > >>>>>>>>>>>> > >>>>>>>>>>>> - Jungtaek Lim (HeartSaVioR) > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> 2016년 11월 15일 (화) 오전 5:55, P. Taylor Goetz <[email protected] > >>> 님이 > >>>>>> 작성: > >>>>>>>>>>>> > >>>>>>>>>>>>> I think we’re very close. I would like to confirm that the > >>>>>>> 1.x-branch > >>>>>>>>>> is > >>>>>>>>>>>>> not affected by STORM-2176. > >>>>>>>>>>>>> > >>>>>>>>>>>>> The worker lifecycle API was added in 1.0, but doesn’t work > in > >>>>> any > >>>>>>>>>>>>> released version due to STORM-2176. > >>>>>>>>>>>>> > >>>>>>>>>>>>> If there are any other open JIRAs that anyone is passionate > >>>>> about, > >>>>>>>>> now > >>>>>>>>>>>>> would be a good time to assign them to the 1.1.0 release epic > >>>>>>>>>>>> (STORM-1856). > >>>>>>>>>>>>> > >>>>>>>>>>>>> -Taylor > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>>> On Oct 27, 2016, at 12:19 PM, Jungtaek Lim < > [email protected] > >>> > >>>>>>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Finally Pacemaker H/A, Supervisor V2, and Storm SQL PRs > which > >>>>>> were > >>>>>>>>>>>> opened > >>>>>>>>>>>>>> at the last mail (4 weeks ago) are all merged to 1.x branch. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> There're some more PRs on Storm SQL opened, but given that > we > >>>>> can > >>>>>>>>>>>> release > >>>>>>>>>>>>>> new minor at any time when we feel it's enough change, I can > >>>>> wait > >>>>>>>>> for > >>>>>>>>>>>> it. > >>>>>>>>>>>>>> They didn't get reviewed yet indeed. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Is there something else we would want to include it to > 1.1.0? > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>> Jungtaek Lim (HeartSaVioR) > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> 2016년 10월 1일 (토) 오전 9:30, Jungtaek Lim <[email protected] > >님이 > >>>>> 작성: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Personally, merging and porting back to three branches are > >>>>>> painful > >>>>>>>>>>>>> enough, > >>>>>>>>>>>>>>> especially we don't have merging script and having verbose > >>>>>> process > >>>>>>>>> (I > >>>>>>>>>>>>> mean > >>>>>>>>>>>>>>> CHANGELOG). > >>>>>>>>>>>>>>> It would be better if merging process is automated (by > >> running > >>>>>>>>> script > >>>>>>>>>>>> or > >>>>>>>>>>>>>>> so), so I'd +1 to revisit Harsha's suggestion (adopting > > Kafka > >>>>>>> merge > >>>>>>>>>>>>> script) > >>>>>>>>>>>>>>> and modify script to fit to Storm. > >>>>>>>>>>>>>>> (It will not work if it's the case we need to handle PRs > for > >>>>>> each > >>>>>>>>>>>>> version > >>>>>>>>>>>>>>> line, since 'Close' in commit log doesn't close the PR if > > its > >>>>>>>>> target > >>>>>>>>>>>>> branch > >>>>>>>>>>>>>>> is not master.) > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Anyway, without automation I don't want to maintain more > >>>>> version > >>>>>>>>>>>> lines. > >>>>>>>>>>>>>>> I'm looking at the announces from other projects, and > others > >>>>> are > >>>>>>>>> only > >>>>>>>>>>>>>>> maintaining two version lines. > >>>>>>>>>>>>>>> Since we maintain 2.0.0 version line we can't reduce > version > >>>>>> lines > >>>>>>>>> to > >>>>>>>>>>>> 2, > >>>>>>>>>>>>>>> but hopefully at most 3. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Btw, let's check pending pull requests and enumerate which > >> can > >>>>>> be > >>>>>>>>>>>>> included > >>>>>>>>>>>>>>> in 1.0.0, and start/finish review and merge them soon. > >>>>>>>>>>>>>>> For me Supervisor V2 and Pacemaker H/A, and pending Storm > > SQL > >>>>>> PRs > >>>>>>>>> can > >>>>>>>>>>>> be > >>>>>>>>>>>>>>> included, since they are small or in reviewing and expected > >> to > >>>>>>> pass > >>>>>>>>>>>>> review > >>>>>>>>>>>>>>> phase soon. > >>>>>>>>>>>>>>> (And some small PRs. There're other valuable PRs in PR list > >>>>> but > >>>>>>> I'm > >>>>>>>>>>>> not > >>>>>>>>>>>>>>> sure we can review them soon. One example is unified API.) > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> One issue which is not clear is STORM-2006 > >>>>>>>>>>>>>>> <https://github.com/apache/storm/pull/1595>. This is a > >>>>>> candidate > >>>>>>>>> for > >>>>>>>>>>>>> me, > >>>>>>>>>>>>>>> but gets blocked while reviewing. If we plan to put great > >>>>> effort > >>>>>>> to > >>>>>>>>>>>>> revise > >>>>>>>>>>>>>>> Metric we can skip this. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Please enumerate other PRs as well if you want to include > in > >>>>>>> 1.1.0. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>> Jungtaek Lim > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> 2016년 9월 30일 (금) 오후 11:09, Bobby Evans > >>>>>>> <[email protected] > >>>>>>>>>> > >>>>>>>>>>>> 님이 > >>>>>>>>>>>>> 작성: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Sounds good to me. It would be nice to get some of the new > >>>>>>>>> features > >>>>>>>>>>>>> out. > >>>>>>>>>>>>>>> Do we expect to maintain both 1.0.x and 1.1.x lines with > bug > >>>>>>> fixes? > >>>>>>>>>>>>> And if > >>>>>>>>>>>>>>> so for how long do we want to do this for? - Bobby > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On Thursday, September 29, 2016 7:35 PM, Jungtaek Lim < > >>>>>>>>>>>>>>> [email protected]> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Hi devs, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> It's been 5 months after releasing Storm 1.0.0, and now 1.x > >>>>>> branch > >>>>>>>>>> has > >>>>>>>>>>>>> lots > >>>>>>>>>>>>>>> of CHANGELOG and also pending reviews. > >>>>>>>>>>>>>>> It's also been a long time after 1.1.0 RC1 is canceled. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> I think it may be good to put some efforts to review and > >> merge > >>>>>>>>>> pending > >>>>>>>>>>>>> pull > >>>>>>>>>>>>>>> requests (except things which takes time to review and > > test), > >>>>>> and > >>>>>>>>>>>>> release > >>>>>>>>>>>>>>> 1.1.0 soon. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> What do you think? > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> I'm also open to volunteer release manager for 1.1.0 after > > we > >>>>>>>>>> document > >>>>>>>>>>>>> the > >>>>>>>>>>>>>>> process of official release. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>> Jungtaek Lim (HeartSaVioR) > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>> > >> > >
