[GitHub] storm pull request #2691: STORM-3061: Update version of hbase

[revans2]

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/2691#discussion_r190432849
  
--- Diff: pom.xml ---
@@ -294,7 +294,7 @@
 0.14.0
 2.6.1
 ${hadoop.version}
-1.1.12
+1.4.4
--- End diff --

In 2.0 TokenUtil is a part of hbase-server.  I will spend some time to see 
if there is something I can do to work around this but I really don't want to 
require the server code be shipped with all of the clients.  Perhaps I can 
refactor autocreds so the nimbus portions are split off into a separate package.


---

[GitHub] storm issue #2692: STORM-3083 Upgrade HikariCP version to 2.4.7

[arunmahadevan]

Github user arunmahadevan commented on the issue:

https://github.com/apache/storm/pull/2692
  
+1


---

[GitHub] storm pull request #2691: STORM-3061: Update version of hbase

[revans2]

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/2691#discussion_r190428653
  
--- Diff: pom.xml ---
@@ -294,7 +294,7 @@
 0.14.0
 2.6.1
 ${hadoop.version}
-1.1.12
+1.4.4
--- End diff --

I am happy to give it a try.  I was a bit cautious with it being a major 
version change, and that there was no 2.0.1 yet.  But it does fit better with 
going to hadoop 3.1 coming in another pull request.  Probably after this one is 
merged in.


---

[GitHub] storm pull request #2691: STORM-3061: Update version of hbase

[arunmahadevan]

Github user arunmahadevan commented on a diff in the pull request:

https://github.com/apache/storm/pull/2691#discussion_r190424484
  
--- Diff: pom.xml ---
@@ -294,7 +294,7 @@
 0.14.0
 2.6.1
 ${hadoop.version}
-1.1.12
+1.4.4
--- End diff --

Since we are upgrading anyway, do we want to attempt Hbase 2.0 ?


---

[GitHub] storm pull request #2692: STORM-3083 Upgrade HikariCP version to 2.4.7

[HeartSaVioR]

GitHub user HeartSaVioR opened a pull request:

https://github.com/apache/storm/pull/2692

STORM-3083 Upgrade HikariCP version to 2.4.7

* Please refer https://issues.apache.org/jira/browse/STORM-3083 for more 
details

This can be safely ported back to 1.x-branch as well.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/HeartSaVioR/storm STORM-3083

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/2692.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2692


commit cbe9ed5a500638401513b475685f74221e8ebe20
Author: Jungtaek Lim 
Date:   2018-05-23T22:40:20Z

STORM-3083 Upgrade HikariCP version to 2.4.7

* Please refer https://issues.apache.org/jira/browse/STORM-3083 for more 
details




---

[GitHub] storm pull request #2691: STORM-3061: Update version of hbase

[revans2]

GitHub user revans2 opened a pull request:

https://github.com/apache/storm/pull/2691

STORM-3061: Update version of hbase

This updates the version of hbase used and cleans up some of the 
dependencies.

The biggest change besides updating the version is that we remove 
storm-server as a dependency, because it was only used for access to a copy of 
StringUtils.

This ends up impacting a lot of packages that were pulling in storm-hbase, 
either directly or indirectly.

storm-autocreds
strom-hdfs (because it depends on storm-autocreds)
flux-core (not really sure why the core of flux needs hbase but it is a 
dependency)
flux-examples 
storm-sql-hdfs (because of storm-autocreds)
storm-hdfs-blobstore (auotcreds again)
storm-hive (autocreds yet again)
storm-starter
storm-hdfs-examples (autocreds)
storm-hbase-examples
storm-hive-examples (autocreds)
storm-perf (autocreds)

I have not run any of the manual tests for this yet.  But I plan on doing 
some soon.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/revans2/incubator-storm 
STORM-3061-hbase-cleanup

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/2691.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2691


commit 63506a0a3b66f7dc2100480282d99fda8537d2f4
Author: Robert (Bobby) Evans 
Date:   2018-05-23T21:53:16Z

STORM-3061: Update version of hbase




---

[GitHub] storm pull request #2688: STORM-3061: Remove unused core dependencies

[HeartSaVioR]

Github user HeartSaVioR commented on a diff in the pull request:

https://github.com/apache/storm/pull/2688#discussion_r190405202
  
--- Diff: bin/storm.py ---
@@ -705,7 +705,6 @@ def 
nimbus(klass="org.apache.storm.daemon.nimbus.Nimbus"):
 cppaths = [CLUSTER_CONF_DIR]
 jvmopts = parse_args(confvalue("nimbus.childopts", cppaths)) + [
 "-Dlogfile.name=nimbus.log",
-
"-DLog4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector",
--- End diff --

Thanks for the kind explanation. Makes totally sense.


---

[GitHub] storm pull request #2688: STORM-3061: Remove unused core dependencies

[revans2]

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/2688#discussion_r190401760
  
--- Diff: bin/storm.py ---
@@ -705,7 +705,6 @@ def 
nimbus(klass="org.apache.storm.daemon.nimbus.Nimbus"):
 cppaths = [CLUSTER_CONF_DIR]
 jvmopts = parse_args(confvalue("nimbus.childopts", cppaths)) + [
 "-Dlogfile.name=nimbus.log",
-
"-DLog4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector",
--- End diff --

So a long time ago we tried out async logging for log4j.  It kind of 
helped, but also caused some issues.  It puts the log messages into a disruptor 
queue and has a background thread handles writing them out.  In my big patch 
after upgrading the version of log4j and removing disruptor from the classpath 
async logging started to get errors about disruptor not being on the classpath. 
 I don't know which caused it, or if it was a combination of things, but I 
thought if I was removing disruptor from the classpath this would be the right 
place to remove the async logging too.


---

[GitHub] storm pull request #2688: STORM-3061: Remove unused core dependencies

[HeartSaVioR]

Github user HeartSaVioR commented on a diff in the pull request:

https://github.com/apache/storm/pull/2688#discussion_r190396554
  
--- Diff: bin/storm.py ---
@@ -705,7 +705,6 @@ def 
nimbus(klass="org.apache.storm.daemon.nimbus.Nimbus"):
 cppaths = [CLUSTER_CONF_DIR]
 jvmopts = parse_args(confvalue("nimbus.childopts", cppaths)) + [
 "-Dlogfile.name=nimbus.log",
-
"-DLog4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector",
--- End diff --

Just curious, we don't touch log4j2 dependency, so can I treat it as 
removing unnecessary option?


---

Re: [VOTE] Release Apache Storm 1.2.2 (rc3)

[Jungtaek Lim]

REMINDER: The vote has been opened from 6 days ago, and still has only two
binding +1s. Please verify the release candidate and vote.

2018년 5월 21일 (월) 오후 11:16, Julien Nioche 님이
작성:

> +1 non binding
>
> compiled + ran StormCrawler topology without problems
>
> Thanks
>
> Julien
>
>
>
> On 17 May 2018 at 20:59, P. Taylor Goetz  wrote:
>
> > This is a call to vote on releasing Apache Storm 1.2.2 (rc3)
> >
> > Full list of changes in this release:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.
> > 2.2-rc3/RELEASE_NOTES.html
> >
> > The tag/commit to be voted upon is v1.2.2:
> >
> > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=
> >
> d29eaa70280dd0ac4b7d393387fab4a91eee9e9d;hb=d2d6f40344e6cc92ab07f3a462d577
> > ef6b61f8b1
> >
> > The source archive being voted upon can be found here:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.
> > 2.2-rc3/apache-storm-1.2.2-src.tar.gz
> >
> > Other release files, signatures and digests can be found here:
> >
> > https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.2.2-rc3/
> >
> > The release artifacts are signed with the following key:
> >
> > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_
> > plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> >
> > The Nexus staging repository for this release is:
> >
> > https://repository.apache.org/content/repositories/orgapachestorm-1068
> >
> > Please vote on releasing this package as Apache Storm 1.2.2.
> >
> > When voting, please list the actions taken to verify the release.
> >
> > This vote will be open for 72 hours or until at least 3 PMC members vote
> > +1.
> >
> > [ ] +1 Release this package as Apache Storm 1.2.2
> > [ ]  0 No opinion
> > [ ] -1 Do not release this package because...
> >
> > Thanks to everyone who contributed to this release.
> >
> > -Taylor
> >
>
>
>
> --
>
> *Open Source Solutions for Text Engineering*
>
> http://www.digitalpebble.com
> http://digitalpebble.blogspot.com/
> #digitalpebble 
>

Re: [VOTE] Release Apache Storm 1.1.3 (rc2)

[Jungtaek Lim]

REMINDER: The vote has been opened from 5 days ago, and still has only one
binding +1s. Please verify the release candidate and vote.

2018년 5월 19일 (토) 오전 10:49, Jungtaek Lim 님이 작성:

> This release candidate contains md5/sha for src archives, unlike 1.2.2
> RCs. May worth to check that.
>
> +1 (binding)
>
> On top to my verification on 1.2.3 rc1 (was +1),
>
> - verification on source/binary dist succeeded
> - build succeeded on source
> - spinning single cluster from binary dist succeeded
> - ran RollingTopWords
>
> The difference between rc1 and rc2 is regarding storm-kafka-client patch,
> so skipped checking storm-core features.
>
> Thanks,
> Jungtaek Lim (HeartSaVioR)
>
> 2018년 5월 19일 (토) 오전 4:39, P. Taylor Goetz 님이 작성:
>
>> This is a call to vote on releasing Apache Storm 1.1.3 (rc2)
>>
>> Full list of changes in this release:
>>
>>
>> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.1.3-rc2/RELEASE_NOTES.html
>>
>> The tag/commit to be voted upon is v1.1.3:
>>
>>
>> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=e983fef3d4dcd35a821cecb462e198505458d1a4;hb=c51c8fc251856291597a5007a07e433f26951464
>>
>> The source archive being voted upon can be found here:
>>
>>
>> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.1.3-rc2/apache-storm-1.1.3-src.tar.gz
>>
>> Other release files, signatures and digests can be found here:
>>
>> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.1.3-rc2/
>>
>> The release artifacts are signed with the following key:
>>
>>
>> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
>>
>> The Nexus staging repository for this release is:
>>
>> https://repository.apache.org/content/repositories/orgapachestorm-1069
>>
>> Please vote on releasing this package as Apache Storm 1.1.3.
>>
>> When voting, please list the actions taken to verify the release.
>>
>> This vote will be open for 72 hours or until at least 3 PMC members vote
>> +1.
>>
>> [ ] +1 Release this package as Apache Storm 1.1.3
>> [ ]  0 No opinion
>> [ ] -1 Do not release this package because...
>>
>> Thanks to everyone who contributed to this release.
>>
>> -Taylor
>>
>

[GitHub] storm pull request #2690: STORM-3061: Clean up some storm-druid dependencies

[revans2]

GitHub user revans2 opened a pull request:

https://github.com/apache/storm/pull/2690

STORM-3061: Clean up some storm-druid dependencies

`mvn dependency:tree` showed that the only real dependency changes were for 
scala and jackson that went from 2.4.6 to 2.9.4.

I am not totally sure how to test this currently as there is an open issue 
right now that it does not work. 

If others want to try this feel welcome to, if not we can try to fix any 
remaining issues as a part of https://issues.apache.org/jira/browse/STORM-2884

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/revans2/incubator-storm STORM-3061-druid

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/2690.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2690


commit 71f78eccda533be2c84e1df26100e5ada9f13c12
Author: Robert (Bobby) Evans 
Date:   2018-05-23T20:44:13Z

STORM-3061: Clean up some storm-druid dependencies




---

[GitHub] storm pull request #2687: STORM-3061: thrift 0.11

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/2687


---

[GitHub] storm issue #2687: STORM-3061: thrift 0.11

[revans2]

Github user revans2 commented on the issue:

https://github.com/apache/storm/pull/2687
  
Yes I reran the integration tests manually and they all passed.


---

[GitHub] storm issue #2687: STORM-3061: thrift 0.11

[revans2]

Github user revans2 commented on the issue:

https://github.com/apache/storm/pull/2687
  
The integration test failures look unrelated to this change.


---

[GitHub] storm issue #2687: STORM-3061: thrift 0.11

[arunmahadevan]

Github user arunmahadevan commented on the issue:

https://github.com/apache/storm/pull/2687
  
+1 LGTM.


---

[GitHub] storm pull request #2689: STORM-3061: rocket, jms, and mqtt updates

[revans2]

GitHub user revans2 opened a pull request:

https://github.com/apache/storm/pull/2689

 STORM-3061: rocket, jms, and mqtt updates

This updates some dependencies for the jms examples, mqtt examples, 
rocketmq examples, and updates activemq implementation used for testing jms, 
rocketmq, and activemq.

I ran some manual tests of the example topologies that I could find. I 
couldn't make all of them really work, because there was no clear documentation 
about most of them on how to setup or run the example topologies.

mqtt examples using flink had no documentation at all so I wasn't able to 
successfully run anything. storm-jms-examples fails with what appears to be a 
JNI issue, but it fails exactly the same way on 1.2.3-SNAPSHOT too.

If someone with more experience with these could improve the documentation 
about how to run them that would be great.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/revans2/incubator-storm STORM-3061-rmq

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/2689.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2689


commit 95ea31da49825632c4fb6068cb50d8d315c59775
Author: Robert (Bobby) Evans 
Date:   2018-05-23T14:41:48Z

STORM-3061: Remove unneeded deps from rocketmq-examples

commit 433689c95526c3be92b3bec2fd3dbcfe6198fb64
Author: Robert (Bobby) Evans 
Date:   2018-05-23T17:53:44Z

STORM-3061: mqtt-update




---

[GitHub] storm issue #2687: STORM-3061: thrift 0.11

[srdo]

Github user srdo commented on the issue:

https://github.com/apache/storm/pull/2687
  
+1


---

[GitHub] storm pull request #2687: STORM-3061: thrift 0.11

[srdo]

Github user srdo commented on a diff in the pull request:

https://github.com/apache/storm/pull/2687#discussion_r190294254
  
--- Diff: storm-client/src/genthrift.sh ---
@@ -17,7 +17,7 @@
 
 rm -rf gen-javabean gen-py py
 rm -rf jvm/org/apache/storm/generated
-thrift --gen java:beans,hashcode,nocamel,generated_annotations=undated 
--gen py:utf8strings storm.thrift
--- End diff --

No, it's fine


---

[GitHub] storm pull request #2687: STORM-3061: thrift 0.11

[revans2]

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/2687#discussion_r190287991
  
--- Diff: storm-client/src/genthrift.sh ---
@@ -17,7 +17,7 @@
 
 rm -rf gen-javabean gen-py py
 rm -rf jvm/org/apache/storm/generated
-thrift --gen java:beans,hashcode,nocamel,generated_annotations=undated 
--gen py:utf8strings storm.thrift
--- End diff --

@srdo do you want me to remove the utf8strings option?


---

[GitHub] storm pull request #2688: STORM-3061: Remove unused core dependencies

[revans2]

GitHub user revans2 opened a pull request:

https://github.com/apache/storm/pull/2688

STORM-3061: Remove unused core dependencies

This is for disruptor and java.jmx

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/revans2/incubator-storm 
STORM-3061-remove-core-deps

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/2688.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2688


commit 100a4daf060a851a247c954cbe2c03d39ea4d8d8
Author: Robert (Bobby) Evans 
Date:   2018-05-23T15:12:06Z

STORM-3061: Remove unused core dependencies




---

[GitHub] storm pull request #2687: STORM-3061: thrift 0.11

[srdo]

Github user srdo commented on a diff in the pull request:

https://github.com/apache/storm/pull/2687#discussion_r190183265
  
--- Diff: storm-client/src/genthrift.sh ---
@@ -17,7 +17,7 @@
 
 rm -rf gen-javabean gen-py py
 rm -rf jvm/org/apache/storm/generated
-thrift --gen java:beans,hashcode,nocamel,generated_annotations=undated 
--gen py:utf8strings storm.thrift
--- End diff --

I think the utf8strings option may be unnecessary 
https://issues.apache.org/jira/browse/THRIFT-3503


---

[GitHub] storm pull request #2687: STORM-3061: thrift 0.11

[srdo]

Github user srdo commented on a diff in the pull request:

https://github.com/apache/storm/pull/2687#discussion_r190175640
  
--- Diff: storm-client/src/genthrift.sh ---
@@ -17,7 +17,7 @@
 
 rm -rf gen-javabean gen-py py
 rm -rf jvm/org/apache/storm/generated
-thrift --gen java:beans,hashcode,nocamel,generated_annotations=undated 
--gen py:utf8strings storm.thrift
--- End diff --

There used to be a hashcode option for Java, but it was removed 
https://github.com/apache/thrift/pull/65. As of Thrift 0.10, it now complains 
if you try to use thrift with invalid arguments 
https://issues.apache.org/jira/browse/THRIFT-3605


---