[GitHub] storm issue #1783: STORM-2204 Adding caching capabilities in HBaseLookupBolt

[ambud]

Github user ambud commented on the issue:

https://github.com/apache/storm/pull/1783
  
Sure thing, I will add the Guava patch code originally added for this to 
1.x and 0.10.x branches


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1783: STORM-2204 Adding caching capabilities in HBaseLookupBolt

[HeartSaVioR]

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1783
  
@ambud 
Yeah if we add unit test for the feature, unit test will fail on JDK 7 
which will be always failing on Travis CI. Nice catch. Could you address this 
to use Guava for 1.x?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1783: STORM-2204 Adding caching capabilities in HBaseLookupBolt

[ambud]

Github user ambud commented on the issue:

https://github.com/apache/storm/pull/1783
  
Caffeine is JDK 8 only so won't work for 1.x, since JDK 7 compilation will 
be tested.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [ANNOUNCE] New Storm Commiter/PMC Member: Xin Wang

[Matthias J. Sax]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Congrats!

On 11/30/16 8:43 PM, Jungtaek Lim wrote:
> Please join me in welcoming Xin Wang as a new Apache Storm
> Committer and PMC member.
> 
> Xin Wang has showed strong commitment to the Apache Storm community
> via code contributions for a long time spanning both core and as
> well as integration components. He also has showed active
> participation on mailing lists.
> 
> Congratulations and welcome Xin!
> 
> - Jungtaek Lim (HeartSaVioR)
> 
-BEGIN PGP SIGNATURE-
Comment: GPGTools - https://gpgtools.org

iQIYBAEBCgAGBQJYP7P1AAoJELz8Z8hxAGOi3egP3RcjYhi9Bof+Y/T30Sy2+TCH
ccBRcYxZcm2zRGZoLgAFO4HolBEaH0NyEhcj26lLzwl59zLrBWlpfxhSf0ruPW/X
Dg5Lm6dtY9LkweUePSgxh7SxnSL0wxh+5ID0Yyv4YuB0p9IE4D9+YM0ziKj+moFd
nlUICUd+eXzbro8oaa28Gi8D28v97N0vBQaZq9ATyzFkbpm9VTl3rEGw91q/b04n
kQfELba51TWG4gQGlsALAIBSuHhhJ9wVbFCGnJVLF4t47jS7gDYOZi7TqXOpPDU3
FS4VhYQUBNPpHgWseeetCSvYe8dGht2+lLYzlj37UDjfS22nGRHRMV3Jh+nQKRW2
CuDFxlZzH3sxHtvR4d7i5axP2cbFm1sqsn/UcmJsTq17jJBM+vZvv4MmxObS76yn
ZHo2i7mvuvkGfmgoByzuDaJb/eo30yCJ0+TC4nYaZaI+cLr3hHRRyI2WejtXCI3r
8vrfHt3dXbbwLwFMRkckvdELqxnsxVI5oFbswPl+0H0SChI33rtL/L/lwvzqp3df
HHb75AJDHjFwHTvAakhuXACuQ/+y3ZezrKfhmjf4mWlYor5mNwMtMEMeH11LUDaV
2CRrPVZ13GPJ3dyhgA/nCYFvsm9K5EINOpMW08saZg/yryK6ExoQDpJa8y8F6E4E
CoiIwY2RoW1P/T4=
=nTHE
-END PGP SIGNATURE-

[ANNOUNCE] New Storm Commiter/PMC Member: Xin Wang

[Jungtaek Lim]

Please join me in welcoming Xin Wang as a new Apache Storm Committer and
PMC member.

Xin Wang has showed strong commitment to the Apache Storm community via
code contributions for a long time spanning both core and as well as
integration components. He also has showed active participation on mailing
lists.

Congratulations and welcome Xin!

- Jungtaek Lim (HeartSaVioR)

[GitHub] storm issue #1783: STORM-2204 Adding caching capabilities in HBaseLookupBolt

[HeartSaVioR]

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1783
  
@ambud I'm OK to use Guava for 1.x. IMHO it would be better to provide 
complete set of features for guaranteed environment (JDK version) instead of 
leaving 'warn' to documentation. 
And some other external modules also use Guava, too.
@revans2 Are you OK to use Guava only for 1.x? Or you would like to keep 
using Caffeine for also 1.x and document it?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1807: fix NullPointException with acked.get(rtp)

[cutd]

GitHub user cutd opened a pull request:

https://github.com/apache/storm/pull/1807

fix NullPointException with acked.get(rtp)

 acked.get(rtp) maybe null when antocommit is true.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/cutd/storm patch-1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1807.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1807


commit 786083b27dc9750075dbddc335bd64f3f81e7c0d
Author: Tandy <982963...@qq.com>
Date:   2016-12-01T03:43:07Z

fix NullPointException with acked.get(rtp)

acked.get(rtp) maybe null when antocommit is true.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1805: STORM-2224 Exposed a method to override in computi...

[satishd]

Github user satishd closed the pull request at:

https://github.com/apache/storm/pull/1805


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1805: STORM-2224 Exposed a method to override in computing the ...

[satishd]

Github user satishd commented on the issue:

https://github.com/apache/storm/pull/1805
  
This PR is merged into master and 1.x-branch.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1801: STORM-2220 Added passing cassandra config to individual C...

[satishd]

Github user satishd commented on the issue:

https://github.com/apache/storm/pull/1801
  
This PR is merged into master and 1.x-branch.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1801: STORM-2220 Added passing cassandra config to indiv...

[satishd]

Github user satishd closed the pull request at:

https://github.com/apache/storm/pull/1801


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1765: STORM-2190: reduce contention between submission and sche...

[ptgoetz]

Github user ptgoetz commented on the issue:

https://github.com/apache/storm/pull/1765
  
One question about the for loop in WordCountTopology. I'm +1 once that's 
answered/addressed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1764: STORM-2190: reduce contention between submission and sche...

[ptgoetz]

Github user ptgoetz commented on the issue:

https://github.com/apache/storm/pull/1764
  
One question about the for loop in WordCountTopology. I'm +1 once that's 
answered/addressed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1764: STORM-2190: reduce contention between submission a...

[ptgoetz]

Github user ptgoetz commented on a diff in the pull request:

https://github.com/apache/storm/pull/1764#discussion_r90317672
  
--- Diff: 
examples/storm-starter/src/jvm/org/apache/storm/starter/WordCountTopology.java 
---
@@ -90,7 +90,9 @@ public static void main(String[] args) throws Exception {
 if (args != null && args.length > 0) {
   conf.setNumWorkers(3);
 
-  StormSubmitter.submitTopologyWithProgressBar(args[0], conf, 
builder.createTopology());
+  for (String name: args) {
--- End diff --

See comment on #1765. Is this change an artifact of manual testing?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1765: STORM-2190: reduce contention between submission a...

[ptgoetz]

Github user ptgoetz commented on a diff in the pull request:

https://github.com/apache/storm/pull/1765#discussion_r90317167
  
--- Diff: 
examples/storm-starter/src/jvm/org/apache/storm/starter/WordCountTopology.java 
---
@@ -90,7 +90,9 @@ public static void main(String[] args) throws Exception {
 if (args != null && args.length > 0) {
   conf.setNumWorkers(3);
 
-  StormSubmitter.submitTopologyWithProgressBar(args[0], conf, 
builder.createTopology());
+  for (String name: args) {
--- End diff --

I'm not sure I understand the for loop here. Isn't this change just 
submitting the same topology multiple times with different names? Is this an 
artifact from manual testing?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1786: STORM-1281: LocalCluster, testing4j and testing.clj to ja...

[ptgoetz]

Github user ptgoetz commented on the issue:

https://github.com/apache/storm/pull/1786
  
+1. I haven't done any manual testing of this patch, but I reviewed the 
code and it looks good.

I wouldn't mind seeing more javadoc comments in some of the new 
classes/interfaces, but it can always be added later, so I don't see that as a 
merge blocker or something that needs to be addressed right away.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1786: STORM-1281: LocalCluster, testing4j and testing.cl...

[ptgoetz]

Github user ptgoetz commented on a diff in the pull request:

https://github.com/apache/storm/pull/1786#discussion_r90311211
  
--- Diff: storm-core/src/jvm/org/apache/storm/testing/CompletableSpout.java 
---
@@ -0,0 +1,39 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.testing;
+
+public interface CompletableSpout {
--- End diff --

Didn't mean to imply that the class should be moved now as part of this 
pull request, but rather that it's something we should consider doing later.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [DISCUSS] Feature Branch for Apache Beam Runner

[P. Taylor Goetz]

Erik is correct, that is the right branch. It was initially unmodified, but I 
just merged the initial pull request so others are open to create pull requests 
against that branch now.

-Taylor

> On Nov 23, 2016, at 1:07 PM, Erik Weathers  
> wrote:
> 
> Hugo,
> 
> This appears to be the branch:
> 
>   - https://github.com/apache/storm/tree/beam-runner
> 
> - Erik
> 
> On Wed, Nov 23, 2016 at 9:34 AM, Hugo Da Cruz Louro 
> wrote:
> 
>> I somehow missed this email … I would like to contribute to this effort as
>> well. Please keep me posted.
>> Thanks.
>> 
>>> On Oct 19, 2016, at 8:51 AM, Satish Duggana 
>> wrote:
>>> 
>>> +1, waiting for that. :)
>>> Currently,there are API changes going on in Beam. It seem they plan to
>> get
>>> that done by the end of 2016.
>>> 
>>> ~Satish.
>>> 
>>> On Wed, Oct 19, 2016 at 9:19 PM, Bobby Evans >> 
>>> wrote:
>>> 
 +1 - Bobby
 
   On Wednesday, October 19, 2016 10:30 AM, Arun Mahadevan <
 ar...@apache.org> wrote:
 
 
 +1
 
 On 10/19/16, 8:58 PM, "P. Taylor Goetz"  wrote:
 
> If there are no objections, I’d like to create the feature branch and
 push what I have so far. I’ve not had too much time lately to work on
>> it,
 but other’s have expressed interest in contributing so I’d like to make
>> it
 available.
> 
> -Taylor
> 
> 
>> On Sep 19, 2016, at 11:15 AM, Bobby Evans >> 
 wrote:
>> 
>> +1 on the idea.  I would love to contribute, but I doubt I will find
 time to do it any time soon. - Bobby
>> 
>>  On Friday, September 16, 2016 12:05 AM, Satish Duggana <
 satish.dugg...@gmail.com> wrote:
>> 
>> 
>> Taylor,
>> I am interested in contributing to this effort. Gone through Beam APIs
>> earlier and had some initial thoughts on Storm runner. We can start
>> with
>> existing core storm constructs but it is better to design in such a
>> way
>> that these can be replaced with new APIs.
>> 
>> Thanks,
>> Satish.
>> 
>> On Fri, Sep 16, 2016 at 3:35 AM, P. Taylor Goetz 
 wrote:
>> 
>>> I'm open to change, but yes, I started with core storm since it
>> offers
 the
>>> most flexibility wrt how Beam constructs are translated.
>>> 
>>> -Taylor
>>> 
 On Sep 15, 2016, at 5:51 PM, Roshan Naik 
 wrote:
 
 Good idea. Will the Beam API be implemented to run on top Storm Core
 primitives ?
 -roshan
 
 
> On 9/15/16, 2:00 PM, "P. Taylor Goetz"  wrote:
> 
> I¹ve been tinkering with implementing an Apache Beam runner on top
>> of
> Storm and would like to open it up so others in the community can
> contribute. To that end I¹d like to propose creating a feature
>> branch
>>> for
> that work if there are others who are interested in getting
 involved. We
> did that a while back when storm-sql was originally developed.
> 
> Basically, review requirements for that branch would be relaxed
 during
> development, with a final, strict review before merging back to one
 of
> our main branches.
> 
> I¹d like to document what I have and future improvements in a
 proposal
> document, and follow that with pushing the code to the feature
>> branch
>>> for
> group collaboration.
> 
> Any thoughts? Anyone interested in contributing to such an effort?
> 
> -Taylor
 
>>> 
>> 
> 
 
 
 
 
 
>> 
>> 

[GitHub] storm pull request #1798: Storm-2203 Add a getAll method to KeyValueState in...

[arunmahadevan]

Github user arunmahadevan commented on a diff in the pull request:

https://github.com/apache/storm/pull/1798#discussion_r90301167
  
--- Diff: storm-core/src/jvm/org/apache/storm/state/KeyValueState.java ---
@@ -45,4 +47,9 @@
  * @return the value or defaultValue if no mapping is found
  */
 V get(K key, V defaultValue);
+
+/**
+ * @return all key value mappings as an unmodifiable map.
+ */
+Map getAll();
--- End diff --

Return an Iterator of the entries in the map than the map itself. That way 
it will be more manageable even if the state is huge.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1800: STORM-2217: Make DRPC pure java

[ptgoetz]

Github user ptgoetz commented on the issue:

https://github.com/apache/storm/pull/1800
  
+1. Looks good.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1783: STORM-2204 Adding caching capabilities in HBaseLookupBolt

[ambud]

Github user ambud commented on the issue:

https://github.com/apache/storm/pull/1783
  
Added documentation to readme file.

Fixed the debug logging by using isDebugEnabled checks

Are we going to use Guava caching implementation for 1.x? @HeartSaVioR 
@revans2 



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1745: STORM-2174: Initial Base for Storm Beam Runner

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1745


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1801: STORM-2220 Added passing cassandra config to individual C...

[ptgoetz]

Github user ptgoetz commented on the issue:

https://github.com/apache/storm/pull/1801
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1802: STORM-2209: 1.x-branch Update documents adding new integr...

[ptgoetz]

Github user ptgoetz commented on the issue:

https://github.com/apache/storm/pull/1802
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1745: STORM-2174: Initial Base for Storm Beam Runner

[ptgoetz]

Github user ptgoetz commented on the issue:

https://github.com/apache/storm/pull/1745
  
@revans2 @arunmahadevan Thanks for the review. I will merge this to the 
beam runner feature branch so others can submit pull requests against it.

@arunmahadevan Yes, the GBK implementation is wrong. It's a WIP to get data 
moving between components.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1783: STORM-2204 Adding caching capabilities in HBaseLookupBolt

[ptgoetz]

Github user ptgoetz commented on the issue:

https://github.com/apache/storm/pull/1783
  
+1, but I agree with @HeartSaVioR that the new cache configuration options 
need to be documented before this is merged.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1470: STORM-1886 Extend KeyValueState iface with delete

[arunmahadevan]

Github user arunmahadevan commented on the issue:

https://github.com/apache/storm/pull/1470
  
@kosii thanks for the PR. Merged it to master. 

Can you also raise a PR for 1.x-branch ? I am getting some merge conflicts 
while trying to cherry-pick.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1783: STORM-2204 Adding caching capabilities in HBaseLookupBolt

[ben-manes]

Github user ben-manes commented on the issue:

https://github.com/apache/storm/pull/1783
  
Should fail, e.g. Class version error.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1783: STORM-2204 Adding caching capabilities in HBaseLookupBolt

[HeartSaVioR]

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1783
  
@ambud 
The code looks good except what @vesense commented. 

Two things more to address:

1. It would be better to document new configurations. Without 
documentation, end-users have no idea about added feature. 
`external/storm-hbase/README.md` and `docs/storm-hbase.md`.

2. The code already uses JDK 8 API (Map.getOrDefault()), so can't get it as 
it is for 1.x. Could you provide a new PR for 1.x branch?
It would be also great if you can test it (with Caffeine) on JRE7 (expected 
to not work but we can document the precondition for JRE version) and JRE8 
(expected to work).
cc. @ben-manes Is my expectation right?

Thanks in advance!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1783: STORM-2204 Adding caching capabilities in HBaseLoo...

[HeartSaVioR]

Github user HeartSaVioR commented on a diff in the pull request:

https://github.com/apache/storm/pull/1783#discussion_r90260820
  
--- Diff: 
external/storm-hbase/src/main/java/org/apache/storm/hbase/bolt/HBaseLookupBolt.java
 ---
@@ -40,51 +48,81 @@
  *
  */
 public class HBaseLookupBolt extends AbstractHBaseBolt {
-private static final Logger LOG = 
LoggerFactory.getLogger(HBaseLookupBolt.class);
+   
+   private static final long serialVersionUID = 1L;
+   private static final Logger LOG = 
LoggerFactory.getLogger(HBaseLookupBolt.class);
+
+   private HBaseValueMapper rowToTupleMapper;
+   private HBaseProjectionCriteria projectionCriteria;
+   private transient LoadingCache cache;
+   private transient boolean cacheEnabled;
 
-private HBaseValueMapper rowToTupleMapper;
+   public HBaseLookupBolt(String tableName, HBaseMapper mapper, 
HBaseValueMapper rowToTupleMapper) {
+  super(tableName, mapper);
+  Validate.notNull(rowToTupleMapper, "rowToTupleMapper can not be 
null");
+  this.rowToTupleMapper = rowToTupleMapper;
+   }
 
-private HBaseProjectionCriteria projectionCriteria;
+   public HBaseLookupBolt withConfigKey(String configKey) {
+  this.configKey = configKey;
+  return this;
+   }
 
-public HBaseLookupBolt(String tableName, HBaseMapper mapper, 
HBaseValueMapper rowToTupleMapper){
-super(tableName, mapper);
-Validate.notNull(rowToTupleMapper, "rowToTupleMapper can not be 
null");
-this.rowToTupleMapper = rowToTupleMapper;
-}
+   public HBaseLookupBolt withProjectionCriteria(HBaseProjectionCriteria 
projectionCriteria) {
+  this.projectionCriteria = projectionCriteria;
+  return this;
+   }
 
-public HBaseLookupBolt withConfigKey(String configKey){
-this.configKey = configKey;
-return this;
-}
+   @SuppressWarnings({ "unchecked", "rawtypes" })
+   @Override
+   public void prepare(Map config, TopologyContext topologyContext, 
OutputCollector collector) {
+  super.prepare(config, topologyContext, collector);
+  cacheEnabled = 
Boolean.parseBoolean(config.getOrDefault("hbase.cache.enable", 
"false").toString());
+  int cacheTTL = 
Integer.parseInt(config.getOrDefault("hbase.cache.ttl.seconds", 
"300").toString());
+  int maxCacheSize = 
Integer.parseInt(config.getOrDefault("hbase.cache.size", "1000").toString());
+  if (cacheEnabled) {
+ cache = Caffeine.newBuilder()
+   
.maximumSize(maxCacheSize).expireAfterWrite(cacheTTL, TimeUnit.SECONDS)
+   .build(new CacheLoader() {
 
-public HBaseLookupBolt withProjectionCriteria(HBaseProjectionCriteria 
projectionCriteria) {
-this.projectionCriteria = projectionCriteria;
-return this;
-}
+  @Override
+  public Result load(byte[] rowKey) throws Exception {
+ Get get = hBaseClient.constructGetRequests(rowKey, 
projectionCriteria);
+ LOG.debug("Cache miss for key:"+new String(rowKey));
--- End diff --

@ambud 
Could you address @vesense comment? 
It makes unnecessary creation of String object for non-debug log level.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1798: Storm 2203 Add a getAll method to KeyValueState interface

[HeartSaVioR]

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1798
  
@aandis 
Looks great. Could you add tests if possible? You may want to rename the 
prefix of the title to `STORM-2203` so that it can be linked to JIRA issue 
automatically.
Thanks in advance!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1799: Fixing a bug related to following exception

[HeartSaVioR]

Github user HeartSaVioR commented on a diff in the pull request:

https://github.com/apache/storm/pull/1799#discussion_r90252874
  
--- Diff: storm-core/src/jvm/org/apache/storm/stats/StatsUtil.java ---
@@ -1516,7 +1516,7 @@ public static ComponentPageInfo aggCompExecsStats(
 
 Integer start = ((Number) key.get(0)).intValue();
 String host = (String) value.get(0);
-Integer port = (Integer) value.get(1);
+Integer port = (Long) value.get(1);
--- End diff --

@ErangaHeshan 
Let's follow what @srdo comments. Could you address it?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1801: STORM-2220 Added passing cassandra config to individual C...

[HeartSaVioR]

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1801
  
Looks good. +1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1805: STORM-2224 Exposed a method to override in computing the ...

[HeartSaVioR]

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1805
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: moving from storm 0.9.3 to storm 1.0.1

[Xin Wang]

Hi Ohad,

You can try setting topology.backpressure.enable: false

-Xin Wang (vesense)

2016-11-30 17:58 GMT+08:00 Ohad Edelstein :

> We are in the process of upgrading our storm infra from 0.9.3 to 1.0.1
> We are running python in the storm.
> We moved for the upgrade from ubuntu14.04 to ubuntu16.04
>
> What we are seeing are very disappointing results,
> Out of the box, with about the same basic params set, we see slowness in
> the new version of about 2%.
> Where we hopped to see an increase of some sort.
> Any ideas with what we can tweak?
> Thanks
>
>

moving from storm 0.9.3 to storm 1.0.1

[Ohad Edelstein]

We are in the process of upgrading our storm infra from 0.9.3 to 1.0.1
We are running python in the storm.
We moved for the upgrade from ubuntu14.04 to ubuntu16.04

What we are seeing are very disappointing results,
Out of the box, with about the same basic params set, we see slowness in the 
new version of about 2%.
Where we hopped to see an increase of some sort.
Any ideas with what we can tweak?
Thanks

[GitHub] storm pull request #1744: STORM-1276: line for line translation of nimbus to...

[erikdw]

Github user erikdw commented on a diff in the pull request:

https://github.com/apache/storm/pull/1744#discussion_r90198668
  
--- Diff: storm-core/src/jvm/org/apache/storm/daemon/nimbus/Nimbus.java ---
@@ -0,0 +1,3729 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.daemon.nimbus;
+
+import static org.apache.storm.metric.StormMetricsRegistry.registerMeter;
+import static org.apache.storm.utils.Utils.OR;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.InterruptedIOException;
+import java.io.OutputStream;
+import java.net.BindException;
+import java.net.ServerSocket;
+import java.nio.ByteBuffer;
+import java.nio.channels.Channels;
+import java.nio.channels.WritableByteChannel;
+import java.security.Principal;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.function.UnaryOperator;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+import javax.security.auth.Subject;
+
+import org.apache.storm.Config;
+import org.apache.storm.StormTimer;
+import org.apache.storm.blobstore.AtomicOutputStream;
+import org.apache.storm.blobstore.BlobStore;
+import org.apache.storm.blobstore.BlobStoreAclHandler;
+import org.apache.storm.blobstore.BlobSynchronizer;
+import org.apache.storm.blobstore.InputStreamWithMeta;
+import org.apache.storm.blobstore.KeySequenceNumber;
+import org.apache.storm.blobstore.LocalFsBlobStore;
+import org.apache.storm.cluster.ClusterStateContext;
+import org.apache.storm.cluster.ClusterUtils;
+import org.apache.storm.cluster.DaemonType;
+import org.apache.storm.cluster.IStormClusterState;
+import org.apache.storm.daemon.DaemonCommon;
+import org.apache.storm.daemon.Shutdownable;
+import org.apache.storm.daemon.StormCommon;
+import org.apache.storm.generated.AlreadyAliveException;
+import org.apache.storm.generated.Assignment;
+import org.apache.storm.generated.AuthorizationException;
+import org.apache.storm.generated.BeginDownloadResult;
+import org.apache.storm.generated.ClusterSummary;
+import org.apache.storm.generated.CommonAggregateStats;
+import org.apache.storm.generated.ComponentAggregateStats;
+import org.apache.storm.generated.ComponentPageInfo;
+import org.apache.storm.generated.ComponentType;
+import org.apache.storm.generated.Credentials;
+import org.apache.storm.generated.DebugOptions;
+import org.apache.storm.generated.ErrorInfo;
+import org.apache.storm.generated.ExecutorInfo;
+import org.apache.storm.generated.ExecutorStats;
+import org.apache.storm.generated.ExecutorSummary;
+import org.apache.storm.generated.GetInfoOptions;
+import org.apache.storm.generated.InvalidTopologyException;
+import org.apache.storm.generated.KeyAlreadyExistsException;
+import org.apache.storm.generated.KeyNotFoundException;
+import org.apache.storm.generated.KillOptions;
+import org.apache.storm.generated.LSTopoHistory;
+import org.apache.storm.generated.ListBlobsResult;
+import org.apache.storm.generated.LogConfig;
+import org.apache.storm.generated.LogLevel;
+import org.apache.storm.generated.LogLevelAction;
+import org.apache.storm.generated.Nimbus.Iface;
+import org.apache.storm.generated.Nimbus.Processor;
+import org.apache.storm.generated.NimbusSummary;
+import org.apache.storm.generated.NodeInfo;
+import org.apache.storm.generated.NotAliveException;
+import org.apache.storm.generated.NumErrorsChoice;
+import 

[GitHub] storm pull request #1744: STORM-1276: line for line translation of nimbus to...

[erikdw]

Github user erikdw commented on a diff in the pull request:

https://github.com/apache/storm/pull/1744#discussion_r90193679
  
--- Diff: storm-core/src/jvm/org/apache/storm/daemon/nimbus/Nimbus.java ---
@@ -0,0 +1,3729 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.daemon.nimbus;
+
+import static org.apache.storm.metric.StormMetricsRegistry.registerMeter;
+import static org.apache.storm.utils.Utils.OR;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.InterruptedIOException;
+import java.io.OutputStream;
+import java.net.BindException;
+import java.net.ServerSocket;
+import java.nio.ByteBuffer;
+import java.nio.channels.Channels;
+import java.nio.channels.WritableByteChannel;
+import java.security.Principal;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.function.UnaryOperator;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+import javax.security.auth.Subject;
+
+import org.apache.storm.Config;
+import org.apache.storm.StormTimer;
+import org.apache.storm.blobstore.AtomicOutputStream;
+import org.apache.storm.blobstore.BlobStore;
+import org.apache.storm.blobstore.BlobStoreAclHandler;
+import org.apache.storm.blobstore.BlobSynchronizer;
+import org.apache.storm.blobstore.InputStreamWithMeta;
+import org.apache.storm.blobstore.KeySequenceNumber;
+import org.apache.storm.blobstore.LocalFsBlobStore;
+import org.apache.storm.cluster.ClusterStateContext;
+import org.apache.storm.cluster.ClusterUtils;
+import org.apache.storm.cluster.DaemonType;
+import org.apache.storm.cluster.IStormClusterState;
+import org.apache.storm.daemon.DaemonCommon;
+import org.apache.storm.daemon.Shutdownable;
+import org.apache.storm.daemon.StormCommon;
+import org.apache.storm.generated.AlreadyAliveException;
+import org.apache.storm.generated.Assignment;
+import org.apache.storm.generated.AuthorizationException;
+import org.apache.storm.generated.BeginDownloadResult;
+import org.apache.storm.generated.ClusterSummary;
+import org.apache.storm.generated.CommonAggregateStats;
+import org.apache.storm.generated.ComponentAggregateStats;
+import org.apache.storm.generated.ComponentPageInfo;
+import org.apache.storm.generated.ComponentType;
+import org.apache.storm.generated.Credentials;
+import org.apache.storm.generated.DebugOptions;
+import org.apache.storm.generated.ErrorInfo;
+import org.apache.storm.generated.ExecutorInfo;
+import org.apache.storm.generated.ExecutorStats;
+import org.apache.storm.generated.ExecutorSummary;
+import org.apache.storm.generated.GetInfoOptions;
+import org.apache.storm.generated.InvalidTopologyException;
+import org.apache.storm.generated.KeyAlreadyExistsException;
+import org.apache.storm.generated.KeyNotFoundException;
+import org.apache.storm.generated.KillOptions;
+import org.apache.storm.generated.LSTopoHistory;
+import org.apache.storm.generated.ListBlobsResult;
+import org.apache.storm.generated.LogConfig;
+import org.apache.storm.generated.LogLevel;
+import org.apache.storm.generated.LogLevelAction;
+import org.apache.storm.generated.Nimbus.Iface;
+import org.apache.storm.generated.Nimbus.Processor;
+import org.apache.storm.generated.NimbusSummary;
+import org.apache.storm.generated.NodeInfo;
+import org.apache.storm.generated.NotAliveException;
+import org.apache.storm.generated.NumErrorsChoice;
+import 

[GitHub] storm pull request #1744: STORM-1276: line for line translation of nimbus to...

[erikdw]

Github user erikdw commented on a diff in the pull request:

https://github.com/apache/storm/pull/1744#discussion_r90195674
  
--- Diff: storm-core/src/jvm/org/apache/storm/daemon/nimbus/Nimbus.java ---
@@ -0,0 +1,3729 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.daemon.nimbus;
+
+import static org.apache.storm.metric.StormMetricsRegistry.registerMeter;
+import static org.apache.storm.utils.Utils.OR;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.InterruptedIOException;
+import java.io.OutputStream;
+import java.net.BindException;
+import java.net.ServerSocket;
+import java.nio.ByteBuffer;
+import java.nio.channels.Channels;
+import java.nio.channels.WritableByteChannel;
+import java.security.Principal;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.function.UnaryOperator;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+import javax.security.auth.Subject;
+
+import org.apache.storm.Config;
+import org.apache.storm.StormTimer;
+import org.apache.storm.blobstore.AtomicOutputStream;
+import org.apache.storm.blobstore.BlobStore;
+import org.apache.storm.blobstore.BlobStoreAclHandler;
+import org.apache.storm.blobstore.BlobSynchronizer;
+import org.apache.storm.blobstore.InputStreamWithMeta;
+import org.apache.storm.blobstore.KeySequenceNumber;
+import org.apache.storm.blobstore.LocalFsBlobStore;
+import org.apache.storm.cluster.ClusterStateContext;
+import org.apache.storm.cluster.ClusterUtils;
+import org.apache.storm.cluster.DaemonType;
+import org.apache.storm.cluster.IStormClusterState;
+import org.apache.storm.daemon.DaemonCommon;
+import org.apache.storm.daemon.Shutdownable;
+import org.apache.storm.daemon.StormCommon;
+import org.apache.storm.generated.AlreadyAliveException;
+import org.apache.storm.generated.Assignment;
+import org.apache.storm.generated.AuthorizationException;
+import org.apache.storm.generated.BeginDownloadResult;
+import org.apache.storm.generated.ClusterSummary;
+import org.apache.storm.generated.CommonAggregateStats;
+import org.apache.storm.generated.ComponentAggregateStats;
+import org.apache.storm.generated.ComponentPageInfo;
+import org.apache.storm.generated.ComponentType;
+import org.apache.storm.generated.Credentials;
+import org.apache.storm.generated.DebugOptions;
+import org.apache.storm.generated.ErrorInfo;
+import org.apache.storm.generated.ExecutorInfo;
+import org.apache.storm.generated.ExecutorStats;
+import org.apache.storm.generated.ExecutorSummary;
+import org.apache.storm.generated.GetInfoOptions;
+import org.apache.storm.generated.InvalidTopologyException;
+import org.apache.storm.generated.KeyAlreadyExistsException;
+import org.apache.storm.generated.KeyNotFoundException;
+import org.apache.storm.generated.KillOptions;
+import org.apache.storm.generated.LSTopoHistory;
+import org.apache.storm.generated.ListBlobsResult;
+import org.apache.storm.generated.LogConfig;
+import org.apache.storm.generated.LogLevel;
+import org.apache.storm.generated.LogLevelAction;
+import org.apache.storm.generated.Nimbus.Iface;
+import org.apache.storm.generated.Nimbus.Processor;
+import org.apache.storm.generated.NimbusSummary;
+import org.apache.storm.generated.NodeInfo;
+import org.apache.storm.generated.NotAliveException;
+import org.apache.storm.generated.NumErrorsChoice;
+import 

[GitHub] storm pull request #1744: STORM-1276: line for line translation of nimbus to...

[erikdw]

Github user erikdw commented on a diff in the pull request:

https://github.com/apache/storm/pull/1744#discussion_r90198469
  
--- Diff: storm-core/src/jvm/org/apache/storm/daemon/nimbus/Nimbus.java ---
@@ -0,0 +1,3729 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.daemon.nimbus;
+
+import static org.apache.storm.metric.StormMetricsRegistry.registerMeter;
+import static org.apache.storm.utils.Utils.OR;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.InterruptedIOException;
+import java.io.OutputStream;
+import java.net.BindException;
+import java.net.ServerSocket;
+import java.nio.ByteBuffer;
+import java.nio.channels.Channels;
+import java.nio.channels.WritableByteChannel;
+import java.security.Principal;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.function.UnaryOperator;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+import javax.security.auth.Subject;
+
+import org.apache.storm.Config;
+import org.apache.storm.StormTimer;
+import org.apache.storm.blobstore.AtomicOutputStream;
+import org.apache.storm.blobstore.BlobStore;
+import org.apache.storm.blobstore.BlobStoreAclHandler;
+import org.apache.storm.blobstore.BlobSynchronizer;
+import org.apache.storm.blobstore.InputStreamWithMeta;
+import org.apache.storm.blobstore.KeySequenceNumber;
+import org.apache.storm.blobstore.LocalFsBlobStore;
+import org.apache.storm.cluster.ClusterStateContext;
+import org.apache.storm.cluster.ClusterUtils;
+import org.apache.storm.cluster.DaemonType;
+import org.apache.storm.cluster.IStormClusterState;
+import org.apache.storm.daemon.DaemonCommon;
+import org.apache.storm.daemon.Shutdownable;
+import org.apache.storm.daemon.StormCommon;
+import org.apache.storm.generated.AlreadyAliveException;
+import org.apache.storm.generated.Assignment;
+import org.apache.storm.generated.AuthorizationException;
+import org.apache.storm.generated.BeginDownloadResult;
+import org.apache.storm.generated.ClusterSummary;
+import org.apache.storm.generated.CommonAggregateStats;
+import org.apache.storm.generated.ComponentAggregateStats;
+import org.apache.storm.generated.ComponentPageInfo;
+import org.apache.storm.generated.ComponentType;
+import org.apache.storm.generated.Credentials;
+import org.apache.storm.generated.DebugOptions;
+import org.apache.storm.generated.ErrorInfo;
+import org.apache.storm.generated.ExecutorInfo;
+import org.apache.storm.generated.ExecutorStats;
+import org.apache.storm.generated.ExecutorSummary;
+import org.apache.storm.generated.GetInfoOptions;
+import org.apache.storm.generated.InvalidTopologyException;
+import org.apache.storm.generated.KeyAlreadyExistsException;
+import org.apache.storm.generated.KeyNotFoundException;
+import org.apache.storm.generated.KillOptions;
+import org.apache.storm.generated.LSTopoHistory;
+import org.apache.storm.generated.ListBlobsResult;
+import org.apache.storm.generated.LogConfig;
+import org.apache.storm.generated.LogLevel;
+import org.apache.storm.generated.LogLevelAction;
+import org.apache.storm.generated.Nimbus.Iface;
+import org.apache.storm.generated.Nimbus.Processor;
+import org.apache.storm.generated.NimbusSummary;
+import org.apache.storm.generated.NodeInfo;
+import org.apache.storm.generated.NotAliveException;
+import org.apache.storm.generated.NumErrorsChoice;
+import 

[GitHub] storm pull request #1744: STORM-1276: line for line translation of nimbus to...

[erikdw]

Github user erikdw commented on a diff in the pull request:

https://github.com/apache/storm/pull/1744#discussion_r90196514
  
--- Diff: storm-core/src/jvm/org/apache/storm/daemon/nimbus/Nimbus.java ---
@@ -0,0 +1,3729 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.daemon.nimbus;
+
+import static org.apache.storm.metric.StormMetricsRegistry.registerMeter;
+import static org.apache.storm.utils.Utils.OR;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.InterruptedIOException;
+import java.io.OutputStream;
+import java.net.BindException;
+import java.net.ServerSocket;
+import java.nio.ByteBuffer;
+import java.nio.channels.Channels;
+import java.nio.channels.WritableByteChannel;
+import java.security.Principal;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.function.UnaryOperator;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+import javax.security.auth.Subject;
+
+import org.apache.storm.Config;
+import org.apache.storm.StormTimer;
+import org.apache.storm.blobstore.AtomicOutputStream;
+import org.apache.storm.blobstore.BlobStore;
+import org.apache.storm.blobstore.BlobStoreAclHandler;
+import org.apache.storm.blobstore.BlobSynchronizer;
+import org.apache.storm.blobstore.InputStreamWithMeta;
+import org.apache.storm.blobstore.KeySequenceNumber;
+import org.apache.storm.blobstore.LocalFsBlobStore;
+import org.apache.storm.cluster.ClusterStateContext;
+import org.apache.storm.cluster.ClusterUtils;
+import org.apache.storm.cluster.DaemonType;
+import org.apache.storm.cluster.IStormClusterState;
+import org.apache.storm.daemon.DaemonCommon;
+import org.apache.storm.daemon.Shutdownable;
+import org.apache.storm.daemon.StormCommon;
+import org.apache.storm.generated.AlreadyAliveException;
+import org.apache.storm.generated.Assignment;
+import org.apache.storm.generated.AuthorizationException;
+import org.apache.storm.generated.BeginDownloadResult;
+import org.apache.storm.generated.ClusterSummary;
+import org.apache.storm.generated.CommonAggregateStats;
+import org.apache.storm.generated.ComponentAggregateStats;
+import org.apache.storm.generated.ComponentPageInfo;
+import org.apache.storm.generated.ComponentType;
+import org.apache.storm.generated.Credentials;
+import org.apache.storm.generated.DebugOptions;
+import org.apache.storm.generated.ErrorInfo;
+import org.apache.storm.generated.ExecutorInfo;
+import org.apache.storm.generated.ExecutorStats;
+import org.apache.storm.generated.ExecutorSummary;
+import org.apache.storm.generated.GetInfoOptions;
+import org.apache.storm.generated.InvalidTopologyException;
+import org.apache.storm.generated.KeyAlreadyExistsException;
+import org.apache.storm.generated.KeyNotFoundException;
+import org.apache.storm.generated.KillOptions;
+import org.apache.storm.generated.LSTopoHistory;
+import org.apache.storm.generated.ListBlobsResult;
+import org.apache.storm.generated.LogConfig;
+import org.apache.storm.generated.LogLevel;
+import org.apache.storm.generated.LogLevelAction;
+import org.apache.storm.generated.Nimbus.Iface;
+import org.apache.storm.generated.Nimbus.Processor;
+import org.apache.storm.generated.NimbusSummary;
+import org.apache.storm.generated.NodeInfo;
+import org.apache.storm.generated.NotAliveException;
+import org.apache.storm.generated.NumErrorsChoice;
+import 

[GitHub] storm pull request #1744: STORM-1276: line for line translation of nimbus to...

[erikdw]

Github user erikdw commented on a diff in the pull request:

https://github.com/apache/storm/pull/1744#discussion_r9024
  
--- Diff: storm-core/src/jvm/org/apache/storm/daemon/nimbus/Nimbus.java ---
@@ -0,0 +1,3729 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.daemon.nimbus;
+
+import static org.apache.storm.metric.StormMetricsRegistry.registerMeter;
+import static org.apache.storm.utils.Utils.OR;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.InterruptedIOException;
+import java.io.OutputStream;
+import java.net.BindException;
+import java.net.ServerSocket;
+import java.nio.ByteBuffer;
+import java.nio.channels.Channels;
+import java.nio.channels.WritableByteChannel;
+import java.security.Principal;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.function.UnaryOperator;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+import javax.security.auth.Subject;
+
+import org.apache.storm.Config;
+import org.apache.storm.StormTimer;
+import org.apache.storm.blobstore.AtomicOutputStream;
+import org.apache.storm.blobstore.BlobStore;
+import org.apache.storm.blobstore.BlobStoreAclHandler;
+import org.apache.storm.blobstore.BlobSynchronizer;
+import org.apache.storm.blobstore.InputStreamWithMeta;
+import org.apache.storm.blobstore.KeySequenceNumber;
+import org.apache.storm.blobstore.LocalFsBlobStore;
+import org.apache.storm.cluster.ClusterStateContext;
+import org.apache.storm.cluster.ClusterUtils;
+import org.apache.storm.cluster.DaemonType;
+import org.apache.storm.cluster.IStormClusterState;
+import org.apache.storm.daemon.DaemonCommon;
+import org.apache.storm.daemon.Shutdownable;
+import org.apache.storm.daemon.StormCommon;
+import org.apache.storm.generated.AlreadyAliveException;
+import org.apache.storm.generated.Assignment;
+import org.apache.storm.generated.AuthorizationException;
+import org.apache.storm.generated.BeginDownloadResult;
+import org.apache.storm.generated.ClusterSummary;
+import org.apache.storm.generated.CommonAggregateStats;
+import org.apache.storm.generated.ComponentAggregateStats;
+import org.apache.storm.generated.ComponentPageInfo;
+import org.apache.storm.generated.ComponentType;
+import org.apache.storm.generated.Credentials;
+import org.apache.storm.generated.DebugOptions;
+import org.apache.storm.generated.ErrorInfo;
+import org.apache.storm.generated.ExecutorInfo;
+import org.apache.storm.generated.ExecutorStats;
+import org.apache.storm.generated.ExecutorSummary;
+import org.apache.storm.generated.GetInfoOptions;
+import org.apache.storm.generated.InvalidTopologyException;
+import org.apache.storm.generated.KeyAlreadyExistsException;
+import org.apache.storm.generated.KeyNotFoundException;
+import org.apache.storm.generated.KillOptions;
+import org.apache.storm.generated.LSTopoHistory;
+import org.apache.storm.generated.ListBlobsResult;
+import org.apache.storm.generated.LogConfig;
+import org.apache.storm.generated.LogLevel;
+import org.apache.storm.generated.LogLevelAction;
+import org.apache.storm.generated.Nimbus.Iface;
+import org.apache.storm.generated.Nimbus.Processor;
+import org.apache.storm.generated.NimbusSummary;
+import org.apache.storm.generated.NodeInfo;
+import org.apache.storm.generated.NotAliveException;
+import org.apache.storm.generated.NumErrorsChoice;
+import 

[GitHub] storm pull request #1744: STORM-1276: line for line translation of nimbus to...

[erikdw]

Github user erikdw commented on a diff in the pull request:

https://github.com/apache/storm/pull/1744#discussion_r90192685
  
--- Diff: storm-core/src/jvm/org/apache/storm/daemon/nimbus/Nimbus.java ---
@@ -0,0 +1,3640 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.daemon.nimbus;
+
+import static org.apache.storm.metric.StormMetricsRegistry.registerMeter;
+import static org.apache.storm.utils.Utils.OR;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.InterruptedIOException;
+import java.io.OutputStream;
+import java.net.BindException;
+import java.net.ServerSocket;
+import java.nio.ByteBuffer;
+import java.nio.channels.Channels;
+import java.nio.channels.WritableByteChannel;
+import java.security.Principal;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.function.UnaryOperator;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+import javax.security.auth.Subject;
+
+import org.apache.storm.Config;
+import org.apache.storm.StormTimer;
+import org.apache.storm.blobstore.AtomicOutputStream;
+import org.apache.storm.blobstore.BlobStore;
+import org.apache.storm.blobstore.BlobStoreAclHandler;
+import org.apache.storm.blobstore.BlobSynchronizer;
+import org.apache.storm.blobstore.InputStreamWithMeta;
+import org.apache.storm.blobstore.KeySequenceNumber;
+import org.apache.storm.blobstore.LocalFsBlobStore;
+import org.apache.storm.cluster.ClusterStateContext;
+import org.apache.storm.cluster.ClusterUtils;
+import org.apache.storm.cluster.DaemonType;
+import org.apache.storm.cluster.IStormClusterState;
+import org.apache.storm.daemon.DaemonCommon;
+import org.apache.storm.daemon.Shutdownable;
+import org.apache.storm.daemon.StormCommon;
+import org.apache.storm.generated.AlreadyAliveException;
+import org.apache.storm.generated.Assignment;
+import org.apache.storm.generated.AuthorizationException;
+import org.apache.storm.generated.BeginDownloadResult;
+import org.apache.storm.generated.ClusterSummary;
+import org.apache.storm.generated.CommonAggregateStats;
+import org.apache.storm.generated.ComponentAggregateStats;
+import org.apache.storm.generated.ComponentPageInfo;
+import org.apache.storm.generated.ComponentType;
+import org.apache.storm.generated.Credentials;
+import org.apache.storm.generated.DebugOptions;
+import org.apache.storm.generated.ErrorInfo;
+import org.apache.storm.generated.ExecutorInfo;
+import org.apache.storm.generated.ExecutorStats;
+import org.apache.storm.generated.ExecutorSummary;
+import org.apache.storm.generated.GetInfoOptions;
+import org.apache.storm.generated.InvalidTopologyException;
+import org.apache.storm.generated.KeyAlreadyExistsException;
+import org.apache.storm.generated.KeyNotFoundException;
+import org.apache.storm.generated.KillOptions;
+import org.apache.storm.generated.LSTopoHistory;
+import org.apache.storm.generated.ListBlobsResult;
+import org.apache.storm.generated.LogConfig;
+import org.apache.storm.generated.LogLevel;
+import org.apache.storm.generated.LogLevelAction;
+import org.apache.storm.generated.Nimbus.Iface;
+import org.apache.storm.generated.Nimbus.Processor;
+import org.apache.storm.generated.NimbusSummary;
+import org.apache.storm.generated.NodeInfo;
+import org.apache.storm.generated.NotAliveException;
+import org.apache.storm.generated.NumErrorsChoice;
+import 

[GitHub] storm pull request #1744: STORM-1276: line for line translation of nimbus to...

[erikdw]

Github user erikdw commented on a diff in the pull request:

https://github.com/apache/storm/pull/1744#discussion_r90191719
  
--- Diff: storm-core/src/jvm/org/apache/storm/daemon/nimbus/Nimbus.java ---
@@ -0,0 +1,3640 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.daemon.nimbus;
+
+import static org.apache.storm.metric.StormMetricsRegistry.registerMeter;
+import static org.apache.storm.utils.Utils.OR;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.InterruptedIOException;
+import java.io.OutputStream;
+import java.net.BindException;
+import java.net.ServerSocket;
+import java.nio.ByteBuffer;
+import java.nio.channels.Channels;
+import java.nio.channels.WritableByteChannel;
+import java.security.Principal;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.function.UnaryOperator;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+import javax.security.auth.Subject;
+
+import org.apache.storm.Config;
+import org.apache.storm.StormTimer;
+import org.apache.storm.blobstore.AtomicOutputStream;
+import org.apache.storm.blobstore.BlobStore;
+import org.apache.storm.blobstore.BlobStoreAclHandler;
+import org.apache.storm.blobstore.BlobSynchronizer;
+import org.apache.storm.blobstore.InputStreamWithMeta;
+import org.apache.storm.blobstore.KeySequenceNumber;
+import org.apache.storm.blobstore.LocalFsBlobStore;
+import org.apache.storm.cluster.ClusterStateContext;
+import org.apache.storm.cluster.ClusterUtils;
+import org.apache.storm.cluster.DaemonType;
+import org.apache.storm.cluster.IStormClusterState;
+import org.apache.storm.daemon.DaemonCommon;
+import org.apache.storm.daemon.Shutdownable;
+import org.apache.storm.daemon.StormCommon;
+import org.apache.storm.generated.AlreadyAliveException;
+import org.apache.storm.generated.Assignment;
+import org.apache.storm.generated.AuthorizationException;
+import org.apache.storm.generated.BeginDownloadResult;
+import org.apache.storm.generated.ClusterSummary;
+import org.apache.storm.generated.CommonAggregateStats;
+import org.apache.storm.generated.ComponentAggregateStats;
+import org.apache.storm.generated.ComponentPageInfo;
+import org.apache.storm.generated.ComponentType;
+import org.apache.storm.generated.Credentials;
+import org.apache.storm.generated.DebugOptions;
+import org.apache.storm.generated.ErrorInfo;
+import org.apache.storm.generated.ExecutorInfo;
+import org.apache.storm.generated.ExecutorStats;
+import org.apache.storm.generated.ExecutorSummary;
+import org.apache.storm.generated.GetInfoOptions;
+import org.apache.storm.generated.InvalidTopologyException;
+import org.apache.storm.generated.KeyAlreadyExistsException;
+import org.apache.storm.generated.KeyNotFoundException;
+import org.apache.storm.generated.KillOptions;
+import org.apache.storm.generated.LSTopoHistory;
+import org.apache.storm.generated.ListBlobsResult;
+import org.apache.storm.generated.LogConfig;
+import org.apache.storm.generated.LogLevel;
+import org.apache.storm.generated.LogLevelAction;
+import org.apache.storm.generated.Nimbus.Iface;
+import org.apache.storm.generated.Nimbus.Processor;
+import org.apache.storm.generated.NimbusSummary;
+import org.apache.storm.generated.NodeInfo;
+import org.apache.storm.generated.NotAliveException;
+import org.apache.storm.generated.NumErrorsChoice;
+import 

[GitHub] storm pull request #1801: STORM-2220 Added passing cassandra config to indiv...

[satishd]

Github user satishd commented on a diff in the pull request:

https://github.com/apache/storm/pull/1801#discussion_r90189064
  
--- Diff: 
external/storm-cassandra/src/main/java/org/apache/storm/cassandra/bolt/BaseCassandraBolt.java
 ---
@@ -91,8 +93,12 @@ public BaseCassandraBolt(CQLStatementTupleMapper 
tupleMapper) {
 public void prepare(Map stormConfig, TopologyContext topologyContext, 
OutputCollector outputCollector) {
 this.outputCollector = outputCollector;
 this.stormConfig = stormConfig;
-this.cassandraConfConfig = new CassandraConf(stormConfig);
-this.client = clientProvider.getClient(this.stormConfig);
+
+Map cassandraClientConfig = cassandraConfig != 
null ? stormConfig : stormConfig;
--- End diff --

@HeartSaVioR Thanks for catching that. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1778: [STORM-2082][SQL] add sql external module storm-sq...

[HeartSaVioR]

Github user HeartSaVioR commented on a diff in the pull request:

https://github.com/apache/storm/pull/1778#discussion_r90187413
  
--- Diff: docs/storm-sql-reference.md ---
@@ -1312,4 +1313,23 @@ You can use below as working reference for 
`--artifacts` option, and change depe
 
 
`org.apache.storm:storm-sql-mongodb:2.0.0-SNAPSHOT,org.apache.storm:storm-mongodb:2.0.0-SNAPSHOT`
 
-Storing record with preserving fields are not supported for now.
\ No newline at end of file
+Storing record with preserving fields are not supported for now.
+
+ HDFS
+
+HDFS data source requires below properties to be set:
+
+* `hdfs.file.path`: HDFS file path
+* `hdfs.file.name`: HDFS file name - please refer to 
[SimpleFileNameFormat]({{page.git-blob-base}}/external/storm-hdfs/src/main/java/org/apache/storm/hdfs/trident/format/SimpleFileNameFormat.java)
+* `hdfs.rotation.size.kb`: HDFS FileSizeRotationPolicy in KB
+* `hdfs.rotation.time.seconds`: HDFS TimedRotationPolicy in seconds
+
+Please note that `hdfs.rotation.size.kb` and `hdfs.rotation.time.seconds` 
only one can be used for hdfs rotation.
+
+And note that `storm-sql-hdfs` requires users to provide `storm-hdfs`.
+You can use below as working reference for `--artifacts` option, and 
change dependencies version if really needed:
+

+`org.apache.storm:storm-sql-hdfs:2.0.0-SNAPSHOT,org.apache.storm:storm-hdfs:2.0.0-SNAPSHOT`
+
+Also, hdfs configuration files should be provided.
+You can put the `core-site.xml` and `hdfs-site.xm` into the `conf` 
directory.
--- End diff --

typo: `hdfs-site.xm` -> `hdfs-site.xml`.
and do you mean putting files to `conf` dir in Storm installation 
directory? Could you explain how they can be referred from workers?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---