date:20160831

[jira] [Resolved] (STORM-2070) Sigar native binary download link went 404

2016-08-31 Thread Jungtaek Lim (JIRA)


 [ 
https://issues.apache.org/jira/browse/STORM-2070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jungtaek Lim resolved STORM-2070.
-
   Resolution: Fixed
Fix Version/s: 1.0.3
   1.1.0
   2.0.0

Merged into master, 1.x, 1.0.x branches.

> Sigar native binary download link went 404
> --
>
> Key: STORM-2070
> URL: https://issues.apache.org/jira/browse/STORM-2070
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-metrics
>Affects Versions: 2.0.0, 1.0.2
>Reporter: Jungtaek Lim
>Assignee: Jungtaek Lim
> Fix For: 2.0.0, 1.1.0, 1.0.3
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> {code}
>   
> 
> 1.6.4
> 
> https://magelan.googlecode.com/files/hyperic-sigar-${sigar.version}.zip
> 8f79d4039ca3ec6c88039d5897a80a268213e6b7
> 
> 
> ${settings.localRepository}/org/fusesource/sigar/${sigar.version}
>   
> {code}
> Sigar download url is set to 
> https://magelan.googlecode.com/files/hyperic-sigar-1.6.4.zip which is not 
> working. 
> Google Code seems changed their download link. Current link of sigar binary 
> 1.6.4 is 
> https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/magelan/hyperic-sigar-1.6.4.zip



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request #1660: STORM-2070 Fix sigar native binary download link

2016-08-31 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1660


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (STORM-2075) Storm SQL Phase III

2016-08-31 Thread Jungtaek Lim (JIRA)


 [ 
https://issues.apache.org/jira/browse/STORM-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jungtaek Lim updated STORM-2075:

Description: 
This epic tracks the effort of the phase III development of StormSQL.

For now Storm SQL only supports Kafka as a data source, which is limited for 
normal use cases. We would need to support others as well. Candidates are 
external modules.

And also consider supporting State since Trident only provides a way to set / 
get via batch. Current way (each) does insert a row 1 by 1.

  was:
This epic tracks the effort of the phase III development of StormSQL.

For now Storm SQL only supports Kafka as a data source, which is limited for 
normal use cases. We would need to support others as well. Candidates are 
external modules.


> Storm SQL Phase III
> ---
>
> Key: STORM-2075
> URL: https://issues.apache.org/jira/browse/STORM-2075
> Project: Apache Storm
>  Issue Type: Epic
>  Components: storm-sql
>Reporter: Jungtaek Lim
>
> This epic tracks the effort of the phase III development of StormSQL.
> For now Storm SQL only supports Kafka as a data source, which is limited for 
> normal use cases. We would need to support others as well. Candidates are 
> external modules.
> And also consider supporting State since Trident only provides a way to set / 
> get via batch. Current way (each) does insert a row 1 by 1.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Which do you think is a better way to handle EXPLAIN in multiple queries on Storm SQL?

2016-08-31 Thread Jungtaek Lim

Hi devs,

I'm going to address STORM-1444
 : Support EXPLAIN
statement in StormSQL, and need our opinions about this.

SqlRunner analyzes sql statements from given file and converts each
statement to Trident topology (without DDL statement) and submits.
(This means SqlRunner may submit multiple topologies.)

If we just handle EXPLAIN only bound to specific sql statement, only that
statement is not submitted but shows the plan, and other statements will be
submitted.
If we don't allow EXPLAIN in statement level but having "explain mode" of
SqlRunner, SqlRunner will show the plan for each sql statement (without DDL
statement).

Which do you think makes more sense to you?

Thanks,
Jungtaek Lim (HeartSaVioR)

ps. I also feel that reading sql statements from file is already
inconvenient for just 'EXPLAIN', but since Storm SQL doesn't register table
to somewhere, DDL statements are needed before DML. Might need to have REPL
eventually Need to think more about this side.

[jira] [Commented] (STORM-1444) Support EXPLAIN statement in StormSQL

2016-08-31 Thread Jungtaek Lim (JIRA)


[ 
https://issues.apache.org/jira/browse/STORM-1444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15454308#comment-15454308
 ] 

Jungtaek Lim commented on STORM-1444:
-

It might need to have different approach.

Storm SQL is creating Trident topology and execute in local cluster or submit 
to remote cluster. The usage of EXPLAIN is opposite, since users don't expect 
topology will be executed or submitted if users want "EXPLAIN".

So while we can still support "EXPLAIN" statement (by not executing or 
submitting topology when query has EXPLAIN),  we can also add explain mode just 
same level as standalone and trident.

Since we have multiple queries on sql file, what we select affects UX of Storm 
SQL Runner.
I'd rather hear voices from dev@ list which is better, and go ahead.

> Support EXPLAIN statement in StormSQL
> -
>
> Key: STORM-1444
> URL: https://issues.apache.org/jira/browse/STORM-1444
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: storm-sql
>Reporter: Haohui Mai
>Assignee: Jungtaek Lim
>
> It is useful to support the `EXPLAIN` statement in StormSQL to allow 
> debugging and customizing the topology generated by StormSQL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm issue #1665: STORM-2074: fix storm-kafka-monitor NPE bug

2016-08-31 Thread vesense

Github user vesense commented on the issue:

https://github.com/apache/storm/pull/1665
  
Thanks @priyank5485 @abellina I think I have addressed your comments.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1666: STORM-1459: allow not specifying producer properties in r...

2016-08-31 Thread HeartSaVioR

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1666
  
Great. Thanks for the quick update. +1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1666: STORM-1459: allow not specifying producer properties in r...

2016-08-31 Thread manuzhang

Github user manuzhang commented on the issue:

https://github.com/apache/storm/pull/1666
  
@HeartSaVioR Updated.  I use "Kafka sink table" instead of "consumer" to 
avoid confusing with kafka consumer.  And I don't think `TBLPROPERTIES` is an 
option for "table as a producer" so I leave it out.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1648: STORM-2054 DependencyResolver should be aware of relative...

2016-08-31 Thread HeartSaVioR

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1648
  
@revans2 This is also a fix for STORM-2059. please review this.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1660: STORM-2070 Fix sigar native binary download link

2016-08-31 Thread hackerwin7

Github user hackerwin7 commented on the issue:

https://github.com/apache/storm/pull/1660
  
the fix is worked to build


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (STORM-2074) NPE bug in storm-kafka-monitor

2016-08-31 Thread Xin Wang (JIRA)


 [ 
https://issues.apache.org/jira/browse/STORM-2074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xin Wang updated STORM-2074:

Description: storm-kafka-monitor will throw an NPE when __--zk-node__ value 
does not exist. this is not friendly.  (was: storm-kafka-monitor will throw an 
NPE when __--zk-node__ value is invalid. this is not friendly.)

> NPE bug in storm-kafka-monitor
> --
>
> Key: STORM-2074
> URL: https://issues.apache.org/jira/browse/STORM-2074
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Xin Wang
>Assignee: Xin Wang
>Priority: Minor
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> storm-kafka-monitor will throw an NPE when __--zk-node__ value does not 
> exist. this is not friendly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request #1665: STORM-2074: fix storm-kafka-monitor NPE bug

2016-08-31 Thread vesense

Github user vesense commented on a diff in the pull request:

https://github.com/apache/storm/pull/1665#discussion_r77109600
  
--- Diff: 
external/storm-kafka-monitor/src/main/java/org/apache/storm/kafka/monitor/KafkaOffsetLagUtil.java
 ---
@@ -373,16 +377,20 @@ private static Options buildOptions () {
 curatorFramework.start();
 String partitionPrefix = "partition_";
 String zkPath = oldKafkaSpoutOffsetQuery.getZkPath();
-if (!zkPath.endsWith("/")) {
-zkPath += "/";
+ if (zkPath.endsWith("/")) {
+zkPath = zkPath.substring(0, zkPath.length()-1);
+}
+if (curatorFramework.checkExists().forPath(zkPath) == null) {
+   System.out.printf("--zk-node '%s' is not exists.%n", zkPath);
--- End diff --

Will fix. Thanks


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (STORM-2075) Storm SQL Phase III

2016-08-31 Thread Jungtaek Lim (JIRA)

Jungtaek Lim created STORM-2075:
---

 Summary: Storm SQL Phase III
 Key: STORM-2075
 URL: https://issues.apache.org/jira/browse/STORM-2075
 Project: Apache Storm
  Issue Type: Epic
  Components: storm-sql
Reporter: Jungtaek Lim


This epic tracks the effort of the phase III development of StormSQL.

For now Storm SQL only supports Kafka as a data source, which is limited for 
normal use cases. We would need to support others as well. Candidates are 
external modules.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request #1665: STORM-2074: fix storm-kafka-monitor NPE bug

2016-08-31 Thread vesense

Github user vesense commented on a diff in the pull request:

https://github.com/apache/storm/pull/1665#discussion_r77107687
  
--- Diff: 
external/storm-kafka-monitor/src/main/java/org/apache/storm/kafka/monitor/KafkaOffsetLagUtil.java
 ---
@@ -373,16 +377,20 @@ private static Options buildOptions () {
 curatorFramework.start();
 String partitionPrefix = "partition_";
 String zkPath = oldKafkaSpoutOffsetQuery.getZkPath();
-if (!zkPath.endsWith("/")) {
-zkPath += "/";
+ if (zkPath.endsWith("/")) {
--- End diff --

I think it is better to check the variable before use it.  and the zkPath 
must not ends with `/`.
the `printUsageAndExit` method needs argument  `Options` which is not 
available in `getOldConsumerOffsetsFromZk`. so I use `exit` directly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1666: STORM-1459: allow not specifying producer properties in r...

2016-08-31 Thread HeartSaVioR

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1666
  
+1 except minor documentation issue.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1666: STORM-1459: allow not specifying producer properti...

2016-08-31 Thread HeartSaVioR

Github user HeartSaVioR commented on a diff in the pull request:

https://github.com/apache/storm/pull/1666#discussion_r77107492
  
--- Diff: docs/storm-sql.md ---
@@ -104,21 +104,22 @@ Let's say there is a Kafka stream that represents the 
transactions of orders. Ea
 The user can specify the following SQL statements in the SQL file:
 
 ```
-CREATE EXTERNAL TABLE ORDERS (ID INT PRIMARY KEY, UNIT_PRICE INT, QUANTITY 
INT) LOCATION 'kafka://localhost:2181/brokers?topic=orders' TBLPROPERTIES 
'{"producer":{"bootstrap.servers":"localhost:9092","acks":"1","key.serializer":"org.apache.org.apache.storm.kafka.IntSerializer","value.serializer":"org.apache.org.apache.storm.kafka.ByteBufferSerializer"}}'
+CREATE EXTERNAL TABLE ORDERS (ID INT PRIMARY KEY, UNIT_PRICE INT, QUANTITY 
INT) LOCATION 'kafka://localhost:2181/brokers?topic=orders'
 CREATE EXTERNAL TABLE LARGE_ORDERS (ID INT PRIMARY KEY, TOTAL INT) 
LOCATION 'kafka://localhost:2181/brokers?topic=large_orders' TBLPROPERTIES 
'{"producer":{"bootstrap.servers":"localhost:9092","acks":"1","key.serializer":"org.apache.org.apache.storm.kafka.IntSerializer","value.serializer":"org.apache.org.apache.storm.kafka.ByteBufferSerializer"}}'
 INSERT INTO LARGE_ORDERS SELECT ID, UNIT_PRICE * QUANTITY AS TOTAL FROM 
ORDERS WHERE UNIT_PRICE * QUANTITY > 50
 ```
 
-The first statement defines the table `ORDER` which represents the input 
stream. The `LOCATION` clause specifies the ZkHost (`localhost:2181`), the path 
of the brokers in ZooKeeper (`/brokers`) and the topic (`orders`). The 
`TBLPROPERTIES` clause specifies the configuration of 
[KafkaProducer](http://kafka.apache.org/documentation.html#producerconfigs).
-Current implementation of `storm-sql-kafka` requires specifying both 
`LOCATION` and `TBLPROPERTIES` clauses even though the table is read-only or 
write-only.
+The first statement defines the table `ORDER` which represents the input 
stream. The `LOCATION` clause specifies the ZkHost (`localhost:2181`), the path 
of the brokers in ZooKeeper (`/brokers`) and the topic (`orders`). 
 
-Similarly, the second statement specifies the table `LARGE_ORDERS` which 
represents the output stream. The third statement is a `SELECT` statement which 
defines the topology: it instructs StormSQL to filter all orders in the 
external table `ORDERS`, calculates the total price and inserts matching 
records into the Kafka stream specified by `LARGE_ORDER`.
+Similarly, the second statement specifies the table `LARGE_ORDERS` which 
represents the output stream. The `TBLPROPERTIES` clause specifies the 
configuration of 
[KafkaProducer](http://kafka.apache.org/documentation.html#producerconfigs). 
--- End diff --

Might be better to clarify again that `TBLPROPERTIES` is optional for table 
as a producer (read) and mandatory for table as a consumer (write).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1665: STORM-2074: fix storm-kafka-monitor NPE bug

2016-08-31 Thread abellina

Github user abellina commented on a diff in the pull request:

https://github.com/apache/storm/pull/1665#discussion_r77107294
  
--- Diff: 
external/storm-kafka-monitor/src/main/java/org/apache/storm/kafka/monitor/KafkaOffsetLagUtil.java
 ---
@@ -373,16 +377,20 @@ private static Options buildOptions () {
 curatorFramework.start();
 String partitionPrefix = "partition_";
 String zkPath = oldKafkaSpoutOffsetQuery.getZkPath();
-if (!zkPath.endsWith("/")) {
-zkPath += "/";
+ if (zkPath.endsWith("/")) {
+zkPath = zkPath.substring(0, zkPath.length()-1);
+}
+if (curatorFramework.checkExists().forPath(zkPath) == null) {
+   System.out.printf("--zk-node '%s' is not exists.%n", zkPath);
--- End diff --

%n? You mean.. println? Also small nit: "is not exists" => "does not exist"

This is also using tabs so the indentation is off. Why not use spaces (4 
spaces)?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1665: STORM-2074: fix storm-kafka-monitor NPE bug

2016-08-31 Thread vesense

Github user vesense commented on a diff in the pull request:

https://github.com/apache/storm/pull/1665#discussion_r77106883
  
--- Diff: 
external/storm-kafka-monitor/src/main/java/org/apache/storm/kafka/monitor/KafkaOffsetLagUtil.java
 ---
@@ -89,6 +89,10 @@ public static void main (String args[]) {
 printUsageAndExit(options, OPTION_ZK_SERVERS_LONG + " 
and " + OPTION_ZK_COMMITTED_NODE_LONG + " are required  with " +
 OPTION_OLD_CONSUMER_LONG);
 }
+String zkNode = 
commandLine.getOptionValue(OPTION_ZK_COMMITTED_NODE_LONG);
--- End diff --

`zkNode == null` is not unnecessary since `hasArg` is set to true. and I 
will remove.
but `zkNode.length() <= 1` is necessary because zkPath should start with 
`/`, so the valid path length should `> 1`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1949) Backpressure can cause spout to stop emitting and stall topology

2016-08-31 Thread Zhuo Liu (JIRA)


[ 
https://issues.apache.org/jira/browse/STORM-1949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15453966#comment-15453966
 ] 

Zhuo Liu commented on STORM-1949:
-

Hi [~roshan_naik], [~abellina] has https://github.com/apache/storm/pull/1632 
merged into 2.x master. Would you mind testing your example with the latest 
master? See if the refactoring has fixed the bug you mentioned. Thank you in 
advance!

> Backpressure can cause spout to stop emitting and stall topology
> 
>
> Key: STORM-1949
> URL: https://issues.apache.org/jira/browse/STORM-1949
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Roshan Naik
>Assignee: Alessandro Bellina
> Attachments: 1.x-branch-works-perfect.png, wordcounttopo.zip
>
>
> Problem can be reproduced by this [Word count 
> topology|https://github.com/hortonworks/storm/blob/perftopos1.x/examples/storm-starter/src/jvm/org/apache/storm/starter/perf/FileReadWordCountTopo.java]
> within a IDE.
> I ran it with 1 spout instance, 2 splitter bolt instances, 2 counter bolt 
> instances.
> The problem is more easily reproduced with WC topology as it causes an 
> explosion of tuples due to splitting a sentence tuple into word tuples. As 
> the bolts have to process more tuples than the  spout is producing, spout 
> needs to operate slower.
> The amount of time it takes for the topology to stall can vary.. but 
> typically under 10 mins. 
> *My theory:*  I suspect there is a race condition in the way ZK is being 
> utilized to enable/disable back pressure. When congested (i.e pressure 
> exceeds high water mark), the bolt's worker records this congested situation 
> in ZK by creating a node. Once the congestion is reduced below the low water 
> mark, it deletes this node. 
> The spout's worker has setup a watch on the parent node, expecting a callback 
> whenever there is change in the child nodes. On receiving the callback the 
> spout's worker lists the parent node to check if there are 0 or more child 
> nodes it is essentially trying to figure out the nature of state change 
> in ZK to determine whether to throttle or not. Subsequently  it setsup 
> another watch in ZK to keep an eye on future changes.
> When there are multiple bolts, there can be rapid creation/deletion of these 
> ZK nodes. Between the time the worker receives a callback and sets up the 
> next watch.. many changes may have undergone in ZK which will go unnoticed by 
> the spout. 
> The condition that the bolts are no longer congested may not get noticed as a 
> result of this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request #1666: STORM-1459: allow not specifying producer properti...

2016-08-31 Thread manuzhang

GitHub user manuzhang opened a pull request:

https://github.com/apache/storm/pull/1666

STORM-1459: allow not specifying producer properties in read-only Kafâ¦

â¦ka table in StormSQL

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/manuzhang/storm storm_sql

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1666.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1666


commit 4f9c80a78a2a65a5eeaf53cc22a74bacf4e4a039
Author: manuzhang 
Date:   2016-08-31T22:54:11Z

STORM-1459: allow not specifying producer properties in read-only Kafka 
table in StormSQL




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1565: STORM-1970: external project examples refator

2016-08-31 Thread HeartSaVioR

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1565
  
I think we have to make sure that users can run example topologies with 
local and also remote.
(Yes these modules are just examples but 'runnable' examples.)

So it might be better to apply two things:

- profile trick to change scope of 'storm-core' : other modules use 
'intellij' profile
- create fat jar for each project : change scope of external module to 
'compile'. Is there a reason to set this to 'provided'?

Other things look good.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1565: STORM-1970: external project examples refator

2016-08-31 Thread HeartSaVioR

Github user HeartSaVioR commented on a diff in the pull request:

https://github.com/apache/storm/pull/1565#discussion_r77088437
  
--- Diff: examples/storm-mqtt-examples/pom.xml ---
@@ -24,18 +24,24 @@
 
   storm-mqtt-examples
 
-  
-org.apache.storm
-storm-mqtt-parent
-2.0.0-SNAPSHOT
-../pom.xml
-  
+
--- End diff --

nit: broken indentation


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (STORM-2056) Bugs in logviewer

2016-08-31 Thread Jungtaek Lim (JIRA)


 [ 
https://issues.apache.org/jira/browse/STORM-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jungtaek Lim resolved STORM-2056.
-
   Resolution: Fixed
Fix Version/s: 1.0.3
   1.1.0
   2.0.0

Thanks [~jerrypeng], I merged into master, 1.x, 1.0.x branches.

> Bugs in logviewer
> -
>
> Key: STORM-2056
> URL: https://issues.apache.org/jira/browse/STORM-2056
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Boyang Jerry Peng
>Assignee: Boyang Jerry Peng
> Fix For: 2.0.0, 1.1.0, 1.0.3
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> 1. Incorrect url for prev,first,last,next buttons when viewing daemon logs 
> via logviewer
> Example:
> http://storm.cluster.com:8000/log?file=nimbus.log&start=0&length=51200
> should be:
> http://storm.cluster.com:8000/daemonlog?file=nimbus.log&start=0&length=51200
> Function with bug:
> https://github.com/apache/storm/blob/master/storm-core/src/clj/org/apache/storm/daemon/logviewer.clj#L374
> 2. Downloading daemon files causes exception to be thrown because of function 
> download-log-file checks authorization via worker.yaml.  Obviously daemon log 
> root will not have this file.
> java.io.FileNotFoundException: 
> /home/y/var/storm/workers-artifacts/supervisor.log/worker.yaml (No such file 
> or directory)
>   at java.io.FileInputStream.open0(Native Method)
>   at java.io.FileInputStream.open(FileInputStream.java:195)
>   at java.io.FileInputStream.(FileInputStream.java:138)
>   at java.io.FileReader.(FileReader.java:72)
>   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>   at 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>   at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
>   at clojure.lang.Reflector.invokeConstructor(Reflector.java:180)
>   at backtype.storm.util$clojure_from_yaml_file.invoke(util.clj:1066)
>   at 
> backtype.storm.daemon.logviewer$get_log_user_group_whitelist.invoke(logviewer.clj:310)
>   at 
> backtype.storm.daemon.logviewer$authorized_log_user_QMARK_.invoke(logviewer.clj:326)
>   at 
> backtype.storm.daemon.logviewer$download_log_file.invoke(logviewer.clj:497)
>   at backtype.storm.daemon.logviewer$fn__11528.invoke(logviewer.clj:1024)
>   at 
> org.apache.storm.shade.compojure.core$make_route$fn__6445.invoke(core.clj:93)
>   at 
> org.apache.storm.shade.compojure.core$if_route$fn__6433.invoke(core.clj:39)
>   at 
> org.apache.storm.shade.compojure.core$if_method$fn__6426.invoke(core.clj:24)
>   at 
> org.apache.storm.shade.compojure.core$routing$fn__6451.invoke(core.clj:106)
>   at clojure.core$some.invoke(core.clj:2515)
>   at org.apache.storm.shade.compojure.core$routing.doInvoke(core.clj:106)
>   at clojure.lang.RestFn.applyTo(RestFn.java:139)
>   at clojure.core$apply.invoke(core.clj:626)
>   at 
> org.apache.storm.shade.compojure.core$routes$fn__6455.invoke(core.clj:111)
> 3. search-log-file should not check for authorized users in worker.yaml
> https://github.com/apache/storm/blob/master/storm-core/src/clj/org/apache/storm/daemon/logviewer.clj#L833



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request #1659: [STORM-2056] - Bugs in logviewer

2016-08-31 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1659


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1658: [STORM-1646] Fix ExponentialBackoffMsgRetryManager...

2016-08-31 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1658


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (STORM-1646) Intermittent test failures in storm-kafka unit tests

2016-08-31 Thread Jungtaek Lim (JIRA)


 [ 
https://issues.apache.org/jira/browse/STORM-1646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jungtaek Lim resolved STORM-1646.
-
   Resolution: Fixed
Fix Version/s: 1.0.3
   1.1.0
   2.0.0

Thanks [~ppoulosk], I merged into master, 1.x, 1.0.x branches.

> Intermittent test failures in storm-kafka unit tests
> 
>
> Key: STORM-1646
> URL: https://issues.apache.org/jira/browse/STORM-1646
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Paul Poulosky
>Assignee: Paul Poulosky
> Fix For: 2.0.0, 1.1.0, 1.0.3
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> We have been seeing intermittent test failures in KafkaUtilsTest on slow 
> hardware and lightly resourced VMs, as well as an intermittent race condition 
> when running the storm-kafka ExponentialBackoffManager  unit test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm issue #1662: [STORM-1344] Remove sql command from storm-jdbc build.

2016-08-31 Thread HeartSaVioR

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1662
  
@ppoulosk 
Could you rebase this into master and squash commits into one? I guess 
discarding first commit would be good. 
Also please add STORM-1344 as prefix to commit message since we don't get 
any information from the commit message.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1662: [STORM-1344] Remove sql command from storm-jdbc build.

2016-08-31 Thread HeartSaVioR

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1662
  
OK. Checked changeset doesn't affect the build result. Nice work. +1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1661: [STORM-2071] Add in retry after rebalance in unit test

2016-08-31 Thread HeartSaVioR

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1661
  
From JIRA comment @revans2 said he fixed this issue from Supervisor V2 
patch.

@revans2 Since we saw builds failing (high chance from my dev.) how about 
addressing build issue before than Supervisor V2?

@ppoulosk @revans2 
Please have a cross look at each patch. We need to choose one which will be 
merged.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[CANCELED] [VOTE] Release Apache Storm 0.10.2 (RC1)

2016-08-31 Thread P. Taylor Goetz

Canceling this vote so the fix for STORM-2042 can be included in the next 
release.

-Taylor


> On Aug 26, 2016, at 12:00 PM, Harsha Chintalapani  wrote:
> 
> We need to get this patch in for 0.10 release
> https://github.com/apache/storm/pull/1645
> 
> Thanks,
> Harsha
> 
> On Fri, Aug 26, 2016 at 7:28 AM Bobby Evans 
> wrote:
> 
>> +1 - Bobby
>> 
>>On Friday, August 26, 2016 9:03 AM, Jungtaek Lim 
>> wrote:
>> 
>> 
>> +1 (binding)
>> 
>> - testing with source distribution : OK
>>  - verify signature (zip, tar.gz): OK
>>  - uncompress : OK
>>  - building from source dist : OK
>>  - how to build: running `mvn clean install` on unzipped source dist.
>> 
>> - testing with binary distribution (one machine) : OK
>>  - verify signature (zip, tar.gz): OK
>> - launch daemons : OK
>> - run RollingTopWords (local) : OK
>> - run RollingTopWords (remote) : OK
>>  - activate / deactivate / rebalance / kill : OK
>>  - logviewer (worker) : OK
>>  - run WordCountTopology (multilang, local) : OK
>>  - run WordCountTopology (multilang, remote) : OK
>> 
>> Btw, would we want to include STORM-2042 to 0.10.2?
>> I agree this is definitely not a blocker, but we have been releasing 0.10.x
>> line slowly so might be worth to consider.
>> As I voted +1 I'm also OK to not include it.
>> 
>> Thanks,
>> Jungtaek Lim (HeartSaVioR)
>> 
>> 2016년 8월 25일 (목) 오전 5:22, P. Taylor Goetz 님이 작성:
>> 
>>> This is a call to vote on releasing Apache Storm 0.10.2 (rc1)
>>> 
>>> Full list of changes in this release:
>>> 
>>> 
>>> 
>> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=CHANGELOG.md;hb=3c24dce8b35527b409cae68a1ebc0e25e5a0b03f
>>> 
>>> The tag/commit to be voted upon is v0.10.2:
>>> 
>>> 
>>> 
>> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=2c356b9c1085d35cfc6d0869fcca1d570910053e;hb=3c24dce8b35527b409cae68a1ebc0e25e5a0b03f
>>> 
>>> The source archive being voted upon can be found here:
>>> 
>>> 
>>> 
>> https://dist.apache.org/repos/dist/dev/storm/apache-storm-0.10.2-rc1/apache-storm-0.10.2-src.tar.gz
>>> 
>>> Other release files, signatures and digests can be found here:
>>> 
>>> https://dist.apache.org/repos/dist/dev/storm/apache-storm-0.10.2-rc1/
>>> 
>>> The release artifacts are signed with the following key:
>>> 
>>> 
>>> 
>> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
>>> 
>>> The Nexus staging repository for this release is:
>>> 
>>> https://repository.apache.org/content/repositories/orgapachestorm-1040
>>> 
>>> Please vote on releasing this package as Apache Storm 0.10.2.
>>> 
>>> When voting, please list the actions taken to verify the release.
>>> 
>>> This vote will be open for at least 72 hours.
>>> 
>>> [ ] +1 Release this package as Apache Storm 0.10.2
>>> [ ]  0 No opinion
>>> [ ] -1 Do not release this package because...
>>> 
>>> Thanks to everyone who contributed to this release.
>>> 
>>> -Taylor
>>> 
>> 
>> 



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: [VOTE] Release Apache Storm 0.9.7 (RC1)

2016-08-31 Thread P. Taylor Goetz

This vote is now closed and passes with 3 binding +1 votes, and no +0 or -1 
votes.

Vote tally:

+1:

P. Taylor Goetz
Jungtaek Lim
Bobby Evans

-Taylor

> On Aug 15, 2016, at 4:10 PM, P. Taylor Goetz  wrote:
> 
> This is a call to vote on releasing Apache Storm 0.9.7 (rc1).
> 
> This release candidate addresses a critical issue with Storm’s multi-lang 
> component where an improperly formed multi-lang spout message would cause the 
> spout to hang indefinitely.
> 
> Full list of changes in this release:
> 
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=CHANGELOG.md;hb=b75d9b2b6dbfd0fda0b114feafec1384bbfa30aa
> 
> The tag/commit to be voted upon is v0.9.7:
> 
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=3d7c7be37ecc1dcc25223fde670d8185a6afbf00;hb=b75d9b2b6dbfd0fda0b114feafec1384bbfa30aa
> 
> The source archive being voted upon can be found here:
> 
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-0.9.7-rc1//apache-storm-0.9.7-src.tar.gz
> 
> Other release files, signatures and digests can be found here:
> 
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-0.9.7-rc1/
> 
> The release artifacts are signed with the following key:
> 
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> 
> The Nexus staging repository for this release is:
> 
> https://repository.apache.org/content/repositories/orgapachestorm-1039
> 
> Please vote on releasing this package as Apache Storm 0.9.7.
> 
> When voting, please list the actions taken to verify the release.
> 
> This vote will be open for at least 72 hours.
> 
> [ ] +1 Release this package as Apache Storm 0.9.7
> [ ]  0 No opinion
> [ ] -1 Do not release this package because...
> 
> Thanks to everyone who contributed to this release.
> 
> -Taylor



signature.asc
Description: Message signed with OpenPGP using GPGMail

[GitHub] storm pull request #1665: STORM-2074: fix storm-kafka-monitor NPE bug

2016-08-31 Thread priyank5485

Github user priyank5485 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1665#discussion_r77034491
  
--- Diff: 
external/storm-kafka-monitor/src/main/java/org/apache/storm/kafka/monitor/KafkaOffsetLagUtil.java
 ---
@@ -373,16 +377,20 @@ private static Options buildOptions () {
 curatorFramework.start();
 String partitionPrefix = "partition_";
 String zkPath = oldKafkaSpoutOffsetQuery.getZkPath();
-if (!zkPath.endsWith("/")) {
-zkPath += "/";
+ if (zkPath.endsWith("/")) {
--- End diff --

Is there a reason for removing trailing / here and then appending in the 
loop below?  The loop already has a check for 
curatorFramework.checkExists().forPath(path). May be we can just print and exit 
therE? Also, can you call the printUsageAndExit method here to be consistent? 
Or is there a reason you could not use it?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1665: STORM-2074: fix storm-kafka-monitor NPE bug

2016-08-31 Thread priyank5485

Github user priyank5485 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1665#discussion_r77033989
  
--- Diff: 
external/storm-kafka-monitor/src/main/java/org/apache/storm/kafka/monitor/KafkaOffsetLagUtil.java
 ---
@@ -89,6 +89,10 @@ public static void main (String args[]) {
 printUsageAndExit(options, OPTION_ZK_SERVERS_LONG + " 
and " + OPTION_ZK_COMMITTED_NODE_LONG + " are required  with " +
 OPTION_OLD_CONSUMER_LONG);
 }
+String zkNode = 
commandLine.getOptionValue(OPTION_ZK_COMMITTED_NODE_LONG);
--- End diff --

While adding OPTION_ZK_NODE_COMMITTED_NODE_LONG to options it is added 
using the method options.addOption(String opt, String longOpt, boolean hasArg, 
String description) The third argument is set to true in the call which 
specifies that the argument must have a value. Does not that take care of null 
or empty value? Do we need to have this explicit check here?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1661: [STORM-2071] Add in retry after rebalance in unit test

2016-08-31 Thread abellina

Github user abellina commented on the issue:

https://github.com/apache/storm/pull/1661
  
+1 thanks @ppoulosk 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1565: STORM-1970: external project examples refator

2016-08-31 Thread abhishekagarwal87

Github user abhishekagarwal87 commented on the issue:

https://github.com/apache/storm/pull/1565
  
+1 Post the conflicts merge. 
@HeartSaVioR - can you also take a look?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1665: STORM-2074: fix storm-kafka-monitor NPE bug

2016-08-31 Thread vesense

Github user vesense commented on the issue:

https://github.com/apache/storm/pull/1665
  
this should be merged into 1.x-branch.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1642: STORM-2018: Supervisor V2.

2016-08-31 Thread revans2

Github user revans2 commented on the issue:

https://github.com/apache/storm/pull/1642
  
@HeartSaVioR take your time.  I want to be sure that I have plenty of 
eyeballs looking at this.  We are doing this mostly because we started to run 
into a lot of race conditions in the supervisor.  We would try to fix one, and 
another would be exposed.  This makes it so that interacting with the container 
is only ever done on a single thread so we can reason about the state of the 
container and avoid those races.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1665: STORM-2074: fix storm-kafka-monitor NPE bug

2016-08-31 Thread vesense

GitHub user vesense opened a pull request:

https://github.com/apache/storm/pull/1665

STORM-2074: fix storm-kafka-monitor NPE bug

https://issues.apache.org/jira/browse/STORM-2074

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/vesense/storm STORM-2074

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1665.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1665


commit e61e7738328913ec429742cd2b3cb22a39eb263f
Author: Xin Wang 
Date:   2016-08-31T15:11:28Z

STORM-2074: fix storm-kafka-monitor NPE bug




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1662: [STORM-1344] Add check for table

2016-08-31 Thread ppoulosk

Github user ppoulosk commented on the issue:

https://github.com/apache/storm/pull/1662
  
Fixed.  Re-ran tests.  Thanks.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1642: STORM-2018: Supervisor V2.

2016-08-31 Thread HeartSaVioR

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1642
  
I'm still reviewing this. Nice work to refactor to introduce greater 
readability. Will leave comment here once I'm done with first pass.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1662: [STORM-1344] Add check for table

2016-08-31 Thread ppoulosk

Github user ppoulosk commented on the issue:

https://github.com/apache/storm/pull/1662
  
@HeartSaVioR, @abellina, will fix as you suggest.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1661: [STORM-2071] Add in retry after rebalance in unit test

2016-08-31 Thread ppoulosk

Github user ppoulosk commented on the issue:

https://github.com/apache/storm/pull/1661
  
Coded fix, retested, and pushed.  Thanks!!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1642: STORM-2018: Supervisor V2.

2016-08-31 Thread HeartSaVioR

Github user HeartSaVioR commented on a diff in the pull request:

https://github.com/apache/storm/pull/1642#discussion_r77000695
  
--- Diff: 
storm-core/src/jvm/org/apache/storm/daemon/supervisor/BasicContainer.java ---
@@ -0,0 +1,628 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.daemon.supervisor;
+
+import java.io.File;
+import java.io.FilenameFilter;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import org.apache.commons.lang.StringUtils;
+import org.apache.storm.Config;
+import org.apache.storm.container.ResourceIsolationInterface;
+import org.apache.storm.generated.LocalAssignment;
+import org.apache.storm.generated.ProfileAction;
+import org.apache.storm.generated.ProfileRequest;
+import org.apache.storm.generated.StormTopology;
+import org.apache.storm.generated.WorkerResources;
+import org.apache.storm.utils.ConfigUtils;
+import org.apache.storm.utils.LocalState;
+import org.apache.storm.utils.Utils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Joiner;
+import com.google.common.collect.Lists;
+
+/**
+ * A container that runs processes on the local box.
+ */
+public class BasicContainer extends Container {
+private static final Logger LOG = 
LoggerFactory.getLogger(BasicContainer.class);
+private static final FilenameFilter jarFilter = new FilenameFilter() {
+@Override
+public boolean accept(File dir, String name) {
+return name.endsWith(".jar");
+}
+};
+private static final Joiner CPJ = 
+Joiner.on(Utils.CLASS_PATH_SEPARATOR).skipNulls();
+
+protected final LocalState _localState;
+protected final String _profileCmd;
+protected volatile boolean _exitedEarly = false;
+
+private class ProcessExitCallback implements ExitCodeCallback {
+private final String _logPrefix;
+
+public ProcessExitCallback(String logPrefix) {
+_logPrefix = logPrefix;
+}
+
+@Override
+public void call(int exitCode) {
+LOG.info("{} exited with code: {}", _logPrefix, exitCode);
+_exitedEarly = true;
+}
+}
+
+//For testing purposes
+public BasicContainer(AdvancedFSOps ops, int port, LocalAssignment 
assignment,
+Map conf, Map topoConf, String 
supervisorId, 
+ResourceIsolationInterface resourceIsolationManager, 
LocalState localState,
+String profileCmd) throws IOException {
+super(ops, port, assignment, conf, topoConf, supervisorId, 
resourceIsolationManager);
+_localState = localState;
+_profileCmd = profileCmd;
+}
+
+public BasicContainer(int port, LocalAssignment assignment, 
Map conf, String supervisorId,
+LocalState localState, ResourceIsolationInterface 
resourceIsolationManager, boolean recover)
+throws IOException {
+super(port, assignment, conf, supervisorId, 
resourceIsolationManager);
+_localState = localState;
+
+if (recover) {
+synchronized (localState) {
+String wid = null;
+Map workerToPort = 
localState.getApprovedWorkers();
+for (Map.Entry entry : 
workerToPort.entrySet()) {
+if (port == entry.getValue().intValue()) {
+wid = entry.getKey();
+}
+}
+if (wid == null) {
+throw new ContainerRecoveryException("Could not find 
worker id for " + port + " " + assignment);
+

[GitHub] storm pull request #1661: [STORM-2071] Add in retry after rebalance in unit ...

2016-08-31 Thread ppoulosk

Github user ppoulosk commented on a diff in the pull request:

https://github.com/apache/storm/pull/1661#discussion_r76999135
  
--- Diff: storm-core/test/clj/org/apache/storm/nimbus_test.clj ---
@@ -1218,8 +1218,16 @@
   (submit-local-topology nimbus "t1" {} topology)
   ;; Instead of sleeping until topology is scheduled, 
rebalance topology so mk-assignments is called.
   (.rebalance nimbus "t1" (doto (RebalanceOptions.) 
(.set_wait_secs 0)))
-  (Thread/sleep 1000)
-  (.deactivate nimbus "t1")
+  ;; One second doesn't work on all systems.  Try it five times
+  (loop [n 5]
+(if-let [result (try
+  [(Thread/sleep 1000)
--- End diff --

Agreed.  Will fix.  Thanks, @abellina.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1642: STORM-2018: Supervisor V2.

2016-08-31 Thread revans2

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/1642#discussion_r76997296
  
--- Diff: 
storm-core/src/jvm/org/apache/storm/daemon/supervisor/AdvancedFSOps.java ---
@@ -0,0 +1,319 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.daemon.supervisor;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileWriter;
+import java.io.IOException;
+import java.io.Writer;
+import java.nio.file.FileSystems;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.StandardCopyOption;
+import java.nio.file.attribute.PosixFilePermission;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.commons.io.FileUtils;
+import org.apache.storm.Config;
+import org.apache.storm.utils.Utils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class AdvancedFSOps {
+private static final Logger LOG = 
LoggerFactory.getLogger(AdvancedFSOps.class);
+
+/**
+ * Factory to create a new AdvancedFSOps
+ * @param conf the configuration of the process
+ * @return the appropriate instance of the class for this config and 
environment.
+ */
+public static AdvancedFSOps make(Map conf) {
+if (Utils.isOnWindows()) {
+return new AdvancedWindowsFSOps(conf);
+}
+if 
(Utils.getBoolean(conf.get(Config.SUPERVISOR_RUN_WORKER_AS_USER), false)) {
+return new AdvancedRunAsUserFSOps(conf);
+}
+return new AdvancedFSOps();
+}
+
+private static class AdvancedRunAsUserFSOps extends AdvancedFSOps {
+private final Map _conf;
+
+public AdvancedRunAsUserFSOps(Map conf) {
+if (Utils.isOnWindows()) {
+throw new UnsupportedOperationException("ERROR: Windows 
doesn't support running workers as different users yet");
+}
+_conf = conf;
+}
+
+@Override
+public void setupBlobPermissions(File path, String user) throws 
IOException {
+String logPrefix = "setup blob permissions for " + path;
+SupervisorUtils.processLauncherAndWait(_conf, user, 
Arrays.asList("blob", path.toString()), null, logPrefix);
+}
+
+@Override
+public void deleteIfExists(File path, String user, String 
logPrefix) throws IOException {
+String absolutePath = path.getAbsolutePath();
+LOG.info("Deleting path {}", absolutePath);
+if (user == null) {
+user = Files.getOwner(path.toPath()).getName();
+}
+List commands = new ArrayList<>();
+commands.add("rmr");
+commands.add(absolutePath);
+SupervisorUtils.processLauncherAndWait(_conf, user, commands, 
null, logPrefix);
+if (Utils.checkFileExists(absolutePath)) {
+throw new RuntimeException(path + " was not deleted.");
+}
+}
+
+@Override
+public void setupStormCodeDir(Map topologyConf, 
File path) throws IOException {
+SupervisorUtils.setupStormCodeDir(_conf, topologyConf, 
path.getCanonicalPath());
+}
+}
+
+/**
+ * Operations that need to override the default ones when running on 
Windows
+ *
+ */
+private static class AdvancedWindowsFSOps extends AdvancedFSOps {
+
+public AdvancedWindowsFSOps(Map conf) {
+if 
(Utils.getBoolean(conf.get(Config.SUPERVISOR_RUN_WORKER_AS_USER), false)) {
+throw new RuntimeException("ERROR: Windows doesn't support 
running workers as different users yet

[jira] [Created] (STORM-2074) NPE bug in storm-kafka-monitor

2016-08-31 Thread Xin Wang (JIRA)

Xin Wang created STORM-2074:
---

 Summary: NPE bug in storm-kafka-monitor
 Key: STORM-2074
 URL: https://issues.apache.org/jira/browse/STORM-2074
 Project: Apache Storm
  Issue Type: Bug
Reporter: Xin Wang
Assignee: Xin Wang
Priority: Minor


storm-kafka-monitor will throw an NPE when __--zk-node__ value is invalid. this 
is not friendly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request #1642: STORM-2018: Supervisor V2.

2016-08-31 Thread HeartSaVioR

Github user HeartSaVioR commented on a diff in the pull request:

https://github.com/apache/storm/pull/1642#discussion_r76991771
  
--- Diff: 
storm-core/src/jvm/org/apache/storm/daemon/supervisor/AdvancedFSOps.java ---
@@ -0,0 +1,319 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.storm.daemon.supervisor;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileWriter;
+import java.io.IOException;
+import java.io.Writer;
+import java.nio.file.FileSystems;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.StandardCopyOption;
+import java.nio.file.attribute.PosixFilePermission;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.commons.io.FileUtils;
+import org.apache.storm.Config;
+import org.apache.storm.utils.Utils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class AdvancedFSOps {
+private static final Logger LOG = 
LoggerFactory.getLogger(AdvancedFSOps.class);
+
+/**
+ * Factory to create a new AdvancedFSOps
+ * @param conf the configuration of the process
+ * @return the appropriate instance of the class for this config and 
environment.
+ */
+public static AdvancedFSOps make(Map conf) {
+if (Utils.isOnWindows()) {
+return new AdvancedWindowsFSOps(conf);
+}
+if 
(Utils.getBoolean(conf.get(Config.SUPERVISOR_RUN_WORKER_AS_USER), false)) {
+return new AdvancedRunAsUserFSOps(conf);
+}
+return new AdvancedFSOps();
+}
+
+private static class AdvancedRunAsUserFSOps extends AdvancedFSOps {
+private final Map _conf;
+
+public AdvancedRunAsUserFSOps(Map conf) {
+if (Utils.isOnWindows()) {
+throw new UnsupportedOperationException("ERROR: Windows 
doesn't support running workers as different users yet");
+}
+_conf = conf;
+}
+
+@Override
+public void setupBlobPermissions(File path, String user) throws 
IOException {
+String logPrefix = "setup blob permissions for " + path;
+SupervisorUtils.processLauncherAndWait(_conf, user, 
Arrays.asList("blob", path.toString()), null, logPrefix);
+}
+
+@Override
+public void deleteIfExists(File path, String user, String 
logPrefix) throws IOException {
+String absolutePath = path.getAbsolutePath();
+LOG.info("Deleting path {}", absolutePath);
+if (user == null) {
+user = Files.getOwner(path.toPath()).getName();
+}
+List commands = new ArrayList<>();
+commands.add("rmr");
+commands.add(absolutePath);
+SupervisorUtils.processLauncherAndWait(_conf, user, commands, 
null, logPrefix);
+if (Utils.checkFileExists(absolutePath)) {
+throw new RuntimeException(path + " was not deleted.");
+}
+}
+
+@Override
+public void setupStormCodeDir(Map topologyConf, 
File path) throws IOException {
+SupervisorUtils.setupStormCodeDir(_conf, topologyConf, 
path.getCanonicalPath());
+}
+}
+
+/**
+ * Operations that need to override the default ones when running on 
Windows
+ *
+ */
+private static class AdvancedWindowsFSOps extends AdvancedFSOps {
+
+public AdvancedWindowsFSOps(Map conf) {
+if 
(Utils.getBoolean(conf.get(Config.SUPERVISOR_RUN_WORKER_AS_USER), false)) {
+throw new RuntimeException("ERROR: Windows doesn't support 
running workers as different users

[GitHub] storm issue #1660: STORM-2070 Fix sigar native binary download link

2016-08-31 Thread revans2

Github user revans2 commented on the issue:

https://github.com/apache/storm/pull/1660
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1638: minor: fix tooltip style

2016-08-31 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1638


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-2071) nimbus-test test-leadership failing with Exception

2016-08-31 Thread Robert Joseph Evans (JIRA)


[ 
https://issues.apache.org/jira/browse/STORM-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15452246#comment-15452246
 ] 

Robert Joseph Evans commented on STORM-2071:


I replaced the sleep with

{code}
(defn wait-for-status [nimbus name status]
  (while-timeout 5000
(let [topo-summary (first (filter (fn [topo] (= name (.get_name topo))) 
(.get_topologies (.getClusterInfo nimbus
  topo-status (if topo-summary (.get_status topo-summary) 
"NOT-RUNNING")]
  (log-message "WAITING FOR "name" TO BE " status " CURRENT " topo-status)
  (not= topo-status status))
(Thread/sleep 100)))
...
(wait-for-status nimbus "t1" "ACTIVE") 
{code}

If someone wants to put this in a separate pull request or have me break it off 
I am happy to do it.

> nimbus-test test-leadership failing with Exception
> --
>
> Key: STORM-2071
> URL: https://issues.apache.org/jira/browse/STORM-2071
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 2.0.0, 1.0.1
> Environment: Mac os X
>Reporter: Paul Poulosky
>Assignee: Robert Joseph Evans
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> When running unit tests on my Mac, I get repeated failures in test-leadership.
> 
> 73752 [main] INFO  o.a.s.l.ThriftAccessLogger - Request ID: 1 access from: 
> null principal: null operation: deactivate
> ]]>
> Uncaught 
> exception, not in assertion.
> expected: nil
>   actual: java.lang.RuntimeException: No transition for event: :inactivate, 
> status: {:type :rebalancing} storm-id: t1-1-1472598899
>  at org.apache.storm.daemon.nimbus$transition_BANG_$get_event__4879.invoke 
> (nimbus.clj:365)
> org.apache.storm.daemon.nimbus$transition_BANG_.invoke (nimbus.clj:373)
> clojure.lang.AFn.applyToHelper (AFn.java:165)
> clojure.lang.AFn.applyTo (AFn.java:144)
> clojure.core$apply.invoke (core.clj:636)
> org.apache.storm.daemon.nimbus$transition_name_BANG_.doInvoke 
> (nimbus.clj:391)
> clojure.lang.RestFn.invoke (RestFn.java:467)
> org.apache.storm.daemon.nimbus$mk_reified_nimbus$reify__5850.deactivate 
> (nimbus.clj:1773)
> sun.reflect.NativeMethodAccessorImpl.invoke0 
> (NativeMethodAccessorImpl.java:-2)
> sun.reflect.NativeMethodAccessorImpl.invoke 
> (NativeMethodAccessorImpl.java:62)
> sun.reflect.DelegatingMethodAccessorImpl.invoke 
> (DelegatingMethodAccessorImpl.java:43)
> java.lang.reflect.Method.invoke (Method.java:497)
> clojure.lang.Reflector.invokeMatchingMethod (Reflector.java:93)
> clojure.lang.Reflector.invokeInstanceMethod (Reflector.java:28)
> org.apache.storm.nimbus_test$fn__1203$fn__1209.invoke 
> (nimbus_test.clj:1222)
> org.apache.storm.nimbus_test/fn (nimbus_test.clj:1210)
> clojure.test$test_var$fn__7670.invoke (test.clj:704)
> clojure.test$test_var.invoke (test.clj:704)
> clojure.test$test_vars$fn__7692$fn__7697.invoke (test.clj:722)
> clojure.test$default_fixture.invoke (test.clj:674)
> clojure.test$test_vars$fn__7692.invoke (test.clj:722)
> clojure.test$default_fixture.invoke (test.clj:674)
> clojure.test$test_vars.invoke (test.clj:718)
> clojure.test$test_all_vars.invoke (test.clj:728)
> (test.clj:747)
> clojure.core$map$fn__4553.invoke (core.clj:2624)
> clojure.lang.LazySeq.sval (LazySeq.java:40)
> clojure.lang.LazySeq.seq (LazySeq.java:49)
> clojure.lang.Cons.next (Cons.java:39)
> clojure.lang.RT.boundedLength (RT.java:1735)
> clojure.lang.RestFn.applyTo (RestFn.java:130)
> clojure.core$apply.invoke (core.clj:632)
> clojure.test$run_tests.doInvoke (test.clj:762)
> clojure.lang.RestFn.invoke (RestFn.java:408)
> 
> org.apache.storm.testrunner$eval8358$iter__8359__8363$fn__8364$fn__8365$fn__8366.invoke
>  (test_runner.clj:107)
> 
> org.apache.storm.testrunner$eval8358$iter__8359__8363$fn__8364$fn__8365.invoke
>  (test_runner.clj:53)
> org.apache.storm.testrunner$eval8358$iter__8359__8363$fn__8364.invoke 
> (test_runner.clj:52)
> clojure.lang.LazySeq.sval (LazySeq.java:40)
> clojure.lang.LazySeq.seq (LazySeq.java:49)
> clojure.lang.RT.seq (RT.java:507)
> clojure.core/seq (core.clj:137)
> clojure.core$dorun.invoke (core.clj:3009)
> org.apache.storm.testrunner$eval8358.invoke (test_runner.clj:52)
> clojure.lang.Compiler.eval (Compiler.java:6782)
> clojure.lang.Compiler.load (Compiler.java:7227)
> clojure.lang.Compiler.loadFile (Compiler.java:7165)
> clojure.main$load_script.invoke (main.clj:275)
> clojure.main$script_opt.invoke (main.clj:337)
> clojure.main$main.doInvoke (main.clj:421)
> clojure.lang.RestFn.invoke (RestFn.java:421)
> clojure.lang.Var.invoke (Var.java:383)
> clojure.lang.AFn.applyToHelper (AFn

[GitHub] storm issue #1638: minor: fix tooltip style

2016-08-31 Thread HeartSaVioR

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1638
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1565: STORM-1970: external project examples refator

2016-08-31 Thread vesense

Github user vesense commented on the issue:

https://github.com/apache/storm/pull/1565
  
@abhishekagarwal87 is this should be continue?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-2071) nimbus-test test-leadership failing with Exception

2016-08-31 Thread Robert Joseph Evans (JIRA)


[ 
https://issues.apache.org/jira/browse/STORM-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15452239#comment-15452239
 ] 

Robert Joseph Evans commented on STORM-2071:


I found and fixed this in my supervisor v2 patch.  I fixed a few bugs because I 
was not sure if they were caused by the supervisor changes or not.

https://github.com/apache/storm/pull/1642

> nimbus-test test-leadership failing with Exception
> --
>
> Key: STORM-2071
> URL: https://issues.apache.org/jira/browse/STORM-2071
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 2.0.0, 1.0.1
> Environment: Mac os X
>Reporter: Paul Poulosky
>Assignee: Robert Joseph Evans
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> When running unit tests on my Mac, I get repeated failures in test-leadership.
> 
> 73752 [main] INFO  o.a.s.l.ThriftAccessLogger - Request ID: 1 access from: 
> null principal: null operation: deactivate
> ]]>
> Uncaught 
> exception, not in assertion.
> expected: nil
>   actual: java.lang.RuntimeException: No transition for event: :inactivate, 
> status: {:type :rebalancing} storm-id: t1-1-1472598899
>  at org.apache.storm.daemon.nimbus$transition_BANG_$get_event__4879.invoke 
> (nimbus.clj:365)
> org.apache.storm.daemon.nimbus$transition_BANG_.invoke (nimbus.clj:373)
> clojure.lang.AFn.applyToHelper (AFn.java:165)
> clojure.lang.AFn.applyTo (AFn.java:144)
> clojure.core$apply.invoke (core.clj:636)
> org.apache.storm.daemon.nimbus$transition_name_BANG_.doInvoke 
> (nimbus.clj:391)
> clojure.lang.RestFn.invoke (RestFn.java:467)
> org.apache.storm.daemon.nimbus$mk_reified_nimbus$reify__5850.deactivate 
> (nimbus.clj:1773)
> sun.reflect.NativeMethodAccessorImpl.invoke0 
> (NativeMethodAccessorImpl.java:-2)
> sun.reflect.NativeMethodAccessorImpl.invoke 
> (NativeMethodAccessorImpl.java:62)
> sun.reflect.DelegatingMethodAccessorImpl.invoke 
> (DelegatingMethodAccessorImpl.java:43)
> java.lang.reflect.Method.invoke (Method.java:497)
> clojure.lang.Reflector.invokeMatchingMethod (Reflector.java:93)
> clojure.lang.Reflector.invokeInstanceMethod (Reflector.java:28)
> org.apache.storm.nimbus_test$fn__1203$fn__1209.invoke 
> (nimbus_test.clj:1222)
> org.apache.storm.nimbus_test/fn (nimbus_test.clj:1210)
> clojure.test$test_var$fn__7670.invoke (test.clj:704)
> clojure.test$test_var.invoke (test.clj:704)
> clojure.test$test_vars$fn__7692$fn__7697.invoke (test.clj:722)
> clojure.test$default_fixture.invoke (test.clj:674)
> clojure.test$test_vars$fn__7692.invoke (test.clj:722)
> clojure.test$default_fixture.invoke (test.clj:674)
> clojure.test$test_vars.invoke (test.clj:718)
> clojure.test$test_all_vars.invoke (test.clj:728)
> (test.clj:747)
> clojure.core$map$fn__4553.invoke (core.clj:2624)
> clojure.lang.LazySeq.sval (LazySeq.java:40)
> clojure.lang.LazySeq.seq (LazySeq.java:49)
> clojure.lang.Cons.next (Cons.java:39)
> clojure.lang.RT.boundedLength (RT.java:1735)
> clojure.lang.RestFn.applyTo (RestFn.java:130)
> clojure.core$apply.invoke (core.clj:632)
> clojure.test$run_tests.doInvoke (test.clj:762)
> clojure.lang.RestFn.invoke (RestFn.java:408)
> 
> org.apache.storm.testrunner$eval8358$iter__8359__8363$fn__8364$fn__8365$fn__8366.invoke
>  (test_runner.clj:107)
> 
> org.apache.storm.testrunner$eval8358$iter__8359__8363$fn__8364$fn__8365.invoke
>  (test_runner.clj:53)
> org.apache.storm.testrunner$eval8358$iter__8359__8363$fn__8364.invoke 
> (test_runner.clj:52)
> clojure.lang.LazySeq.sval (LazySeq.java:40)
> clojure.lang.LazySeq.seq (LazySeq.java:49)
> clojure.lang.RT.seq (RT.java:507)
> clojure.core/seq (core.clj:137)
> clojure.core$dorun.invoke (core.clj:3009)
> org.apache.storm.testrunner$eval8358.invoke (test_runner.clj:52)
> clojure.lang.Compiler.eval (Compiler.java:6782)
> clojure.lang.Compiler.load (Compiler.java:7227)
> clojure.lang.Compiler.loadFile (Compiler.java:7165)
> clojure.main$load_script.invoke (main.clj:275)
> clojure.main$script_opt.invoke (main.clj:337)
> clojure.main$main.doInvoke (main.clj:421)
> clojure.lang.RestFn.invoke (RestFn.java:421)
> clojure.lang.Var.invoke (Var.java:383)
> clojure.lang.AFn.applyToHelper (AFn.java:156)
> clojure.lang.Var.applyTo (Var.java:700)
> clojure.main.main (main.java:37)
> 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm issue #1638: minor: fix tooltip style

2016-08-31 Thread abellina

Github user abellina commented on the issue:

https://github.com/apache/storm/pull/1638
  
ð 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (STORM-2071) nimbus-test test-leadership failing with Exception

2016-08-31 Thread Robert Joseph Evans (JIRA)


 [ 
https://issues.apache.org/jira/browse/STORM-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Joseph Evans reassigned STORM-2071:
--

Assignee: Robert Joseph Evans

> nimbus-test test-leadership failing with Exception
> --
>
> Key: STORM-2071
> URL: https://issues.apache.org/jira/browse/STORM-2071
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 2.0.0, 1.0.1
> Environment: Mac os X
>Reporter: Paul Poulosky
>Assignee: Robert Joseph Evans
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> When running unit tests on my Mac, I get repeated failures in test-leadership.
> 
> 73752 [main] INFO  o.a.s.l.ThriftAccessLogger - Request ID: 1 access from: 
> null principal: null operation: deactivate
> ]]>
> Uncaught 
> exception, not in assertion.
> expected: nil
>   actual: java.lang.RuntimeException: No transition for event: :inactivate, 
> status: {:type :rebalancing} storm-id: t1-1-1472598899
>  at org.apache.storm.daemon.nimbus$transition_BANG_$get_event__4879.invoke 
> (nimbus.clj:365)
> org.apache.storm.daemon.nimbus$transition_BANG_.invoke (nimbus.clj:373)
> clojure.lang.AFn.applyToHelper (AFn.java:165)
> clojure.lang.AFn.applyTo (AFn.java:144)
> clojure.core$apply.invoke (core.clj:636)
> org.apache.storm.daemon.nimbus$transition_name_BANG_.doInvoke 
> (nimbus.clj:391)
> clojure.lang.RestFn.invoke (RestFn.java:467)
> org.apache.storm.daemon.nimbus$mk_reified_nimbus$reify__5850.deactivate 
> (nimbus.clj:1773)
> sun.reflect.NativeMethodAccessorImpl.invoke0 
> (NativeMethodAccessorImpl.java:-2)
> sun.reflect.NativeMethodAccessorImpl.invoke 
> (NativeMethodAccessorImpl.java:62)
> sun.reflect.DelegatingMethodAccessorImpl.invoke 
> (DelegatingMethodAccessorImpl.java:43)
> java.lang.reflect.Method.invoke (Method.java:497)
> clojure.lang.Reflector.invokeMatchingMethod (Reflector.java:93)
> clojure.lang.Reflector.invokeInstanceMethod (Reflector.java:28)
> org.apache.storm.nimbus_test$fn__1203$fn__1209.invoke 
> (nimbus_test.clj:1222)
> org.apache.storm.nimbus_test/fn (nimbus_test.clj:1210)
> clojure.test$test_var$fn__7670.invoke (test.clj:704)
> clojure.test$test_var.invoke (test.clj:704)
> clojure.test$test_vars$fn__7692$fn__7697.invoke (test.clj:722)
> clojure.test$default_fixture.invoke (test.clj:674)
> clojure.test$test_vars$fn__7692.invoke (test.clj:722)
> clojure.test$default_fixture.invoke (test.clj:674)
> clojure.test$test_vars.invoke (test.clj:718)
> clojure.test$test_all_vars.invoke (test.clj:728)
> (test.clj:747)
> clojure.core$map$fn__4553.invoke (core.clj:2624)
> clojure.lang.LazySeq.sval (LazySeq.java:40)
> clojure.lang.LazySeq.seq (LazySeq.java:49)
> clojure.lang.Cons.next (Cons.java:39)
> clojure.lang.RT.boundedLength (RT.java:1735)
> clojure.lang.RestFn.applyTo (RestFn.java:130)
> clojure.core$apply.invoke (core.clj:632)
> clojure.test$run_tests.doInvoke (test.clj:762)
> clojure.lang.RestFn.invoke (RestFn.java:408)
> 
> org.apache.storm.testrunner$eval8358$iter__8359__8363$fn__8364$fn__8365$fn__8366.invoke
>  (test_runner.clj:107)
> 
> org.apache.storm.testrunner$eval8358$iter__8359__8363$fn__8364$fn__8365.invoke
>  (test_runner.clj:53)
> org.apache.storm.testrunner$eval8358$iter__8359__8363$fn__8364.invoke 
> (test_runner.clj:52)
> clojure.lang.LazySeq.sval (LazySeq.java:40)
> clojure.lang.LazySeq.seq (LazySeq.java:49)
> clojure.lang.RT.seq (RT.java:507)
> clojure.core/seq (core.clj:137)
> clojure.core$dorun.invoke (core.clj:3009)
> org.apache.storm.testrunner$eval8358.invoke (test_runner.clj:52)
> clojure.lang.Compiler.eval (Compiler.java:6782)
> clojure.lang.Compiler.load (Compiler.java:7227)
> clojure.lang.Compiler.loadFile (Compiler.java:7165)
> clojure.main$load_script.invoke (main.clj:275)
> clojure.main$script_opt.invoke (main.clj:337)
> clojure.main$main.doInvoke (main.clj:421)
> clojure.lang.RestFn.invoke (RestFn.java:421)
> clojure.lang.Var.invoke (Var.java:383)
> clojure.lang.AFn.applyToHelper (AFn.java:156)
> clojure.lang.Var.applyTo (Var.java:700)
> clojure.main.main (main.java:37)
> 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm issue #1603: STORM-2012: Upgrade Kafka to 0.8.2.2

2016-08-31 Thread vesense

Github user vesense commented on the issue:

https://github.com/apache/storm/pull/1603
  
@darionyaphet two important fixes for snappy compression. please take a 
look the above link first, thanks.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1638: minor: fix tooltip style

2016-08-31 Thread vesense

Github user vesense commented on the issue:

https://github.com/apache/storm/pull/1638
  
hi @HeartSaVioR Done.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1344) storm-jdbc build error "object name already exists: USER_DETAILS in statement"

2016-08-31 Thread Robert Joseph Evans (JIRA)


[ 
https://issues.apache.org/jira/browse/STORM-1344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15452218#comment-15452218
 ] 

Robert Joseph Evans commented on STORM-1344:


Do you have a patch for that?  I can see it being at least a step in the right 
direction on fixing this.

> storm-jdbc build error "object name already exists: USER_DETAILS in statement"
> --
>
> Key: STORM-1344
> URL: https://issues.apache.org/jira/browse/STORM-1344
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-jdbc
>Affects Versions: 1.0.0
> Environment: os X, jdk7
>Reporter: Longda Feng
>Priority: Critical
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> ```
> [ERROR] Failed to execute goal org.codehaus.mojo:sql-maven-plugin:1.5:execute 
> (create-db) on project storm-jdbc: object name already exists: USER_DETAILS 
> in statement [ /** * Licensed to the Apache Software Foundation (ASF) under 
> one * or more contributor license agreements.  See the NOTICE file * 
> distributed with this work for additional information * regarding copyright 
> ownership.  The ASF licenses this file * to you under the Apache License, 
> Version 2.0 (the * "License"); you may not use this file except in compliance 
> * with the License.  You may obtain a copy of the License at * * 
> http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable 
> law or agreed to in writing, software * distributed under the License is 
> distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY 
> KIND, either express or implied. * See the License for the specific language 
> governing permissions and * limitations under the License. */
> [ERROR] create table user_details (id integer, user_name varchar(100), 
> create_date date)]
> [ERROR] -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute 
> goal org.codehaus.mojo:sql-maven-plugin:1.5:execute (create-db) on project 
> storm-jdbc: object name already exists: USER_DETAILS in statement [ /** * 
> Licensed to the Apache Software Foundation (ASF) under one * or more 
> contributor license agreements.  See the NOTICE file * distributed with this 
> work for additional information * regarding copyright ownership.  The ASF 
> licenses this file * to you under the Apache License, Version 2.0 (the * 
> "License"); you may not use this file except in compliance * with the 
> License.  You may obtain a copy of the License at * * 
> http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable 
> law or agreed to in writing, software * distributed under the License is 
> distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY 
> KIND, either express or implied. * See the License for the specific language 
> governing permissions and * limitations under the License. */
>  create table user_details (id integer, user_name varchar(100), create_date 
> date)]
>   at 
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
>   at 
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
>   at 
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
>   at 
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
>   at 
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
>   at 
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
>   at 
> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
>   at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
>   at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
>   at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
>   at org.apache.maven.cli.MavenCli.execute(MavenCli.java:862)
>   at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:286)
>   at org.apache.maven.cli.MavenCli.main(MavenCli.java:197)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:606)
>   at 
> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
>   at 
> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
>   at 
> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
>   at 
> org.codehaus.plexus.classw

Assignment of threads to tcp ports

2016-08-31 Thread Walid Aljoby

Hi everyone,
Is it easier to specify in which tcp port the tasks (threads) will be 
assigned?Any API can help to do that?
Thank You--Regards

[jira] [Resolved] (STORM-2039) Backpressure refactoring in worker and executor

2016-08-31 Thread Jungtaek Lim (JIRA)


 [ 
https://issues.apache.org/jira/browse/STORM-2039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jungtaek Lim resolved STORM-2039.
-
   Resolution: Fixed
Fix Version/s: 1.0.3
   1.1.0
   2.0.0

Thanks [~abellina], I merged this into master, 1.x, 1.0.x branches.

NOTE: This is not a bug, but not also a kind of new feature or improvement so I 
just merged this to 1.0.x. It also reduces divergence.

> Backpressure refactoring in worker and executor
> ---
>
> Key: STORM-2039
> URL: https://issues.apache.org/jira/browse/STORM-2039
> Project: Apache Storm
>  Issue Type: Story
>Reporter: Alessandro Bellina
>Assignee: Alessandro Bellina
>Priority: Minor
> Fix For: 2.0.0, 1.1.0, 1.0.3
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> * Use backpressure flags directly from disruptor queue instead of in the 
> disruptor-backpressure-handlers in worker and executor
> * Other minor refactoring (eliminate redundant function)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request #1627: STORM-2039: backpressure refactoring in worker and...

2016-08-31 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1627


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1632: Storm 2039 backpressure refactoring in worker and ...

2016-08-31 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1632


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1627: STORM-2039: backpressure refactoring in worker and execut...

2016-08-31 Thread HeartSaVioR

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1627
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1638: minor: fix tooltip style

2016-08-31 Thread HeartSaVioR

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1638
  
@vesense Nice finding. I'll merge once you rebase this to current master. 
Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1613: Fix emitting of tuples from bolts with Python 3

2016-08-31 Thread HeartSaVioR

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1613
  
@ThijsMergaert 
Sorry to comment too late.

I'm not sure how much python users use Storm python multi-lang directly, 
and what's more preferred version, 2 vs 3. If many users still rely on 2, it 
might be better to avoid creating list and find other workaround. (checking 
version and if-else?)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1614: STORM-1890: looks like too many calls were moved t...

2016-08-31 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1614


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1615: STORM-1890: looks like too many calls were moved t...

2016-08-31 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1615


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1612: STORM-1890: looks like too many calls were moved t...

2016-08-31 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1612


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1615: STORM-1890: looks like too many calls were moved to getSt...

2016-08-31 Thread HeartSaVioR

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1615
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1612: STORM-1890: looks like too many calls were moved to getSt...

2016-08-31 Thread HeartSaVioR

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1612
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1614: STORM-1890: looks like too many calls were moved to getSt...

2016-08-31 Thread HeartSaVioR

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1614
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (STORM-2042) Nimbus client connections not closed properly causing connection leaks

2016-08-31 Thread Jungtaek Lim (JIRA)


 [ 
https://issues.apache.org/jira/browse/STORM-2042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jungtaek Lim updated STORM-2042:

Fix Version/s: 0.10.3

> Nimbus client connections not closed properly causing connection leaks
> --
>
> Key: STORM-2042
> URL: https://issues.apache.org/jira/browse/STORM-2042
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Arun Mahadevan
>Assignee: Arun Mahadevan
> Fix For: 2.0.0, 1.1.0, 1.0.3, 0.10.3
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> The nimbus client connections are not closed properly causing connection 
> leaks. After the number of connections exceed nimbus.thrift.threads, a 
> RejectedExecutionException is thrown.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request #1645: [STORM-2042] Nimbus client connections not closed ...

2016-08-31 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1645


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Question regarding design of the Trident (operations)

2016-08-31 Thread Jungtaek Lim

FYI: Just worked to have one -
http://issues.apache.org/jira/browse/STORM-2072

This adds map and flatMap to have additional parameter: outputFields. If
outputFields are specified, it replaces origin fields with output fields.

- Jungtaek Lim (HeartSaVioR)


2016년 8월 31일 (수) 오전 10:27, Jungtaek Lim 님이 작성:

> FYI: I just found that Stream class has TODO comment above of class:
> *// TODO: need to be able to replace existing fields with the function
> fields (like Cascading Fields.REPLACE)*
> which is written at Aug. 2012, initial importing Trident source code.
>
> According to the TODO comment, it seems to be a just missing feature.
>
> Thanks,
> Jungtaek Lim (HeartSaVioR)
>
>
> 2016년 8월 31일 (수) 오전 7:29, Jungtaek Lim 님이 작성:
>
>> Hi Manu,
>>
>> I'm finding for 1:1 tuple transition, not aggregation. In other words,
>> I'm finding like 'V map(T, Function)' which seems fit for many use
>> case but Trident doesn't support this yet so I'd like to find out why.
>>
>> Thanks,
>> Jungtaek Lim (HeartSaVioR)
>>
>> 2016년 8월 31일 (수) 오전 7:09, Manu Zhang 님이 작성:
>>
>>> I think we have 'aggregate' in Trident, where the function output fields
>>> replace the input fields.
>>>
>>> Thanks,
>>> Manu Zhang
>>>
>>> On Tue, Aug 30, 2016 at 4:17 PM Jungtaek Lim  wrote:
>>>
>>> > Hi devs,
>>> >
>>> > While implementing some features of Storm SQL on top of Trident, I
>>> realized
>>> > that there's no Trident operation which provides a way to add fields
>>> from
>>> > some Function and remove existing fields within one operation.
>>> > 'each' appends the function output fields to origin input fields, and
>>> 'map'
>>> > and 'flatMap' use same output fields, so no luck.
>>> >
>>> > Is this a missing feature, or there's a reason for doing so? Trident
>>> uses
>>> > TridentTupleView so I guess this is related but I didn't have a deep
>>> look
>>> > for Trident itself.
>>> > If anyone knows about the history please let me know.
>>> >
>>> > Thanks,
>>> > Jungtaek Lim (HeartSaVioR)
>>> >
>>> > ps. It might be better to explain why I'm finding that feature. I'm
>>> trying
>>> > to minimize below each -> project -> each -> project chain into one.
>>> >
>>> >
>>> https://github.com/apache/storm/blob/master/external/sql/storm-sql-core/src/jvm/org/apache/storm/sql/compiler/backends/trident/TridentLogicalPlanCompiler.java#L164-L167
>>> > Please note that Trident doesn't allow duplicated field name so using
>>> > temporary field name is necessary if we should preserve existing
>>> fields.
>>> >
>>>
>>

[jira] [Resolved] (STORM-2064) Add storm name and function, access result and function to log-thrift-access

2016-08-31 Thread Jungtaek Lim (JIRA)


 [ 
https://issues.apache.org/jira/browse/STORM-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jungtaek Lim resolved STORM-2064.
-
   Resolution: Fixed
Fix Version/s: 1.0.3
   1.1.0
   2.0.0

Thanks [~abellina], I merged this into master, 1.x, 1.0.x branches.

> Add storm name and function, access result and function to log-thrift-access
> 
>
> Key: STORM-2064
> URL: https://issues.apache.org/jira/browse/STORM-2064
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Alessandro Bellina
>Assignee: Alessandro Bellina
>Priority: Minor
> Fix For: 2.0.0, 1.1.0, 1.0.3
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Improves overall usefulness of the thrift access log.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request #1651: STORM-2064: add function, storm name and auth resu...

2016-08-31 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1651


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1650: STORM-2064: add function, storm name and auth resu...

2016-08-31 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1650


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1480: STORM-1881: Create fat-jar of storm-redis which contains ...

2016-08-31 Thread sakanaou

Github user sakanaou commented on the issue:

https://github.com/apache/storm/pull/1480
  
Closed as requested due to STORM-2016.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1480: STORM-1881: Create fat-jar of storm-redis which co...

2016-08-31 Thread sakanaou

Github user sakanaou closed the pull request at:

https://github.com/apache/storm/pull/1480


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1645: [STORM-2042] Nimbus client connections not closed properl...

2016-08-31 Thread HeartSaVioR

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1645
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1650: STORM-2064: add function, storm name and auth result to t...

2016-08-31 Thread HeartSaVioR

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1650
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm issue #1651: STORM-2064: add function, storm name and auth result to t...

2016-08-31 Thread HeartSaVioR

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1651
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (STORM-2063) Add thread name in worker logs

2016-08-31 Thread Jungtaek Lim (JIRA)


 [ 
https://issues.apache.org/jira/browse/STORM-2063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jungtaek Lim resolved STORM-2063.
-
   Resolution: Fixed
Fix Version/s: 1.0.3
   1.1.0
   2.0.0

Thanks [~abellina], I merged into master, 1.x, 1.0.x branches.

> Add thread name in worker logs
> --
>
> Key: STORM-2063
> URL: https://issues.apache.org/jira/browse/STORM-2063
> Project: Apache Storm
>  Issue Type: Improvement
>Reporter: Alessandro Bellina
>Assignee: Alessandro Bellina
>Priority: Minor
> Fix For: 2.0.0, 1.1.0, 1.0.3
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request #1653: STORM-2063: turn on thread name in worker.xml

2016-08-31 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1653


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request #1652: Storm 2063 thread name in worker log improvements

2016-08-31 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1652


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Closed] (STORM-1703) In local mode, process is not shutting down clearly

2016-08-31 Thread Jungtaek Lim (JIRA)


 [ 
https://issues.apache.org/jira/browse/STORM-1703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jungtaek Lim closed STORM-1703.
---
Resolution: Duplicate

> In local mode, process is not shutting down clearly
> ---
>
> Key: STORM-1703
> URL: https://issues.apache.org/jira/browse/STORM-1703
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 1.0.0, 2.0.0
>Reporter: Jungtaek Lim
> Attachments: RollingTopWords-2.0.0-SNAPSHOT.jstack, 
> RollingTopWords-2.0.0-SNAPSHOT.log, RollingTopWords-thread-dump.jstack, 
> RollingTopWords.log
>
>
> Process is not shutting down clearly in local mode, but ‘Ctrl + C’ can 
> terminate the process.
> Will attach log file and jstack dump file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm issue #1659: [STORM-2056] - Bugs in logviewer

2016-08-31 Thread HeartSaVioR

Github user HeartSaVioR commented on the issue:

https://github.com/apache/storm/pull/1659
  
+1 Nice work.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

87 matches

Mail list logo