[GitHub] storm issue #1521: STORM-1901: Avro Integration for Storm-Kafka

2016-06-27 Thread vesense
Github user vesense commented on the issue: https://github.com/apache/storm/pull/1521 @dossett BTW, I have a question about `ConfluentAvroSerializer`: I can't see any unit tests for `ConfluentAvroSerializer`. I wonder how kryo create the object by using construction

[jira] [Commented] (STORM-1901) Avro Integration for Storm-Kafka

2016-06-27 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/STORM-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352393#comment-15352393 ] ASF GitHub Bot commented on STORM-1901: --- Github user vesense commented on the issue:

[GitHub] storm issue #1521: STORM-1901: Avro Integration for Storm-Kafka

2016-06-27 Thread vesense
Github user vesense commented on the issue: https://github.com/apache/storm/pull/1521 The reason why I define the new interface `DirectAvroSerializer` and move avro to a sharable module is explained as above. @harshach This PR is based on STORM-1504, you can take a look at

[jira] [Commented] (STORM-1932) Change Duration's value data type

2016-06-27 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/STORM-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352307#comment-15352307 ] ASF GitHub Bot commented on STORM-1932: --- GitHub user darionyaphet opened a pull request:

[GitHub] storm pull request #1527: [STORM-1932] Change Duration's value data type

2016-06-27 Thread darionyaphet
GitHub user darionyaphet opened a pull request: https://github.com/apache/storm/pull/1527 [STORM-1932] Change Duration's value data type [STORM-1932](https://issues.apache.org/jira/browse/STORM-1932) BaseWindowedBolt Duration value is integer now . Use long to

[jira] [Updated] (STORM-1932) Change Duration's value data type

2016-06-27 Thread darion yaphet (JIRA)
[ https://issues.apache.org/jira/browse/STORM-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] darion yaphet updated STORM-1932: - Description: BaseWindowedBolt Duration value is integer now . Use long to express time interval

[jira] [Updated] (STORM-1932) Change Duration's value data type

2016-06-27 Thread darion yaphet (JIRA)
[ https://issues.apache.org/jira/browse/STORM-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] darion yaphet updated STORM-1932: - Description: BaseWindowedBolt Duration value is integer . Use long to express time interval maybe

[jira] [Updated] (STORM-1932) Change Duration's value data type

2016-06-27 Thread darion yaphet (JIRA)
[ https://issues.apache.org/jira/browse/STORM-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] darion yaphet updated STORM-1932: - Description: BaseWindowedBolt > Change Duration's value data type >

[jira] [Created] (STORM-1932) Change Duration's value data type

2016-06-27 Thread darion yaphet (JIRA)
darion yaphet created STORM-1932: Summary: Change Duration's value data type Key: STORM-1932 URL: https://issues.apache.org/jira/browse/STORM-1932 Project: Apache Storm Issue Type:

[jira] [Resolved] (STORM-1924) Add a config file parameter to HDFS test topologies

2016-06-27 Thread Jungtaek Lim (JIRA)
[ https://issues.apache.org/jira/browse/STORM-1924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved STORM-1924. - Resolution: Fixed Assignee: Sivaguru Kannan Fix Version/s: 1.1.0

[GitHub] storm issue #1512: Adding hdfs-conf.yaml to all test cases to support securi...

2016-06-27 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1512 Merged (cherry-pick) to master, 1.x, 1.0.x branches. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[jira] [Commented] (STORM-1924) Add a config file parameter to HDFS test topologies

2016-06-27 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/STORM-1924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352191#comment-15352191 ] ASF GitHub Bot commented on STORM-1924: --- Github user HeartSaVioR commented on the issue:

[GitHub] storm issue #1512: Adding hdfs-conf.yaml to all test cases to support securi...

2016-06-27 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1512 Sorry I need to revert merging this and set commit message to contain STORM-1924. Sign-off information is overwritten to me (eac9641469b79251f5146be4dcf8e52e6914448f), so please feel free to

[jira] [Commented] (STORM-1901) Avro Integration for Storm-Kafka

2016-06-27 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/STORM-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352188#comment-15352188 ] ASF GitHub Bot commented on STORM-1901: --- Github user vesense commented on a diff in the pull

[GitHub] storm pull request #1521: STORM-1901: Avro Integration for Storm-Kafka

2016-06-27 Thread vesense
Github user vesense commented on a diff in the pull request: https://github.com/apache/storm/pull/1521#discussion_r68685946 --- Diff: external/storm-kafka/src/jvm/org/apache/storm/kafka/AvroScheme.java --- @@ -0,0 +1,59 @@ +/** + * Licensed to the Apache Software

[GitHub] storm pull request #1512: Adding hdfs-conf.yaml to all test cases to support...

2016-06-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/1512 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] storm issue #1512: Adding hdfs-conf.yaml to all test cases to support securi...

2016-06-27 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1512 +1 Thanks @csivaguru. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] storm pull request #1521: STORM-1901: Avro Integration for Storm-Kafka

2016-06-27 Thread vesense
Github user vesense commented on a diff in the pull request: https://github.com/apache/storm/pull/1521#discussion_r68683162 --- Diff: external/storm-kafka/README.md --- @@ -360,6 +361,44 @@ For Trident: StormSubmitter.submitTopology("kafkaTridentTest", conf,

[jira] [Commented] (STORM-1901) Avro Integration for Storm-Kafka

2016-06-27 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/STORM-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352151#comment-15352151 ] ASF GitHub Bot commented on STORM-1901: --- Github user vesense commented on a diff in the pull

[GitHub] storm pull request #1521: STORM-1901: Avro Integration for Storm-Kafka

2016-06-27 Thread dossett
Github user dossett commented on a diff in the pull request: https://github.com/apache/storm/pull/1521#discussion_r68658648 --- Diff: external/storm-kafka/src/jvm/org/apache/storm/kafka/AvroScheme.java --- @@ -0,0 +1,59 @@ +/** + * Licensed to the Apache Software

[jira] [Commented] (STORM-1901) Avro Integration for Storm-Kafka

2016-06-27 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/STORM-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15351852#comment-15351852 ] ASF GitHub Bot commented on STORM-1901: --- Github user dossett commented on the issue:

[jira] [Commented] (STORM-1901) Avro Integration for Storm-Kafka

2016-06-27 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/STORM-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15351846#comment-15351846 ] ASF GitHub Bot commented on STORM-1901: --- Github user dossett commented on a diff in the pull

[GitHub] storm issue #1521: STORM-1901: Avro Integration for Storm-Kafka

2016-06-27 Thread dossett
Github user dossett commented on the issue: https://github.com/apache/storm/pull/1521 @harshach My interpretation of the PR is that it moves some Avro speficif code I added to storm-hdfs into a more sharable module. @vesense I like the idea of making it easier to read/write

[GitHub] storm pull request #1521: STORM-1901: Avro Integration for Storm-Kafka

2016-06-27 Thread dossett
Github user dossett commented on a diff in the pull request: https://github.com/apache/storm/pull/1521#discussion_r68657788 --- Diff: external/storm-kafka/README.md --- @@ -360,6 +361,44 @@ For Trident: StormSubmitter.submitTopology("kafkaTridentTest", conf,

[jira] [Commented] (STORM-1901) Avro Integration for Storm-Kafka

2016-06-27 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/STORM-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15351832#comment-15351832 ] ASF GitHub Bot commented on STORM-1901: --- Github user dossett commented on a diff in the pull

[GitHub] storm issue #1512: Adding hdfs-conf.yaml to all test cases to support securi...

2016-06-27 Thread csivaguru
Github user csivaguru commented on the issue: https://github.com/apache/storm/pull/1512 Can this be merged in? @harshach @HeartSaVioR --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

What is roles of __system-executor?

2016-06-27 Thread Junguk Cho
Hi, In one worker, by default Storm creates one-executor and one-executor-disruptor threads pair. In detail, it created __system, __acker, and __bolt or __spout. I understood __acker and __bolt or __spout, but what is the roles of __system and __system-disruptor threads? Another question is a

[jira] [Commented] (STORM-1928) ShellSpout should check heartbeat while ShellSpout is waiting for subprocess to sync

2016-06-27 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/STORM-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15351219#comment-15351219 ] ASF GitHub Bot commented on STORM-1928: --- GitHub user HeartSaVioR opened a pull request:

[GitHub] storm pull request #1526: STORM-1928 ShellSpout should check heartbeat while...

2016-06-27 Thread HeartSaVioR
GitHub user HeartSaVioR opened a pull request: https://github.com/apache/storm/pull/1526 STORM-1928 ShellSpout should check heartbeat while ShellSpout is waiting for subprocess to sync When nextTuple is not calling at any chances (max spout pending, backpressure introduced at

[jira] [Updated] (STORM-1928) ShellSpout should check heartbeat while ShellSpout is waiting for subprocess to sync

2016-06-27 Thread Jungtaek Lim (JIRA)
[ https://issues.apache.org/jira/browse/STORM-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated STORM-1928: Summary: ShellSpout should check heartbeat while ShellSpout is waiting for subprocess to sync

[jira] [Commented] (STORM-1931) Share mapper and selector in Storm-Kafka

2016-06-27 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/STORM-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15351115#comment-15351115 ] ASF GitHub Bot commented on STORM-1931: --- GitHub user darionyaphet opened a pull request:

[GitHub] storm pull request #1525: [STORM-1931] Share mapper and selector in Storm-Ka...

2016-06-27 Thread darionyaphet
GitHub user darionyaphet opened a pull request: https://github.com/apache/storm/pull/1525 [STORM-1931] Share mapper and selector in Storm-Kafka [STORM-1931](https://issues.apache.org/jira/browse/STORM-1931) Storm Kafka's mapper and selector and Storm Kafka trident's mapper

[jira] [Created] (STORM-1931) Share mapper and selector in Storm-Kafka

2016-06-27 Thread darion yaphet (JIRA)
darion yaphet created STORM-1931: Summary: Share mapper and selector in Storm-Kafka Key: STORM-1931 URL: https://issues.apache.org/jira/browse/STORM-1931 Project: Apache Storm Issue Type:

[GitHub] storm issue #1096: Send activate and deactivate command to ShellSpout

2016-06-27 Thread obiwabrakenobi
Github user obiwabrakenobi commented on the issue: https://github.com/apache/storm/pull/1096 i'll update the docs @HeartSaVioR --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[jira] [Commented] (STORM-1919) Introduce FilterBolt on storm-redis

2016-06-27 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/STORM-1919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15350669#comment-15350669 ] ASF GitHub Bot commented on STORM-1919: --- Github user darionyaphet commented on the issue:

[GitHub] storm issue #1517: STORM-1919 Introduce FilterBolt on storm-redis

2016-06-27 Thread darionyaphet
Github user darionyaphet commented on the issue: https://github.com/apache/storm/pull/1517 @HeartSaVioR Thank you for you Great Work . I have a small question about `Mapper` extention structure . --- If your project is set up for it, you can reply to this email and have your reply

[jira] [Commented] (STORM-1919) Introduce FilterBolt on storm-redis

2016-06-27 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/STORM-1919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15350663#comment-15350663 ] ASF GitHub Bot commented on STORM-1919: --- Github user darionyaphet commented on a diff in the pull

[GitHub] storm pull request #1517: STORM-1919 Introduce FilterBolt on storm-redis

2016-06-27 Thread darionyaphet
Github user darionyaphet commented on a diff in the pull request: https://github.com/apache/storm/pull/1517#discussion_r68543199 --- Diff: external/storm-redis/README.md --- @@ -21,13 +21,16 @@ use it as a maven dependency: ### For normal Bolt -Storm-redis

[jira] [Commented] (STORM-1719) Introduce REST API: Topology metric stats for stream

2016-06-27 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/STORM-1719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15350637#comment-15350637 ] ASF GitHub Bot commented on STORM-1719: --- GitHub user HeartSaVioR opened a pull request:

[GitHub] storm pull request #1524: STORM-1719 Introduce REST API: Topology metric sta...

2016-06-27 Thread HeartSaVioR
GitHub user HeartSaVioR opened a pull request: https://github.com/apache/storm/pull/1524 STORM-1719 Introduce REST API: Topology metric stats for stream * Path: /api/v1/topology/:id/metrics * This API provides detailed metrics for topology ** shows metrics per component,

[jira] [Commented] (STORM-1311) port backtype.storm.ui.core to java

2016-06-27 Thread Jark Wu (JIRA)
[ https://issues.apache.org/jira/browse/STORM-1311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15350630#comment-15350630 ] Jark Wu commented on STORM-1311: Hi [~darion], this is an origin discussion about REST framework adoption

[jira] [Commented] (STORM-1886) Extend KeyValueState interface with delete method

2016-06-27 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/STORM-1886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15350621#comment-15350621 ] ASF GitHub Bot commented on STORM-1886: --- Github user arunmahadevan commented on the issue:

[GitHub] storm issue #1470: STORM-1886 Extend KeyValueState iface with delete

2016-06-27 Thread arunmahadevan
Github user arunmahadevan commented on the issue: https://github.com/apache/storm/pull/1470 @kosii I have gone through the latest changes and left a few comments. The idea of using tombstone looks nice and once the comments are addressed its good to merge. --- If your project is

[jira] [Commented] (STORM-1886) Extend KeyValueState interface with delete method

2016-06-27 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/STORM-1886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15350577#comment-15350577 ] ASF GitHub Bot commented on STORM-1886: --- Github user arunmahadevan commented on a diff in the pull

[jira] [Commented] (STORM-1886) Extend KeyValueState interface with delete method

2016-06-27 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/STORM-1886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15350573#comment-15350573 ] ASF GitHub Bot commented on STORM-1886: --- Github user arunmahadevan commented on a diff in the pull

[jira] [Commented] (STORM-1886) Extend KeyValueState interface with delete method

2016-06-27 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/STORM-1886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15350575#comment-15350575 ] ASF GitHub Bot commented on STORM-1886: --- Github user arunmahadevan commented on a diff in the pull

[GitHub] storm pull request #1470: STORM-1886 Extend KeyValueState iface with delete

2016-06-27 Thread arunmahadevan
Github user arunmahadevan commented on a diff in the pull request: https://github.com/apache/storm/pull/1470#discussion_r68537734 --- Diff: external/storm-redis/src/main/java/org/apache/storm/redis/state/RedisKeyValueState.java --- @@ -161,7 +176,11 @@ public void

[jira] [Commented] (STORM-1886) Extend KeyValueState interface with delete method

2016-06-27 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/STORM-1886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15350576#comment-15350576 ] ASF GitHub Bot commented on STORM-1886: --- Github user arunmahadevan commented on a diff in the pull

[GitHub] storm pull request #1470: STORM-1886 Extend KeyValueState iface with delete

2016-06-27 Thread arunmahadevan
Github user arunmahadevan commented on a diff in the pull request: https://github.com/apache/storm/pull/1470#discussion_r68537622 --- Diff: external/storm-redis/src/main/java/org/apache/storm/redis/state/RedisKeyValueState.java --- @@ -60,22 +64,23 @@ public

[GitHub] storm pull request #1470: STORM-1886 Extend KeyValueState iface with delete

2016-06-27 Thread arunmahadevan
Github user arunmahadevan commented on a diff in the pull request: https://github.com/apache/storm/pull/1470#discussion_r68537628 --- Diff: external/storm-redis/src/main/java/org/apache/storm/redis/state/RedisKeyValueState.java --- @@ -60,22 +64,23 @@ public

[GitHub] storm pull request #1470: STORM-1886 Extend KeyValueState iface with delete

2016-06-27 Thread arunmahadevan
Github user arunmahadevan commented on a diff in the pull request: https://github.com/apache/storm/pull/1470#discussion_r68537606 --- Diff: external/storm-redis/src/main/java/org/apache/storm/redis/state/RedisKeyValueState.java --- @@ -60,22 +64,23 @@ public

[jira] [Commented] (STORM-1742) More accurate 'complete latency'

2016-06-27 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/STORM-1742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15350457#comment-15350457 ] ASF GitHub Bot commented on STORM-1742: --- Github user HeartSaVioR commented on the issue:

[GitHub] storm issue #1379: STORM-1742 (1.x) More accurate 'complete latency'

2016-06-27 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1379 Also created pull request against master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[jira] [Commented] (STORM-1742) More accurate 'complete latency'

2016-06-27 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/STORM-1742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15350455#comment-15350455 ] ASF GitHub Bot commented on STORM-1742: --- GitHub user HeartSaVioR opened a pull request:

[GitHub] storm pull request #1523: STORM-1742 More accurate 'complete latency'

2016-06-27 Thread HeartSaVioR
GitHub user HeartSaVioR opened a pull request: https://github.com/apache/storm/pull/1523 STORM-1742 More accurate 'complete latency' PR for 1.x: #1379 * Acker computes 'complete latency' and sends back to Spout * start time: Acker receiving ACK_INIT from Spout