[jira] [Comment Edited] (STORM-2983) Some topologies not working properly
[
https://issues.apache.org/jira/browse/STORM-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427559#comment-16427559
]
Roshan Naik edited comment on STORM-2983 at 4/5/18 8:44 PM:
Thanks [~revans2] for those clarifications.
Thanks [~ethanli] for the revised fix.
Similar fixes are needed elsewhere as well, since they are all probably broken:
-
[https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/daemon/StormCommon.java#L276]
-
[https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/daemon/StormCommon.java#L363-L364]
-
[https://github.com/apache/storm/blob/master/examples/storm-loadgen/src/main/java/org/apache/storm/loadgen/CaptureLoad.java#L109-L112]
-
[https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/daemon/worker/Worker.java#L306]
was (Author: roshan_naik):
Thanks [~revans2] for those clarifications.
Thanks [~ethanli] for the revised fix.
Similar fixes are needed elsewhere as well, since they are all probably broken:
-
[https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/daemon/StormCommon.java#L276]
-
[https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/daemon/StormCommon.java#L363-L364]
-
[https://github.com/apache/storm/blob/master/examples/storm-loadgen/src/main/java/org/apache/storm/loadgen/CaptureLoad.java#L109-L112]
-
[https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/daemon/worker/Worker.java#L306]
> Some topologies not working properly
> -
>
> Key: STORM-2983
> URL: https://issues.apache.org/jira/browse/STORM-2983
> Project: Apache Storm
> Issue Type: Bug
>Reporter: Ethan Li
>Assignee: Ethan Li
>Priority: Critical
> Labels: pull-request-available
> Fix For: 2.0.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> For example,
> {code:java}
> bin/storm jar storm-loadgen-*.jar
> org.apache.storm.loadgen.ThroughputVsLatency --spouts 1 --splitters 2
> --counters 1 -c topology.debug=true
> {code}
> on ResourceAwareScheduler not working properly.
> With default cluster settings, there will be only one __acker-executor and it
> will be on a separate worker. And it looks like the __acker-executor was not
> able to receive messages from spouts and bolts. And spouts and bolts
> continued to retry sending messages to acker. It then led to another problem:
> STORM-2970
> I tried to run on storm right before
> [https://github.com/apache/storm/pull/2502] and right after and confirmed
> that this bug should be related to it
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (STORM-2983) Some topologies not working properly
[
https://issues.apache.org/jira/browse/STORM-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427559#comment-16427559
]
Roshan Naik commented on STORM-2983:
Thanks [~revans2] for those clarifications.
Thanks [~ethanli] for the revised fix.
Similar fixes are needed elsewhere as well, since they are all probably broken:
-
[https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/daemon/StormCommon.java#L276]
-
[https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/daemon/StormCommon.java#L363-L364]
-
[https://github.com/apache/storm/blob/master/examples/storm-loadgen/src/main/java/org/apache/storm/loadgen/CaptureLoad.java#L109-L112]
-
[https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/daemon/worker/Worker.java#L306]
> Some topologies not working properly
> -
>
> Key: STORM-2983
> URL: https://issues.apache.org/jira/browse/STORM-2983
> Project: Apache Storm
> Issue Type: Bug
>Reporter: Ethan Li
>Assignee: Ethan Li
>Priority: Critical
> Labels: pull-request-available
> Fix For: 2.0.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> For example,
> {code:java}
> bin/storm jar storm-loadgen-*.jar
> org.apache.storm.loadgen.ThroughputVsLatency --spouts 1 --splitters 2
> --counters 1 -c topology.debug=true
> {code}
> on ResourceAwareScheduler not working properly.
> With default cluster settings, there will be only one __acker-executor and it
> will be on a separate worker. And it looks like the __acker-executor was not
> able to receive messages from spouts and bolts. And spouts and bolts
> continued to retry sending messages to acker. It then led to another problem:
> STORM-2970
> I tried to run on storm right before
> [https://github.com/apache/storm/pull/2502] and right after and confirmed
> that this bug should be related to it
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Closed] (STORM-2753) Avoid shutting down netty server on netty exception
[ https://issues.apache.org/jira/browse/STORM-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Li closed STORM-2753. --- Resolution: Won't Fix > Avoid shutting down netty server on netty exception > --- > > Key: STORM-2753 > URL: https://issues.apache.org/jira/browse/STORM-2753 > Project: Apache Storm > Issue Type: Bug > Components: storm-client >Reporter: Ethan Li >Assignee: Ethan Li >Priority: Minor > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > We should avoid shutting down netty server on netty exception -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (STORM-2838) Replace log4j-over-slf4j with log4j-1.2-api
[
https://issues.apache.org/jira/browse/STORM-2838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Li closed STORM-2838.
---
Resolution: Won't Fix
> Replace log4j-over-slf4j with log4j-1.2-api
> ---
>
> Key: STORM-2838
> URL: https://issues.apache.org/jira/browse/STORM-2838
> Project: Apache Storm
> Issue Type: Improvement
>Reporter: Ethan Li
>Assignee: Ethan Li
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> I tried to setup HdfsBlobStore and an exception shows up when I launch the
> nimbus.
> {code:java}
> Detected both log4j-over-slf4j.jar AND slf4j-log4j12.jar on the class path,
> preempting StackOverflowError.
> {code}
> Found an explanation: https://www.slf4j.org/codes.html#log4jDelegationLoop
> This is because storm and hadoop use different logging system:
> {code:java}
> Storm: log4j-over-slf4j --> slf4j --> log4j2 or slf4j --> log4j2
> Hadoop: slf4j --> log4j1.2 or log4j1.2
> (note: --> means redirecting)
> {code}
> When we add hadoop common lib classpath to nimbus, log4j-over-slf4j.jar and
> slf4j-log4j12.jar coexist.
> One way to let storm work with hadoop is to replace log4j-over-slf4j in storm
> with log4j1.2.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (STORM-2983) Some topologies not working properly
[
https://issues.apache.org/jira/browse/STORM-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427441#comment-16427441
]
Robert Joseph Evans commented on STORM-2983:
Thanks [~kabhwan] and [~ethanli] for getting that in.
[~roshan_naik] I am happy to continue any discussion on the mailing list or on
follow on JIRAs.
> Some topologies not working properly
> -
>
> Key: STORM-2983
> URL: https://issues.apache.org/jira/browse/STORM-2983
> Project: Apache Storm
> Issue Type: Bug
>Reporter: Ethan Li
>Assignee: Ethan Li
>Priority: Critical
> Labels: pull-request-available
> Fix For: 2.0.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> For example,
> {code:java}
> bin/storm jar storm-loadgen-*.jar
> org.apache.storm.loadgen.ThroughputVsLatency --spouts 1 --splitters 2
> --counters 1 -c topology.debug=true
> {code}
> on ResourceAwareScheduler not working properly.
> With default cluster settings, there will be only one __acker-executor and it
> will be on a separate worker. And it looks like the __acker-executor was not
> able to receive messages from spouts and bolts. And spouts and bolts
> continued to retry sending messages to acker. It then led to another problem:
> STORM-2970
> I tried to run on storm right before
> [https://github.com/apache/storm/pull/2502] and right after and confirmed
> that this bug should be related to it
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (STORM-2687) Group Topology executors by network proximity needs and schedule them on "network wise" close slots
[ https://issues.apache.org/jira/browse/STORM-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated STORM-2687: -- Labels: pull-request-available (was: ) > Group Topology executors by network proximity needs and schedule them on > "network wise" close slots > --- > > Key: STORM-2687 > URL: https://issues.apache.org/jira/browse/STORM-2687 > Project: Apache Storm > Issue Type: Sub-task > Components: storm-core, storm-server >Reporter: Ethan Li >Assignee: Ethan Li >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (STORM-2983) Some topologies not working properly
[
https://issues.apache.org/jira/browse/STORM-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jungtaek Lim updated STORM-2983:
Priority: Critical (was: Major)
> Some topologies not working properly
> -
>
> Key: STORM-2983
> URL: https://issues.apache.org/jira/browse/STORM-2983
> Project: Apache Storm
> Issue Type: Bug
>Reporter: Ethan Li
>Assignee: Ethan Li
>Priority: Critical
> Labels: pull-request-available
> Fix For: 2.0.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> For example,
> {code:java}
> bin/storm jar storm-loadgen-*.jar
> org.apache.storm.loadgen.ThroughputVsLatency --spouts 1 --splitters 2
> --counters 1 -c topology.debug=true
> {code}
> on ResourceAwareScheduler not working properly.
> With default cluster settings, there will be only one __acker-executor and it
> will be on a separate worker. And it looks like the __acker-executor was not
> able to receive messages from spouts and bolts. And spouts and bolts
> continued to retry sending messages to acker. It then led to another problem:
> STORM-2970
> I tried to run on storm right before
> [https://github.com/apache/storm/pull/2502] and right after and confirmed
> that this bug should be related to it
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (STORM-2983) Some topologies not working properly
[
https://issues.apache.org/jira/browse/STORM-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jungtaek Lim resolved STORM-2983.
-
Resolution: Fixed
Fix Version/s: 2.0.0
Thanks [~ethanli], I merged into master.
[~roshan_naik] [~revans2]
Please note that [https://github.com/apache/storm/pull/2614] fixed the issue
via proposed way (even better) : it can also cover multiple all-in-one workers
use case as well.
We can continue discussion, but maybe better to move to dev. list or file
issue(s) since the topic(s) are out of topic for this issue.
> Some topologies not working properly
> -
>
> Key: STORM-2983
> URL: https://issues.apache.org/jira/browse/STORM-2983
> Project: Apache Storm
> Issue Type: Bug
>Reporter: Ethan Li
>Assignee: Ethan Li
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.0.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> For example,
> {code:java}
> bin/storm jar storm-loadgen-*.jar
> org.apache.storm.loadgen.ThroughputVsLatency --spouts 1 --splitters 2
> --counters 1 -c topology.debug=true
> {code}
> on ResourceAwareScheduler not working properly.
> With default cluster settings, there will be only one __acker-executor and it
> will be on a separate worker. And it looks like the __acker-executor was not
> able to receive messages from spouts and bolts. And spouts and bolts
> continued to retry sending messages to acker. It then led to another problem:
> STORM-2970
> I tried to run on storm right before
> [https://github.com/apache/storm/pull/2502] and right after and confirmed
> that this bug should be related to it
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
