[jira] [Commented] (STORM-3195) IllegalArgumentException when rolling upgrade nimbus from 1.x to 2.x
[
https://issues.apache.org/jira/browse/STORM-3195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582511#comment-16582511
]
Ethan Li commented on STORM-3195:
-
Yeah this is our internal code. We have a piece of code in nimbu.clj similar to
https://github.com/apache/storm/pull/2325 but it's in clojure:
{code:java}
(defn estimated-worker-count-for-ras-topo [storm-conf topology]
(let [component-parallelism-mp (into {} (map #(vector (first %)
(component-parallelism storm-conf (second %))) (all-components topology)))
bolt-memory-requirement (into {} (map #(vector (first %)
(.getOnHeapMemoryMb (second %))) (into {} (ResourceUtils/getBoltsResources
topology storm-conf
spout-memory-requirement (into {} (map #(vector (first %)
(.getOnHeapMemoryMb (second %))) (into {} (ResourceUtils/getSpoutsResources
topology storm-conf
component-memory-requirement (merge bolt-memory-requirement
spout-memory-requirement)
total-memory-required (reduce + (vals (merge-with *
component-memory-requirement component-parallelism-mp)))
worker-heap-memory (get storm-conf WORKER-HEAP-MEMORY-MB)
]
(/ total-memory-required worker-heap-memory)))
{code}
This returns a double value.
We don't need to worry about it in the community code.
> IllegalArgumentException when rolling upgrade nimbus from 1.x to 2.x
>
>
> Key: STORM-3195
> URL: https://issues.apache.org/jira/browse/STORM-3195
> Project: Apache Storm
> Issue Type: Bug
>Reporter: Ethan Li
>Priority: Major
>
> {code:java}
> 2018-08-15 15:49:06.550 o.a.s.d.n.Nimbus timer [ERROR] Error while processing
> event
> java.lang.RuntimeException: java.lang.IllegalArgumentException: Don't know
> how to convert 1.76041667 to int
> at
> org.apache.storm.daemon.nimbus.Nimbus.lambda$launchServer$48(Nimbus.java:2837)
> ~[storm-server-2.0.0.y.jar:2.0.0.y]
> at org.apache.storm.StormTimer$1.run(StormTimer.java:110)
> ~[storm-client-2.0.0.y.jar:2.0.0.y]
> at
> org.apache.storm.StormTimer$StormTimerTask.run(StormTimer.java:226)
> [storm-client-2.0.0.y.jar:2.0.0.y]
> Caused by: java.lang.IllegalArgumentException: Don't know how to convert
> 1.76041667 to int
> at org.apache.storm.utils.ObjectReader.getInt(ObjectReader.java:78)
> ~[storm-client-2.0.0.y.jar:2.0.0.y]
> at org.apache.storm.daemon.StormCommon.addAcker(StormCommon.java:256)
> ~[storm-client-2.0.0.y.jar:2.0.0.y]
> at
> org.apache.storm.daemon.StormCommon.systemTopologyImpl(StormCommon.java:523)
> ~[storm-client-2.0.0.y.jar:2.0.0.y]
> at
> org.apache.storm.daemon.StormCommon.systemTopology(StormCommon.java:440)
> ~[storm-client-2.0.0.y.jar:2.0.0.y]
> at
> org.apache.storm.daemon.StormCommon.stormTaskInfoImpl(StormCommon.java:544)
> ~[storm-client-2.0.0.y.jar:2.0.0.y]
> at
> org.apache.storm.daemon.StormCommon.stormTaskInfo(StormCommon.java:460)
> ~[storm-client-2.0.0.y.jar:2.0.0.y]
> at
> org.apache.storm.daemon.nimbus.Nimbus.computeExecutors(Nimbus.java:1732)
> ~[storm-server-2.0.0.y.jar:2.0.0.y]
> at
> org.apache.storm.daemon.nimbus.Nimbus.getOrUpdateExecutors(Nimbus.java:1357)
> ~[storm-server-2.0.0.y.jar:2.0.0.y]
> at
> org.apache.storm.daemon.nimbus.Nimbus.computeExecutorToComponent(Nimbus.java:1752)
> ~[storm-server-2.0.0.y.jar:2.0.0.y]
> at
> org.apache.storm.daemon.nimbus.Nimbus.readTopologyDetails(Nimbus.java:1582)
> ~[storm-server-2.0.0.y.jar:2.0.0.y]
> at
> org.apache.storm.daemon.nimbus.Nimbus.getResourcesForTopology(Nimbus.java:2034)
> ~[storm-server-2.0.0.y.jar:2.0.0.y]
> at
> org.apache.storm.daemon.nimbus.Nimbus.getClusterInfoImpl(Nimbus.java:2676)
> ~[storm-server-2.0.0.y.jar:2.0.0.y]
> at
> org.apache.storm.daemon.nimbus.Nimbus.sendClusterMetricsToExecutors(Nimbus.java:2701)
> ~[storm-server-2.0.0.y.jar:2.0.0.y]
> at
> org.apache.storm.daemon.nimbus.Nimbus.lambda$launchServer$48(Nimbus.java:2834)
> ~[storm-server-2.0.0.y.jar:2.0.0.y]
> ... 2 more
> {code}
> We were rolling upgrading our storm cluster to 2.x and we found this error
> from nimbus.log. It's not necessary to do rolling upgrade but good news is we
> basically made it work.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (STORM-3195) IllegalArgumentException when rolling upgrade nimbus from 1.x to 2.x
[
https://issues.apache.org/jira/browse/STORM-3195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16581928#comment-16581928
]
Jungtaek Lim commented on STORM-3195:
-
[~ethanli]
I'm wondering the symptom itself: shouldn't worker count as well as acker
executor number be integer? No one would say worker count is 1.66 and acker
executor count is 2.33, so the value itself looks somewhat incorrect to me.
> IllegalArgumentException when rolling upgrade nimbus from 1.x to 2.x
>
>
> Key: STORM-3195
> URL: https://issues.apache.org/jira/browse/STORM-3195
> Project: Apache Storm
> Issue Type: Bug
>Reporter: Ethan Li
>Priority: Major
>
> {code:java}
> 2018-08-15 15:49:06.550 o.a.s.d.n.Nimbus timer [ERROR] Error while processing
> event
> java.lang.RuntimeException: java.lang.IllegalArgumentException: Don't know
> how to convert 1.76041667 to int
> at
> org.apache.storm.daemon.nimbus.Nimbus.lambda$launchServer$48(Nimbus.java:2837)
> ~[storm-server-2.0.0.y.jar:2.0.0.y]
> at org.apache.storm.StormTimer$1.run(StormTimer.java:110)
> ~[storm-client-2.0.0.y.jar:2.0.0.y]
> at
> org.apache.storm.StormTimer$StormTimerTask.run(StormTimer.java:226)
> [storm-client-2.0.0.y.jar:2.0.0.y]
> Caused by: java.lang.IllegalArgumentException: Don't know how to convert
> 1.76041667 to int
> at org.apache.storm.utils.ObjectReader.getInt(ObjectReader.java:78)
> ~[storm-client-2.0.0.y.jar:2.0.0.y]
> at org.apache.storm.daemon.StormCommon.addAcker(StormCommon.java:256)
> ~[storm-client-2.0.0.y.jar:2.0.0.y]
> at
> org.apache.storm.daemon.StormCommon.systemTopologyImpl(StormCommon.java:523)
> ~[storm-client-2.0.0.y.jar:2.0.0.y]
> at
> org.apache.storm.daemon.StormCommon.systemTopology(StormCommon.java:440)
> ~[storm-client-2.0.0.y.jar:2.0.0.y]
> at
> org.apache.storm.daemon.StormCommon.stormTaskInfoImpl(StormCommon.java:544)
> ~[storm-client-2.0.0.y.jar:2.0.0.y]
> at
> org.apache.storm.daemon.StormCommon.stormTaskInfo(StormCommon.java:460)
> ~[storm-client-2.0.0.y.jar:2.0.0.y]
> at
> org.apache.storm.daemon.nimbus.Nimbus.computeExecutors(Nimbus.java:1732)
> ~[storm-server-2.0.0.y.jar:2.0.0.y]
> at
> org.apache.storm.daemon.nimbus.Nimbus.getOrUpdateExecutors(Nimbus.java:1357)
> ~[storm-server-2.0.0.y.jar:2.0.0.y]
> at
> org.apache.storm.daemon.nimbus.Nimbus.computeExecutorToComponent(Nimbus.java:1752)
> ~[storm-server-2.0.0.y.jar:2.0.0.y]
> at
> org.apache.storm.daemon.nimbus.Nimbus.readTopologyDetails(Nimbus.java:1582)
> ~[storm-server-2.0.0.y.jar:2.0.0.y]
> at
> org.apache.storm.daemon.nimbus.Nimbus.getResourcesForTopology(Nimbus.java:2034)
> ~[storm-server-2.0.0.y.jar:2.0.0.y]
> at
> org.apache.storm.daemon.nimbus.Nimbus.getClusterInfoImpl(Nimbus.java:2676)
> ~[storm-server-2.0.0.y.jar:2.0.0.y]
> at
> org.apache.storm.daemon.nimbus.Nimbus.sendClusterMetricsToExecutors(Nimbus.java:2701)
> ~[storm-server-2.0.0.y.jar:2.0.0.y]
> at
> org.apache.storm.daemon.nimbus.Nimbus.lambda$launchServer$48(Nimbus.java:2834)
> ~[storm-server-2.0.0.y.jar:2.0.0.y]
> ... 2 more
> {code}
> We were rolling upgrading our storm cluster to 2.x and we found this error
> from nimbus.log. It's not necessary to do rolling upgrade but good news is we
> basically made it work.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
