[
https://issues.apache.org/jira/browse/STORM-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated STORM-2820:
----------------------------------
Labels: pull-request-available (was: )
> validateTopologyWorkerMaxHeapSizeConfigs function never picks up the value
> set by nimbus
> -----------------------------------------------------------------------------------------
>
> Key: STORM-2820
> URL: https://issues.apache.org/jira/browse/STORM-2820
> Project: Apache Storm
> Issue Type: Bug
> Reporter: Ethan Li
> Assignee: Ethan Li
> Labels: pull-request-available
>
> https://github.com/apache/storm/blob/master/storm-server/src/main/java/org/apache/storm/daemon/nimbus/Nimbus.java#L2548
> {code:java}
> @VisibleForTesting
> static void validateTopologyWorkerMaxHeapSizeConfigs(
> Map<String, Object> stormConf, StormTopology topology) {
> double largestMemReq = getMaxExecutorMemoryUsageForTopo(topology,
> stormConf);
> double topologyWorkerMaxHeapSize =
>
> ObjectReader.getDouble(stormConf.get(Config.TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB),
> 768.0);
> if (topologyWorkerMaxHeapSize < largestMemReq) {
> throw new IllegalArgumentException(
> "Topology will not be able to be successfully scheduled:
> Config "
> + "TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB="
> + topologyWorkerMaxHeapSize
> + " < " + largestMemReq + " (Largest memory requirement
> of a component in the topology)."
> + " Perhaps set TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB to a
> larger amount");
> }
> }
> {code}
> The topologyWorkerMaxHeapSize in the above code is either the value from
> topology configuration (set by the topology) or 768.0. It never picks up the
> value set by nimbus .
> A test:
> I set
> {code:java}
> topology.worker.max.heap.size.mb: 2000.0
> {code}
> in storm.yaml.
> And have the WordCountTopology to have the WordCount bolt with memory load of
> 1024MB.
> {code:java}
> builder.setBolt("count", new WordCount(), 12).fieldsGrouping("split", new
> Fields("word")).setMemoryLoad(1024);
> {code}
> I got an error when submitting this topology. The nimbus log shows
> {code:java}
> 2017-11-15 19:46:43.085 o.a.s.d.n.Nimbus pool-14-thread-2 [WARN] Topology
> submission exception. (topology name='wc')
> java.lang.IllegalArgumentException: Topology will not be able to be
> successfully scheduled: Config TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB=768.0 <
> 1024.0 (Largest memory requirement of a component in the topology). Perhaps
> set TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB to a larger amount
> {code}
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
