[
https://issues.apache.org/jira/browse/STORM-3668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Li updated STORM-3668:
----------------------------
Description:
This
[validateTopologyWorkerMaxHeapSizeConfigs|https://github.com/apache/storm/blob/master/storm-server/src/main/java/org/apache/storm/daemon/nimbus/Nimbus.java#L3150-L3151]
takes topoConf to get the default heap size for components. But at this point,
the topoConf might not have "topology.component.resources.onheap.memory.mb".
And in this case, validateTopologyWorkerMaxHeapSizeConfigs just passes.
For example, submission with
{code:java}
storm jar storm-starter.jar org.apache.storm.starter.WordCountTopology -c
topology.worker.max.heap.size.mb=100 wc
{code}
will succeed while the config "topology.component.resources.onheap.memory.mb"
is default to 120mb > 100mb.
While submission succeeds, nimbus is not able to schedule the topology.
{code:java}
2020-07-06 21:41:07.376 o.a.s.s.r.s.s.BaseResourceAwareStrategy
pool-21-thread-1 [WARN] Not Enough Resources to schedule Task [6, 6] - split
Normalized resources: {onheap.memory.mb=128.0, cpu.pcore.percent=10.0,
offheap.memory.mb=0.0}
{code}
Note: this is when ResourceAwareScheduler is being used.
was:
This
[validateTopologyWorkerMaxHeapSizeConfigs|https://github.com/apache/storm/blob/master/storm-server/src/main/java/org/apache/storm/daemon/nimbus/Nimbus.java#L3150-L3151]
takes topoConf to get the default heap size for components. But at this point,
the topoConf might not have "topology.component.resources.onheap.memory.mb".
And in this case, validateTopologyWorkerMaxHeapSizeConfigs just passes.
For example, submission with
{code:java}
storm jar storm-starter.jar org.apache.storm.starter.WordCountTopology -c
topology.worker.max.heap.size.mb=100 wc
{code}
will succeed while the config "topology.component.resources.onheap.memory.mb"
is default to 120mb > 100mb.
While submission succeeds, nimbus is not able to schedule the topology.
{code:java}
2020-07-06 21:41:07.376 o.a.s.s.r.s.s.BaseResourceAwareStrategy
pool-21-thread-1 [WARN] Not Enough Resources to schedule Task [6, 6] - split
Normalized resources: {onheap.memory.mb=128.0, cpu.pcore.percent=10.0,
offheap.memory.mb=0.0}
{code}
> Nimbus.validateTopologyWorkerMaxHeapSizeConfigs doesn't work properly when a
> topology doesn't set component heap size explicitly
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: STORM-3668
> URL: https://issues.apache.org/jira/browse/STORM-3668
> Project: Apache Storm
> Issue Type: Bug
> Reporter: Ethan Li
> Priority: Minor
>
> This
> [validateTopologyWorkerMaxHeapSizeConfigs|https://github.com/apache/storm/blob/master/storm-server/src/main/java/org/apache/storm/daemon/nimbus/Nimbus.java#L3150-L3151]
> takes topoConf to get the default heap size for components. But at this
> point, the topoConf might not have
> "topology.component.resources.onheap.memory.mb". And in this case,
> validateTopologyWorkerMaxHeapSizeConfigs just passes.
> For example, submission with
> {code:java}
> storm jar storm-starter.jar org.apache.storm.starter.WordCountTopology -c
> topology.worker.max.heap.size.mb=100 wc
> {code}
> will succeed while the config "topology.component.resources.onheap.memory.mb"
> is default to 120mb > 100mb.
> While submission succeeds, nimbus is not able to schedule the topology.
> {code:java}
> 2020-07-06 21:41:07.376 o.a.s.s.r.s.s.BaseResourceAwareStrategy
> pool-21-thread-1 [WARN] Not Enough Resources to schedule Task [6, 6] - split
> Normalized resources: {onheap.memory.mb=128.0, cpu.pcore.percent=10.0,
> offheap.memory.mb=0.0}
> {code}
> Note: this is when ResourceAwareScheduler is being used.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
