Hi Pavithra, The assignedMemory in topology summary is the total memory assigned to your topologies. So it changes when you change your number of executors. You can visit the topology page and it will show you how much memory is assigned for each component.
I suggest to not change Xmx in worker.childopts since worker.childopts is actually a template that will be override before the supervisor launches the worker based on the executors scheduled in this worker. https://github.com/apache/storm/blob/master/conf/defaults.yaml#L194 <https://github.com/apache/storm/blob/master/conf/defaults.yaml#L194> https://github.com/apache/storm/blob/master/storm-server/src/main/java/org/apache/storm/daemon/supervisor/BasicContainer.java#L427-L444 <https://github.com/apache/storm/blob/master/storm-server/src/main/java/org/apache/storm/daemon/supervisor/BasicContainer.java#L427-L444> By default, the memory requested by each component is 128MB. https://github.com/apache/storm/blob/master/conf/defaults.yaml#L333 <https://github.com/apache/storm/blob/master/conf/defaults.yaml#L333> That should explain the UI is showing 61696mb (=128mb * 482). Since you changed worker.childopts to be 16384mb, every processes are running with 16384mb. But UI doesn’t know that. UI uses the scheduling result to determine how much memory is “assigned” to the topology (and to the workers), so it shows 61696mb for the topology on UI. -Ethan > On Jun 19, 2020, at 12:57 AM, Pavithra Gunasekara > <[email protected]> wrote: > > I'm still not able to figure out why assignedMemOnHeap for topology is > showing as 61696mb in storm UI and how this value is being calculated or > where it is being read. We have 482 executors/tasks, and I noticed, whenever > the number of executors/tasks are changed the assigned memory is getting > changed as well. > > Really appreciate it if anyone can explain this to me. > > /api/v1/topology/summary > {"schedulerDisplayResource":false,"topologies":[{"owner":"root","requestedCpu":4820.0,"topologyVersion":null,"replicationCount":3,"stormVersion":"2.1.0","executorsTotal":482,"assignedMemOnHeap":61696.0,"assignedTotalMem":61696.0,"assignedCpu":4820.0,"requestedMemOnHeap":61696.0,"encodedId":"retail_network-1-1592465357","uptimeSeconds":80726,"uptime":"22h > 25m > 26s","schedulerInfo":null,"requestedTotalMem":61696.0,"assignedMemOffHeap":0.0,"workersTotal":1,"requestedMemOffHeap":0.0,"name":"retail_network","id":"retail_network-1-1592465357","tasksTotal":482,"status":"ACTIVE"},{"owner":"root","requestedCpu":4820.0,"topologyVersion":null,"replicationCount":3,"stormVersion":"2.1.0","executorsTotal":482,"assignedMemOnHeap":61696.0,"assignedTotalMem":61696.0,"assignedCpu":4820.0,"requestedMemOnHeap":61696.0,"encodedId":"test_topo-2-1592465447","uptimeSeconds":80635,"uptime":"22h > 23m > 55s","schedulerInfo":null,"requestedTotalMem":61696.0,"assignedMemOffHeap":0.0,"workersTotal":1,"requestedMemOffHeap":0.0,"name":"test-topo","id":"test_topo-2-1592465447","tasksTotal":482,"status":"ACTIVE"}]} > > Thanks > -Pavithra > > On Thu, Jun 18, 2020 at 5:12 PM Pavithra Gunasekara > <[email protected] <mailto:[email protected]>> wrote: > Hi all, > > We are in the process of migrating from Storm 1.1.3 to 2.1.0. In storm.yaml > we use worker.childopts to adjust the worker assigned memory. However in > Storm UI, it shows a different value for Assigned Mem(MB). > > We have following configs in storm.yaml among the others. > > nimbus.childopts: "-Xmx2048m" > supervisor.childopts: "-Xmx2048m" > worker.childopts: "-Xmx16384m" > > Could someone please help us find answers for the following questions? > > 1. Are we missing any memory related configs in storm.yaml? > 2. Why is the UI showing a different value(61696 mb) than the one we have > used in the configuration file(16384 mb), from where does this value come > from? > > Thanks & Regards, > -Pavithra > > > -- > -Pavithra
