Thanks for explanation. Do you know how to disable resource aware sheduler? I don't really need it. I have only one topology with known constant number of workers, so I need very basic sheduler.
Best regards, Michal ________________________________ From: Roshan Naik <[email protected]> Sent: Saturday, 10 August 2019 07:45:41 To: [email protected] <[email protected]> Subject: Re: Storm 2.0 and memory for worker I think the change in behavior has to do with the resource aware scheduler. I have noticed this setting impact the heap size. - topology.component.resources.onheap.memory.mb It (i think) allocates that much mem to each component instance running in any given worker. So each worker's heap can be diff. If RAS is disabled, I suspect the behavior *might* revert to what you are used to 1.x. Perhaps someone better familiar with RAS can comment. -roshan On Thursday, August 1, 2019, 01:23:53 AM PDT, Stig Rohde Døssing <[email protected]> wrote: Here are a couple of links to the source that might help you. The system property is coming from https://github.com/apache/storm/blob/eaed3cbf3610bbefb9466f2c40db1e0e2a20ca9c/storm-server/src/main/java/org/apache/storm/daemon/supervisor/BasicContainer.java#L574 It is being calculated by https://github.com/apache/storm/blob/eaed3cbf3610bbefb9466f2c40db1e0e2a20ca9c/storm-server/src/main/java/org/apache/storm/daemon/supervisor/BasicContainer.java#L805 See also https://github.com/apache/storm/blob/eaed3cbf3610bbefb9466f2c40db1e0e2a20ca9c/storm-server/src/main/java/org/apache/storm/daemon/supervisor/BasicContainer.java#L579 Den ons. 31. jul. 2019 kl. 18.41 skrev Michal Koziorowski <[email protected]>: > > > Hi, > > > > > In storm 1.1 I could set Xmx for a worker by setting worker.heap.memory.mb > in storm.yaml. > > It seems ignored in my upgraded Storm 2.0 configuration and Xmx value was > taken from -Dworker.memory_limit_mb=5888 which I didn't set. > > How to make setting from storm 1.1 working in 2.0? > > > > > I have following properties in storm.yaml: > > worker.heap.memory.mb: 16000 > > > worker.childopts: "-Xmx%HEAP-MEM%m > -Xlog:gc*,gc+age=trace,safepoint:file=artifacts/gc.log:utctime,pid,tags:filecount=32,filesize=10m > -Djava.net.preferIPv4Stack=true" > > > > > > Previously 16000 was used as %HEAP-MEM% > > > > >
