Dell - Internal Use - Confidential Never mind found this on a previous thread right after I sent this email.
https://xumingming.sinaapp.com/885/twitter-storm-how-to-develop-a-pluggable-scheduler/ From: Kalluri, Rajesh - Dell Team Sent: Friday, May 22, 2015 5:17 PM To: [email protected] Subject: RE: Is it supported to set different JVM options for different tasks(workers)? Dell - Internal Use - Confidential Nathan, can you point to some documentation on custom schedulers to use with Storm. From: Nathan Leung [mailto:[email protected]] Sent: Friday, May 22, 2015 5:07 PM To: user Subject: Re: Is it supported to set different JVM options for different tasks(workers)? I would rather write a custom scheduler and put certain topologies on specific hosts. On May 22, 2015 1:09 PM, "Mark Zang" <[email protected]<mailto:[email protected]>> wrote: In a topology, some task might consumes more memories and some consumes less. So is it supported to set task level / worker level JVM options to set worker process JVM options? For example, the physical host have 48G memories. The main task prefer to use 8G memories. But the rest just need 1G or less. So it is needed to specify a -Xmx8g in workerOpt In this way, the 48G physical host can start only 6 workers. But it is risky to specify 8G memories and start more than 6 workers per physical host. Is there a best practice to config this?
