Re: Resource Aware Scheduler Kills Nimbus

2018-02-20 Thread Wuyang Zhang
Hi Alessio, Thank you for your reply. I think the storm community can consider to introduce some automatic resource analysis mechanisms to set up those values. The resource-aware scheduling algorithm itself is great, while it may not perform well without accurately setting the task resource

Re: Resource Aware Scheduler Kills Nimbus

2018-02-19 Thread Alessio Pagliari
Hi Wuyang, This is a nice question, I didn’t find anything that computes it automatically, yet. By now you have to test your topology tuning the resource requirements until you don’t find pleasing values. Cheers, Alessio > On 10 Feb 2018, at 21:42, Wuyang Zhang

Re: Resource Aware Scheduler Kills Nimbus

2018-02-10 Thread Wuyang Zhang
Hi Jerry, I am not familiar with the pull operation on the Storm website... Can you do it when you get a chance? Also, I wonder how can I accurately specify the resource requirement, especially the cpu demand, for each task? Best, Wuyang ᐧ On Sat, Feb 10, 2018 at 3:21 PM, Jerry Peng

Re: Resource Aware Scheduler Kills Nimbus

2018-02-10 Thread Jerry Peng
Yup, an error in the documentation Feel free to submit a pull request to fix the documentation. Best, Jerry On Sat, Feb 10, 2018 at 1:42 PM, Wuyang Zhang wrote: > Ok, I find it works by adding > storm.scheduler: org.apache.storm.scheduler.resource. >

Re: Resource Aware Scheduler Kills Nimbus

2018-02-10 Thread Wuyang Zhang
Ok, I find it works by adding storm.scheduler: org.apache.storm.scheduler.resource.ResourceAwareScheduler instead of adding storm.scheduler: “org.apache.storm.scheduler.resource.ResourceAwareScheduler” suggested by the website.. ᐧ On Sat, Feb 10, 2018 at 2:22 PM, Wuyang Zhang

Re: Resource Aware Scheduler Kills Nimbus

2018-02-10 Thread Wuyang Zhang
When I redo the procedure, here is the more log *2018-02-10 14:10:32.791 o.a.s.d.nimbus main [INFO] Using custom scheduler: “org.apache.storm.scheduler.resource.ResourceAwareScheduler”* *2018-02-10 14:10:32.793 o.a.s.d.nimbus main [ERROR] Error on initialization of server service-handler*

Re: Resource Aware Scheduler Kills Nimbus

2018-02-10 Thread Wuyang Zhang
Hi Jerry, I just retry to use the resource aware scheduler in a single node with 4 slots without any further settings. The scheduler still kills the nimbus process. Please find nimbus.log and storm.yaml attached. Without add the line storm.scheduler:

Re: Resource Aware Scheduler Kills Nimbus

2018-02-10 Thread Zhang Wuyang
Hi Jerry, Thank you for the reply. Please find the storm.yaml below: ### These MUST be filled in for a storm configuration storm.zookeeper.servers: - "10.10.21.1" - "10.10.21.2" - "10.10.21.3" - "10.10.21.4" - "10.10.21.5" - "10.10.21.6" -