Hi Jerry, I just retry to use the resource aware scheduler in a single node with 4 slots without any further settings. The scheduler still kills the nimbus process. Please find nimbus.log and storm.yaml attached.
Without add the line storm.scheduler: “org.apache.storm.scheduler.resource.ResourceAwareScheduler” in storm.yaml, everything works fine. I also tried conf.setTopologyStrategy(org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy.class); The cluster can run, but I cannot see any log related to DefaultResourceAware. It seems like that it doesn't really run. Please take a look at the log and see if I miss anything to enable them to work? Best, Wuyang ᐧ On Sat, Feb 10, 2018 at 12:23 PM, Zhang Wuyang <qingshanyou...@gmail.com> wrote: > Hi Jerry, > > Thank you for the reply. > > Please find the storm.yaml below: > > ########### These MUST be filled in for a storm configuration > storm.zookeeper.servers: > - "10.10.21.1" > - "10.10.21.2" > - "10.10.21.3" > - "10.10.21.4" > - "10.10.21.5" > - "10.10.21.6" > - "10.10.21.7" > > nimbus.seeds: ["10.10.21.1"] > > storm.zookeeper.port: 2181 > ###### These may optionally be filled in: > > worker.heap.memory.mb: 8192 > > supervisor.scheduler.meta: > tag: 10.10.20.1 > > #storm.scheduler: storm.EmptyScheduler > storm.scheduler: “org.apache.storm.scheduler.resource. > ResourceAwareScheduler” > supervisor.memory.capacity.mb: 2048.0 > supervisor.cpu.capacity: 4000.0 > supervisor.slots.ports: > - 6700 > - 6701 > - 6702 > - 6703 > > worker.childopts: "-Xmx%HEAP-MEM%m -Xms6000m -XX:+PrintGCDetails > -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps > -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M > -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump" > > On Sat, Feb 10, 2018 at 12:16 PM, Jerry Peng <jerry.boyang.p...@gmail.com> > wrote: > >> Wuyang, >> >> Can you provide your whole storm.yaml ? >> >> Best, >> >> Jerry >> >> On Fri, Feb 9, 2018 at 9:44 PM, Wuyang Zhang <wuy...@winlab.rutgers.edu> >> wrote: >> >>> Hi, >>> >>> I have the difficulty with the resource aware scheduler from the version >>> 1.1.1. >>> >>> As suggested by the website, I add the following lines to storm.yaml: >>> >>> *storm.scheduler: “org.apache.storm.scheduler.re >>> <http://org.apache.storm.scheduler.re>source.ResourceAwareScheduler”* >>> * supervisor.memory.capacity.mb: 2048.0* >>> * supervisor.cpu.capacity: 400.0* >>> >>> Afterwards, I rerun the storm and the nimbus process has been killed. >>> The last log I found from nimbus.log is: >>> o.a.s.d.nimbus main [INFO] Using custom scheduler: “ >>> org.apache.storm.scheduler.resource.ResourceAwareScheduler”. >>> >>> I suppose something goes wrong when it runs the scheduler. >>> >>> Do I miss anything to enable this scheduler? >>> >>> Best, >>> Wuyang >>> ᐧ >>> >> >> >
nimbus.log
Description: Binary data
storm.yaml
Description: Binary data
