Hi, I am using DRPC for my log processing. We have a very heavy task and trying to do it with drpc. Input to drpc is a file name , output is usually 1GB string data.
Problem : DRPC is not assigning task to all workers. Currently running 5 nodes.Other 4 nodes using given configuration server as drpc. Some workers will get job immediately but some of them not getting assigned in 20-30 mins. Kindly help. Configuration is : storm.zookeeper.session.timeout: 250000 storm.zookeeper.connection.timeout: 250000 nimbus.task.timeout.secs: 300 nimbus.monitor.freq.secs: 10 storm.local.dir: "/mnt/storm" supervisor.worker.timeout.secs: 250 worker.heartbeat.frequency.secs: 5 supervisor.monitor.frequency.secs: 10 supervisor.heartbeat.frequency.secs: 5 drpc.request.timeout.secs: 3500 drpc.worker.threads: 640 drpc.max_buffer_size: 10485760 drpc.queue.size: 1280 drpc.invocations.threads: 640 drpc.childopts: "-Xmx7680m" drpc.servers: - "localhost" storm.zookeeper.servers: - "localhost" nimbus.host: "localhost" storm.local.dir: "/mnt/storm" worker.childopts: "-Xmx25000m" supervisor.slots.ports: - 6700 - 6701 - 6702 - 6703..more workers Thanks Satyavrat
