java.lang.IllegalArgumentException: timeout value is negative
I'm getting this exception 2014-06-03 19:59:13 STDIO [ERROR][id:] Jun 03, 2014 7:59:13 PM org.jboss.netty.channel.DefaultChannelPipeline WARNING: An exception was thrown by a user handler while handling an exception event ([id: 0xdcf3be42] EXCEPTION: java.net.ConnectException: Connection refused) java.lang.IllegalArgumentException: timeout value is negative at java.lang.Thread.sleep(Native Method) at backtype.storm.messaging.netty.Client.reconnect(Client.java:94) at backtype.storm.messaging.netty.StormClientHandler.exceptionCaught(StormClientHandler.java:118) at org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.java:377) *Here's my storm.yaml (from nimbus/zookeeper only server hence ports are commented)* storm.zookeeper.servers: - ma-app05.corp.mydomain.com storm.zookeeper.port: 2181 storm.zookeeper.root: /storm storm.zookeeper.session.timeout: 2 storm.zookeeper.connection.timeout: 15000 storm.zookeeper.retry.times: 5 storm.zookeeper.retry.interval: 1000 storm.zookeeper.retry.intervalceiling.millis: 3 nimbus.host: ma-app05.corp.mydomain.com nimbus.thrift.port: 6627 nimbus.thrift.max_buffer_size: 1048576 nimbus.childopts: -Xmx1024m nimbus.task.timeout.secs: 30 nimbus.supervisor.timeout.secs: 60 nimbus.monitor.freq.secs: 10 nimbus.cleanup.inbox.freq.secs: 600 nimbus.inbox.jar.expiration.secs: 3600 nimbus.task.launch.secs: 120 nimbus.reassign: true nimbus.file.copy.expiration.secs: 600 nimbus.topology.validator: backtype.storm.nimbus.DefaultTopologyValidator #No supervisor in Nimbus. Pls. note that storm might still be loading the default conf, if it requires. #supervisor.slots.ports: # - 9951 # - 9952 # - 9953 # - 9954 # - 9955 #supervisor.worker.start.timeout.secs: 120 #supervisor.worker.timeout.secs: 30 #supervisor.monitor.frequency.secs: 3 #supervisor.heartbeat.frequency.secs: 5 #supervisor.enable: true ui.port: 8181 ui.childopts: -Xmx256m logviewer.port: 8000 logviewer.childopts: -Xmx128m logviewer.appender.name: A1 worker.childopts: -Xmx512m -XX:MaxPermSize=256m -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -verbose:gc -Xloggc:/app/local/var/logs/jvm-stat/gc-storm-worker-%ID%.log worker.heartbeat.frequency.secs: 1 task.heartbeat.frequency.secs: 3 task.refresh.poll.secs: 10 #zmq.threads: 1 #zmq.linger.millis: 5000 #zmq.hwm: 0 storm.local.dir: /ngs/app/isdbd/local/var/run/storm/data storm.local.mode.zmq: false storm.cluster.mode: distributed storm.thrift.transport: backtype.storm.security.auth.SimpleTransportPlugin storm.messaging.transport: backtype.storm.messaging.netty.Context storm.messaging.transport: backtype.storm.messaging.netty.Context storm.messaging.netty.server_worker_threads: 1 storm.messaging.netty.client_worker_threads: 1 storm.messaging.netty.buffer_size: 5242880 storm.messaging.netty.max_retries: 100 storm.messaging.netty.max_wait_ms: 1000 storm.messaging.netty.min_wait_ms: 500 java.library.path: /usr/local/lib:/opt/local/lib:/usr/lib topology.enable.message.timeouts: true topology.debug: false topology.optimize: true topology.workers: 5 topology.acker.executors: null topology.tasks: null topology.message.timeout.secs: 30 topology.skip.missing.kryo.registrations: false topology.max.task.parallelism: null topology.max.spout.pending: 20 topology.state.synchronization.timeout.secs: 60 topology.stats.sample.rate: 0.05 topology.builtin.metrics.bucket.size.secs: 60 topology.fall.back.on.java.serialization: true topology.worker.childopts: -Xmx512m topology.executor.receive.buffer.size: 1024 topology.executor.send.buffer.size: 1024 topology.receiver.buffer.size: 8 topology.transfer.buffer.size: 1024 topology.tick.tuple.freq.secs: null topology.worker.shared.thread.pool.size: 4 topology.disruptor.wait.strategy: com.lmax.disruptor.BlockingWaitStrategy topology.spout.wait.strategy: backtype.storm.spout.SleepSpoutWaitStrategy topology.sleep.spout.wait.strategy.time.ms: 1 topology.error.throttle.interval.secs: 10 topology.max.error.report.per.interval: 5 topology.kryo.factory: backtype.storm.serialization.DefaultKryoFactory topology.tuple.serializer: backtype.storm.serialization.types.ListDelegateSerializer topology.trident.batch.emit.interval.millis: 500 #dev.zookeeper.path: /tmp/dev-storm-zookeeper
Re: java.lang.IllegalArgumentException: timeout value is negative - seen in Worker logs
This is my netty config, perhaps this will help? Swap yours for this, does the problem persist? Introduce your preferred values in a binary search pattern to track down the problem configuration item. Hopefully that specified below will get your topology working again.. storm.messaging.transport: backtype.storm.messaging.netty.Context storm.messaging.netty.buffer_size: 16384 storm.messaging.netty.max_retries: 10 storm.messaging.netty.min_wait_ms: 1000 storm.messaging.netty.max_wait_ms: 5000 D From: Binita Bharatimailto:binita.bhar...@gmail.com Sent: ?Wednesday?, ?2? ?April? ?2014 ?08?:?18 To: user@storm.incubator.apache.orgmailto:user@storm.incubator.apache.org The same issue is still observed in worker logs. Anyone has any other inputs ? Thanks On Tue, Apr 1, 2014 at 8:25 AM, Binita Bharati binita.bhar...@gmail.com wrote: Thanks , will give it a shot.. On Fri, Mar 28, 2014 at 9:27 AM, bijoy deb bijoy.comput...@gmail.com wrote: Hi, Try modifying storm.messaging.netty.max_retries: 100 to a lower value,e.g. storm.messaging.netty.max_retries: 10. Thanks Bijoy On Fri, Mar 28, 2014 at 8:48 AM, Binita Bharati binita.bhar...@gmail.com wrote: Hi all, Am using storm-0.9.0.1. The following error is seen in the worker logs: 2014-03-25 16:18:24 STDIO [ERROR] Mar 25, 2014 4:18:24 PM org.jboss.netty.channel.DefaultChannelPipeline WARNING: An exception was thrown by a user handler while handling an exception event ([id: 0x8068e4b0] EXCEPTION: java.net.ConnectException: Connection refused) java.lang.IllegalArgumentException: timeout value is negative at java.lang.Thread.sleep(Native Method) at backtype.storm.messaging.netty.Client.reconnect(Client.java:78) at backtype.storm.messaging.netty.StormClientHandler.exceptionCaught(StormClientHandler.java:108) at org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.java:377) at org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:525) at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:109) at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:78) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:41) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Relevant Netty config from storm.yaml: storm.messaging.transport: backtype.storm.messaging.netty.Context storm.messaging.netty.server_worker_threads: 1 storm.messaging.netty.client_worker_threads: 1 storm.messaging.netty.buffer_size: 5242880 storm.messaging.netty.max_retries: 100 storm.messaging.netty.max_wait_ms: 1000 storm.messaging.netty.min_wait_ms: 100 Anyone knows why ? Thanks Binita
java.lang.IllegalArgumentException: timeout value is negative - seen in Worker logs
Hi all, Am using storm-0.9.0.1. The following error is seen in the worker logs: 2014-03-25 16:18:24 STDIO [ERROR] Mar 25, 2014 4:18:24 PM org.jboss.netty.channel.DefaultChannelPipeline WARNING: An exception was thrown by a user handler while handling an exception event ([id: 0x8068e4b0] EXCEPTION: java.net.ConnectException: Connection refused) java.lang.IllegalArgumentException: timeout value is negative at java.lang.Thread.sleep(Native Method) at backtype.storm.messaging.netty.Client.reconnect(Client.java:78) at backtype.storm.messaging.netty.StormClientHandler.exceptionCaught(StormClientHandler.java:108) at org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.java:377) at org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:525) at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:109) at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:78) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:41) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Relevant Netty config from storm.yaml: storm.messaging.transport: backtype.storm.messaging.netty.Context storm.messaging.netty.server_worker_threads: 1 storm.messaging.netty.client_worker_threads: 1 storm.messaging.netty.buffer_size: 5242880 storm.messaging.netty.max_retries: 100 storm.messaging.netty.max_wait_ms: 1000 storm.messaging.netty.min_wait_ms: 100 Anyone knows why ? Thanks Binita
