Just started getting this error this morning. On startup the consumer trying to reload the kv-store changelog starts throwing this exception and keeps restarting and failing.
2015-11-20 10:16:19 [main] TaskStorageManager [DEBUG] Stopping consumers for stores. 2015-11-20 10:16:19 [main] BrokerProxy [INFO] Shutting down BrokerProxy for devstream03.chartbeat.net:9092 2015-11-20 10:16:19 [main] BrokerProxy [INFO] closing simple consumer... 2015-11-20 10:16:19 [main] DefaultFetchSimpleConsumer [DEBUG] Disconnecting from devstream03.chartbeat.net:9092 2015-11-20 10:16:19 [SAMZA-BROKER-PROXY-BrokerProxy thread pointed at devstream03.chartbeat.net:9092 for client samza_consumer-t_key_aggregator_dev-dev-1448032450060-10] DefaultFetchSimpleConsumer [INFO] Reconnect due to socket error: java.nio.channels.ClosedChannelException 2015-11-20 10:16:19 [SAMZA-BROKER-PROXY-BrokerProxy thread pointed at devstream03.chartbeat.net:9092 for client samza_consumer-t_key_aggregator_dev-dev-1448032450060-10] DefaultFetchSimpleConsumer [DEBUG] Disconnecting from devstream03.chartbeat.net:9092 2015-11-20 10:16:19 [SAMZA-BROKER-PROXY-BrokerProxy thread pointed at devstream03.chartbeat.net:9092 for client samza_consumer-t_key_aggregator_dev-dev-1448032450060-10] DefaultFetchSimpleConsumer [DEBUG] Disconnecting from devstream03.chartbeat.net:9092 2015-11-20 10:16:19 [SAMZA-BROKER-PROXY-BrokerProxy thread pointed at devstream03.chartbeat.net:9092 for client samza_consumer-t_key_aggregator_dev-dev-1448032450060-10] DefaultFetchSimpleConsumer [DEBUG] Disconnecting from devstream03.chartbeat.net:9092 2015-11-20 10:16:19 [SAMZA-BROKER-PROXY-BrokerProxy thread pointed at devstream03.chartbeat.net:9092 for client samza_consumer-t_key_aggregator_dev-dev-1448032450060-10] BrokerProxy [WARN] Restarting consumer due to java.nio.channels.ClosedChannelException. Releasing ownership of all partitions, and restarting consumer. Turn on debugging to get a full stack trace. 2015-11-20 10:16:19 [SAMZA-BROKER-PROXY-BrokerProxy thread pointed at devstream03.chartbeat.net:9092 for client samza_consumer-t_key_aggregator_dev-dev-1448032450060-10] BrokerProxy [DEBUG] Exception detail: java.nio.channels.ClosedChannelException at kafka.network.BlockingChannel.send(BlockingChannel.scala:100) at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:78) at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:68) at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SimpleConsumer.scala:112) at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:112) at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:112) at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33) at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(SimpleConsumer.scala:111) at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:111) at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:111) at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33) at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:110) at org.apache.samza.system.kafka.DefaultFetchSimpleConsumer.fetch(DefaultFetchSimpleConsumer.scala:48) at org.apache.samza.system.kafka.DefaultFetchSimpleConsumer.defaultFetch(DefaultFetchSimpleConsumer.scala:41) at org.apache.samza.system.kafka.BrokerProxy.org$apache$samza$system$kafka$BrokerProxy$$fetchMessages(BrokerProxy.scala:179) at org.apache.samza.system.kafka.BrokerProxy$$anon$1$$anonfun$run$1.apply(BrokerProxy.scala:147) at org.apache.samza.system.kafka.BrokerProxy$$anon$1$$anonfun$run$1.apply(BrokerProxy.scala:134) at org.apache.samza.util.ExponentialSleepStrategy.run(ExponentialSleepStrategy.scala:82) at org.apache.samza.system.kafka.BrokerProxy$$anon$1.run(BrokerProxy.scala:133) at java.lang.Thread.run(Thread.java:745)
signature.asc
Description: Message signed with OpenPGP using GPGMail