Hi Alex, Thanks for your response earlier I can confirm that change output to elasticsearch_http solved the data insertion problem . However I think there is still some issues in ES threadpool queue .
Thanks Aamir On 14 February 2014 11:41, Alexander Reelsen <[email protected]> wrote: > Hey > > logstash ships with an elasticsearch 0.90 version included. You can try > changing the logstash output from 'elasticsearch' to 'elasticsearch_http', > which is using HTTP as communication protocol between elasticsearch and > logstash instead of the binary one (which needs to have the same versions > on both ends). > > > --Alex > > > On Fri, Feb 14, 2014 at 12:22 PM, Aamir Khan <[email protected]>wrote: > >> Hi, >> I yesterday moved my our cluster which have single node from 09.11 to 1.0 >> and I am getting following message from elasticsearch log >> >> [2014-02-14 11:14:40,528][WARN ][transport.netty ] [ >> mipLoggingCenter.mi-pay.com] Message not fully read (request) for [0] >> and action [], resetting >> [2014-02-14 11:14:41,907][WARN ][transport.netty ] [ >> mipLoggingCenter.mi-pay.com] Message not fully read (request) for [2] >> and action [], resetting >> [2014-02-14 11:14:43,429][WARN ][transport.netty ] [ >> mipLoggingCenter.mi-pay.com] Message not fully read (request) for [4] >> and action [], resetting >> [2014-02-14 11:14:44,439][WARN ][transport.netty ] [ >> mipLoggingCenter.mi-pay.com] Message not fully read (request) for [6] >> and action [], resetting >> >> >> On the logstash 1.3.3 side I am getting following messages >> log4j, [2014-02-14T11:14:40.576] WARN: >> org.elasticsearch.discovery.zen.ping.unicast: [Redeemer] failed to send >> ping to [[#zen_unicast_1#][inet[ >> mipLoggingCenter.mi-pay.com/10.3.57.34:9300]<http://mipLoggingCenter.mi-pay.com/10.3.57.34:9300%5D> >> ]] >> org.elasticsearch.transport.RemoteTransportException: Failed to >> deserialize exception response from stream >> Caused by: org.elasticsearch.transport.TransportSerializationException: >> Failed to deserialize exception response from stream >> at >> org.elasticsearch.transport.netty.MessageChannelHandler.handlerResponseError(MessageChannelHandler.java:169) >> at >> org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:123) >> at >> org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) >> at >> org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) >> at >> org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) >> at >> org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:296) >> at >> org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) >> at >> org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) >> at >> org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) >> at >> org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) >> at >> org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) >> at >> org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) >> at >> org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268) >> at >> org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255) >> at >> org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) >> at >> org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108) >> at >> org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318) >> at >> org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) >> at >> org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) >> at >> org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) >> at >> org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:724) >> Caused by: java.io.InvalidClassException: failed to read class descriptor >> at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601) >> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1515) >> at >> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1620) >> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1515) >> at >> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1620) >> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1515) >> at >> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1620) >> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1515) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1769) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) >> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) >> at >> org.elasticsearch.transport.netty.MessageChannelHandler.handlerResponseError(MessageChannelHandler.java:167) >> ... 23 more >> Caused by: java.lang.ClassNotFoundException: >> org.elasticsearch.ElasticsearchException >> at java.net.URLClassLoader$1.run(URLClassLoader.java:366) >> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >> at org.jruby.util.JRubyClassLoader.findClass(JRubyClassLoader.java:128) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >> at >> org.elasticsearch.common.io.ThrowableObjectInputStream.loadClass(ThrowableObjectInputStream.java:93) >> at >> org.elasticsearch.common.io.ThrowableObjectInputStream.readClassDescriptor(ThrowableObjectInputStream.java:67) >> at >> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1599) >> ... 34 more >> log4j, [2014-02-14T11:14:40.582] WARN: >> org.elasticsearch.transport.netty: [Redeemer] Message not fully read >> (response) for [0] handler >> org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$4@450da6b4, >> error [true], resetting >> log4j, [2014-02-14T11:14:41.906] WARN: >> org.elasticsearch.discovery.zen.ping.unicast: [Redeemer] failed to send >> ping to [[#zen_unicast_1#][inet[ >> mipLoggingCenter.mi-pay.com/10.3.57.34:9300]<http://mipLoggingCenter.mi-pay.com/10.3.57.34:9300%5D> >> ]] >> org.elasticsearch.transport.RemoteTransportException: Failed to >> deserialize exception response from stream >> Caused by: org.elasticsearch.transport.TransportSerializationException: >> Failed to deserialize exception response from stream >> at >> org.elasticsearch.transport.netty.MessageChannelHandler.handlerResponseError(MessageChannelHandler.java:169) >> at >> org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:123) >> at >> org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) >> at >> org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) >> at >> org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) >> at >> org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:296) >> at >> org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) >> at >> org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) >> at >> org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) >> at >> org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) >> at >> org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) >> at >> org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) >> at >> org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268) >> at >> org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255) >> at >> org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) >> at >> org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108) >> at >> org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318) >> at >> org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) >> at >> org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) >> at >> org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) >> at >> org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:724) >> Caused by: java.io.InvalidClassException: failed to read class descriptor >> at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601) >> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1515) >> at >> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1620) >> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1515) >> at >> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1620) >> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1515) >> at >> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1620) >> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1515) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1769) >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) >> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) >> at >> org.elasticsearch.transport.netty.MessageChannelHandler.handlerResponseError(MessageChannelHandler.java:167) >> ... 23 more >> Caused by: java.lang.ClassNotFoundException: >> org.elasticsearch.ElasticsearchException >> at java.net.URLClassLoader$1.run(URLClassLoader.java:366) >> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >> at org.jruby.util.JRubyClassLoader.findClass(JRubyClassLoader.java:128) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >> at >> org.elasticsearch.common.io.ThrowableObjectInputStream.loadClass(ThrowableObjectInputStream.java:93) >> at >> org.elasticsearch.common.io.ThrowableObjectInputStream.readClassDescriptor(ThrowableObjectInputStream.java:67) >> at >> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1599) >> ... 34 more >> >> >> >> One more thing I pointe out that same configuration working fine on 09.11 >> >> Thanks >> Aamir >> >> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/3e262fba-052a-465e-b464-88de74df48eb%40googlegroups.com >> . >> For more options, visit https://groups.google.com/groups/opt_out. >> > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/elasticsearch/bsepjwqN-vg/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/CAGCwEM-wnJz4pVo_BrgWct5g5FHMydyMzB8A5ZFJcz7R%2B7uysQ%40mail.gmail.com > . > > For more options, visit https://groups.google.com/groups/opt_out. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAA%3DQfXUYsCF%2B6NO%2BO7LunjLOZZ5ttaJrmJQdWaj%2BeB%2BTTN8GKw%40mail.gmail.com. For more options, visit https://groups.google.com/groups/opt_out.
