oh yes, my graylog2_deflector index was totally out of it, it had timestamp pegged as a string. Probably caused by elasticsearch running out of memory a while ago, and restarting seemed to fix it...
Well I deleted the whole index and now its all better. /Martin On Wednesday, 28 May 2014 09:15:19 UTC+2, Martin René Mortensen wrote: > > This problem persist after downgrading to 0.20.1 - something must have > corrupted the data Im thinking. or the graylog2-web service is asking > something wrong. Google mentions trying to create a histogram on a field > that is not a date, so I might just have corrupt data. > > On Tuesday, 27 May 2014 15:20:13 UTC+2, Martin René Mortensen wrote: >> >> After upgrading from 0.20.1 to 0.20.2 all my requests fail with http 500. >> Something with a cast. >> >> Help! >> >> /Martin >> >> <http://syslog/> >> >> - Search <http://syslog/> >> - Streams <http://syslog/streams> >> - Dashboards <http://syslog/dashboards> >> - Sources <http://syslog/sources> >> - System <http://syslog/system?page=0> >> - <http://syslog/system?page=0> >> >> *527* msg/s on 1 node. <http://syslog/logout> >> Oh no, something went wrong! >> >> (You caused a lib.APIException. API call failed GET http://@ >> syslog.adm.ku.dk:12900/search/universal/relative/histogram?interval=minute&query=*&range=300&range_type=relative&filter=*returned >> 500 Internal Server Error body: Failed to execute phase [query], >> all shards failed; shardFailures >> {[xspVvDzYT0Gy-e0nJucfxw][graylog2_120][2]: >> RemoteTransportException[[Demogoblin][inet[/130.225.127.90:9300]][search/phase/query]]; >> >> nested: SearchParseException[[graylog2_120][2]: >> query[ConstantScore(*:*)],from[-1],size[-1]: Parse Failure [Failed to parse >> source >> [{"query":{"query_string":{"query":"*","allow_leading_wildcard":false}},"facets":{"histogram":{"date_histogram":{"field":"timestamp","interval":"minute"},"facet_filter":{"bool":{"must":{"range":{"timestamp":{"from":"2014-05-27 >> >> 13:11:37.248","to":"2014-05-27 >> 13:16:37.248","include_lower":true,"include_upper":true}}}}}}}}]]]; nested: >> ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData >> >> cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]; >> }{[xspVvDzYT0Gy-e0nJucfxw][graylog2_120][0]: >> RemoteTransportException[[Demogoblin][inet[/130.225.127.90:9300]][search/phase/query]]; >> >> nested: SearchParseException[[graylog2_120][0]: >> query[ConstantScore(*:*)],from[-1],size[-1]: Parse Failure [Failed to parse >> source >> [{"query":{"query_string":{"query":"*","allow_leading_wildcard":false}},"facets":{"histogram":{"date_histogram":{"field":"timestamp","interval":"minute"},"facet_filter":{"bool":{"must":{"range":{"timestamp":{"from":"2014-05-27 >> >> 13:11:37.248","to":"2014-05-27 >> 13:16:37.248","include_lower":true,"include_upper":true}}}}}}}}]]]; nested: >> ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData >> >> cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]; >> }{[xspVvDzYT0Gy-e0nJucfxw][graylog2_120][1]: >> RemoteTransportException[[Demogoblin][inet[/130.225.127.90:9300]][search/phase/query]]; >> >> nested: SearchParseException[[graylog2_120][1]: >> query[ConstantScore(*:*)],from[-1],size[-1]: Parse Failure [Failed to parse >> source >> [{"query":{"query_string":{"query":"*","allow_leading_wildcard":false}},"facets":{"histogram":{"date_histogram":{"field":"timestamp","interval":"minute"},"facet_filter":{"bool":{"must":{"range":{"timestamp":{"from":"2014-05-27 >> >> 13:11:37.248","to":"2014-05-27 >> 13:16:37.248","include_lower":true,"include_upper":true}}}}}}}}]]]; nested: >> ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData >> >> cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]; >> }{[xspVvDzYT0Gy-e0nJucfxw][graylog2_120][3]: >> RemoteTransportException[[Demogoblin][inet[/130.225.127.90:9300]][search/phase/query]]; >> >> nested: SearchParseException[[graylog2_120][3]: >> query[ConstantScore(*:*)],from[-1],size[-1]: Parse Failure [Failed to parse >> source >> [{"query":{"query_string":{"query":"*","allow_leading_wildcard":false}},"facets":{"histogram":{"date_histogram":{"field":"timestamp","interval":"minute"},"facet_filter":{"bool":{"must":{"range":{"timestamp":{"from":"2014-05-27 >> >> 13:11:37.248","to":"2014-05-27 >> 13:16:37.248","include_lower":true,"include_upper":true}}}}}}}}]]]; nested: >> ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData >> >> cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]; >> } org.elasticsearch.action.search.SearchPhaseExecutionException: Failed to >> execute phase [query], all shards failed; shardFailures >> {[xspVvDzYT0Gy-e0nJucfxw][graylog2_120][2]: >> RemoteTransportException[[Demogoblin][inet[/130.225.127.90:9300]][search/phase/query]]; >> >> nested: SearchParseException[[graylog2_120][2]: >> query[ConstantScore(*:*)],from[-1],size[-1]: Parse Failure [Failed to parse >> source >> [{"query":{"query_string":{"query":"*","allow_leading_wildcard":false}},"facets":{"histogram":{"date_histogram":{"field":"timestamp","interval":"minute"},"facet_filter":{"bool":{"must":{"range":{"timestamp":{"from":"2014-05-27 >> >> 13:11:37.248","to":"2014-05-27 >> 13:16:37.248","include_lower":true,"include_upper":true}}}}}}}}]]]; nested: >> ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData >> >> cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]; >> }{[xspVvDzYT0Gy-e0nJucfxw][graylog2_120][0]: >> RemoteTransportException[[Demogoblin][inet[/130.225.127.90:9300]][search/phase/query]]; >> >> nested: SearchParseException[[graylog2_120][0]: >> query[ConstantScore(*:*)],from[-1],size[-1]: Parse Failure [Failed to parse >> source >> [{"query":{"query_string":{"query":"*","allow_leading_wildcard":false}},"facets":{"histogram":{"date_histogram":{"field":"timestamp","interval":"minute"},"facet_filter":{"bool":{"must":{"range":{"timestamp":{"from":"2014-05-27 >> >> 13:11:37.248","to":"2014-05-27 >> 13:16:37.248","include_lower":true,"include_upper":true}}}}}}}}]]]; nested: >> ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData >> >> cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]; >> }{[xspVvDzYT0Gy-e0nJucfxw][graylog2_120][1]: >> RemoteTransportException[[Demogoblin][inet[/130.225.127.90:9300]][search/phase/query]]; >> >> nested: SearchParseException[[graylog2_120][1]: >> query[ConstantScore(*:*)],from[-1],size[-1]: Parse Failure [Failed to parse >> source >> [{"query":{"query_string":{"query":"*","allow_leading_wildcard":false}},"facets":{"histogram":{"date_histogram":{"field":"timestamp","interval":"minute"},"facet_filter":{"bool":{"must":{"range":{"timestamp":{"from":"2014-05-27 >> >> 13:11:37.248","to":"2014-05-27 >> 13:16:37.248","include_lower":true,"include_upper":true}}}}}}}}]]]; nested: >> ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData >> >> cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]; >> }{[xspVvDzYT0Gy-e0nJucfxw][graylog2_120][3]: >> RemoteTransportException[[Demogoblin][inet[/130.225.127.90:9300]][search/phase/query]]; >> >> nested: SearchParseException[[graylog2_120][3]: >> query[ConstantScore(*:*)],from[-1],size[-1]: Parse Failure [Failed to parse >> source >> [{"query":{"query_string":{"query":"*","allow_leading_wildcard":false}},"facets":{"histogram":{"date_histogram":{"field":"timestamp","interval":"minute"},"facet_filter":{"bool":{"must":{"range":{"timestamp":{"from":"2014-05-27 >> >> 13:11:37.248","to":"2014-05-27 >> 13:16:37.248","include_lower":true,"include_upper":true}}}}}}}}]]]; nested: >> ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData >> >> cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]; >> } at >> org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:272) >> >> at >> org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$3.onFailure(TransportSearchTypeAction.java:224) >> >> at >> org.elasticsearch.search.action.SearchServiceTransportAction$4.handleException(SearchServiceTransportAction.java:222) >> >> at >> org.elasticsearch.transport.netty.MessageChannelHandler.handleException(MessageChannelHandler.java:181) >> >> at >> org.elasticsearch.transport.netty.MessageChannelHandler.handlerResponseError(MessageChannelHandler.java:171) >> >> at >> org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:123) >> >> at >> org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) >> >> at >> org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) >> >> at >> org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) >> >> at >> org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:296) >> >> at >> org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) >> >> at >> org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) >> >> at >> org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) >> >> at >> org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) >> >> at >> org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) >> >> at >> org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) >> >> at >> org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268) >> >> at >> org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255) >> >> at >> org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) >> >> at >> org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108) >> >> at >> org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318) >> >> at >> org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) >> >> at >> org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) >> >> at >> org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) >> >> at >> org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) >> >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> >> at java.lang.Thread.run(Thread.java:744) ) >> >> *Reason:* There was a problem with your search. We expected HTTP 200, >> but got a HTTP 500. >> >> Need help?Do not hesitate to consult the Graylog2 community if your >> questions are not answered in the >> documentation<http://support.torch.sh/help/kb> >> . >> >> - Forum / Mailing >> list<http://support.torch.sh/help/kb/general/forums-mailing-list> >> - Issue trackers<http://support.torch.sh/help/kb/general/issue-trackers> >> - Commercial support <http://www.torch.sh/> >> >> Stacktrace >> >> - lib.ApiClientImpl$ApiRequestBuilder#execute ( >> *ApiClientImpl.java:372*) >> - models.UniversalSearch#dateHistogram (*UniversalSearch.java:156*) >> - controllers.SearchController#index (*SearchController.java:109*) >> - Routes$$anonfun$routes$1$$anonfun$applyOrElse$7$$anonfun$apply$19#apply >> (*routes_routing.scala:693*) >> - Routes$$anonfun$routes$1$$anonfun$applyOrElse$7$$anonfun$apply$19#apply >> (*routes_routing.scala:693*) >> - play.core.Router$HandlerInvoker$$anon$7$$anon$2#invocation ( >> *Router.scala:183*) >> - play.core.Router$Routes$$anon$1#invocation (*Router.scala:377*) >> - play.core.j.JavaAction$$anon$1#call (*JavaAction.scala:56*) >> - play.GlobalSettings$1#call (*GlobalSettings.java:64*) >> - play.mvc.Security$AuthenticatedAction#call (*Security.java:45*) >> - play.core.j.JavaAction$$anon$3#apply (*JavaAction.scala:91*) >> - play.core.j.JavaAction$$anon$3#apply (*JavaAction.scala:90*) >> - play.core.j.FPromiseHelper$$anonfun$flatMap$1#apply ( >> *FPromiseHelper.scala:82*) >> - play.core.j.FPromiseHelper$$anonfun$flatMap$1#apply ( >> *FPromiseHelper.scala:82*) >> - scala.concurrent.Future$$anonfun$flatMap$1#apply (*Future.scala:251* >> ) >> - scala.concurrent.Future$$anonfun$flatMap$1#apply (*Future.scala:249* >> ) >> - scala.concurrent.impl.CallbackRunnable#run (*Promise.scala:32*) >> - play.core.j.HttpExecutionContext$$anon$2#run ( >> *HttpExecutionContext.scala:37*) >> - akka.dispatch.TaskInvocation#run (*AbstractDispatcher.scala:42*) >> - akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask#exec ( >> *AbstractDispatcher.scala:386*) >> - scala.concurrent.forkjoin.ForkJoinTask#doExec ( >> *ForkJoinTask.java:260*) >> - scala.concurrent.forkjoin.ForkJoinPool$WorkQueue#runTask ( >> *ForkJoinPool.java:1339*) >> - scala.concurrent.forkjoin.ForkJoinPool#runWorker ( >> *ForkJoinPool.java:1979*) >> - scala.concurrent.forkjoin.ForkJoinWorkerThread#run ( >> *ForkJoinWorkerThread.java:107*) >> >> Request informationMethodGETQuery >> >> - relative=300 >> - to= >> - q= >> - fields= >> - from= >> - rangetype=relative >> >> X-Forwarded-HostsyslogRefererhttp://syslog/X-Forwarded-For10.76.7.39 >> X-Forwarded-Serversyslog.adm.ku.dkConnectionKeep-AliveAccept >> text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 >> Accept-Languageen-US,en;q=0.8,da;q=0.6Accept-Encodinggzip,deflate,sdch >> User-AgentMozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, >> like Gecko) Chrome/34.0.1847.137 Safari/537.36Host130.225.127.90:9000 >> > -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
