I also wanted to add that even on my interface it shows:
Your current configuration is 20,000,000 documents per index and a maximum
number of 20 indices.
On Wednesday, May 28, 2014 3:41:49 PM UTC-6, Tyler Bell wrote:
>
> Thanks Kay! I did upgrade from an older version of Graylog and
> ElasticSearch once v0.20.0 came out.
>
> I ran that job from the web interface, and it looks like it cycled through
> all my indexes with messages like:
>
> 2014-05-28 14:34:30,460 INFO : org.graylog2.system.jobs.SystemJobManager -
> Submitted SystemJob <da2267c5-e6af-11e3-a691-005056b274fe>
> [org.graylog2.indexer.ranges.RebuildIndexRangesJob]
> 2014-05-28 14:34:30,460 INFO :
> org.graylog2.indexer.ranges.RebuildIndexRangesJob - Re-calculating index
> ranges.
> 2014-05-28 14:34:30,729 INFO :
> org.graylog2.indexer.ranges.RebuildIndexRangesJob - Calculated range of
> [graylog2_56] in [239ms].
> 2014-05-28 14:34:30,959 INFO :
> org.graylog2.indexer.ranges.RebuildIndexRangesJob - Calculated range of
> [graylog2_55] in [229ms].
> 2014-05-28 14:34:31,158 INFO :
> org.graylog2.indexer.ranges.RebuildIndexRangesJob - Calculated range of
> [graylog2_54] in [199ms].
> 2014-05-28 14:34:31,345 INFO :
> org.graylog2.indexer.ranges.RebuildIndexRangesJob - Calculated range of
> [graylog2_53] in [187ms].
>
> .....etc, then when it hits graylog2_recent
>
> 2014-05-28 14:34:51,242 INFO :
> org.graylog2.indexer.ranges.RebuildIndexRangesJob - Could not calculate
> range of index [graylog2_recent]. Skipping.
> org.elasticsearch.action.search.SearchPhaseExecutionException: Failed to
> execute phase [query], all shards failed; shardFailures
> {[cjdL5h8tSo6VK3zE-Gpvxw][graylog2_recent][0]:
> RemoteTransportException[[GLNode1][inet[/:9300]][search/phase/query]];
> nested: SearchParseException[[graylog2_recent][0]:
> query[ConstantScore(*:*)],from[-1],size[1]: Parse Failure [Failed to parse
> source
> [{"size":1,"query":{"match_all":{}},"sort":[{"timestamp":{"order":"desc"}}]}]]];
>
> nested: SearchParseException[[graylog2_recent][0]:
> query[ConstantScore(*:*)],from[-1],size[1]: Parse Failure [No mapping found
> for [timestamp] in order to sort on]];
> }{[XB6yIWzpRHOa8wOJtuKe1g][graylog2_recent][3]:
> RemoteTransportException[[GLNode2][inet[/:9300]][search/phase/query]];
> nested: SearchParseException[[graylog2_recent][3]:
> query[ConstantScore(*:*)],from[-1],size[1]: Parse Failure [Failed to parse
> source
> [{"size":1,"query":{"match_all":{}},"sort":[{"timestamp":{"order":"desc"}}]}]]];
>
> nested: SearchParseException[[graylog2_recent][3]:
> query[ConstantScore(*:*)],from[-1],size[1]: Parse Failure [No mapping found
> for [timestamp] in order to sort on]];
> }{[XB6yIWzpRHOa8wOJtuKe1g][graylog2_recent][2]:
> RemoteTransportException[[GLNode2][inet[/:9300]][search/phase/query]];
> nested: SearchParseException[[graylog2_recent][2]:
> query[ConstantScore(*:*)],from[-1],size[1]: Parse Failure [Failed to parse
> source
> [{"size":1,"query":{"match_all":{}},"sort":[{"timestamp":{"order":"desc"}}]}]]];
>
> nested: SearchParseException[[graylog2_recent][2]:
> query[ConstantScore(*:*)],from[-1],size[1]: Parse Failure [No mapping found
> for [timestamp] in order to sort on]];
> }{[cjdL5h8tSo6VK3zE-Gpvxw][graylog2_recent][1]:
> RemoteTransportException[[GLNode1][inet[/:9300]][search/phase/query]];
> nested: SearchParseException[[graylog2_recent][1]:
> query[ConstantScore(*:*)],from[-1],size[1]: Parse Failure [Failed to parse
> source
> [{"size":1,"query":{"match_all":{}},"sort":[{"timestamp":{"order":"desc"}}]}]]];
>
> nested: SearchParseException[[graylog2_recent][1]:
> query[ConstantScore(*:*)],from[-1],size[1]: Parse Failure [No mapping found
> for [timestamp] in order to sort on]]; }
> at
> org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:272)
> at
> org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$3.onFailure(TransportSearchTypeAction.java:224)
> at
> org.elasticsearch.search.action.SearchServiceTransportAction$4.handleException(SearchServiceTransportAction.java:222)
> at
> org.elasticsearch.transport.netty.MessageChannelHandler.handleException(MessageChannelHandler.java:181)
> at
> org.elasticsearch.transport.netty.MessageChannelHandler.handlerResponseError(MessageChannelHandler.java:171)
> at
> org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:123)
> at
> org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> at
> org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at
> org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> at
> org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> at
> org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
> at
> org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
> at
> org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
> at
> org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> at
> org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at
> org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
> at
> org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> at
> org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> at
> org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
> at
> org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
> at
> org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
> at
> org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
> at
> org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
> at
> org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> at
> org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
>
> Then at the end:
>
> 2014-05-28 14:35:05,052 INFO :
> org.graylog2.indexer.ranges.RebuildIndexRangesJob - Done calculating index
> ranges for 172 indices. Took 34562ms.
> 2014-05-28 14:35:05,052 INFO : org.graylog2.system.jobs.SystemJobManager -
> SystemJob <da2267c5-e6af-11e3-a691-005056b274fe>
> [org.graylog2.indexer.ranges.RebuildIndexRangesJob] finished in 34591ms.
>
> What does that job actually do? Should those older indices be marked for
> deletion now?
>
>
> On Wednesday, May 28, 2014 3:13:33 PM UTC-6, Kay Röpke wrote:
>>
>> Hi!
>>
>> graylog2_recent sounds like you upgraded from an older version. Those
>> indices might not get collected correctly.
>>
>> Please try to use the "recalculate index ranges" from the action menu on
>> the system/indices page in the web interface. After that the server should
>> perform retention properly. If not, you can still manually clear out
>> unwanted indices, but be sure to perform that maintenance task from the
>> page.
>>
>> Best,
>> Kay
>> On May 28, 2014 11:04 PM, "Tyler Bell" <[email protected]> wrote:
>>
>>> Hi All - Using Graylog v0.20.1 and ES 0.90.10, and am having my disk
>>> space maxed out because indices are not rotating out. My elasticsearch data
>>> directory shows indices graylog2_recent, graylog2_0, ...., graylog2_173.
>>>
>>> Anyone have experience with this? I'm going to use the ES API to clear
>>> out some older indices and get my setup working again, but need to figure
>>> out the rotation issue for longterm resolution.
>>>
>>> I'm using default config settings:
>>>
>>> # Embedded elasticsearch configuration file
>>> # pay attention to the working directory of the server, maybe use an
>>> absolute path here
>>> elasticsearch_config_file = /etc/graylog2-elasticsearch.yml
>>>
>>> elasticsearch_max_docs_per_index = 20000000
>>>
>>> # How many indices do you want to keep?
>>> #
>>> elasticsearch_max_number_of_indices*elasticsearch_max_docs_per_index=total
>>> number of messages in your setup
>>> elasticsearch_max_number_of_indices = 20
>>>
>>> # Decide what happens with the oldest indices when the maximum number of
>>> indices is reached.
>>> # The following strategies are availble:
>>> # - delete # Deletes the index completely (Default)
>>> # - close # Closes the index and hides it from the system. Can be
>>> re-opened later.
>>> retention_strategy = delete
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "graylog2" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
--
You received this message because you are subscribed to the Google Groups
"graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
