Even more interesting is that using the head plugin, I can see that all the
settings on each server JVM seem to report that the index level settings
are what I want them to be, but they still are getting changed to some
other values...
- settings: {
- index: {
- codec: {
- bloom: {
- load: false
}
}
- number_of_replicas: 1
- number_of_shards: 6
- translog: {
- flush_threshold_size: 1GB
}
- search: {
- slowlog: {
- threshold: {
- fetch: {
- warn: 2s
- info: 1s
}
- index: {
- warn: 10s
- info: 5s
}
- query: {
- warn: 10s
- info: 5s
}
}
}
}
- refresh_interval: 60s
- merge: {
- scheduler: {
- type: concurrent
- max_thread_count: 1
}
- policy: {
- type: tiered
- max_merged_segment: 1gb
- max_merge_at_once_explicit: 4
- max_merge_at_once: 4
- segments_per_tier: 4
}
}
}
- bootstrap: {
- mlockall: true
}
-
[2015-01-13 02:12:52,062][INFO ][index.merge.policy ]
[elasticsearch-test] [test-20150113][1] updating [segments_per_tier] from
[4.0] to [10.0]
[2015-01-13 02:12:52,062][INFO ][index.merge.policy ]
[elasticsearch-test] [test-20150113][1] updating [max_merge_at_once] from
[4] to [10]
[2015-01-13 02:12:52,062][INFO ][index.merge.policy ]
[elasticsearch-test] [test-20150113][1] updating
[max_merge_at_once_explicit] from [4] to [30]
[2015-01-13 02:12:52,062][INFO ][index.merge.policy ]
[elasticsearch-test] [test-20150113][1] updating [max_merged_segment] from
[1024.0mb] to [5gb]
This is quite strange.
On Tue, Jan 13, 2015 at 1:21 PM, Chris Neal <[email protected]>
wrote:
> I hate to bump my own thread, but I still have not been able to figure
> this one out. Just wanted to put it in front of people's eyes again in
> hopes that someone might have an idea where these index settings are coming
> from.
>
> Thanks everyone.
>
> On Thu, Jan 8, 2015 at 3:51 AM, Chris Neal <[email protected]>
> wrote:
>
>> Thanks again Mark.
>>
>> Hopefully someone will chime in on this one. The default settings are
>> too much for the development environment I'm in, and causing some memory
>> issues. I'd really like to find out why the index settings are there, and
>> how to remove them!
>>
>> Chris
>>
>> On Tue, Jan 6, 2015 at 10:31 PM, Mark Walkom <[email protected]>
>> wrote:
>>
>>> Generally index settings on the cluster level are over written by
>>> settings specified at the index level which appears to be what is happening.
>>>
>>> Though I just checked a marvel mapping and I cannot see where that is
>>> set, so I'm not much more help there I am afraid, someone else should have
>>> a better idea.
>>>
>>> On 7 January 2015 at 13:59, Chris Neal <[email protected]> wrote:
>>>
>>>> I also found this very old post:
>>>>
>>>>
>>>> http://elasticsearch-users.115913.n3.nabble.com/How-to-set-max-merged-segment-at-startup-td3282574.html
>>>>
>>>> This thread talks about the configuration working when it is applied on
>>>> a master node when the index is created. I have the configuration on *all*
>>>> nodes in the cluster, master or not, and they are still being overwritten
>>>> by some mysterious method.
>>>>
>>>> I'm not sure if a 3 year old post is even still valid, but wanted to
>>>> mention it. I could always just move the configuration settings into the
>>>> index template, but I want to understand why the configuration file is not
>>>> applying, when I don't have any index specific settings!
>>>>
>>>> Again, thanks for reading :)
>>>> Chris
>>>>
>>>> On Tue, Jan 6, 2015 at 8:47 PM, Chris Neal <[email protected]>
>>>> wrote:
>>>>
>>>>> Thanks Mark.
>>>>>
>>>>> To my knowledge, I've never applied any index level settings. Any
>>>>> idea where they might be coming from? Even my "regular" ES cluster (I
>>>>> have
>>>>> Marvel on a separate cluster) is undoing my settings:
>>>>>
>>>>> [2015-01-06 23:58:34,991][INFO ][index.merge.policy ]
>>>>> [elasticsearch-ip-10-0-0-45] [test-20150107][0] updating
>>>>> [segments_per_tier] from [4.0] to [10.0]
>>>>> [2015-01-06 23:58:35,032][INFO ][index.merge.policy ]
>>>>> [elasticsearch-ip-10-0-0-45] [test-20150107][0] updating
>>>>> [max_merge_at_once] from [4] to [10]
>>>>> [2015-01-06 23:58:35,032][INFO ][index.merge.policy ]
>>>>> [elasticsearch-ip-10-0-0-45] [test-20150107][0] updating
>>>>> [max_merge_at_once_explicit] from [4] to [30]
>>>>> [2015-01-06 23:58:35,032][INFO ][index.merge.policy ]
>>>>> [elasticsearch-ip-10-0-0-45] [test-20150107][0] updating
>>>>> [max_merged_segment] from [1024.0mb] to [5gb]
>>>>>
>>>>> Querying the index settings doesn't show anything specific either:
>>>>>
>>>>> root@ip-10-0-0-45:bddevw07[1005]:~/elasticsearch> curl -XGET '
>>>>> http://localhost:9200/derbysoft-20150106/_settings'
>>>>>
>>>>> {"test-20150106":{"settings":{"index":{"creation_date":"1420502319287","uuid":"yuHSFauVTL-SwKVAwaRdCg","number_of_replicas":"1","number_of_shards":"3","version":{"created":"1040199"}}}}}
>>>>> root@ip-10-0-0-45:bddevw07[1006]:~/elasticsearch>
>>>>>
>>>>> I'm still stumped!
>>>>> Thanks so much for your time!
>>>>> Chris
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Jan 6, 2015 at 1:05 AM, Mark Walkom <[email protected]>
>>>>> wrote:
>>>>>
>>>>>> index level settings will override cluster level ones.
>>>>>>
>>>>>> On 6 January 2015 at 15:11, Chris Neal <[email protected]>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi all.
>>>>>>>
>>>>>>> My elasticsearch.yml file has these settings with regards to merging:
>>>>>>>
>>>>>>> index:
>>>>>>> codec:
>>>>>>> bloom:
>>>>>>> load: false
>>>>>>> merge:
>>>>>>> policy:
>>>>>>> max_merge_at_once: 4
>>>>>>> max_merge_at_once_explicit: 4
>>>>>>> max_merged_segment: 1gb
>>>>>>> segments_per_tier: 4
>>>>>>> type: tiered
>>>>>>> scheduler:
>>>>>>> max_thread_count: 1
>>>>>>> type: concurrent
>>>>>>> number_of_replicas: 0
>>>>>>> number_of_shards: 1
>>>>>>> refresh_interval: 5s
>>>>>>>
>>>>>>> My elasticsearch.log file on server startup shows me this:
>>>>>>>
>>>>>>> [2015-01-06 04:04:53,320][INFO ][cluster.metadata ]
>>>>>>> [elasticsearch-ip-10-0-0-42] [.marvel-2015.01.06] update_mapping
>>>>>>> [cluster_state] (dynamic)
>>>>>>> [2015-01-06 04:04:56,704][INFO ][index.merge.policy ]
>>>>>>> [elasticsearch-ip-10-0-0-42] [.marvel-2015.01.06][0] updating
>>>>>>> [segments_per_tier] from [4.0] to [10.0]
>>>>>>> [2015-01-06 04:04:56,704][INFO ][index.merge.policy ]
>>>>>>> [elasticsearch-ip-10-0-0-42] [.marvel-2015.01.06][0] updating
>>>>>>> [max_merge_at_once] from [4] to [10]
>>>>>>> [2015-01-06 04:04:56,704][INFO ][index.merge.policy ]
>>>>>>> [elasticsearch-ip-10-0-0-42] [.marvel-2015.01.06][0] updating
>>>>>>> [max_merge_at_once_explicit] from [4] to [30]
>>>>>>> [2015-01-06 04:04:56,704][INFO ][index.merge.policy ]
>>>>>>> [elasticsearch-ip-10-0-0-42] [.marvel-2015.01.06][0] updating
>>>>>>> [max_merged_segment] from [1024.0mb] to [5gb]
>>>>>>>
>>>>>>> It sure looks like ES is overriding what is in the .yml file to me,
>>>>>>> but I'm not sure why. Does anyone have an idea?
>>>>>>>
>>>>>>> Much appreciated!
>>>>>>> Chris
>>>>>>>
>>>>>>> --
>>>>>>> You received this message because you are subscribed to the Google
>>>>>>> Groups "elasticsearch" group.
>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>> send an email to [email protected].
>>>>>>> To view this discussion on the web visit
>>>>>>> https://groups.google.com/d/msgid/elasticsearch/CAND3DpjXD1XqRUDxQt5zOoG-ho%2BdRrUgY5ejKw9%2BXMy_sbYkLA%40mail.gmail.com
>>>>>>> <https://groups.google.com/d/msgid/elasticsearch/CAND3DpjXD1XqRUDxQt5zOoG-ho%2BdRrUgY5ejKw9%2BXMy_sbYkLA%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>>>>> .
>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "elasticsearch" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to [email protected].
>>>>>> To view this discussion on the web visit
>>>>>> https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_JdfH3SzG1OwYk4TH6z7q%2BZgFyA_qNzUjwOE8Xt959cw%40mail.gmail.com
>>>>>> <https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_JdfH3SzG1OwYk4TH6z7q%2BZgFyA_qNzUjwOE8Xt959cw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>>>> .
>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>
>>>>>
>>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "elasticsearch" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected].
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/elasticsearch/CAND3DpjEy5HFUMuCiynK7V_XkwOsA-t3oxo9TZg%3DzRUzQ%3Do0eQ%40mail.gmail.com
>>>> <https://groups.google.com/d/msgid/elasticsearch/CAND3DpjEy5HFUMuCiynK7V_XkwOsA-t3oxo9TZg%3DzRUzQ%3Do0eQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elasticsearch/CAEYi1X9ZAR%3DMxj4k%3DwMns1vF%3D7XT6-xSDtqBbp6saepfuxjkoA%40mail.gmail.com
>>> <https://groups.google.com/d/msgid/elasticsearch/CAEYi1X9ZAR%3DMxj4k%3DwMns1vF%3D7XT6-xSDtqBbp6saepfuxjkoA%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAND3Dpg6QvrGOhus3EzFK%2Bb60CHgYOiuVyOhx2gXM_1GfkFn0g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
