Hi Alex,
Thank you for your comments.
You can see the curl command result (for security reason, ip addresses
replaced with ****). For all 3 nodes, bootstrap.mlockall is true.
Also doesn't matter on which node I run the command.
{"ok":true,"cluster_name":"****","nodes":{"rqptXnAMS7O-scqGJTIssQ":{"name":
"****.42:10000","transport_address":"inet[bdm01-test/****.42:9300]",
"hostname":"bdm01-test","version":"0.90.3","http_address":
"inet[/****.42:9200]","settings":{"path.home":"/home/nli/opt/bdm","pidfile":
"/home/nli/opt/bdm/pid/****.42:10000/elasticsearch.pid","path.logs":
"/home/nli/opt/bdm/log/elasticsearch/****.42:10000","path.work":
"/tmp/elasticsearch/****.42:10000","max-open-files":"true","path.data":
"/home/nli/data/elasticsearchdata","config":
"/home/nli/opt/bdm/config/****.42:10000.yml",
"discovery.zen.ping.unicast.hosts.2":"****.44:9300",
"discovery.zen.ping.unicast.hosts.1":"****.43:9300",
"gateway.recovery_after_time":"5m","discovery.zen.ping.unicast.hosts.0":
"****.42:9300","indices.ttl.interval":"86400s","bootstrap.mlockall":"true",
"cluster.name":"****","http.port":"9200","network.publish_host":"****.42",
"indices.memory.index_buffer_size":"20%",
"discovery.zen.minimum_master_nodes":"2","discovery.zen.fd.ping_timeout":
"60s","gateway.recovery_after_nodes":"3","gateway.expected_nodes":"3",
"discovery.zen.ping.timeout":"30s","indices.fielddata.cache.expire":"1m",
"discovery.zen.fd.ping_retries":"10","node.name":"****.42:10000",
"indices.fielddata.cache.size":"10%","action.auto_create_index":"false",
"transport.tcp.compress":"true","discovery.zen.ping.multicast.enabled":
"false","name":"****.42:10000"}},"g302MXTgRY2qG7OfTXbevQ":{"name":
"****.43:10000","transport_address":"inet[/****.43:9300]","hostname":
"bdm02-test","version":"0.90.3","http_address":"inet[/****.43:9200]",
"settings":{"path.home":"/home/nli/opt/bdm","pidfile":
"/home/nli/opt/bdm/pid/****.43:10000/elasticsearch.pid","path.logs":
"/home/nli/opt/bdm/log/elasticsearch/****.43:10000","path.work":
"/tmp/elasticsearch/****.43:10000","max-open-files":"true","path.data":
"/home/nli/data/elasticsearchdata","config":
"/home/nli/opt/bdm/config/****.43:10000.yml",
"discovery.zen.ping.unicast.hosts.2":"****.44:9300",
"discovery.zen.ping.unicast.hosts.1":"****.43:9300",
"gateway.recovery_after_time":"5m","discovery.zen.ping.unicast.hosts.0":
"****.42:9300","indices.ttl.interval":"86400s","bootstrap.mlockall":"true",
"cluster.name":"****","http.port":"9200","network.publish_host":"****.43",
"indices.memory.index_buffer_size":"20%",
"discovery.zen.minimum_master_nodes":"2","discovery.zen.fd.ping_timeout":
"60s","gateway.recovery_after_nodes":"3","gateway.expected_nodes":"3",
"discovery.zen.ping.timeout":"30s","indices.fielddata.cache.expire":"1m",
"discovery.zen.fd.ping_retries":"10","node.name":"****.43:10000",
"indices.fielddata.cache.size":"10%","action.auto_create_index":"false",
"transport.tcp.compress":"true","discovery.zen.ping.multicast.enabled":
"false","name":"****.43:10000"}},"WxvcZ71jSom2aklRu2LYjg":{"name":
"****.44:10000","transport_address":"inet[/****.44:9300]","hostname":
"bdm03-test","version":"0.90.3","http_address":"inet[/****.44:9200]",
"settings":{"path.home":"/home/nli/opt/bdm","pidfile":
"/home/nli/opt/bdm/pid/****.44:10000/elasticsearch.pid","path.logs":
"/home/nli/opt/bdm/log/elasticsearch/****.44:10000","path.work":
"/tmp/elasticsearch/****.44:10000","max-open-files":"true","path.data":
"/home/nli/data/elasticsearchdata","config":
"/home/nli/opt/bdm/config/****.44:10000.yml",
"discovery.zen.ping.unicast.hosts.2":"****.44:9300",
"discovery.zen.ping.unicast.hosts.1":"****.43:9300",
"gateway.recovery_after_time":"5m","discovery.zen.ping.unicast.hosts.0":
"****.42:9300","indices.ttl.interval":"86400s","bootstrap.mlockall":"true",
"cluster.name":"****","http.port":"9200","network.publish_host":"****.44",
"indices.memory.index_buffer_size":"20%",
"discovery.zen.minimum_master_nodes":"2","discovery.zen.fd.ping_timeout":
"60s","gateway.recovery_after_nodes":"3","gateway.expected_nodes":"3",
"discovery.zen.ping.timeout":"30s","indices.fielddata.cache.expire":"1m",
"discovery.zen.fd.ping_retries":"10","node.name":"****.44:10000",
"indices.fielddata.cache.size":"10%","action.auto_create_index":"false",
"transport.tcp.compress":"true","discovery.zen.ping.multicast.enabled":
"false","name":"****.44:10000"}}}}Enter code here...
Best,
Vahid
On Wednesday, February 12, 2014 2:45:46 PM UTC+1, Alexander Reelsen wrote:
>
> Hey
>
> minor question/just a wild guess: Could this one node (1gb is the default
> heap size) be a client node, where you didnot configure the mlockall
> setting as it is started inside of another java application? If the
> mlockall setting was successful, I highly doubt, that any of these
> processes swap.
>
> All the nodes connected to the cluster should show in the nodes info and
> have the mlockall setting set to true.
>
>
> --Alex
>
>
> On Tue, Feb 11, 2014 at 4:51 PM, Vahid <[email protected] <javascript:>>wrote:
>
>> Hi Alex,
>> thank you.
>>
>> I've run the command and also it shows that mlockall is set to true !
>>
>> On Tuesday, February 11, 2014 2:33:49 PM UTC+1, Alexander Reelsen wrote:
>>
>>> Hey,
>>>
>>> with recent elasticsearch versions (including newer 0.90), you can see
>>> if bootstrap.mlockall setting is really applied in the nodes info. So make
>>> sure setting it, was really successful.
>>>
>>> curl -XGET 'http://localhost:9200/_nodes' and search for mlockall,
>>> which must be set to true.
>>>
>>>
>>> --Alex
>>>
>>>
>>> On Tue, Feb 11, 2014 at 10:08 AM, Vahid <[email protected]> wrote:
>>>
>>>> Thank you Tony and Mark,
>>>>
>>>> atm I have no more information about the virtualization because it's
>>>> our customer systems, maybe later I can provide more information regarding
>>>> that.
>>>> Other processes are our java applications which use ES to index and
>>>> search data.
>>>>
>>>> From htop/top I can see that almost all the swap memory is used, and by
>>>> running a bash script I can see how much swap space is used by which
>>>> process and mosthly is used by ES.
>>>>
>>>>
>>>> On Monday, February 10, 2014 10:44:00 PM UTC+1, Mark Walkom wrote:
>>>>
>>>>> How are you seeing this, ie what monitoring are you using?
>>>>> Also, what is the "~3 gb other processes" exactly for?
>>>>>
>>>>> Regards,
>>>>> Mark Walkom
>>>>>
>>>>> Infrastructure Engineer
>>>>> Campaign Monitor
>>>>> email: [email protected]
>>>>> web: www.campaignmonitor.com
>>>>>
>>>>>
>>>>> On 11 February 2014 04:16, Vahid <[email protected]> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> ES is running on a ubuntu 64bit in VM environment with the following
>>>>>> memory configuration:
>>>>>>
>>>>>> 12 gb total memory
>>>>>> 5 gb elasticsearch
>>>>>> ~3 gb other processes
>>>>>> and 4 gb left for OS.
>>>>>>
>>>>>> ES cluster configured with 3 nodes.
>>>>>> In the elasticsearch.yml bootstrap.mlockall set to true. On one of
>>>>>> the nodes I can see that ES is using about 1gb of swap space.
>>>>>> Also no warn log printed on the ES log file and only one node has
>>>>>> this problem.
>>>>>>
>>>>>> Is there any settings missed there?
>>>>>>
>>>>>> Many thanks,
>>>>>>
>>>>>> Vahid
>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "elasticsearch" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to [email protected].
>>>>>>
>>>>>> To view this discussion on the web visit https://groups.google.com/d/
>>>>>> msgid/elasticsearch/e9953624-3f35-4b86-a779-ac58dd0e30ba%40goo
>>>>>> glegroups.com.
>>>>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>>>>
>>>>>
>>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "elasticsearch" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected].
>>>> To view this discussion on the web visit https://groups.google.com/d/
>>>> msgid/elasticsearch/791e40ad-c688-4de4-af78-c1f68c0d9569%
>>>> 40googlegroups.com.
>>>>
>>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>>
>>>
>>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/b284a9c3-bf09-42e3-8393-8683e9d1425e%40googlegroups.com
>> .
>>
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/aaae80a9-9ca2-4c65-8421-b69e226b42cd%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
