Hi,
I'm a beginner in elasticsearch. I'm using also ubuntu for
elasticsearch(version 1.1.0). My issue : where should I set the
ES_MIN/MAX_SIZE(= ES_HEAP_SIZE)? In /etc/elasticsearch/elasticsearch.yml or
/etc/default/elasticsearch or /etc/init.d/elasticsearch? From what i read i
should set the ES_MIN/MAX_SIZE = ram / 2.
My application is reading data from a Sqs and pushing it to elasticseach.
This is done from 2 consumers each with a thread pool of 5 threads. That
means 10 threads who are inserting documents in elasticsearch. I have a
separate server for elasticsearch with 8gb of ram.
From my configuration : I set ES_MIN/MAX_SIZE = 4g(=ram / 2) only in
/etc/elasticsearch/elasticsearch.yml and restarting the elasticsearch. When
I call the htop, I see for elasticsearch user the next command which is
duplicated(..10 threads => more line like this one) :
/usr/lib/jvm/java-7-openjdk-amd64//bin/java -Xms256m -Xmx1g -Xss256k
-Djava.awt. Shouldn't it be -Xms2g -Xmx2g, because I set in
elasticsearch.yml 2g?
From what i read in your post, you set in "service wrapper configuration"
set.default.ES_HEAP_SIZE=8192.
First of all : which file is linked to "service wrapper configuration"
/etc/default/elasticsearch or /etc/init.d/elasticsearch?
Second : why you have different values in elasticsearch.yml ES_MIN_MEM:
5g ES_MAX_MEM: 5g than in "service wrapper configuration"
(ES_HEAP_SIZE=8192) ?
Third : where should i set the wrapper.java.additional.1 ... 15 and
wrapper.java.initmemory=%ES_HEAP_SIZE% ?
joi, 13 martie 2014, 13:46:01 UTC+2, Hicham Mallah a scris:
>
> Hello,
>
> I have been using elasticsearch on a ubuntu server for a year now, and
> everything was going great. I had an index of 150,000,000 entries of domain
> names, running small queries on it, just filtering by 1 term no sorting no
> wildcard nothing. Now we moved servers, I have now a CentOS 6 server, 32GB
> ram and running elasticserach but now we have 2 indices, of about 150
> million entries each 32 shards, still running the same queries on them
> nothing changed in the queries. But since we went online with the new
> server, I have to restart elasticsearch every 2 hours before OOM killer
> kills it.
>
> What's happening is that elasticsearch starts using memory till 50% then
> it goes back down to about 30% gradually then starts to go up again
> gradually and never goes back down.
>
> I have tried all the solutions I found on the net, I am a developer not a
> server admin.
>
> *I have these setting in my service wrapper configuration*
>
> set.default.ES_HOME=/home/elasticsearch
> set.default.ES_HEAP_SIZE=8192
> set.default.MAX_OPEN_FILES=65535
> set.default.MAX_LOCKED_MEMORY=10240
> set.default.CONF_DIR=/home/elasticsearch/conf
> set.default.WORK_DIR=/home/elasticsearch/tmp
> set.default.DIRECT_SIZE=4g
>
> # Java Additional Parameters
> wrapper.java.additional.1=-Delasticsearch-service
> wrapper.java.additional.2=-Des.path.home=%ES_HOME%
> wrapper.java.additional.3=-Xss256k
> wrapper.java.additional.4=-XX:+UseParNewGC
> wrapper.java.additional.5=-XX:+UseConcMarkSweepGC
> wrapper.java.additional.6=-XX:CMSInitiatingOccupancyFraction=75
> wrapper.java.additional.7=-XX:+UseCMSInitiatingOccupancyOnly
> wrapper.java.additional.8=-XX:+HeapDumpOnOutOfMemoryError
> wrapper.java.additional.9=-Djava.awt.headless=true
> wrapper.java.additional.10=-XX:MinHeapFreeRatio=40
> wrapper.java.additional.11=-XX:MaxHeapFreeRatio=70
> wrapper.java.additional.12=-XX:CMSInitiatingOccupancyFraction=75
> wrapper.java.additional.13=-XX:+UseCMSInitiatingOccupancyOnly
> wrapper.java.additional.15=-XX:MaxDirectMemorySize=4g
> # Initial Java Heap Size (in MB)
> wrapper.java.initmemory=%ES_HEAP_SIZE%
>
> *And these in elasticsearch.yml*
> ES_MIN_MEM: 5g
> ES_MAX_MEM: 5g
> #index.store.type=mmapfs
> index.cache.field.type: soft
> index.cache.field.max_size: 10000
> index.cache.field.expire: 10m
> index.term_index_interval: 256
> index.term_index_divisor: 5
>
> *java version: *
> java version "1.7.0_51"
> Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
> Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)
>
> *Elasticsearch version*
> "version" : {
> "number" : "1.0.0",
> "build_hash" : "a46900e9c72c0a623d71b54016357d5f94c8ea32",
> "build_timestamp" : "2014-02-12T16:18:34Z",
> "build_snapshot" : false,
> "lucene_version" : "4.6"
> }
>
> Using elastica PHP
>
>
> I have tried playing with values up and down to try to make it work, but
> nothing is changing.
>
> Please any help would be highly appreciated.
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/227cfc27-b44d-4804-af22-36b26cec0152%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
