I ran out of memory on some big indexes when using solr 1.4. Found out that
increasing
termInfosIndexDivisor
in solrconfig.xml could help a lot.
It may slow down your searching your index.
cheers,
:-Dennis
On 02/06/2011, at 01.16, Alexey Serba wrote:
Hey Denis,
* How big is your
Hey Denis,
* How big is your index in terms of number of documents and index size?
5 cores, average 250.000 documents, one with about 1 million (but
without text, just int/float fields), one with about 10 million
id/name documents, but with n-gram.
Size: 4 databases about 1G (sum),
Commits are divided into 2 groups:
- often but small (last changed
info)
1) Make sure that it's not too often and you don't have commit
overlapping problem.
http://wiki.apache.org/solr/FAQ#What_does_.22PERFORMANCE_WARNING:_Overlapping_onDeckSearchers.3DX.22_mean_in_my_logs.3F
2) You may
My OS is also CentOS (5.4). If it were 10gb all the time it would be
ok, but it grows for 13-15gb, and hurts other services =\
It could be environment specific (specific of your top command
implementation, OS, etc)
I have on CentOS 2986m virtual memory showing although -Xmx2g
You have
Here is output after about 24 hours running solr. Maybe there is some
way to limit memory consumption? :(
test@d6 ~/solr/example $ java -Xms3g-Xmx6g-D64
-Dsolr.solr.home=/home/test/solr/example/multicore/ -jar start.jar
2011-05-31 17:05:14.265:INFO::Logging to STDERR via
Are you in fact out of swap space, as the java error suggested?
The way JVM's work always, if you tell it -Xmx6g, it WILL use all 6g
eventually. The JVM doesn't Garbage Collect until it's going to run out
of heap space, until it gets to your Xmx. It will keep using RAM until
it reaches your
So what should i do to evoid that error?
I can use 10G on server, now i try to run with flags:
java -Xms6G -Xmx6G -XX:MaxPermSize=1G -XX:PermSize=512M -D64
Or should i set xmx to lower numbers and what about other params?
Sorry, i don't know much about java/jvm =(
Wednesday, June 1, 2011,
There is no simple answer.
All I can say is you don't usually want to use an Xmx that's more than
you actually have available RAM, and _can't_ use more than you have
available ram+swap, and the Java error seems to be suggesting you are
using more than is available in ram+swap. That may not be
Overall memory on server is 24G, and 24G of swap, mostly all the time
swap is free and is not used at all, that's why no free swap sound
strange to me..
There is no simple answer.
All I can say is you don't usually want to use an Xmx that's more than
you actually have available RAM, and
PermSize and MaxPermSize don't need to be higher than 64M. You should read on
JVM tuning. The permanent generation is only used for the code that's being
executed.
So what should i do to evoid that error?
I can use 10G on server, now i try to run with flags:
java -Xms6G -Xmx6G
Could be related to your crazy high MaxPermSize like Marcus said.
I'm no JVM tuning expert either. Few people are, it's confusing. So if
you don't understand it either, why are you trying to throw in very
non-standard parameters you don't understand? Just start with whatever
the Solr example
There were no parameters at all, and java hitted out of memory
almost every day, then i tried to add parameters but nothing changed.
Xms/Xmx - did not solve the problem too. Now i try the MaxPermSize,
because it's the last thing i didn't try yet :(
Wednesday, June 1, 2011, 9:00:56 PM,
Hey Denis,
* How big is your index in terms of number of documents and index size?
* Is it production system where you have many search requests?
* Is there any pattern for OOM errors? I.e. right after you start your
Solr app, after some search activity or specific Solr queries, etc?
* What are
It could be environment specific (specific of your top command
implementation, OS, etc)
I have on CentOS 2986m virtual memory showing although -Xmx2g
You have 10g virtual although -Xmx6g
Don't trust it too much... top command may count OS buffers for opened
files, network sockets, JVM DLLs
14 matches
Mail list logo
