On 03/14/2013 03:11 PM, Toke Eskildsen wrote:
On Thu, 2013-03-14 at 13:10 +0100, Arkadi Colson wrote:
When I shutdown tomcat free -m and top keeps telling me the same values.
Almost no free memory...
Any idea?
Are you reading top & free right? It is standard behaviour for most
modern operating systems to have very little free memory. As long as the
sum of free memory and cache is high, everything is fine.
Looking at the stats you gave previously we have
*top*
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+
COMMAND
&nbs
p;
13666 root 20 0 86.8g 4.7g 248m S 101 39.7 478:37.45
4.7GB physical memory used and ~80GB used for memory mapping the index.
*free -m**
* total used free shared buffers cached
Mem: 12047 11942 105 0 180 6363
-/+ buffers/cache: 5399 6648
Swap: 956 75 881
So 6648MB used for either general disk cache or memory mapped index.
This really translates to 6648MB (plus the 105MB above) available memory
as any application asking for memory will get it immediately from that
pool (sorry if this is basic stuff for you).
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.OutOfMemoryError
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.<init>(ZipFile.java:127)
at java.util.zip.ZipFile.<init>(ZipFile.java:144)
at
org.apache.poi.openxml4j.opc.internal.ZipHelper.openZipFile(ZipHelper.java:157)
[...]
Java HotSpot(TM) 64-Bit Server VM warning: Attempt to allocate stack
guard pages failed.
mmap failed for CEN and END part of zip file
A quick search shows that other people have had problems with ZipFile in
at least some sub-versions of Java 1.7. However, another very common
cause for OOM with memory mapping is that the limit for allocating
virtual memory is too low.
We do not index zip files so that could not cause the problem
Try doing a
ulimit -v
on the machine. If the number is somewhere around 100000000 (100GB),
Lucene's memory mapping of your index (the 80GB) plus the ZipFile's
memory mapping plus other processes might hit the ceiling. If that is
the case, simply raise the limit.
- Toke
ulimit -v shows me unlimited
I decreased the hard commit time to 10 seconds and set ramBufferSizeMB
to 250. Hope this helps...
Will keep you informed!
Thanks for the explanation!
