On Thu, 2013-03-14 at 13:10 +0100, Arkadi Colson wrote: > When I shutdown tomcat free -m and top keeps telling me the same values. > Almost no free memory... > > Any idea?
Are you reading top & free right? It is standard behaviour for most modern operating systems to have very little free memory. As long as the sum of free memory and cache is high, everything is fine. Looking at the stats you gave previously we have > > *top* > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ > > COMMAND > > > > &nbs > > p; > > 13666 root 20 0 86.8g 4.7g 248m S 101 39.7 478:37.45 4.7GB physical memory used and ~80GB used for memory mapping the index. > > *free -m** > > * total used free shared buffers cached > > Mem: 12047 11942 105 0 180 6363 > > -/+ buffers/cache: 5399 6648 > > Swap: 956 75 881 So 6648MB used for either general disk cache or memory mapped index. This really translates to 6648MB (plus the 105MB above) available memory as any application asking for memory will get it immediately from that pool (sorry if this is basic stuff for you). > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > > at java.lang.Thread.run(Thread.java:662) > > Caused by: java.lang.OutOfMemoryError > > at java.util.zip.ZipFile.open(Native Method) > > at java.util.zip.ZipFile.<init>(ZipFile.java:127) > > at java.util.zip.ZipFile.<init>(ZipFile.java:144) > > at > > org.apache.poi.openxml4j.opc.internal.ZipHelper.openZipFile(ZipHelper.java:157) [...] > > Java HotSpot(TM) 64-Bit Server VM warning: Attempt to allocate stack > > guard pages failed. > > mmap failed for CEN and END part of zip file A quick search shows that other people have had problems with ZipFile in at least some sub-versions of Java 1.7. However, another very common cause for OOM with memory mapping is that the limit for allocating virtual memory is too low. Try doing a ulimit -v on the machine. If the number is somewhere around 100000000 (100GB), Lucene's memory mapping of your index (the 80GB) plus the ZipFile's memory mapping plus other processes might hit the ceiling. If that is the case, simply raise the limit. - Toke
