Re: Solr memory usage off-heap

2014-03-20 Thread Avishai Ish-Shalom
thanks! On Tue, Mar 18, 2014 at 4:37 PM, Erick Erickson erickerick...@gmail.comwrote: Avishai: It sounds like you already understand mmap. Even so you might be interested in this excellent writeup of MMapDirectory and Lucene by Uwe:

Solr memory usage off-heap

2014-03-18 Thread Avishai Ish-Shalom
Hi, My solr instances are configured with 10GB heap (Xmx) but linux shows resident size of 16-20GB. even with thread stack and permgen taken into account i'm still far off from these numbers. Could it be that jvm IO buffers take so much space? does lucene use JNI/JNA memory allocations?

RE: Solr memory usage off-heap

2014-03-18 Thread Doug Turnbull
-and-storage-for-lucenesolr/ Sent from my Windows Phone From: Avishai Ish-Shalom Sent: 3/18/2014 7:31 AM To: solr-user@lucene.apache.org Subject: Solr memory usage off-heap Hi, My solr instances are configured with 10GB heap (Xmx) but linux shows resident size of 16-20GB. even with thread stack and permgen

Re: Solr memory usage off-heap

2014-03-18 Thread Shawn Heisey
On 3/18/2014 5:30 AM, Avishai Ish-Shalom wrote: My solr instances are configured with 10GB heap (Xmx) but linux shows resident size of 16-20GB. even with thread stack and permgen taken into account i'm still far off from these numbers. Could it be that jvm IO buffers take so much space? does

Re: Solr memory usage off-heap

2014-03-18 Thread Erick Erickson
Avishai: It sounds like you already understand mmap. Even so you might be interested in this excellent writeup of MMapDirectory and Lucene by Uwe: http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html Best, Erick On Tue, Mar 18, 2014 at 7:23 AM, Avishai Ish-Shalom

Re: Solr memory usage off-heap

2014-03-18 Thread Shawn Heisey
On 3/18/2014 8:37 AM, Erick Erickson wrote: It sounds like you already understand mmap. Even so you might be interested in this excellent writeup of MMapDirectory and Lucene by Uwe: http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html There is some actual bad memory