Hi Shawn,
Wow! Thank you for your considered reply!
I'm going to dig into these issues, but I have a few questions:
Regarding memory: Including duplicate data in shard replicas the entire
index is 350GB. Each server hosts a total of 44GB of data. Each server has
28GB of memory. I haven't been
On 11/22/2013 8:13 AM, Dave Seltzer wrote:
Regarding memory: Including duplicate data in shard replicas the entire
index is 350GB. Each server hosts a total of 44GB of data. Each server has
28GB of memory. I haven't been setting -Xmx or -Xms, in the hopes that Java
would take the memory it needs
On 11/22/2013 10:01 AM, Shawn Heisey wrote:
You can see how much the max heap is in the Solr admin UI dashboard -
it'll be the right-most number on the JVM-Memory graph. On my 64-bit
linux development machine with 16GB of RAM, it looks like Java
defaults to a 4GB max heap. I have the heap
Thanks so much Shawn,
I think you (and others) are completely right about this being heap and GC
related. I just did a test while not indexing data and the same periodic
slowness was observable.
On to GC/Memory Tuning!
Many Thanks!
-Dave
On Fri, Nov 22, 2013 at 12:09 PM, Shawn Heisey
You mentioned earlier that you are not setting -Xms/-Xmx; the values actually
in use would then depend on the Java version, whether you're running 32- or
64-bit Java, whether Java thinks your machines are servers, and whether you
have specified the -server flag – and possibly a few other
Wow. That is one noisy command!
Full output is below. The grepped output looks like:
[solr@searchtest07 ~]$ java -XX:+PrintFlagsFinal -version | grep -i -E
'heapsize|permsize|version'
uintx AdaptivePermSizeWeight= 20
{product}
uintx ErgoHeapSizeLimit
So I made a few changes, but I still seem to be dealing with this pesky
periodic slowness.
Changes:
1) I'm now only forcing commits every 5 minutes. This was done by
specifying commitWithin=30 when doing document adds.
2) I'm specifying an -Xmx12g to force the java heap to take more memory
3)
On 11/22/2013 2:17 PM, Dave Seltzer wrote:
So I made a few changes, but I still seem to be dealing with this pesky
periodic slowness.
Changes:
1) I'm now only forcing commits every 5 minutes. This was done by
specifying commitWithin=30 when doing document adds.
2) I'm specifying an -Xmx12g
How real time is NRT? In particular, what are you commit settings?
And can you characterize periodic slowness? Queries that usually
take 500ms not tail 10s? Or 1s? How often? How are you measuring?
Details matter, a lot...
Best,
Erick
On Thu, Nov 21, 2013 at 6:03 PM, Dave Seltzer
Yes, more details…
Solr version, which garbage collector, how does heap usage look, cpu, etc.
- Mark
On Nov 21, 2013, at 6:46 PM, Erick Erickson erickerick...@gmail.com wrote:
How real time is NRT? In particular, what are you commit settings?
And can you characterize periodic slowness?
Lots of questions. Okay.
In digging a little deeper and looking at the config I see that
nrtModetrue/nrtMode is commented out. I believe this is the default
setting. So I don't know if NRT is enabled or not. Maybe just a red herring.
I don't know what Garbage Collector we're using. In this test
: Periodic Slowness on Solr Cloud
Lots of questions. Okay.
In digging a little deeper and looking at the config I see that
nrtModetrue/nrtMode is commented out. I believe this is the default
setting. So I don't know if NRT is enabled or not. Maybe just a red herring.
I don't know what Garbage
.
Doug
Sent from my Windows Phone From: Dave Seltzer
Sent: 11/21/2013 8:42 PM
To: solr-user@lucene.apache.org
Subject: Re: Periodic Slowness on Solr Cloud
Lots of questions. Okay.
In digging a little deeper and looking at the config I see that
nrtModetrue/nrtMode is commented out. I believe
: Re: Periodic Slowness on Solr Cloud
Lots of questions. Okay.
In digging a little deeper and looking at the config I see that
nrtModetrue/nrtMode is commented out. I believe this is the default
setting. So I don't know if NRT is enabled or not. Maybe just a red
herring.
I don't know what
On 11/21/2013 6:41 PM, Dave Seltzer wrote:
In digging a little deeper and looking at the config I see that
nrtModetrue/nrtMode is commented out. I believe this is the default
setting. So I don't know if NRT is enabled or not. Maybe just a red herring.
I had never seen this setting before.
15 matches
Mail list logo
