Seriously, at least try JVM argument -XX:+UseConcMarkSweepGC . That argument took care of very similar symptoms I was having. I never did figure out exactly what was causing them, but at some point I tried that JVM argument, and they went away never to come back (which I guess is a clue about what was causing the slowdown, but the JVM still confuses me). ________________________________________ From: Mason Hale [masonh...@gmail.com] Sent: Wednesday, October 27, 2010 9:03 PM To: solr-user@lucene.apache.org Subject: Re: Inconsistent slave performance after optimize
On Wed, Oct 27, 2010 at 7:18 PM, Ken Krugler <kkrugler_li...@transpac.com>wrote: > Normally I'd say like you were getting into swap hell, but based on your > settings you only have 5GB of JVM space being used, on a 16GB box. > > Just to confirm, nothing else is using lots of memory, right? And the "top" > command isn't showing any swap usage, right? > > Correct. Only thing of note running on this machine is Solr. I don't have a poor performing server on hand at the moment, but I recall checking top when it was tanking, and it was not showing any swap usage. > When you encounter very slow search times, what does the top command say > about system load and cpu vs. I/O percentages? > I did look at iostat -x when the server was running slow and IO util was 100%. This lead me to believe the problem was cache-warming related, and that data needed to be loaded into Solr caches and/or files loaded into the file system cache. Does that yield any additional clues. In this does happen again, what stats should I collect? (Note to self: need to install sar on these servers to collect historical performance data...) Mason
