Re: Lucene/Solr Filesystem tunings
Just to add to the pile...use the Deadline or NOOP I/O scheduler. -Z On Sat, Jun 8, 2013 at 4:40 PM, Mark Miller markrmil...@gmail.com wrote: Turning swappiness down to 0 can have some decent performance impact. - http://en.wikipedia.org/wiki/Swappiness In the past, I've seen better performance with ext3 over ext4 around commits/fsync. Test were actually enough slower (lots of these operations), that I made a special ext3 partition workspace for lucene/solr dev. (Still use ext4 for root and home). Have not checked that recently, and it may not be a large concern for many use cases. - Mark On Jun 4, 2013, at 6:48 PM, Tim Vaillancourt t...@elementspace.com wrote: Hey all, Does anyone have any advice or special filesytem tuning to share for Lucene/Solr, and which file systems they like more? Also, does Lucene/Solr care about access times if I turn them off (I think I doesn't care)? A bit unrelated: What are people's opinions on reducing some consistency things like filesystem journaling, etc (ext2?) due to SolrCloud's additional HA with replicas? How about RAID 0 x 3 replicas or so? Thanks! Tim Vaillancourt
Re: Lucene/Solr Filesystem tunings
Turning swappiness down to 0 can have some decent performance impact. - http://en.wikipedia.org/wiki/Swappiness In the past, I've seen better performance with ext3 over ext4 around commits/fsync. Test were actually enough slower (lots of these operations), that I made a special ext3 partition workspace for lucene/solr dev. (Still use ext4 for root and home). Have not checked that recently, and it may not be a large concern for many use cases. - Mark On Jun 4, 2013, at 6:48 PM, Tim Vaillancourt t...@elementspace.com wrote: Hey all, Does anyone have any advice or special filesytem tuning to share for Lucene/Solr, and which file systems they like more? Also, does Lucene/Solr care about access times if I turn them off (I think I doesn't care)? A bit unrelated: What are people's opinions on reducing some consistency things like filesystem journaling, etc (ext2?) due to SolrCloud's additional HA with replicas? How about RAID 0 x 3 replicas or so? Thanks! Tim Vaillancourt
Re: Lucene/Solr Filesystem tunings
I figured as much for atime, thanks Otis! I haven't ran benchmarks just yet, but I'll be sure to share whatever I find. I plan to try ext4 vs xfs. I am also curious what effect disabling journaling (ext2) would have, relying on SolrCloud to manage 'consistency' over many instances vs FS journaling. Anyone have opinions there? If I test I'll share the results. Cheers, Tim On 4 June 2013 16:11, Otis Gospodnetic otis.gospodne...@gmail.com wrote: Hi, You can use noatime, nodiratime, nothing in Solr depends on that as far as I know. We tend to use ext4. Some people love xfs. Want to run some benchmarks and publish the results? :) Otis -- Solr ElasticSearch Support http://sematext.com/ On Tue, Jun 4, 2013 at 6:48 PM, Tim Vaillancourt t...@elementspace.com wrote: Hey all, Does anyone have any advice or special filesytem tuning to share for Lucene/Solr, and which file systems they like more? Also, does Lucene/Solr care about access times if I turn them off (I think I doesn't care)? A bit unrelated: What are people's opinions on reducing some consistency things like filesystem journaling, etc (ext2?) due to SolrCloud's additional HA with replicas? How about RAID 0 x 3 replicas or so? Thanks! Tim Vaillancourt
Lucene/Solr Filesystem tunings
Hey all, Does anyone have any advice or special filesytem tuning to share for Lucene/Solr, and which file systems they like more? Also, does Lucene/Solr care about access times if I turn them off (I think I doesn't care)? A bit unrelated: What are people's opinions on reducing some consistency things like filesystem journaling, etc (ext2?) due to SolrCloud's additional HA with replicas? How about RAID 0 x 3 replicas or so? Thanks! Tim Vaillancourt
Re: Lucene/Solr Filesystem tunings
Hi, You can use noatime, nodiratime, nothing in Solr depends on that as far as I know. We tend to use ext4. Some people love xfs. Want to run some benchmarks and publish the results? :) Otis -- Solr ElasticSearch Support http://sematext.com/ On Tue, Jun 4, 2013 at 6:48 PM, Tim Vaillancourt t...@elementspace.com wrote: Hey all, Does anyone have any advice or special filesytem tuning to share for Lucene/Solr, and which file systems they like more? Also, does Lucene/Solr care about access times if I turn them off (I think I doesn't care)? A bit unrelated: What are people's opinions on reducing some consistency things like filesystem journaling, etc (ext2?) due to SolrCloud's additional HA with replicas? How about RAID 0 x 3 replicas or so? Thanks! Tim Vaillancourt
