Michael, Actually, one more thing - you said you changed the store/BufferedIndexOutput.BUFFER_SIZE from 1024 to 4096 and that turned out to yield the fastest indexing. Does your FS block size also happen to be 4k (dumpe2fs output) on that FC3 box? If so, I wonder if this is more than just a coincidence...
Thanks, Otis ----- Original Message ---- From: Michael D. Curtin <[EMAIL PROTECTED]> To: java-user@lucene.apache.org Sent: Fri 10 Feb 2006 05:05:07 PM EST Subject: Re: Performance and FS block size Otis Gospodnetic wrote: > Hi, > > I'm wondering if anyone has tested Lucene indexing/search performance with > different file system block sizes? > > I just realized one of the servers where I run a lot of Lucene indexing and > searching has an FS with blocks of only 1K in size (typically they are 4k or > 8k, I believe), so I started wondering what's better for Lucene - smaller or > larger blocks? I have a feeling 1K is too small, although I don't know > enough to back up this feeling. :( On my system (dual Xeon with a couple 120GB S-ATA drives (not RAIDed), running Fedora Core 3) I changed BUFFER_SIZE in storage/OutputStream.java to 4096, achieving about 30% better performance in indexing. The search improvement was smaller, enough smaller that it was on order what I thought my measurement error was. I tried values up to 64K, but there wasn't much change on my system after 4K. --MDC --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
