On Jun 9, 2005, at 11:52 PM, Chris Collins wrote:

In that case I have a different performance issue, that is that FSInputStream and FSOutputStream inherit the buffer size of 1k from OS and IS This would be useful to increase to reduce the amount of RPC's to the filer when doing merges ..... assuming that reads and writes are sequential (CIFS supports a 64k block and NFS supports upto I think 32k). I haven't spent much time on this so far so its not like I know its hard todo. From preliminary experiments its obvious
that changing the OS buffersize is not the thing todo.

If anyone has successfully increased the FSOutputStream and FSInputStream buffers and got it not to blow up on array copies I would love to know the
short cut.

I just started up with Lucene, and I have been looking at the NFS issues. Since the OS doesn't report the block size in use by the Netapp, EMC, whatever, you need to tweak it manually. I found this in src/java/org/apache/lucene/store/OutputStream.java:

/** Abstract class for output to a file in a Directory. A random- access output
* stream.  Used for all Lucene index output operations.
* @see Directory
* @see InputStream
*/
public abstract class OutputStream {
  static final int BUFFER_SIZE = 1024;

I changed that value to 8k, and based on the truss output from an index run, it is working. Haven't gotten much beyond that to see if it causes problems elsewhere. The value also needs to be altered on the read end of things. Ideally, this will be made settable via a system property.

Peter


---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Reply via email to