I think that using a pool of cloned inputStreams would be the best solution. I've implemented such a solution locally using two pools of 3 readers each (configurable via system properties) and will post the diff after I do some testing to confirm accuracy and speed improvements.
Could you also benchmark this against a version that clones new streams for each call? That sounds extravagant, but it removes a configuration parameter, always a good thing.
Sure. On some thought I expect them to perform about the same so I may just go with the simplier method if testing supports that thought.
One thing I saw while tracing back the clone() call is that InputStream.clone() has this remark:
Expert: Subclasses must ensure that clones may be positioned at different points in the input from each other and from the stream they were cloned from.
I'm not actually certain if that's the case for FSInputStream. As I see it I don't think the file variable in the FSInputStream class will be cloned correctly and will cause issues when cloned InputStreams are used.
From Object.clone():
... this method creates a new instance of the class of this object and initializes all its fields with exactly the contents of the corresponding fields of this object, as if by assignment; the contents of the fields are not themselves cloned. Thus, this method performs a "shallow copy" of this object, not a "deep copy" operation.
The solution I think is to recreate the FSInputStream.file object whenever a FSInputStream is cloned. I've attached what I think is a fix for the issue below.
diff -u -r1.20 FSDirectory.java --- src/java/org/apache/lucene/store/FSDirectory.java 29 May 2003 20:18:18 -0000 1.20 +++ src/java/org/apache/lucene/store/FSDirectory.java 15 Sep 2003 22:14:36 -0000 @@ -381,9 +381,11 @@
final class FSInputStream extends InputStream {
private class Descriptor extends RandomAccessFile {
+ File f = null;
public long position;
public Descriptor(File file, String mode) throws IOException {
super(file, mode);
+ this.f = file;
}
}@@ -431,6 +433,8 @@
public Object clone() {
FSInputStream clone = (FSInputStream)super.clone();
clone.isClone = true;
+ try { clone.file = new Descriptor(file.f, "r"); }
+ catch (IOException e) { /* umm? */ }
return clone;
}
}Regards,
Bruce Ritchie
smime.p7s
Description: S/MIME Cryptographic Signature
