[
https://issues.apache.org/jira/browse/LUCENE-753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12609916#action_12609916
]
Yonik Seeley commented on LUCENE-753:
-------------------------------------
bq. (clones are made & used up quickly and closed)
IIRC, clones are often not closed at all.
And for term expanding queries, you can get a *lot* of them all at once.
bq. And yes we cannot just open a new fd on demand if the file has been
deleted. But I'm thinking that may not matter in practice. Ie if the pool wants
to open a new fd, it can attempt to do so, and if the file was deleted it must
then return a shared access wrapper to the fd it already has open.
At first blush, sounds a bit too complex for the benefits.
- one would have to reserve the last fd for synchronized access... can't really
hand it out for unsynchronized exclusive access and then go and share it later.
- the shared access should use pread... not seek+read
bq. But on Linux I saw 44% speedup for serial=true case with 4 threads using
SeparateFile vs ChannelPread, which I was very surprised by.
In the serial case, there are half the system calls (no seek). When both
implementations have a single single system call, all the extra code and
complexity that Sun threw into FileChannel comes into play. Compare that with
RandomAccessFile.read() which drops down to a native call and presumably just
the read with little overhead. I wish Sun would just add a
RandomAccessFile.read with a file position.
If access will be truly serial sometimes, larger buffers would help with that
larger read() setup cost.
> Use NIO positional read to avoid synchronization in FSIndexInput
> ----------------------------------------------------------------
>
> Key: LUCENE-753
> URL: https://issues.apache.org/jira/browse/LUCENE-753
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Store
> Reporter: Yonik Seeley
> Attachments: FileReadTest.java, FileReadTest.java, FileReadTest.java,
> FileReadTest.java, FileReadTest.java, FileReadTest.java, FileReadTest.java,
> FSIndexInput.patch, FSIndexInput.patch, lucene-753.patch, lucene-753.patch
>
>
> As suggested by Doug, we could use NIO pread to avoid synchronization on the
> underlying file.
> This could mitigate any MT performance drop caused by reducing the number of
> files in the index format.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
