Perfectly safe to use SimpleFSDirectory. When you measure the
performance, are you measuring a lot of concurrent requests? That's
where you should see the win.
- Mark
On 5/1/10 6:38 PM, John Wang wrote:
We are seeing this issue as well in your production. (using Zoie on top
of lucene 2.9.1)
After some performance comparisons, we do NOT see performance gain with
NIO, rather these nasty ClosedChannelExceptions.
I think the performance gains ppl are seeing with 2.9.1 can be due to
many different things. From what we seen, they are not related to
NIOFSDirectory.
Our solution is to avoid calling FSDirectory.open(), instead just call
new SimpleFSDirectory(). Is this safe?
-John
On Fri, Jan 29, 2010 at 12:32 PM, Mark Miller <markrmil...@gmail.com
<mailto:markrmil...@gmail.com>> wrote:
Perhaps - one of the things they are supposed to be addressing is
extendability.
nio2 does have FileSystemProvider, which would actually allow you to
create a custom channel !
I have not dug in enough to know much more than that though.
*But*, another really interesting thing is that in Java 7,
FileDescriptors are ref counted ! (though users can't inc/dec).
But, FileInputStream and OutputStream have a new constructor that takes
a FileDescriptor.
So possibly, you could just make one that sits around to keep the
FileDescriptor valid, and get your channel off
FileInputStream/FileOutputStream?
And then if it goes down, make a new one using the FileDescriptor which
was not actually closed because there was a still a ref to it.
Possibly .... ;)
Michael McCandless wrote:
> Does anyone know if nio2 has improved this...?
>
> Mike
>
> On Fri, Jan 29, 2010 at 2:00 PM, Jason Rutherglen
> <jason.rutherg...@gmail.com <mailto:jason.rutherg...@gmail.com>>
wrote:
>
>> Defaulting NIOFSDir could account for some of the recent speed
>> improvements users have been reporting in Lucene 2.9. So
removing it
>> as a default could reverse those and people could then report Lucene
>> 3.X has slowed...
>>
>> On Thu, Jan 28, 2010 at 5:24 AM, Michael McCandless
>> <luc...@mikemccandless.com <mailto:luc...@mikemccandless.com>>
wrote:
>>
>>> Bummer.
>>>
>>> So the only viable workarounds are 1) don't use
Thread.interrupt (nor,
>>> things like Future.cancel, which in turn use Thread.interrupt) with
>>> NIOFSDir, or 2) we fix NIOFSDir to reopen the channel AND the
app must
>>> make a deletion policy that keeps a commit alive if any reader is
>>> using it. Or, 3) don't use NIOFSDir!
>>>
>>> Mike
>>>
>>> On Thu, Jan 28, 2010 at 7:29 AM, Simon Willnauer
>>> <simon.willna...@googlemail.com
<mailto:simon.willna...@googlemail.com>> wrote:
>>>
>>>> On Thu, Jan 28, 2010 at 12:43 PM, Michael McCandless
>>>> <luc...@mikemccandless.com <mailto:luc...@mikemccandless.com>>
wrote:
>>>>
>>>>> On Thu, Jan 28, 2010 at 6:38 AM, Uwe Schindler
<u...@thetaphi.de <mailto:u...@thetaphi.de>> wrote:
>>>>>
>>>>>
>>>>>> So I checked the code of NIOFSIndexInput, my last comment
was not really correct:
>>>>>> NIOFSIndexInput extends SimpleFSIndexInput and that opens
the RAF. In the ctor RAF.getChannel() is called. The RAF keeps open
until the file is closed (and also the channel).
>>>>>>
>>>>>> So it's really simple to fix in my opinion, just call
getChannel() again on this exception. Because the RAF should still
be open?
>>>>>>
>>>> Short answer:
>>>> public final FileChannel getChannel() {
>>>> synchronized (this) {
>>>> if (channel == null)
>>>> channel = FileChannelImpl.open(fd, true, rw, this);
>>>> return channel;
>>>> }
>>>> }
>>>>
>>>> this is not gonna work I tried it before. The RandomAccessFile
buffers
>>>> the channel!!
>>>>
>>>> simon
>>>>
>>>>> I think we need a definitive answer on what happens to the
RAF when
>>>>> the FileChannel was closed by Thread.Interrupt. Simon can
you test
>>>>> this?
>>>>>
>>>>> Mike
>>>>>
>>>>>
---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail:
java-dev-unsubscr...@lucene.apache.org
<mailto:java-dev-unsubscr...@lucene.apache.org>
>>>>> For additional commands, e-mail:
java-dev-h...@lucene.apache.org <mailto:java-dev-h...@lucene.apache.org>
>>>>>
>>>>>
>>>>>
>>>>
---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
<mailto:java-dev-unsubscr...@lucene.apache.org>
>>>> For additional commands, e-mail:
java-dev-h...@lucene.apache.org <mailto:java-dev-h...@lucene.apache.org>
>>>>
>>>>
>>>>
>>>
---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
<mailto:java-dev-unsubscr...@lucene.apache.org>
>>> For additional commands, e-mail:
java-dev-h...@lucene.apache.org <mailto:java-dev-h...@lucene.apache.org>
>>>
>>>
>>>
>>
---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
<mailto:java-dev-unsubscr...@lucene.apache.org>
>> For additional commands, e-mail: java-dev-h...@lucene.apache.org
<mailto:java-dev-h...@lucene.apache.org>
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
<mailto:java-dev-unsubscr...@lucene.apache.org>
> For additional commands, e-mail: java-dev-h...@lucene.apache.org
<mailto:java-dev-h...@lucene.apache.org>
>
>
--
- Mark
http://www.lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
<mailto:java-dev-unsubscr...@lucene.apache.org>
For additional commands, e-mail: java-dev-h...@lucene.apache.org
<mailto:java-dev-h...@lucene.apache.org>
--
- Mark
http://www.lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
