Yep, will do so tonight. I saw the code in RAMDirectory and couldn't
figure out why the same logic wasn't in DSDirectory.
I'll test on OSX and linux and report back.
Dmitry Serebrennikov said:
> Sounds like the same error that I fixed in the RAMDirectory last week. I
> didn't think that FSDirectory would have this problem because I thought
> that the OS filesystem would take care of this. Apparently it does not.
>
> Hani, please take a look at the RAMDirectory touchFile method (line
> 148). It's basically the same fix you proposed except that it has some
> commented out monitoring code and it uses Thread.sleep(0,1) to release
> the CPU while waiting. In my measurements I was seeing 5-10 loops with
> the Thread.sleep call (regardless of the actual sleep duration that I
> specified), and around 10000 loops without a sleep call. But of course I
> was on a different hardware/OS combo and probably a different VM.
>
> Hani, will you be able to try a variant of this fix for the FSDirectory?
> I'm currious to see what kinds of results will observed on OS X.
>
> Thanks.
> Dmitry.
>
>
> Hani Suleiman wrote:
>
>> Ok, I figured out what's going on.
>>
>> I suspect this will only manifest itself on faster machines (I'm on a
>> dual 1.42ghz G4)
>>
>> What happens basically is that the the timer resolution on
>> file.lastModified isn't accurate enough, so when a file is touched,
>> it returns the same value for lastModified. This in turn causes the
>> segment staleness checks in IndexReader to fail (around line 276,
>> lastModified(directory)==segmentInfosAge), hence the testcase fails.
>>
>> I believe this is a bug, but I can't come up with a satisfactory
>> workaround. My solution is to modify FSDirectory.touchFile(String
>> name) to have the following block;
>>
>> long oldModified = file.lastModified();
>> do
>> {
>> file.setLastModified(System.currentTimeMillis());
>> }
>> while(oldModified==file.lastModified());
>>
>> This ensures that a touch ensures the file is considered 'fresher'
>> that it was before, which makes everything happy. It's a bit of a
>> hack though, so I'm not sure of how you guys would prefer handling
>> this, take the performance hit with the extra lastModified checks, if
>> make it an even bigger hack by checking for OSX, or something far
>> more clever and not detrimental to performance ;)
>>
>> Hani
>>
>> On Tuesday, September 30, 2003, at 12:42 AM, Hani Suleiman wrote:
>>
>>> here you go, happens to me on linux too (with JDK 1.4.0)
>>>
>>> OSX using latest 1.4.1
>>>
>>> [junit] ------------- ---------------- ---------------
>>> [junit] Testsuite: org.apache.lucene.index.TestIndexReader
>>> [junit] Tests run: 5, Failures: 2, Errors: 0, Time elapsed:
>>> 2.883 sec
>>>
>>> [junit] Testcase:
>>> testDeleteReaderWriterConflict(org.apache.lucene.index.TestIndexReader)
>>> : FAILED
>>> [junit] Delete allowed on an index reader with stale segment
>>> information
>>> [junit] junit.framework.AssertionFailedError: Delete allowed on
>>> an index reader with stale segment information
>>> [junit] at
>>> org.apache.lucene.index.TestIndexReader.testDeleteReaderWriterConflict(
>>> TestIndexReader.java:271)
>>> [junit] at
>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> [junit] at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja
>>> va:39)
>>> [junit] at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso
>>> rImpl.java:25)
>>>
>>>
>>> [junit] Testcase:
>>> testDeleteReaderReaderConflict(org.apache.lucene.index.TestIndexReader)
>>> : FAILED
>>> [junit] Delete allowed from a stale index reader
>>> [junit] junit.framework.AssertionFailedError: Delete allowed
>>> from a stale index reader
>>> [junit] at
>>> org.apache.lucene.index.TestIndexReader.testDeleteReaderReaderConflict(
>>> TestIndexReader.java:397)
>>> [junit] at
>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> [junit] at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja
>>> va:39)
>>> [junit] at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso
>>> rImpl.java:25)
>>>
>>>
>>>
>>> On Tuesday, September 30, 2003, at 12:37 AM, Erik Hatcher wrote:
>>>
>>>> Hani,
>>>>
>>>> I don't see this test failure on OS X. Thanks to your patch on the
>>>> RussianAnalyzer, all my tests now successfully pass. Could you
>>>> give us the stack trace for the problem you're seeing?
>>>>
>>>> Erik
>>>>
>>>>
>>>> On Monday, September 29, 2003, at 11:28 PM, Hani Suleiman wrote:
>>>>
>>>>> I'm seeing a fairly disturbing testcase failure on OSX in
>>>>> TestIndexReader.testDeleteReaderWriterConflict and
>>>>> TestIndexReader.testDeleteReaderReaderConflict
>>>>>
>>>>> This only happens on OSX though, is this a known issue? Any have
>>>>> an ideas of what sort of thing is going wrong before I dig in
>>>>> further?
>>>>>
>>>>> Hani
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>>>>> For additional commands, e-mail: [EMAIL PROTECTED]
>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>>>> For additional commands, e-mail: [EMAIL PROTECTED]
>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [EMAIL PROTECTED] For
>>> additional commands, e-mail: [EMAIL PROTECTED]
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [EMAIL PROTECTED] For
>> additional commands, e-mail: [EMAIL PROTECTED]
>>
>>
>
>
>
> --------------------------------------------------------------------- To
> unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
