Spookily, I believe this is in fact some sort of JVM hotspot compiler
bug. Adding -Xbatch (forces up-front compilation) prevents (works
around) the problem.
I added a counter, to IndexOutput, of total bytes written, and then
in SegmentMerger.mergeFields, I added an assert that the #docs
written through FieldsWriter matched the number of bytes written to
the fdx file. Insanely, that assert would fail when the issue was
hit, which is just not possible from the code. It's as if when the
BG compilation finished and the native method got swapped into the
JVM, it somehow "missed" a call to IndexOutput.writeLong.
I committed some of these asserts into SegmentMerger on both 2.3 &
trunk.
I'll respond on the java-user thread to bring closure here.
Mike
Yonik Seeley wrote:
On Wed, Mar 19, 2008 at 1:18 PM, Michael McCandless
<[EMAIL PROTECTED]> wrote:
I'm CC'ing java-dev. To sum up: sometimes when we merge fields, the
fdx file ends up exactly one document too short. In adding numerous
asserts around this code in SegmentMerger.java, insanely, somehow
the
call to indexStream.writeLong fails to actually "happen" when we
call
FieldsWriter.addDocument, even though from looking at the code I see
no way to explain that. As hard as it is to believe, it really is
looking like a strange JVM bug at this point...
JDK 1.6_04 was a major upgrade of hotspot.
Is there a way to dump the compiled machine code that hotspot
eventually generates for that method?
-Yonik
Has anyone else seen any odd behavior on update 4 or 5 of JDK 1.6?
The issue does not happen on previous updates of JDK 1.6.
Mike
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
