[
https://issues.apache.org/jira/browse/LUCENE-4948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638191#comment-13638191
]
Uwe Schindler edited comment on LUCENE-4948 at 4/22/13 5:26 PM:
----------------------------------------------------------------
This is easy to explain: J9 does not know compressed oops. On Oracle 64 bit
JVMs the pointers are compressed, so a reference is 32 bits, which is identical
to the size of ints. So they grow identical. In IBM J9 (64 bit, it only fails
with 64 bits!) a reference is 8 bytes. So the reference array grows different
than the int[] array.
This bug also affect useres with Oracle JDK, if they have > 32 Gig of heap...
Whenever we use ArrayUtil.grow() we have to take care that the sizes may
differ. Please don't change the growing pattern, the code should (when it
requires that several arrays are sized in parallel) calculate the new size
before and use a suitable type to do that.
A good test would be: We make the growing pattern in ArrayUtil allow "test
mode" maybe and make the factor random? Just an idea. Maybe we can do this
before running tests using bytecode modification.
was (Author: thetaphi):
This is easy to explain: J9 does not know compressed oops. On Oracle 64 bit
JVMs the pointers are compressed, so a reference is 32 bits, which is identical
to the size of ints. So they grow identical. In IBM J9 (64 bit, it only fails
with 64 bits!) a reference is 8 bytes. So the reference array grows different
than the int[] array.
This bug also affect useres with Oracle JDK, if the have > 32 Gig of heap...
Whenever we use ArrayUtil.grow() we have to take care that the sizes may
differ. Please don't change the growing pattern, the code should (when it
requires that several arrays are sized in paralel, calculate the new size
before and use a suitable type to do that).
A good test would be: We make the growing pattern in ArrayUtil allow "test
mode" maybe and make the factor random? Just an idea. Maybe we can do this
before running tests using bytecode modification.
> Stink bug in PostingsHighlighter
> --------------------------------
>
> Key: LUCENE-4948
> URL: https://issues.apache.org/jira/browse/LUCENE-4948
> Project: Lucene - Core
> Issue Type: Bug
> Components: modules/highlighter
> Reporter: Michael McCandless
> Fix For: 5.0, 4.4
>
>
> This test fail reproduces on IBM J9:
> {noformat}
> NOTE: reproduce with: ant test -Dtestcase=TestPostingsHighlighter
> -Dtests.method=testCambridgeMA -Dtests.seed=2A9A93DAC39E0938
> -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_HN
> -Dtests.timezone=America/Yellowknife -Dtests.file.encoding=UTF-8
> {noformat}
> {noformat}
> Stack Trace:
> java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 37
> at
> __randomizedtesting.SeedInfo.seed([2A9A93DAC39E0938:AB8FF071AD305139]:0)
> at
> org.apache.lucene.search.postingshighlight.Passage.addMatch(Passage.java:53)
> at
> org.apache.lucene.search.postingshighlight.PostingsHighlighter.highlightDoc(PostingsHighlighter.java:547)
> at
> org.apache.lucene.search.postingshighlight.PostingsHighlighter.highlightField(PostingsHighlighter.java:425)
> at
> org.apache.lucene.search.postingshighlight.PostingsHighlighter.highlightFields(PostingsHighlighter.java:364)
> at
> org.apache.lucene.search.postingshighlight.PostingsHighlighter.highlightFields(PostingsHighlighter.java:268)
> at
> org.apache.lucene.search.postingshighlight.PostingsHighlighter.highlight(PostingsHighlighter.java:198)
> at
> org.apache.lucene.search.postingshighlight.TestPostingsHighlighter.testCambridgeMA(TestPostingsHighlighter.java:373)
> {noformat}
> I think it's because J9 grows arrays in a different progression than other
> JVMs ... we should fix PostingsHighlighter to forcefully grow the arrays to
> the same length instead of this:
> {noformat}
> if (numMatches == matchStarts.length) {
> matchStarts = ArrayUtil.grow(matchStarts, numMatches+1);
> matchEnds = ArrayUtil.grow(matchEnds, numMatches+1);
> BytesRef newMatchTerms[] = new
> BytesRef[ArrayUtil.oversize(numMatches+1,
> RamUsageEstimator.NUM_BYTES_OBJECT_REF)];
> System.arraycopy(matchTerms, 0, newMatchTerms, 0, numMatches);
> matchTerms = newMatchTerms;
> }
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
