Rodrigo Trujillo created LUCENE-4999:
----------------------------------------
Summary: Lucene test (testCambridgeMA) fails when JVM 64-bit does
not use memory compression
Key: LUCENE-4999
URL: https://issues.apache.org/jira/browse/LUCENE-4999
Project: Lucene - Core
Issue Type: Bug
Affects Versions: 4.2.1, 4.3
Environment: Red Hat 6.3
IBM Java 6 - SR13
Ant 1.9.0
Reporter: Rodrigo Trujillo
Priority: Critical
When I ran the Lucene (4.2.1/4.3) test suite with IBM Java I get the following
error:
[junit4:junit4] Suite:
org.apache.lucene.search.postingshighlight.TestPostingsHighlighter
[junit4:junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestPostingsHighlighter -Dtests.method=testCambridgeMA
-Dtests.seed=571E16AEAF72C9F9 -Dtests.s
low=true -Dtests.locale=mt_MT -Dtests.timezone=Pacific/Kiritimati
-Dtests.file.encoding=UTF-8
[junit4:junit4] ERROR 0.71s J2 | TestPostingsHighlighter.testCambridgeMA <<<
[junit4:junit4] > Throwable #1: java.lang.ArrayIndexOutOfBoundsException:
Array index out of range: 37
[junit4:junit4] > at
__randomizedtesting.SeedInfo.seed([571E16AEAF72C9F9:D60B7505C1DC91F8]:0)
[junit4:junit4] > at
org.apache.lucene.search.postingshighlight.Passage.addMatch(Passage.java:53)
[junit4:junit4] > at
org.apache.lucene.search.postingshighlight.PostingsHighlighter.highlightDoc(PostingsHighlighter.java:547)
[junit4:junit4] > at
org.apache.lucene.search.postingshighlight.PostingsHighlighter.highlightField(PostingsHighlighter.java:425)
[junit4:junit4] > at
org.apache.lucene.search.postingshighlight.PostingsHighlighter.highlightFields(PostingsHighlighter.java:364)
[junit4:junit4] > at
org.apache.lucene.search.postingshighlight.PostingsHighlighter.highlightFields(PostingsHighlighter.java:268)
[junit4:junit4] > at
org.apache.lucene.search.postingshighlight.PostingsHighlighter.highlight(PostingsHighlighter.java:198)
[junit4:junit4] > at
org.apache.lucene.search.postingshighlight.TestPostingsHighlighter.testCambridgeMA(TestPostingsHighlighter.java:373)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:738)
[junit4:junit4] 2> NOTE: test params are:
codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FA
ST_DECOMPRESSION, chunkSize=386),
termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION,
chunkSize=386)), sim=RandomSimilarityProv
ider(queryNorm=false,coord=yes): {body=DFR I(n)Z(0.3), title=DFR I(F)Z(0.3),
id=DFR I(n)2}, locale=mt_MT, timezone=Pacific/Kiritimati
[junit4:junit4] 2> NOTE: Linux 2.6.32-279.el6.x86_64 amd64/IBM Corporation
1.6.0 (64-bit)/cpus=4,threads=1,free=10783032,total=24030208
[junit4:junit4] 2> NOTE: All tests run in this JVM: [FieldQueryTest,
FieldPhraseListTest, SimpleFragListBuilderTest, FieldTermStackTest,
OffsetLimitTokenFil
terTest, TokenSourcesTest, TestPostingsHighlighter]
[junit4:junit4] Completed on J2 in 2.46s, 23 tests, 1 error <<< FAILURES!
This error is not seem with Oracle Java.
A Google search showed that this error has already occurred in community builds
and the solution proposed was disable the IBM Java in the community tests.
I took a look in the code and found that the root of the problem is due to the
assignment of the variable "referenceSize" in RamUsageEstimator.java:
// get object reference size by getting scale factor of Object[] arrays:
try {
final Method arrayIndexScaleM = unsafeClass.getMethod("arrayIndexScale",
Class.class);
referenceSize = ((Number) arrayIndexScaleM.invoke(theUnsafe,
Object[].class)).intValue();
supportedFeatures.add(JvmFeature.OBJECT_REFERENCE_SIZE);
} catch (Exception e) {
// ignore.
}
The Java Object reference size for arrays have 8 bytes in 64-bit machines
(Oracle or IBM) and can be reduced to 4 bytes (like 32-bit JVMs) using
Compressed References and Compressed Ordinary Object Pointers (OOPs).
This options seems to be enabled by default in Oracle Java when the heap size
is under 32GB, but is not in IBM Java.
As workaround, when testing with IBM JVM I can pass the options
"-Xcompressedrefs" or "-XX:+UseCompressedOops" to Junit.
Similarly, you can reproduce the error if you pass the option
"-XX:-UseCompressedOops" when testing with Oracle Java.
The bug is in oversize method of ArrayUtil.java. It does nothing when the
object reference size (bytesPerElement) is 8.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
