[
https://issues.apache.org/jira/browse/HADOOP-14479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16067320#comment-16067320
]
SammiChen commented on HADOOP-14479:
------------------------------------
First of all, these three failed unit tests are not related with ISA-L version
change. There are actually hidden issues.
1.TestHHXORErasureCoder
The implementation of native XOR encoder/decoder has array out of index issue
which cause the JVM crash.
2.TestRSErasureCoder
One failed unit test is to test underlying encoder reuse by execution
testCoding twice, while the underlying encoder is released explicitly by
ErasureEncodingStep.finish. By review the context, I think
ErasureEncodingStep.finish is not the right place to release the encoder.
3. TestCodecRawCoderMapping
One failed unit test is because it doesn't consider the native RS coder
situation.
I have tried the patch locally.
[~Ayappan], thanks for reporting this. You can have a try with the new patch.
Besides, I will close HADOOP-14593 later since I have merged fixes for 3 unit
cases into one patch.
> Erasurecode testcase failures with native enabled
> -------------------------------------------------
>
> Key: HADOOP-14479
> URL: https://issues.apache.org/jira/browse/HADOOP-14479
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
> Affects Versions: 3.0.0-alpha3
> Environment: x86_64 Ubuntu 16.04.02 LTS
> Reporter: Ayappan
> Assignee: SammiChen
> Priority: Critical
> Labels: hdfs-ec-3.0-must-do
> Attachments: HADOOP-14479.001.patch
>
>
> I built hadoop with ISA-L support. I took the ISA-L code from
> https://github.com/01org/isa-l (tag v2.18.0) and built it. While running the
> UTs , following three testcases are failing
> 1)TestHHXORErasureCoder
> Tests run: 7, Failures: 3, Errors: 0, Skipped: 0, Time elapsed: 1.106 sec <<<
> FAILURE! - in org.apache.hadoop.io.erasurecode.coder.TestHHXORErasureCoder
> testCodingDirectBuffer_10x4_erasing_p1(org.apache.hadoop.io.erasurecode.coder.TestHHXORErasureCoder)
> Time elapsed: 0.029 sec <<< FAILURE!
> java.lang.AssertionError: Decoding and comparing failed.
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.hadoop.io.erasurecode.TestCoderBase.compareAndVerify(TestCoderBase.java:170)
> at
> org.apache.hadoop.io.erasurecode.coder.TestErasureCoderBase.compareAndVerify(TestErasureCoderBase.java:141)
> at
> org.apache.hadoop.io.erasurecode.coder.TestErasureCoderBase.performTestCoding(TestErasureCoderBase.java:98)
> at
> org.apache.hadoop.io.erasurecode.coder.TestErasureCoderBase.testCoding(TestErasureCoderBase.java:69)
> at
> org.apache.hadoop.io.erasurecode.coder.TestHHXORErasureCoder.testCodingDirectBuffer_10x4_erasing_p1(TestHHXORErasureCoder.java:64)
> 2)TestRSErasureCoder
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.591 sec -
> in org.apache.hadoop.io.erasurecode.coder.TestXORCoder
> Running org.apache.hadoop.io.erasurecode.coder.TestRSErasureCoder
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> # SIGSEGV (0xb) at pc=0x00007f486a28a6e4, pid=8970, tid=0x00007f4850927700
> #
> # JRE version: OpenJDK Runtime Environment (8.0_121-b13) (build
> 1.8.0_121-8u121-b13-0ubuntu1.16.04.2-b13)
> # Java VM: OpenJDK 64-Bit Server VM (25.121-b13 mixed mode linux-amd64
> compressed oops)
> # Problematic frame:
> # C [libc.so.6+0x8e6e4]
> #
> # Failed to write core dump. Core dumps have been disabled. To enable core
> dumping, try "ulimit -c unlimited" before starting Java again
> #
> # An error report file with more information is saved as:
> # /home/ayappan/hadoop/hadoop-common-project/hadoop-common/hs_err_pid8970.log
> #
> # If you would like to submit a bug report, please visit:
> # http://bugreport.java.com/bugreport/crash.jsp
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> #
> 3)TestCodecRawCoderMapping
> Running org.apache.hadoop.io.erasurecode.TestCodecRawCoderMapping
> Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.559 sec <<<
> FAILURE! - in org.apache.hadoop.io.erasurecode.TestCodecRawCoderMapping
> testRSDefaultRawCoder(org.apache.hadoop.io.erasurecode.TestCodecRawCoderMapping)
> Time elapsed: 0.015 sec <<< FAILURE!
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.io.erasurecode.TestCodecRawCoderMapping.testRSDefaultRawCoder(TestCodecRawCoderMapping.java:58)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
