[
https://issues.apache.org/jira/browse/HDDS-2308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anu Engineer resolved HDDS-2308.
--------------------------------
Fix Version/s: 0.5.0
Resolution: Fixed
Committed to the build branch.
> Switch to centos with the apache/ozone-build docker image
> ---------------------------------------------------------
>
> Key: HDDS-2308
> URL: https://issues.apache.org/jira/browse/HDDS-2308
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Reporter: Marton Elek
> Assignee: Marton Elek
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.5.0
>
> Attachments: hs_err_pid16346.log
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> I realized multiple JVM crashes in the daily builds:
>
> {code:java}
> ERROR] ExecutionException The forked VM terminated without properly saying
> goodbye. VM crash or System.exit called?
>
>
> [ERROR] Command was /bin/sh -c cd /workdir/hadoop-ozone/ozonefs &&
> /usr/lib/jvm/java-1.8-openjdk/jre/bin/java -Xmx2048m
> -XX:+HeapDumpOnOutOfMemoryError -jar
> /workdir/hadoop-ozone/ozonefs/target/surefire/surefirebooter9018689154779946208.jar
> /workdir/hadoop-ozone/ozonefs/target/surefire
> 2019-10-06T14-52-40_697-jvmRun1 surefire7569723928289175829tmp
> surefire_947955725320624341206tmp
>
>
> [ERROR] Error occurred in starting fork, check output in log
>
>
> [ERROR] Process Exit Code: 139
>
>
> [ERROR] Crashed tests:
>
>
> [ERROR] org.apache.hadoop.fs.ozone.contract.ITestOzoneContractRename
>
>
> [ERROR] ExecutionException The forked VM terminated without properly
> saying goodbye. VM crash or System.exit called?
>
>
> [ERROR] Command was /bin/sh -c cd /workdir/hadoop-ozone/ozonefs &&
> /usr/lib/jvm/java-1.8-openjdk/jre/bin/java -Xmx2048m
> -XX:+HeapDumpOnOutOfMemoryError -jar
> /workdir/hadoop-ozone/ozonefs/target/surefire/surefirebooter5429192218879128313.jar
> /workdir/hadoop-ozone/ozonefs/target/surefire
> 2019-10-06T14-52-40_697-jvmRun1 surefire7227403571189445391tmp
> surefire_1011197392458143645283tmp
>
>
> [ERROR] Error occurred in starting fork, check output in log
>
>
> [ERROR] Process Exit Code: 139
>
>
> [ERROR] Crashed tests:
>
>
> [ERROR] org.apache.hadoop.fs.ozone.contract.ITestOzoneContractDistCp
>
>
> [ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException:
> ExecutionException The forked VM terminated without properly saying goodbye.
> VM crash or System.exit called?
>
>
> [ERROR] Command was /bin/sh -c cd /workdir/hadoop-ozone/ozonefs &&
> /usr/lib/jvm/java-1.8-openjdk/jre/bin/java -Xmx2048m
> -XX:+HeapDumpOnOutOfMemoryError -jar
> /workdir/hadoop-ozone/ozonefs/target/surefire/surefirebooter1355604543311368443.jar
> /workdir/hadoop-ozone/ozonefs/target/surefire
> 2019-10-06T14-52-40_697-jvmRun1 surefire3938612864214747736tmp
> surefire_933162535733309260236tmp
>
>
> [ERROR] Error occurred in starting fork, check output in log
>
>
> [ERROR] Process Exit Code: 139
>
>
> [ERROR] ExecutionException The forked VM terminated without properly
> saying goodbye. VM crash or System.exit called?
>
>
> [ERROR] Command was /bin/sh -c cd /workdir/hadoop-ozone/ozonefs &&
> /usr/lib/jvm/java-1.8-openjdk/jre/bin/java -Xmx2048m
> -XX:+HeapDumpOnOutOfMemoryError -jar
> /workdir/hadoop-ozone/ozonefs/target/surefire/surefirebooter9018689154779946208.jar
> /workdir/hadoop-ozone/ozonefs/target/surefire
> 2019-10-06T14-52-40_697-jvmRun1 surefire7569723928289175829tmp
> surefire_947955725320624341206tmp
>
>
> [ERROR] Error occurred in starting fork, check output in log
>
>
> [ERROR] Process Exit Code: 139 {code}
>
> Based on the crash log (uploaded) it's related to the rocksdb JNI interface.
> In the current ozone-build docker image (which provides the environment for
> build) we use alpine where musl libc is used instead of the main glibc. I
> think it would be more safe to use the same glibc what is used in production.
> I tested with centos based docker image and it seems to be more stable.
> Didn't see any more JVM crashes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
