[
https://issues.apache.org/jira/browse/SPARK-35181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17410492#comment-17410492
]
angerszhu commented on SPARK-35181:
-----------------------------------
{code}
2021-08-28 22:38:19 INFO CodecPool:184 - Got brand-new decompressor [.zst]
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007f2a0df54ebf, pid=80960, tid=0x00007f2a1cefd700
#
# JRE version: OpenJDK Runtime Environment (8.0_181-b13) (build 1.8.0_181-b13)
# Java VM: OpenJDK 64-Bit Server VM (25.181-b13 mixed mode linux-amd64
compressed oops)
# Problematic frame:
# C [libzstd.so.1+0xeebf]
#
# Core dump written. Default location:
/mnt/ssd/0/yarn/nm-local-dir/usercache/staging_data_trafficmart/appcache/application_1629291751925_3137692/container_e212_1629291751925_3137692_01_000118/core
or core.80960
#
# An error report file with more information is saved as:
#
/mnt/ssd/0/yarn/nm-local-dir/usercache/staging_data_trafficmart/appcache/application_1629291751925_3137692/container_e212_1629291751925_3137692_01_000118/hs_err_pid80960.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
{code}
We meet a lot jvm crash issue in spark 3.1.2 when use zstd as default. Have you
guys meet this before too?
> Use zstd for spark.io.compression.codec by default
> --------------------------------------------------
>
> Key: SPARK-35181
> URL: https://issues.apache.org/jira/browse/SPARK-35181
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 3.2.0
> Reporter: Dongjoon Hyun
> Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
