Starting last week we started seeing the error below which terminates the
drill service pid.. My research suggests that it is a space issue with /tmp,
but we have plenty of free space..
I'm using dfs.tmp to convert JSON files (2 to 3 gig each) into Parquet..
Anyone encounter this issue before? I've tried using different versions of the
JVM with no success..
Output from drillbit_out
672 values, 512,796B raw, 465,932B comp, 1 pages, encodings: [RLE,
PLAIN_DICTIONARY], dic { 3,376 entries, 27,008B raw, 3,376B comp}
Jun 15, 2017 1:39#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007f724c052570, pid=33814, tid=140128310007552
#
# JRE version: Java(TM) SE Runtime Environment (8.0_45-b14) (build 1.8.0_45-b14)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.45-b02 mixed mode linux-amd64
compressed oops)
# Problematic frame:
# v ~StubRoutines::jlong_disjoint_arraycopy
#
# Core dump written. Default location: /usr/hdp/2.3.4.0-3485/core or core.33814
#
# An error report file with more information is saved as:
# /tmp/hs_err_pid33814.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
#
Changed parquet settings:
store.parquet.block-size CHANGED 134217728 < Matches HDFS
block size. I FTP the files from DFS to HDFS afterwards.
store.parquet.enable_dictionary_encoding CHANGED true <
Dictionary Encoding turned on.
This message may contain information that is confidential or privileged. If you
are not the intended recipient, please advise the sender immediately and delete
this message. See
http://www.blackrock.com/corporate/en-us/compliance/email-disclaimers for
further information. Please refer to
http://www.blackrock.com/corporate/en-us/compliance/privacy-policy for more
information about BlackRock’s Privacy Policy.
For a list of BlackRock's office addresses worldwide, see
http://www.blackrock.com/corporate/en-us/about-us/contacts-locations.
© 2017 BlackRock, Inc. All rights reserved.
