Mitesh created SPARK-20112:
------------------------------
Summary: SIGSEGV in GeneratedIterator.sort_addToSorter
Key: SPARK-20112
URL: https://issues.apache.org/jira/browse/SPARK-20112
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.0.2
Environment: AWS m4.10xlarge with EBS (io1 drive, 400g, 4000iops)
Reporter: Mitesh
I'm seeing a very weird crash in GeneratedIterator.sort_addToSorter. The
hs_err_pid and codegen file are attached (with query plans). Its not a
deterministic repro, but running a big query load, I eventually see it come up
within a few minutes.
Here is some interesting repro information:
- Using r3.8xlarge machines, which have ephermal attached drives, I can't repro
this. So I think that means its not an issue with the code-gen, but I cant
figure out what the difference in behavior is.
- The broadcast joins in the plan are all small tables. I have
autoJoinBroadcast=-1 because I always hint which tables should be broadcast.
- As you can see from the plan, all the sources are cached memory tables
{noformat}
# A fatal error has been detected by the Java Runtime Environment:
#
# [thread 139872345896704 also had an error]
SIGSEGV (0xb) at pc=0x00007f38a378caa3, pid=19271, tid=139872342738688
#
# JRE version: Java(TM) SE Runtime Environment (8.0_60-b27) (build 1.8.0_60-b27)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.60-b23 mixed mode linux-amd64
compressed oops)
[thread 139872348002048 also had an error]# Problematic frame:
#
J 28454 C1
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.sort_addToSorter$(Lorg/apache/spark/sql/catalyst/expressions/GeneratedClass$GeneratedIterator;)V
(369 bytes) @ 0x00007f38a378caa3 [0x00007f38a378b5e0+0x14c3]
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
