Hong Shen created SPARK-5421:
--------------------------------
Summary: SparkSql throw OOM at shuffle
Key: SPARK-5421
URL: https://issues.apache.org/jira/browse/SPARK-5421
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 1.2.0
Reporter: Hong Shen
ExternalAppendOnlyMap if only for the spark job that aggregator isDefined, but
sparkSQL's shuffledRDD haven't define aggregator, so sparkSQL won't spill at
shuffle, it's very easy to throw OOM at shuffle.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
