Josh Rosen created SPARK-9702:
---------------------------------
Summary: Repartition operator should use Exchange to perform its
shuffle
Key: SPARK-9702
URL: https://issues.apache.org/jira/browse/SPARK-9702
Project: Spark
Issue Type: Improvement
Components: SQL
Reporter: Josh Rosen
Spark SQL's {{Repartition}} operator is implemented in terms of Spark Core's
repartition operator, which means that it has to perform lots of unnecessary
row copying and inefficient row serialization. Instead, it would be better if
this was implemented using some of Exchange's internals so that it can avoid
row format conversions and generic getters / hashcodes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
