Xiangrui Meng created SPARK-4112:
------------------------------------
Summary: Have a reserved copy of Sorter/SortDataFormat
Key: SPARK-4112
URL: https://issues.apache.org/jira/browse/SPARK-4112
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 1.2.0
Reporter: Xiangrui Meng
After SPARK-4084, developers can use Sorter with their own SortDataFormat.
However, if there are multiple subclasses of SortDataFormat instantiated, JIT
won't inline the methods in SortDataFormat and virtual method table lookup is
slow. One solution could be making two copies of the code and reserve one for
shuffle only, and expose the other to developers.
Before we do that, we should compare the performance with/without JIT and check
whether it is worth the extra code complexity.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
