Cheng Hao created SPARK-3196:
--------------------------------
Summary: Expression Evaluation Performance Improvement
Key: SPARK-3196
URL: https://issues.apache.org/jira/browse/SPARK-3196
Project: Spark
Issue Type: Improvement
Components: SQL
Reporter: Cheng Hao
The expression id generations depend on a atomic long object internally, which
will cause the performance drop dramatically in a multi-threading execution.
I'd like to create 2 sub tasks(maybe more) for the improvements:
1) Reduce the expression tree object creation from the aggregation functions
(min/max), as they will create expression trees for each single row.
2) Improve the expression id generation algorithm, by not using the AtomicLong.
And remove the expression object creation as many as possible, where we have
the expression evaluation. (I will create couple of subtask soon).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
