[
https://issues.apache.org/jira/browse/SPARK-34906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Herman van Hövell resolved SPARK-34906.
---------------------------------------
Fix Version/s: 3.2.0
Assignee: Ali Afroozeh
Resolution: Fixed
> Refactor TreeNode's children handling methods into specialized traits
> ---------------------------------------------------------------------
>
> Key: SPARK-34906
> URL: https://issues.apache.org/jira/browse/SPARK-34906
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.1.1
> Reporter: Ali Afroozeh
> Assignee: Ali Afroozeh
> Priority: Major
> Fix For: 3.2.0
>
>
> Spark query plan node hierarchy has specialized traits (or abstract classes)
> for handling nodes with fixed number of children, for example
> UnaryExpression, UnaryNode and UnaryExec for representing an expression, a
> logical plan and a physical plan with only one child, respectively. This PR
> refactors the TreeNode hierarchy by extracting the children handling
> functionality into the following traits. UnaryExpression` and other similar
> classes now extend the corresponding new trait:
> {{trait LeafLike[T <: TreeNode[T]] { self: TreeNode[T] =>}}
> {{ override final def children: Seq[T] = Nil}}
> {{}}}
> {{trait UnaryLike[T <: TreeNode[T]] { self: TreeNode[T] =>}}
> {{ def child: T}}
> {{ @transient override final lazy val children: Seq[T] = child :: Nil}}
> {{}}}
> {{trait BinaryLike[T <: TreeNode[T]] { self: TreeNode[T] =>}}
> {{ def left: T}}
> {{ def right: T}}
> {{ @transient override final lazy val children: Seq[T] = left :: right ::
> Nil}}
> {{}}}
> {{trait TernaryLike[T <: TreeNode[T]] { self: TreeNode[T] =>}}
> {{ def first: T}}
> {{ def second: T}}
> {{ def third: T}}
> {{ @transient override final lazy val children: Seq[T] = first :: second ::
> third :: Nil}}
> {{}}}
>
> This refactoring, which is part of a bigger effort to make tree
> transformations in Spark more efficient, has two benefits:
> * It moves the children handling to a single place, instead of being spread
> in specific subclasses, which will help the future optimizations for tree
> traversals.
> * It allows to mix in these traits with some concrete node types that could
> not extend the previous classes. For example, expressions with one child that
> extend AggregateFunction cannot extend UnaryExpression as AggregateFunction
> defines the foldable method final while UnaryExpression defines it as non
> final. With the new traits, we can directly extend the concrete class from
> UnaryLike in these cases. Classes with more specific child handling will make
> tree traversal methods faster.
> In this PR we have also updated many concrete node types to extend these
> traits to benefit from more specific child handling.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
