peter-toth opened a new pull request, #38034:
URL: https://github.com/apache/spark/pull/38034
### What changes were proposed in this pull request?
This PR introduce `TreeNode.multiTransform()` methods to be able to
recursively transform a `TreeNode` (and so a tree) into multiple alternatives.
These functions are particularly useful if we want to transform an expression
with a projection in which subexpressions can be aliased with multiple
different attributes.
E.g. if we have a partitioning expression `HashPartitioning(a + b)` and we
have a `Project` node that aliases `a` as `a1` and `a2` and `b` as `b1` and
`b2` we can easily generate a stream of alternative transformations of the
original partitioning:
```
val partitioning = HashPartitioning(Add(a, b))
val aliases: Map[Expression, Seq[Attribute]] = ... // collect the alias map
from project
val s = partitioning.multiTransform {
case e: Expression if aliases.contains(e.canonicalized) =>
aliases(e.canonicalized)
}
s // Stream(HashPartitioning(Add(a1, b1)), HashPartitioning(Add(a1, b2)),
HashPartitioning(Add(a2, b2)), HashPartitioning(Add(a2, b2)))
```
The result of `multiTransform` is a lazy stream to be able to limit the
number of alternatives generated at the caller side as needed.
### Why are the changes needed?
`TreeNode.multiTransform()` is a useful helper method.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
New UTs are added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
