L. C. Hsieh created SPARK-52921:
-----------------------------------
Summary: Specify outputPartitioning for UnionExec for partitioner
aware case
Key: SPARK-52921
URL: https://issues.apache.org/jira/browse/SPARK-52921
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 4.1.0
Reporter: L. C. Hsieh
Currently the output partitioning of UnionExec is simply unknown. But if the
partitioner is known to be the same for all children RDDs, SparkContext.union
produces a PartitionerAwareUnionRDD which reuses the partition. For such cases,
the output partitioning of UnionExec is actually known to be the same as its
children nodes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
