Github user yucai commented on a diff in the pull request:
https://github.com/apache/spark/pull/21564#discussion_r195652026
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala
---
@@ -170,6 +170,8 @@ case class InMemoryTableScanExec(
override def outputPartitioning: Partitioning = {
relation.cachedPlan.outputPartitioning match {
case h: HashPartitioning =>
updateAttribute(h).asInstanceOf[HashPartitioning]
+ case r: RangePartitioning =>
+ r.copy(ordering =
r.ordering.map(updateAttribute(_).asInstanceOf[SortOrder]))
--- End diff --
@viirya From `updateAttribute`, `relation.cachedPlan.output` and
`relation.output` one to one.
```
private def updateAttribute(expr: Expression): Expression = {
....
val attrMap =
AttributeMap(relation.cachedPlan.output.zip(relation.output))
....
}
```
It means "[i#54, j#55, m#58, n#59]" corresponds to "[i#5, j#6, m#15,
n#16]", so we can always replace `HashPartitioning(i#5)` to
`HashPartitioning(i#54)`.
Any idea?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
