Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21564#discussion_r195630214
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala
---
@@ -170,6 +170,8 @@ case class InMemoryTableScanExec(
override def outputPartitioning: Partitioning = {
relation.cachedPlan.outputPartitioning match {
case h: HashPartitioning =>
updateAttribute(h).asInstanceOf[HashPartitioning]
+ case r: RangePartitioning =>
+ r.copy(ordering =
r.ordering.map(updateAttribute(_).asInstanceOf[SortOrder]))
--- End diff --
For `PartitioningCollection`, I think it is harder to treat it like
`HashPartitioning` and `RangePartitioning` when replacing attributes.
In above example, `PartitioningCollection` contains `HashPartitioning(i#5)`
and `HashPartitioning(m#15)`, the output of `InMemoryRelation` is `[i#54, j#55,
m#58, n#59]`. Can we still replace attributes based on the location of
attribute in output?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
