xinrong-meng commented on code in PR #47884:
URL: https://github.com/apache/spark/pull/47884#discussion_r1741341219
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala:
##########
@@ -1474,6 +1474,34 @@ case class Pivot(
override protected def withNewChildInternal(newChild: LogicalPlan): Pivot =
copy(child = newChild)
}
+/**
+ * A constructor for creating a transpose, which will later be converted
+ * to a [[LocalRelation]] at ReplaceTranspose during the query optimization.
+ *
+ * The result of the transpose operation is held in the `data` field, and the
corresponding
+ * schema is stored in the `output` field. The `Transpose` node does not
depend on any child
+ * logical plans after the data has been collected and transposed.
+ *
+ * @param output A sequence of output attributes representing the schema of
the transposed data.
+ * @param data A sequence of [[InternalRow]] containing the transposed data.
+ * @param hasNonIndexColumns A flag indicating whether there are non-index
columns in the
+ * original DataFrame. This is used to determine if
the transposed data
+ * should be populated or if it should remain empty.
Review Comment:
The flag is necessary to decide whether a Transpose node is resolved or not
as shown below:
```scala
case class Transpose(...
...
override lazy val resolved: Boolean = {
(output.isEmpty && !hasNonIndexColumns) ||
(output.nonEmpty && (!hasNonIndexColumns || data.nonEmpty))
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
