cloud-fan commented on a change in pull request #31666:
URL: https://github.com/apache/spark/pull/31666#discussion_r612501598
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/package.scala
##########
@@ -201,4 +201,30 @@ package object util extends Logging {
def truncatedString[T](seq: Seq[T], sep: String, maxFields: Int): String = {
truncatedString(seq, "", sep, "", maxFields)
}
+
+ val METADATA_COL_ATTR_KEY = "__metadata_col"
+
+ /**
+ * Hidden columns are a type of metadata column that are candidates during
qualified star
+ * star expansions. They are propagated through Projects that have hidden
children output,
+ * so that nested hidden output is not lost.
+ */
+ val HIDDEN_COL_ATTR_KEY = "__hidden_col"
Review comment:
The semantic is clear now, let's refine the naming.
We only have metadata column, and metadata column can be included in
qualified star if required. We can just add a new property to metadata columns
to indicate it.
The property name can be `__support_qualified_star`, and the helper class
can be
```
implicit class MetadataColumnHelper(attr: Attribute) {
def isMetadataCol: Boolean ...
def supportQualifiedStar: Boolean ...
def markAsSupportQualifiedStar: Attribute ...
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
