Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18907#discussion_r133061185
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala
---
@@ -402,52 +402,20 @@ object CatalogTypes {
/**
- * A [[LogicalPlan]] that represents a table.
+ * The logical plan representing a catalog table.
*/
-case class CatalogRelation(
- tableMeta: CatalogTable,
- dataCols: Seq[AttributeReference],
- partitionCols: Seq[AttributeReference]) extends LeafNode with
MultiInstanceRelation {
- assert(tableMeta.identifier.database.isDefined)
- assert(tableMeta.partitionSchema.sameType(partitionCols.toStructType))
- assert(tableMeta.dataSchema.sameType(dataCols.toStructType))
-
- // The partition column should always appear after data columns.
- override def output: Seq[AttributeReference] = dataCols ++ partitionCols
-
- def isPartitioned: Boolean = partitionCols.nonEmpty
-
- override def equals(relation: Any): Boolean = relation match {
- case other: CatalogRelation => tableMeta == other.tableMeta && output
== other.output
- case _ => false
- }
-
- override def hashCode(): Int = {
- Objects.hashCode(tableMeta.identifier, output)
- }
+trait CatalogRelation extends LeafNode {
+ def tableMeta: CatalogTable
+}
- override lazy val canonicalized: LogicalPlan = copy(
- tableMeta = tableMeta.copy(
- storage = CatalogStorageFormat.empty,
- createTime = -1
- ),
- dataCols = dataCols.zipWithIndex.map {
- case (attr, index) => attr.withExprId(ExprId(index))
- },
- partitionCols = partitionCols.zipWithIndex.map {
- case (attr, index) => attr.withExprId(ExprId(index +
dataCols.length))
- })
+/**
+ * A placeholder for a table relation, which will be replaced by concrete
relation like
+ * `LogicalRelation` or `HiveTableRelation`, during analysis.
+ */
+case class UnresolvedCatalogRelation(tableMeta: CatalogTable) extends
CatalogRelation {
--- End diff --
Is that possible we move `HiveTableRelation` to our core package? Then,
many rules become very clear. We have a case for `LogicalRelation` and another
case for `HiveTableRelation`
Or another way is to not let `UnresolvedCatalogRelation` extend
`CatalogRelation`? Then, `CatalogRelation` can be a pure node for representing
`HiveTableRelation`. We can rename it to a more easy-to-understand name.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
