szehon-ho commented on code in PR #54411:
URL: https://github.com/apache/spark/pull/54411#discussion_r2836055777
##########
sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/InMemoryTable.scala:
##########
@@ -261,3 +263,71 @@ object InMemoryTable {
}
}
}
+
+/**
+ * A metadata table that returns snapshot (commit) information for a parent
table.
+ * Simulates data source metadata tables like Iceberg's db.table.snapshots.
+ */
+class InMemorySnapshotsTable(parentTable: InMemoryTable) extends Table with
SupportsRead {
+ override def name(): String = parentTable.name + ".snapshots"
+
+ override def schema(): StructType = StructType(Seq(
+ StructField("committed_at", LongType, nullable = false),
+ StructField("snapshot_id", LongType, nullable = false)
+ ))
+
+ override def capabilities(): util.Set[TableCapability] = {
Review Comment:
Yea, as I guess you know, Iceberg implements the metadata table behind
loadTable, matching the tableIdentifier with the form
$table.$metadata_table_name.
It'd be nice for Spark to support metadata table officially in DSV2 one day.
Then its more clear to Spark code base.
Until then, this test mocks current Iceberg behavior, is because Spark is
often not aware of this unexpected behavior in Iceberg and breaks it. So its a
regression test for https://github.com/apache/spark/pull/54247 case.
By the way, what do you mean read permission? I only see
TableWritePrivelege in DSV2 so far
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
