AnishMahto commented on code in PR #56378:
URL: https://github.com/apache/spark/pull/56378#discussion_r3377116523
##########
sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/txns.scala:
##########
@@ -279,6 +279,10 @@ class SharedTablesInMemoryRowLevelOperationTableCatalog
super.initialize(name, options)
tables = SharedTablesInMemoryRowLevelOperationTableCatalog.sharedTables
}
+
+ // Return the live table instance (not a snapshot copy) so that TRUNCATE and
DROP
+ // mutations affect the shared catalog state immediately.
+ override def loadTable(ident: Identifier): Table = liveTable(ident)
Review Comment:
I don't have any expertise here but is this really addressing the root
problem or is it more of a bandaid/test-only fix?
Naively consulting Claude, it looks like `TableCatalog` defines two
`loadTable` overrides; `loadTable(ident)` for loading on a table read-only
path, and `loadTable(ident, writePrivileges)` for loading tables on a write
path.
It seems like `SPARK-56995` has implemented the correct override for
`loadTable` in the in-memory catalog, where the read-only path reads the cached
version, while the write path still reads the live version of the table.
But the disconnect seems to be that the table reference inside `TRUNCATE
<table>` is parsed as an `UnresolvedTable` (rather than, say,
`UnresolvedRelation` with the `requireWritePrivileges` API) that ultimately
triggers the read overload of `loadTable` during resolution despite
fundamentally being a write operation. Maybe `UnresolvedTable` should be
storing/propagating the correct write privileges too, similar to
`UnresolvedRelation`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
