amaliujia commented on code in PR #36641:
URL: https://github.com/apache/spark/pull/36641#discussion_r898293801
##########
sql/core/src/main/scala/org/apache/spark/sql/internal/CatalogImpl.scala:
##########
@@ -250,8 +251,18 @@ class CatalogImpl(sparkSession: SparkSession) extends
Catalog {
* table/view. This throws an `AnalysisException` when no `Table` can be
found.
*/
override def getTable(tableName: String): Table = {
- val tableIdent =
sparkSession.sessionState.sqlParser.parseTableIdentifier(tableName)
- getTable(tableIdent.database.orNull, tableIdent.table)
+ // calling `sqlParser.parseTableIdentifier` to parse tableName. If it
contains only table name
+ // and optionally contains a database name(thus a TableIdentifier), then
that is used to get
+ // the table. Otherwise we try `sqlParser.parseMultipartIdentifier` to
have a sequence of string
Review Comment:
This is the place that was also confusing to me:
In the past there is only HMS and people call API based on that assumption,
in that case does we think whether users know there is a current catalog
concept?
This is vague to me: people may or may not use the API with current catalog
in mind. For example, one user may not know that we can set a current catalog
then play against it. One user might just always believe whatever they called,
it will hit HMS. This matches with the old implementation I feel like because
the code directly check sessionCatalog which is HMS.
I am just trying to understand to maintain backwards compatibility, what is
that old behaviors.
If you think the old behavior is one that we already respected current
catalog. I can change this implementation: when there is 2 part namespace, we
first get current catalog, then check within that catalog .
However I am thinking the idea above might break backwards behavior:
1. A HMS user set current catalog but not to HMS.
2. The user uses 2 part name and and the user expects to read from HMS.
In the past even the user does the step 1, step 2 will still work as we just
verify, for example, a DB in sessionCatalog.
With respecting current catalog, step 2 will not succeed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
