yadavay-amzn commented on code in PR #56627:
URL: https://github.com/apache/spark/pull/56627#discussion_r3566946871
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala:
##########
@@ -2045,6 +2045,17 @@ object SQLConf {
.intConf
.createWithDefault(200)
+ val THRIFTSERVER_CATALOG_METADATA_ENABLED =
+ buildConf("spark.sql.thriftServer.catalogMetadata.enabled")
+ .doc("When true, JDBC metadata operations (getCatalogs, getSchemas,
getTables, " +
+ "getColumns) are DataSource V2 catalog-aware: getCatalogs returns all
loaded catalogs, " +
+ "and TABLE_CAT is populated with the real catalog name. When false,
the legacy behavior " +
+ "is preserved (empty TABLE_CAT and getCatalogs returns no rows).")
+ .version("5.0.0")
Review Comment:
Changed to .version("4.3.0") - agreed, this is a backportable, conf-gated
feature so it should ship in branch-4.x rather than master-only.
##########
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkGetSchemasOperation.scala:
##########
@@ -71,15 +72,51 @@ private[hive] class SparkGetSchemasOperation(
try {
val schemaPattern = convertSchemaPattern(schemaName)
- catalog.listDatabases(schemaPattern).foreach { dbName =>
- rowSet.addRow(Array[AnyRef](dbName, DEFAULT_HIVE_CATALOG))
- }
-
- val globalTempViewDb = catalog.globalTempDatabase
- val databasePattern =
Pattern.compile(CLIServiceUtils.patternToRegex(schemaName))
- if (schemaName == null || schemaName.isEmpty ||
- databasePattern.matcher(globalTempViewDb).matches()) {
- rowSet.addRow(Array[AnyRef](globalTempViewDb, DEFAULT_HIVE_CATALOG))
+ if (isCatalogMetadataEnabled) {
+ // The JDBC catalogName parameter is intentionally not used for
filtering;
+ // resolution always uses the current catalog (SPARK-57518; per-catalog
+ // filtering deferred).
+ val resolvedCatalog = catalogManager.currentCatalog
+ val catalogNameValue = resolvedCatalog.name()
+ if (catalogNameValue == CatalogManager.SESSION_CATALOG_NAME) {
+ // For spark_catalog, use the V1 SessionCatalog directly
(transparent delegation)
+ catalog.listDatabases(schemaPattern).foreach { dbName =>
+ rowSet.addRow(Array[AnyRef](dbName, catalogNameValue))
+ }
+ // Global temp view database is only relevant for spark_catalog
+ val globalTempViewDb = catalog.globalTempDatabase
+ val databasePattern =
Pattern.compile(CLIServiceUtils.patternToRegex(schemaName))
+ if (schemaName == null || schemaName.isEmpty ||
+ databasePattern.matcher(globalTempViewDb).matches()) {
+ rowSet.addRow(Array[AnyRef](globalTempViewDb, catalogNameValue))
+ }
+ } else {
+ resolvedCatalog match {
+ case nsCatalog: SupportsNamespaces =>
+ val databasePattern = Pattern.compile(
+ CLIServiceUtils.patternToRegex(schemaName))
+ nsCatalog.listNamespaces().foreach { ns =>
+ // Only top-level namespaces (depth=1) are exposed as JDBC
schemas.
+ val nsName = ns.head
+ if (schemaName == null || schemaName.isEmpty ||
+ databasePattern.matcher(nsName).matches()) {
+ rowSet.addRow(Array[AnyRef](nsName, catalogNameValue))
+ }
+ }
+ case _ =>
+ // Catalog doesn't support namespaces — return empty
Review Comment:
Fixed - replaced the em-dash with ASCII '--' (applied your suggestion), and
confirmed no other non-ASCII in the changed lines.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
