Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/11938#discussion_r57447378
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
---
@@ -31,17 +32,34 @@ import
org.apache.spark.sql.catalyst.plans.logical.{LogicalPlan, SubqueryAlias}
* proxy to the underlying metastore (e.g. Hive Metastore) and it also
manages temporary
* tables and functions of the Spark Session that it belongs to.
*/
-class SessionCatalog(externalCatalog: ExternalCatalog) {
+class SessionCatalog(externalCatalog: ExternalCatalog, conf: CatalystConf)
{
import ExternalCatalog._
- private[this] val tempTables = new ConcurrentHashMap[String, LogicalPlan]
- private[this] val tempFunctions = new ConcurrentHashMap[String,
CatalogFunction]
+ def this(externalCatalog: ExternalCatalog) {
+ this(externalCatalog, new SimpleCatalystConf(true))
+ }
+
+ protected[this] val tempTables = new ConcurrentHashMap[String,
LogicalPlan]
+ protected[this] val tempFunctions = new ConcurrentHashMap[String,
CatalogFunction]
// Note: we track current database here because certain operations do
not explicitly
// specify the database (e.g. DROP TABLE my_table). In these cases we
must first
// check whether the temporary table or function exists, then, if not,
operate on
// the corresponding item in the current database.
- private[this] var currentDb = "default"
+ protected[this] var currentDb = {
+ val defaultName = "default"
+ val defaultDbDefinition = CatalogDatabase(defaultName, "default
database", "", Map())
--- End diff --
The `currentDb` is also shared by `HiveSessionCatalog`. This is a very
important variable in the Session Catalog. We could use the property of this
variable in the other functions. If the value is not right, it could cause
hidden bugs.
Just after this line, we are trying to create such a default database
https://github.com/apache/spark/pull/11938/files#diff-b3f9800839b9b9a1df9da9cbfc01adf8R53
This looks a little bit confusing. Can we assume that the default database
in Hive always exists? If so, in `HiveSessionCatalog`, we will not call
`createDatabase`. Instead, we can just call `getDatabase` and use the returned
value to populate the default `currentDb` property?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
