Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/14897#discussion_r81281510
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -2433,31 +2433,65 @@ class Dataset[T] private[sql](
}
/**
- * Creates a temporary view using the given name. The lifetime of this
+ * Creates a local temporary view using the given name. The lifetime of
this
* temporary view is tied to the [[SparkSession]] that was used to
create this Dataset.
*
+ * Local temporary view is session-scoped. Its lifetime is the lifetime
of the session that
+ * created it, i.e. it will be automatically dropped when the session
terminates. It's not
+ * tied to any databases, i.e. we can't use `db1.view1` to reference a
local temporary view.
+ *
* @throws AnalysisException if the view name already exists
*
* @group basic
* @since 2.0.0
*/
@throws[AnalysisException]
def createTempView(viewName: String): Unit = withPlan {
- createViewCommand(viewName, replace = false)
+ createTempViewCommand(viewName, replace = false, global = false)
}
+
+
/**
- * Creates a temporary view using the given name. The lifetime of this
+ * Creates a local temporary view using the given name. The lifetime of
this
* temporary view is tied to the [[SparkSession]] that was used to
create this Dataset.
*
* @group basic
* @since 2.0.0
*/
def createOrReplaceTempView(viewName: String): Unit = withPlan {
- createViewCommand(viewName, replace = true)
+ createTempViewCommand(viewName, replace = true, global = false)
}
- private def createViewCommand(viewName: String, replace: Boolean):
CreateViewCommand = {
+ /**
+ * Creates a global temporary view using the given name. The lifetime of
this
+ * temporary view is tied to this Spark application.
+ *
+ * Global temporary view is cross-session. Its lifetime is the lifetime
of the Spark application,
+ * i.e. it will be automatically dropped when the application
terminates. It's tied to a system
+ * preserved database `_global_temp`, and we must use the qualified name
to refer a global temp
+ * view, e.g. `SELECT * FROM _global_temp.view1`.
+ *
+ * @throws TempTableAlreadyExistsException if the view name already
exists
+ *
+ * @group basic
+ * @since 2.1.0
+ */
+ @throws[AnalysisException]
+ def createGlobalTempView(viewName: String): Unit = withPlan {
+ createTempViewCommand(viewName, replace = false, global = true)
--- End diff --
```Scala
Seq(1 -> "a").toDF("i", "j").createGlobalTempView(s"$globalTempDB.src")
```
We will get the following error:
```
org.apache.spark.sql.AnalysisException: It is not allowed to add database
prefix `global_temp` for the TEMPORARY view name.;
```
I am wondering if we should also support it for the global temp view?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
