Github user jiangxb1987 commented on a diff in the pull request:
https://github.com/apache/spark/pull/16613#discussion_r96562319
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala ---
@@ -207,29 +209,26 @@ case class CreateViewCommand(
}
/**
- * Returns a [[CatalogTable]] that can be used to save in the catalog.
This comment canonicalize
- * SQL based on the analyzed plan, and also creates the proper schema
for the view.
+ * Returns a [[CatalogTable]] that can be used to save in the catalog.
Generate the view-specific
+ * properties(e.g. view default database, view query output column
names) and store them as
+ * properties in the CatalogTable, and also creates the proper schema
for the view.
*/
- private def prepareTable(sparkSession: SparkSession, aliasedPlan:
LogicalPlan): CatalogTable = {
- val viewSQL: String = new SQLBuilder(aliasedPlan).toSQL
-
- // Validate the view SQL - make sure we can parse it and analyze it.
- // If we cannot analyze the generated query, there is probably a bug
in SQL generation.
- try {
- sparkSession.sql(viewSQL).queryExecution.assertAnalyzed()
- } catch {
- case NonFatal(e) =>
- throw new RuntimeException(s"Failed to analyze the canonicalized
SQL: $viewSQL", e)
+ private def prepareTable(session: SparkSession, aliasedPlan:
LogicalPlan): CatalogTable = {
+ if (originalText.isEmpty) {
+ throw new AnalysisException(
+ "It is not allowed to create a persisted view from the Dataset
API")
}
+ val newProperties = generateViewProperties(properties, session,
originalText.get)
+
CatalogTable(
identifier = name,
tableType = CatalogTableType.VIEW,
storage = CatalogStorageFormat.empty,
schema = aliasedPlan.schema,
- properties = properties,
+ properties = newProperties,
viewOriginalText = originalText,
- viewText = Some(viewSQL),
+ viewText = originalText,
--- End diff --
I'm afraid that would require changes of several tens of places, should we
do that in a seprated PR?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
