LantaoJin opened a new pull request #28901: URL: https://github.com/apache/spark/pull/28901
### What changes were proposed in this pull request? Many databases and data warehouse SQL engines support temporary tables. A temporary table, as its named implied, is a short-lived table that its life will be only for current session. [Hive Temporary Table](https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.0.1/using-hiveql/content/hive_create_a_hive_temporary_table.html) [Teradata Volatile Table](https://docs.teradata.com/reader/rgAb27O_xRmMVc_aQq2VGw/mpJF1z_vSlpMbZYxFmRJfA) [PostgreSQL Temporary Table](https://www.postgresql.org/docs/12/sql-createtable.html) In Spark, there is no temporary table. the DDL “CREATE TEMPORARY TABLE AS SELECT” will create a temporary view. A temporary view is totally different with a temporary table. This ticket to support Spark native temporary table. More details are described in [DESIGN DOCS](https://docs.google.com/document/d/1RS4Q3VbxlZ_Yy0fdWgTJ-k0QxFd1dToCqpLAYvIJ34U/edit?usp=sharing) ### Why are the changes needed? A temporary view is just a VIEW. It doesn’t materialize data in storage. So it has below shortage: 1. View will not give improved performance. Materialize intermediate data in temporary tables for a complex query will accurate queries, especially in an ETL pipeline. 2. View which calls other views can cause severe performance issues. Even, executing a very complex view may fail in Spark. 3. Temporary view has no database namespace. In some complex ETL pipelines or data warehouse applications, without database prefix is not convenient. It needs some tables which only used in current session. ### Does this PR introduce _any_ user-facing change? YES. ```sql CREATE TEMPORARY TABLE tt1 AS SELECT .. ``` before the patch, it will create a local temporary VIEW. After this patch, it will create a temporary table. ```sql CREATE TEMPORARY TABLE tt1 USING .. ``` before the patch, it will throw exception. After this patch, it will create a temporary table. ### How was this patch tested? Add unit tests. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
