Yin Huai created SPARK-15712:
--------------------------------
Summary: Proper temp table support
Key: SPARK-15712
URL: https://issues.apache.org/jira/browse/SPARK-15712
Project: Spark
Issue Type: Bug
Components: SQL
Reporter: Yin Huai
For proper temp table support, I am proposing to create a temp dir for every
SparkSession. This can be done through the following changes:
1. We add a root temp dir (the path is configurable).
2. For every SparkSession, we create a temp dir for this SparkSession in the
root temp dir.
To make the semantic of CREATE TEMP TABLE ... USING ... OPTIONS ... AS SELECT
easier to understand, I am proposing the following changes:
1. We only allow HadoopFsRelations used by this command.
2. We make USING clause a optional clause and by default we use the format set
in spark.sql.sources.default.
3. We do not allow users to add path in the OPTIONS
4. We save the temp table in the temp location associated with the current
SparkSession.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
