GitHub user hvanhovell opened a pull request:
https://github.com/apache/spark/pull/15832
[SPARK-18370][SQL] Add table information to
InsertIntoHadoopFsRelationCommand
## What changes were proposed in this pull request?
`InsertIntoHadoopFsRelationCommand` does not keep track if it inserts into
a table and what table it inserts to. This can make debugging these statements
problematic. This PR adds table information the
`InsertIntoHadoopFsRelationCommand`. Explaining this SQL command `insert into
prq select * from range(0, 100000)` now yields the following executed plan:
```
== Physical Plan ==
ExecutedCommand
+- InsertIntoHadoopFsRelationCommand
file:/dev/assembly/spark-warehouse/prq, ParquetFormat, <function1>,
Map(serialization.format -> 1, path -> file:/dev/assembly/spark-warehouse/prq),
Append, CatalogTable(
Table: `default`.`prq`
Owner: hvanhovell
Created: Wed Nov 09 17:42:30 CET 2016
Last Access: Thu Jan 01 01:00:00 CET 1970
Type: MANAGED
Schema: [StructField(id,LongType,true)]
Provider: parquet
Properties: [transient_lastDdlTime=1478709750]
Storage(Location: file:/dev/assembly/spark-warehouse/prq, InputFormat:
org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat, OutputFormat:
org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat, Serde:
org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe, Properties:
[serialization.format=1]))
+- Project [id#7L]
+- Range (0, 100000, step=1, splits=None)
```
## How was this patch tested?
Added extra checks to the `ParquetMetastoreSuite`
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/hvanhovell/spark SPARK-18370
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/15832.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #15832
----
commit dea8a579bbf1a9935d8ebda6d7b94acb9fb04755
Author: Herman van Hovell <[email protected]>
Date: 2016-11-09T16:47:18Z
Add table information to InsertIntoHadoopFsRelationCommand.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
