[
https://issues.apache.org/jira/browse/SPARK-6495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14379243#comment-14379243
]
Chaozhong Yang edited comment on SPARK-6495 at 3/25/15 4:08 AM:
----------------------------------------------------------------
Thanks! Maybe what you point at is the resolved issue
https://issues.apache.org/jira/browse/SPARK-3851. Reading data from parquet
files with different but compatible schemas has been support in Spark 1.3.0.
https://spark.apache.org/docs/latest/sql-programming-guide.html#schema-merging
was (Author: debugger87):
Thanks! Maybe what you point at is the resolved issue
https://issues.apache.org/jira/browse/SPARK-3851. Reading data from parquet
files with different but compatible schemas has been support in Spark 1.3.0.
> DataFrame#insertInto method should support insert rows with sub-columns
> -----------------------------------------------------------------------
>
> Key: SPARK-6495
> URL: https://issues.apache.org/jira/browse/SPARK-6495
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Reporter: Chaozhong Yang
>
> The original table's schema is like this:
> |-- a: string (nullable = true)
> |-- b: string (nullable = true)
> |-- c: string (nullable = true)
> |-- d: string (nullable = true)
> If we want to insert one row(can be transformed into DataFrame) with this
> schema:
> |-- a: string (nullable = true)
> |-- b: string (nullable = true)
> |-- c: string (nullable = true)
> Of course, that operation will fail. Actually, in many cases, people need to
> insert new rows with columns which is the subset of original table columns.
> If we can support and fix those issue, Spark SQL's insertion can be more
> valuable to users.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
