Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/15155
Yea. I meant if we want to read old/new Parquet files without user-given
schema with enabling merging schemas, then, we'd face SPARK-15516 first. This
is why I thought that JIRA blocks this
Github user wgtmac commented on the issue:
https://github.com/apache/spark/pull/15155
@HyukjinKwon Yup this PR is very similar to yours.
For merging parquet schema, it won't work. Think about this: the table
contains two parquet files, one has int, one has long. The DataFrame
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/15155
-1 : As far as I know, we are picking up a single Parquet file to read
Spark-side schema. In this case, it is ambiguous to decide which one is "new"
and "old". So, sometimes it'd be failed to
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/15155
Do you mind if I ask to fix the title to be complete withoit ... ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15155
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
