Muraliraja created SPARK-24289:
----------------------------------
Summary: Unable to Infer schema
Key: SPARK-24289
URL: https://issues.apache.org/jira/browse/SPARK-24289
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 2.2.1
Reporter: Muraliraja
Fix For: 2.2.1
When we run a spark job and when the datatypes during schema inference change
and the same is written as parquet files, then when we try to read the data, it
fails or mergeSchema fails.
A better way would be to upcast the datatypes to the highest types via a spark
option.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
