Ewan Leith created SPARK-10947: ---------------------------------- Summary: With schema inference from JSON into a Dataframe, add option to infer all primitive object types as strings Key: SPARK-10947 URL: https://issues.apache.org/jira/browse/SPARK-10947 Project: Spark Issue Type: Improvement Components: SQL Affects Versions: 1.5.1 Reporter: Ewan Leith Priority: Minor
Currently, when a schema is inferred from a JSON file using sqlContext.read.json, the primitive object types are inferred as string, long, boolean, etc. However, if the inferred type is too specific (JSON obviously does not enforce types itself), this causes issues with merging dataframe schemas. Instead, we would like an option in the JSON inferField function to treat all primitive objects as strings. We'll create and submit a pull request for this for review. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org