[
https://issues.apache.org/jira/browse/SPARK-9139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcelo Vanzin updated SPARK-9139:
----------------------------------
Priority: Major (was: Critical)
> Add backwards-compatibility tests for DataType.fromJson()
> ---------------------------------------------------------
>
> Key: SPARK-9139
> URL: https://issues.apache.org/jira/browse/SPARK-9139
> Project: Spark
> Issue Type: Test
> Components: SQL
> Reporter: Josh Rosen
> Priority: Major
>
> SQL's DataType.fromJson is a public API and thus must be
> backwards-compatible; there are also backwards-compatibility concerns related
> to persistence of DataType JSON in metastores.
> Unfortunately, we do not have any backwards-compatibility tests which attempt
> to read old JSON values that were written by earlier versions of Spark.
> DataTypeSuite has "roundtrip" tests that test fromJson(toJson(foo)), but this
> doesn't ensure compatibility.
> I think that we should address this by capuring the JSON strings produced in
> Spark 1.3's DataFrameSuite and adding test cases that try to create DataTypes
> from those strings.
> This might be a good starter task for someone who wants to contribute to SQL
> tests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
