Josh Rosen created SPARK-9139:
---------------------------------
Summary: Add backwards-compatibility tests for DataType.fromJson()
Key: SPARK-9139
URL: https://issues.apache.org/jira/browse/SPARK-9139
Project: Spark
Issue Type: Test
Reporter: Josh Rosen
Priority: Critical
SQL's DataType.fromJson is a public API and thus must be backwards-compatible;
there are also backwards-compatibility concerns related to persistence of
DataType JSON in metastores.
Unfortunately, we do not have any backwards-compatibility tests which attempt
to read old JSON values that were written by earlier versions of Spark.
DataTypeSuite has "roundtrip" tests that test fromJson(toJson(x)), but this
doesn't ensure compatibility.
I think that we should address this by capuring the JSON strings produced in
Spark 1.3's DataFrameSuite and adding test cases that try to create DataTypes
from those strings.
This might be a good starter task for someone who wants to contribute to SQL
tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
