Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/3009#discussion_r20759212
--- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala ---
@@ -31,6 +31,21 @@ import org.apache.spark.scheduler._
import org.apache.spark.storage._
import org.apache.spark._
+/**
+ * Serializes SparkListener events to/from JSON. This protocol provides
strong backwards-
+ * and forwards-compatibility guarantees: any version of Spark should be
able to read JSON output
+ * written by any other version, including newer versions.
+ *
+ * JsonProtocolSuite contains backwards-compatibility tests which check
that the current version of
+ * JsonProtocol is able to read output written by earlier versions. We do
not currently have tests
+ * for reading newer JSON output with older Spark versions.
+ *
+ * To ensure that we provide these guarantees, follow these rules when
modifying these methods:
+ *
+ * - Never delete any JSON fields.
+ * - Any new JSON fields should be optional; use `Utils.jsonOption` when
reading these fields
+ * in `*FromJson` methods.
+ */
--- End diff --
I agree; I've filed https://issues.apache.org/jira/browse/SPARK-4555 for
this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
