Github user aarondav commented on a diff in the pull request:
https://github.com/apache/spark/pull/3073#discussion_r19923183
--- Diff: core/src/main/scala/org/apache/spark/TaskEndReason.scala ---
@@ -88,10 +88,32 @@ case class FetchFailed(
case class ExceptionFailure(
className: String,
description: String,
- stackTrace: Array[StackTraceElement],
+ stackTrace: Array[StackTraceElement], // backwards compatibility
--- End diff --
@zsxwing To be clear, adding new fields in the constructor actually breaks
compatibility for case classes, because you can do things like
```scala
failure match {
case ExceptionFailure(className, description, stackTrace, metrics) =>
}
```
which would fail to compile when you add a new parameter.
However, Patrick's point was that this behavior makes extending these
classes very difficult, so we should make a one-time breaking change where we
get rid of the case-class-ness, and just use regular classes, where you _can_
add new fields arbitrarily. You can even remove fields if you provide a `def`
which fakes them.
In this patch, let's go ahead and break compatibility by adding a new field
as you said. Let's additionally file a JIRA to make these normal classes
instead of case classes in a later patch, and, if possible, try to get both
that and this into 1.2.0 to just embrace the compatibility issues.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
