Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/6625#discussion_r32773925
--- Diff:
core/src/main/scala/org/apache/spark/serializer/SerializationDebugger.scala ---
@@ -169,18 +208,53 @@ private[spark] object SerializationDebugger extends
Logging {
}
j += 1
}
-
}
i += 1
}
return List.empty
}
+
+ /**
+ * Visit a serializable object which has the writeObject() defined.
+ * Since writeObject() can choose add arbitrary objects at the time of
serialization,
+ * the only way to capture all the objects it will serialize is by
using a
+ * dummy ObjectOutputStream that collects all the relevant fields for
further testing.
+ */
+ private def visitSerializableWithWriteObjectMethod(
+ o: Object, stack: List[String]): List[String] = {
+ val innerObjectsCatcher = new ListObjectOutputStream
+ var notSerializableFound = false
+ try {
+ innerObjectsCatcher.writeObject(o)
+ } catch {
+ case io: IOException =>
+ notSerializableFound = true
+ }
+
+ // If something was not serializable, then visit the captured
objects.
+ // Otherwise, all the captured objects are safely serializable, so
no need to visit them.
+ // As an optimization, just added them to the visited list.
--- End diff --
it doesn't really matter as long as we get the message across... if you
prefer we can rephrase this as "Only visit the captured objects if everything
is serializable"
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
