Github user yzhou2001 commented on a diff in the pull request:
https://github.com/apache/spark/pull/12598#discussion_r61183058
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala
---
@@ -324,7 +324,9 @@ private[joins] object UnsafeHashedRelation {
*
* see
http://java-performance.info/implementing-world-fastest-java-int-to-int-hash-map/
*/
-private[execution] final class LongToUnsafeRowMap(var mm:
TaskMemoryManager, capacity: Int)
+private[execution] final class LongToUnsafeRowMap(
+ @transient val mm: TaskMemoryManager,
--- End diff --
@srowen Thanks for reviewing. I think the issue is whether the
TaskMemoryManager should be serialized and used by the receiving end or not. If
it should not be used by the receiving end, not by accident but by design as I
believe, then it should not be serialized and sent over at all.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
