Github user squito commented on the issue:
https://github.com/apache/spark/pull/15505
@witgo -- right, I'm saying that we should make local mode work to make
debugging & development as easy as possible, not to maximize performance. That
means we *should* do the extra serialization in local mode, to mimic the
behavior of running on a cluster.
It doesn't look like task serialization still happens in local mode to me.
Tasks only get serialized inside `TaskDescription.encode`, but that is only
called by `CoarseGrainedSchedulerBackend`. Before this change, task
serialization happened in
[`localScheduler.reviveOffers`](https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/scheduler/local/LocalSchedulerBackend.scala#L85),
because it happened inside the `TaskSchedulerImpl`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
