Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/17596
@mridulm I actually have the same plan, I think it's an overkill to
implement TaskMetrics with accumulators, we don't need to merge the accumulator
updates at driver side for TaskMetrics accumulators. We should send back
TaskMetrics directly with hearbeat, task failure and task finish, then we can
just send a bunch of `long`s and compress.
One thing I'm not 100% agree with you is about the bitmask. According to my
experiment, most of the task metrics will not be 0, so the bitmask may not be
very useful.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
