Github user holdenk commented on the pull request:
https://github.com/apache/spark/pull/11105#issuecomment-208963389
@viirya - I think having consistent accumulators only be consistent inside
some of the Spark operations might be a little confusing, although that's also
where I started the implementation, and I think one still needs to do some
wrapping or explicitly pass the RDD ID in to the accumulator to determine which
RDD exactly it's coming from when it's chained together on the same iterator.
This PR only has the overhead if the task has a consistent accumulator
otherwise no wrapping will occur.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
