Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/4067#issuecomment-72975805
Hey @ksakellis - I did a pretty thorough review here, any chance you could
take a look? Most of the things were small. The only major thing is that I
think if we are going to always show the output metrics for Hadoop (even for
older versions) we need to make sure that we at least do a "best effort"
presentation of the output size for older versions. I think it would be good if
we could sneak that in, since otherwise that output metrics thing is not so
useful for many people running with older versions. However, I'd also be okay
just not showing either bytes or count in that case (which I think was the
behavior before).
A second issue I realized (let's open a new JIRA for this) is that we never
track output metrics (size or bytes) when data is written to cache, which is
confusing because we track input metrics when data is read from cache.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
