Github user sryza commented on the pull request:
https://github.com/apache/spark/pull/3000#issuecomment-74034874
@jkbradley sorry for the delay in responding here. Your breakdown of
operations makes sense to me.
A stats collector seems like a good idea. I also wonder if there's some
way to hook it in with Hive table statistics so we can avoid a pass over the
data, but maybe that should be saved for future. If you aren't planning to get
to this in the near future, but think you'll have bandwidth to review, I'd be
happy to work on it. Otherwise, I'm happy to look over whatever you put up.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
