Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/7770#issuecomment-127098293
This looks good to me overall. I think that there are a couple of small
tasks that we might want to defer to smaller followup patches:
- Figuring out whether we want to account for broadcast variables via a
separate "shared task memory" metric.
- If necessary, blocking one of the SQL tests on cleanup of TestSQLContext
(this can happen during the QA period).
The only minor blockers at this point are some slight fixes to where we
update metrics in order to guard against breakage due to future changes (e.g.
ensure that the update will still be correct if a non-destructive iterator was
updated to be destructive).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
