Github user lianhuiwang commented on the pull request:
https://github.com/apache/spark/pull/4474#issuecomment-73627335
@andrewor14 @pwendell yes, Now we have
conf.get("spark.driver.maxResultSize", "1g") to control the memory of driver.
but once OOM is happened at TaskResultGetter, the application will be hold and
waste resouces. so i think this is a very serious and at least we should to let
application abort and let user know why his job is wrong.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
