GitHub user liupc opened a pull request:
https://github.com/apache/spark/pull/20090
[SPARK-22907]Clean broadcast garbage when IOException occurred in
MetadataFetch
## What changes were proposed in this pull request?
This PR will clean garbage broadcast when IOException occurred in
MetadataFetch to avoid
the MedataFetchFailedException happens again which is caused by 'broadcast
already present' exception
details see: https://issues.apache.org/jira/browse/SPARK-22907
(Please fill in changes proposed in this fix)
## How was this patch tested?
manual
(Please explain how this patch was tested. E.g. unit tests, integration
tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise,
remove this)
Please review http://spark.apache.org/contributing.html before opening a
pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/liupc/spark
Clean-broadcast-garbage-when-ioexception-occurred-in-metadataFetch
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20090.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20090
----
commit df9e05200b76c36647147131e2d2f3dc60c6804b
Author: liupengcheng <liupengcheng@...>
Date: 2017-12-27T09:06:28Z
Clean broadcast garbage when IOException occurred in MetadataFetch
Summary: Ref T10399
Test Plan: n/a
Reviewers: liushaohui, peng.zhang, zhoukang, chenfan
Subscribers: cloud-computing
Maniphest Tasks: T10399
Differential Revision: https://phabricator.d.xiaomi.net/D79279
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
