stczwd opened a new pull request #28280:
URL: https://github.com/apache/spark/pull/28280
### What changes were proposed in this pull request?
In Spark, we need do some hook after job cleaned, such as cleaning hive
external temporary paths. This has already discussed in [GitHub Pull Request
#28129](https://github.com/apache/spark/pull/28129).
The JobEnd Status is not suitable for this. As JobEnd is responsible for Job
finished, once all result has generated, it should be finished. After finish,
Scheduler will leave the still running tasks to be zombie tasks and delete
abnormal tasks asynchronously.
Thus, we add JobCleaned Status to enable user to do some hook after all
tasks cleaned in the job and add a JobCleanedListener to do some hooks.
### Does this PR introduce any user-facing change?
NO
### How was this patch tested?
run all tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
