GitHub user bowenli86 opened a pull request:
https://github.com/apache/flink/pull/4798
[FLINK-6505] Proactively cleanup local FS for RocksDBKeyedStateBackend on
startup
## What is the purpose of the change
In `RocksDBKeyedStateBackend`, the `instanceBasePath` is cleared on
`dispose()`. It also make sense to also clear this directory when the backend
is created, in case something crashed and the backend never reached
`dispose()`. At least for previous runs of the same job, we can know what to
delete on restart.
In general, it is very important for this backend to clean up the local FS,
because the local quota might be very limited compared to the DFS. And a node
that runs out of local disk space can bring down the whole job, with no way to
recover (it might always get rescheduled to that node).
## Brief change log
clear `instanceBasePath` when `RocksDBKeyedStateBackend ` is created
## Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
## Does this pull request potentially affect one of the following parts:
none
## Documentation
none
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/bowenli86/flink FLINK-6505
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/4798.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4798
----
commit 3c4ea092759f2df052cc3dc02403d041f4c16b2d
Author: Bowen Li <[email protected]>
Date: 2017-10-10T05:31:17Z
[FLIN-6505] Proactively cleanup local FS for RocksDBKeyedStateBackend on
startup
commit 22a761736d43114fb5b935d53df65bcf3832f02d
Author: Bowen Li <[email protected]>
Date: 2017-10-11T10:41:24Z
add comment
----
---
