Nico Kruber commented on FLINK-14378:

I believe a proper cleanup should cover both scenarios and a fix for this one 
probably also fixes the other issue. I'm closing FLINK-15011 as a duplicate.


Just to clarify here: we should also cleanup the {{rocksdb-lib-<uid>}} 
directory upon graceful shutdown.

> Cleanup rocksDB lib folder if fail to load library
> --------------------------------------------------
>                 Key: FLINK-14378
>                 URL: https://issues.apache.org/jira/browse/FLINK-14378
>             Project: Flink
>          Issue Type: Improvement
>          Components: Runtime / State Backends
>            Reporter: Yun Tang
>            Assignee: Yun Tang
>            Priority: Major
> This improvement is inspired due to some of our machines need some time to 
> load the rocksDB library. When some other unrecoverable exceptions continue 
> to happen and the process to load library would be interrupted which cause 
> the {{rocksdb-lib}} folder created but not cleaned up. As the job continues 
> to failover, the {{rocksdb-lib}} folder would be created more and more. We 
> even come across that machine was running out of inodes!
> Details could refer to current 
> [implementation|https://github.com/apache/flink/blob/80b27a150026b7b5cb707bd9fa3e17f565bb8112/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBStateBackend.java#L860]

This message was sent by Atlassian Jira

Reply via email to