[
https://issues.apache.org/jira/browse/FLINK-10423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629155#comment-16629155
]
Seth Wiesman edited comment on FLINK-10423 at 10/9/18 7:34 PM:
---------------------------------------------------------------
-I have a question about the best way to handle resource management of the
RocksDB native handler.-
-The basic implementation uses a TimerTask to periodically pull metrics from
RocksDB, this works fine until it is time to release the resource. The question
I have is how to be a good citizen with the ResourceGuard. Manually calling
cancel on the TimerTask before calling the blocking cancel on ResourceGuard
seems brittle in the face of refactoring and it does not seem there is any way
to access a CancellableRegistry from within RocksDBStateBackend.-
-The simplest solution I've been able to come up with is to add a new method
to ResourceGuard that effectively acts as a way to aquire a weak reference.-
{code:java}
public boolean runIfResourceAvailable(Runnable function) {
synchronized (lock) {
if (!closed) {
function.run();
}
return closed;
}
}
{code}
-Run atomically only if the resource is available and return if the function
was run or not. When this method returns false the calling class knows the
resource is being or has been closed and it is time to shut down. The concern I
have here is the potential of putting to much work inside of the synchronized
block.-
-I'm curious what you think is the best path forward.-
Nevermind
was (Author: sjwiesman):
[~srichter] I have a question about the best way to handle resource management
of the RocksDB native handler.{color}{color}
The basic implementation uses a TimerTask to periodically pull metrics from
RocksDB, this works fine until it is time to release the resource. The question
I have is how to be a good citizen with the ResourceGuard. Manually calling
cancel on the TimerTask before calling the blocking cancel on ResourceGuard
seems brittle in the face of refactoring and it does not seem there is any way
to access a CancellableRegistry from within RocksDBStateBackend.
The simplest solution I've been able to come up with is to add a new method to
ResourceGuard that effectively acts as a way to aquire a weak reference.
{code:java}
public boolean runIfResourceAvailable(Runnable function) {
synchronized (lock) {
if (!closed) {
function.run();
}
return closed;
}
}
{code}
Run atomically only if the resource is available and return if the function was
run or not. When this method returns false the calling class knows the resource
is being or has been closed and it is time to shut down. The concern I have
here is the potential of putting to much work inside of the synchronized block.
I'm curious what you think is the best path forward.
> Forward RocksDB native metrics to Flink metrics reporter
> ---------------------------------------------------------
>
> Key: FLINK-10423
> URL: https://issues.apache.org/jira/browse/FLINK-10423
> Project: Flink
> Issue Type: New Feature
> Components: Metrics, State Backends, Checkpointing
> Reporter: Seth Wiesman
> Assignee: Seth Wiesman
> Priority: Major
> Labels: pull-request-available
>
> RocksDB contains a number of metrics at the column family level about current
> memory usage, open memtables, etc that would be useful to users wishing
> greater insight what rocksdb is doing. This work is inspired heavily by the
> comments on this rocksdb issue thread
> (https://github.com/facebook/rocksdb/issues/3216#issuecomment-348779233)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
