[
https://issues.apache.org/jira/browse/FLINK-18242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17132880#comment-17132880
]
Yun Tang edited comment on FLINK-18242 at 6/11/20, 4:07 AM:
------------------------------------------------------------
[~NicoK] The root cause is current
{{[RocksDBResourceContainer|https://github.com/apache/flink/blob/88cc44afbfb5267e2674ecb735561365e735d2b0/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBResourceContainer.java#L90]}}
would only deal with new interface
{{DefaultConfigurableOptionsFactory#createDBOptions(DBOptions currentOptions,
Collection<AutoCloseable> handlesToClose)}}.
As you only override the deprecated interface, that's why the new options not
take effect.
I think this bug would only occur if user extends
{{DefaultConfigurableOptionsFactory}} as we migrate
{{DefaultConfigurableOptionsFactory}} to new {{RocksDBOptionsFactory}} but
still keep previous interfaces. Not sure whether extending
{{DefaultConfigurableOptionsFactory}} has been ‘abused’ among users.
was (Author: yunta):
[~NicoK] The root cause is current
{{[RocksDBResourceContainer|https://github.com/apache/flink/blob/88cc44afbfb5267e2674ecb735561365e735d2b0/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBResourceContainer.java#L90]}}
would only deal with new interface
{{DefaultConfigurableOptionsFactory#createDBOptions(DBOptions currentOptions,
Collection<AutoCloseable> handlesToClose)}}.
As you only override the deprecated interface, that's why the new options not
take effect.
This is certainly a bug since we migrate {{DefaultConfigurableOptionsFactory}}
to new {{RocksDBOptionsFactory}}.
> Custom OptionsFactory settings seem to have no effect on RocksDB
> ----------------------------------------------------------------
>
> Key: FLINK-18242
> URL: https://issues.apache.org/jira/browse/FLINK-18242
> Project: Flink
> Issue Type: Bug
> Components: Runtime / State Backends
> Affects Versions: 1.10.0, 1.10.1, 1.11.0
> Reporter: Nico Kruber
> Priority: Major
> Attachments: DefaultConfigurableOptionsFactoryWithLog.java
>
>
> When I configure a custom {{OptionsFactory}} for RocksDB like this
> (similarly by specifying it via the {{state.backend.rocksdb.options-factory}}
> configuration):
> {code:java}
> Configuration globalConfig = GlobalConfiguration.loadConfiguration();
> String checkpointDataUri =
> globalConfig.getString(CheckpointingOptions.CHECKPOINTS_DIRECTORY);
> RocksDBStateBackend stateBackend = new RocksDBStateBackend(checkpointDataUri);
> stateBackend.setOptions(new DefaultConfigurableOptionsFactoryWithLog());
> env.setStateBackend((StateBackend) stateBackend);{code}
> it seems to be loaded
> {code:java}
> 2020-06-10 12:54:20,720 INFO
> org.apache.flink.contrib.streaming.state.RocksDBStateBackend - Using
> predefined options: DEFAULT.
> 2020-06-10 12:54:20,721 INFO
> org.apache.flink.contrib.streaming.state.RocksDBStateBackend - Using
> application-defined options factory:
> DefaultConfigurableOptionsFactoryWithLog{DefaultConfigurableOptionsFactory{configuredOptions={}}}.
> {code}
> but it seems like none of the options defined in there is actually used. Just
> as an example, my factory does set the info log level to {{INFO_LEVEL}} but
> this is what you will see in the created RocksDB instance:
> {code:java}
> > cat /tmp/flink-io-c95e8f48-0daa-4fb9-a9a7-0e4fb42e9135/*/db/OPTIONS*|grep
> > info_log_level
> info_log_level=HEADER_LEVEL
> info_log_level=HEADER_LEVEL{code}
> Together with the bug from FLINK-18241, it seems I cannot re-activate the
> RocksDB log that we disabled in FLINK-15068. FLINK-15747 was aiming at
> changing that particular configuration, but the problem seems broader since
> {{setDbLogDir()}} was actually also ignored and Flink itself does not change
> that setting.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
