[
https://issues.apache.org/jira/browse/FLINK-10198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588897#comment-16588897
]
ASF GitHub Bot commented on FLINK-10198:
----------------------------------------
StefanRRichter opened a new pull request #6603: [FLINK-10198][state] Set Env
object in DBOptions for RocksDB
URL: https://github.com/apache/flink/pull/6603
## What is the purpose of the change
This PR always sets a default environment when creating the `DBOptions`.
This could simplify resource management for multiple RocksDB instances on one
machine.
See https://github.com/facebook/rocksdb/wiki/rocksdb-basics:
Support for Multiple Embedded Databases in the same process
A common use-case for RocksDB is that applications inherently partition
their data set into logical partitions or shards. This technique benefits
application load balancing and fast recovery from faults. This means that a
single server process should be able to operate multiple RocksDB databases
simultaneously. This is done via an environment object named Env. Among other
things, a thread pool is associated with an Env. If applications want to share
a common thread pool (for background compactions) among multiple database
instances, then it should use the same Env object for opening those databases.
Similarly, multiple database instances may share the same block cache.
## Verifying this change
This change added tests and can be verified as follows:
`RocksDBStateBackendConfigTest#testSetDefaultEnvInOptions`
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (yes)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not applicable)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Set Env object in DBOptions for RocksDB
> ---------------------------------------
>
> Key: FLINK-10198
> URL: https://issues.apache.org/jira/browse/FLINK-10198
> Project: Flink
> Issue Type: Improvement
> Affects Versions: 1.7.0
> Reporter: Stefan Richter
> Assignee: Stefan Richter
> Priority: Major
> Labels: pull-request-available
>
> I think we should consider to always set a default environment when we create
> the DBOptions.
> See https://github.com/facebook/rocksdb/wiki/rocksdb-basics:
> *Support for Multiple Embedded Databases in the same process*
> A common use-case for RocksDB is that applications inherently partition their
> data set into logical partitions or shards. This technique benefits
> application load balancing and fast recovery from faults. This means that a
> single server process should be able to operate multiple RocksDB databases
> simultaneously. This is done via an environment object named Env. Among other
> things, a thread pool is associated with an Env. If applications want to
> share a common thread pool (for background compactions) among multiple
> database instances, then it should use the same Env object for opening those
> databases.
> Similarly, multiple database instances may share the same block cache.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
