[
https://issues.apache.org/jira/browse/FLINK-20684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17252832#comment-17252832
]
Yun Tang commented on FLINK-20684:
----------------------------------
Before we decide to add this option in \{{RocksDBConfigurableOptions}}, we have
several questions to answer:
* What kind of options we want to support. We can set the overall compression
type or we can also set different compression type at different level.
* Can this option break the checkpoint compatibility? If user choose snappy as
previous compression type, and then he decide to switch to LZ4 as current
compression type. Can the new job restore from previous job correctly in both
incremental and full checkpoint format?
I think adding this option is not something complicated, but we might meet the
risk to break the checkpoint compatibility. Last but not least, we do not
provide such option in Flink currently, user still could config the compression
type programmatically.
> Add compression type option in RocksDBConfigurableOptions
> ---------------------------------------------------------
>
> Key: FLINK-20684
> URL: https://issues.apache.org/jira/browse/FLINK-20684
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
> Affects Versions: 1.11.3
> Reporter: Jiayi Liao
> Priority: Major
>
> Currently RocksDB uses snappy compression by default, but this could be a
> huge cost on CPU time, and for most cases we may prefer using more disk space
> than CPU.
> I think Flink should provide an compression type option for users to choose
> in different use cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
