[jira] [Assigned] (ZOOKEEPER-3179) Add snapshot compression to reduce the disk IO
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yisong Yue reassigned ZOOKEEPER-3179: - Assignee: Yisong Yue > Add snapshot compression to reduce the disk IO > -- > > Key: ZOOKEEPER-3179 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3179 > Project: ZooKeeper > Issue Type: Improvement >Reporter: Fangmin Lv >Assignee: Yisong Yue >Priority: Major > Fix For: 3.6.0 > > > When the snapshot becomes larger, the periodically snapshot after certain > number of txns will be more expensive. Which will in turn affect the maximum > throughput we can support within SLA, because of the disk contention between > snapshot and txn when they're on the same drive. > > With compression like zstd/snappy/gzip, the actual snapshot size could be > much smaller, the compress ratio depends on the actual data. It might make > the recovery time (loading from disk) faster in some cases, but will take > longer sometimes because of the extra time used to compress/decompress. > > Based on the production traffic, the performance various with different > compress method as well, that's why we provided different implementations, we > can select different compress method for different use cases. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (ZOOKEEPER-3179) Add snapshot compression to reduce the disk IO
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fangmin Lv reassigned ZOOKEEPER-3179: - Assignee: (was: Suyog Mapara) > Add snapshot compression to reduce the disk IO > -- > > Key: ZOOKEEPER-3179 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3179 > Project: ZooKeeper > Issue Type: Improvement >Reporter: Fangmin Lv >Priority: Major > Fix For: 3.6.0 > > > When the snapshot becomes larger, the periodically snapshot after certain > number of txns will be more expensive. Which will in turn affect the maximum > throughput we can support within SLA, because of the disk contention between > snapshot and txn when they're on the same drive. > > With compression like zstd/snappy/gzip, the actual snapshot size could be > much smaller, the compress ratio depends on the actual data. It might make > the recovery time (loading from disk) faster in some cases, but will take > longer sometimes because of the extra time used to compress/decompress. > > Based on the production traffic, the performance various with different > compress method as well, that's why we provided different implementations, we > can select different compress method for different use cases. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
