szaszm commented on code in PR #1480:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1480#discussion_r1100041964
##########
extensions/rocksdb-repos/FlowFileRepository.h:
##########
@@ -131,11 +131,35 @@ class FlowFileRepository : public ThreadedRepository,
public SwapManager {
// To avoid DB write issues during heavy load it's recommended to have
high number of buffer.
// Rocksdb's stall feature can also trigger in case the number of buffers
is >= 3.
// The more buffers we have the more memory rocksdb can utilize without
significant memory consumption under low load.
- auto cf_options = [] (rocksdb::ColumnFamilyOptions& cf_opts) {
+ auto cf_options = [&configure] (rocksdb::ColumnFamilyOptions& cf_opts) {
cf_opts.OptimizeForPointLookup(4);
cf_opts.write_buffer_size = 8ULL << 20U;
cf_opts.max_write_buffer_number = 20;
cf_opts.min_write_buffer_number_to_merge = 1;
+ std::string value;
+ if (configure->get(Configure::nifi_flow_repository_rocksdb_compression,
value) && !value.empty()) {
+#ifdef WIN32
+ if (value == "xpress") {
+ cf_opts.compression = rocksdb::CompressionType::kXpressCompression;
+ } else {
+ throw Exception(REPOSITORY_EXCEPTION, "RocksDB compression type not
supported: " + value);
+ }
+#else
+ if (value == "zlib") {
+ cf_opts.compression = rocksdb::CompressionType::kZlibCompression;
+ } else if (value == "bzip2") {
+ cf_opts.compression = rocksdb::CompressionType::kBZip2Compression;
Review Comment:
I propose to add an option like "auto" or "yes", which chooses a good
default compression algorithm on all platforms. This way we can keep the config
platform-independent, and people don't need to become experts in compression
algorithms to make a good choice.
On Windows, it obviously has to be xpress, since there's nothing else. On
others, I'd go for zstd.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
