http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_posts/2016-01-29-compaction_pri.markdown ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_posts/2016-01-29-compaction_pri.markdown b/thirdparty/rocksdb/docs/_posts/2016-01-29-compaction_pri.markdown new file mode 100644 index 0000000..ba9ee62 --- /dev/null +++ b/thirdparty/rocksdb/docs/_posts/2016-01-29-compaction_pri.markdown @@ -0,0 +1,51 @@ +--- +title: Option of Compaction Priority +layout: post +author: sdong +category: blog +redirect_from: + - /blog/2921/compaction_pri/ +--- + +The most popular compaction style of RocksDB is level-based compaction, which is an improved version of LevelDB's compaction algorithm. Page 9- 16 of this [slides](https://github.com/facebook/rocksdb/blob/gh-pages/talks/2015-09-29-HPTS-Siying-RocksDB.pdf) gives an illustrated introduction of this compaction style. The basic idea that: data is organized by multiple levels with exponential increasing target size. Except a special level 0, every level is key-range partitioned into many files. When size of a level exceeds its target size, we pick one or more of its files, and merge the file into the next level. + +<!--truncate--> + +Which file to pick to compact is an interesting question. LevelDB only uses one thread for compaction and it always picks files in round robin manner. We implemented multi-thread compaction in RocksDB by picking multiple files from the same level and compact them in parallel. We had to move away from LevelDB's file picking approach. Recently, we created an option [options.compaction_pri](https://github.com/facebook/rocksdb/blob/d6c838f1e130d8860407bc771fa6d4ac238859ba/include/rocksdb/options.h#L83-L93), which indicated three different algorithms to pick files to compact. + +Why do we need to multiple algorithms to choose from? Because there are different factors to consider when picking the files, and we now don't yet know how to balance them automatically, so we expose it to users to choose. Here are factors to consider: + +**Write amplification** + +When we estimate write amplification, we usually simplify the problem by assuming keys are uniformly distributed inside each level. In reality, it is not the case, even if user updates are uniformly distributed across the whole key range. For instance, when we compact one file of a level to the next level, it creates a hole. Over time, incoming compaction will fill data to the hole, but the density will still be lower for a while. Picking a file with keys least densely populated is more expensive to get the file to the next level, because there will be more overlapping files in the next level so we need to rewrite more data. For example, assume a file is 100MB, if an L2 file overlaps with 8 L3 files, we need to rewrite about 800MB of data to get the file to L3. If the file overlaps with 12 L3 files, we'll need to rewrite about 1200MB to get a file of the same size out of L2. It uses 50% more writes. (This analysis ignores the key density of the next level, because the range covers N times of files in that level so one hole only impacts write amplification by 1/N) + +If all the updates are uniformly distributed, LevelDB's approach optimizes write amplification, because a file being picked covers a range whose last compaction time to the next level is the oldest, so the range will accumulated keys from incoming compactions for the longest and the density is the highest. + +We created a compaction priority **kOldestSmallestSeqFirst** for the same effect. With this mode, we always pick the file covers the oldest updates in the level, which usually is contains the densest key range. If you have a use case where writes are uniformly distributed across the key space and you want to reduce write amplification, you should set options.compaction_pri=kOldestSmallestSeqFirst. + +**Optimize for small working set** + +We are assuming updates are uniformly distributed across the whole key space in previous analysis. However, in many use cases, there are subset of keys that are frequently updated while other key ranges are very cold. In this case, keeping hot key ranges from compacting to deeper levels will benefit write amplification, as well as space amplification. For example, if in a DB only key 150-160 are updated and other keys are seldom updated. If level 1 contains 20 keys, we want to keep 150-160 all stay in level 1. Because when next level 0 -> 1 compaction comes, it will simply overwrite existing keys so size level 1 doesn't increase, so no need to schedule further compaction for level 1->2. On the other hand, if we compact key 150-155 to level2, when a new Level 1->2 compaction comes, it increases the size of level 1, making size of level 1 exceed target size and more compactions will be needed, which generates more writes. + +The compaction priority **kOldestLargestSeqFirst** optimizes this use case. In this mode, we will pick a file whose latest update is the oldest. It means there is no incoming data for the range for the longest. Usually it is the coldest range. By compacting coldest range first, we leave the hot ranges in the level. If your use case is to overwrite existing keys in a small range, try options.compaction_pri=kOldestLargestSeqFirst**.** + +**Drop delete marker sooner** + +If one file contains a lot of delete markers, it may slow down iterating over this area, because we still need to iterate those deleted keys just to ignore them. Furthermore, the sooner we compact delete keys into the last level, the sooner the disk space is reclaimed, so it is good for space efficiency. + +Our default compaction priority **kByCompensatedSize** considers the case. If number of deletes in a file exceeds number of inserts, it is more likely to be picked for compaction. The more number of deletes exceed inserts, the more likely it is being compacted. The optimization is added to avoid the worst performance of space efficiency and query performance when a large percentage of the DB is deleted. + +**Efficiency of compaction filter** + +Usually people use [compaction filters](https://github.com/facebook/rocksdb/blob/v4.1/include/rocksdb/options.h#L201-L226) to clean up old data to free up space. Picking files to compact may impact space efficiency. We don't yet have a a compaction priority to optimize this case. In some of our use cases, we solved the problem in a different way: we have an external service checking modify time of all SST files. If any of the files is too old, we force the single file to compaction by calling DB::CompactFiles() using the single file. In this way, we can provide a time bound of data passing through compaction filters. + + +In all, there three choices of compaction priority modes optimizing different scenarios. if you have a new use case, we suggest you start with `options.compaction_pri=kOldestSmallestSeqFirst` (note it is not the default one for backward compatible reason). If you want to further optimize your use case, you can try other two use cases if your use cases apply. + +If you have good ideas about better compaction picker approach, you are welcome to implement and benchmark it. We'll be glad to review and merge your a pull requests. + +### Comments + +**[Mark Callaghan]([email protected])** + +Performance results for compaction_pri values and linkbench are explained at [http://smalldatum.blogspot.com/2016/02/compaction-priority-in-rocksdb.html](http://smalldatum.blogspot.com/2016/02/compaction-priority-in-rocksdb.html)
http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_posts/2016-02-24-rocksdb-4-2-release.markdown ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_posts/2016-02-24-rocksdb-4-2-release.markdown b/thirdparty/rocksdb/docs/_posts/2016-02-24-rocksdb-4-2-release.markdown new file mode 100644 index 0000000..409015c --- /dev/null +++ b/thirdparty/rocksdb/docs/_posts/2016-02-24-rocksdb-4-2-release.markdown @@ -0,0 +1,41 @@ +--- +title: RocksDB 4.2 Release! +layout: post +author: sdong +category: blog +redirect_from: + - /blog/3017/rocksdb-4-2-release/ +--- + +New RocksDB release - 4.2! + + +**New Features** + + 1. Introduce CreateLoggerFromOptions(), this function create a Logger for provided DBOptions. + + + 2. Add GetAggregatedIntProperty(), which returns the sum of the GetIntProperty of all the column families. + + + 3. Add MemoryUtil in rocksdb/utilities/memory.h. It currently offers a way to get the memory usage by type from a list rocksdb instances. + + +<!--truncate--> + + +**Public API changes** + + 1. CompactionFilter::Context includes information of Column Family ID + + + 2. The need-compaction hint given by TablePropertiesCollector::NeedCompact() will be persistent and recoverable after DB recovery. This introduces a breaking format change. If you use this experimental feature, including NewCompactOnDeletionCollectorFactory() in the new version, you may not be able to directly downgrade the DB back to version 4.0 or lower. + + + 3. TablePropertiesCollectorFactory::CreateTablePropertiesCollector() now takes an option Context, containing the information of column family ID for the file being written. + + + 4. Remove DefaultCompactionFilterFactory. + + +[https://github.com/facebook/rocksdb/releases/tag/v4.2](https://github.com/facebook/rocksdb/releases/tag/v4.2) http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_posts/2016-02-25-rocksdb-ama.markdown ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_posts/2016-02-25-rocksdb-ama.markdown b/thirdparty/rocksdb/docs/_posts/2016-02-25-rocksdb-ama.markdown new file mode 100644 index 0000000..2ba04f3 --- /dev/null +++ b/thirdparty/rocksdb/docs/_posts/2016-02-25-rocksdb-ama.markdown @@ -0,0 +1,20 @@ +--- +title: RocksDB AMA +layout: post +author: yhchiang +category: blog +redirect_from: + - /blog/3065/rocksdb-ama/ +--- + +RocksDB developers are doing a Reddit Ask-Me-Anything now at 10AM â 11AM PDT! We welcome you to stop by and ask any RocksDB related questions, including existing / upcoming features, tuning tips, or database design. + +Here are some enhancements that we'd like to focus on over the next six months: + +* 2-Phase Commit +* Lua support in some custom functions +* Backup and repair tools +* Direct I/O to bypass OS cache +* RocksDB Java API + +[https://www.reddit.com/r/IAmA/comments/47k1si/we_are_rocksdb_developers_ask_us_anything/](https://www.reddit.com/r/IAmA/comments/47k1si/we_are_rocksdb_developers_ask_us_anything/) http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_posts/2016-03-07-rocksdb-options-file.markdown ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_posts/2016-03-07-rocksdb-options-file.markdown b/thirdparty/rocksdb/docs/_posts/2016-03-07-rocksdb-options-file.markdown new file mode 100644 index 0000000..703449b --- /dev/null +++ b/thirdparty/rocksdb/docs/_posts/2016-03-07-rocksdb-options-file.markdown @@ -0,0 +1,24 @@ +--- +title: RocksDB Options File +layout: post +author: yhciang +category: blog +redirect_from: + - /blog/3089/rocksdb-options-file/ +--- + +In RocksDB 4.3, we added a new set of features that makes managing RocksDB options easier. Specifically: + + * **Persisting Options Automatically**: Each RocksDB database will now automatically persist its current set of options into an INI file on every successful call of DB::Open(), SetOptions(), and CreateColumnFamily() / DropColumnFamily(). + + + + * **Load Options from File**: We added [LoadLatestOptions() / LoadOptionsFromFile()](https://github.com/facebook/rocksdb/blob/4.3.fb/include/rocksdb/utilities/options_util.h#L48-L58) that enables developers to construct RocksDB options object from an options file. + + + + * **Sanity Check Options**: We added [CheckOptionsCompatibility](https://github.com/facebook/rocksdb/blob/4.3.fb/include/rocksdb/utilities/options_util.h#L64-L77) that performs compatibility check on two sets of RocksDB options. + +<!--truncate--> + +Want to know more about how to use this new features? Check out the [RocksDB Options File wiki page](https://github.com/facebook/rocksdb/wiki/RocksDB-Options-File) and start using this new feature today! http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_posts/2016-04-26-rocksdb-4-5-1-released.markdown ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_posts/2016-04-26-rocksdb-4-5-1-released.markdown b/thirdparty/rocksdb/docs/_posts/2016-04-26-rocksdb-4-5-1-released.markdown new file mode 100644 index 0000000..247768d --- /dev/null +++ b/thirdparty/rocksdb/docs/_posts/2016-04-26-rocksdb-4-5-1-released.markdown @@ -0,0 +1,60 @@ +--- +title: RocksDB 4.5.1 Released! +layout: post +author: sdong +category: blog +redirect_from: + - /blog/3179/rocksdb-4-5-1-released/ +--- + +## 4.5.1 (3/25/2016) + +### Bug Fixes + + * Â Fix failures caused by the destorying order of singleton objects. + +<br/> + +## 4.5.0 (2/5/2016) + +### Public API Changes + + * Add a new perf context level between kEnableCount and kEnableTime. Level 2 now does not include timers for mutexes. + * Statistics of mutex operation durations will not be measured by default. If you want to have them enabled, you need to set Statistics::stats_level_ to kAll. + * DBOptions::delete_scheduler and NewDeleteScheduler() are removed, please use DBOptions::sst_file_manager and NewSstFileManager() instead + +### New Features + * ldb tool now supports operations to non-default column families. + * Add kPersistedTier to ReadTier. This option allows Get and MultiGet to read only the persited data and skip mem-tables if writes were done with disableWAL = true. + * Add DBOptions::sst_file_manager. Use NewSstFileManager() in include/rocksdb/sst_file_manager.h to create a SstFileManager that can be used to track the total size of SST files and control the SST files deletion rate. + +<br/> + +<!--truncate--> + +## 4.4.0 (1/14/2016) + +### Public API Changes + + * Change names in CompactionPri and add a new one. + * Deprecate options.soft_rate_limit and add options.soft_pending_compaction_bytes_limit. + * If options.max_write_buffer_number > 3, writes will be slowed down when writing to the last write buffer to delay a full stop. + * Introduce CompactionJobInfo::compaction_reason, this field include the reason to trigger the compaction. + * After slow down is triggered, if estimated pending compaction bytes keep increasing, slowdown more. + * Increase default options.delayed_write_rate to 2MB/s. + * Added a new parameter --path to ldb tool. --path accepts the name of either MANIFEST, SST or a WAL file. Either --db or --path can be used when calling ldb. + +<br/> + +## 4.3.0 (12/8/2015) + +### New Features + + * CompactionFilter has new member function called IgnoreSnapshots which allows CompactionFilter to be called even if there are snapshots later than the key. + * RocksDB will now persist options under the same directory as the RocksDB database on successful DB::Open, CreateColumnFamily, DropColumnFamily, and SetOptions. + * Introduce LoadLatestOptions() in rocksdb/utilities/options_util.h. This function can construct the latest DBOptions / ColumnFamilyOptions used by the specified RocksDB intance. + * Introduce CheckOptionsCompatibility() in rocksdb/utilities/options_util.h. This function checks whether the input set of options is able to open the specified DB successfully. + +### Public API Changes + + * When options.db_write_buffer_size triggers, only the column family with the largest column family size will be flushed, not all the column families. http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_posts/2016-07-26-rocksdb-4-8-released.markdown ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_posts/2016-07-26-rocksdb-4-8-released.markdown b/thirdparty/rocksdb/docs/_posts/2016-07-26-rocksdb-4-8-released.markdown new file mode 100644 index 0000000..b42a66e --- /dev/null +++ b/thirdparty/rocksdb/docs/_posts/2016-07-26-rocksdb-4-8-released.markdown @@ -0,0 +1,48 @@ +--- +title: RocksDB 4.8 Released! +layout: post +author: yiwu +category: blog +redirect_from: + - /blog/3239/rocksdb-4-8-released/ +--- + +## 4.8.0 (5/2/2016) + +### [](https://github.com/facebook/rocksdb/blob/master/HISTORY.md#public-api-change-1)Public API Change + + * Allow preset compression dictionary for improved compression of block-based tables. This is supported for zlib, zstd, and lz4. The compression dictionary's size is configurable via CompressionOptions::max_dict_bytes. + * Delete deprecated classes for creating backups (BackupableDB) and restoring from backups (RestoreBackupableDB). Now, BackupEngine should be used for creating backups, and BackupEngineReadOnly should be used for restorations. For more details, see [https://github.com/facebook/rocksdb/wiki/How-to-backup-RocksDB%3F](https://github.com/facebook/rocksdb/wiki/How-to-backup-RocksDB%3F) + * Expose estimate of per-level compression ratio via DB property: "rocksdb.compression-ratio-at-levelN". + * Added EventListener::OnTableFileCreationStarted. EventListener::OnTableFileCreated will be called on failure case. User can check creation status via TableFileCreationInfo::status. + +### [](https://github.com/facebook/rocksdb/blob/master/HISTORY.md#new-features-2)New Features + + * Add ReadOptions::readahead_size. If non-zero, NewIterator will create a new table reader which performs reads of the given size. + +<br/> + +<!--truncate--> + +## [](https://github.com/facebook/rocksdb/blob/master/HISTORY.md#470-482016)4.7.0 (4/8/2016) + +### [](https://github.com/facebook/rocksdb/blob/master/HISTORY.md#public-api-change-2)Public API Change + + * rename options compaction_measure_io_stats to report_bg_io_stats and include flush too. + * Change some default options. Now default options will optimize for server-workloads. Also enable slowdown and full stop triggers for pending compaction bytes. These changes may cause sub-optimal performance or significant increase of resource usage. To avoid these risks, users can open existing RocksDB with options extracted from RocksDB option files. See [https://github.com/facebook/rocksdb/wiki/RocksDB-Options-File](https://github.com/facebook/rocksdb/wiki/RocksDB-Options-File) for how to use RocksDB option files. Or you can call Options.OldDefaults() to recover old defaults. DEFAULT_OPTIONS_HISTORY.md will track change history of default options. + +<br/> + +## [](https://github.com/facebook/rocksdb/blob/master/HISTORY.md#460-3102016)4.6.0 (3/10/2016) + +### [](https://github.com/facebook/rocksdb/blob/master/HISTORY.md#public-api-changes-1)Public API Changes + + * Change default of BlockBasedTableOptions.format_version to 2. It means default DB created by 4.6 or up cannot be opened by RocksDB version 3.9 or earlier + * Added strict_capacity_limit option to NewLRUCache. If the flag is set to true, insert to cache will fail if no enough capacity can be free. Signature of Cache::Insert() is updated accordingly. + * Tickers [NUMBER_DB_NEXT, NUMBER_DB_PREV, NUMBER_DB_NEXT_FOUND, NUMBER_DB_PREV_FOUND, ITER_BYTES_READ] are not updated immediately. The are updated when the Iterator is deleted. + * Add monotonically increasing counter (DB property "rocksdb.current-super-version-number") that increments upon any change to the LSM tree. + +### [](https://github.com/facebook/rocksdb/blob/master/HISTORY.md#new-features-3)New Features + + * Add CompactionPri::kMinOverlappingRatio, a compaction picking mode friendly to write amplification. + * Deprecate Iterator::IsKeyPinned() and replace it with Iterator::GetProperty() with prop_name="rocksdb.iterator.is.key.pinned" http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_posts/2016-09-28-rocksdb-4-11-2-released.markdown ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_posts/2016-09-28-rocksdb-4-11-2-released.markdown b/thirdparty/rocksdb/docs/_posts/2016-09-28-rocksdb-4-11-2-released.markdown new file mode 100644 index 0000000..87c20eb --- /dev/null +++ b/thirdparty/rocksdb/docs/_posts/2016-09-28-rocksdb-4-11-2-released.markdown @@ -0,0 +1,49 @@ +--- +title: RocksDB 4.11.2 Released! +layout: post +author: sdong +category: blog +--- +We abandoned release candidates 4.10.x and directly go to 4.11.2 from 4.9, to make sure the latest release is stable. In 4.11.2, we fixed several data corruption related bugs introduced in 4.9.0. + +## 4.11.2 (9/15/2016) + +### Bug fixes + + * Segfault when failing to open an SST file for read-ahead iterators. + * WAL without data for all CFs is not deleted after recovery. + +<!--truncate--> + +## 4.11.1 (8/30/2016) + +### Bug Fixes + + * Mitigate the regression bug of deadlock condition during recovery when options.max_successive_merges hits. + * Fix data race condition related to hash index in block based table when putting indexes in the block cache. + +## 4.11.0 (8/1/2016) + +### Public API Change + + * options.memtable_prefix_bloom_huge_page_tlb_size => memtable_huge_page_size. When it is set, RocksDB will try to allocate memory from huge page for memtable too, rather than just memtable bloom filter. + +### New Features + + * A tool to migrate DB after options change. See include/rocksdb/utilities/option_change_migration.h. + * Add ReadOptions.background_purge_on_iterator_cleanup. If true, we avoid file deletion when destorying iterators. + +## 4.10.0 (7/5/2016) + +### Public API Change + + * options.memtable_prefix_bloom_bits changes to options.memtable_prefix_bloom_bits_ratio and deprecate options.memtable_prefix_bloom_probes + * enum type CompressionType and PerfLevel changes from char to unsigned char. Value of all PerfLevel shift by one. + * Deprecate options.filter_deletes. + +### New Features + + * Add avoid_flush_during_recovery option. + * Add a read option background_purge_on_iterator_cleanup to avoid deleting files in foreground when destroying iterators. Instead, a job is scheduled in high priority queue and would be executed in a separate background thread. + * RepairDB support for column families. RepairDB now associates data with non-default column families using information embedded in the SST/WAL files (4.7 or later). For data written by 4.6 or earlier, RepairDB associates it with the default column family. + * Add options.write_buffer_manager which allows users to control total memtable sizes across multiple DB instances. http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_posts/2017-01-06-rocksdb-5-0-1-released.markdown ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_posts/2017-01-06-rocksdb-5-0-1-released.markdown b/thirdparty/rocksdb/docs/_posts/2017-01-06-rocksdb-5-0-1-released.markdown new file mode 100644 index 0000000..fb04130 --- /dev/null +++ b/thirdparty/rocksdb/docs/_posts/2017-01-06-rocksdb-5-0-1-released.markdown @@ -0,0 +1,26 @@ +--- +title: RocksDB 5.0.1 Released! +layout: post +author: yiwu +category: blog +--- + +### Public API Change + + * Options::max_bytes_for_level_multiplier is now a double along with all getters and setters. + * Support dynamically change `delayed_write_rate` and `max_total_wal_size` options via SetDBOptions(). + * Introduce DB::DeleteRange for optimized deletion of large ranges of contiguous keys. + * Support dynamically change `delayed_write_rate` option via SetDBOptions(). + * Options::allow_concurrent_memtable_write and Options::enable_write_thread_adaptive_yield are now true by default. + * Remove Tickers::SEQUENCE_NUMBER to avoid confusion if statistics object is shared among RocksDB instance. Alternatively DB::GetLatestSequenceNumber() can be used to get the same value. + * Options.level0_stop_writes_trigger default value changes from 24 to 32. + * New compaction filter API: CompactionFilter::FilterV2(). Allows to drop ranges of keys. + * Removed flashcache support. + * DB::AddFile() is deprecated and is replaced with DB::IngestExternalFile(). DB::IngestExternalFile() remove all the restrictions that existed for DB::AddFile. + +### New Features + + * Add avoid_flush_during_shutdown option, which speeds up DB shutdown by not flushing unpersisted data (i.e. with disableWAL = true). Unpersisted data will be lost. The options is dynamically changeable via SetDBOptions(). + * Add memtable_insert_with_hint_prefix_extractor option. The option is mean to reduce CPU usage for inserting keys into memtable, if keys can be group by prefix and insert for each prefix are sequential or almost sequential. See include/rocksdb/options.h for more details. + * Add LuaCompactionFilter in utilities. This allows developers to write compaction filters in Lua. To use this feature, LUA_PATH needs to be set to the root directory of Lua. + * No longer populate "LATEST_BACKUP" file in backup directory, which formerly contained the number of the latest backup. The latest backup can be determined by finding the highest numbered file in the "meta/" subdirectory. http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_posts/2017-02-07-rocksdb-5-1-2-released.markdown ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_posts/2017-02-07-rocksdb-5-1-2-released.markdown b/thirdparty/rocksdb/docs/_posts/2017-02-07-rocksdb-5-1-2-released.markdown new file mode 100644 index 0000000..35bafb2 --- /dev/null +++ b/thirdparty/rocksdb/docs/_posts/2017-02-07-rocksdb-5-1-2-released.markdown @@ -0,0 +1,15 @@ +--- +title: RocksDB 5.1.2 Released! +layout: post +author: maysamyabandeh +category: blog +--- + +### Public API Change +* Support dynamically change `delete_obsolete_files_period_micros` option via SetDBOptions(). +* Added EventListener::OnExternalFileIngested which will be called when IngestExternalFile() add a file successfully. +* BackupEngine::Open and BackupEngineReadOnly::Open now always return error statuses matching those of the backup Env. + +### Bug Fixes +* Fix the bug that if 2PC is enabled, checkpoints may loss some recent transactions. +* When file copying is needed when creating checkpoints or bulk loading files, fsync the file after the file copying. http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_posts/2017-02-17-bulkoad-ingest-sst-file.markdown ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_posts/2017-02-17-bulkoad-ingest-sst-file.markdown b/thirdparty/rocksdb/docs/_posts/2017-02-17-bulkoad-ingest-sst-file.markdown new file mode 100644 index 0000000..9a43a84 --- /dev/null +++ b/thirdparty/rocksdb/docs/_posts/2017-02-17-bulkoad-ingest-sst-file.markdown @@ -0,0 +1,50 @@ +--- +title: Bulkloading by ingesting external SST files +layout: post +author: IslamAbdelRahman +category: blog +--- + +## Introduction + +One of the basic operations of RocksDB is writing to RocksDB, Writes happen when user call (DB::Put, DB::Write, DB::Delete ... ), but what happens when you write to RocksDB ? .. this is a brief description of what happens. +- User insert a new key/value by calling DB::Put() (or DB::Write()) +- We create a new entry for the new key/value in our in-memory structure (memtable / SkipList by default) and we assign it a new sequence number. +- When the memtable exceeds a specific size (64 MB for example), we convert this memtable to a SST file, and put this file in level 0 of our LSM-Tree +- Later, compaction will kick in and move data from level 0 to level 1, and then from level 1 to level 2 .. and so on + +But what if we can skip these steps and add data to the lowest possible level directly ? This is what bulk-loading does + +## Bulkloading + +- Write all of our keys and values into SST file outside of the DB +- Add the SST file into the LSM directly + +This is bulk-loading, and in specific use-cases it allow users to achieve faster data loading and better write-amplification. + +and doing it is as simple as +```cpp +Options options; +SstFileWriter sst_file_writer(EnvOptions(), options, options.comparator); +Status s = sst_file_writer.Open(file_path); +assert(s.ok()); + +// Insert rows into the SST file, note that inserted keys must be +// strictly increasing (based on options.comparator) +for (...) { + s = sst_file_writer.Add(key, value); + assert(s.ok()); +} + +// Ingest the external SST file into the DB +s = db_->IngestExternalFile({"/home/usr/file1.sst"}, IngestExternalFileOptions()); +assert(s.ok()); +``` + +You can find more details about how to generate SST files and ingesting them into RocksDB in this [wiki page](https://github.com/facebook/rocksdb/wiki/Creating-and-Ingesting-SST-files) + +## Use cases +There are multiple use cases where bulkloading could be useful, for example +- Generating SST files in offline jobs in Hadoop, then downloading and ingesting the SST files into RocksDB +- Migrating shards between machines by dumping key-range in SST File and loading the file in a different machine +- Migrating from a different storage (InnoDB to RocksDB migration in MyRocks) http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_posts/2017-03-02-rocksdb-5-2-1-released.markdown ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_posts/2017-03-02-rocksdb-5-2-1-released.markdown b/thirdparty/rocksdb/docs/_posts/2017-03-02-rocksdb-5-2-1-released.markdown new file mode 100644 index 0000000..c6ce27d --- /dev/null +++ b/thirdparty/rocksdb/docs/_posts/2017-03-02-rocksdb-5-2-1-released.markdown @@ -0,0 +1,22 @@ +--- +title: RocksDB 5.2.1 Released! +layout: post +author: sdong +category: blog +--- + +### Public API Change +* NewLRUCache() will determine number of shard bits automatically based on capacity, if the user doesn't pass one. This also impacts the default block cache when the user doesn't explict provide one. +* Change the default of delayed slowdown value to 16MB/s and further increase the L0 stop condition to 36 files. + +### New Features +* Added new overloaded function GetApproximateSizes that allows to specify if memtable stats should be computed only without computing SST files' stats approximations. +* Added new function GetApproximateMemTableStats that approximates both number of records and size of memtables. +* (Experimental) Two-level indexing that partition the index and creates a 2nd level index on the partitions. The feature can be enabled by setting kTwoLevelIndexSearch as IndexType and configuring index_per_partition. + +### Bug Fixes +* RangeSync() should work if ROCKSDB_FALLOCATE_PRESENT is not set +* Fix wrong results in a data race case in Get() +* Some fixes related to 2PC. +* Fix several bugs in Direct I/O supports. +* Fix a regression bug which can cause Seek() to miss some keys if the return key has been updated many times after the snapshot which is used by the iterator. http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_posts/2017-05-12-partitioned-index-filter.markdown ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_posts/2017-05-12-partitioned-index-filter.markdown b/thirdparty/rocksdb/docs/_posts/2017-05-12-partitioned-index-filter.markdown new file mode 100644 index 0000000..fb4f62c --- /dev/null +++ b/thirdparty/rocksdb/docs/_posts/2017-05-12-partitioned-index-filter.markdown @@ -0,0 +1,34 @@ +--- +title: Partitioned Index/Filters +layout: post +author: maysamyabandeh +category: blog +--- + +As DB/mem ratio gets larger, the memory footprint of filter/index blocks becomes non-trivial. Although `cache_index_and_filter_blocks` allows storing only a subset of them in block cache, their relatively large size negatively affects the performance by i) occupying the block cache space that could otherwise be used for caching data, ii) increasing the load on the disk storage by loading them into the cache after a miss. Here we illustrate these problems in more detail and explain how partitioning index/filters alleviates the overhead. + +### How large are the index/filter blocks? + +RocksDB has by default one index/filter block per SST file. The size of the index/filter varies based on the configuration but for a SST of size 256MB the index/filter block of size 0.5/5MB is typical, which is much larger than the typical data block size of 4-32KB. That is fine when all index/filters fit perfectly into memory and hence are read once per SST lifetime, not so much when they compete with data blocks for the block cache space and are also likely to be re-read many times from the disk. + +### What is the big deal with large index/filter blocks? + +When index/filter blocks are stored in block cache they are effectively competing with data blocks (as well as with each other) on this scarce resource. A filter of size 5MB is occupying the space that could otherwise be used to cache 1000s of data blocks (of size 4KB). This would result in more cache misses for data blocks. The large index/filters also kick each other out of the block cache more often and exacerbate their own cache miss rate too. This is while only a small part of the index/filter block might have been actually used during its lifetime in the cache. + +After the cache miss of an index/filter, it has to be reloaded from the disk, and its large size is not helping in reducing the IO cost. While a simple point lookup might need at most a couple of data block reads (of size 4KB) one from each layer of LSM, it might end up also loading multiple megabytes of index/filter blocks. If that happens often then the disk is spending more time serving index/filters rather than the actual data blocks. + +## What is partitioned index/filters? + +With partitioning, the index/filter of a SST file is partitioned into smaller blocks with an additional top-level index on them. When reading an index/filter, only top-level index is loaded into memory. The partitioned index/filter then uses the top-level index to load on demand into the block cache the partitions that are required to perform the index/filter query. The top-level index, which has much smaller memory footprint, can be stored in heap or block cache depending on the `cache_index_and_filter_blocks` setting. + +### Success stories + +#### HDD, 100TB DB + +In this example we have a DB of size 86G on HDD and emulate the small memory that is present to a node with 100TB of data by using direct IO (skipping OS file cache) and a very small block cache of size 60MB. Partitioning improves throughput by 11x from 5 op/s to 55 op/s. + +#### SSD, Linkbench + +In this example we have a DB of size 300G on SSD and emulate the small memory that would be available in presence of other DBs on the same node by by using direct IO (skipping OS file cache) and block cache of size 6G and 2G. Without partitioning the linkbench throughput drops from 38k tps to 23k when reducing block cache size from 6G to 2G. With partitioning the throughput drops from 38k to only 30k. + +Learn more (here)[https://github.com/facebook/rocksdb/wiki/Partitioned-Index-Filters]. http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_posts/2017-05-14-core-local-stats.markdown ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_posts/2017-05-14-core-local-stats.markdown b/thirdparty/rocksdb/docs/_posts/2017-05-14-core-local-stats.markdown new file mode 100644 index 0000000..a806541 --- /dev/null +++ b/thirdparty/rocksdb/docs/_posts/2017-05-14-core-local-stats.markdown @@ -0,0 +1,106 @@ +--- +title: Core-local Statistics +layout: post +author: ajkr +category: blog +--- + +## Origins: Global Atomics + +Until RocksDB 4.12, ticker/histogram statistics were implemented with std::atomic values shared across the entire program. A ticker consists of a single atomic, while a histogram consists of several atomics to represent things like min/max/per-bucket counters. These statistics could be updated by all user/background threads. + +For concurrent/high-throughput workloads, cache line bouncing of atomics caused high CPU utilization. For example, we have tickers that count block cache hits and misses. Almost every user read increments these tickers a few times. Many concurrent user reads would cause the cache lines containing these atomics to bounce between cores. + +### Performance + +Here are perf results for 32 reader threads where most reads (99%+) are served by uncompressed block cache. Such a scenario stresses the statistics code heavily. + +Benchmark command: `TEST_TMPDIR=/dev/shm/ perf record -g ./db_bench -statistics -use_existing_db=true -benchmarks=readrandom -threads=32 -cache_size=1048576000 -num=1000000 -reads=1000000 && perf report -g --children` + +Perf snippet for "cycles" event: + +``` + Children Self Command Shared Object Symbol ++ 30.33% 30.17% db_bench db_bench [.] rocksdb::StatisticsImpl::recordTick ++ 3.65% 0.98% db_bench db_bench [.] rocksdb::StatisticsImpl::measureTime +``` + +Perf snippet for "cache-misses" event: + +``` + Children Self Command Shared Object Symbol ++ 19.54% 19.50% db_bench db_bench [.] rocksdb::StatisticsImpl::recordTick ++ 3.44% 0.57% db_bench db_bench [.] rocksdb::StatisticsImpl::measureTime +``` + +The high CPU overhead for updating tickers and histograms corresponds well to the high cache misses. + +## Thread-locals: Faster Updates + +Since RocksDB 4.12, ticker/histogram statistics use thread-local storage. Each thread has a local set of atomic values that no other thread can update. This prevents the cache line bouncing problem described above. Even though updates to a given value are always made by the same thread, atomics are still useful to synchronize with aggregations for querying statistics. + +Implementing this approach involved a couple challenges. First, each query for a statistic's global value must aggregate all threads' local values. This adds some overhead, which may pass unnoticed if statistics are queried infrequently. Second, exited threads' local values are still needed to provide accurate statistics. We handle this by merging a thread's local values into process-wide variables upon thread exit. + +### Performance + +Update benchmark setup is same as before. CPU overhead improved 7.8x compared to global atomics, corresponding to a 17.8x reduction in cache-misses overhead. + +Perf snippet for "cycles" event: + +``` + Children Self Command Shared Object Symbol ++ 2.96% 0.87% db_bench db_bench [.] rocksdb::StatisticsImpl::recordTick ++ 1.37% 0.10% db_bench db_bench [.] rocksdb::StatisticsImpl::measureTime +``` + +Perf snippet for "cache-misses" event: + +``` + Children Self Command Shared Object Symbol ++ 1.21% 0.65% db_bench db_bench [.] rocksdb::StatisticsImpl::recordTick + 0.08% 0.00% db_bench db_bench [.] rocksdb::StatisticsImpl::measureTime +``` + +To measure statistics query latency, we ran sysbench with 4K OLTP clients concurrently with one client that queries statistics repeatedly. Times shown are in milliseconds. + +``` + min: 18.45 + avg: 27.91 + max: 231.65 + 95th percentile: 55.82 +``` + +## Core-locals: Faster Querying + +The thread-local approach is working well for applications calling RocksDB from only a few threads, or polling statistics infrequently. Eventually, though, we found use cases where those assumptions do not hold. For example, one application has per-connection threads and typically runs into performance issues when connection count grows very high. For debugging such issues, they want high-frequency statistics polling to correlate issues in their application with changes in RocksDB's state. + +Once [PR #2258](https://github.com/facebook/rocksdb/pull/2258) lands, ticker/histogram statistics will be local to each CPU core. Similarly to thread-local, each core updates only its local values, thus avoiding cache line bouncing. Local values are still atomics to make aggregation possible. With this change, query work depends only on number of cores, not the number of threads. So, applications with many more threads than cores can no longer impact statistics query latency. + +### Performance + +Update benchmark setup is same as before. CPU overhead worsened ~23% compared to thread-local, while cache performance was unchanged. + +Perf snippet for "cycles" event: + +``` + Children Self Command Shared Object Symbol ++ 2.96% 0.87% db_bench db_bench [.] rocksdb::StatisticsImpl::recordTick ++ 1.37% 0.10% db_bench db_bench [.] rocksdb::StatisticsImpl::measureTime +``` + +Perf snippet for "cache-misses" event: + +``` + Children Self Command Shared Object Symbol ++ 1.21% 0.65% db_bench db_bench [.] rocksdb::StatisticsImpl::recordTick + 0.08% 0.00% db_bench db_bench [.] rocksdb::StatisticsImpl::measureTime +``` + +Query latency is measured same as before with times in milliseconds. Average latency improved by 6.3x compared to thread-local. + +``` + min: 2.47 + avg: 4.45 + max: 91.13 + 95th percentile: 7.56 +``` http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_posts/2017-05-26-rocksdb-5-4-5-released.markdown ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_posts/2017-05-26-rocksdb-5-4-5-released.markdown b/thirdparty/rocksdb/docs/_posts/2017-05-26-rocksdb-5-4-5-released.markdown new file mode 100644 index 0000000..561dab4 --- /dev/null +++ b/thirdparty/rocksdb/docs/_posts/2017-05-26-rocksdb-5-4-5-released.markdown @@ -0,0 +1,39 @@ +--- +title: RocksDB 5.4.5 Released! +layout: post +author: sagar0 +category: blog +--- + +### Public API Change +* Support dynamically changing `stats_dump_period_sec` option via SetDBOptions(). +* Added ReadOptions::max_skippable_internal_keys to set a threshold to fail a request as incomplete when too many keys are being skipped while using iterators. +* DB::Get in place of std::string accepts PinnableSlice, which avoids the extra memcpy of value to std::string in most of cases. + * PinnableSlice releases the pinned resources that contain the value when it is destructed or when ::Reset() is called on it. + * The old API that accepts std::string, although discouraged, is still supported. +* Replace Options::use_direct_writes with Options::use_direct_io_for_flush_and_compaction. See Direct IO wiki for details. + +### New Features +* Memtable flush can be avoided during checkpoint creation if total log file size is smaller than a threshold specified by the user. +* Introduce level-based L0->L0 compactions to reduce file count, so write delays are incurred less often. +* (Experimental) Partitioning filters which creates an index on the partitions. The feature can be enabled by setting partition_filters when using kFullFilter. Currently the feature also requires two-level indexing to be enabled. Number of partitions is the same as the number of partitions for indexes, which is controlled by metadata_block_size. +* DB::ResetStats() to reset internal stats. +* Added CompactionEventListener and EventListener::OnFlushBegin interfaces. +* Added DB::CreateColumnFamilie() and DB::DropColumnFamilies() to bulk create/drop column families. +* Facility for cross-building RocksJava using Docker. + +### Bug Fixes +* Fix WriteBatchWithIndex address use after scope error. +* Fix WritableFile buffer size in direct IO. +* Add prefetch to PosixRandomAccessFile in buffered io. +* Fix PinnableSlice access invalid address when row cache is enabled. +* Fix huge fallocate calls fail and make XFS unhappy. +* Fix memory alignment with logical sector size. +* Fix alignment in ReadaheadRandomAccessFile. +* Fix bias with read amplification stats (READ_AMP_ESTIMATE_USEFUL_BYTES and READ_AMP_TOTAL_READ_BYTES). +* Fix a manual / auto compaction data race. +* Fix CentOS 5 cross-building of RocksJava. +* Build and link with ZStd when creating the static RocksJava build. +* Fix snprintf's usage to be cross-platform. +* Fix build errors with blob DB. +* Fix readamp test type inconsistency. http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_posts/2017-06-26-17-level-based-changes.markdown ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_posts/2017-06-26-17-level-based-changes.markdown b/thirdparty/rocksdb/docs/_posts/2017-06-26-17-level-based-changes.markdown new file mode 100644 index 0000000..9e838eb --- /dev/null +++ b/thirdparty/rocksdb/docs/_posts/2017-06-26-17-level-based-changes.markdown @@ -0,0 +1,60 @@ +--- +title: Level-based Compaction Changes +layout: post +author: ajkr +category: blog +--- + +### Introduction + +RocksDB provides an option to limit the number of L0 files, which bounds read-amplification. Since L0 files (unlike files at lower levels) can span the entire key-range, a key might be in any file, thus reads need to check them one-by-one. Users often wish to configure a low limit to improve their read latency. + +Although, the mechanism with which we enforce L0's file count limit may be unappealing. When the limit is reached, RocksDB intentionally delays user writes. This slows down accumulation of files in L0, and frees up resources for compacting files down to lower levels. But adding delays will significantly increase user-visible write latency jitter. + +Also, due to how L0 files can span the entire key-range, compaction parallelization is limited. Files at L0 or L1 may be locked due to involvement in pending L0->L1 or L1->L2 compactions. We can only schedule a parallel L0->L1 compaction if it does not require any of the locked files, which is typically not the case. + +To handle these constraints better, we added a new type of compaction, L0->L0. It quickly reduces file count in L0 and can be scheduled even when L1 files are locked, unlike L0->L1. We also changed the L0->L1 picking algorithm to increase opportunities for parallelism. + +### Old L0->L1 Picking Logic + +Previously, our logic for picking which L0 file to compact was the same as every other level: pick the largest file in the level. One special property of L0->L1 compaction is that files can overlap in the input level, so those overlapping files must be pulled in as well. For example, a compaction may look like this: + + + +This compaction pulls in every L0 and L1 file. This happens regardless of which L0 file is initially chosen as each file overlaps with every other file. + +Users may insert their data less uniformly in the key-range. For example, a database may look like this during L0->L1 compaction: + + + +Let's say the third file from the top is the largest, and let's say the top two files are created after the compaction started. When the compaction is picked, the fourth L0 file and six rightmost L1 files are pulled in due to overlap. Notice this leaves the database in a state where we might not be able to schedule parallel compactions. For example, if the sixth file from the top is the next largest, we can't compact it because it overlaps with the top two files, which overlap with the locked L0 files. + +We can now see the high-level problems with this approach more clearly. First, locked files in L0 or L1 prevent us from parallelizing compactions. When locked files block L0->L1 compaction, there is nothing we can do to eliminate L0 files. Second, L0->L1 compactions are relatively slow. As we saw, when keys are uniformly distributed, L0->L1 compacts two entire levels. While this is happening, new files are being flushed to L0, advancing towards the file count limit. + +### New L0->L0 Algorithm + +We introduced compaction within L0 to improve both parallelization and speed of reducing L0 file count. An L0->L0 compaction may look like this: + + + +Say the L1->L2 compaction started first. Now L0->L1 is prevented by the locked L1 file. In this case, we compact files within L0. This allows us to start the work for eliminating L0 files earlier. It also lets us do less work since we don't pull in any L1 files, whereas L0->L1 compaction would've pulled in all of them. This lets us quickly reduce L0 file count to keep read-amp low while sustaining large bursts of writes (i.e., fast accumulation of L0 files). + +The tradeoff is this increases total compaction work, as we're now compacting files without contributing towards our eventual goal of moving them towards lower levels. Our benchmarks, though, consistently show less compaction stalls and improved write throughput. One justification is that L0 file data is highly likely in page cache and/or block cache due to it being recently written and frequently accessed. So, this type of compaction is relatively cheap compared to compactions at lower levels. + +This feature is available since RocksDB 5.4. + +### New L0->L1 Picking Logic + +Recall how the old L0->L1 picking algorithm chose the largest L0 file for compaction. This didn't fit well with L0->L0 compaction, which operates on a span of files. That span begins at the newest L0 file, and expands towards older files as long as they're not being compacted. Since the largest file may be anywhere, the old L0->L1 picking logic could arbitrarily prevent us from getting a long span of files. See the second illustration in this post for a scenario where this would happen. + +So, we changed the L0->L1 picking algorithm to start from the oldest file and expand towards newer files as long as they're not being compacted. For example: + + + +Now, there can never be L0 files unreachable for L0->L0 due to L0->L1 selecting files in the middle. When longer spans of files are available for L0->L0, we perform less compaction work per deleted L0 file, thus improving efficiency. + +This feature will be available in RocksDB 5.7. + +### Performance Changes + +Mark Callaghan did the most extensive benchmarking of this feature's impact on MyRocks. See his results [here](http://smalldatum.blogspot.com/2017/05/innodb-myrocks-and-tokudb-on-insert.html). Note the primary change between his March 17 and April 14 builds is the latter performs L0->L0 compaction. http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_posts/2017-06-29-rocksdb-5-5-1-released.markdown ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_posts/2017-06-29-rocksdb-5-5-1-released.markdown b/thirdparty/rocksdb/docs/_posts/2017-06-29-rocksdb-5-5-1-released.markdown new file mode 100644 index 0000000..d785608 --- /dev/null +++ b/thirdparty/rocksdb/docs/_posts/2017-06-29-rocksdb-5-5-1-released.markdown @@ -0,0 +1,22 @@ +--- +title: RocksDB 5.5.1 Released! +layout: post +author: lightmark +category: blog +--- + +### New Features +* FIFO compaction to support Intra L0 compaction too with CompactionOptionsFIFO.allow_compaction=true. +* Statistics::Reset() to reset user stats. +* ldb add option --try_load_options, which will open DB with its own option file. +* Introduce WriteBatch::PopSavePoint to pop the most recent save point explicitly. +* Support dynamically change `max_open_files` option via SetDBOptions() +* Added DB::CreateColumnFamilie() and DB::DropColumnFamilies() to bulk create/drop column families. +* Add debugging function `GetAllKeyVersions` to see internal versions of a range of keys. +* Support file ingestion with universal compaction style +* Support file ingestion behind with option `allow_ingest_behind` +* New option enable_pipelined_write which may improve write throughput in case writing from multiple threads and WAL enabled. + +### Bug Fixes +* Fix the bug that Direct I/O uses direct reads for non-SST file +* Fix the bug that flush doesn't respond to fsync result http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_posts/2017-07-25-rocksdb-5-6-1-released.markdown ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_posts/2017-07-25-rocksdb-5-6-1-released.markdown b/thirdparty/rocksdb/docs/_posts/2017-07-25-rocksdb-5-6-1-released.markdown new file mode 100644 index 0000000..3b54ffd --- /dev/null +++ b/thirdparty/rocksdb/docs/_posts/2017-07-25-rocksdb-5-6-1-released.markdown @@ -0,0 +1,22 @@ +--- +title: RocksDB 5.6.1 Released! +layout: post +author: yiwu +category: blog +--- + +### Public API Change +* Scheduling flushes and compactions in the same thread pool is no longer supported by setting `max_background_flushes=0`. Instead, users can achieve this by configuring their high-pri thread pool to have zero threads. See https://github.com/facebook/rocksdb/wiki/Thread-Pool for more details. +* Replace `Options::max_background_flushes`, `Options::max_background_compactions`, and `Options::base_background_compactions` all with `Options::max_background_jobs`, which automatically decides how many threads to allocate towards flush/compaction. +* options.delayed_write_rate by default take the value of options.rate_limiter rate. +* Replace global variable `IOStatsContext iostats_context` with `IOStatsContext* get_iostats_context()`; replace global variable `PerfContext perf_context` with `PerfContext* get_perf_context()`. + +### New Features +* Change ticker/histogram statistics implementations to use core-local storage. This improves aggregation speed compared to our previous thread-local approach, particularly for applications with many threads. See http://rocksdb.org/blog/2017/05/14/core-local-stats.html for more details. +* Users can pass a cache object to write buffer manager, so that they can cap memory usage for memtable and block cache using one single limit. +* Flush will be triggered when 7/8 of the limit introduced by write_buffer_manager or db_write_buffer_size is triggered, so that the hard threshold is hard to hit. See https://github.com/facebook/rocksdb/wiki/Write-Buffer-Manager for more details. +* Introduce WriteOptions.low_pri. If it is true, low priority writes will be throttled if the compaction is behind. See https://github.com/facebook/rocksdb/wiki/Low-Priority-Write for more details. +* `DB::IngestExternalFile()` now supports ingesting files into a database containing range deletions. + +### Bug Fixes +* Shouldn't ignore return value of fsync() in flush. http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_sass/_base.scss ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_sass/_base.scss b/thirdparty/rocksdb/docs/_sass/_base.scss new file mode 100644 index 0000000..6d26d9f --- /dev/null +++ b/thirdparty/rocksdb/docs/_sass/_base.scss @@ -0,0 +1,492 @@ +body { + background: $secondary-bg; + color: $text; + font: normal #{$base-font-size}/#{$base-line-height} $base-font-family; + height: 100vh; + text-align: left; + text-rendering: optimizeLegibility; +} + +img { + max-width: 100%; +} + +article { + p { + img { + max-width: 100%; + display:block; + margin-left: auto; + margin-right: auto; + } + } +} + +a { + border-bottom: 1px dotted $primary-bg; + color: $text; + text-decoration: none; + -webkit-transition: background 0.3s, color 0.3s; + transition: background 0.3s, color 0.3s; +} + +blockquote { + padding: 15px 30px 15px 15px; + margin: 20px 0 0 10px; + background-color: rgba(204, 122, 111, 0.1); + border-left: 10px solid rgba(191, 87, 73, 0.2); +} + +#fb_oss a { + border: 0; +} + +h1, h2, h3, h4 { + font-family: $header-font-family; + font-weight: 900; +} + +.navPusher { + border-top: $header-height + $header-ptop + $header-pbot solid $primary-bg; + height: 100%; + left: 0; + position: relative; + z-index: 99; +} + +.homeContainer { + background: $primary-bg; + color: $primary-overlay; + + a { + color: $primary-overlay; + } + + .homeSplashFade { + color: white; + } + + .homeWrapper { + padding: 2em 10px; + text-align: left; + + .wrapper { + margin: 0px auto; + max-width: $content-width; + padding: 0 20px; + } + + .projectLogo { + img { + height: 100px; + margin-bottom: 0px; + } + } + + h1#project_title { + font-family: $header-font-family; + font-size: 300%; + letter-spacing: -0.08em; + line-height: 1em; + margin-bottom: 80px; + } + + h2#project_tagline { + font-family: $header-font-family; + font-size: 200%; + letter-spacing: -0.04em; + line-height: 1em; + } + } +} + +.wrapper { + margin: 0px auto; + max-width: $content-width; + padding: 0 10px; +} + +.projectLogo { + display: none; + + img { + height: 100px; + margin-bottom: 0px; + } +} + +section#intro { + margin: 40px 0; +} + +.fbossFontLight { + font-family: $base-font-family; + font-weight: 300; + font-style: normal; +} + +.fb-like { + display: block; + margin-bottom: 20px; + width: 100%; +} + +.center { + display: block; + text-align: center; +} + +.mainContainer { + background: $secondary-bg; + overflow: auto; + + .mainWrapper { + padding: 4vh 10px; + text-align: left; + + .allShareBlock { + padding: 10px 0; + + .pluginBlock { + margin: 12px 0; + padding: 0; + } + } + + a { + &:hover, + &:focus { + background: $primary-bg; + color: $primary-overlay; + } + } + + em, i { + font-style: italic; + } + + strong, b { + font-weight: bold; + } + + h1 { + font-size: 300%; + line-height: 1em; + padding: 1.4em 0 1em; + text-align: left; + } + + h2 { + font-size: 250%; + line-height: 1em; + margin-bottom: 20px; + padding: 1.4em 0 20px; + text-align: left; + + & { + border-bottom: 1px solid darken($primary-bg, 10%); + color: darken($primary-bg, 10%); + font-size: 22px; + padding: 10px 0; + } + + &.blockHeader { + border-bottom: 1px solid white; + color: white; + font-size: 22px; + margin-bottom: 20px; + padding: 10px 0; + } + } + + h3 { + font-size: 150%; + line-height: 1.2em; + padding: 1em 0 0.8em; + } + + h4 { + font-size: 130%; + line-height: 1.2em; + padding: 1em 0 0.8em; + } + + p { + padding: 0.8em 0; + } + + ul { + list-style: disc; + } + + ol, ul { + padding-left: 24px; + li { + padding-bottom: 4px; + padding-left: 6px; + } + } + + strong { + font-weight: bold; + } + + .post { + position: relative; + + .katex { + font-weight: 700; + } + + &.basicPost { + margin-top: 30px; + } + + a { + color: $primary-bg; + + &:hover, + &:focus { + color: #fff; + } + } + + h2 { + border-bottom: 4px solid $primary-bg; + font-size: 130%; + } + + h3 { + border-bottom: 1px solid $primary-bg; + font-size: 110%; + } + + ol { + list-style: decimal outside none; + } + + .post-header { + padding: 1em 0; + + h1 { + font-size: 150%; + line-height: 1em; + padding: 0.4em 0 0; + + a { + border: none; + } + } + + .post-meta { + color: $primary-bg; + font-family: $header-font-family; + text-align: center; + } + } + + .postSocialPlugins { + padding-top: 1em; + } + + .docPagination { + background: $primary-bg; + bottom: 0px; + left: 0px; + position: absolute; + right: 0px; + + .pager { + display: inline-block; + width: 50%; + } + + .pagingNext { + float: right; + text-align: right; + } + + a { + border: none; + color: $primary-overlay; + display: block; + padding: 4px 12px; + + &:hover { + background-color: $secondary-bg; + color: $text; + } + + .pagerLabel { + display: inline; + } + + .pagerTitle { + display: none; + } + } + } + } + + .posts { + .post { + margin-bottom: 6vh; + } + } + } +} + +#integrations_title { + font-size: 250%; + margin: 80px 0; +} + +.ytVideo { + height: 0; + overflow: hidden; + padding-bottom: 53.4%; /* 16:9 */ + padding-top: 25px; + position: relative; +} + +.ytVideo iframe, +.ytVideo object, +.ytVideo embed { + height: 100%; + left: 0; + position: absolute; + top: 0; + width: 100%; +} + +@media only screen and (min-width: 480px) { + h1#project_title { + font-size: 500%; + } + + h2#project_tagline { + font-size: 250%; + } + + .projectLogo { + img { + margin-bottom: 10px; + height: 200px; + } + } + + .homeContainer .homeWrapper { + padding-left: 10px; + padding-right: 10px; + } + + .mainContainer { + .mainWrapper { + .post { + h2 { + font-size: 180%; + } + + h3 { + font-size: 120%; + } + + .docPagination { + a { + .pagerLabel { + display: none; + } + .pagerTitle { + display: inline; + } + } + } + } + } + } +} + +@media only screen and (min-width: 900px) { + .homeContainer { + .homeWrapper { + position: relative; + + #inner { + box-sizing: border-box; + max-width: 600px; + padding-right: 40px; + } + + .projectLogo { + align-items: center; + bottom: 0; + display: flex; + justify-content: flex-end; + left: 0; + padding: 2em 20px 4em; + position: absolute; + right: 20px; + top: 0; + + img { + height: 100%; + max-height: 250px; + } + } + } + } +} + +@media only screen and (min-width: 1024px) { + .mainContainer { + .mainWrapper { + .post { + box-sizing: border-box; + display: block; + + .post-header { + h1 { + font-size: 250%; + } + } + } + + .posts { + .post { + margin-bottom: 4vh; + width: 100%; + } + } + } + } +} + +@media only screen and (min-width: 1200px) { + .homeContainer { + .homeWrapper { + #inner { + max-width: 750px; + } + } + } + + .wrapper { + max-width: 1100px; + } +} + +@media only screen and (min-width: 1500px) { + .homeContainer { + .homeWrapper { + #inner { + max-width: 1100px; + padding-bottom: 40px; + padding-top: 40px; + } + } + } + + .wrapper { + max-width: 1400px; + } +} http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_sass/_blog.scss ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_sass/_blog.scss b/thirdparty/rocksdb/docs/_sass/_blog.scss new file mode 100644 index 0000000..74335d1 --- /dev/null +++ b/thirdparty/rocksdb/docs/_sass/_blog.scss @@ -0,0 +1,45 @@ +.blogContainer { + .posts { + margin-top: 60px; + + .post { + border: 1px solid $primary-bg; + border-radius: 3px; + padding: 10px 20px 20px; + } + } + + .lonePost { + margin-top: 60px; + + .post { + padding: 10px 0px 0px; + } + } + + .post-header { + h1 { + text-align: center; + } + + .post-authorName { + color: rgba($text, 0.7); + font-size: 14px; + font-weight: 900; + margin-top: 0; + padding: 0; + text-align: center; + } + + .authorPhoto { + border-radius: 50%; + height: 50px; + left: 50%; + margin-left: -25px; + overflow: hidden; + position: absolute; + top: -25px; + width: 50px; + } + } +} \ No newline at end of file http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_sass/_buttons.scss ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_sass/_buttons.scss b/thirdparty/rocksdb/docs/_sass/_buttons.scss new file mode 100644 index 0000000..a037161 --- /dev/null +++ b/thirdparty/rocksdb/docs/_sass/_buttons.scss @@ -0,0 +1,47 @@ +.button { + border: 1px solid $primary-bg; + border-radius: 3px; + color: $primary-bg; + display: inline-block; + font-size: 14px; + font-weight: 900; + line-height: 1.2em; + padding: 10px; + text-transform: uppercase; + transition: background 0.3s, color 0.3s; + + &:hover { + background: $primary-bg; + color: $primary-overlay; + } +} + +.homeContainer { + .button { + border-color: $primary-overlay; + border-width: 1px; + color: $primary-overlay; + + &:hover { + background: $primary-overlay; + color: $primary-bg; + } + } +} + +.blockButton { + display: block; +} + +.edit-page-link { + float: right; + font-size: 14px; + font-weight: normal; + line-height: 20px; + opacity: 0.6; + transition: opacity 0.5s; +} + +.edit-page-link:hover { + opacity: 1; +} http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_sass/_footer.scss ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_sass/_footer.scss b/thirdparty/rocksdb/docs/_sass/_footer.scss new file mode 100644 index 0000000..5b74395 --- /dev/null +++ b/thirdparty/rocksdb/docs/_sass/_footer.scss @@ -0,0 +1,82 @@ +.footerContainer { + background: $secondary-bg; + color: $primary-bg; + overflow: hidden; + padding: 0 10px; + text-align: left; + + .footerWrapper { + border-top: 1px solid $primary-bg; + padding: 0; + + .footerBlocks { + align-items: center; + align-content: center; + display: flex; + flex-flow: row wrap; + margin: 0 -20px; + padding: 10px 0; + } + + .footerSection { + box-sizing: border-box; + flex: 1 1 25%; + font-size: 14px; + min-width: 275px; + padding: 0px 20px; + + a { + border: 0; + color: inherit; + display: inline-block; + line-height: 1.2em; + } + + .footerLink { + padding-right: 20px; + } + } + + .fbOpenSourceFooter { + align-items: center; + display: flex; + flex-flow: row nowrap; + max-width: 25%; + + .facebookOSSLogoSvg { + flex: 0 0 31px; + height: 30px; + margin-right: 10px; + width: 31px; + + path { + fill: $primary-bg; + } + + .middleRing { + opacity: 0.7; + } + + .innerRing { + opacity: 0.45; + } + } + + h2 { + display: block; + font-weight: 900; + line-height: 1em; + } + } + } +} + +@media only screen and (min-width: 900px) { + .footerSection { + &.rightAlign { + margin-left: auto; + max-width: 25%; + text-align: right; + } + } +} \ No newline at end of file http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_sass/_gridBlock.scss ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_sass/_gridBlock.scss b/thirdparty/rocksdb/docs/_sass/_gridBlock.scss new file mode 100644 index 0000000..679b31c --- /dev/null +++ b/thirdparty/rocksdb/docs/_sass/_gridBlock.scss @@ -0,0 +1,115 @@ +.gridBlock { + margin: -5px 0; + padding: 0; + padding-bottom: 20px; + + .blockElement { + padding: 5px 0; + + img { + max-width: 100%; + } + + h3 { + border-bottom: 1px solid rgba($primary-bg, 0.5); + color: $primary-bg; + font-size: 18px; + margin: 0; + padding: 10px 0; + } + } + + .gridClear { + clear: both; + } + +} + +.gridBlock .alignCenter { + text-align: center; +} +.gridBlock .alignRight { + text-align: right; +} +.gridBlock .imageAlignSide { + align-items: center; + display: flex; + flex-flow: row wrap; +} +.blockImage { + max-width: 150px; + width: 50%; +} +.imageAlignTop .blockImage { + margin-bottom: 20px; +} +.imageAlignTop.alignCenter .blockImage { + margin-left: auto; + margin-right: auto; +} +.imageAlignSide .blockImage { + flex: 0 1 100px; + margin-right: 20px; +} +.imageAlignSide .blockContent { + flex: 1 1; +} + +@media only screen and (max-width: 1023px) { + .responsiveList .blockContent { + position: relative; + } + .responsiveList .blockContent > div { + padding-left: 20px; + } + .responsiveList .blockContent::before { + content: "\2022"; + position: absolute; + } +} + +@media only screen and (min-width: 1024px) { + .gridBlock { + display: flex; + flex-direction: row; + flex-wrap: wrap; + margin: -10px -10px 10px -10px; + + .twoByGridBlock { + box-sizing: border-box; + flex: 1 0 50%; + padding: 10px; + } + + .fourByGridBlock { + box-sizing: border-box; + flex: 1 0 25%; + padding: 10px; + } + } + + h2 + .gridBlock { + padding-top: 20px; + } +} + +@media only screen and (min-width: 1400px) { + .gridBlock { + display: flex; + flex-direction: row; + flex-wrap: wrap; + margin: -10px -20px 10px -20px; + + .twoByGridBlock { + box-sizing: border-box; + flex: 1 0 50%; + padding: 10px 20px; + } + + .fourByGridBlock { + box-sizing: border-box; + flex: 1 0 25%; + padding: 10px 20px; + } + } +} \ No newline at end of file http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_sass/_header.scss ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_sass/_header.scss b/thirdparty/rocksdb/docs/_sass/_header.scss new file mode 100644 index 0000000..b4cd071 --- /dev/null +++ b/thirdparty/rocksdb/docs/_sass/_header.scss @@ -0,0 +1,138 @@ +.fixedHeaderContainer { + background: $primary-bg; + color: $primary-overlay; + height: $header-height; + padding: $header-ptop 0 $header-pbot; + position: fixed; + width: 100%; + z-index: 9999; + + a { + align-items: center; + border: 0; + color: $primary-overlay; + display: flex; + flex-flow: row nowrap; + height: $header-height; + } + + header { + display: flex; + flex-flow: row nowrap; + position: relative; + text-align: left; + + img { + height: 24px; + margin-right: 10px; + } + + h2 { + display: block; + font-family: $header-font-family; + font-weight: 900; + line-height: 18px; + position: relative; + } + } +} + +.navigationFull { + height: 34px; + margin-left: auto; + + nav { + position: relative; + + ul { + display: flex; + flex-flow: row nowrap; + margin: 0 -10px; + + li { + padding: 0 10px; + display: block; + + a { + border: 0; + color: $primary-overlay-special; + font-size: 16px; + font-weight: 400; + line-height: 1.2em; + + &:hover { + border-bottom: 2px solid $primary-overlay; + color: $primary-overlay; + } + } + + &.navItemActive { + a { + color: $primary-overlay; + } + } + } + } + } +} + +/* 900px + + + .fixedHeaderContainer { + .navigationWrapper { + nav { + padding: 0 1em; + position: relative; + top: -9px; + + ul { + margin: 0 -0.4em; + li { + display: inline-block; + + a { + padding: 14px 0.4em; + border: 0; + color: $primary-overlay-special; + display: inline-block; + + &:hover { + color: $primary-overlay; + } + } + + &.navItemActive { + a { + color: $primary-overlay; + } + } + } + } + } + + &.navigationFull { + display: inline-block; + } + + &.navigationSlider { + display: none; + } + } + } + + 1200px + + .fixedHeaderContainer { + header { + max-width: 1100px; + } + } + + 1500px + .fixedHeaderContainer { + header { + max-width: 1400px; + } + } + */ \ No newline at end of file http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_sass/_poweredby.scss ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_sass/_poweredby.scss b/thirdparty/rocksdb/docs/_sass/_poweredby.scss new file mode 100644 index 0000000..4155b60 --- /dev/null +++ b/thirdparty/rocksdb/docs/_sass/_poweredby.scss @@ -0,0 +1,69 @@ +.poweredByContainer { + background: $primary-bg; + color: $primary-overlay; + margin-bottom: 20px; + + a { + color: $primary-overlay; + } + + .poweredByWrapper { + h2 { + border-color: $primary-overlay-special; + color: $primary-overlay-special; + } + } + + .poweredByMessage { + color: $primary-overlay-special; + font-size: 14px; + padding-top: 20px; + } +} + +.poweredByItems { + display: flex; + flex-flow: row wrap; + margin: 0 -10px; +} + +.poweredByItem { + box-sizing: border-box; + flex: 1 0 50%; + line-height: 1.1em; + padding: 5px 10px; + + &.itemLarge { + flex-basis: 100%; + padding: 10px; + text-align: center; + + &:nth-child(4) { + padding-bottom: 20px; + } + + img { + max-height: 30px; + } + } +} + +@media only screen and (min-width: 480px) { + .itemLarge { + flex-basis: 50%; + max-width: 50%; + } +} + +@media only screen and (min-width: 1024px) { + .poweredByItem { + flex-basis: 25%; + max-width: 25%; + + &.itemLarge { + padding-bottom: 20px; + text-align: left; + } + } +} + http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_sass/_promo.scss ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_sass/_promo.scss b/thirdparty/rocksdb/docs/_sass/_promo.scss new file mode 100644 index 0000000..8c9a809 --- /dev/null +++ b/thirdparty/rocksdb/docs/_sass/_promo.scss @@ -0,0 +1,55 @@ +.promoSection { + display: flex; + flex-flow: column wrap; + font-size: 125%; + line-height: 1.6em; + margin: -10px 0; + position: relative; + z-index: 99; + + .promoRow { + padding: 10px 0; + + .pluginWrapper { + display: block; + + &.ghWatchWrapper, &.ghStarWrapper { + height: 28px; + } + } + + .pluginRowBlock { + display: flex; + flex-flow: row wrap; + margin: 0 -2px; + + .pluginWrapper { + padding: 0 2px; + } + } + } +} + +iframe.pluginIframe { + height: 500px; + margin-top: 20px; + width: 100%; +} + +.iframeContent { + display: none; +} + +.iframePreview { + display: inline-block; + margin-top: 20px; +} + +@media only screen and (min-width: 1024px) { + .iframeContent { + display: block; + } + .iframePreview { + display: none; + } +} \ No newline at end of file http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_sass/_react_docs_nav.scss ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_sass/_react_docs_nav.scss b/thirdparty/rocksdb/docs/_sass/_react_docs_nav.scss new file mode 100644 index 0000000..f0a651e --- /dev/null +++ b/thirdparty/rocksdb/docs/_sass/_react_docs_nav.scss @@ -0,0 +1,332 @@ +.docsNavContainer { + background: $sidenav; + height: 35px; + left: 0; + position: fixed; + width: 100%; + z-index: 100; +} + +.docMainWrapper { + .wrapper { + &.mainWrapper { + padding-left: 0; + padding-right: 0; + padding-top: 10px; + } + } +} + +.docsSliderActive { + .docsNavContainer { + box-sizing: border-box; + height: 100%; + overflow-y: auto; + -webkit-overflow-scrolling: touch; + padding-bottom: 50px; + } + + .mainContainer { + display: none; + } +} + +.navBreadcrumb { + box-sizing: border-box; + display: flex; + flex-flow: row nowrap; + font-size: 12px; + height: 35px; + overflow: hidden; + padding: 5px 10px; + + a, span { + border: 0; + color: $sidenav-text; + } + + i { + padding: 0 3px; + } +} + +nav.toc { + position: relative; + + section { + padding: 0px; + position: relative; + + .navGroups { + display: none; + padding: 40px 10px 10px; + } + } + + .toggleNav { + background: $sidenav; + color: $sidenav-text; + position: relative; + transition: background-color 0.3s, color 0.3s; + + .navToggle { + cursor: pointer; + height: 24px; + margin-right: 10px; + position: relative; + text-align: left; + width: 18px; + + &::before, &::after { + content: ""; + position: absolute; + top: 50%; + left: 0; + left: 8px; + width: 3px; + height: 6px; + border: 5px solid $sidenav-text; + border-width: 5px 0; + margin-top: -8px; + transform: rotate(45deg); + z-index: 1; + } + + &::after { + transform: rotate(-45deg); + } + + i { + &::before, &::after { + content: ""; + position: absolute; + top: 50%; + left: 2px; + background: transparent; + border-width: 0 5px 5px; + border-style: solid; + border-color: transparent $sidenav-text; + height: 0; + margin-top: -7px; + opacity: 1; + width: 5px; + z-index: 10; + } + + &::after { + border-width: 5px 5px 0; + margin-top: 2px; + } + } + } + + .navGroup { + background: $sidenav-overlay; + margin: 1px 0; + + ul { + display: none; + } + + h3 { + background: $sidenav-overlay; + color: $sidenav-text; + cursor: pointer; + font-size: 14px; + font-weight: 400; + line-height: 1.2em; + padding: 10px; + transition: color 0.2s; + + i:not(:empty) { + width: 16px; + height: 16px; + display: inline-block; + box-sizing: border-box; + text-align: center; + color: rgba($sidenav-text, 0.5); + margin-right: 10px; + transition: color 0.2s; + } + + &:hover { + color: $primary-bg; + + i:not(:empty) { + color: $primary-bg; + } + } + } + + &.navGroupActive { + background: $sidenav-active; + color: $sidenav-text; + + ul { + display: block; + padding-bottom: 10px; + padding-top: 10px; + } + + h3 { + background: $primary-bg; + color: $primary-overlay; + + i { + display: none; + } + } + } + } + + ul { + padding-left: 0; + padding-right: 24px; + + li { + list-style-type: none; + padding-bottom: 0; + padding-left: 0; + + a { + border: none; + color: $sidenav-text; + display: inline-block; + font-size: 14px; + line-height: 1.1em; + margin: 2px 10px 5px; + padding: 5px 0 2px; + transition: color 0.3s; + + &:hover, + &:focus { + color: $primary-bg; + } + + &.navItemActive { + color: $primary-bg; + font-weight: 900; + } + } + } + } + } + + .toggleNavActive { + .navBreadcrumb { + background: $sidenav; + margin-bottom: 20px; + position: fixed; + width: 100%; + } + + section { + .navGroups { + display: block; + } + } + + + .navToggle { + &::before, &::after { + border-width: 6px 0; + height: 0px; + margin-top: -6px; + } + + i { + opacity: 0; + } + } + } +} + +.docsNavVisible { + .navPusher { + .mainContainer { + padding-top: 35px; + } + } +} + +@media only screen and (min-width: 900px) { + .navBreadcrumb { + padding: 5px 0; + } + + nav.toc { + section { + .navGroups { + padding: 40px 0 0; + } + } + } +} + +@media only screen and (min-width: 1024px) { + .navToggle { + display: none; + } + + .docsSliderActive { + .mainContainer { + display: block; + } + } + + .docsNavVisible { + .navPusher { + .mainContainer { + padding-top: 0; + } + } + } + + .docsNavContainer { + background: none; + box-sizing: border-box; + height: auto; + margin: 40px 40px 0 0; + overflow-y: auto; + position: relative; + width: 300px; + } + + nav.toc { + section { + .navGroups { + display: block; + padding-top: 0px; + } + } + + .toggleNavActive { + .navBreadcrumb { + margin-bottom: 0; + position: relative; + } + } + } + + .docMainWrapper { + display: flex; + flex-flow: row nowrap; + margin-bottom: 40px; + + .wrapper { + padding-left: 0; + padding-right: 0; + + &.mainWrapper { + padding-top: 0; + } + } + } + + .navBreadcrumb { + display: none; + h2 { + padding: 0 10px; + } + } +} \ No newline at end of file http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_sass/_react_header_nav.scss ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_sass/_react_header_nav.scss b/thirdparty/rocksdb/docs/_sass/_react_header_nav.scss new file mode 100644 index 0000000..13c0e56 --- /dev/null +++ b/thirdparty/rocksdb/docs/_sass/_react_header_nav.scss @@ -0,0 +1,141 @@ +.navigationFull { + display: none; +} + +.navigationSlider { + position: absolute; + right: 0px; + + .navSlideout { + cursor: pointer; + padding-top: 4px; + position: absolute; + right: 10px; + top: 0; + transition: top 0.3s; + z-index: 101; + } + + .slidingNav { + background: $secondary-bg; + box-sizing: border-box; + height: 0px; + overflow-x: hidden; + padding: 0; + position: absolute; + right: 0px; + top: 0; + transition: height 0.3s cubic-bezier(0.68, -0.55, 0.265, 1.55), width 0.3s cubic-bezier(0.68, -0.55, 0.265, 1.55); + width: 0; + + ul { + flex-flow: column nowrap; + list-style: none; + padding: 10px; + + li { + margin: 0; + padding: 2px 0; + + a { + color: $primary-bg; + display: inline; + margin: 3px 5px; + padding: 2px 0px; + transition: background-color 0.3s; + + &:focus, + &:hover { + border-bottom: 2px solid $primary-bg; + } + } + } + } + } + + .navSlideoutActive { + .slidingNav { + height: auto; + padding-top: $header-height + $header-pbot; + width: 300px; + } + + .navSlideout { + top: -2px; + .menuExpand { + span:nth-child(1) { + background-color: $text; + top: 16px; + transform: rotate(45deg); + } + span:nth-child(2) { + opacity: 0; + } + span:nth-child(3) { + background-color: $text; + transform: rotate(-45deg); + } + } + } + } +} + +.menuExpand { + display: flex; + flex-flow: column nowrap; + height: 20px; + justify-content: space-between; + + span { + background: $primary-overlay; + border-radius: 3px; + display: block; + flex: 0 0 4px; + height: 4px; + position: relative; + top: 0; + transition: background-color 0.3s, top 0.3s, opacity 0.3s, transform 0.3s; + width: 20px; + } +} + +.navPusher { + border-top: $header-height + $header-ptop + $header-pbot solid $primary-bg; + position: relative; + left: 0; + z-index: 99; + height: 100%; + + &::after { + position: absolute; + top: 0; + right: 0; + width: 0; + height: 0; + background: rgba(0,0,0,0.4); + content: ''; + opacity: 0; + -webkit-transition: opacity 0.5s, width 0.1s 0.5s, height 0.1s 0.5s; + transition: opacity 0.5s, width 0.1s 0.5s, height 0.1s 0.5s; + } + + .sliderActive &::after { + width: 100%; + height: 100%; + opacity: 1; + -webkit-transition: opacity 0.5s; + transition: opacity 0.5s; + z-index: 100; + } +} + + +@media only screen and (min-width: 1024px) { + .navigationFull { + display: block; + } + + .navigationSlider { + display: none; + } +} \ No newline at end of file http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_sass/_reset.scss ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_sass/_reset.scss b/thirdparty/rocksdb/docs/_sass/_reset.scss new file mode 100644 index 0000000..0e5f2e0 --- /dev/null +++ b/thirdparty/rocksdb/docs/_sass/_reset.scss @@ -0,0 +1,43 @@ +html, body, div, span, applet, object, iframe, +h1, h2, h3, h4, h5, h6, p, blockquote, pre, +a, abbr, acronym, address, big, cite, code, +del, dfn, em, img, ins, kbd, q, s, samp, +small, strike, strong, sub, sup, tt, var, +b, u, i, center, +dl, dt, dd, ol, ul, li, +fieldset, form, label, legend, +table, caption, tbody, tfoot, thead, tr, th, td, +article, aside, canvas, details, embed, +figure, figcaption, footer, header, hgroup, +menu, nav, output, ruby, section, summary, +time, mark, audio, video { + margin: 0; + padding: 0; + border: 0; + font-size: 100%; + font: inherit; + vertical-align: baseline; +} +/* HTML5 display-role reset for older browsers */ +article, aside, details, figcaption, figure, +footer, header, hgroup, menu, nav, section { + display: block; +} +body { + line-height: 1; +} +ol, ul { + list-style: none; +} +blockquote, q { + quotes: none; +} +blockquote:before, blockquote:after, +q:before, q:after { + content: ''; + content: none; +} +table { + border-collapse: collapse; + border-spacing: 0; +}
