http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_docs/faq.md ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_docs/faq.md b/thirdparty/rocksdb/docs/_docs/faq.md new file mode 100644 index 0000000..0887a09 --- /dev/null +++ b/thirdparty/rocksdb/docs/_docs/faq.md @@ -0,0 +1,48 @@ +--- +docid: support-faq +title: FAQ +layout: docs +permalink: /docs/support/faq.html +--- + +Here is an ever-growing list of frequently asked questions around RocksDB + +## What is RocksDB? + +RocksDB is an embeddable persistent key-value store for fast storage. RocksDB can also be the foundation for a client-server database but our current focus is on embedded workloads. + +RocksDB builds on [LevelDB](https://code.google.com/p/leveldb/) to be scalable to run on servers with many CPU cores, to efficiently use fast storage, to support IO-bound, in-memory and write-once workloads, and to be flexible to allow for innovation. + +For the latest details, watch [Mark Callaghanâs and Igor Canadiâs talk at CMU on 10/2015](https://scs.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=f4e0eb37-ae18-468f-9248-cb73edad3e56). [Dhruba Borthakurâs introductory talk](https://github.com/facebook/rocksdb/blob/gh-pages-old/intro.pdf?raw=true) from the Data @ Scale 2013 conference provides some perspective about how RocksDB has evolved. + +## How does performance compare? + +We benchmarked LevelDB and found that it was unsuitable for our server workloads. The [benchmark results](http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html) look awesome at first sight, but we quickly realized that those results were for a database whose size was smaller than the size of RAM on the test machine â where the entire database could fit in the OS page cache. When we performed the same benchmarks on a database that was at least 5 times larger than main memory, the performance results were dismal. + +By contrast, weâve published the [RocksDB benchmark results](https://github.com/facebook/rocksdb/wiki/Performance-Benchmarks) for server side workloads on Flash. We also measured the performance of LevelDB on these server-workload benchmarks and found that RocksDB solidly outperforms LevelDB for these IO bound workloads. We found that LevelDBâs single-threaded compaction process was insufficient to drive server workloads. We saw frequent write-stalls with LevelDB that caused 99-percentile latency to be tremendously large. We found that mmap-ing a file into the OS cache introduced performance bottlenecks for reads. We could not make LevelDB consume all the IOs offered by the underlying Flash storage. + +## What is RocksDB suitable for? + +RocksDB can be used by applications that need low latency database accesses. Possibilities include: + +* A user-facing application that stores the viewing history and state of users of a website. +* A spam detection application that needs fast access to big data sets. +* A graph-search query that needs to scan a data set in realtime. +* A cache data from Hadoop, thereby allowing applications to query Hadoop data in realtime. +* A message-queue that supports a high number of inserts and deletes. + +## How big is RocksDB adoption? + +RocksDB is an embedded storage engine that is used in a number of backend systems at Facebook. In the Facebook newsfeedâs backend, it replaced another internal storage engine called Centrifuge and is one of the many components used. ZippyDB, a distributed key value store service used by Facebook products relies RocksDB. Details on ZippyDB are in [Muthu Annamalaiâs talk at Data@Scale in Seattle](https://youtu.be/DfiN7pG0D0k). Dragon, a distributed graph query engine part of the social graph infrastructure, is using RocksDB to store data. Parse has been running [MongoDB on RocksDB in production](http://blog.parse.com/announcements/mongodb-rocksdb-parse/) since early 2015. + +RocksDB is proving to be a useful component for a lot of other groups in the industry. For a list of projects currently using RocksDB, take a look at our USERS.md list on github. + +## How good is RocksDB as a database storage engine? + +Our engineering team at Facebook firmly believes that RocksDB has great potential as storage engine for databases. It has been proven in production with MongoDB: [MongoRocks](https://github.com/mongodb-partners/mongo-rocks) is the RocksDB based storage engine for MongoDB. + +[MyRocks](https://code.facebook.com/posts/190251048047090/myrocks-a-space-and-write-optimized-mysql-database/) is the RocksDB based storage engine for MySQL. Using RocksDB we have managed to achieve 2x better compression and 10x less write amplification for our benchmarks compared to our existing MySQL setup. Given our current results, work is currently underway to develop MyRocks into a production ready solution for web-scale MySQL workloads. Follow along on [GitHub](https://github.com/facebook/mysql-5.6)! + +## Why is RocksDB open sourced? + +We are open sourcing this project on [GitHub](http://github.com/facebook/rocksdb) because we think it will be useful beyond Facebook. We are hoping that software programmers and database developers will use, enhance, and customize RocksDB for their use-cases. We would also like to engage with the academic community on topics related to efficiency for modern database algorithms.
http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_docs/getting-started.md ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_docs/getting-started.md b/thirdparty/rocksdb/docs/_docs/getting-started.md new file mode 100644 index 0000000..8b01dfe --- /dev/null +++ b/thirdparty/rocksdb/docs/_docs/getting-started.md @@ -0,0 +1,78 @@ +--- +docid: getting-started +title: Getting started +layout: docs +permalink: /docs/getting-started.html +--- + +## Overview + +The RocksDB library provides a persistent key value store. Keys and values are arbitrary byte arrays. The keys are ordered within the key value store according to a user-specified comparator function. + +The library is maintained by the Facebook Database Engineering Team, and is based on [LevelDB](https://github.com/google/leveldb), by Sanjay Ghemawat and Jeff Dean at Google. + +This overview gives some simple examples of how RocksDB is used. For the story of why RocksDB was created in the first place, see [Dhruba Borthakurâs introductory talk](https://github.com/facebook/rocksdb/blob/gh-pages-old/intro.pdf?raw=true) from the Data @ Scale 2013 conference. + +## Opening A Database + +A rocksdb database has a name which corresponds to a file system directory. All of the contents of database are stored in this directory. The following example shows how to open a database, creating it if necessary: + +```c++ +#include <assert> +#include "rocksdb/db.h" + +rocksdb::DB* db; +rocksdb::Options options; +options.create_if_missing = true; +rocksdb::Status status = + rocksdb::DB::Open(options, "/tmp/testdb", &db); +assert(status.ok()); +... +``` + +If you want to raise an error if the database already exists, add the following line before the rocksdb::DB::Open call: + +```c++ +options.error_if_exists = true; +``` + +## Status + +You may have noticed the `rocksdb::Status` type above. Values of this type are returned by most functions in RocksDB that may encounter +an error. You can check if such a result is ok, and also print an associated error message: + +```c++ +rocksdb::Status s = ...; +if (!s.ok()) cerr << s.ToString() << endl; +``` + +## Closing A Database + +When you are done with a database, just delete the database object. For example: + +```c++ +/* open the db as described above */ +/* do something with db */ +delete db; +``` + +## Reads And Writes + +The database provides Put, Delete, and Get methods to modify/query the database. For example, the following code moves the value stored under `key1` to `key2`. + +```c++ +std::string value; +rocksdb::Status s = db->Get(rocksdb::ReadOptions(), key1, &value); +if (s.ok()) s = db->Put(rocksdb::WriteOptions(), key2, value); +if (s.ok()) s = db->Delete(rocksdb::WriteOptions(), key1); +``` + +## Further documentation + +These are just simple examples of how RocksDB is used. The full documentation is currently on the [GitHub wiki](https://github.com/facebook/rocksdb/wiki). + +Here are some specific details about the RocksDB implementation: + +- [Architecture Guide](https://github.com/facebook/rocksdb/wiki/Rocksdb-Architecture-Guide) +- [Format of an immutable Table file](https://github.com/facebook/rocksdb/wiki/Rocksdb-Table-Format) +- [Format of a log file](https://github.com/facebook/rocksdb/wiki/Write-Ahead-Log-File-Format) http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/blog_pagination.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/blog_pagination.html b/thirdparty/rocksdb/docs/_includes/blog_pagination.html new file mode 100644 index 0000000..6a1f334 --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/blog_pagination.html @@ -0,0 +1,28 @@ +<!-- Pagination links - copied from http://jekyllrb.com/docs/pagination/ --> +{% if paginator.total_pages > 1 %} +<br /> +<div class="pagination"> + {% if paginator.previous_page %} + <a href="{{ paginator.previous_page_path | replace: '//', '/' }}">« Prev</a> + {% else %} + <span>« Prev</span> + {% endif %} + + {% for page in (1..paginator.total_pages) %} + {% if page == paginator.page %} + <em>{{ page }}</em> + {% elsif page == 1 %} + <a href="{{ '/blog' }}">{{ page }}</a> + {% else %} + <a href="{{ site.paginate_path | replace: '//', '/' | replace: ':num', page }}">{{ page }}</a> + {% endif %} + {% endfor %} + + {% if paginator.next_page %} + <a href="{{ paginator.next_page_path | replace: '//', '/' }}">Next »</a> + {% else %} + <span>Next »</span> + {% endif %} +</div> +<br /> +{% endif %} http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/content/gridblocks.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/content/gridblocks.html b/thirdparty/rocksdb/docs/_includes/content/gridblocks.html new file mode 100644 index 0000000..49c5e59 --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/content/gridblocks.html @@ -0,0 +1,5 @@ +<div class="gridBlock"> +{% for item in {{include.data_source}} %} + {% include content/items/gridblock.html item=item layout=include.layout imagealign=include.imagealign align=include.align %} +{% endfor %} +</div> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/content/items/gridblock.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/content/items/gridblock.html b/thirdparty/rocksdb/docs/_includes/content/items/gridblock.html new file mode 100644 index 0000000..58c9e7f --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/content/items/gridblock.html @@ -0,0 +1,37 @@ +{% if include.layout == "fourColumn" %} + {% assign layout = "fourByGridBlock" %} +{% else %} + {% assign layout = "twoByGridBlock" %} +{% endif %} + +{% if include.imagealign == "side" %} + {% assign imagealign = "imageAlignSide" %} +{% else %} + {% if item.image %} + {% assign imagealign = "imageAlignTop" %} + {% else %} + {% assign imagealign = "" %} + {% endif %} +{% endif %} + +{% if include.align == "right" %} + {% assign align = "alignRight" %} +{% elsif include.align == "center" %} + {% assign align = "alignCenter" %} +{% else %} + {% assign align = "alignLeft" %} +{% endif %} + +<div class="blockElement {{ layout }} {{ imagealign }} {{ align }}"> + {% if item.image %} + <div class="blockImage"> + <img src="/static/{{ item.image }}" alt="{{ item.title }}" title="{{ item.title }}" /> + </div> + {% endif %} + <div class="blockContent"> + <h3>{{ item.title }}</h3> + {% if item.text %} + {{ item.text | markdownify }} + {% endif %} + </div> +</div> http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/doc.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/doc.html b/thirdparty/rocksdb/docs/_includes/doc.html new file mode 100644 index 0000000..a795000 --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/doc.html @@ -0,0 +1,25 @@ +<div class="post"> + <header class="post-header"> + <h1 class="post-title">{% if include.truncate %}<a href="{{ page.url | absolute_url }}">{{ page.title }}</a>{% else %}{{ page.title }}{% endif %}</h1> + </header> + + <article class="post-content"> + {% if include.truncate %} + {% if page.content contains '<!--truncate-->' %} + {{ page.content | split:'<!--truncate-->' | first }} + <div class="read-more"> + <a href="{{ page.url | absolute_url }}" > + ...Read More + </a> + </div> + {% else %} + {{ page.content }} + {% endif %} + {% else %} + {{ content }} + + <p><a class="edit-page-link" href="https://github.com/{{ site.ghrepo }}/blob/master/docs/{{ page.path }}" target="_blank">Edit on GitHub</a></p> + {% endif %} + </article> + {% include doc_paging.html %} +</div> http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/doc_paging.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/doc_paging.html b/thirdparty/rocksdb/docs/_includes/doc_paging.html new file mode 100644 index 0000000..e69de29 http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/footer.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/footer.html b/thirdparty/rocksdb/docs/_includes/footer.html new file mode 100644 index 0000000..dd9494a --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/footer.html @@ -0,0 +1,33 @@ +<div class="footerContainer"> + <div id="footer_wrap" class="wrapper footerWrapper"> + <div class="footerBlocks"> + <div id="fb_oss" class="footerSection fbOpenSourceFooter"> + <svg class="facebookOSSLogoSvg" viewBox="0 0 1133.9 1133.9" x="0px" y="0px"> + <g> + <path class="logoRing outerRing" d="M 498.3 3.7 c 153.6 88.9 307.3 177.7 461.1 266.2 c 7.6 4.4 10.3 9.1 10.3 17.8 c -0.3 179.1 -0.2 358.3 0 537.4 c 0 8.1 -2.4 12.8 -9.7 17.1 c -154.5 88.9 -308.8 178.1 -462.9 267.5 c -9 5.2 -15.5 5.3 -24.6 0.1 c -153.9 -89.2 -307.9 -178 -462.1 -266.8 C 3 838.8 0 833.9 0 825.1 c 0.3 -179.1 0.2 -358.3 0 -537.4 c 0 -8.6 2.6 -13.6 10.2 -18 C 164.4 180.9 318.4 92 472.4 3 C 477 -1.5 494.3 -0.7 498.3 3.7 Z M 48.8 555.3 c 0 79.9 0.2 159.9 -0.2 239.8 c -0.1 10 3 15.6 11.7 20.6 c 137.2 78.8 274.2 157.8 411 237.3 c 9.9 5.7 17 5.7 26.8 0.1 c 137.5 -79.8 275.2 -159.2 412.9 -238.5 c 7.4 -4.3 10.5 -8.9 10.5 -17.8 c -0.3 -160.2 -0.3 -320.5 0 -480.7 c 0 -8.8 -2.8 -13.6 -10.3 -18 C 772.1 218 633.1 137.8 494.2 57.4 c -6.5 -3.8 -11.5 -4.5 -18.5 -0.5 C 336.8 137.4 197.9 217.7 58.8 297.7 c -7.7 4.4 -10.2 9.2 -10.2 17.9 C 48.9 395.5 48.8 475.4 48.8 555.3 Z" /> + <path class="logoRing middleRing" d="M 184.4 555.9 c 0 -33.3 -1 -66.7 0.3 -100 c 1.9 -48 24.1 -86 64.7 -110.9 c 54.8 -33.6 110.7 -65.5 167 -96.6 c 45.7 -25.2 92.9 -24.7 138.6 1 c 54.4 30.6 108.7 61.5 162.2 93.7 c 44 26.5 67.3 66.8 68 118.4 c 0.9 63.2 0.9 126.5 0 189.7 c -0.7 50.6 -23.4 90.7 -66.6 116.9 c -55 33.4 -110.8 65.4 -167.1 96.5 c -43.4 24 -89 24.2 -132.3 0.5 c -57.5 -31.3 -114.2 -64 -170 -98.3 c -41 -25.1 -62.9 -63.7 -64.5 -112.2 C 183.5 621.9 184.3 588.9 184.4 555.9 Z M 232.9 556.3 c 0 29.5 0.5 59.1 -0.1 88.6 c -0.8 39.2 16.9 67.1 50.2 86.2 c 51.2 29.4 102.2 59.2 153.4 88.4 c 31.4 17.9 63.6 18.3 95 0.6 c 53.7 -30.3 107.1 -61.2 160.3 -92.5 c 29.7 -17.5 45 -44.5 45.3 -78.8 c 0.6 -61.7 0.5 -123.5 0 -185.2 c -0.3 -34.4 -15.3 -61.5 -44.9 -79 C 637.7 352.6 583 320.8 527.9 290 c -27.5 -15.4 -57.2 -16.1 -84.7 -0.7 c -56.9 31.6 -113.4 64 -169.1 97.6 c -26.4 15.9 -40.7 41.3 -41.1 72.9 C 232.6 491.9 232.9 524.1 232.9 556.3 Z" /> + <path class="logoRing innerRing" d="M 484.9 424.4 c 69.8 -2.8 133.2 57.8 132.6 132 C 617 630 558.5 688.7 484.9 689.1 c -75.1 0.4 -132.6 -63.6 -132.7 -132.7 C 352.1 485 413.4 421.5 484.9 424.4 Z M 401.3 556.7 c -3.4 37.2 30.5 83.6 83 84.1 c 46.6 0.4 84.8 -37.6 84.9 -84 c 0.1 -46.6 -37.2 -84.4 -84.2 -84.6 C 432.2 472.1 397.9 518.3 401.3 556.7 Z" /> + </g> + </svg> + <h2>Facebook Open Source</h2> + </div> + <div class="footerSection"> + <a class="footerLink" href="https://code.facebook.com/projects/"; target="_blank">Open Source Projects</a> + <a class="footerLink" href="https://github.com/facebook/"; target="_blank">GitHub</a> + <a class="footerLink" href="https://twitter.com/fbOpenSource"; target="_blank">Twitter</a> + </div> + <div class="footerSection rightAlign"> + <a class="footerLink" href="https://github.com/{{ site.ghrepo }}" target="_blank">Contribute to this project on GitHub</a> + </div> + </div> + </div> +</div> +<script> + (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ + (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), + m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) + })(window,document,'script','//www.google-analytics.com/analytics.js','ga'); + + ga('create', '{{ site.gacode }}', 'auto'); + ga('send', 'pageview'); +</script> http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/head.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/head.html b/thirdparty/rocksdb/docs/_includes/head.html new file mode 100644 index 0000000..10845ec --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/head.html @@ -0,0 +1,23 @@ +<head> + <meta charset="utf-8"> + <meta http-equiv="X-UA-Compatible" content="IE=edge"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + + <meta property="og:url" content="{{ page.url | replace:'index.html','' | absolute_url }}" /> + <meta property="og:site_name" content="{{ site.title }}"/> + <meta property="og:title" content="{% if page.title %}{{ page.title }}{% else %}{{ site.title }}{% endif %}" /> + <meta property="og:image" content="{{ '/static/og_image.png' | absolute_url }}" /> + <meta property="og:description" content="{% if page.excerpt %}{{ page.excerpt | strip_html | strip_newlines | truncate: 160 }}{% else %}{{ site.description }}{% endif %}" /> + + <link rel="stylesheet" href="{{ '/css/main.css' }}" media="screen"> + <link rel="icon" href="{{ '/static/favicon.png' }}" type="image/x-icon"> + {% if site.searchconfig %} + <link rel="stylesheet" href="https://cdn.jsdelivr.net/docsearch.js/1/docsearch.min.css"; /> + {% endif %} + + <title>{% if page.title %}{{ page.title }} | {{ site.title }}{% else %}{{ site.title }}{% endif %}</title> + <meta name="description" content="{% if page.excerpt %}{{ page.excerpt | strip_html | strip_newlines | truncate: 160 }}{% else %}{{ site.description }}{% endif %}"> + + <link rel="canonical" href="{{ page.url | replace:'index.html','' | absolute_url }}"> + <link rel="alternate" type="application/rss+xml" title="{{ site.title }}" href="{{ '/feed.xml' | absolute_url }}" /> +</head> http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/header.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/header.html b/thirdparty/rocksdb/docs/_includes/header.html new file mode 100644 index 0000000..8108d22 --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/header.html @@ -0,0 +1,19 @@ +<div class="headerContainer"> + <div id="header_wrap" class="wrapper headerWrapper"> + <div class="inner"> + <img class="projectLogo" height="200px" src="{{ '/static/logo.svg' }}" alt="{{ site.title }}" title="{{ site.title }}" /> + <h1 id="project_title">{{ site.title }}</h1> + <h2 id="project_tagline" class="fbossFontLight">{{ site.tagline }}</h2> + + <section id="intro"> + <p>{% if page.excerpt %}{{ page.excerpt | strip_html }}{% else %}{{ site.description }}{% endif %}</p> + </section> + <div id="promo" class="section promoSection"> + {% for promo in site.data.promo %} + {% include plugins/{{promo.type}}.html button_href=promo.href button_text=promo.text %} + <div class="gridClear"></div> + {% endfor %} + </div> + </div> + </div> +</div> http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/hero.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/hero.html b/thirdparty/rocksdb/docs/_includes/hero.html new file mode 100644 index 0000000..e69de29 http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/home_header.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/home_header.html b/thirdparty/rocksdb/docs/_includes/home_header.html new file mode 100644 index 0000000..90880d1 --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/home_header.html @@ -0,0 +1,22 @@ +<div class="homeContainer"> + <div class="homeSplashFade"> + <div id="home_wrap" class="wrapper homeWrapper"> + <div id="inner"> + <h2 id="project_tagline">{{ site.tagline }}</h2> + <section id="intro"> + <p>{% if page.excerpt %}{{ page.excerpt | strip_html }}{% else %}{{ site.description }}{% endif %}</p> + </section> + <div id="promo" class="section promoSection"> + {% for promo in site.data.promo %} + <div class="promoRow"> + {% include plugins/{{promo.type}}.html href=promo.href text=promo.text children=promo.children %} + </div> + {% endfor %} + </div> + </div> + <div class="projectLogo"> + <img src="{{ '/static/logo.svg' }}" alt="{{ site.title }}"> + </div> + </div> + </div> +</div> http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/katex_import.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/katex_import.html b/thirdparty/rocksdb/docs/_includes/katex_import.html new file mode 100644 index 0000000..6d6b7cf --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/katex_import.html @@ -0,0 +1,3 @@ +<script src="//code.jquery.com/jquery-1.11.1.min.js"></script> +<link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/KaTeX/0.2.0/katex.min.css"> +<script src="//cdnjs.cloudflare.com/ajax/libs/KaTeX/0.2.0/katex.min.js"></script> http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/katex_render.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/katex_render.html b/thirdparty/rocksdb/docs/_includes/katex_render.html new file mode 100644 index 0000000..56e2e89 --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/katex_render.html @@ -0,0 +1,210 @@ +<script type="text/javascript"> +/* global katex */ + +var findEndOfMath = function(delimiter, text, startIndex) { + // Adapted from + // https://github.com/Khan/perseus/blob/master/src/perseus-markdown.jsx + var index = startIndex; + var braceLevel = 0; + + var delimLength = delimiter.length; + + while (index < text.length) { + var character = text[index]; + + if (braceLevel <= 0 && + text.slice(index, index + delimLength) === delimiter) { + return index; + } else if (character === "\\") { + index++; + } else if (character === "{") { + braceLevel++; + } else if (character === "}") { + braceLevel--; + } + + index++; + } + + return -1; +}; + +var splitAtDelimiters = function(startData, leftDelim, rightDelim, display) { + var finalData = []; + + for (var i = 0; i < startData.length; i++) { + if (startData[i].type === "text") { + var text = startData[i].data; + + var lookingForLeft = true; + var currIndex = 0; + var nextIndex; + + nextIndex = text.indexOf(leftDelim); + if (nextIndex !== -1) { + currIndex = nextIndex; + finalData.push({ + type: "text", + data: text.slice(0, currIndex) + }); + lookingForLeft = false; + } + + while (true) { + if (lookingForLeft) { + nextIndex = text.indexOf(leftDelim, currIndex); + if (nextIndex === -1) { + break; + } + + finalData.push({ + type: "text", + data: text.slice(currIndex, nextIndex) + }); + + currIndex = nextIndex; + } else { + nextIndex = findEndOfMath( + rightDelim, + text, + currIndex + leftDelim.length); + if (nextIndex === -1) { + break; + } + + finalData.push({ + type: "math", + data: text.slice( + currIndex + leftDelim.length, + nextIndex), + rawData: text.slice( + currIndex, + nextIndex + rightDelim.length), + display: display + }); + + currIndex = nextIndex + rightDelim.length; + } + + lookingForLeft = !lookingForLeft; + } + + finalData.push({ + type: "text", + data: text.slice(currIndex) + }); + } else { + finalData.push(startData[i]); + } + } + + return finalData; +}; + +var splitWithDelimiters = function(text, delimiters) { + var data = [{type: "text", data: text}]; + for (var i = 0; i < delimiters.length; i++) { + var delimiter = delimiters[i]; + data = splitAtDelimiters( + data, delimiter.left, delimiter.right, + delimiter.display || false); + } + return data; +}; + +var renderMathInText = function(text, delimiters) { + var data = splitWithDelimiters(text, delimiters); + + var fragment = document.createDocumentFragment(); + + for (var i = 0; i < data.length; i++) { + if (data[i].type === "text") { + fragment.appendChild(document.createTextNode(data[i].data)); + } else { + var span = document.createElement("span"); + var math = data[i].data; + try { + katex.render(math, span, { + displayMode: data[i].display + }); + } catch (e) { + if (!(e instanceof katex.ParseError)) { + throw e; + } + console.error( + "KaTeX auto-render: Failed to parse `" + data[i].data + + "` with ", + e + ); + fragment.appendChild(document.createTextNode(data[i].rawData)); + continue; + } + fragment.appendChild(span); + } + } + + return fragment; +}; + +var renderElem = function(elem, delimiters, ignoredTags) { + for (var i = 0; i < elem.childNodes.length; i++) { + var childNode = elem.childNodes[i]; + if (childNode.nodeType === 3) { + // Text node + var frag = renderMathInText(childNode.textContent, delimiters); + i += frag.childNodes.length - 1; + elem.replaceChild(frag, childNode); + } else if (childNode.nodeType === 1) { + // Element node + var shouldRender = ignoredTags.indexOf( + childNode.nodeName.toLowerCase()) === -1; + + if (shouldRender) { + renderElem(childNode, delimiters, ignoredTags); + } + } + // Otherwise, it's something else, and ignore it. + } +}; + +var defaultOptions = { + delimiters: [ + {left: "$$", right: "$$", display: true}, + {left: "\\[", right: "\\]", display: true}, + {left: "\\(", right: "\\)", display: false} + // LaTeX uses this, but it ruins the display of normal `$` in text: + // {left: "$", right: "$", display: false} + ], + + ignoredTags: [ + "script", "noscript", "style", "textarea", "pre", "code" + ] +}; + +var extend = function(obj) { + // Adapted from underscore.js' `_.extend`. See LICENSE.txt for license. + var source, prop; + for (var i = 1, length = arguments.length; i < length; i++) { + source = arguments[i]; + for (prop in source) { + if (Object.prototype.hasOwnProperty.call(source, prop)) { + obj[prop] = source[prop]; + } + } + } + return obj; +}; + +var renderMathInElement = function(elem, options) { + if (!elem) { + throw new Error("No element provided to render"); + } + + options = extend({}, defaultOptions, options); + + renderElem(elem, options.delimiters, options.ignoredTags); +}; + +renderMathInElement(document.body); + +</script> http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/nav.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/nav.html b/thirdparty/rocksdb/docs/_includes/nav.html new file mode 100644 index 0000000..9c6fed0 --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/nav.html @@ -0,0 +1,37 @@ +<div id="fixed_header" class="fixedHeaderContainer{% if include.alwayson %} visible{% endif %}"> + <div class="headerWrapper wrapper"> + <header> + <a href="{{ '/' | absolute_url }}"> + <img src="{{ '/static/logo.svg' }}"> + <h2>{{ site.title }}</h2> + </a> + + <div class="navigationWrapper navigationFull" id="flat_nav"> + <nav class="navigation"> + <ul> + {% for item in site.data.nav %} + <li class="navItem{% if page.collection == item.category or page.category == item.category %} navItemActive{% endif %}"> + {% if item.category == "external" %} + <a href="{{ item.href }}">{{ item.title }}</a> + {% else %} + {% comment %} + I removed `relative_url` from here for now until the problem we are having with + GitHub pages is resolved. Yes, I know this is exactly the same as the if above. + See: https://github.com/facebook/rocksdb/commit/800e51553ee029f29581f7f338cbc988c7f6da62 + {% endcomment %} + <a href="{{ item.href }}">{{ item.title }}</a> + {% endif %} + </li> + {% endfor %} + {% if site.searchconfig %} + {% include nav_search.html inputselector="search_input" %} + {% endif %} + </ul> + </nav> + </div> + <div class="navigationWrapper navigationSlider" id="navigation_wrap"> + {% include nav/header_nav.html %} + </div> + </header> + </div> +</div> http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/nav/collection_nav.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/nav/collection_nav.html b/thirdparty/rocksdb/docs/_includes/nav/collection_nav.html new file mode 100644 index 0000000..a3c7a2d --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/nav/collection_nav.html @@ -0,0 +1,64 @@ +<div class="docsNavContainer"> + <nav class="toc" id="doc_nav"> + <div class="toggleNav" id="collection_nav"> + <section class="navWrapper wrapper"> + <div class="navBreadcrumb wrapper"> + <div class="navToggle" id="collection_nav_toggler"> + <i></i> + </div> + <h2> + <a href="{{ include.sectionpath }}">{{ include.sectiontitle }}</a> + {% if include.currentgroup %} + <i>âº</i> + <span>{{ include.currentgroup }}</span> + {% endif %} + </h2> + </div> + <div class="navGroups"> + {% if include.type == "blog" %} + {% assign grouptitle = "All Posts" %} + {% assign groupitems = include.navdata %} + {% include nav/collection_nav_group.html %} + {% else %} + {% for group in include.navdata %} + {% assign grouptitle = group.title %} + {% for item in group.items %} + {% if item.id == page.docid %} + {% assign currentgroup = group %} + {% endif %} + {% endfor %} + {% include nav/collection_nav_group.html %} + {% endfor %} + {% endif %} + </div> + </section> + </div> + </nav> +</div> +<script> + var docsevent = document.createEvent('Event'); + docsevent.initEvent('docs_slide', true, true); + document.addEventListener('docs_slide', function (e) { + document.body.classList.toggle('docsSliderActive'); + }, false); + + var collectionNav = document.getElementById('collection_nav'); + var collectionNavToggler = + document.getElementById('collection_nav_toggler'); + collectionNavToggler.addEventListener('click', function(e) { + collectionNav.classList.toggle('toggleNavActive'); + document.dispatchEvent(docsevent); + }); + + var groups = document.getElementsByClassName('navGroup'); + for(var i = 0; i < groups.length; i++) { + var thisGroup = groups[i]; + thisGroup.onclick = function() { + for(var j = 0; j < groups.length; j++) { + var group = groups[j]; + group.classList.remove('navGroupActive'); + } + this.classList.add('navGroupActive'); + } + } +</script> http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/nav/collection_nav_group.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/nav/collection_nav_group.html b/thirdparty/rocksdb/docs/_includes/nav/collection_nav_group.html new file mode 100644 index 0000000..b236ac5 --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/nav/collection_nav_group.html @@ -0,0 +1,19 @@ +<div class="navGroup{% if currentgroup == group %} navGroupActive navGroupCurrent{% endif %}"> + <h3><i>+</i><span>{{ grouptitle }}</span></h3> + <ul> + {% if include.data_collection %} + {% for item in group.items %} + {% for collectionitem in include.data_collection %} + {% if collectionitem.docid == item.id %} + {% assign groupitem = collectionitem %} + {% include nav/collection_nav_group_item.html %} + {% endif %} + {% endfor %} + {% endfor %} + {% else %} + {% for groupitem in groupitems %} + {% include nav/collection_nav_group_item.html %} + {% endfor %} + {% endif %} + </ul> +</div> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/nav/collection_nav_group_item.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/nav/collection_nav_group_item.html b/thirdparty/rocksdb/docs/_includes/nav/collection_nav_group_item.html new file mode 100644 index 0000000..fbb063d --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/nav/collection_nav_group_item.html @@ -0,0 +1 @@ +<li class="navListItem"><a class="navItem" href="{{ groupitem.url | absolute_url }}">{{ groupitem.title }}</a></li> http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/nav/header_nav.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/nav/header_nav.html b/thirdparty/rocksdb/docs/_includes/nav/header_nav.html new file mode 100644 index 0000000..0fe945c --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/nav/header_nav.html @@ -0,0 +1,30 @@ +<div id="header_nav"> + <div class="navSlideout"> + <i class="menuExpand" id="header_nav_expander"><span></span><span></span><span></span></i> + </div> + <nav class="slidingNav"> + <ul> + {% for item in site.data.nav %} + <li class="navItem"> + <a href="{{ item.href }}"{% if item.category == "external" %} target="_blank"{% endif %}>{{ item.title }}</a> + </li> + {% endfor %} + {% if site.searchconfig %} + {% include nav_search.html inputselector="search_input_react" %} + {% endif %} + </ul> + </nav> +</div> +<script> + var event = document.createEvent('Event'); + event.initEvent('slide', true, true); + document.addEventListener('slide', function (e) { + document.body.classList.toggle('sliderActive'); + }, false); + var headerNav = document.getElementById('header_nav'); + var headerNavExpander = document.getElementById('header_nav_expander'); + headerNavExpander.addEventListener('click', function(e) { + headerNav.classList.toggle('navSlideoutActive'); + document.dispatchEvent(event); + }, false); +</script> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/nav_search.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/nav_search.html b/thirdparty/rocksdb/docs/_includes/nav_search.html new file mode 100644 index 0000000..84956b9 --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/nav_search.html @@ -0,0 +1,15 @@ +<li class="navSearchWrapper"> + <input id="{{ include.inputselector }}" type="search" /> +</li> +<script type="text/javascript" src="https://cdn.jsdelivr.net/docsearch.js/1/docsearch.min.js";></script> +<script> +// For Algolia search +(function() { + // Algolia + docsearch({ + apiKey: '{{ site.searchconfig.apikey }}', + indexName: '{{ site.searchconfig.indexname }}', + inputSelector: '#{{ include.inputselector }}', + }); +}()); +</script> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/plugins/all_share.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/plugins/all_share.html b/thirdparty/rocksdb/docs/_includes/plugins/all_share.html new file mode 100644 index 0000000..59b00d6 --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/plugins/all_share.html @@ -0,0 +1,3 @@ +<div class="pluginBlock allShareBlock"> + {% include plugins/like_button.html %}{% include plugins/twitter_share.html %}{% include plugins/google_share.html %} +</div> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/plugins/ascii_cinema.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/plugins/ascii_cinema.html b/thirdparty/rocksdb/docs/_includes/plugins/ascii_cinema.html new file mode 100644 index 0000000..7d3f971 --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/plugins/ascii_cinema.html @@ -0,0 +1,2 @@ +<div class="ascii-cinema pluginBlock"></div> +<script type="text/javascript" src="https://asciinema.org/a/{{ include.href }}.js" id="asciicast-{{ include.href }}" async data-autoplay="true" data-loop="true" data-speed="2" data-t="23"></script> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/plugins/button.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/plugins/button.html b/thirdparty/rocksdb/docs/_includes/plugins/button.html new file mode 100644 index 0000000..9e499fe --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/plugins/button.html @@ -0,0 +1,6 @@ +<div class="pluginWrapper buttonWrapper"> + <a + class="button" + href="{{ include.href }}" + >{{ include.text }}</a> +</div> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/plugins/github_star.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/plugins/github_star.html b/thirdparty/rocksdb/docs/_includes/plugins/github_star.html new file mode 100644 index 0000000..6aea70f --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/plugins/github_star.html @@ -0,0 +1,4 @@ +<div class="pluginWrapper ghStarWrapper"> + <a aria-label="Star {{ site.ghrepo }} on GitHub" data-count-aria-label="# stargazers on GitHub" data-count-api="/repos/{{ site.ghrepo }}#stargazers_count" data-count-href="/{{ site.ghrepo }}/stargazers" data-style="mega" data-icon="octicon-star" href="https://github.com/{{ site.ghrepo }}" class="github-button">Star</a> +</div> +<script async defer id="github-bjs" src="https://buttons.github.io/buttons.js";></script> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/plugins/github_watch.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/plugins/github_watch.html b/thirdparty/rocksdb/docs/_includes/plugins/github_watch.html new file mode 100644 index 0000000..64233b5 --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/plugins/github_watch.html @@ -0,0 +1,4 @@ +<div class="pluginWrapper ghWatchWrapper"> + <a aria-label="Watch {{ site.ghrepo }} on GitHub" data-count-aria-label="# watchers on GitHub" data-count-api="/repos/{{ site.ghrepo }}#subscribers_count" data-count-href="/{{ site.ghrepo }}/watchers" data-style="mega" data-icon="octicon-eye" href="https://github.com/{{ site.ghrepo }}" class="github-button">Watch</a> +</div> +<script async defer id="github-bjs" src="https://buttons.github.io/buttons.js";></script> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/plugins/google_share.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/plugins/google_share.html b/thirdparty/rocksdb/docs/_includes/plugins/google_share.html new file mode 100644 index 0000000..1b557db --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/plugins/google_share.html @@ -0,0 +1,5 @@ +<div class="pluginBlock"> + <div class="g-plusone" data-size="medium"></div> +</div> + +<script src="https://apis.google.com/js/platform.js"; async defer></script> http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/plugins/iframe.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/plugins/iframe.html b/thirdparty/rocksdb/docs/_includes/plugins/iframe.html new file mode 100644 index 0000000..525b59f --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/plugins/iframe.html @@ -0,0 +1,6 @@ +<div class="iframeContent"> + <iframe class="pluginIframe" src="{{ include.href }}" seamless></iframe> +</div> +<div class="iframePreview"> + {% include plugins/button.html href=include.href text=include.text %} +</div> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/plugins/like_button.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/plugins/like_button.html b/thirdparty/rocksdb/docs/_includes/plugins/like_button.html new file mode 100644 index 0000000..bcb8a7b --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/plugins/like_button.html @@ -0,0 +1,18 @@ +<div class="fb-like pluginWrapper likeButtonWrapper" data-layout="button_count" data-action="like" data-show-faces="true" data-share="true"></div> +<script> + window.fbAsyncInit = function() { + FB.init({ + appId : '{{ site.fbappid }}', + xfbml : true, + version : 'v2.3' + }); + }; + + (function(d, s, id){ + var js, fjs = d.getElementsByTagName(s)[0]; + if (d.getElementById(id)) {return;} + js = d.createElement(s); js.id = id; + js.src = "//connect.facebook.net/en_US/sdk.js"; + fjs.parentNode.insertBefore(js, fjs); + }(document, 'script', 'facebook-jssdk')); +</script> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/plugins/plugin_row.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/plugins/plugin_row.html b/thirdparty/rocksdb/docs/_includes/plugins/plugin_row.html new file mode 100644 index 0000000..800f50b --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/plugins/plugin_row.html @@ -0,0 +1,5 @@ +<div class="pluginRowBlock"> +{% for child in include.children %} + {% include plugins/{{child.type}}.html href=child.href text=child.text %} +{% endfor %} +</div> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/plugins/post_social_plugins.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/plugins/post_social_plugins.html b/thirdparty/rocksdb/docs/_includes/plugins/post_social_plugins.html new file mode 100644 index 0000000..cb4d6e2 --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/plugins/post_social_plugins.html @@ -0,0 +1,34 @@ +<div class="postSocialPlugins"> + <a + href="https://twitter.com/share"; + class="twitter-share-button" + data-url="{{ page.url | replace:'index.html','' | absolute_url }}" + data-text="{% if page.title %}{{ page.title }}{% else %}{{ site.title }}{% endif %}" + data-hashtags="flowtype">Tweet</a> + <div + class="fb-like" + data-href="{{ page.url | replace:'index.html','' | absolute_url }}" + data-layout="button_count" + data-action="like" + data-show-faces="false" + data-share="true"></div> +</div> +<script> + window.fbAsyncInit = function() { + FB.init({ + appId : '{{ site.fbappid }}', + xfbml : true, + version : 'v2.2' + }); + }; + + (function(d, s, id){ + var js, fjs = d.getElementsByTagName(s)[0]; + if (d.getElementById(id)) {return;} + js = d.createElement(s); js.id = id; + js.src = "//connect.facebook.net/en_US/sdk.js"; + fjs.parentNode.insertBefore(js, fjs); + }(document, 'script', 'facebook-jssdk')); +</script> + +<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script> http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/plugins/slideshow.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/plugins/slideshow.html b/thirdparty/rocksdb/docs/_includes/plugins/slideshow.html new file mode 100644 index 0000000..69fa2b3 --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/plugins/slideshow.html @@ -0,0 +1,88 @@ +<div class="slideshowBlock pluginWrapper" id="slideshow"></div> +<script> + var slideshowData = [ + {% for image in site.data.slideshow %} + { + id : "{{ image.id }}", + imagesrc : "{{ image.src }}", + tooltip : "{{ image.tooltip }}", + href : "{{ image.link }}", + }, + {% endfor %} + ]; +</script> +<script src="http://fb.me/react-with-addons-0.13.1.min.js";></script> +<script type="text/javascript"> + var Slideshow = React.createClass({displayName: "Slideshow", + getInitialState: function() { + return { + currentSlide: 0, + }; + }, + getDefaultProps: function() { + return { + data: slideshowData, + }; + }, + handleSelect: function(id) { + var index = this.props.data.map(function (el, elIndex) { + return ( + elIndex + ); + }); + var currentIndex = index.indexOf(id); + this.setState({ + currentSlide: currentIndex, + }); + }, + render: function() { + return ( + React.createElement("div", {className: "slideshow"}, + React.createElement("div", {className: "slides"}, + this.props.data.map(this.renderSlide) + ), + React.createElement("div", {className: "pagination"}, + this.props.data.map(this.renderPager) + ) + ) + ); + }, + renderSlide: function(child, index) { + var classes = React.addons.classSet({ + 'slide': true, + 'slideActive': this.state.currentSlide === index, + }); + if (child.href) { + return ( + React.createElement("div", {key: index, className: classes}, + React.createElement("a", {href: child.href, alt: child.tooltip, title: child.tooltip}, + React.createElement("img", {src: child.imagesrc, alt: child.tooltip, title: child.tooltip}) + ) + ) + ); + } + return ( + React.createElement("div", {key: index, className: classes}, + React.createElement("img", {src: child.imagesrc, alt: child.tooltip}) + ) + ); + }, + renderPager: function(child, index) { + var classes = React.addons.classSet({ + 'pager': true, + 'pagerActive': this.state.currentSlide === index, + }); + return ( + React.createElement("span", {key: index, className: classes, onClick: this.handleSelect.bind(this, index)}) + ); + }, + }); + + function render(slideshowData) { + React.render( + React.createElement(Slideshow, {data: slideshowData}), + document.getElementById('slideshow') + ); + } + render(slideshowData); +</script> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/plugins/twitter_follow.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/plugins/twitter_follow.html b/thirdparty/rocksdb/docs/_includes/plugins/twitter_follow.html new file mode 100644 index 0000000..9443db3 --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/plugins/twitter_follow.html @@ -0,0 +1,5 @@ +<div class="pluginBlock"> + <a href="https://twitter.com/{{ include.href }}" class="twitter-follow-button pluginBlock" data-show-count="false">Follow @{{ include.href }}</a> +</div> + +<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/plugins/twitter_share.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/plugins/twitter_share.html b/thirdparty/rocksdb/docs/_includes/plugins/twitter_share.html new file mode 100644 index 0000000..67f1d62 --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/plugins/twitter_share.html @@ -0,0 +1,4 @@ +<div class="pluginWrapper twitterSharePlugin"> + <a href="https://twitter.com/share"; class="twitter-share-button" data-hashtags="{{ site.title| replace: ' ', '' }}">Tweet</a> +</div> +<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/post.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/post.html b/thirdparty/rocksdb/docs/_includes/post.html new file mode 100644 index 0000000..4c4b5b1 --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/post.html @@ -0,0 +1,35 @@ +<div class="post"> + {% assign author = site.data.authors[page.author] %} + <header class="post-header"> + {% if author.fbid %} + <div class="authorPhoto"> + <img src="http://graph.facebook.com/{{ author.fbid }}/picture/" alt="{{ author.fullname }}" title="{{ author.fullname }}" /> + </div> + {% endif %} + {% if author.full_name %} + <p class="post-authorName">{{ author.full_name }}</p> + {% endif %} + <h1 class="post-title">{% if include.truncate %}<a href="{{ page.url | absolute_url }}">{{ page.title }}</a>{% else %}{{ page.title }}{% endif %}</h1> + <p class="post-meta">Posted {{ page.date | date: '%B %d, %Y' }}{% if page.meta %} ⢠{{ page.meta }}{% endif %}</p> + </header> + + <article class="post-content"> + {% if include.truncate %} + {% if page.content contains '<!--truncate-->' %} + {{ page.content | split:'<!--truncate-->' | first | markdownify }} + <div class="read-more"> + <a href="{{ page.url | absolute_url }}" > + Read More + </a> + </div> + {% else %} + {{ page.content | markdownify }} + {% endif %} + {% else %} + {{ content }} + {% endif %} + {% unless include.truncate %} + {% include plugins/like_button.html %} + {% endunless %} + </article> +</div> http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/powered_by.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/powered_by.html b/thirdparty/rocksdb/docs/_includes/powered_by.html new file mode 100644 index 0000000..c629429 --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/powered_by.html @@ -0,0 +1,28 @@ +{% if site.data.powered_by.first.items or site.data.powered_by_highlight.first.items %} +<div class="poweredByContainer"> + <div class="wrapper mainWrapper poweredByWrapper"> + {% if site.data.powered_by_highlight.first.title %} + <h2>{{ site.data.powered_by_highlight.first.title }}</h2> + {% else %} + <h2>{{ site.data.powered_by.first.title }}</h2> + {% endif %} + {% if site.data.powered_by_highlight.first.items %} + <div class="poweredByItems"> + {% for item in site.data.powered_by_highlight.first.items %} + <div class="poweredByItem itemLarge"> + <a href="{{ item.url }}" target="_blank"><img src="{{ item.img }}" alt="{{ item.name }}" /></a> + </div> + {% endfor %} + </div> + {% endif %} + <div class="poweredByItems"> + {% for item in site.data.powered_by.first.items %} + <div class="poweredByItem itemSmall"> + <a href="{{ item.url }}" target="_blank">{{ item.name }}</a> + </div> + {% endfor %} + </div> + <div class="poweredByMessage">Does your app use {{ site.title }}? Add it to this list with <a href="https://github.com/{{ site.ghrepo }}/edit/gh-pages/_data/powered_by.yml" target="_blank">a pull request!</a></div> + </div> +</div> +{% endif %} http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/social_plugins.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/social_plugins.html b/thirdparty/rocksdb/docs/_includes/social_plugins.html new file mode 100644 index 0000000..db4f1ae --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/social_plugins.html @@ -0,0 +1,24 @@ +<a + href="https://twitter.com/share"; + class="twitter-share-button" + data-url="http://facebook.github.io/fresco{{ page.url }}" + data-text="Fresco | {{ page.title }}" + data-hashtags="fresco">Tweet</a> +<div + class="fb-like" + data-href="http://facebook.github.io/fresco{{ page.url }}" + data-layout="standard" + data-action="like" + data-show-faces="true" + data-share="true"></div> + +<div id="fb-root"></div> +<script>(function(d, s, id) { + var js, fjs = d.getElementsByTagName(s)[0]; + if (d.getElementById(id)) return; + js = d.createElement(s); js.id = id; + js.src = "//connect.facebook.net/en_US/sdk.js#xfbml=1&version=v2.0"; + fjs.parentNode.insertBefore(js, fjs); +}(document, 'script', 'facebook-jssdk'));</script> + +<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script> http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_includes/ui/button.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_includes/ui/button.html b/thirdparty/rocksdb/docs/_includes/ui/button.html new file mode 100644 index 0000000..729ccc3 --- /dev/null +++ b/thirdparty/rocksdb/docs/_includes/ui/button.html @@ -0,0 +1 @@ +<span class="buttonWrap {{ include.align }}"><a class="button blockButton fbossFontLight pluginBlock margin{{ include.margin }}" target="{{ include.button_target }}" href="{{ include.button_href }}">{{ include.button_text }}</a></span> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_layouts/basic.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_layouts/basic.html b/thirdparty/rocksdb/docs/_layouts/basic.html new file mode 100644 index 0000000..65bd210 --- /dev/null +++ b/thirdparty/rocksdb/docs/_layouts/basic.html @@ -0,0 +1,12 @@ +--- +layout: doc_default +--- + +<div class="mainContainer blogContainer postContainer"> + <div id="main_wrap" class="wrapper mainWrapper"> + <div class="post basicPost"> + {{ content }} + </div> + </div> +</div> + http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_layouts/blog.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_layouts/blog.html b/thirdparty/rocksdb/docs/_layouts/blog.html new file mode 100644 index 0000000..1b0da41 --- /dev/null +++ b/thirdparty/rocksdb/docs/_layouts/blog.html @@ -0,0 +1,11 @@ +--- +category: blog +layout: blog_default +--- + +<div class="mainContainer blogContainer postContainer"> + <div id="main_wrap" class="wrapper mainWrapper"> + {{ content }} + </div> +</div> + http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_layouts/blog_default.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_layouts/blog_default.html b/thirdparty/rocksdb/docs/_layouts/blog_default.html new file mode 100644 index 0000000..a29d58d --- /dev/null +++ b/thirdparty/rocksdb/docs/_layouts/blog_default.html @@ -0,0 +1,14 @@ +<!DOCTYPE html> +<html> + {% include head.html %} + <body class="docsNavVisible"> + {% include nav.html alwayson=true %} + <div class="navPusher"> + <div class="docMainWrapper wrapper"> + {% include nav/collection_nav.html navdata=site.posts type="blog" sectionpath="/blog/" sectiontitle="Blog" %} + {{ content }} + </div> + {% include footer.html %} + </div> + </body> +</html> http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_layouts/default.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_layouts/default.html b/thirdparty/rocksdb/docs/_layouts/default.html new file mode 100644 index 0000000..0167d9f --- /dev/null +++ b/thirdparty/rocksdb/docs/_layouts/default.html @@ -0,0 +1,12 @@ +<!DOCTYPE html> +<html> + {% include head.html %} + <body> + {% include nav.html alwayson=true %} + <div class="navPusher"> + {{ content }} + {% include footer.html %} + </div> + </body> + +</html> http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_layouts/doc_default.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_layouts/doc_default.html b/thirdparty/rocksdb/docs/_layouts/doc_default.html new file mode 100644 index 0000000..4a41392 --- /dev/null +++ b/thirdparty/rocksdb/docs/_layouts/doc_default.html @@ -0,0 +1,14 @@ +<!DOCTYPE html> +<html> + {% include head.html %} + <body class="docsNavVisible"> + {% include nav.html alwayson=true %} + <div class="navPusher"> + <div class="docMainWrapper wrapper"> + {% include nav/collection_nav.html navdata=site.data.nav_docs type="docs" sectionpath="/docs/" sectiontitle="Docs" data_collection=site.docs %} + {{ content }} + </div> + {% include footer.html %} + </div> + </body> +</html> http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_layouts/doc_page.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_layouts/doc_page.html b/thirdparty/rocksdb/docs/_layouts/doc_page.html new file mode 100644 index 0000000..dba761e --- /dev/null +++ b/thirdparty/rocksdb/docs/_layouts/doc_page.html @@ -0,0 +1,10 @@ +--- +layout: doc_default +--- + +<div class="mainContainer documentContainer postContainer"> + <div id="main_wrap" class="wrapper mainWrapper"> + {{ content }} + </div> +</div> + http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_layouts/docs.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_layouts/docs.html b/thirdparty/rocksdb/docs/_layouts/docs.html new file mode 100644 index 0000000..749dafa --- /dev/null +++ b/thirdparty/rocksdb/docs/_layouts/docs.html @@ -0,0 +1,5 @@ +--- +layout: doc_page +--- + +{% include doc.html %} \ No newline at end of file http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_layouts/home.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_layouts/home.html b/thirdparty/rocksdb/docs/_layouts/home.html new file mode 100644 index 0000000..e3c320f --- /dev/null +++ b/thirdparty/rocksdb/docs/_layouts/home.html @@ -0,0 +1,17 @@ +<!DOCTYPE html> +<html> + {% include head.html %} + <body> + {% include nav.html alwayson=true %} + <div class="navPusher"> + {% include home_header.html %} + <div class="mainContainer"> + <div id="main_wrap" class="wrapper mainWrapper"> + {{ content }} + </div> + {% include powered_by.html %} + </div> + {% include footer.html %} + </div> + </body> +</html> http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_layouts/page.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_layouts/page.html b/thirdparty/rocksdb/docs/_layouts/page.html new file mode 100644 index 0000000..bec3680 --- /dev/null +++ b/thirdparty/rocksdb/docs/_layouts/page.html @@ -0,0 +1,3 @@ +--- +layout: blog +--- http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_layouts/plain.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_layouts/plain.html b/thirdparty/rocksdb/docs/_layouts/plain.html new file mode 100644 index 0000000..fccc02c --- /dev/null +++ b/thirdparty/rocksdb/docs/_layouts/plain.html @@ -0,0 +1,10 @@ +--- +layout: default +--- + +<div class="mainContainer blogContainer postContainer"> + <div id="main_wrap" class="wrapper mainWrapper"> + {{ content }} + </div> +</div> + http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_layouts/post.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_layouts/post.html b/thirdparty/rocksdb/docs/_layouts/post.html new file mode 100644 index 0000000..4c92cf2 --- /dev/null +++ b/thirdparty/rocksdb/docs/_layouts/post.html @@ -0,0 +1,8 @@ +--- +collection: blog +layout: blog +--- + +<div class="lonePost"> +{% include post.html %} +</div> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_layouts/redirect.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_layouts/redirect.html b/thirdparty/rocksdb/docs/_layouts/redirect.html new file mode 100644 index 0000000..c24f817 --- /dev/null +++ b/thirdparty/rocksdb/docs/_layouts/redirect.html @@ -0,0 +1,6 @@ +<html> +<head> + <meta http-equiv="refresh" content="0; {{ page.destination }}"> +</head> +<body></body> +</html> http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_layouts/top-level.html ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_layouts/top-level.html b/thirdparty/rocksdb/docs/_layouts/top-level.html new file mode 100644 index 0000000..fccc02c --- /dev/null +++ b/thirdparty/rocksdb/docs/_layouts/top-level.html @@ -0,0 +1,10 @@ +--- +layout: default +--- + +<div class="mainContainer blogContainer postContainer"> + <div id="main_wrap" class="wrapper mainWrapper"> + {{ content }} + </div> +</div> + http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_posts/2014-03-27-how-to-backup-rocksdb.markdown ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_posts/2014-03-27-how-to-backup-rocksdb.markdown b/thirdparty/rocksdb/docs/_posts/2014-03-27-how-to-backup-rocksdb.markdown new file mode 100644 index 0000000..f9e4a54 --- /dev/null +++ b/thirdparty/rocksdb/docs/_posts/2014-03-27-how-to-backup-rocksdb.markdown @@ -0,0 +1,135 @@ +--- +title: How to backup RocksDB? +layout: post +author: icanadi +category: blog +redirect_from: + - /blog/191/how-to-backup-rocksdb/ +--- + +In RocksDB, we have implemented an easy way to backup your DB. Here is a simple example: + + + + #include "rocksdb/db.h" + #include "utilities/backupable_db.h" + using namespace rocksdb; + + DB* db; + DB::Open(Options(), "/tmp/rocksdb", &db); + BackupableDB* backupable_db = new BackupableDB(db, BackupableDBOptions("/tmp/rocksdb_backup")); + backupable_db->Put(...); // do your thing + backupable_db->CreateNewBackup(); + delete backupable_db; // no need to also delete db + +<!--truncate--> + + +This simple example will create a backup of your DB in "/tmp/rocksdb_backup". Creating new BackupableDB consumes DB* and you should be calling all the DB methods on object `backupable_db` going forward. + +Restoring is also easy: + + + + RestoreBackupableDB* restore = new RestoreBackupableDB(Env::Default(), BackupableDBOptions("/tmp/rocksdb_backup")); + restore->RestoreDBFromLatestBackup("/tmp/rocksdb", "/tmp/rocksdb"); + delete restore; + + + + +This code will restore the backup back to "/tmp/rocksdb". The second parameter is the location of log files (In some DBs they are different from DB directory, but usually they are the same. See Options::wal_dir for more info). + +An alternative API for backups is to use BackupEngine directly: + + + + #include "rocksdb/db.h" + #include "utilities/backupable_db.h" + using namespace rocksdb; + + DB* db; + DB::Open(Options(), "/tmp/rocksdb", &db); + db->Put(...); // do your thing + BackupEngine* backup_engine = BackupEngine::NewBackupEngine(Env::Default(), BackupableDBOptions("/tmp/rocksdb_backup")); + backup_engine->CreateNewBackup(db); + delete db; + delete backup_engine; + + + + +Restoring with BackupEngine is similar to RestoreBackupableDB: + + + + BackupEngine* backup_engine = BackupEngine::NewBackupEngine(Env::Default(), BackupableDBOptions("/tmp/rocksdb_backup")); + backup_engine->RestoreDBFromLatestBackup("/tmp/rocksdb", "/tmp/rocksdb"); + delete backup_engine; + + + + +Backups are incremental. You can create a new backup with `CreateNewBackup()` and only the new data will be copied to backup directory (for more details on what gets copied, see "Under the hood"). Checksum is always calculated for any backuped file (including sst, log, and etc). It is used to make sure files are kept sound in the file system. Checksum is also verified for files from the previous backups even though they do not need to be copied. A checksum mismatch aborts the current backup (see "Under the hood" for more details). Once you have more backups saved, you can issue `GetBackupInfo()` call to get a list of all backups together with information on timestamp of the backup and the size (please note that sum of all backups' sizes is bigger than the actual size of the backup directory because some data is shared by multiple backups). Backups are identified by their always-increasing IDs. `GetBackupInfo()` is available both in `BackupableDB` and `RestoreBackupableDB`. + +You probably want to keep around only small number of backups. To delete old backups, just call `PurgeOldBackups(N)`, where N is how many backups you'd like to keep. All backups except the N newest ones will be deleted. You can also choose to delete arbitrary backup with call `DeleteBackup(id)`. + +`RestoreDBFromLatestBackup()` will restore the DB from the latest consistent backup. An alternative is `RestoreDBFromBackup()` which takes a backup ID and restores that particular backup. Checksum is calculated for any restored file and compared against the one stored during the backup time. If a checksum mismatch is detected, the restore process is aborted and `Status::Corruption` is returned. Very important thing to note here: Let's say you have backups 1, 2, 3, 4. If you restore from backup 2 and start writing more data to your database, newly created backup will delete old backups 3 and 4 and create new backup 3 on top of 2. + + + +## Advanced usage + + +Let's say you want to backup your DB to HDFS. There is an option in `BackupableDBOptions` to set `backup_env`, which will be used for all file I/O related to backup dir (writes when backuping, reads when restoring). If you set it to HDFS Env, all the backups will be stored in HDFS. + +`BackupableDBOptions::info_log` is a Logger object that is used to print out LOG messages if not-nullptr. + +If `BackupableDBOptions::sync` is true, we will sync data to disk after every file write, guaranteeing that backups will be consistent after a reboot or if machine crashes. Setting it to false will speed things up a bit, but some (newer) backups might be inconsistent. In most cases, everything should be fine, though. + +If you set `BackupableDBOptions::destroy_old_data` to true, creating new `BackupableDB` will delete all the old backups in the backup directory. + +`BackupableDB::CreateNewBackup()` method takes a parameter `flush_before_backup`, which is false by default. When `flush_before_backup` is true, `BackupableDB` will first issue a memtable flush and only then copy the DB files to the backup directory. Doing so will prevent log files from being copied to the backup directory (since flush will delete them). If `flush_before_backup` is false, backup will not issue flush before starting the backup. In that case, the backup will also include log files corresponding to live memtables. Backup will be consistent with current state of the database regardless of `flush_before_backup` parameter. + + + +## Under the hood + + +`BackupableDB` implements `DB` interface and adds four methods to it: `CreateNewBackup()`, `GetBackupInfo()`, `PurgeOldBackups()`, `DeleteBackup()`. Any `DB` interface calls will get forwarded to underlying `DB` object. + +When you call `BackupableDB::CreateNewBackup()`, it does the following: + + + + + + 1. Disable file deletions + + + + 2. Get live files (this includes table files, current and manifest file). + + + + 3. Copy live files to the backup directory. Since table files are immutable and filenames unique, we don't copy a table file that is already present in the backup directory. For example, if there is a file `00050.sst` already backed up and `GetLiveFiles()` returns `00050.sst`, we will not copy that file to the backup directory. However, checksum is calculated for all files regardless if a file needs to be copied or not. If a file is already present, the calculated checksum is compared against previously calculated checksum to make sure nothing crazy happened between backups. If a mismatch is detected, backup is aborted and the system is restored back to the state before `BackupableDB::CreateNewBackup()` is called. One thing to note is that a backup abortion could mean a corruption from a file in backup directory or the corresponding live file in current DB. Both manifest and current files are copied, since they are not immutable. + + + + 4. If `flush_before_backup` was set to false, we also need to copy log files to the backup directory. We call `GetSortedWalFiles()` and copy all live files to the backup directory. + + + + 5. Enable file deletions + + + + +Backup IDs are always increasing and we have a file `LATEST_BACKUP` that contains the ID of the latest backup. If we crash in middle of backing up, on a restart we will detect that there are newer backup files than `LATEST_BACKUP` claims there are. In that case, we will delete any backup newer than `LATEST_BACKUP` and clean up all the files since some of the table files might be corrupted. Having corrupted table files in the backup directory is dangerous because of our deduplication strategy. + + + +## Further reading + + +For the API details, see `include/utilities/backupable_db.h`. For the implementation, see `utilities/backupable/backupable_db.cc`. http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_posts/2014-03-27-how-to-persist-in-memory-rocksdb-database.markdown ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_posts/2014-03-27-how-to-persist-in-memory-rocksdb-database.markdown b/thirdparty/rocksdb/docs/_posts/2014-03-27-how-to-persist-in-memory-rocksdb-database.markdown new file mode 100644 index 0000000..89ffb2d --- /dev/null +++ b/thirdparty/rocksdb/docs/_posts/2014-03-27-how-to-persist-in-memory-rocksdb-database.markdown @@ -0,0 +1,54 @@ +--- +title: How to persist in-memory RocksDB database? +layout: post +author: icanadi +category: blog +redirect_from: + - /blog/245/how-to-persist-in-memory-rocksdb-database/ +--- + +In recent months, we have focused on optimizing RocksDB for in-memory workloads. With growing RAM sizes and strict low-latency requirements, lots of applications decide to keep their entire data in memory. Running in-memory database with RocksDB is easy -- just mount your RocksDB directory on tmpfs or ramfs [1]. Even if the process crashes, RocksDB can recover all of your data from in-memory filesystem. However, what happens if the machine reboots? + +<!--truncate--> + +In this article we will explain how you can recover your in-memory RocksDB database even after a machine reboot. + +Every update to RocksDB is written to two places - one is an in-memory data structure called memtable and second is write-ahead log. Write-ahead log can be used to completely recover the data in memtable. By default, when we flush the memtable to table file, we also delete the current log, since we don't need it anymore for recovery (the data from the log is "persisted" in the table file -- we say that the log file is obsolete). However, if your table file is stored in in-memory file system, you may need the obsolete write-ahead log to recover the data after the machine reboots. Here's how you can do that. + +Options::wal_dir is the directory where RocksDB stores write-ahead log files. If you configure this directory to be on flash or disk, you will not lose current log file on machine reboot. +Options::WAL_ttl_seconds is the timeout when we delete the archived log files. If the timeout is non-zero, obsolete log files will be moved to `archive/` directory under Options::wal_dir. Those archived log files will only be deleted after the specified timeout. + +Let's assume Options::wal_dir is a directory on persistent storage and Options::WAL_ttl_seconds is set to one day. To fully recover the DB, we also need to backup the current snapshot of the database (containing table and metadata files) with a frequency of less than one day. RocksDB provides an utility that enables you to easily backup the snapshot of your database. You can learn more about it here: [How to backup RocksDB?](https://github.com/facebook/rocksdb/wiki/How-to-backup-RocksDB%3F) + +You should configure the backup process to avoid backing up log files, since they are already stored in persistent storage. To do that, set BackupableDBOptions::backup_log_files to false. + +Restore process by default cleans up entire DB and WAL directory. Since we didn't include log files in the backup, we need to make sure that restoring the database doesn't delete log files in WAL directory. When restoring, configure RestoreOptions::keep_log_file to true. That option will also move any archived log files back to WAL directory, enabling RocksDB to replay all archived log files and rebuild the in-memory database state. + +To reiterate, here's what you have to do: + + + + + * Set DB directory to tmpfs or ramfs mounted drive + + + + * Set Options::wal_log to a directory on persistent storage + + + + * Set Options::WAL_ttl_seconds to T seconds + + + + * Backup RocksDB every T/2 seconds, with BackupableDBOptions::backup_log_files = false + + + + * When you lose data, restore from backup with RestoreOptions::keep_log_file = true + + + + + +[1] You might also want to consider using [PlainTable format](https://github.com/facebook/rocksdb/wiki/PlainTable-Format) for table files http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_posts/2014-04-02-the-1st-rocksdb-local-meetup-held-on-march-27-2014.markdown ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_posts/2014-04-02-the-1st-rocksdb-local-meetup-held-on-march-27-2014.markdown b/thirdparty/rocksdb/docs/_posts/2014-04-02-the-1st-rocksdb-local-meetup-held-on-march-27-2014.markdown new file mode 100644 index 0000000..7ccbdba --- /dev/null +++ b/thirdparty/rocksdb/docs/_posts/2014-04-02-the-1st-rocksdb-local-meetup-held-on-march-27-2014.markdown @@ -0,0 +1,53 @@ +--- +title: The 1st RocksDB Local Meetup Held on March 27, 2014 +layout: post +author: xjin +category: blog +redirect_from: + - /blog/323/the-1st-rocksdb-local-meetup-held-on-march-27-2014/ +--- + +On Mar 27, 2014, RocksDB team @ Facebook held the 1st RocksDB local meetup in FB HQ (Menlo Park, California). We invited around 80 guests from 20+ local companies, including LinkedIn, Twitter, Dropbox, Square, Pinterest, MapR, Microsoft and IBM. Finally around 50 guests showed up, totaling around 60% show-up rate. + +<!--truncate--> + +[](/static/images/Resize-of-20140327_200754-300x225.jpg) + +RocksDB team @ Facebook gave four talks about the latest progress and experience on RocksDB: + + + + + * [Supporting a 1PB In-Memory Workload](https://github.com/facebook/rocksdb/raw/gh-pages/talks/2014-03-27-RocksDB-Meetup-Haobo-RocksDB-In-Memory.pdf) + + + + + * [Column Families in RocksDB](https://github.com/facebook/rocksdb/raw/gh-pages/talks/2014-03-27-RocksDB-Meetup-Igor-Column-Families.pdf) + + + + + * ["Lockless" Get() in RocksDB?](https://github.com/facebook/rocksdb/raw/gh-pages/talks/2014-03-27-RocksDB-Meetup-Lei-Lockless-Get.pdf) + + + + + * [Prefix Hashing in RocksDB](https://github.com/facebook/rocksdb/raw/gh-pages/talks/2014-03-27-RocksDB-Meetup-Siying-Prefix-Hash.pdf) + + +A very interesting question asked by a massive number of guests is: does RocksDB plan to provide replication functionality? Obviously, many applications need a resilient and distributed storage solution, not just single-node storage. We are considering how to approach this issue. + +When will be the next meetup? We haven't decided yet. We will see whether the community is interested in it and how it can help RocksDB grow. + +If you have any questions or feedback for the meetup or RocksDB, please let us know in [our Facebook group](https://www.facebook.com/groups/rocksdb.dev/). + +### Comments + +**[Rajiv]([email protected])** + +Have any of these talks been recorded and if so will they be published? + +**[Igor Canadi]([email protected])** + +Yes, I think we plan to publish them soon. http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_posts/2014-04-07-rocksdb-2-8-release.markdown ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_posts/2014-04-07-rocksdb-2-8-release.markdown b/thirdparty/rocksdb/docs/_posts/2014-04-07-rocksdb-2-8-release.markdown new file mode 100644 index 0000000..7be7842 --- /dev/null +++ b/thirdparty/rocksdb/docs/_posts/2014-04-07-rocksdb-2-8-release.markdown @@ -0,0 +1,40 @@ +--- +title: RocksDB 2.8 release +layout: post +author: icanadi +category: blog +redirect_from: + - /blog/371/rocksdb-2-8-release/ +--- + +Check out the new RocksDB 2.8 release on [Github](https://github.com/facebook/rocksdb/releases/tag/2.8.fb). + +RocksDB 2.8. is mostly focused on improving performance for in-memory workloads. We are seeing read QPS as high as 5M (we will write a separate blog post on this). + +<!--truncate--> + +Here is the summary of new features: + + * Added a new table format called PlainTable, which is optimized for RAM storage (ramfs or tmpfs). You can read more details about it on [our wiki](https://github.com/facebook/rocksdb/wiki/PlainTable-Format). + + + * New prefixed memtable format HashLinkedList, which is optimized for cases where there are only a few keys for each prefix. + + + * Merge operator supports a new function PartialMergeMulti() that allows users to do partial merges against multiple operands. This function enables big speedups for workloads that use merge operators. + + + * Added a V2 compaction filter interface. It buffers the kv-pairs sharing the same key prefix, process them in batches, and return the batched results back to DB. + + + * Geo-spatial support for locations and radial-search. + + + * Improved read performance using thread local cache for frequently accessed data. + + + * Stability improvements -- we're now ignoring partially written tailing record to MANIFEST or WAL files. + + + +We have also introduced small incompatible API changes (mostly for advanced users). You can see full release notes in our [HISTORY.my](https://github.com/facebook/rocksdb/blob/2.8.fb/HISTORY.md) file. http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/rocksdb/docs/_posts/2014-04-21-indexing-sst-files-for-better-lookup-performance.markdown ---------------------------------------------------------------------- diff --git a/thirdparty/rocksdb/docs/_posts/2014-04-21-indexing-sst-files-for-better-lookup-performance.markdown b/thirdparty/rocksdb/docs/_posts/2014-04-21-indexing-sst-files-for-better-lookup-performance.markdown new file mode 100644 index 0000000..368055d --- /dev/null +++ b/thirdparty/rocksdb/docs/_posts/2014-04-21-indexing-sst-files-for-better-lookup-performance.markdown @@ -0,0 +1,28 @@ +--- +title: Indexing SST Files for Better Lookup Performance +layout: post +author: leijin +category: blog +redirect_from: + - /blog/431/indexing-sst-files-for-better-lookup-performance/ +--- + +For a `Get()` request, RocksDB goes through mutable memtable, list of immutable memtables, and SST files to look up the target key. SST files are organized in levels. + +On level 0, files are sorted based on the time they are flushed. Their key range (as defined by FileMetaData.smallest and FileMetaData.largest) are mostly overlapped with each other. So it needs to look up every L0 file. + +<!--truncate--> + +Compaction is scheduled periodically to pick up files from an upper level and merges them with files from lower level. As a result, key/values are moved from L0 down the LSM tree gradually. Compaction sorts key/values and split them into files. From level 1 and below, SST files are sorted based on key. Their key range are mutually exclusive. Instead of scanning through each SST file and checking if a key falls into its range, RocksDB performs a binary search based on FileMetaData.largest to locate a candidate file that can potentially contain the target key. This reduces complexity from O(N) to O(log(N)). However, log(N) can still be large for bottom levels. For a fan-out ratio of 10, level 3 can have 1000 files. That requires 10 comparisons to locate a candidate file. This is a significant cost for an in-memory database when you can do [several million gets per second](https://github.com/facebook/rocksdb/wiki/RocksDB-In-Memory-Workload-Performance-Benchmarks). + +One observation to this problem is that: after the LSM tree is built, an SST file's position in its level is fixed. Furthermore, its order relative to files from the next level is also fixed. Based on this idea, we can perform [fractional cascading](http://en.wikipedia.org/wiki/Fractional_cascading) kind of optimization to narrow down the binary search range. Here is an example: + +[](/static/images/tree_example1.png) + +Level 1 has 2 files and level 2 has 8 files. Now, we want to look up key 80. A binary search based FileMetaData.largest tells you file 1 is the candidate. Then key 80 is compared with its FileMetaData.smallest and FileMetaData.largest to decide if it falls into the range. The comparison shows 80 is less than FileMetaData.smallest (100), so file 1 does not possibly contain key 80. We to proceed to check level 2. Usually, we need to do binary search among all 8 files on level 2. But since we already know target key 80 is less than 100 and only file 1 to file 3 can contain key less than 100, we can safely exclude other files from the search. As a result we cut down the search space from 8 files to 3 files. + +Let's look at another example. We want to get key 230. A binary search on level 1 locates to file 2 (this also implies key 230 is larger than file 1's FileMetaData.largest 200). A comparison with file 2's range shows the target key is smaller than file 2's FileMetaData.smallest 300. Even though, we couldn't find key on level 1, we have derived hints that target key is in range between 200 and 300. Any files on level 2 that cannot overlap with [200, 300] can be safely excluded. As a result, we only need to look at file 5 and file 6 on level 2. + +Inspired by this concept, we pre-build pointers at compaction time on level 1 files that point to a range of files on level 2. For example, file 1 on level 1 points to file 3 (on level 2) on the left and file 4 on the right. File 2 will point to level 2 files 6 and 7. At query time, these pointers are used to determine the actual binary search range based on comparison result. + +Our benchmark shows that this optimization improves lookup QPS by ~5% for similar setup mentioned [here](https://github.com/facebook/rocksdb/wiki/RocksDB-In-Memory-Workload-Performance-Benchmarks).
