http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/leveldb-1.18/doc/index.html ---------------------------------------------------------------------- diff --git a/thirdparty/leveldb-1.18/doc/index.html b/thirdparty/leveldb-1.18/doc/index.html deleted file mode 100755 index 3ed0ed9..0000000 --- a/thirdparty/leveldb-1.18/doc/index.html +++ /dev/null @@ -1,549 +0,0 @@ -<!DOCTYPE html> -<html> -<head> -<link rel="stylesheet" type="text/css" href="doc.css" /> -<title>Leveldb</title> -</head> - -<body> -<h1>Leveldb</h1> -<address>Jeff Dean, Sanjay Ghemawat</address> -<p> -The <code>leveldb</code> library provides a persistent key value store. Keys and -values are arbitrary byte arrays. The keys are ordered within the key -value store according to a user-specified comparator function. - -<p> -<h1>Opening A Database</h1> -<p> -A <code>leveldb</code> database has a name which corresponds to a file system -directory. All of the contents of database are stored in this -directory. The following example shows how to open a database, -creating it if necessary: -<p> -<pre> - #include <assert> - #include "leveldb/db.h" - - leveldb::DB* db; - leveldb::Options options; - options.create_if_missing = true; - leveldb::Status status = leveldb::DB::Open(options, "/tmp/testdb", &db); - assert(status.ok()); - ... -</pre> -If you want to raise an error if the database already exists, add -the following line before the <code>leveldb::DB::Open</code> call: -<pre> - options.error_if_exists = true; -</pre> -<h1>Status</h1> -<p> -You may have noticed the <code>leveldb::Status</code> type above. Values of this -type are returned by most functions in <code>leveldb</code> that may encounter an -error. You can check if such a result is ok, and also print an -associated error message: -<p> -<pre> - leveldb::Status s = ...; - if (!s.ok()) cerr << s.ToString() << endl; -</pre> -<h1>Closing A Database</h1> -<p> -When you are done with a database, just delete the database object. -Example: -<p> -<pre> - ... open the db as described above ... - ... do something with db ... - delete db; -</pre> -<h1>Reads And Writes</h1> -<p> -The database provides <code>Put</code>, <code>Delete</code>, and <code>Get</code> methods to -modify/query the database. For example, the following code -moves the value stored under key1 to key2. -<pre> - std::string value; - leveldb::Status s = db->Get(leveldb::ReadOptions(), key1, &value); - if (s.ok()) s = db->Put(leveldb::WriteOptions(), key2, value); - if (s.ok()) s = db->Delete(leveldb::WriteOptions(), key1); -</pre> - -<h1>Atomic Updates</h1> -<p> -Note that if the process dies after the Put of key2 but before the -delete of key1, the same value may be left stored under multiple keys. -Such problems can be avoided by using the <code>WriteBatch</code> class to -atomically apply a set of updates: -<p> -<pre> - #include "leveldb/write_batch.h" - ... - std::string value; - leveldb::Status s = db->Get(leveldb::ReadOptions(), key1, &value); - if (s.ok()) { - leveldb::WriteBatch batch; - batch.Delete(key1); - batch.Put(key2, value); - s = db->Write(leveldb::WriteOptions(), &batch); - } -</pre> -The <code>WriteBatch</code> holds a sequence of edits to be made to the database, -and these edits within the batch are applied in order. Note that we -called <code>Delete</code> before <code>Put</code> so that if <code>key1</code> is identical to <code>key2</code>, -we do not end up erroneously dropping the value entirely. -<p> -Apart from its atomicity benefits, <code>WriteBatch</code> may also be used to -speed up bulk updates by placing lots of individual mutations into the -same batch. - -<h1>Synchronous Writes</h1> -By default, each write to <code>leveldb</code> is asynchronous: it -returns after pushing the write from the process into the operating -system. The transfer from operating system memory to the underlying -persistent storage happens asynchronously. The <code>sync</code> flag -can be turned on for a particular write to make the write operation -not return until the data being written has been pushed all the way to -persistent storage. (On Posix systems, this is implemented by calling -either <code>fsync(...)</code> or <code>fdatasync(...)</code> or -<code>msync(..., MS_SYNC)</code> before the write operation returns.) -<pre> - leveldb::WriteOptions write_options; - write_options.sync = true; - db->Put(write_options, ...); -</pre> -Asynchronous writes are often more than a thousand times as fast as -synchronous writes. The downside of asynchronous writes is that a -crash of the machine may cause the last few updates to be lost. Note -that a crash of just the writing process (i.e., not a reboot) will not -cause any loss since even when <code>sync</code> is false, an update -is pushed from the process memory into the operating system before it -is considered done. - -<p> -Asynchronous writes can often be used safely. For example, when -loading a large amount of data into the database you can handle lost -updates by restarting the bulk load after a crash. A hybrid scheme is -also possible where every Nth write is synchronous, and in the event -of a crash, the bulk load is restarted just after the last synchronous -write finished by the previous run. (The synchronous write can update -a marker that describes where to restart on a crash.) - -<p> -<code>WriteBatch</code> provides an alternative to asynchronous writes. -Multiple updates may be placed in the same <code>WriteBatch</code> and -applied together using a synchronous write (i.e., -<code>write_options.sync</code> is set to true). The extra cost of -the synchronous write will be amortized across all of the writes in -the batch. - -<p> -<h1>Concurrency</h1> -<p> -A database may only be opened by one process at a time. -The <code>leveldb</code> implementation acquires a lock from the -operating system to prevent misuse. Within a single process, the -same <code>leveldb::DB</code> object may be safely shared by multiple -concurrent threads. I.e., different threads may write into or fetch -iterators or call <code>Get</code> on the same database without any -external synchronization (the leveldb implementation will -automatically do the required synchronization). However other objects -(like Iterator and WriteBatch) may require external synchronization. -If two threads share such an object, they must protect access to it -using their own locking protocol. More details are available in -the public header files. -<p> -<h1>Iteration</h1> -<p> -The following example demonstrates how to print all key,value pairs -in a database. -<p> -<pre> - leveldb::Iterator* it = db->NewIterator(leveldb::ReadOptions()); - for (it->SeekToFirst(); it->Valid(); it->Next()) { - cout << it->key().ToString() << ": " << it->value().ToString() << endl; - } - assert(it->status().ok()); // Check for any errors found during the scan - delete it; -</pre> -The following variation shows how to process just the keys in the -range <code>[start,limit)</code>: -<p> -<pre> - for (it->Seek(start); - it->Valid() && it->key().ToString() < limit; - it->Next()) { - ... - } -</pre> -You can also process entries in reverse order. (Caveat: reverse -iteration may be somewhat slower than forward iteration.) -<p> -<pre> - for (it->SeekToLast(); it->Valid(); it->Prev()) { - ... - } -</pre> -<h1>Snapshots</h1> -<p> -Snapshots provide consistent read-only views over the entire state of -the key-value store. <code>ReadOptions::snapshot</code> may be non-NULL to indicate -that a read should operate on a particular version of the DB state. -If <code>ReadOptions::snapshot</code> is NULL, the read will operate on an -implicit snapshot of the current state. -<p> -Snapshots are created by the DB::GetSnapshot() method: -<p> -<pre> - leveldb::ReadOptions options; - options.snapshot = db->GetSnapshot(); - ... apply some updates to db ... - leveldb::Iterator* iter = db->NewIterator(options); - ... read using iter to view the state when the snapshot was created ... - delete iter; - db->ReleaseSnapshot(options.snapshot); -</pre> -Note that when a snapshot is no longer needed, it should be released -using the DB::ReleaseSnapshot interface. This allows the -implementation to get rid of state that was being maintained just to -support reading as of that snapshot. -<h1>Slice</h1> -<p> -The return value of the <code>it->key()</code> and <code>it->value()</code> calls above -are instances of the <code>leveldb::Slice</code> type. <code>Slice</code> is a simple -structure that contains a length and a pointer to an external byte -array. Returning a <code>Slice</code> is a cheaper alternative to returning a -<code>std::string</code> since we do not need to copy potentially large keys and -values. In addition, <code>leveldb</code> methods do not return null-terminated -C-style strings since <code>leveldb</code> keys and values are allowed to -contain '\0' bytes. -<p> -C++ strings and null-terminated C-style strings can be easily converted -to a Slice: -<p> -<pre> - leveldb::Slice s1 = "hello"; - - std::string str("world"); - leveldb::Slice s2 = str; -</pre> -A Slice can be easily converted back to a C++ string: -<pre> - std::string str = s1.ToString(); - assert(str == std::string("hello")); -</pre> -Be careful when using Slices since it is up to the caller to ensure that -the external byte array into which the Slice points remains live while -the Slice is in use. For example, the following is buggy: -<p> -<pre> - leveldb::Slice slice; - if (...) { - std::string str = ...; - slice = str; - } - Use(slice); -</pre> -When the <code>if</code> statement goes out of scope, <code>str</code> will be destroyed and the -backing storage for <code>slice</code> will disappear. -<p> -<h1>Comparators</h1> -<p> -The preceding examples used the default ordering function for key, -which orders bytes lexicographically. You can however supply a custom -comparator when opening a database. For example, suppose each -database key consists of two numbers and we should sort by the first -number, breaking ties by the second number. First, define a proper -subclass of <code>leveldb::Comparator</code> that expresses these rules: -<p> -<pre> - class TwoPartComparator : public leveldb::Comparator { - public: - // Three-way comparison function: - // if a < b: negative result - // if a > b: positive result - // else: zero result - int Compare(const leveldb::Slice& a, const leveldb::Slice& b) const { - int a1, a2, b1, b2; - ParseKey(a, &a1, &a2); - ParseKey(b, &b1, &b2); - if (a1 < b1) return -1; - if (a1 > b1) return +1; - if (a2 < b2) return -1; - if (a2 > b2) return +1; - return 0; - } - - // Ignore the following methods for now: - const char* Name() const { return "TwoPartComparator"; } - void FindShortestSeparator(std::string*, const leveldb::Slice&) const { } - void FindShortSuccessor(std::string*) const { } - }; -</pre> -Now create a database using this custom comparator: -<p> -<pre> - TwoPartComparator cmp; - leveldb::DB* db; - leveldb::Options options; - options.create_if_missing = true; - options.comparator = &cmp; - leveldb::Status status = leveldb::DB::Open(options, "/tmp/testdb", &db); - ... -</pre> -<h2>Backwards compatibility</h2> -<p> -The result of the comparator's <code>Name</code> method is attached to the -database when it is created, and is checked on every subsequent -database open. If the name changes, the <code>leveldb::DB::Open</code> call will -fail. Therefore, change the name if and only if the new key format -and comparison function are incompatible with existing databases, and -it is ok to discard the contents of all existing databases. -<p> -You can however still gradually evolve your key format over time with -a little bit of pre-planning. For example, you could store a version -number at the end of each key (one byte should suffice for most uses). -When you wish to switch to a new key format (e.g., adding an optional -third part to the keys processed by <code>TwoPartComparator</code>), -(a) keep the same comparator name (b) increment the version number -for new keys (c) change the comparator function so it uses the -version numbers found in the keys to decide how to interpret them. -<p> -<h1>Performance</h1> -<p> -Performance can be tuned by changing the default values of the -types defined in <code>include/leveldb/options.h</code>. - -<p> -<h2>Block size</h2> -<p> -<code>leveldb</code> groups adjacent keys together into the same block and such a -block is the unit of transfer to and from persistent storage. The -default block size is approximately 4096 uncompressed bytes. -Applications that mostly do bulk scans over the contents of the -database may wish to increase this size. Applications that do a lot -of point reads of small values may wish to switch to a smaller block -size if performance measurements indicate an improvement. There isn't -much benefit in using blocks smaller than one kilobyte, or larger than -a few megabytes. Also note that compression will be more effective -with larger block sizes. -<p> -<h2>Compression</h2> -<p> -Each block is individually compressed before being written to -persistent storage. Compression is on by default since the default -compression method is very fast, and is automatically disabled for -uncompressible data. In rare cases, applications may want to disable -compression entirely, but should only do so if benchmarks show a -performance improvement: -<p> -<pre> - leveldb::Options options; - options.compression = leveldb::kNoCompression; - ... leveldb::DB::Open(options, name, ...) .... -</pre> -<h2>Cache</h2> -<p> -The contents of the database are stored in a set of files in the -filesystem and each file stores a sequence of compressed blocks. If -<code>options.cache</code> is non-NULL, it is used to cache frequently used -uncompressed block contents. -<p> -<pre> - #include "leveldb/cache.h" - - leveldb::Options options; - options.cache = leveldb::NewLRUCache(100 * 1048576); // 100MB cache - leveldb::DB* db; - leveldb::DB::Open(options, name, &db); - ... use the db ... - delete db - delete options.cache; -</pre> -Note that the cache holds uncompressed data, and therefore it should -be sized according to application level data sizes, without any -reduction from compression. (Caching of compressed blocks is left to -the operating system buffer cache, or any custom <code>Env</code> -implementation provided by the client.) -<p> -When performing a bulk read, the application may wish to disable -caching so that the data processed by the bulk read does not end up -displacing most of the cached contents. A per-iterator option can be -used to achieve this: -<p> -<pre> - leveldb::ReadOptions options; - options.fill_cache = false; - leveldb::Iterator* it = db->NewIterator(options); - for (it->SeekToFirst(); it->Valid(); it->Next()) { - ... - } -</pre> -<h2>Key Layout</h2> -<p> -Note that the unit of disk transfer and caching is a block. Adjacent -keys (according to the database sort order) will usually be placed in -the same block. Therefore the application can improve its performance -by placing keys that are accessed together near each other and placing -infrequently used keys in a separate region of the key space. -<p> -For example, suppose we are implementing a simple file system on top -of <code>leveldb</code>. The types of entries we might wish to store are: -<p> -<pre> - filename -> permission-bits, length, list of file_block_ids - file_block_id -> data -</pre> -We might want to prefix <code>filename</code> keys with one letter (say '/') and the -<code>file_block_id</code> keys with a different letter (say '0') so that scans -over just the metadata do not force us to fetch and cache bulky file -contents. -<p> -<h2>Filters</h2> -<p> -Because of the way <code>leveldb</code> data is organized on disk, -a single <code>Get()</code> call may involve multiple reads from disk. -The optional <code>FilterPolicy</code> mechanism can be used to reduce -the number of disk reads substantially. -<pre> - leveldb::Options options; - options.filter_policy = NewBloomFilterPolicy(10); - leveldb::DB* db; - leveldb::DB::Open(options, "/tmp/testdb", &db); - ... use the database ... - delete db; - delete options.filter_policy; -</pre> -The preceding code associates a -<a href="http://en.wikipedia.org/wiki/Bloom_filter";>Bloom filter</a> -based filtering policy with the database. Bloom filter based -filtering relies on keeping some number of bits of data in memory per -key (in this case 10 bits per key since that is the argument we passed -to NewBloomFilterPolicy). This filter will reduce the number of unnecessary -disk reads needed for <code>Get()</code> calls by a factor of -approximately a 100. Increasing the bits per key will lead to a -larger reduction at the cost of more memory usage. We recommend that -applications whose working set does not fit in memory and that do a -lot of random reads set a filter policy. -<p> -If you are using a custom comparator, you should ensure that the filter -policy you are using is compatible with your comparator. For example, -consider a comparator that ignores trailing spaces when comparing keys. -<code>NewBloomFilterPolicy</code> must not be used with such a comparator. -Instead, the application should provide a custom filter policy that -also ignores trailing spaces. For example: -<pre> - class CustomFilterPolicy : public leveldb::FilterPolicy { - private: - FilterPolicy* builtin_policy_; - public: - CustomFilterPolicy() : builtin_policy_(NewBloomFilterPolicy(10)) { } - ~CustomFilterPolicy() { delete builtin_policy_; } - - const char* Name() const { return "IgnoreTrailingSpacesFilter"; } - - void CreateFilter(const Slice* keys, int n, std::string* dst) const { - // Use builtin bloom filter code after removing trailing spaces - std::vector<Slice> trimmed(n); - for (int i = 0; i < n; i++) { - trimmed[i] = RemoveTrailingSpaces(keys[i]); - } - return builtin_policy_->CreateFilter(&trimmed[i], n, dst); - } - - bool KeyMayMatch(const Slice& key, const Slice& filter) const { - // Use builtin bloom filter code after removing trailing spaces - return builtin_policy_->KeyMayMatch(RemoveTrailingSpaces(key), filter); - } - }; -</pre> -<p> -Advanced applications may provide a filter policy that does not use -a bloom filter but uses some other mechanism for summarizing a set -of keys. See <code>leveldb/filter_policy.h</code> for detail. -<p> -<h1>Checksums</h1> -<p> -<code>leveldb</code> associates checksums with all data it stores in the file system. -There are two separate controls provided over how aggressively these -checksums are verified: -<p> -<ul> -<li> <code>ReadOptions::verify_checksums</code> may be set to true to force - checksum verification of all data that is read from the file system on - behalf of a particular read. By default, no such verification is - done. -<p> -<li> <code>Options::paranoid_checks</code> may be set to true before opening a - database to make the database implementation raise an error as soon as - it detects an internal corruption. Depending on which portion of the - database has been corrupted, the error may be raised when the database - is opened, or later by another database operation. By default, - paranoid checking is off so that the database can be used even if - parts of its persistent storage have been corrupted. -<p> - If a database is corrupted (perhaps it cannot be opened when - paranoid checking is turned on), the <code>leveldb::RepairDB</code> function - may be used to recover as much of the data as possible -<p> -</ul> -<h1>Approximate Sizes</h1> -<p> -The <code>GetApproximateSizes</code> method can used to get the approximate -number of bytes of file system space used by one or more key ranges. -<p> -<pre> - leveldb::Range ranges[2]; - ranges[0] = leveldb::Range("a", "c"); - ranges[1] = leveldb::Range("x", "z"); - uint64_t sizes[2]; - leveldb::Status s = db->GetApproximateSizes(ranges, 2, sizes); -</pre> -The preceding call will set <code>sizes[0]</code> to the approximate number of -bytes of file system space used by the key range <code>[a..c)</code> and -<code>sizes[1]</code> to the approximate number of bytes used by the key range -<code>[x..z)</code>. -<p> -<h1>Environment</h1> -<p> -All file operations (and other operating system calls) issued by the -<code>leveldb</code> implementation are routed through a <code>leveldb::Env</code> object. -Sophisticated clients may wish to provide their own <code>Env</code> -implementation to get better control. For example, an application may -introduce artificial delays in the file IO paths to limit the impact -of <code>leveldb</code> on other activities in the system. -<p> -<pre> - class SlowEnv : public leveldb::Env { - .. implementation of the Env interface ... - }; - - SlowEnv env; - leveldb::Options options; - options.env = &env; - Status s = leveldb::DB::Open(options, ...); -</pre> -<h1>Porting</h1> -<p> -<code>leveldb</code> may be ported to a new platform by providing platform -specific implementations of the types/methods/functions exported by -<code>leveldb/port/port.h</code>. See <code>leveldb/port/port_example.h</code> for more -details. -<p> -In addition, the new platform may need a new default <code>leveldb::Env</code> -implementation. See <code>leveldb/util/env_posix.h</code> for an example. - -<h1>Other Information</h1> - -<p> -Details about the <code>leveldb</code> implementation may be found in -the following documents: -<ul> -<li> <a href="impl.html">Implementation notes</a> -<li> <a href="table_format.txt">Format of an immutable Table file</a> -<li> <a href="log_format.txt">Format of a log file</a> -</ul> - -</body> -</html>
http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/leveldb-1.18/doc/log_format.txt ---------------------------------------------------------------------- diff --git a/thirdparty/leveldb-1.18/doc/log_format.txt b/thirdparty/leveldb-1.18/doc/log_format.txt deleted file mode 100755 index 4cca5ef..0000000 --- a/thirdparty/leveldb-1.18/doc/log_format.txt +++ /dev/null @@ -1,75 +0,0 @@ -The log file contents are a sequence of 32KB blocks. The only -exception is that the tail of the file may contain a partial block. - -Each block consists of a sequence of records: - block := record* trailer? - record := - checksum: uint32 // crc32c of type and data[] ; little-endian - length: uint16 // little-endian - type: uint8 // One of FULL, FIRST, MIDDLE, LAST - data: uint8[length] - -A record never starts within the last six bytes of a block (since it -won't fit). Any leftover bytes here form the trailer, which must -consist entirely of zero bytes and must be skipped by readers. - -Aside: if exactly seven bytes are left in the current block, and a new -non-zero length record is added, the writer must emit a FIRST record -(which contains zero bytes of user data) to fill up the trailing seven -bytes of the block and then emit all of the user data in subsequent -blocks. - -More types may be added in the future. Some Readers may skip record -types they do not understand, others may report that some data was -skipped. - -FULL == 1 -FIRST == 2 -MIDDLE == 3 -LAST == 4 - -The FULL record contains the contents of an entire user record. - -FIRST, MIDDLE, LAST are types used for user records that have been -split into multiple fragments (typically because of block boundaries). -FIRST is the type of the first fragment of a user record, LAST is the -type of the last fragment of a user record, and MIDDLE is the type of -all interior fragments of a user record. - -Example: consider a sequence of user records: - A: length 1000 - B: length 97270 - C: length 8000 -A will be stored as a FULL record in the first block. - -B will be split into three fragments: first fragment occupies the rest -of the first block, second fragment occupies the entirety of the -second block, and the third fragment occupies a prefix of the third -block. This will leave six bytes free in the third block, which will -be left empty as the trailer. - -C will be stored as a FULL record in the fourth block. - -=================== - -Some benefits over the recordio format: - -(1) We do not need any heuristics for resyncing - just go to next -block boundary and scan. If there is a corruption, skip to the next -block. As a side-benefit, we do not get confused when part of the -contents of one log file are embedded as a record inside another log -file. - -(2) Splitting at approximate boundaries (e.g., for mapreduce) is -simple: find the next block boundary and skip records until we -hit a FULL or FIRST record. - -(3) We do not need extra buffering for large records. - -Some downsides compared to recordio format: - -(1) No packing of tiny records. This could be fixed by adding a new -record type, so it is a shortcoming of the current implementation, -not necessarily the format. - -(2) No compression. Again, this could be fixed by adding new record types. http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/leveldb-1.18/doc/table_format.txt ---------------------------------------------------------------------- diff --git a/thirdparty/leveldb-1.18/doc/table_format.txt b/thirdparty/leveldb-1.18/doc/table_format.txt deleted file mode 100755 index ca8f9b4..0000000 --- a/thirdparty/leveldb-1.18/doc/table_format.txt +++ /dev/null @@ -1,104 +0,0 @@ -File format -=========== - - <beginning_of_file> - [data block 1] - [data block 2] - ... - [data block N] - [meta block 1] - ... - [meta block K] - [metaindex block] - [index block] - [Footer] (fixed size; starts at file_size - sizeof(Footer)) - <end_of_file> - -The file contains internal pointers. Each such pointer is called -a BlockHandle and contains the following information: - offset: varint64 - size: varint64 -See https://developers.google.com/protocol-buffers/docs/encoding#varints -for an explanation of varint64 format. - -(1) The sequence of key/value pairs in the file are stored in sorted -order and partitioned into a sequence of data blocks. These blocks -come one after another at the beginning of the file. Each data block -is formatted according to the code in block_builder.cc, and then -optionally compressed. - -(2) After the data blocks we store a bunch of meta blocks. The -supported meta block types are described below. More meta block types -may be added in the future. Each meta block is again formatted using -block_builder.cc and then optionally compressed. - -(3) A "metaindex" block. It contains one entry for every other meta -block where the key is the name of the meta block and the value is a -BlockHandle pointing to that meta block. - -(4) An "index" block. This block contains one entry per data block, -where the key is a string >= last key in that data block and before -the first key in the successive data block. The value is the -BlockHandle for the data block. - -(6) At the very end of the file is a fixed length footer that contains -the BlockHandle of the metaindex and index blocks as well as a magic number. - metaindex_handle: char[p]; // Block handle for metaindex - index_handle: char[q]; // Block handle for index - padding: char[40-p-q]; // zeroed bytes to make fixed length - // (40==2*BlockHandle::kMaxEncodedLength) - magic: fixed64; // == 0xdb4775248b80fb57 (little-endian) - -"filter" Meta Block -------------------- - -If a "FilterPolicy" was specified when the database was opened, a -filter block is stored in each table. The "metaindex" block contains -an entry that maps from "filter.<N>" to the BlockHandle for the filter -block where "<N>" is the string returned by the filter policy's -"Name()" method. - -The filter block stores a sequence of filters, where filter i contains -the output of FilterPolicy::CreateFilter() on all keys that are stored -in a block whose file offset falls within the range - - [ i*base ... (i+1)*base-1 ] - -Currently, "base" is 2KB. So for example, if blocks X and Y start in -the range [ 0KB .. 2KB-1 ], all of the keys in X and Y will be -converted to a filter by calling FilterPolicy::CreateFilter(), and the -resulting filter will be stored as the first filter in the filter -block. - -The filter block is formatted as follows: - - [filter 0] - [filter 1] - [filter 2] - ... - [filter N-1] - - [offset of filter 0] : 4 bytes - [offset of filter 1] : 4 bytes - [offset of filter 2] : 4 bytes - ... - [offset of filter N-1] : 4 bytes - - [offset of beginning of offset array] : 4 bytes - lg(base) : 1 byte - -The offset array at the end of the filter block allows efficient -mapping from a data block offset to the corresponding filter. - -"stats" Meta Block ------------------- - -This meta block contains a bunch of stats. The key is the name -of the statistic. The value contains the statistic. -TODO(postrelease): record following stats. - data size - index size - key size (uncompressed) - value size (uncompressed) - number of entries - number of data blocks http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/leveldb-1.18/helpers/memenv/memenv.cc ---------------------------------------------------------------------- diff --git a/thirdparty/leveldb-1.18/helpers/memenv/memenv.cc b/thirdparty/leveldb-1.18/helpers/memenv/memenv.cc deleted file mode 100755 index 43ef2e0..0000000 --- a/thirdparty/leveldb-1.18/helpers/memenv/memenv.cc +++ /dev/null @@ -1,385 +0,0 @@ -// Copyright (c) 2011 The LevelDB Authors. All rights reserved. -// Use of this source code is governed by a BSD-style license that can be -// found in the LICENSE file. See the AUTHORS file for names of contributors. - -#include "helpers/memenv/memenv.h" - -#include "leveldb/env.h" -#include "leveldb/status.h" -#include "port/port.h" -#include "util/mutexlock.h" -#include <map> -#include <string.h> -#include <string> -#include <vector> - -namespace leveldb { - -namespace { - -class FileState { - public: - // FileStates are reference counted. The initial reference count is zero - // and the caller must call Ref() at least once. - FileState() : refs_(0), size_(0) {} - - // Increase the reference count. - void Ref() { - MutexLock lock(&refs_mutex_); - ++refs_; - } - - // Decrease the reference count. Delete if this is the last reference. - void Unref() { - bool do_delete = false; - - { - MutexLock lock(&refs_mutex_); - --refs_; - assert(refs_ >= 0); - if (refs_ <= 0) { - do_delete = true; - } - } - - if (do_delete) { - delete this; - } - } - - uint64_t Size() const { return size_; } - - Status Read(uint64_t offset, size_t n, Slice* result, char* scratch) const { - if (offset > size_) { - return Status::IOError("Offset greater than file size."); - } - const uint64_t available = size_ - offset; - if (n > available) { - n = static_cast<size_t>(available); - } - if (n == 0) { - *result = Slice(); - return Status::OK(); - } - - assert(offset / kBlockSize <= SIZE_MAX); - size_t block = static_cast<size_t>(offset / kBlockSize); - size_t block_offset = offset % kBlockSize; - - if (n <= kBlockSize - block_offset) { - // The requested bytes are all in the first block. - *result = Slice(blocks_[block] + block_offset, n); - return Status::OK(); - } - - size_t bytes_to_copy = n; - char* dst = scratch; - - while (bytes_to_copy > 0) { - size_t avail = kBlockSize - block_offset; - if (avail > bytes_to_copy) { - avail = bytes_to_copy; - } - memcpy(dst, blocks_[block] + block_offset, avail); - - bytes_to_copy -= avail; - dst += avail; - block++; - block_offset = 0; - } - - *result = Slice(scratch, n); - return Status::OK(); - } - - Status Append(const Slice& data) { - const char* src = data.data(); - size_t src_len = data.size(); - - while (src_len > 0) { - size_t avail; - size_t offset = size_ % kBlockSize; - - if (offset != 0) { - // There is some room in the last block. - avail = kBlockSize - offset; - } else { - // No room in the last block; push new one. - blocks_.push_back(new char[kBlockSize]); - avail = kBlockSize; - } - - if (avail > src_len) { - avail = src_len; - } - memcpy(blocks_.back() + offset, src, avail); - src_len -= avail; - src += avail; - size_ += avail; - } - - return Status::OK(); - } - - private: - // Private since only Unref() should be used to delete it. - ~FileState() { - for (std::vector<char*>::iterator i = blocks_.begin(); i != blocks_.end(); - ++i) { - delete [] *i; - } - } - - // No copying allowed. - FileState(const FileState&); - void operator=(const FileState&); - - port::Mutex refs_mutex_; - int refs_; // Protected by refs_mutex_; - - // The following fields are not protected by any mutex. They are only mutable - // while the file is being written, and concurrent access is not allowed - // to writable files. - std::vector<char*> blocks_; - uint64_t size_; - - enum { kBlockSize = 8 * 1024 }; -}; - -class SequentialFileImpl : public SequentialFile { - public: - explicit SequentialFileImpl(FileState* file) : file_(file), pos_(0) { - file_->Ref(); - } - - ~SequentialFileImpl() { - file_->Unref(); - } - - virtual Status Read(size_t n, Slice* result, char* scratch) { - Status s = file_->Read(pos_, n, result, scratch); - if (s.ok()) { - pos_ += result->size(); - } - return s; - } - - virtual Status Skip(uint64_t n) { - if (pos_ > file_->Size()) { - return Status::IOError("pos_ > file_->Size()"); - } - const uint64_t available = file_->Size() - pos_; - if (n > available) { - n = available; - } - pos_ += n; - return Status::OK(); - } - - private: - FileState* file_; - uint64_t pos_; -}; - -class RandomAccessFileImpl : public RandomAccessFile { - public: - explicit RandomAccessFileImpl(FileState* file) : file_(file) { - file_->Ref(); - } - - ~RandomAccessFileImpl() { - file_->Unref(); - } - - virtual Status Read(uint64_t offset, size_t n, Slice* result, - char* scratch) const { - return file_->Read(offset, n, result, scratch); - } - - private: - FileState* file_; -}; - -class WritableFileImpl : public WritableFile { - public: - WritableFileImpl(FileState* file) : file_(file) { - file_->Ref(); - } - - ~WritableFileImpl() { - file_->Unref(); - } - - virtual Status Append(const Slice& data) { - return file_->Append(data); - } - - virtual Status Close() { return Status::OK(); } - virtual Status Flush() { return Status::OK(); } - virtual Status Sync() { return Status::OK(); } - - private: - FileState* file_; -}; - -class NoOpLogger : public Logger { - public: - virtual void Logv(const char* format, va_list ap) { } -}; - -class InMemoryEnv : public EnvWrapper { - public: - explicit InMemoryEnv(Env* base_env) : EnvWrapper(base_env) { } - - virtual ~InMemoryEnv() { - for (FileSystem::iterator i = file_map_.begin(); i != file_map_.end(); ++i){ - i->second->Unref(); - } - } - - // Partial implementation of the Env interface. - virtual Status NewSequentialFile(const std::string& fname, - SequentialFile** result) { - MutexLock lock(&mutex_); - if (file_map_.find(fname) == file_map_.end()) { - *result = NULL; - return Status::IOError(fname, "File not found"); - } - - *result = new SequentialFileImpl(file_map_[fname]); - return Status::OK(); - } - - virtual Status NewRandomAccessFile(const std::string& fname, - RandomAccessFile** result) { - MutexLock lock(&mutex_); - if (file_map_.find(fname) == file_map_.end()) { - *result = NULL; - return Status::IOError(fname, "File not found"); - } - - *result = new RandomAccessFileImpl(file_map_[fname]); - return Status::OK(); - } - - virtual Status NewWritableFile(const std::string& fname, - WritableFile** result) { - MutexLock lock(&mutex_); - if (file_map_.find(fname) != file_map_.end()) { - DeleteFileInternal(fname); - } - - FileState* file = new FileState(); - file->Ref(); - file_map_[fname] = file; - - *result = new WritableFileImpl(file); - return Status::OK(); - } - - virtual bool FileExists(const std::string& fname) { - MutexLock lock(&mutex_); - return file_map_.find(fname) != file_map_.end(); - } - - virtual Status GetChildren(const std::string& dir, - std::vector<std::string>* result) { - MutexLock lock(&mutex_); - result->clear(); - - for (FileSystem::iterator i = file_map_.begin(); i != file_map_.end(); ++i){ - const std::string& filename = i->first; - - if (filename.size() >= dir.size() + 1 && filename[dir.size()] == '/' && - Slice(filename).starts_with(Slice(dir))) { - result->push_back(filename.substr(dir.size() + 1)); - } - } - - return Status::OK(); - } - - void DeleteFileInternal(const std::string& fname) { - if (file_map_.find(fname) == file_map_.end()) { - return; - } - - file_map_[fname]->Unref(); - file_map_.erase(fname); - } - - virtual Status DeleteFile(const std::string& fname) { - MutexLock lock(&mutex_); - if (file_map_.find(fname) == file_map_.end()) { - return Status::IOError(fname, "File not found"); - } - - DeleteFileInternal(fname); - return Status::OK(); - } - - virtual Status CreateDir(const std::string& dirname) { - return Status::OK(); - } - - virtual Status DeleteDir(const std::string& dirname) { - return Status::OK(); - } - - virtual Status GetFileSize(const std::string& fname, uint64_t* file_size) { - MutexLock lock(&mutex_); - if (file_map_.find(fname) == file_map_.end()) { - return Status::IOError(fname, "File not found"); - } - - *file_size = file_map_[fname]->Size(); - return Status::OK(); - } - - virtual Status RenameFile(const std::string& src, - const std::string& target) { - MutexLock lock(&mutex_); - if (file_map_.find(src) == file_map_.end()) { - return Status::IOError(src, "File not found"); - } - - DeleteFileInternal(target); - file_map_[target] = file_map_[src]; - file_map_.erase(src); - return Status::OK(); - } - - virtual Status LockFile(const std::string& fname, FileLock** lock) { - *lock = new FileLock; - return Status::OK(); - } - - virtual Status UnlockFile(FileLock* lock) { - delete lock; - return Status::OK(); - } - - virtual Status GetTestDirectory(std::string* path) { - *path = "/test"; - return Status::OK(); - } - - virtual Status NewLogger(const std::string& fname, Logger** result) { - *result = new NoOpLogger; - return Status::OK(); - } - - private: - // Map from filenames to FileState objects, representing a simple file system. - typedef std::map<std::string, FileState*> FileSystem; - port::Mutex mutex_; - FileSystem file_map_; // Protected by mutex_. -}; - -} // namespace - -Env* NewMemEnv(Env* base_env) { - return new InMemoryEnv(base_env); -} - -} // namespace leveldb http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/leveldb-1.18/helpers/memenv/memenv.h ---------------------------------------------------------------------- diff --git a/thirdparty/leveldb-1.18/helpers/memenv/memenv.h b/thirdparty/leveldb-1.18/helpers/memenv/memenv.h deleted file mode 100755 index 03b88de..0000000 --- a/thirdparty/leveldb-1.18/helpers/memenv/memenv.h +++ /dev/null @@ -1,20 +0,0 @@ -// Copyright (c) 2011 The LevelDB Authors. All rights reserved. -// Use of this source code is governed by a BSD-style license that can be -// found in the LICENSE file. See the AUTHORS file for names of contributors. - -#ifndef STORAGE_LEVELDB_HELPERS_MEMENV_MEMENV_H_ -#define STORAGE_LEVELDB_HELPERS_MEMENV_MEMENV_H_ - -namespace leveldb { - -class Env; - -// Returns a new environment that stores its data in memory and delegates -// all non-file-storage tasks to base_env. The caller must delete the result -// when it is no longer needed. -// *base_env must remain live while the result is in use. -Env* NewMemEnv(Env* base_env); - -} // namespace leveldb - -#endif // STORAGE_LEVELDB_HELPERS_MEMENV_MEMENV_H_ http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/leveldb-1.18/helpers/memenv/memenv_test.cc ---------------------------------------------------------------------- diff --git a/thirdparty/leveldb-1.18/helpers/memenv/memenv_test.cc b/thirdparty/leveldb-1.18/helpers/memenv/memenv_test.cc deleted file mode 100755 index a44310f..0000000 --- a/thirdparty/leveldb-1.18/helpers/memenv/memenv_test.cc +++ /dev/null @@ -1,232 +0,0 @@ -// Copyright (c) 2011 The LevelDB Authors. All rights reserved. -// Use of this source code is governed by a BSD-style license that can be -// found in the LICENSE file. See the AUTHORS file for names of contributors. - -#include "helpers/memenv/memenv.h" - -#include "db/db_impl.h" -#include "leveldb/db.h" -#include "leveldb/env.h" -#include "util/testharness.h" -#include <string> -#include <vector> - -namespace leveldb { - -class MemEnvTest { - public: - Env* env_; - - MemEnvTest() - : env_(NewMemEnv(Env::Default())) { - } - ~MemEnvTest() { - delete env_; - } -}; - -TEST(MemEnvTest, Basics) { - uint64_t file_size; - WritableFile* writable_file; - std::vector<std::string> children; - - ASSERT_OK(env_->CreateDir("/dir")); - - // Check that the directory is empty. - ASSERT_TRUE(!env_->FileExists("/dir/non_existent")); - ASSERT_TRUE(!env_->GetFileSize("/dir/non_existent", &file_size).ok()); - ASSERT_OK(env_->GetChildren("/dir", &children)); - ASSERT_EQ(0, children.size()); - - // Create a file. - ASSERT_OK(env_->NewWritableFile("/dir/f", &writable_file)); - delete writable_file; - - // Check that the file exists. - ASSERT_TRUE(env_->FileExists("/dir/f")); - ASSERT_OK(env_->GetFileSize("/dir/f", &file_size)); - ASSERT_EQ(0, file_size); - ASSERT_OK(env_->GetChildren("/dir", &children)); - ASSERT_EQ(1, children.size()); - ASSERT_EQ("f", children[0]); - - // Write to the file. - ASSERT_OK(env_->NewWritableFile("/dir/f", &writable_file)); - ASSERT_OK(writable_file->Append("abc")); - delete writable_file; - - // Check for expected size. - ASSERT_OK(env_->GetFileSize("/dir/f", &file_size)); - ASSERT_EQ(3, file_size); - - // Check that renaming works. - ASSERT_TRUE(!env_->RenameFile("/dir/non_existent", "/dir/g").ok()); - ASSERT_OK(env_->RenameFile("/dir/f", "/dir/g")); - ASSERT_TRUE(!env_->FileExists("/dir/f")); - ASSERT_TRUE(env_->FileExists("/dir/g")); - ASSERT_OK(env_->GetFileSize("/dir/g", &file_size)); - ASSERT_EQ(3, file_size); - - // Check that opening non-existent file fails. - SequentialFile* seq_file; - RandomAccessFile* rand_file; - ASSERT_TRUE(!env_->NewSequentialFile("/dir/non_existent", &seq_file).ok()); - ASSERT_TRUE(!seq_file); - ASSERT_TRUE(!env_->NewRandomAccessFile("/dir/non_existent", &rand_file).ok()); - ASSERT_TRUE(!rand_file); - - // Check that deleting works. - ASSERT_TRUE(!env_->DeleteFile("/dir/non_existent").ok()); - ASSERT_OK(env_->DeleteFile("/dir/g")); - ASSERT_TRUE(!env_->FileExists("/dir/g")); - ASSERT_OK(env_->GetChildren("/dir", &children)); - ASSERT_EQ(0, children.size()); - ASSERT_OK(env_->DeleteDir("/dir")); -} - -TEST(MemEnvTest, ReadWrite) { - WritableFile* writable_file; - SequentialFile* seq_file; - RandomAccessFile* rand_file; - Slice result; - char scratch[100]; - - ASSERT_OK(env_->CreateDir("/dir")); - - ASSERT_OK(env_->NewWritableFile("/dir/f", &writable_file)); - ASSERT_OK(writable_file->Append("hello ")); - ASSERT_OK(writable_file->Append("world")); - delete writable_file; - - // Read sequentially. - ASSERT_OK(env_->NewSequentialFile("/dir/f", &seq_file)); - ASSERT_OK(seq_file->Read(5, &result, scratch)); // Read "hello". - ASSERT_EQ(0, result.compare("hello")); - ASSERT_OK(seq_file->Skip(1)); - ASSERT_OK(seq_file->Read(1000, &result, scratch)); // Read "world". - ASSERT_EQ(0, result.compare("world")); - ASSERT_OK(seq_file->Read(1000, &result, scratch)); // Try reading past EOF. - ASSERT_EQ(0, result.size()); - ASSERT_OK(seq_file->Skip(100)); // Try to skip past end of file. - ASSERT_OK(seq_file->Read(1000, &result, scratch)); - ASSERT_EQ(0, result.size()); - delete seq_file; - - // Random reads. - ASSERT_OK(env_->NewRandomAccessFile("/dir/f", &rand_file)); - ASSERT_OK(rand_file->Read(6, 5, &result, scratch)); // Read "world". - ASSERT_EQ(0, result.compare("world")); - ASSERT_OK(rand_file->Read(0, 5, &result, scratch)); // Read "hello". - ASSERT_EQ(0, result.compare("hello")); - ASSERT_OK(rand_file->Read(10, 100, &result, scratch)); // Read "d". - ASSERT_EQ(0, result.compare("d")); - - // Too high offset. - ASSERT_TRUE(!rand_file->Read(1000, 5, &result, scratch).ok()); - delete rand_file; -} - -TEST(MemEnvTest, Locks) { - FileLock* lock; - - // These are no-ops, but we test they return success. - ASSERT_OK(env_->LockFile("some file", &lock)); - ASSERT_OK(env_->UnlockFile(lock)); -} - -TEST(MemEnvTest, Misc) { - std::string test_dir; - ASSERT_OK(env_->GetTestDirectory(&test_dir)); - ASSERT_TRUE(!test_dir.empty()); - - WritableFile* writable_file; - ASSERT_OK(env_->NewWritableFile("/a/b", &writable_file)); - - // These are no-ops, but we test they return success. - ASSERT_OK(writable_file->Sync()); - ASSERT_OK(writable_file->Flush()); - ASSERT_OK(writable_file->Close()); - delete writable_file; -} - -TEST(MemEnvTest, LargeWrite) { - const size_t kWriteSize = 300 * 1024; - char* scratch = new char[kWriteSize * 2]; - - std::string write_data; - for (size_t i = 0; i < kWriteSize; ++i) { - write_data.append(1, static_cast<char>(i)); - } - - WritableFile* writable_file; - ASSERT_OK(env_->NewWritableFile("/dir/f", &writable_file)); - ASSERT_OK(writable_file->Append("foo")); - ASSERT_OK(writable_file->Append(write_data)); - delete writable_file; - - SequentialFile* seq_file; - Slice result; - ASSERT_OK(env_->NewSequentialFile("/dir/f", &seq_file)); - ASSERT_OK(seq_file->Read(3, &result, scratch)); // Read "foo". - ASSERT_EQ(0, result.compare("foo")); - - size_t read = 0; - std::string read_data; - while (read < kWriteSize) { - ASSERT_OK(seq_file->Read(kWriteSize - read, &result, scratch)); - read_data.append(result.data(), result.size()); - read += result.size(); - } - ASSERT_TRUE(write_data == read_data); - delete seq_file; - delete [] scratch; -} - -TEST(MemEnvTest, DBTest) { - Options options; - options.create_if_missing = true; - options.env = env_; - DB* db; - - const Slice keys[] = {Slice("aaa"), Slice("bbb"), Slice("ccc")}; - const Slice vals[] = {Slice("foo"), Slice("bar"), Slice("baz")}; - - ASSERT_OK(DB::Open(options, "/dir/db", &db)); - for (size_t i = 0; i < 3; ++i) { - ASSERT_OK(db->Put(WriteOptions(), keys[i], vals[i])); - } - - for (size_t i = 0; i < 3; ++i) { - std::string res; - ASSERT_OK(db->Get(ReadOptions(), keys[i], &res)); - ASSERT_TRUE(res == vals[i]); - } - - Iterator* iterator = db->NewIterator(ReadOptions()); - iterator->SeekToFirst(); - for (size_t i = 0; i < 3; ++i) { - ASSERT_TRUE(iterator->Valid()); - ASSERT_TRUE(keys[i] == iterator->key()); - ASSERT_TRUE(vals[i] == iterator->value()); - iterator->Next(); - } - ASSERT_TRUE(!iterator->Valid()); - delete iterator; - - DBImpl* dbi = reinterpret_cast<DBImpl*>(db); - ASSERT_OK(dbi->TEST_CompactMemTable()); - - for (size_t i = 0; i < 3; ++i) { - std::string res; - ASSERT_OK(db->Get(ReadOptions(), keys[i], &res)); - ASSERT_TRUE(res == vals[i]); - } - - delete db; -} - -} // namespace leveldb - -int main(int argc, char** argv) { - return leveldb::test::RunAllTests(); -} http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/leveldb-1.18/include/leveldb/c.h ---------------------------------------------------------------------- diff --git a/thirdparty/leveldb-1.18/include/leveldb/c.h b/thirdparty/leveldb-1.18/include/leveldb/c.h deleted file mode 100755 index 1048fe3..0000000 --- a/thirdparty/leveldb-1.18/include/leveldb/c.h +++ /dev/null @@ -1,290 +0,0 @@ -/* Copyright (c) 2011 The LevelDB Authors. All rights reserved. - Use of this source code is governed by a BSD-style license that can be - found in the LICENSE file. See the AUTHORS file for names of contributors. - - C bindings for leveldb. May be useful as a stable ABI that can be - used by programs that keep leveldb in a shared library, or for - a JNI api. - - Does not support: - . getters for the option types - . custom comparators that implement key shortening - . custom iter, db, env, cache implementations using just the C bindings - - Some conventions: - - (1) We expose just opaque struct pointers and functions to clients. - This allows us to change internal representations without having to - recompile clients. - - (2) For simplicity, there is no equivalent to the Slice type. Instead, - the caller has to pass the pointer and length as separate - arguments. - - (3) Errors are represented by a null-terminated c string. NULL - means no error. All operations that can raise an error are passed - a "char** errptr" as the last argument. One of the following must - be true on entry: - *errptr == NULL - *errptr points to a malloc()ed null-terminated error message - (On Windows, *errptr must have been malloc()-ed by this library.) - On success, a leveldb routine leaves *errptr unchanged. - On failure, leveldb frees the old value of *errptr and - set *errptr to a malloc()ed error message. - - (4) Bools have the type unsigned char (0 == false; rest == true) - - (5) All of the pointer arguments must be non-NULL. -*/ - -#ifndef STORAGE_LEVELDB_INCLUDE_C_H_ -#define STORAGE_LEVELDB_INCLUDE_C_H_ - -#ifdef __cplusplus -extern "C" { -#endif - -#include <stdarg.h> -#include <stddef.h> -#include <stdint.h> - -/* Exported types */ - -typedef struct leveldb_t leveldb_t; -typedef struct leveldb_cache_t leveldb_cache_t; -typedef struct leveldb_comparator_t leveldb_comparator_t; -typedef struct leveldb_env_t leveldb_env_t; -typedef struct leveldb_filelock_t leveldb_filelock_t; -typedef struct leveldb_filterpolicy_t leveldb_filterpolicy_t; -typedef struct leveldb_iterator_t leveldb_iterator_t; -typedef struct leveldb_logger_t leveldb_logger_t; -typedef struct leveldb_options_t leveldb_options_t; -typedef struct leveldb_randomfile_t leveldb_randomfile_t; -typedef struct leveldb_readoptions_t leveldb_readoptions_t; -typedef struct leveldb_seqfile_t leveldb_seqfile_t; -typedef struct leveldb_snapshot_t leveldb_snapshot_t; -typedef struct leveldb_writablefile_t leveldb_writablefile_t; -typedef struct leveldb_writebatch_t leveldb_writebatch_t; -typedef struct leveldb_writeoptions_t leveldb_writeoptions_t; - -/* DB operations */ - -extern leveldb_t* leveldb_open( - const leveldb_options_t* options, - const char* name, - char** errptr); - -extern void leveldb_close(leveldb_t* db); - -extern void leveldb_put( - leveldb_t* db, - const leveldb_writeoptions_t* options, - const char* key, size_t keylen, - const char* val, size_t vallen, - char** errptr); - -extern void leveldb_delete( - leveldb_t* db, - const leveldb_writeoptions_t* options, - const char* key, size_t keylen, - char** errptr); - -extern void leveldb_write( - leveldb_t* db, - const leveldb_writeoptions_t* options, - leveldb_writebatch_t* batch, - char** errptr); - -/* Returns NULL if not found. A malloc()ed array otherwise. - Stores the length of the array in *vallen. */ -extern char* leveldb_get( - leveldb_t* db, - const leveldb_readoptions_t* options, - const char* key, size_t keylen, - size_t* vallen, - char** errptr); - -extern leveldb_iterator_t* leveldb_create_iterator( - leveldb_t* db, - const leveldb_readoptions_t* options); - -extern const leveldb_snapshot_t* leveldb_create_snapshot( - leveldb_t* db); - -extern void leveldb_release_snapshot( - leveldb_t* db, - const leveldb_snapshot_t* snapshot); - -/* Returns NULL if property name is unknown. - Else returns a pointer to a malloc()-ed null-terminated value. */ -extern char* leveldb_property_value( - leveldb_t* db, - const char* propname); - -extern void leveldb_approximate_sizes( - leveldb_t* db, - int num_ranges, - const char* const* range_start_key, const size_t* range_start_key_len, - const char* const* range_limit_key, const size_t* range_limit_key_len, - uint64_t* sizes); - -extern void leveldb_compact_range( - leveldb_t* db, - const char* start_key, size_t start_key_len, - const char* limit_key, size_t limit_key_len); - -/* Management operations */ - -extern void leveldb_destroy_db( - const leveldb_options_t* options, - const char* name, - char** errptr); - -extern void leveldb_repair_db( - const leveldb_options_t* options, - const char* name, - char** errptr); - -/* Iterator */ - -extern void leveldb_iter_destroy(leveldb_iterator_t*); -extern unsigned char leveldb_iter_valid(const leveldb_iterator_t*); -extern void leveldb_iter_seek_to_first(leveldb_iterator_t*); -extern void leveldb_iter_seek_to_last(leveldb_iterator_t*); -extern void leveldb_iter_seek(leveldb_iterator_t*, const char* k, size_t klen); -extern void leveldb_iter_next(leveldb_iterator_t*); -extern void leveldb_iter_prev(leveldb_iterator_t*); -extern const char* leveldb_iter_key(const leveldb_iterator_t*, size_t* klen); -extern const char* leveldb_iter_value(const leveldb_iterator_t*, size_t* vlen); -extern void leveldb_iter_get_error(const leveldb_iterator_t*, char** errptr); - -/* Write batch */ - -extern leveldb_writebatch_t* leveldb_writebatch_create(); -extern void leveldb_writebatch_destroy(leveldb_writebatch_t*); -extern void leveldb_writebatch_clear(leveldb_writebatch_t*); -extern void leveldb_writebatch_put( - leveldb_writebatch_t*, - const char* key, size_t klen, - const char* val, size_t vlen); -extern void leveldb_writebatch_delete( - leveldb_writebatch_t*, - const char* key, size_t klen); -extern void leveldb_writebatch_iterate( - leveldb_writebatch_t*, - void* state, - void (*put)(void*, const char* k, size_t klen, const char* v, size_t vlen), - void (*deleted)(void*, const char* k, size_t klen)); - -/* Options */ - -extern leveldb_options_t* leveldb_options_create(); -extern void leveldb_options_destroy(leveldb_options_t*); -extern void leveldb_options_set_comparator( - leveldb_options_t*, - leveldb_comparator_t*); -extern void leveldb_options_set_filter_policy( - leveldb_options_t*, - leveldb_filterpolicy_t*); -extern void leveldb_options_set_create_if_missing( - leveldb_options_t*, unsigned char); -extern void leveldb_options_set_error_if_exists( - leveldb_options_t*, unsigned char); -extern void leveldb_options_set_paranoid_checks( - leveldb_options_t*, unsigned char); -extern void leveldb_options_set_env(leveldb_options_t*, leveldb_env_t*); -extern void leveldb_options_set_info_log(leveldb_options_t*, leveldb_logger_t*); -extern void leveldb_options_set_write_buffer_size(leveldb_options_t*, size_t); -extern void leveldb_options_set_max_open_files(leveldb_options_t*, int); -extern void leveldb_options_set_cache(leveldb_options_t*, leveldb_cache_t*); -extern void leveldb_options_set_block_size(leveldb_options_t*, size_t); -extern void leveldb_options_set_block_restart_interval(leveldb_options_t*, int); - -enum { - leveldb_no_compression = 0, - leveldb_snappy_compression = 1 -}; -extern void leveldb_options_set_compression(leveldb_options_t*, int); - -/* Comparator */ - -extern leveldb_comparator_t* leveldb_comparator_create( - void* state, - void (*destructor)(void*), - int (*compare)( - void*, - const char* a, size_t alen, - const char* b, size_t blen), - const char* (*name)(void*)); -extern void leveldb_comparator_destroy(leveldb_comparator_t*); - -/* Filter policy */ - -extern leveldb_filterpolicy_t* leveldb_filterpolicy_create( - void* state, - void (*destructor)(void*), - char* (*create_filter)( - void*, - const char* const* key_array, const size_t* key_length_array, - int num_keys, - size_t* filter_length), - unsigned char (*key_may_match)( - void*, - const char* key, size_t length, - const char* filter, size_t filter_length), - const char* (*name)(void*)); -extern void leveldb_filterpolicy_destroy(leveldb_filterpolicy_t*); - -extern leveldb_filterpolicy_t* leveldb_filterpolicy_create_bloom( - int bits_per_key); - -/* Read options */ - -extern leveldb_readoptions_t* leveldb_readoptions_create(); -extern void leveldb_readoptions_destroy(leveldb_readoptions_t*); -extern void leveldb_readoptions_set_verify_checksums( - leveldb_readoptions_t*, - unsigned char); -extern void leveldb_readoptions_set_fill_cache( - leveldb_readoptions_t*, unsigned char); -extern void leveldb_readoptions_set_snapshot( - leveldb_readoptions_t*, - const leveldb_snapshot_t*); - -/* Write options */ - -extern leveldb_writeoptions_t* leveldb_writeoptions_create(); -extern void leveldb_writeoptions_destroy(leveldb_writeoptions_t*); -extern void leveldb_writeoptions_set_sync( - leveldb_writeoptions_t*, unsigned char); - -/* Cache */ - -extern leveldb_cache_t* leveldb_cache_create_lru(size_t capacity); -extern void leveldb_cache_destroy(leveldb_cache_t* cache); - -/* Env */ - -extern leveldb_env_t* leveldb_create_default_env(); -extern void leveldb_env_destroy(leveldb_env_t*); - -/* Utility */ - -/* Calls free(ptr). - REQUIRES: ptr was malloc()-ed and returned by one of the routines - in this file. Note that in certain cases (typically on Windows), you - may need to call this routine instead of free(ptr) to dispose of - malloc()-ed memory returned by this library. */ -extern void leveldb_free(void* ptr); - -/* Return the major version number for this release. */ -extern int leveldb_major_version(); - -/* Return the minor version number for this release. */ -extern int leveldb_minor_version(); - -#ifdef __cplusplus -} /* end extern "C" */ -#endif - -#endif /* STORAGE_LEVELDB_INCLUDE_C_H_ */ http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/leveldb-1.18/include/leveldb/cache.h ---------------------------------------------------------------------- diff --git a/thirdparty/leveldb-1.18/include/leveldb/cache.h b/thirdparty/leveldb-1.18/include/leveldb/cache.h deleted file mode 100755 index 1a201e5..0000000 --- a/thirdparty/leveldb-1.18/include/leveldb/cache.h +++ /dev/null @@ -1,99 +0,0 @@ -// Copyright (c) 2011 The LevelDB Authors. All rights reserved. -// Use of this source code is governed by a BSD-style license that can be -// found in the LICENSE file. See the AUTHORS file for names of contributors. -// -// A Cache is an interface that maps keys to values. It has internal -// synchronization and may be safely accessed concurrently from -// multiple threads. It may automatically evict entries to make room -// for new entries. Values have a specified charge against the cache -// capacity. For example, a cache where the values are variable -// length strings, may use the length of the string as the charge for -// the string. -// -// A builtin cache implementation with a least-recently-used eviction -// policy is provided. Clients may use their own implementations if -// they want something more sophisticated (like scan-resistance, a -// custom eviction policy, variable cache sizing, etc.) - -#ifndef STORAGE_LEVELDB_INCLUDE_CACHE_H_ -#define STORAGE_LEVELDB_INCLUDE_CACHE_H_ - -#include <stdint.h> -#include "leveldb/slice.h" - -namespace leveldb { - -class Cache; - -// Create a new cache with a fixed size capacity. This implementation -// of Cache uses a least-recently-used eviction policy. -extern Cache* NewLRUCache(size_t capacity); - -class Cache { - public: - Cache() { } - - // Destroys all existing entries by calling the "deleter" - // function that was passed to the constructor. - virtual ~Cache(); - - // Opaque handle to an entry stored in the cache. - struct Handle { }; - - // Insert a mapping from key->value into the cache and assign it - // the specified charge against the total cache capacity. - // - // Returns a handle that corresponds to the mapping. The caller - // must call this->Release(handle) when the returned mapping is no - // longer needed. - // - // When the inserted entry is no longer needed, the key and - // value will be passed to "deleter". - virtual Handle* Insert(const Slice& key, void* value, size_t charge, - void (*deleter)(const Slice& key, void* value)) = 0; - - // If the cache has no mapping for "key", returns NULL. - // - // Else return a handle that corresponds to the mapping. The caller - // must call this->Release(handle) when the returned mapping is no - // longer needed. - virtual Handle* Lookup(const Slice& key) = 0; - - // Release a mapping returned by a previous Lookup(). - // REQUIRES: handle must not have been released yet. - // REQUIRES: handle must have been returned by a method on *this. - virtual void Release(Handle* handle) = 0; - - // Return the value encapsulated in a handle returned by a - // successful Lookup(). - // REQUIRES: handle must not have been released yet. - // REQUIRES: handle must have been returned by a method on *this. - virtual void* Value(Handle* handle) = 0; - - // If the cache contains entry for key, erase it. Note that the - // underlying entry will be kept around until all existing handles - // to it have been released. - virtual void Erase(const Slice& key) = 0; - - // Return a new numeric id. May be used by multiple clients who are - // sharing the same cache to partition the key space. Typically the - // client will allocate a new id at startup and prepend the id to - // its cache keys. - virtual uint64_t NewId() = 0; - - private: - void LRU_Remove(Handle* e); - void LRU_Append(Handle* e); - void Unref(Handle* e); - - struct Rep; - Rep* rep_; - - // No copying allowed - Cache(const Cache&); - void operator=(const Cache&); -}; - -} // namespace leveldb - -#endif // STORAGE_LEVELDB_INCLUDE_CACHE_H_ http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/leveldb-1.18/include/leveldb/comparator.h ---------------------------------------------------------------------- diff --git a/thirdparty/leveldb-1.18/include/leveldb/comparator.h b/thirdparty/leveldb-1.18/include/leveldb/comparator.h deleted file mode 100755 index a8d43ae..0000000 --- a/thirdparty/leveldb-1.18/include/leveldb/comparator.h +++ /dev/null @@ -1,66 +0,0 @@ -// Copyright (c) 2011 The LevelDB Authors. All rights reserved. -// Use of this source code is governed by a BSD-style license that can be -// found in the LICENSE file. See the AUTHORS file for names of contributors. - -#ifndef STORAGE_LEVELDB_INCLUDE_COMPARATOR_H_ -#define STORAGE_LEVELDB_INCLUDE_COMPARATOR_H_ - -#include <string> - -namespace leveldb { - -class Slice; - -// A Comparator object provides a total order across slices that are -// used as keys in an sstable or a database. A Comparator implementation -// must be thread-safe since leveldb may invoke its methods concurrently -// from multiple threads. -class Comparator { - public: - virtual ~Comparator(); - - // Three-way comparison. Returns value: - // < 0 iff "a" < "b", - // == 0 iff "a" == "b", - // > 0 iff "a" > "b" - virtual int Compare(const Slice& a, const Slice& b) const = 0; - - // The name of the comparator. Used to check for comparator - // mismatches (i.e., a DB created with one comparator is - // accessed using a different comparator. - // - // The client of this package should switch to a new name whenever - // the comparator implementation changes in a way that will cause - // the relative ordering of any two keys to change. - // - // Names starting with "leveldb." are reserved and should not be used - // by any clients of this package. - virtual const char* Name() const = 0; - - // Advanced functions: these are used to reduce the space requirements - // for internal data structures like index blocks. - - // If *start < limit, changes *start to a short string in [start,limit). - // Simple comparator implementations may return with *start unchanged, - // i.e., an implementation of this method that does nothing is correct. - virtual void FindShortestSeparator( - std::string* start, - const Slice& limit) const = 0; - - // Changes *key to a short string >= *key. - // Simple comparator implementations may return with *key unchanged, - // i.e., an implementation of this method that does nothing is correct. - virtual void FindShortSuccessor(std::string* key) const = 0; -}; - -// Return a builtin comparator that uses lexicographic byte-wise -// ordering. The result remains the property of this module and -// must not be deleted. -extern const Comparator* BytewiseComparator(); - -// memory leaks hunting -extern void UnsafeDeallocateBytewiseComparator(); - -} // namespace leveldb - -#endif // STORAGE_LEVELDB_INCLUDE_COMPARATOR_H_ http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/leveldb-1.18/include/leveldb/db.h ---------------------------------------------------------------------- diff --git a/thirdparty/leveldb-1.18/include/leveldb/db.h b/thirdparty/leveldb-1.18/include/leveldb/db.h deleted file mode 100755 index 4c169bf..0000000 --- a/thirdparty/leveldb-1.18/include/leveldb/db.h +++ /dev/null @@ -1,161 +0,0 @@ -// Copyright (c) 2011 The LevelDB Authors. All rights reserved. -// Use of this source code is governed by a BSD-style license that can be -// found in the LICENSE file. See the AUTHORS file for names of contributors. - -#ifndef STORAGE_LEVELDB_INCLUDE_DB_H_ -#define STORAGE_LEVELDB_INCLUDE_DB_H_ - -#include <stdint.h> -#include <stdio.h> -#include "leveldb/iterator.h" -#include "leveldb/options.h" - -namespace leveldb { - -// Update Makefile if you change these -static const int kMajorVersion = 1; -static const int kMinorVersion = 18; - -struct Options; -struct ReadOptions; -struct WriteOptions; -class WriteBatch; - -// Abstract handle to particular state of a DB. -// A Snapshot is an immutable object and can therefore be safely -// accessed from multiple threads without any external synchronization. -class Snapshot { - protected: - virtual ~Snapshot(); -}; - -// A range of keys -struct Range { - Slice start; // Included in the range - Slice limit; // Not included in the range - - Range() { } - Range(const Slice& s, const Slice& l) : start(s), limit(l) { } -}; - -// A DB is a persistent ordered map from keys to values. -// A DB is safe for concurrent access from multiple threads without -// any external synchronization. -class DB { - public: - // Open the database with the specified "name". - // Stores a pointer to a heap-allocated database in *dbptr and returns - // OK on success. - // Stores NULL in *dbptr and returns a non-OK status on error. - // Caller should delete *dbptr when it is no longer needed. - static Status Open(const Options& options, - const std::string& name, - DB** dbptr); - - DB() { } - virtual ~DB(); - - // Set the database entry for "key" to "value". Returns OK on success, - // and a non-OK status on error. - // Note: consider setting options.sync = true. - virtual Status Put(const WriteOptions& options, - const Slice& key, - const Slice& value) = 0; - - // Remove the database entry (if any) for "key". Returns OK on - // success, and a non-OK status on error. It is not an error if "key" - // did not exist in the database. - // Note: consider setting options.sync = true. - virtual Status Delete(const WriteOptions& options, const Slice& key) = 0; - - // Apply the specified updates to the database. - // Returns OK on success, non-OK on failure. - // Note: consider setting options.sync = true. - virtual Status Write(const WriteOptions& options, WriteBatch* updates) = 0; - - // If the database contains an entry for "key" store the - // corresponding value in *value and return OK. - // - // If there is no entry for "key" leave *value unchanged and return - // a status for which Status::IsNotFound() returns true. - // - // May return some other Status on an error. - virtual Status Get(const ReadOptions& options, - const Slice& key, std::string* value) = 0; - - // Return a heap-allocated iterator over the contents of the database. - // The result of NewIterator() is initially invalid (caller must - // call one of the Seek methods on the iterator before using it). - // - // Caller should delete the iterator when it is no longer needed. - // The returned iterator should be deleted before this db is deleted. - virtual Iterator* NewIterator(const ReadOptions& options) = 0; - - // Return a handle to the current DB state. Iterators created with - // this handle will all observe a stable snapshot of the current DB - // state. The caller must call ReleaseSnapshot(result) when the - // snapshot is no longer needed. - virtual const Snapshot* GetSnapshot() = 0; - - // Release a previously acquired snapshot. The caller must not - // use "snapshot" after this call. - virtual void ReleaseSnapshot(const Snapshot* snapshot) = 0; - - // DB implementations can export properties about their state - // via this method. If "property" is a valid property understood by this - // DB implementation, fills "*value" with its current value and returns - // true. Otherwise returns false. - // - // - // Valid property names include: - // - // "leveldb.num-files-at-level<N>" - return the number of files at level <N>, - // where <N> is an ASCII representation of a level number (e.g. "0"). - // "leveldb.stats" - returns a multi-line string that describes statistics - // about the internal operation of the DB. - // "leveldb.sstables" - returns a multi-line string that describes all - // of the sstables that make up the db contents. - virtual bool GetProperty(const Slice& property, std::string* value) = 0; - - // For each i in [0,n-1], store in "sizes[i]", the approximate - // file system space used by keys in "[range[i].start .. range[i].limit)". - // - // Note that the returned sizes measure file system space usage, so - // if the user data compresses by a factor of ten, the returned - // sizes will be one-tenth the size of the corresponding user data size. - // - // The results may not include the sizes of recently written data. - virtual void GetApproximateSizes(const Range* range, int n, - uint64_t* sizes) = 0; - - // Compact the underlying storage for the key range [*begin,*end]. - // In particular, deleted and overwritten versions are discarded, - // and the data is rearranged to reduce the cost of operations - // needed to access the data. This operation should typically only - // be invoked by users who understand the underlying implementation. - // - // begin==NULL is treated as a key before all keys in the database. - // end==NULL is treated as a key after all keys in the database. - // Therefore the following call will compact the entire database: - // db->CompactRange(NULL, NULL); - virtual void CompactRange(const Slice* begin, const Slice* end) = 0; - - private: - // No copying allowed - DB(const DB&); - void operator=(const DB&); -}; - -// Destroy the contents of the specified database. -// Be very careful using this method. -Status DestroyDB(const std::string& name, const Options& options); - -// If a DB cannot be opened, you may attempt to call this method to -// resurrect as much of the contents of the database as possible. -// Some data may be lost, so be careful when calling this function -// on a database that contains important information. -Status RepairDB(const std::string& dbname, const Options& options); - -} // namespace leveldb - -#endif // STORAGE_LEVELDB_INCLUDE_DB_H_ http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/leveldb-1.18/include/leveldb/dumpfile.h ---------------------------------------------------------------------- diff --git a/thirdparty/leveldb-1.18/include/leveldb/dumpfile.h b/thirdparty/leveldb-1.18/include/leveldb/dumpfile.h deleted file mode 100755 index 3f97fda..0000000 --- a/thirdparty/leveldb-1.18/include/leveldb/dumpfile.h +++ /dev/null @@ -1,25 +0,0 @@ -// Copyright (c) 2014 The LevelDB Authors. All rights reserved. -// Use of this source code is governed by a BSD-style license that can be -// found in the LICENSE file. See the AUTHORS file for names of contributors. - -#ifndef STORAGE_LEVELDB_INCLUDE_DUMPFILE_H_ -#define STORAGE_LEVELDB_INCLUDE_DUMPFILE_H_ - -#include <string> -#include "leveldb/env.h" -#include "leveldb/status.h" - -namespace leveldb { - -// Dump the contents of the file named by fname in text format to -// *dst. Makes a sequence of dst->Append() calls; each call is passed -// the newline-terminated text corresponding to a single item found -// in the file. -// -// Returns a non-OK result if fname does not name a leveldb storage -// file, or if the file cannot be read. -Status DumpFile(Env* env, const std::string& fname, WritableFile* dst); - -} // namespace leveldb - -#endif // STORAGE_LEVELDB_INCLUDE_DUMPFILE_H_ http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/48867732/thirdparty/leveldb-1.18/include/leveldb/env.h ---------------------------------------------------------------------- diff --git a/thirdparty/leveldb-1.18/include/leveldb/env.h b/thirdparty/leveldb-1.18/include/leveldb/env.h deleted file mode 100755 index 6c25c7c..0000000 --- a/thirdparty/leveldb-1.18/include/leveldb/env.h +++ /dev/null @@ -1,339 +0,0 @@ -// Copyright (c) 2011 The LevelDB Authors. All rights reserved. -// Use of this source code is governed by a BSD-style license that can be -// found in the LICENSE file. See the AUTHORS file for names of contributors. -// -// An Env is an interface used by the leveldb implementation to access -// operating system functionality like the filesystem etc. Callers -// may wish to provide a custom Env object when opening a database to -// get fine gain control; e.g., to rate limit file system operations. -// -// All Env implementations are safe for concurrent access from -// multiple threads without any external synchronization. - -#ifndef STORAGE_LEVELDB_INCLUDE_ENV_H_ -#define STORAGE_LEVELDB_INCLUDE_ENV_H_ - -#include <string> -#include <vector> -#include <thread> -#include <stdarg.h> -#include <stdint.h> -#include "leveldb/status.h" - -namespace leveldb { - -class FileLock; -class Logger; -class RandomAccessFile; -class SequentialFile; -class Slice; -class WritableFile; - -class Env { - public: - Env() { } - virtual ~Env(); - - // Return a default environment suitable for the current operating - // system. Sophisticated users may wish to provide their own Env - // implementation instead of relying on this default environment. - // - // The result of Default() belongs to leveldb and must never be deleted. - static Env* Default(); - - // use this call to deallocate memory when you know you no longer need it - // totally not thread safe but needed to prevent memory analyzers from crying - static void UnsafeDeallocate(); - - // needed if you want to set affinity or adjust thread priority - virtual std::thread::native_handle_type GetBackgroundThreadHandle() = 0; - - // Create a brand new sequentially-readable file with the specified name. - // On success, stores a pointer to the new file in *result and returns OK. - // On failure stores NULL in *result and returns non-OK. If the file does - // not exist, returns a non-OK status. - // - // The returned file will only be accessed by one thread at a time. - virtual Status NewSequentialFile(const std::string& fname, - SequentialFile** result) = 0; - - // Create a brand new random access read-only file with the - // specified name. On success, stores a pointer to the new file in - // *result and returns OK. On failure stores NULL in *result and - // returns non-OK. If the file does not exist, returns a non-OK - // status. - // - // The returned file may be concurrently accessed by multiple threads. - virtual Status NewRandomAccessFile(const std::string& fname, - RandomAccessFile** result) = 0; - - // Create an object that writes to a new file with the specified - // name. Deletes any existing file with the same name and creates a - // new file. On success, stores a pointer to the new file in - // *result and returns OK. On failure stores NULL in *result and - // returns non-OK. - // - // The returned file will only be accessed by one thread at a time. - virtual Status NewWritableFile(const std::string& fname, - WritableFile** result) = 0; - - // Returns true iff the named file exists. - virtual bool FileExists(const std::string& fname) = 0; - - // Store in *result the names of the children of the specified directory. - // The names are relative to "dir". - // Original contents of *results are dropped. - virtual Status GetChildren(const std::string& dir, - std::vector<std::string>* result) = 0; - - // Delete the named file. - virtual Status DeleteFile(const std::string& fname) = 0; - - // Create the specified directory. - virtual Status CreateDir(const std::string& dirname) = 0; - - // Delete the specified directory. - virtual Status DeleteDir(const std::string& dirname) = 0; - - // Store the size of fname in *file_size. - virtual Status GetFileSize(const std::string& fname, uint64_t* file_size) = 0; - - // Rename file src to target. - virtual Status RenameFile(const std::string& src, - const std::string& target) = 0; - - // Lock the specified file. Used to prevent concurrent access to - // the same db by multiple processes. On failure, stores NULL in - // *lock and returns non-OK. - // - // On success, stores a pointer to the object that represents the - // acquired lock in *lock and returns OK. The caller should call - // UnlockFile(*lock) to release the lock. If the process exits, - // the lock will be automatically released. - // - // If somebody else already holds the lock, finishes immediately - // with a failure. I.e., this call does not wait for existing locks - // to go away. - // - // May create the named file if it does not already exist. - virtual Status LockFile(const std::string& fname, FileLock** lock) = 0; - - // Release the lock acquired by a previous successful call to LockFile. - // REQUIRES: lock was returned by a successful LockFile() call - // REQUIRES: lock has not already been unlocked. - virtual Status UnlockFile(FileLock* lock) = 0; - - // Arrange to run "(*function)(arg)" once in a background thread. - // - // "function" may run in an unspecified thread. Multiple functions - // added to the same Env may run concurrently in different threads. - // I.e., the caller may not assume that background work items are - // serialized. - virtual void Schedule( - void (*function)(void* arg), - void* arg) = 0; - - // *path is set to a temporary directory that can be used for testing. It may - // or many not have just been created. The directory may or may not differ - // between runs of the same process, but subsequent calls will return the - // same directory. - virtual Status GetTestDirectory(std::string* path) = 0; - - // Create and return a log file for storing informational messages. - virtual Status NewLogger(const std::string& fname, Logger** result) = 0; - - // Returns the number of micro-seconds since some fixed point in time. Only - // useful for computing deltas of time. - virtual uint64_t NowMicros() = 0; - - // Sleep/delay the thread for the prescribed number of micro-seconds. - virtual void SleepForMicroseconds(int micros) = 0; - - private: - // No copying allowed - Env(const Env&); - void operator=(const Env&); -}; - -// A file abstraction for reading sequentially through a file -class SequentialFile { - public: - SequentialFile() { } - virtual ~SequentialFile(); - - // Read up to "n" bytes from the file. "scratch[0..n-1]" may be - // written by this routine. Sets "*result" to the data that was - // read (including if fewer than "n" bytes were successfully read). - // May set "*result" to point at data in "scratch[0..n-1]", so - // "scratch[0..n-1]" must be live when "*result" is used. - // If an error was encountered, returns a non-OK status. - // - // REQUIRES: External synchronization - virtual Status Read(size_t n, Slice* result, char* scratch) = 0; - - // Skip "n" bytes from the file. This is guaranteed to be no - // slower that reading the same data, but may be faster. - // - // If end of file is reached, skipping will stop at the end of the - // file, and Skip will return OK. - // - // REQUIRES: External synchronization - virtual Status Skip(uint64_t n) = 0; - - private: - // No copying allowed - SequentialFile(const SequentialFile&); - void operator=(const SequentialFile&); -}; - -// A file abstraction for randomly reading the contents of a file. -class RandomAccessFile { - public: - RandomAccessFile() { } - virtual ~RandomAccessFile(); - - // Read up to "n" bytes from the file starting at "offset". - // "scratch[0..n-1]" may be written by this routine. Sets "*result" - // to the data that was read (including if fewer than "n" bytes were - // successfully read). May set "*result" to point at data in - // "scratch[0..n-1]", so "scratch[0..n-1]" must be live when - // "*result" is used. If an error was encountered, returns a non-OK - // status. - // - // Safe for concurrent use by multiple threads. - virtual Status Read(uint64_t offset, size_t n, Slice* result, - char* scratch) const = 0; - - private: - // No copying allowed - RandomAccessFile(const RandomAccessFile&); - void operator=(const RandomAccessFile&); -}; - -// A file abstraction for sequential writing. The implementation -// must provide buffering since callers may append small fragments -// at a time to the file. -class WritableFile { - public: - WritableFile() { } - virtual ~WritableFile(); - - virtual Status Append(const Slice& data) = 0; - virtual Status Close() = 0; - virtual Status Flush() = 0; - virtual Status Sync() = 0; - - private: - // No copying allowed - WritableFile(const WritableFile&); - void operator=(const WritableFile&); -}; - -// An interface for writing log messages. -class Logger { - public: - Logger() { } - virtual ~Logger(); - - // Write an entry to the log file with the specified format. - virtual void Logv(const char* format, va_list ap) = 0; - - private: - // No copying allowed - Logger(const Logger&); - void operator=(const Logger&); -}; - - -// Identifies a locked file. -class FileLock { - public: - FileLock() { } - virtual ~FileLock(); - private: - // No copying allowed - FileLock(const FileLock&); - void operator=(const FileLock&); -}; - -// Log the specified data to *info_log if info_log is non-NULL. -extern void Log(Logger* info_log, const char* format, ...) -# if defined(__GNUC__) || defined(__clang__) - __attribute__((__format__ (__printf__, 2, 3))) -# endif - ; - -// A utility routine: write "data" to the named file. -extern Status WriteStringToFile(Env* env, const Slice& data, - const std::string& fname); - -// A utility routine: read contents of named file into *data -extern Status ReadFileToString(Env* env, const std::string& fname, - std::string* data); - -// An implementation of Env that forwards all calls to another Env. -// May be useful to clients who wish to override just part of the -// functionality of another Env. -class EnvWrapper : public Env { - public: - // Initialize an EnvWrapper that delegates all calls to *t - explicit EnvWrapper(Env* t) : target_(t) { } - virtual ~EnvWrapper(); - - // Return the target to which this Env forwards all calls - Env* target() const { return target_; } - - std::thread::native_handle_type GetBackgroundThreadHandle() { - return target_->GetBackgroundThreadHandle(); - } - - // The following text is boilerplate that forwards all methods to target() - Status NewSequentialFile(const std::string& f, SequentialFile** r) { - return target_->NewSequentialFile(f, r); - } - Status NewRandomAccessFile(const std::string& f, RandomAccessFile** r) { - return target_->NewRandomAccessFile(f, r); - } - Status NewWritableFile(const std::string& f, WritableFile** r) { - return target_->NewWritableFile(f, r); - } - bool FileExists(const std::string& f) { return target_->FileExists(f); } - Status GetChildren(const std::string& dir, std::vector<std::string>* r) { - return target_->GetChildren(dir, r); - } - Status DeleteFile(const std::string& f) { return target_->DeleteFile(f); } - Status CreateDir(const std::string& d) { return target_->CreateDir(d); } - Status DeleteDir(const std::string& d) { return target_->DeleteDir(d); } - Status GetFileSize(const std::string& f, uint64_t* s) { - return target_->GetFileSize(f, s); - } - Status RenameFile(const std::string& s, const std::string& t) { - return target_->RenameFile(s, t); - } - Status LockFile(const std::string& f, FileLock** l) { - return target_->LockFile(f, l); - } - Status UnlockFile(FileLock* l) { return target_->UnlockFile(l); } - void Schedule(void (*f)(void*), void* a) { - return target_->Schedule(f, a); - } - - virtual Status GetTestDirectory(std::string* path) { - return target_->GetTestDirectory(path); - } - virtual Status NewLogger(const std::string& fname, Logger** result) { - return target_->NewLogger(fname, result); - } - uint64_t NowMicros() { - return target_->NowMicros(); - } - void SleepForMicroseconds(int micros) { - target_->SleepForMicroseconds(micros); - } - private: - Env* target_; -}; - -} // namespace leveldb - -#endif // STORAGE_LEVELDB_INCLUDE_ENV_H_
