Dear Wiki user, You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.
The following page has been changed by JimKellerman: http://wiki.apache.org/lucene-hadoop/Hbase/HbaseArchitecture ------------------------------------------------------------------------------ * [#ipcclientmaster Client to Master Messages] * [#ipcclienttablet Client to Tablet Server Messages] * [#ipcmastertablet Master to Tablet Server Messages] + * [#ipctabletmaster Tablet Server to Master Messages] * [#clientlib Client Library] * [#api API] * [#other Other] @@ -300, +301 @@ * When a tablet is "brand new", all reads can be satisfied from the memtable. * When the memtable gets too large ''(how large?)'', the memtable is written to a new SSTable. This is a ''minor'' compaction. - ''Since this is not a tablet split, this implies that information about the new SSTable is not written to the METADATA table. Is this correct? Maybe not because a minor compaction does write a new redo point into the METADATA table and the redo point could contain a pointer to the new SSTable.'' + ''Minor and merging compactions write a new redo point into the METADATA table. The redo point contains information about the new SSTables.'' * Reads * Checks that the request is well-formed @@ -331, +332 @@ * Block Cache: caches blocks read from the SSTables * Bloom Filter * Optional in-memory structure that reduces disk access by - * API - {{{ - LoadTablet() - }}} [[Anchor(sstable)]] = SSTable = @@ -378, +375 @@ * Root tablet stores the location of all tablets in a METADATA table * The "root tablet" is just the first tablet in the METADATA table, it is never split - ''I really don't like this definition (and I know it is from the Bigtable paper). Aside from the fact that it is never split, the root tablet is special in another way: it is the metadata table for the METADATA table.'' ''It '''does''' however have the same format as the rest of the METADATA table.'' + ''I really don't like this definition (and I know it is from the Bigtable paper). Aside from the fact that it is never split, the root tablet is special in another way: it is the metadata table for the METADATA table.'' ''It '''does''' however have the same format as the rest of the METADATA table.'' * The row key is a combination of the table name, column or map name and the __'''first row key'''__ of the tablet. @@ -388, +385 @@ 1. ''If the first key in a tablet is stored instead, the minimum value could be represented as the empty string "" and a simple string comparison between a key and the first key would result in key > first key.'' If you represented the maximum row key as the empty string, that would require a special case instead of just a simple compare.'' 1. ''If the keys go from 1 to n, when the tablet is split, the first tablet still has the row key "" as its first key and the second tablet has a first row key of n/2. So if a key is presented and its value is < n/2, you know that if it exists, it is in the first tablet and if the value >= n/2 it is in the second tablet.'' + + ''If it turns out that there is a good reason that the last row key should be stored instead of the first row key, then this can be changed in the future.'' * The "location" map has the ''InMemory'' tuning parameter set * Each row stores approximately 1KB of data in memory @@ -456, +455 @@ [[Anchor(ipcclienttablet)]] == Client to TabletServer Messages == + * read (row-key, column-name, [column-key (for maps)], [timestamp]) returns value + * for maps, if no column-key is specified, returns the entire map for the specified row key + * ''Question: Is there a need to split up large return values into multiple messages?'' + * The table name is implied since this message is sent directly to the tablet server for the specified row-key + + * scan (starting-row-key, column-name, [starting-timestamp]) returns a tablet block size of data for the specified column and starting point. + + * write (row-key, column-name, [column-key (for maps)], timestamp, value) + + * batchUpdate (row-key, List<parameters>, timestamp) + * batchUpdates can only be used when updating multiple values in a map for a specific row key or if the tablets for all the specified columns are all served by the same tablet server. + * the parameter list consists of column name - value pairs (for columns) or column-name - column-key - value triples (for maps) + + * incrementCounter (row-key, column-name, timestamp, increment) + [[Anchor(ipcmastertablet)]] == Master to Tablet Server Messages == + + * ''Note: the master uses the client API to read METADATA tablets, so no special messages are required for this activity.'' + + * "Are you ok?" + * response (indicates tablet server alive) + * if the tablet server is serving any tablets, the response will also include a list of tablets being served. + + * start serving tablets (list-of-tablets-to-serve) + * response indicates tablets being served and any that it failed to start serving + + * major compaction (tablet) + * response indicates compaction complete + + * stop serving tablets (list-of-tablets-to-stop-serving) + * response indicates that tablets are no longer being served + + * shut down + * response indicates that shutdown was successful + + [[Anchor(ipctabletmaster)]] + == Tablet Server to Master Messages == + + * tablet split (old-tablet, new-tablet-1, new-tablet-2) [[Anchor(clientlib)]] = Client Library =
