Repository: hbase Updated Branches: refs/heads/master c3685760f -> 6ee2dcf48
HBASE-17089 Add doc on experience running with hedged reads Project: http://git-wip-us.apache.org/repos/asf/hbase/repo Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/6ee2dcf4 Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/6ee2dcf4 Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/6ee2dcf4 Branch: refs/heads/master Commit: 6ee2dcf480dd95877a20e33086a020eb1a19e41f Parents: c368576 Author: Michael Stack <st...@apache.org> Authored: Mon Nov 14 10:27:58 2016 -0800 Committer: Michael Stack <st...@apache.org> Committed: Mon Nov 14 10:28:09 2016 -0800 ---------------------------------------------------------------------- src/main/asciidoc/_chapters/performance.adoc | 12 +++++++++++- src/main/asciidoc/_chapters/protobuf.adoc | 5 +++-- 2 files changed, 14 insertions(+), 3 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hbase/blob/6ee2dcf4/src/main/asciidoc/_chapters/performance.adoc ---------------------------------------------------------------------- diff --git a/src/main/asciidoc/_chapters/performance.adoc b/src/main/asciidoc/_chapters/performance.adoc index 5f27640..bc8b48a 100644 --- a/src/main/asciidoc/_chapters/performance.adoc +++ b/src/main/asciidoc/_chapters/performance.adoc @@ -156,6 +156,10 @@ See <<block.cache>> See <<recommended_configurations>>. +[[perf.99th.percentile]] +=== Improving the 99th Percentile +Try link:[hedged_reads]. + [[perf.compactions.and.splits]] === Managing Compactions @@ -751,9 +755,10 @@ Most people should leave this alone. Default = 7, or can collapse to at least 1/128th of original size. See the _Development Process_ section of the document link:https://issues.apache.org/jira/secure/attachment/12444007/Bloom_Filters_in_HBase.pdf[BloomFilters in HBase] for more on what this option means. +[[hedged.reads]] === Hedged Reads -Hedged reads are a feature of HDFS, introduced in link:https://issues.apache.org/jira/browse/HDFS-5776[HDFS-5776]. +Hedged reads are a feature of HDFS, introduced in Hadoop 2.4.0 with link:https://issues.apache.org/jira/browse/HDFS-5776[HDFS-5776]. Normally, a single thread is spawned for each read request. However, if hedged reads are enabled, the client waits some configurable amount of time, and if the read does not return, the client spawns a second read request, against a different block replica of the same data. Whichever read returns first is used, and the other read request is discarded. @@ -790,6 +795,11 @@ See <<hbase_metrics>> for more information. * hedgeReadOpsWin - the number of times the hedged read thread was faster than the original thread. This could indicate that a given RegionServer is having trouble servicing requests. +Concerns while running with hedged reads enabled include: + +* They may lead to network congestion. See link:https://issues.apache.org/jira/browse/HBASE-17083[HBASE-17083] +* Make sure you set the thread pool large enough so as blocking on the pool does not become a bottleneck (Again see link:https://issues.apache.org/jira/browse/HBASE-17083[HBASE-17083]) + [[perf.deleting]] == Deleting from HBase http://git-wip-us.apache.org/repos/asf/hbase/blob/6ee2dcf4/src/main/asciidoc/_chapters/protobuf.adoc ---------------------------------------------------------------------- diff --git a/src/main/asciidoc/_chapters/protobuf.adoc b/src/main/asciidoc/_chapters/protobuf.adoc index fa63127..1c2cc47 100644 --- a/src/main/asciidoc/_chapters/protobuf.adoc +++ b/src/main/asciidoc/_chapters/protobuf.adoc @@ -27,6 +27,8 @@ :icons: font :experimental: + +== Protobuf HBase uses Google's link:http://protobuf.protobufs[protobufs] wherever it persists metadata -- in the tail of hfiles or Cells written by HBase into the system hbase;meta table or when HBase writes znodes @@ -36,12 +38,11 @@ Interfaces (Services) we expose to clients, for example the `Admin` and `Client` Interfaces that the RegionServer fields, or specifying the arbitrary extensions added by developers via our xref:cp[Coprocessor Endpoint] mechanism. + In this chapter we go into detail for developers who are looking to understand better how it all works. This chapter is of particular use to those who would amend or extend HBase functionality. -== Protobuf - With protobuf, you describe serializations and services in a `.protos` file. You then feed these descriptors to a protobuf tool, the `protoc` binary, to generate classes that can marshall and unmarshall the described serializations
