[
https://issues.apache.org/jira/browse/HBASE-5162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288832#comment-14288832
]
Hudson commented on HBASE-5162:
-------------------------------
SUCCESS: Integrated in HBase-1.1 #99 (See
[https://builds.apache.org/job/HBase-1.1/99/])
HBASE-12730 Backport HBASE-5162 (Basic client pushback mechanism) to branch-1
(apurtell: rev 04a003d6a2b628bc551a55ae556c1b44485e3022)
*
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestDelayingRunner.java
* hbase-protocol/src/main/protobuf/Client.proto
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/DelayingRunner.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/MultiAction.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RegionCoprocessorRpcChannel.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionManager.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/backoff/ClientBackoffPolicyFactory.java
*
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestClientExponentialBackoff.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/StatsTrackingRpcRetryingCaller.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/backoff/ClientBackoffPolicy.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerFactory.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/Result.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ServerStatisticTracker.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/ResultStatsUtil.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClusterConnection.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/backoff/ExponentialClientBackoffPolicy.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALEditsReplaySink.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/backoff/ServerStatistics.java
*
hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionAdapter.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestClientPushback.java
> Basic client pushback mechanism
> -------------------------------
>
> Key: HBASE-5162
> URL: https://issues.apache.org/jira/browse/HBASE-5162
> Project: HBase
> Issue Type: New Feature
> Affects Versions: 0.92.0
> Reporter: Jean-Daniel Cryans
> Assignee: Jesse Yates
> Fix For: 2.0.0
>
> Attachments: 5162-addendum2.txt, hbase-5162-branch-1-v0.patch,
> hbase-5162-trunk-addendum.patch, hbase-5162-trunk-v0.patch,
> hbase-5162-trunk-v1.patch, hbase-5162-trunk-v10.patch,
> hbase-5162-trunk-v11.patch, hbase-5162-trunk-v12-committed.patch,
> hbase-5162-trunk-v2.patch, hbase-5162-trunk-v3.patch,
> hbase-5162-trunk-v4.patch, hbase-5162-trunk-v5.patch,
> hbase-5162-trunk-v6.patch, hbase-5162-trunk-v7.patch,
> hbase-5162-trunk-v8.patch, java_HBASE-5162.patch
>
>
> The current blocking we do when we are close to some limits (memstores over
> the multiplier factor, too many store files, global memstore memory) is bad,
> too coarse and confusing. After hitting HBASE-5161, it really becomes obvious
> that we need something better.
> I did a little brainstorm with Stack, we came up quickly with two solutions:
> - Send some exception to the client, like OverloadedException, that's thrown
> when some situation happens like getting past the low memory barrier. It
> would be thrown when the client gets a handler and does some check while
> putting or deleting. The client would treat this a retryable exception but
> ideally wouldn't check .META. for a new location. It could be fancy and have
> multiple levels of pushback, like send the exception to 25% of the clients,
> and then go up if the situation persists. Should be "easy" to implement but
> we'll be using a lot more IO to send the payload over and over again (but at
> least it wouldn't sit in the RS's memory).
> - Send a message alongside a successful put or delete to tell the client to
> slow down a little, this way we don't have to do back and forth with the
> payload between the client and the server. It's a cleaner (I think) but more
> involved solution.
> In every case the RS should do very obvious things to notify the operators of
> this situation, through logs, web UI, metrics, etc.
> Other ideas?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
