[ https://issues.apache.org/jira/browse/HBASE-5162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14249462#comment-14249462 ]
stack commented on HBASE-5162: ------------------------------ The test this patch adds has been failing. You lot on it (i'm sure you are!) > Basic client pushback mechanism > ------------------------------- > > Key: HBASE-5162 > URL: https://issues.apache.org/jira/browse/HBASE-5162 > Project: HBase > Issue Type: New Feature > Affects Versions: 0.92.0 > Reporter: Jean-Daniel Cryans > Assignee: Jesse Yates > Fix For: 1.0.0, 2.0.0 > > Attachments: hbase-5162-branch-1-v0.patch, > hbase-5162-trunk-addendum.patch, hbase-5162-trunk-v0.patch, > hbase-5162-trunk-v1.patch, hbase-5162-trunk-v10.patch, > hbase-5162-trunk-v11.patch, hbase-5162-trunk-v12-committed.patch, > hbase-5162-trunk-v2.patch, hbase-5162-trunk-v3.patch, > hbase-5162-trunk-v4.patch, hbase-5162-trunk-v5.patch, > hbase-5162-trunk-v6.patch, hbase-5162-trunk-v7.patch, > hbase-5162-trunk-v8.patch, java_HBASE-5162.patch > > > The current blocking we do when we are close to some limits (memstores over > the multiplier factor, too many store files, global memstore memory) is bad, > too coarse and confusing. After hitting HBASE-5161, it really becomes obvious > that we need something better. > I did a little brainstorm with Stack, we came up quickly with two solutions: > - Send some exception to the client, like OverloadedException, that's thrown > when some situation happens like getting past the low memory barrier. It > would be thrown when the client gets a handler and does some check while > putting or deleting. The client would treat this a retryable exception but > ideally wouldn't check .META. for a new location. It could be fancy and have > multiple levels of pushback, like send the exception to 25% of the clients, > and then go up if the situation persists. Should be "easy" to implement but > we'll be using a lot more IO to send the payload over and over again (but at > least it wouldn't sit in the RS's memory). > - Send a message alongside a successful put or delete to tell the client to > slow down a little, this way we don't have to do back and forth with the > payload between the client and the server. It's a cleaner (I think) but more > involved solution. > In every case the RS should do very obvious things to notify the operators of > this situation, through logs, web UI, metrics, etc. > Other ideas? -- This message was sent by Atlassian JIRA (v6.3.4#6332)