[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340614#comment-15340614
 ] 

Hudson commented on HBASE-16023:


SUCCESS: Integrated in HBase-1.3-IT #722 (See 
[https://builds.apache.org/job/HBase-1.3-IT/722/])
HBASE-16023 Fastpath for the FIFO rpcscheduler AMENDMENT (stack: rev 
6c519f36210f9f917e8cf40c96223cac755bb272)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/FifoWithFastPathBalancedQueueRpcExecutor.java


> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.addenum.patch, 
> HBASE-16023.branch-1.001.patch, hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15339517#comment-15339517
 ] 

Hudson commented on HBASE-16023:


FAILURE: Integrated in HBase-1.4 #227 (See 
[https://builds.apache.org/job/HBase-1.4/227/])
HBASE-16023 Fastpath for the FIFO rpcscheduler AMENDMENT (stack: rev 
d34e65327e77338d24ace159b0756836fd91406d)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/FifoWithFastPathBalancedQueueRpcExecutor.java


> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.addenum.patch, 
> HBASE-16023.branch-1.001.patch, hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15339372#comment-15339372
 ] 

Hudson commented on HBASE-16023:


FAILURE: Integrated in HBase-Trunk_matrix #1081 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/1081/])
HBASE-16023 Fastpath for the FIFO rpcscheduler AMENDMENT (stack: rev 
01684c3417861350a0c82cd2faba039f65ca7a79)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/FifoWithFastPathBalancedQueueRpcExecutor.java


> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.addenum.patch, 
> HBASE-16023.branch-1.001.patch, hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15339313#comment-15339313
 ] 

Hudson commented on HBASE-16023:


SUCCESS: Integrated in HBase-1.3 #748 (See 
[https://builds.apache.org/job/HBase-1.3/748/])
HBASE-16023 Fastpath for the FIFO rpcscheduler AMENDMENT (stack: rev 
6c519f36210f9f917e8cf40c96223cac755bb272)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/FifoWithFastPathBalancedQueueRpcExecutor.java


> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.addenum.patch, 
> HBASE-16023.branch-1.001.patch, hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15339112#comment-15339112
 ] 

stack commented on HBASE-16023:
---

Oh, I tried the addendum on cluster before commit just to be sure I hadn't 
broken anything.

> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.addenum.patch, 
> HBASE-16023.branch-1.001.patch, hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15339111#comment-15339111
 ] 

stack commented on HBASE-16023:
---

Pushed addendum to branch-1.3+ to address review by [~ikeda]

commit 01684c3417861350a0c82cd2faba039f65ca7a79
Author: stack 
Date:   Mon Jun 20 08:28:39 2016 +0100

HBASE-16023 Fastpath for the FIFO rpcscheduler AMENDMENT

Thanks for the extra review. Nice.

> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.addenum.patch, 
> HBASE-16023.branch-1.001.patch, hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[Hiroshi Ikeda (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338883#comment-15338883
 ] 

Hiroshi Ikeda commented on HBASE-16023:
---

Other worrying things:

{code}
protected CallRunner getCallRunner() throws InterruptedException {
...skip...
if (this.fastPathHandlerStack != null) {
  this.fastPathHandlerStack.push(this);
  this.semaphore.acquire();
  cr = this.loadedCallRunner;
} else {
{code}

The loadedCallRunner should be set to null after passing, otherwise that would 
become sort of memory leak (in low load).


It is not a bug but
{code}
private Semaphore semaphore = new Semaphore(1);
...skip...
FastPathHandler(String name, double handlerFailureThreshhold, 
BlockingQueue q,
...skip...
  this.semaphore.drainPermits();
{code}

It is better to just create an instance of Semaphore with no permit.

> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.addenum.patch, 
> HBASE-16023.branch-1.001.patch, hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337580#comment-15337580
 ] 

Hudson commented on HBASE-16023:


FAILURE: Integrated in HBase-1.4 #224 (See 
[https://builds.apache.org/job/HBase-1.4/224/])
HBASE-16023 Fastpath for the FIFO rpcscheduler Adds an executor that (stack: 
rev 411e3cdb6d47e622bf3aaec1c19380ba42d22b21)
* hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcExecutor.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestSimpleRpcScheduler.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/BalancedQueueRpcExecutor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/FifoWithFastPathBalancedQueueRpcExecutor.java


> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.addenum.patch, 
> HBASE-16023.branch-1.001.patch, hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[Mikhail Antonov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337527#comment-15337527
 ] 

Mikhail Antonov commented on HBASE-16023:
-

ah! somehow I was pretty sure this was pushed a day ago.

> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.addenum.patch, 
> HBASE-16023.branch-1.001.patch, hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337510#comment-15337510
 ] 

stack commented on HBASE-16023:
---

Pushed this patch to branch-1.3 sir.

> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.addenum.patch, 
> HBASE-16023.branch-1.001.patch, hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337453#comment-15337453
 ] 

Hudson commented on HBASE-16023:


SUCCESS: Integrated in HBase-1.3-IT #715 (See 
[https://builds.apache.org/job/HBase-1.3-IT/715/])
HBASE-16023 Fastpath for the FIFO rpcscheduler Adds an executor that (stack: 
rev 0f8debc4c4b4403c5fff61e1ae7338d80831d988)
* hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java


> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.addenum.patch, 
> HBASE-16023.branch-1.001.patch, hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337451#comment-15337451
 ] 

Hudson commented on HBASE-16023:


SUCCESS: Integrated in HBase-1.3 #745 (See 
[https://builds.apache.org/job/HBase-1.3/745/])
HBASE-16023 Fastpath for the FIFO rpcscheduler Adds an executor that (stack: 
rev 0f8debc4c4b4403c5fff61e1ae7338d80831d988)
* hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java


> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.addenum.patch, 
> HBASE-16023.branch-1.001.patch, hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[Mikhail Antonov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337332#comment-15337332
 ] 

Mikhail Antonov commented on HBASE-16023:
-

[~stack] pushed what?

> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.addenum.patch, 
> HBASE-16023.branch-1.001.patch, hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337316#comment-15337316
 ] 

stack commented on HBASE-16023:
---

I made HBASE-16063. 

> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.addenum.patch, 
> HBASE-16023.branch-1.001.patch, hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337286#comment-15337286
 ] 

stack commented on HBASE-16023:
---

[~ikeda] Thank you. That makes sense. If all handlers went through step 3 
before we got to 4, it would be an interesting situation. Let me make a new 
issue w/ this in it.

> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.addenum.patch, 
> HBASE-16023.branch-1.001.patch, hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337272#comment-15337272
 ] 

stack commented on HBASE-16023:
---

[~mantonov] I just pushed this to branch-1.3 and branch-1. FYI.

> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.addenum.patch, 
> HBASE-16023.branch-1.001.patch, hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337266#comment-15337266
 ] 

stack commented on HBASE-16023:
---

Thanks [~chenheng]

> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.addenum.patch, 
> HBASE-16023.branch-1.001.patch, hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[Heng Chen (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331383#comment-15331383
 ] 

Heng Chen commented on HBASE-16023:
---

Push addendum patch to master

> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.addenum.patch, 
> HBASE-16023.branch-1.001.patch, hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[Hiroshi Ikeda (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331172#comment-15331172
 ] 

Hiroshi Ikeda commented on HBASE-16023:
---

Sorry my lack of explanation.

An concrete example of the race condition:

1. Worker checks no task.
2. Reader checks no ready handler.
3. Worker pushes itself as a ready handler and waits on the semaphore.
4. Reader queues a task to the queue, without directly passing it to the ready 
handler nor releasing the semaphore.

(1,3) and (2,4) should be exclusively executed. That depends on luck, and it 
might be not severe(?)


> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.branch-1.001.patch, 
> hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331135#comment-15331135
 ] 

stack commented on HBASE-16023:
---

bq. Checking no task and waiting on the semaphore is not atomic, and it is 
possible that a reader queues a task while a worker are going to waiting on the 
semaphore.

Even if the Reader semaphore release runs before we get to the acquire, thats 
ok? The acquire will just not block when called? Let me know if I misunderstand.

> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.branch-1.001.patch, 
> hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[Hiroshi Ikeda (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331123#comment-15331123
 ] 

Hiroshi Ikeda commented on HBASE-16023:
---

{code}
protected CallRunner getCallRunner() throws InterruptedException {
  // Get a callrunner if one in the Q.
  CallRunner cr = this.q.poll();
  if (cr == null) {
// Else, if a fastPathHandlerStack present and no callrunner in Q, 
register ourselves for
// the fastpath handoff done via fastPathHandlerStack.
if (this.fastPathHandlerStack != null) {
  this.fastPathHandlerStack.push(this);
  this.semaphore.acquire();
{code}

Checking no task and waiting on the semaphore is not atomic, and it is possible 
that a reader queues a task while a worker are going to waiting on the 
semaphore.

There might be no simple way to implement without a flaw.

> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.branch-1.001.patch, 
> hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[Mikhail Antonov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330691#comment-15330691
 ] 

Mikhail Antonov commented on HBASE-16023:
-

[~stack] looks good to me - yeah, let's get it in branch-1.3. Thanks!

> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Attachments: HBASE-16023.branch-1.001.patch, 
> hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330559#comment-15330559
 ] 

Hadoop QA commented on HBASE-16023:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 
21s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 43s 
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s 
{color} | {color:green} branch-1 passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 
7s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
19s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
56s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s 
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s 
{color} | {color:green} branch-1 passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 38s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s 
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 31s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 
0s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
11m 58s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 9s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 37s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s 
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 88m 54s {color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
17s {color} | {color:green} Patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 119m 10s {color} 
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.coprocessor.TestRowProcessorEndpoint |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12810581/HBASE-16023.branch-1.001.patch
 |
| JIRA Issue | HBASE-16023 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | branch-1 / 286934f |
| Default Java | 1.7.0_79 |
| Multi-JDK versions |  /home/jenkins/tools/java/jdk1.8.0:1.8.0 
/usr/local/jenkins/java/jdk1.7.0_79:1.7.0_79 |
| findbugs | 

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330203#comment-15330203
 ] 

stack commented on HBASE-16023:
---

[~mantonov] for branch-1.3?

> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Attachments: HBASE-16023.branch-1.001.patch, 
> hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[ramkrishna.s.vasudevan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330080#comment-15330080
 ] 

ramkrishna.s.vasudevan commented on HBASE-16023:


Nice graphs and nice findings.

> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance
>Reporter: stack
>Assignee: stack
> Attachments: hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)