[jira] [Comment Edited] (HBASE-15921) Add first AsyncTable impl and create TableImpl based on it
[ https://issues.apache.org/jira/browse/HBASE-15921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15508442#comment-15508442 ] Heng Chen edited comment on HBASE-15921 at 9/21/16 2:12 AM: The patch [~carp84] supplied missed some thing but i think it has been implemented in AP 1. How to deal with batch? For batch puts, how to control traffic? 2. How to deal with replica calls? 3. How to deal with retry and different Exception handling? And after you deal with these things, It is just another AsyncProcess i think. was (Author: chenheng): The patch [~carp84] supplied missed some thing but i think it has been implemented in AP 1. How to deal with batch? For batch puts, how to control traffic? 2. How to deal with replica calls? 3. How to deal with retry and different Exception handling? > Add first AsyncTable impl and create TableImpl based on it > -- > > Key: HBASE-15921 > URL: https://issues.apache.org/jira/browse/HBASE-15921 > Project: HBase > Issue Type: Improvement >Reporter: Jurriaan Mous >Assignee: Jurriaan Mous > Attachments: HBASE-15921.demo.patch, HBASE-15921.patch, > HBASE-15921.v1.patch > > > First we create an AsyncTable interface with implementation without the Scan > functionality. Those will land in a separate patch since they need a refactor > of existing scans. > Also added is a new TableImpl to replace HTable. It uses the AsyncTableImpl > internally and should be a bit faster because it does jump through less hoops > to do ProtoBuf transportation. This way we can run all existing tests on the > AsyncTableImpl to guarantee its quality. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-15921) Add first AsyncTable impl and create TableImpl based on it
[ https://issues.apache.org/jira/browse/HBASE-15921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15489255#comment-15489255 ] Duo Zhang edited comment on HBASE-15921 at 9/14/16 3:44 AM: I think we need to redesign the retry and callable logic since now we have callback support. [~carp84] I know you guys are implementing async table on your own HBase branch, can you share your experience? Thanks. was (Author: apache9): I think we need to redesign the retry and callable logic since now we have callback support. [~liyu] I know you guys are implementing async table on your own HBase branch, can you share your experience? Thanks. > Add first AsyncTable impl and create TableImpl based on it > -- > > Key: HBASE-15921 > URL: https://issues.apache.org/jira/browse/HBASE-15921 > Project: HBase > Issue Type: Improvement >Reporter: Jurriaan Mous >Assignee: Jurriaan Mous > Attachments: HBASE-15921.patch, HBASE-15921.v1.patch > > > First we create an AsyncTable interface with implementation without the Scan > functionality. Those will land in a separate patch since they need a refactor > of existing scans. > Also added is a new TableImpl to replace HTable. It uses the AsyncTableImpl > internally and should be a bit faster because it does jump through less hoops > to do ProtoBuf transportation. This way we can run all existing tests on the > AsyncTableImpl to guarantee its quality. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-15921) Add first AsyncTable impl and create TableImpl based on it
[
https://issues.apache.org/jira/browse/HBASE-15921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15486656#comment-15486656
]
Duo Zhang edited comment on HBASE-15921 at 9/13/16 8:36 AM:
In fact, we do not need a thread pool to execute the request with async rpc
stub(maybe we still need a thread to trigger retry), so the implementation will
be greatly different... The whole {{RetryingCallable}} should be redesigned and
reimplemented...
was (Author: apache9):
In fact, we do not need to thread pool to execute the request with async rpc
stub(maybe we still need a thread to trigger retry), so the implementation will
be greatly different... The whole {{RetryingCallable}} should be redesigned and
reimplemented...
> Add first AsyncTable impl and create TableImpl based on it
> --
>
> Key: HBASE-15921
> URL: https://issues.apache.org/jira/browse/HBASE-15921
> Project: HBase
> Issue Type: Improvement
>Reporter: Jurriaan Mous
>Assignee: Jurriaan Mous
> Attachments: HBASE-15921.patch, HBASE-15921.v1.patch
>
>
> First we create an AsyncTable interface with implementation without the Scan
> functionality. Those will land in a separate patch since they need a refactor
> of existing scans.
> Also added is a new TableImpl to replace HTable. It uses the AsyncTableImpl
> internally and should be a bit faster because it does jump through less hoops
> to do ProtoBuf transportation. This way we can run all existing tests on the
> AsyncTableImpl to guarantee its quality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-15921) Add first AsyncTable impl and create TableImpl based on it
[ https://issues.apache.org/jira/browse/HBASE-15921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423827#comment-15423827 ] Duo Zhang edited comment on HBASE-15921 at 8/17/16 6:39 AM: OK, you use it in this patch... I left a comment on rb, I do not think it is a good idea to jump over the stub layer of protobuf to call a RpcChannel directly, the method name mapping is fragile, and we also bypass the type check which is not good... Thanks. was (Author: apache9): OK, you use it in this patch... I left a comment on rb, I do not think it is a good idea to jump over the stub layer of protobuf to call a RpcChannel directly, the method name mapping is fragile, and we also eliminate the type check... Thanks. > Add first AsyncTable impl and create TableImpl based on it > -- > > Key: HBASE-15921 > URL: https://issues.apache.org/jira/browse/HBASE-15921 > Project: HBase > Issue Type: Improvement >Reporter: Jurriaan Mous >Assignee: Jurriaan Mous > Attachments: HBASE-15921.patch, HBASE-15921.v1.patch > > > First we create an AsyncTable interface with implementation without the Scan > functionality. Those will land in a separate patch since they need a refactor > of existing scans. > Also added is a new TableImpl to replace HTable. It uses the AsyncTableImpl > internally and should be a bit faster because it does jump through less hoops > to do ProtoBuf transportation. This way we can run all existing tests on the > AsyncTableImpl to guarantee its quality. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-15921) Add first AsyncTable impl and create TableImpl based on it
[
https://issues.apache.org/jira/browse/HBASE-15921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15311070#comment-15311070
]
Jurriaan Mous edited comment on HBASE-15921 at 6/1/16 8:42 PM:
---
{quote}
This has to be this way? (Lots of abstract classes...)
public abstract class AsyncRegionServerCallable extends
AbstractRegionServerCallable
It probably has to be given you are coming into a convoluted hierarchy that has
accreted over a long period of time. Was just wondering if could have a
shallower hierarchy. No issue if can't be done easily... or has to wait till
later after you've gotten your async client in.
Or, you just moved this existing class out to its own file?
{quote}
I have to add the abstract classes right now to share code between the sync and
async variants. As usage of the sync variants are replaced with async variants
we can remove the abstract versions again.
bq. AsyncTable returns Future only? Not CompletableFuture? Consumers won't be
able to consume AsyncTable in an event-driven way? We need a callback?
This is indeed as [~Apache9] mentions the Netty and now also HBase variant with
event driven methods.
bq. Why we let out EventExecutor? Especially given it a netty class. Can we
contain the fact that the implementation is betty-based?
At the moment I use the event executors for (new/failed/succes) promise
creation. I can take a fresh look at it to see if I can contain it more but am
not sure if it is possible.
bq. In successful future it has a class comment "25 * A Failed Response
future"
Will fix it.
bq. Any chance of a note on how the PromiseKeepers work?
Yes I will. It is the async way to combine multiple calls. So it replaces the
batch callback in places.
bq. The replacement of HTable by TableImpl comes later?
It is already replaced at getTable in Connection in this patch. I want to make
sure the performance is ok and then I find a solution for the remaining HTable
refs.
Created review: https://reviews.apache.org/r/48152/
was (Author: jurmous):
{quote}
This has to be this way? (Lots of abstract classes...)
public abstract class AsyncRegionServerCallable extends
AbstractRegionServerCallable
It probably has to be given you are coming into a convoluted hierarchy that has
accreted over a long period of time. Was just wondering if could have a
shallower hierarchy. No issue if can't be done easily... or has to wait till
later after you've gotten your async client in.
Or, you just moved this existing class out to its own file?
{quote}
I have to add the abstract classes right now to share code between the sync and
async variants. As usage of the sync variants are replaced with async variants
we can remove the abstract versions again.
bq. AsyncTable returns Future only? Not CompletableFuture? Consumers won't be
able to consume AsyncTable in an event-driven way? We need a callback?
This is indeed as [~Apache9] mentions the Netty variant with event driven
methods.
bq. Why we let out EventExecutor? Especially given it a netty class. Can we
contain the fact that the implementation is betty-based?
At the moment I use the event executors for (new/failed/succes) promise
creation. I can take a fresh look at it to see if I can contain it more but am
not sure if it is possible.
bq. In successful future it has a class comment "25 * A Failed Response
future"
Will fix it.
bq. Any chance of a note on how the PromiseKeepers work?
Yes I will. It is the async way to combine multiple calls. So it replaces the
batch callback in places.
bq. The replacement of HTable by TableImpl comes later?
It is already replaced at getTable in Connection in this patch. I want to make
sure the performance is ok and then I find a solution for the remaining HTable
refs.
Created review: https://reviews.apache.org/r/48152/
> Add first AsyncTable impl and create TableImpl based on it
> --
>
> Key: HBASE-15921
> URL: https://issues.apache.org/jira/browse/HBASE-15921
> Project: HBase
> Issue Type: Improvement
>Reporter: Jurriaan Mous
>Assignee: Jurriaan Mous
> Attachments: HBASE-15921.patch, HBASE-15921.v1.patch
>
>
> First we create an AsyncTable interface with implementation without the Scan
> functionality. Those will land in a separate patch since they need a refactor
> of existing scans.
> Also added is a new TableImpl to replace HTable. It uses the AsyncTableImpl
> internally and should be a bit faster because it does jump through less hoops
> to do ProtoBuf transportation. This way we can run all existing tests on the
> AsyncTableImpl to guarantee its quality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
