[
https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15149715#comment-15149715
]
Duo Zhang commented on HBASE-15264:
-----------------------------------
Basically, the stream works as follow
1. {{write}} will just copy the data into a buffer without sending it to
datanode, and we do not have a auto flush mechanism, you always need to flush
manually.
2. {{flush}} will fan out data to datanodes and notify the caller using the
CompletionHander with the file length after this flush.
3. {{close}} will complete the file normally.
4. whenever you got an exception when calling {{flush}} or {{close}}, you
should call {{recoverAndClose}} method to close the file. Now we just call
recoverFileLease. As mentioned in HBASE-14790, we need a little change on the
DTP protocol to support client recovery.
There is an {{EventLoop}} assioated with the output. All connections to
datanodes are managed by this EventLoop. For {{write}} and {{flush}}, we will
switch to the EventLoop thread if we are not in the thread currently, and for
{{close}}, {{recoverAndClose}} and the creation method of this class(the
{{createOutput}} method in helper class), you should not call them inside the
EventLoop thread because they call block for a while and you should never block
the EventLoop thread.
> Implement a fan out HDFS OutputStream
> -------------------------------------
>
> Key: HBASE-15264
> URL: https://issues.apache.org/jira/browse/HBASE-15264
> Project: HBase
> Issue Type: Sub-task
> Reporter: Duo Zhang
> Assignee: Duo Zhang
> Attachments: HBASE-15264-v1.patch, HBASE-15264-v2.patch,
> HBASE-15264.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
