[
https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15149959#comment-15149959
]
Duo Zhang commented on HBASE-15264:
-----------------------------------
[~stack] You can see this, it is almost done.
https://github.com/Apache9/hbase/blob/HBASE-15265/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AsyncFSWAL.java
I still use a queue here, the rpc handler will put WAL entry in the queue and
schedule a task to event loop(Of course I have added some batching logic here
to avoid scheduling one task per WAL entry). The task will call write/flush.
See the {{consumer}} field and {{consume}} methods. We will call flush if we
write all queued entries and there are sync requests, or we find that the
buffered bytes is larger than the configured batchSize(64KB default) when
writing.
On error, we just open a new writer and write the unacked WAL entries to the
new writer. We will schedule a task to close the old broken writer. In
recoverAndClose, I will stop renew lease for this file first, and then call
recoverFileLease. With no one renew its lease, I think finally namenode will
close the file because of lease expired even if the recoverFileLease failed?
Thanks.
> Implement a fan out HDFS OutputStream
> -------------------------------------
>
> Key: HBASE-15264
> URL: https://issues.apache.org/jira/browse/HBASE-15264
> Project: HBase
> Issue Type: Sub-task
> Reporter: Duo Zhang
> Assignee: Duo Zhang
> Attachments: HBASE-15264-v1.patch, HBASE-15264-v2.patch,
> HBASE-15264.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
