[
https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15158505#comment-15158505
]
Duo Zhang commented on HBASE-15264:
-----------------------------------
Commit this on master is a bit simple for me since I haven't worked well on a
branch yet. Does review board and pre-commit check still work for a commit on
branch other than the several main branches(the pre-commit is broken when I was
doing HDFS-7966...). And the permissions of git repo... If these are all
simple, I would prefer doing this on a feature branch since it is more
flexible. If it is troublesome then I would say just doing this on master since
we only have one big commit, not a series of new patches :)
{quote}
That is a fat bunch of params to pass the constructor:
312 FanOutOneBlockAsyncDFSOutput(Configuration conf, FSUtils fsUtils,
DistributedFileSystem dfs,
313 DFSClient client, ClientProtocol namenode, String clientName, String
src, long fileId,
314 LocatedBlock locatedBlock, EventLoop eventLoop, List<Channel>
datanodeList,
315 DataChecksum summer, ByteBufAllocator alloc) {
{quote}
The constructor is not suppose to be called by end user, we still need a
private constructor with the builder pattern, so I think you are saying the
Helper.createOutput method. It is same with FileSystem.createNonRecursive
method. But this is a good suggestion, can change to a builder pattern when
moving it to HDFS.
{quote}
What did we say about this:
ByteBuf checksumBuf = alloc.directBuffer(checksumLen);
That we are allocating offheap. when writing always.... The offheap allocations
are relatively slow... Can use a pool later?
... and here: ByteBuf newBuf =
alloc.directBuffer().ensureWritable(trailingPartialChunkLen);
{quote}
This is done by netty itself. I use PooledByteBufAllocator.
{code:title=FanOutOneBlockAsyncDFSOutputHelper .java}
private static final ByteBufAllocator ALLOC = PooledByteBufAllocator.DEFAULT;
{code}
Will add more comments on how things are going on here. Thanks.
> Implement a fan out HDFS OutputStream
> -------------------------------------
>
> Key: HBASE-15264
> URL: https://issues.apache.org/jira/browse/HBASE-15264
> Project: HBase
> Issue Type: Sub-task
> Components: util, wal
> Reporter: Duo Zhang
> Assignee: Duo Zhang
> Fix For: 2.0.0
>
> Attachments: HBASE-15264-v1.patch, HBASE-15264-v2.patch,
> HBASE-15264-v3.patch, HBASE-15264-v4.patch, HBASE-15264-v5.patch,
> HBASE-15264-v6.patch, HBASE-15264.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
