[
https://issues.apache.org/jira/browse/HDFS-7889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhe Zhang updated HDFS-7889:
----------------------------
Attachment: HDFS-7889-009.patch
Thanks Bo for updating the patch! It looks good overall.
# The current {{stripeBlocks}} is not type safe. I think we can use an
ArrayList here.
# {{blockGroupSize}} and {{blockGroupDataBlocks}} should come from the
constructor. We can change it later, when EC policy is returned as part of file
status.
# In {{encode}}, can we directly use the given buffers, instead of making a
copy first?
# Is this really the last block? Shouldn't it be the block from the current
streamer?
{code}
// get last block before destroying the streamer
ExtendedBlock lastBlock = streamers.get(0).getBlock();
{code}
# It seems {{writeParityCellsForLastStripe}} uses stale data in the
cellBuffers? In the below code, shouldn't we clear cellBuffers beyond {{index}}?
{code}
if (i >= index) {
cellBuffers[i].position(0);
cellBuffers[i].limit(parityCellSize);
}
{code}
# You already have the hook to open a file in striped format, maybe need to
change the below in the test?
{code}
DFSStripedOutputStream dos = (DFSStripedOutputStream)
Whitebox.getInternalState(os,
"wrappedStream");
{code}
# Could you also make a pass and split long lines (each line should be <= 80
chars)?
I made some minor changes in 009 patch. Since HDFS-7839 moved EC policy to
XAttr, we don't have a way to check the EC policy from file status yet. The
current code checks if replication factor is 0, which works but if not ideal.
Will address after HDFS-7859.
> Subclass DFSOutputStream to support writing striping layout files
> -----------------------------------------------------------------
>
> Key: HDFS-7889
> URL: https://issues.apache.org/jira/browse/HDFS-7889
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Li Bo
> Assignee: Li Bo
> Attachments: HDFS-7889-001.patch, HDFS-7889-002.patch,
> HDFS-7889-003.patch, HDFS-7889-004.patch, HDFS-7889-005.patch,
> HDFS-7889-006.patch, HDFS-7889-007.patch, HDFS-7889-008.patch,
> HDFS-7889-009.patch
>
>
> After HDFS-7888, we can subclass {{DFSOutputStream}} to support writing
> striping layout files.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
