[
https://issues.apache.org/jira/browse/HDFS-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16776655#comment-16776655
]
Vinayakumar B commented on HDFS-7663:
-------------------------------------
Thanks [~ayushtkn] for taking this up.
Following are the comments from patch.
1.
{code}
protected final AtomicReference<CachingStrategy> cachingStrategy;
- private FileEncryptionInfo fileEncryptionInfo;
+ protected FileEncryptionInfo fileEncryptionInfo;
private int writePacketSize;
{code}
{code}
- private DFSOutputStream(DFSClient dfsClient, String src,
+ protected DFSOutputStream(DFSClient dfsClient, String src,
EnumSet<CreateFlag> flag,
{code}
These changes are unnecessary
2.
{code}
+ /** Construct a new output stream for appending to a file. */
+ DFSStripedOutputStream(DFSClient dfsClient, String src,
+ EnumSet<CreateFlag> flags, Progressable progress, LocatedBlock lastBlock,
+ HdfsFileStatus stat, DataChecksum checksum, String[] favoredNodes)
+ throws IOException {
+ super(dfsClient, src, stat, flags, progress, checksum, favoredNodes,
false);
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Creating DFSStripedOutputStream for " + src);
+ }
.
.
.
{code}
You can re-use the existing constructor itself. Just add remaining statements.
No need to validate the NEW_BLOCK flag here. It will be already validated
during namenode call itself.
3.
{code}
// not support appending file with striped blocks
- if (file.isStriped()) {
+ if (file.isStriped() && file.isUnderConstruction()) {
throw new UnsupportedOperationException(
"Cannot append to files with striped block " + path);
}
{code}
Remove the entire check itself. For Underconstruction,
AlreadyBeingCreatedException will be thrown later.
4. Also, add Validation of NEW_BLOCK flag in the namenode-side, until support
of append to existing blocks.
> Erasure Coding: Append on striped file
> --------------------------------------
>
> Key: HDFS-7663
> URL: https://issues.apache.org/jira/browse/HDFS-7663
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: 3.0.0-alpha1
> Reporter: Jing Zhao
> Assignee: Ayush Saxena
> Priority: Major
> Attachments: HDFS-7663-02.patch, HDFS-7663-03.patch,
> HDFS-7663-04.patch, HDFS-7663.00.txt, HDFS-7663.01.patch
>
>
> Append should be easy if we have variable length block support from
> HDFS-3689, i.e., the new data will be appended to a new block. We need to
> revisit whether and how to support appending data to the original last block.
> 1. Append to a closed striped file, with NEW_BLOCK flag enabled (this)
> 2. Append to a under-construction striped file, with NEW_BLOCK flag enabled
> (HDFS-9173)
> 3. Append to a striped file, by appending to last block group (follow-on)
> This jira attempts to implement the #1, and also track #2, #3.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
