[
https://issues.apache.org/jira/browse/HDFS-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695179#comment-13695179
]
Chris Nauroth commented on HDFS-4905:
-------------------------------------
Hi, Arpit. This looks very useful!
# I believe this TODO won't be necessary as the norm is to pass these via
-Dkey=value arguments and let {{GenericOptionsParser}} take care of it.
Existing commands like -put handle it this way.
{quote}
// TODO: Accept options to set the replication factor
// and the blocksize.
{quote}
# When I run the command with one argument, it seems to default to appending to
the user's home path in HDFS (/user/chris in my case).
{{CommandWithDestination#getRemoteDestination}} has a special case to assume
this is the destination when there are fewer than 2 arguments. Then,
{{AppendToFile#processArguments}} always attempts to create the destination
file if it doesn't exist. Then, there are no more remaining arguments, so it
doesn't append anything to this newly created file. Perhaps it would be best
to abort if there are fewer than 2 args to prevent confusion?
# {{sources}}, {{replication}}, and {{blockSize}} appear to be unused.
(They're probably not needed, related to point 1.)
# I think the following logic could be simplified by using
{{IOUtils#copyBytes}}.
{code}
FSDataOutputStream fos = dst.fs.append(dst.path);
byte[] buffer = new byte[DEFAULT_IO_LENGTH];
// Read in each input file and write to the target.
for (PathData source : args) {
FileInputStream fis = new FileInputStream(source.toFile());
int bytesRead = fis.read(buffer, 0, buffer.length);
while (bytesRead != -1) {
fos.write(buffer, 0, bytesRead);
bytesRead = fis.read(buffer, 0, buffer.length);
}
fis.close();
}
fos.close();
{code}
# Also related to the above code, can you put the close calls into finally
blocks?
# Can you please add documentation for the new command to
hadoop-common-project/hadoop-common/src/site/apt/FileSystemShell.apt.vm?
> Add appendToFile command to "hdfs dfs"
> --------------------------------------
>
> Key: HDFS-4905
> URL: https://issues.apache.org/jira/browse/HDFS-4905
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: tools
> Affects Versions: 3.0.0
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
> Priority: Minor
> Attachments: HDFS-4905.patch
>
>
> A "hdfs dfs -appendToFile..." option would be quite useful for quick testing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
