[
https://issues.apache.org/jira/browse/HDFS-12654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16344419#comment-16344419
]
Masatake Iwasaki commented on HDFS-12654:
-----------------------------------------
Thanks for the info.
WebHDFS does not create a new file when append is requested. It returns 404.
NamenodeWebHdfsMethods#chooseDataNode::
{noformat}
} else if (op == GetOpParam.Op.OPEN
|| op == GetOpParam.Op.GETFILECHECKSUM
|| op == PostOpParam.Op.APPEND) {
//choose a datanode containing a replica
final NamenodeProtocols np = getRPCServer(namenode);
final HdfsFileStatus status = np.getFileInfo(path);
if (status == null) {
throw new FileNotFoundException("File " + path + " not found.");
}
{noformat}
The non-existent file seems to be crated by fluent-plugin-webhdfs.
out_webhdfs.rb::
{noformat}
def send_data(path, data)
if @append
begin
@client.append(path, data)
rescue WebHDFS::FileNotFoundError
@client.create(path, data)
end
{noformat}
The issue stated in the ticket is that WebHDFS returns 404 but HttpFs returns
500. I could not reproduce this.
{quote}
WebHDFS::ServerError means that the client (fluentd) receives HTTP response
code 500 from HttpFs server. WebHDFS server returns 404 for such cases.
{quote}
> APPEND API call is different in HTTPFS and NameNode REST
> --------------------------------------------------------
>
> Key: HDFS-12654
> URL: https://issues.apache.org/jira/browse/HDFS-12654
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs, httpfs, namenode
> Affects Versions: 2.6.0, 2.7.0, 2.8.0, 3.0.0-beta1
> Reporter: Andras Czesznak
> Priority: Major
>
> The APPEND REST API call behaves differently in the NameNode REST and the
> HTTPFS codes. The NameNode version creates the target file the new data being
> appended to if it does not exist at the time of the call issued. The HTTPFS
> version assumes the target file exists when APPEND is called and can append
> only the new data but does not create the target file it doesn't exist.
> The two implementations should be standardized, preferably the HTTPFS version
> should be modified to execute an implicit CREATE if the target file does not
> exist.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
