[ https://issues.apache.org/jira/browse/HDFS-12654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16344419#comment-16344419 ]
Masatake Iwasaki commented on HDFS-12654: ----------------------------------------- Thanks for the info. WebHDFS does not create a new file when append is requested. It returns 404. NamenodeWebHdfsMethods#chooseDataNode:: {noformat} } else if (op == GetOpParam.Op.OPEN || op == GetOpParam.Op.GETFILECHECKSUM || op == PostOpParam.Op.APPEND) { //choose a datanode containing a replica final NamenodeProtocols np = getRPCServer(namenode); final HdfsFileStatus status = np.getFileInfo(path); if (status == null) { throw new FileNotFoundException("File " + path + " not found."); } {noformat} The non-existent file seems to be crated by fluent-plugin-webhdfs. out_webhdfs.rb:: {noformat} def send_data(path, data) if @append begin @client.append(path, data) rescue WebHDFS::FileNotFoundError @client.create(path, data) end {noformat} The issue stated in the ticket is that WebHDFS returns 404 but HttpFs returns 500. I could not reproduce this. {quote} WebHDFS::ServerError means that the client (fluentd) receives HTTP response code 500 from HttpFs server. WebHDFS server returns 404 for such cases. {quote} > APPEND API call is different in HTTPFS and NameNode REST > -------------------------------------------------------- > > Key: HDFS-12654 > URL: https://issues.apache.org/jira/browse/HDFS-12654 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs, httpfs, namenode > Affects Versions: 2.6.0, 2.7.0, 2.8.0, 3.0.0-beta1 > Reporter: Andras Czesznak > Priority: Major > > The APPEND REST API call behaves differently in the NameNode REST and the > HTTPFS codes. The NameNode version creates the target file the new data being > appended to if it does not exist at the time of the call issued. The HTTPFS > version assumes the target file exists when APPEND is called and can append > only the new data but does not create the target file it doesn't exist. > The two implementations should be standardized, preferably the HTTPFS version > should be modified to execute an implicit CREATE if the target file does not > exist. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org