[
https://issues.apache.org/jira/browse/HDFS-12654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336386#comment-16336386
]
Andras Czesznak commented on HDFS-12654:
----------------------------------------
Hello [~iwasakims] San, the issue occurred with the FluentD app and described
here:
https://github.com/fluent/fluent-plugin-webhdfs/issues/46
>From the HTTPFS's log:
```
2017-10-03 16:20:59,204 WARN org.apache.hadoop.security.UserGroupInformation:
PriviledgedActionException as:<username> (auth:PROXY) via httpfs (auth:SIMPLE)
cause:java.io.FileNotFoundException: failed to append to non-existent file
<path>/fluentd/process/<process>.log for client <IP address>
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.appendFileInternal(FSNamesystem.java:2930)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.appendFileInt(FSNamesystem.java:3227)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.appendFile(FSNamesystem.java:3191)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.append(NameNodeRpcServer.java:614)
at
org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.append(AuthorizationProviderProxyClientProtocol.java:126)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.append(ClientNamenodeProtocolServerSideTranslatorPB.java:416)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2086)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2082)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2080)
```
This is what it looks like in FluentD log
```
2017-10-04 17:23:54 -0700 [warn]: failed to communicate hdfs cluster, path:
<path>/fluentd/process/<process>.log
2017-10-04 17:23:54 -0700 [warn]: temporarily failed to flush the buffer.
next_retry=2017-10-04 17:24:24 -0700 error_class="WebHDFS::ServerError"
error="Failed to connect to host <FQDN>:14000, Broken pipe - sendfile"
plugin_id="object:3f8778538560"
2017-10-04 17:23:54 -0700 [warn]: suppressed same stacktrace
The source code snippets:
1) HTTPFS: direct FS output stream creation
/hadoop/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/client/HttpFSFileSystem.java
...
551 /**
552 * Append to an existing file (optional operation).
553 * <p/>
554 * IMPORTANT: The <code>Progressable</code> parameter is not used.
555 *
556 * @param f the existing file to be appended.
557 * @param bufferSize the size of the buffer to be used.
558 * @param progress for reporting progress if it is not null.
559 *
560 * @throws IOException
561 */
562 @Override
563 public FSDataOutputStream append(Path f, int bufferSize,
564 Progressable progress) throws IOException {
565 Map<String, String> params = new HashMap<String, String>();
566 params.put(OP_PARAM, Operation.APPEND.toString());
567 return uploadData(Operation.APPEND.getMethod(), f, params, bufferSize,
568 HttpURLConnection.HTTP_OK);
569 }
...
2) WebHDFS: indirect FS output stream creation thru DFS
...
/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
...
54 import org.apache.hadoop.fs.FSDataOutputStream;
...
795 /**
796 * Handle create/append output streams
797 */
798 class FsPathOutputStreamRunner extends
AbstractFsPathRunner<FSDataOutputStream> {
799 private final int bufferSize;
800
801 FsPathOutputStreamRunner(Op op, Path fspath, int bufferSize,
802 Param<?,?>... parameters) {
803 super(op, fspath, parameters);
804 this.bufferSize = bufferSize;
805 }
806
807 @Override
808 FSDataOutputStream getResponse(final HttpURLConnection conn)
809 throws IOException {
810 return new FSDataOutputStream(new BufferedOutputStream(
811 conn.getOutputStream(), bufferSize), statistics) {
812 @Override
813 public void close() throws IOException {
814 try {
815 super.close();
816 } finally {
817 try {
818 validateResponse(op, conn, true);
819 } finally {
820 conn.disconnect();
821 }
822 }
823 }
824 };
825 }
826 }
...
> APPEND API call is different in HTTPFS and NameNode REST
> --------------------------------------------------------
>
> Key: HDFS-12654
> URL: https://issues.apache.org/jira/browse/HDFS-12654
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs, httpfs, namenode
> Affects Versions: 2.6.0, 2.7.0, 2.8.0, 3.0.0-beta1
> Reporter: Andras Czesznak
> Priority: Major
>
> The APPEND REST API call behaves differently in the NameNode REST and the
> HTTPFS codes. The NameNode version creates the target file the new data being
> appended to if it does not exist at the time of the call issued. The HTTPFS
> version assumes the target file exists when APPEND is called and can append
> only the new data but does not create the target file it doesn't exist.
> The two implementations should be standardized, preferably the HTTPFS version
> should be modified to execute an implicit CREATE if the target file does not
> exist.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
