[
https://issues.apache.org/jira/browse/HDFS-13400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427218#comment-16427218
]
Erik Krogen commented on HDFS-13400:
------------------------------------
Sure. Sorry for fighting with you on JIRA status, just didn't want to make it
seem like someone had posted a patch fixing the issue.
So the issue comes from here:
{code}
/**
* Handle create/append output streams
*/
class FsPathOutputStreamRunner
extends AbstractFsPathRunner<FSDataOutputStream> {
private final int bufferSize;
FsPathOutputStreamRunner(Op op, Path fspath, int bufferSize,
Param<?,?>... parameters) {
super(op, fspath, parameters);
this.bufferSize = bufferSize;
}
@Override
FSDataOutputStream getResponse(final HttpURLConnection conn)
throws IOException {
return new FSDataOutputStream(new BufferedOutputStream(
conn.getOutputStream(), bufferSize), statistics) {
@Override
public void close() throws IOException {
try {
super.close();
} finally {
try {
validateResponse(op, conn, true);
} finally {
// This is a connection to DataNode. Let's disconnect since
// there is little chance that the connection will be reused
// any time soonl
conn.disconnect();
}
}
}
};
}
}
{code}
We see that the {{FSDataOutputStream}} is being created using the two-arg
constructor, but we need to use the three-arg constructor which allows for
specifying a nonzero {{startPosition}}. That part is easy. How to get the
correct {{startPosition}} is hard...
The attached test simply fails on the assert:
{code}
java.lang.AssertionError:
Expected :10
Actual :0
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.hadoop.hdfs.web.TestWebHDFS.testWebHdfsSuccessfulAppend(TestWebHDFS.java:1498)
{code}
> WebHDFS append returned stream has incorrectly set position
> -----------------------------------------------------------
>
> Key: HDFS-13400
> URL: https://issues.apache.org/jira/browse/HDFS-13400
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
> Affects Versions: 2.9.0, 2.8.3, 2.7.5, 3.0.1
> Reporter: Erik Krogen
> Priority: Minor
> Attachments: HDFS-13400-unit-test.patch
>
>
> If you call {{getPos()}} on an {{FSDataOutputStream}} returned by
> {{WebHdfsFileSystem#append}}, it will initially return 0, even if the file
> already had content. This method should return the initial length of the file
> before appending.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
