[
https://issues.apache.org/jira/browse/HDFS-7175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294338#comment-14294338
]
Subbu commented on HDFS-7175:
-----------------------------
I apologize for the delay in verification of this bug.
I have now verified that no matter what the value is for frequency of flush,
the solution does NOT work. Basically, the flush() call has no effect since
there are no bytes to flush.
Here is what I did to verify this:
* Brought up a single node cluster.
* I changed the frequency of flush to 1 (instead of 10k or 100k).
* Ran fsck on a small directory with 10 files, both with and without
-showprogress option.
* Ran tcpdump on the namenode port to capture packets during the session.
I could see that the dots were sent out in in the channel when -showprogress
was specified, but the channel was quiet when it was not.
So, we need to think of another way to solve the problem.
> Client-side SocketTimeoutException during Fsck
> ----------------------------------------------
>
> Key: HDFS-7175
> URL: https://issues.apache.org/jira/browse/HDFS-7175
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Reporter: Carl Steinbach
> Assignee: Akira AJISAKA
> Attachments: HDFS-7175.2.patch, HDFS-7175.3.patch, HDFS-7175.patch,
> HDFS-7175.patch
>
>
> HDFS-2538 disabled status reporting for the fsck command (it can optionally
> be enabled with the -showprogress option). We have observed that without
> status reporting the client will abort with read timeout:
> {noformat}
> [hdfs@lva1-hcl0030 ~]$ hdfs fsck /
> Connecting to namenode via http://lva1-tarocknn01.grid.linkedin.com:50070
> 14/09/30 06:03:41 WARN security.UserGroupInformation:
> PriviledgedActionException as:[email protected] (auth:KERBEROS)
> cause:java.net.SocketTimeoutException: Read timed out
> Exception in thread "main" java.net.SocketTimeoutException: Read timed out
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(SocketInputStream.java:152)
> at java.net.SocketInputStream.read(SocketInputStream.java:122)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
> at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:687)
> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633)
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
> at org.apache.hadoop.hdfs.tools.DFSck.doWork(DFSck.java:312)
> at org.apache.hadoop.hdfs.tools.DFSck.access$000(DFSck.java:72)
> at org.apache.hadoop.hdfs.tools.DFSck$1.run(DFSck.java:149)
> at org.apache.hadoop.hdfs.tools.DFSck$1.run(DFSck.java:146)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
> at org.apache.hadoop.hdfs.tools.DFSck.run(DFSck.java:145)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at org.apache.hadoop.hdfs.tools.DFSck.main(DFSck.java:346)
> {noformat}
> Since there's nothing for the client to read it will abort if the time
> required to complete the fsck operation is longer than the client's read
> timeout setting.
> I can think of a couple ways to fix this:
> # Set an infinite read timeout on the client side (not a good idea!).
> # Have the server-side write (and flush) zeros to the wire and instruct the
> client to ignore these characters instead of echoing them.
> # It's possible that flushing an empty buffer on the server-side will trigger
> an HTTP response with a zero length payload. This may be enough to keep the
> client from hanging up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
