[
https://issues.apache.org/jira/browse/HDFS-9455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15082774#comment-15082774
]
Daisuke Kobayashi commented on HDFS-9455:
-----------------------------------------
I see, [~archanat]. When using normal webhdfs protocol over SSL/TLS enabled
cluster, it should catch {{SocketException}} and print a stack trace. Here's a
result of my quick test, for instance:
{noformat}
[daisuke@test1 ~]$ hadoop distcp /user/daisuke/hosts webhdfs://target:50470/
16/01/05 19:11:34 WARN security.UserGroupInformation:
PriviledgedActionException as:daisuke@HADOOP (auth:KERBEROS)
cause:java.net.SocketException: Unexpected end of file from server
16/01/05 19:11:34 WARN security.UserGroupInformation:
PriviledgedActionException as:daisuke@HADOOP (auth:KERBEROS)
cause:java.net.SocketException: Unexpected end of file from server
16/01/05 19:11:34 ERROR tools.DistCp: Log.error:
java.net.SocketException: Unexpected end of file from server
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:772)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:769)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633)
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
at
java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:336)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$200(WebHdfsFileSystem.java:91)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:614)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$100(WebHdfsFileSystem.java:464)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:493)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.run(WebHdfsFileSystem.java:489)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getDelegationToken(WebHdfsFileSystem.java:1307)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getDelegationToken(WebHdfsFileSystem.java:239)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getAuthParameters(WebHdfsFileSystem.java:429)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.toUrl(WebHdfsFileSystem.java:450)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractFsPathRunner.getUrl(WebHdfsFileSystem.java:697)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:609)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$100(WebHdfsFileSystem.java:464)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:493)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.run(WebHdfsFileSystem.java:489)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getHdfsFileStatus(WebHdfsFileSystem.java:844)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getFileStatus(WebHdfsFileSystem.java:859)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1409)
at org.apache.hadoop.tools.DistCp.setTargetPathExists(DistCp.java:206)
at org.apache.hadoop.tools.DistCp.run(DistCp.java:113)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.tools.DistCp.main(DistCp.java:435)
LOG.error: Unexpected end of file from server
{noformat}
Does this make sense?
> In distcp, Invalid Argument Error thrown in case of filesystem operation
> failure
> --------------------------------------------------------------------------------
>
> Key: HDFS-9455
> URL: https://issues.apache.org/jira/browse/HDFS-9455
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: distcp, security
> Reporter: Archana T
> Assignee: Archana T
> Priority: Minor
>
> When Filesystem Operation failure happens during discp,
> Wrong exception : Invalid Argument thrown along with distcp command Usage.
> {color:red}
> hadoop distcp webhdfs://IP:25003/test/testfile webhdfs://IP:25003/myp
> Invalid arguments: Unexpected end of file from server
> usage: distcp OPTIONS [source_path...] <target_path>
> OPTIONS
> -append Reuse existing data in target files and
> append new data to them if possible
> -async Should distcp execution be blocking
> -atomic Commit all changes or none
> -bandwidth <arg> Specify bandwidth per map in MB
> -delete Delete from target, files missing in source
> -diff <arg> Use snapshot diff report to identify the
> difference between source and target
> -f <arg> List of files that need to be copied
> -filelimit <arg> (Deprecated!) Limit number of files copied
> to <= n
> -i Ignore failures during copy
> .....
> {color}
> Instead Proper Exception has to be thrown.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
