[
https://issues.apache.org/jira/browse/HDFS-4262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510680#comment-13510680
]
Yu Li commented on HDFS-4262:
-----------------------------
Thanks Alejandro.
After applying the two patches you supplied, UT result in my env is like:
===============================================================
Tests in error:
testOperation[0](org.apache.hadoop.fs.http.client.TestWebhdfsFileSystem):
java.io.IOException: Server returned HTTP response code: 500 for URL:
http://bdvm072.svl.ibm.com:48763/webhdfs/v1/?op=GETHOMEDIRECTORY&doas=biadmin
testOperationDoAs[0](org.apache.hadoop.fs.http.client.TestWebhdfsFileSystem):
java.io.IOException: Server returned HTTP response code: 401 for URL:
http://bdvm072.svl.ibm.com:57519/webhdfs/v1/?op=GETHOMEDIRECTORY&doas=user1
testOperation[0](org.apache.hadoop.fs.http.client.TestHttpFSFileSystem):
java.io.IOException: Server returned HTTP response code: 500 for URL:
http://bdvm072.svl.ibm.com:57757/webhdfs/v1/?op=GETHOMEDIRECTORY&doas=biadmin
testOperationDoAs[0](org.apache.hadoop.fs.http.client.TestHttpFSFileSystem):
java.io.IOException: Server returned HTTP response code: 401 for URL:
http://bdvm072.svl.ibm.com:56289/webhdfs/v1/?op=GETHOMEDIRECTORY&doas=user1
Tests run: 177, Failures: 0, Errors: 4, Skipped: 0
===============================================================
Then I did some investigation and made another patch,
"03-resolve-proxyuser-related-issue.patch", which could resolve the UT failure.
I also merged the three patches together to "HDFS-4262-github.patch", as
attached.
After getting all UT pass, I also build-out a httpfs tar ball and tested it on
a hadoop-1.0.3 environment, and most function worked. However, I found the API
documents, either the ones attached with HDFS-2178 or on
http://cloudera.github.com/httpfs/UsingHttpTools.html, are out of date. Like to
get the name dir of one specified user, the request should be:
<p><i>curl -X GET
"http://shihc024-public.cn.ibm.com:14000/webhdfs/v1?user.name=biadmin&op=gethomedirectory";</i></p>
rather than
<p><i>curl -i "http://<HTTPFSHOSTNAME>:14000?user.name=babu&op=homedir"</i></p>
Anybody could tell me where I can get the latest API doc, so I could apply a
full sanity test?
> Backport HTTPFS to Branch 1
> ---------------------------
>
> Key: HDFS-4262
> URL: https://issues.apache.org/jira/browse/HDFS-4262
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Environment: IBM JDK, RHEL 6.3
> Reporter: Eric Yang
> Assignee: Yu Li
> Attachments: 01-retrofit-httpfs-cdh3u4-for-hadoop1.patch,
> 02-cookie-from-authenticated-url-is-not-getting-to-auth-filter.patch,
> 03-resolve-proxyuser-related-issue.patch, HDFS-4262-github.patch
>
>
> There are interests to backport HTTPFS back to Hadoop 1 branch. After the
> initial investigation, there're quite some changes in HDFS-2178, and several
> related patches, including:
> HDFS-2284 Write Http access to HDFS
> HDFS-2646 Hadoop HttpFS introduced 4 findbug warnings
> HDFS-2649 eclipse:eclipse build fails for hadoop-hdfs-httpfs
> HDFS-2657 TestHttpFSServer and TestServerWebApp are failing on trunk
> HDFS-2658 HttpFS introduced 70 javadoc warnings
> The most challenge of backporting is all these patches, including HDFS-2178
> are for 2.X, which code base has been refactored a lot and quite different
> from 1.X, so it seems we have to backport the changes manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
