[
https://issues.apache.org/jira/browse/HDFS-11189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15710288#comment-15710288
]
Weiwei Yang commented on HDFS-11189:
------------------------------------
Hi [~guadilla]
It is not clear what was the problem. Webhdfs was designed not to automatically
follow redirects, you don't want to send too many data to NN server directly,
instead you want NN tells you which DN you want to write, then use another
request to DN to transfer data. Would you provide more info about how webhdfs
was crashed? At this point, can't tell this is a webhdfs problem or not.
> WebHDFS and FollowRedirects
> ---------------------------
>
> Key: HDFS-11189
> URL: https://issues.apache.org/jira/browse/HDFS-11189
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
> Affects Versions: 2.7.3
> Environment: We are using webhdfs from a j2ee environment with java
> 6. In the web application we have a lot of libraries: spring, jersey,
> jackson...
> Reporter: Oscar Guadilla
>
> In some cases with simple operations (get file - non two step operation) we
> detect that FollowRedirects flag of http comes as "false" and it makes
> webhdfs crash.
> We don't really know which library change this behavior and it is really
> strange because the change is done directly in the jvm because in other
> wars/ears of the same j2ee server it fails also. If we restart the j2ee
> server it starts working again.
> To fix the problem we changed the WebHdfsFileSystem class adding
> "setInstanceFollowRedirects(true)" in the connection management instead of
> supposing that it should be true and it works fine.
> The same problem arises both in 1.x and 2.x webhdfs. We didn't test in 3.x
> Could you fix it? We did it in our environment but it would fine in it could
> be included in the next releases.
> Thanks in advance,
> Oscar
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
