[
https://issues.apache.org/jira/browse/HADOOP-11556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14319382#comment-14319382
]
Lei (Eddy) Xu commented on HADOOP-11556:
----------------------------------------
[[email protected]] Thanks for confirming this.
It also appears to me that if a partitioned upload fails in the middle, a
{{bar._COPYING_}} directory is there as side effect. So I am wondering whether
should we describe it as a {{ObjectStore}} in HADOOP-9565, in which the {{PUT}}
failure should have no side effect.
But for this particular issue, I have doubt about this line of the log:
{code}
15/02/06 21:00:49 DEBUG methods.EntityEnclosingMethod: Request body has not
been specified
{code}
It looks more like a small bug to me. If it is the case, partitioned upload
should not work at all?
> SwiftNativeFileSystem rename being interrupted during "fs -put"
> ---------------------------------------------------------------
>
> Key: HADOOP-11556
> URL: https://issues.apache.org/jira/browse/HADOOP-11556
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, fs/swift
> Affects Versions: 2.6.0
> Reporter: Lei (Eddy) Xu
>
> When running {{hadoop fs -put bar swift://... }}, {{FsShell}} writes to
> {{bar._COPYING_}} first, and then renames it to "bar".
> However, it fails when the file is partitioned uploaded.
> {code}
> 15/02/06 21:00:49 DEBUG snative.SwiftNativeFileSystemStore: Source file
> appears to be partitioned. copying file and deleting children
> 15/02/06 21:00:49 DEBUG params.DefaultHttpParams: Set parameter
> http.method.retry-handler =
> org.apache.commons.httpclient.DefaultHttpMethodRetryHandler@26bf4e60
> 15/02/06 21:00:49 DEBUG params.DefaultHttpParams: Set parameter
> http.connection.timeout = 15000
> 15/02/06 21:00:49 DEBUG params.DefaultHttpParams: Set parameter
> http.socket.timeout = 60000
> 15/02/06 21:00:49 DEBUG http.SwiftRestClient: COPY
> https://snet-storage101.iad3.clouddrive.com/v1/MossoCloudFS_946265/foo/bar._COPYING_
> X-Auth-Token: ...
> Destination: foo/bar
> User-Agent: Apache Hadoop Swift Client 3.0.0-SNAPSHOT from ... by root source
> checksum ...
> 15/02/06 21:00:49 DEBUG httpclient.HttpConnection: Open connection to
> snet-storage101.iad3.clouddrive.com:443
> 15/02/06 21:00:49 DEBUG wire.header: >> "COPY
> /v1/MossoCloudFS_946265/foo/bar._COPYING_ HTTP/1.1[\r][\n]"
> 15/02/06 21:00:49 DEBUG httpclient.HttpMethodBase: Adding Host request header
> 15/02/06 21:00:49 DEBUG wire.header: >> "X-Auth-Token: ...[\r][\n]"
> 15/02/06 21:00:49 DEBUG wire.header: >> "Destination: foo/bar[\r][\n]"
> 15/02/06 21:00:49 DEBUG wire.header: >> "User-Agent: Apache Hadoop Swift
> Client 3.0.0-SNAPSHOT from ... by root source checksum ...[\r][\n]"
> 15/02/06 21:00:49 DEBUG wire.header: >> "Host:
> snet-storage101.iad3.clouddrive.com[\r][\n]"
> 15/02/06 21:00:49 DEBUG wire.header: >> "Content-Length: 0[\r][\n]"
> 15/02/06 21:00:49 DEBUG wire.header: >> "[\r][\n]"
> 15/02/06 21:00:49 DEBUG methods.EntityEnclosingMethod: Request body has not
> been specified
> 15/02/06 21:01:49 DEBUG httpclient.HttpMethodDirector: Closing the connection.
> 15/02/06 21:01:49 DEBUG httpclient.HttpMethodDirector: Method retry handler
> returned false. Automatic recovery will not be attempted
> 15/02/06 21:01:49 DEBUG httpclient.HttpConnection: Releasing connection back
> to connection manager.
> 15/02/06 21:01:49 DEBUG httpclient.HttpConnection: Releasing connection back
> to connection manager.
> {code}
> The {{bar}} file eventually shows up in Swift. However, the {{bar._COPYING_}}
> directories and the manifest file are not deleted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
