[
https://issues.apache.org/jira/browse/HADOOP-12057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14658851#comment-14658851
]
Steve Loughran commented on HADOOP-12057:
-----------------------------------------
All the rename tests are failing for me now
{code}
------------------------------------------------------
Running org.apache.hadoop.fs.swift.TestSwiftFileSystemRename
Tests run: 14, Failures: 0, Errors: 4, Skipped: 0, Time elapsed: 187.855 sec
<<< FAILURE! - in org.apache.hadoop.fs.swift.TestSwiftFileSystemRename
testRenameFileAndVerifyContents(org.apache.hadoop.fs.swift.TestSwiftFileSystemRename)
Time elapsed: 29.876 sec <<< ERROR!
java.io.FileNotFoundException: Operation COPY on
https://storage101.dfw1.clouddrive.com/v1/MossoCloudFS_8b95cbaa-d1b8-43a3-97f8-dbc8d4b3174a/jenkins/test/home/user/documents/file.txt/file.txt
at
org.apache.hadoop.fs.swift.http.SwiftRestClient.buildException(SwiftRestClient.java:1449)
at
org.apache.hadoop.fs.swift.http.SwiftRestClient.perform(SwiftRestClient.java:1402)
at
org.apache.hadoop.fs.swift.http.SwiftRestClient.copyObject(SwiftRestClient.java:923)
at
org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystemStore.copyObject(SwiftNativeFileSystemStore.java:815)
at
org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystemStore.movePartitionedFile(SwiftNativeFileSystemStore.java:746)
at
org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystemStore.rename(SwiftNativeFileSystemStore.java:628)
at
org.apache.hadoop.fs.swift.SwiftFileSystemBaseTest.renameToSuccess(SwiftFileSystemBaseTest.java:283)
at
org.apache.hadoop.fs.swift.SwiftFileSystemBaseTest.rename(SwiftFileSystemBaseTest.java:227)
at
org.apache.hadoop.fs.swift.TestSwiftFileSystemRename.testRenameFileAndVerifyContents(TestSwiftFileSystemRename.java:197)
testRenameFile(org.apache.hadoop.fs.swift.TestSwiftFileSystemRename) Time
elapsed: 20.657 sec <<< ERROR!
java.io.FileNotFoundException: Operation COPY on
https://storage101.dfw1.clouddrive.com/v1/MossoCloudFS_8b95cbaa-d1b8-43a3-97f8-dbc8d4b3174a/jenkins/test/alice/file/file
at
org.apache.hadoop.fs.swift.http.SwiftRestClient.buildException(SwiftRestClient.java:1449)
at
org.apache.hadoop.fs.swift.http.SwiftRestClient.perform(SwiftRestClient.java:1402)
at
org.apache.hadoop.fs.swift.http.SwiftRestClient.copyObject(SwiftRestClient.java:923)
at
org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystemStore.copyObject(SwiftNativeFileSystemStore.java:815)
at
org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystemStore.movePartitionedFile(SwiftNativeFileSystemStore.java:746)
at
org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystemStore.rename(SwiftNativeFileSystemStore.java:628)
at
org.apache.hadoop.fs.swift.SwiftFileSystemBaseTest.renameToSuccess(SwiftFileSystemBaseTest.java:283)
at
org.apache.hadoop.fs.swift.SwiftFileSystemBaseTest.rename(SwiftFileSystemBaseTest.java:227)
at
org.apache.hadoop.fs.swift.TestSwiftFileSystemRename.testRenameFile(TestSwiftFileSystemRename.java:76)
testRenamedConsistence(org.apache.hadoop.fs.swift.TestSwiftFileSystemRename)
Time elapsed: 24.578 sec <<< ERROR!
java.io.FileNotFoundException: Operation COPY on
https://storage101.dfw1.clouddrive.com/v1/MossoCloudFS_8b95cbaa-d1b8-43a3-97f8-dbc8d4b3174a/jenkins/test/home/user/documents/file.txt/file.txt
at
org.apache.hadoop.fs.swift.http.SwiftRestClient.buildException(SwiftRestClient.java:1449)
at
org.apache.hadoop.fs.swift.http.SwiftRestClient.perform(SwiftRestClient.java:1402)
at
org.apache.hadoop.fs.swift.http.SwiftRestClient.copyObject(SwiftRestClient.java:923)
at
org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystemStore.copyObject(SwiftNativeFileSystemStore.java:815)
at
org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystemStore.movePartitionedFile(SwiftNativeFileSystemStore.java:746)
at
org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystemStore.rename(SwiftNativeFileSystemStore.java:628)
at
org.apache.hadoop.fs.swift.SwiftFileSystemBaseTest.renameToSuccess(SwiftFileSystemBaseTest.java:283)
at
org.apache.hadoop.fs.swift.SwiftFileSystemBaseTest.rename(SwiftFileSystemBaseTest.java:227)
at
org.apache.hadoop.fs.swift.TestSwiftFileSystemRename.testRenamedConsistence(TestSwiftFileSystemRename.java:259)
testRenameFileIntoExistingDirectory(org.apache.hadoop.fs.swift.TestSwiftFileSystemRename)
Time elapsed: 26.019 sec <<< ERROR!
java.io.FileNotFoundException: Operation COPY on
https://storage101.dfw1.clouddrive.com/v1/MossoCloudFS_8b95cbaa-d1b8-43a3-97f8-dbc8d4b3174a/jenkins/test/olddir/file/file
at
org.apache.hadoop.fs.swift.http.SwiftRestClient.buildException(SwiftRestClient.java:1449)
at
org.apache.hadoop.fs.swift.http.SwiftRestClient.perform(SwiftRestClient.java:1402)
at
org.apache.hadoop.fs.swift.http.SwiftRestClient.copyObject(SwiftRestClient.java:923)
at
org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystemStore.copyObject(SwiftNativeFileSystemStore.java:815)
at
org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystemStore.movePartitionedFile(SwiftNativeFileSystemStore.java:746)
at
org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystemStore.rename(SwiftNativeFileSystemStore.java:628)
at
org.apache.hadoop.fs.swift.SwiftFileSystemBaseTest.renameToSuccess(SwiftFileSystemBaseTest.java:283)
at
org.apache.hadoop.fs.swift.SwiftFileSystemBaseTest.rename(SwiftFileSystemBaseTest.java:227)
at
org.apache.hadoop.fs.swift.TestSwiftFileSystemRename.testRenameFileIntoExistingDirectory(TestSwiftFileSystemRename.java:51)
{code}
> swiftfs rename on partitioned file attempts to consolidate partitions
> ---------------------------------------------------------------------
>
> Key: HADOOP-12057
> URL: https://issues.apache.org/jira/browse/HADOOP-12057
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/swift
> Reporter: David Dobbins
> Assignee: David Dobbins
> Attachments: HADOOP-12057.patch, HADOOP-12057.patch,
> HADOOP-12057.patch, HADOOP-12057.patch, HADOOP-12057.patch
>
>
> In the swift filesystem for openstack, a rename operation on a partitioned
> file uses the swift COPY operation, which attempts to consolidate all of the
> partitions into a single object. This causes the rename to fail when the
> total size of all the partitions exceeds the maximum object size for swift.
> Since partitioned files are primarily created to allow a file to exceed the
> maximum object size, this bug makes writing to swift extremely unreliable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
