raulcd opened a new issue, #36447: URL: https://github.com/apache/arrow/issues/36447
### Describe the bug, including details regarding any error messages, version, and platform. It seems that the `arrow-s3fs-test` fails with `file exists` for some of the tests: ``` Error: mkdir /tmp/s3fs-test-9kk8hwry/bucket/somefile/: file exists (*fs.PathError) ``` This is happening on: * [test-alpine-linux-cpp](https://github.com/ursacomputing/crossbow/actions/runs/5449610923/jobs/9914016936) * [test-debian-11-cpp-amd64](https://github.com/ursacomputing/crossbow/actions/runs/5449604499/jobs/9914003220) * [test-skyhook-integration](https://github.com/ursacomputing/crossbow/actions/runs/5449610638/jobs/9914016297) * [test-ubuntu-20.04-cpp](https://github.com/ursacomputing/crossbow/actions/runs/5449609783/jobs/9914014289) * [test-ubuntu-22.04-cpp](https://github.com/ursacomputing/crossbow/actions/runs/5449608290/jobs/9914011015) The full error log: ``` 75/108 Test #69: arrow-s3fs-test ...........................***Failed 21.30 sec Running arrow-s3fs-test, redirecting output into /build/cpp/build/test-logs/arrow-s3fs-test.txt (attempt 1/1) /arrow/cpp/build-support/run-test.sh: line 88: 30269 Aborted (core dumped) $TEST_EXECUTABLE "$@" > $LOGFILE.raw 2>&1 Running main() from /home/buildozer/aports/main/gtest/src/googletest-14aa11db02d9851d957f93ef9fddb110c1aafdc6/googletest/src/gtest_main.cc [==========] Running 62 tests from 6 test suites. [----------] Global test environment set-up. [----------] 3 tests from S3OptionsTest [ RUN ] S3OptionsTest.FromUri [ OK ] S3OptionsTest.FromUri (579 ms) [ RUN ] S3OptionsTest.FromAccessKey [ OK ] S3OptionsTest.FromAccessKey (132 ms) [ RUN ] S3OptionsTest.FromAssumeRole [ OK ] S3OptionsTest.FromAssumeRole (135 ms) [----------] 3 tests from S3OptionsTest (847 ms total) [----------] 4 tests from S3RegionResolutionTest [ RUN ] S3RegionResolutionTest.PublicBucket [ OK ] S3RegionResolutionTest.PublicBucket (372 ms) [ RUN ] S3RegionResolutionTest.RestrictedBucket [ OK ] S3RegionResolutionTest.RestrictedBucket (341 ms) [ RUN ] S3RegionResolutionTest.NonExistentBucket [ OK ] S3RegionResolutionTest.NonExistentBucket (1490 ms) [ RUN ] S3RegionResolutionTest.InvalidBucketName [ OK ] S3RegionResolutionTest.InvalidBucketName (135 ms) [----------] 4 tests from S3RegionResolutionTest (2340 ms total) [----------] 2 tests from S3FileSystemRegionTest [ RUN ] S3FileSystemRegionTest.Default [ OK ] S3FileSystemRegionTest.Default (135 ms) [ RUN ] S3FileSystemRegionTest.EnvironmentVariable [ OK ] S3FileSystemRegionTest.EnvironmentVariable (136 ms) [----------] 2 tests from S3FileSystemRegionTest (271 ms total) [----------] 1 test from TestMinioServer [ RUN ] TestMinioServer.Connect [ OK ] TestMinioServer.Connect (706 ms) [----------] 1 test from TestMinioServer (706 ms total) [----------] 26 tests from TestS3FS [ RUN ] TestS3FS.GetFileInfoRoot [ OK ] TestS3FS.GetFileInfoRoot (197 ms) [ RUN ] TestS3FS.GetFileInfoBucket [ OK ] TestS3FS.GetFileInfoBucket (244 ms) [ RUN ] TestS3FS.GetFileInfoObject [ OK ] TestS3FS.GetFileInfoObject (300 ms) [ RUN ] TestS3FS.GetFileInfoSelector [ OK ] TestS3FS.GetFileInfoSelector (257 ms) [ RUN ] TestS3FS.GetFileInfoSelectorRecursive [ OK ] TestS3FS.GetFileInfoSelectorRecursive (277 ms) [ RUN ] TestS3FS.GetFileInfoGenerator [ OK ] TestS3FS.GetFileInfoGenerator (240 ms) [ RUN ] TestS3FS.CreateDir API: PutObject(bucket=bucket, object=somefile/) Time: 01:32:48 UTC 07/04/2023 DeploymentID: 010901b4-1aed-45ad-a0ee-e0af17423c04 RequestID: 176E861CC5143A4A RemoteHost: 127.0.0.1 Host: 127.0.0.8:54373 UserAgent: aws-sdk-cpp/1.10.55 Linux/5.15.0-1040-azure x86_64 GCC/11.2.1 Error: mkdir /tmp/s3fs-test-9kk8hwry/bucket/somefile/: file exists (*fs.PathError) 5: internal/logger/logger.go:278:logger.LogIf() 4: cmd/fs-v1.go:1036:cmd.(*FSObjects).putObject() 3: cmd/fs-v1.go:1012:cmd.(*FSObjects).PutObject() 2: cmd/object-handlers.go:1830:cmd.objectAPIHandlers.PutObjectHandler() 1: net/http/server.go:2047:http.HandlerFunc.ServeHTTP() API: PutObject(bucket=bucket, object=somefile/) Time: 01:32:48 UTC 07/04/2023 DeploymentID: 010901b4-1aed-45ad-a0ee-e0af17423c04 RequestID: 176E861CC5143A4A RemoteHost: 127.0.0.1 Host: 127.0.0.8:54373 UserAgent: aws-sdk-cpp/1.10.55 Linux/5.15.0-1040-azure x86_64 GCC/11.2.1 Error: mkdir /tmp/s3fs-test-9kk8hwry/bucket/somefile/: file exists (*fs.PathError) 5: internal/logger/logger.go:278:logger.LogIf() 4: cmd/api-errors.go:2123:cmd.toAPIErrorCode() 3: cmd/api-errors.go:2148:cmd.toAPIError() 2: cmd/object-handlers.go:1832:cmd.objectAPIHandlers.PutObjectHandler() 1: net/http/server.go:2047:http.HandlerFunc.ServeHTTP() [ OK ] TestS3FS.CreateDir (335 ms) [ RUN ] TestS3FS.DeleteFile [ OK ] TestS3FS.DeleteFile (243 ms) [ RUN ] TestS3FS.DeleteDir [ OK ] TestS3FS.DeleteDir (294 ms) [ RUN ] TestS3FS.DeleteDirContents [ OK ] TestS3FS.DeleteDirContents (278 ms) [ RUN ] TestS3FS.DeleteDirContentsAsync API: PutObject(bucket=bucket, object=somefile/) Time: 01:32:49 UTC 07/04/2023 DeploymentID: a9a8af82-9903-4a4b-9d24-8647ccf47604 RequestID: 176E861D0468B79A API: CopyObject(bucket=s3fs-test-bucket, object=xxx/mno) Time: 01:32:58 UTC 07/04/2023 DeploymentID: b07a70c3-9139-46de-abf3-6bdfc538bf1c RequestID: 176E861F1AAC1E91 RemoteHost: 127.0.0.1 Host: 127.0.0.36:33197 UserAgent: aws-sdk-cpp/1.10.55 Linux/5.15.0-1040-azure x86_64 GCC/11.2.1 Error: file access denied (cmd.StorageErr) 6: internal/logger/logger.go:278:logger.LogIf() 5: cmd/fs-v1-helpers.go:360:cmd.fsRenameFile() 4: cmd/fs-v1.go:1107:cmd.(*FSObjects).putObject() 3: cmd/fs-v1.go:685:cmd.(*FSObjects).CopyObject() 2: cmd/object-handlers.go:1501:cmd.objectAPIHandlers.CopyObjectHandler() 1: net/http/server.go:2047:http.HandlerFunc.ServeHTTP() [ OK ] TestS3FSGeneric.MoveFile (380 ms) [ RUN ] TestS3FSGeneric.MoveDir /arrow/cpp/src/arrow/filesystem/test_util.cc:453: Skipped Filesystem doesn't allow moving directories [ SKIPPED ] TestS3FSGeneric.MoveDir (160 ms) [ RUN ] TestS3FSGeneric.CopyFile API: CopyObject(bucket=s3fs-test-bucket, object=def/mno) Time: 01:32:58 UTC 07/04/2023 DeploymentID: d53f2a07-9c5c-4a8e-aa2a-e6ba8713c057 RequestID: 176E861F397DC956 RemoteHost: 127.0.0.1 Host: 127.0.0.38:52109 UserAgent: aws-sdk-cpp/1.10.55 Linux/5.15.0-1040-azure x86_64 GCC/11.2.1 Error: file access denied (cmd.StorageErr) 6: internal/logger/logger.go:278:logger.LogIf() 5: cmd/fs-v1-helpers.go:360:cmd.fsRenameFile() 4: cmd/fs-v1.go:1107:cmd.(*FSObjects).putObject() 3: cmd/fs-v1.go:685:cmd.(*FSObjects).CopyObject() 2: cmd/object-handlers.go:1501:cmd.objectAPIHandlers.CopyObjectHandler() 1: net/http/server.go:2047:http.HandlerFunc.ServeHTTP() [ OK ] TestS3FSGeneric.CopyFile (368 ms) [ RUN ] TestS3FSGeneric.GetFileInfo [ OK ] TestS3FSGeneric.GetFileInfo (242 ms) [ RUN ] TestS3FSGeneric.GetFileInfoVector [ OK ] TestS3FSGeneric.GetFileInfoVector (295 ms) [ RUN ] TestS3FSGeneric.GetFileInfoSelector [ OK ] TestS3FSGeneric.GetFileInfoSelector (318 ms) [ RUN ] TestS3FSGeneric.GetFileInfoSelectorWithRecursion [ OK ] TestS3FSGeneric.GetFileInfoSelectorWithRecursion (398 ms) [ RUN ] TestS3FSGeneric.GetFileInfoAsync [ OK ] TestS3FSGeneric.GetFileInfoAsync (231 ms) [ RUN ] TestS3FSGeneric.GetFileInfoGenerator [ OK ] TestS3FSGeneric.GetFileInfoGenerator (298 ms) [ RUN ] TestS3FSGeneric.OpenOutputStream [ OK ] TestS3FSGeneric.OpenOutputStream (385 ms) [ RUN ] TestS3FSGeneric.OpenAppendStream /arrow/cpp/src/arrow/filesystem/test_util.cc:932: Skipped Filesystem doesn't allow file appends [ SKIPPED ] TestS3FSGeneric.OpenAppendStream (215 ms) [ RUN ] TestS3FSGeneric.OpenInputStream [ OK ] TestS3FSGeneric.OpenInputStream (288 ms) [ RUN ] TestS3FSGeneric.OpenInputStreamWithFileInfo [ OK ] TestS3FSGeneric.OpenInputStreamWithFileInfo (280 ms) [ RUN ] TestS3FSGeneric.OpenInputStreamAsync [ OK ] TestS3FSGeneric.OpenInputStreamAsync (253 ms) [ RUN ] TestS3FSGeneric.OpenInputFile [ OK ] TestS3FSGeneric.OpenInputFile (265 ms) [ RUN ] TestS3FSGeneric.OpenInputFileWithFileInfo [ OK ] TestS3FSGeneric.OpenInputFileWithFileInfo (281 ms) [ RUN ] TestS3FSGeneric.OpenInputFileAsync [ OK ] TestS3FSGeneric.OpenInputFileAsync (260 ms) [ RUN ] TestS3FSGeneric.SpecialChars [ OK ] TestS3FSGeneric.SpecialChars (301 ms) [----------] 26 tests from TestS3FSGeneric (8043 ms total) [----------] Global test environment tear-down [==========] 62 tests from 6 test suites ran. (21022 ms total) [ PASSED ] 60 tests. [ SKIPPED ] 2 tests, listed below: [ SKIPPED ] TestS3FSGeneric.MoveDir [ SKIPPED ] TestS3FSGeneric.OpenAppendStream pure virtual method called terminate called without an active exception /build/cpp/src/arrow/filesystem ``` ### Component(s) C++, Continuous Integration -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
