raulcd commented on issue #47908: URL: https://github.com/apache/arrow/issues/47908#issuecomment-4630648970
After some more careful examination I've found we have quite a lot of minio/mc fixtures/specifics which are going to be time consuming to migrate. I've tested seaweedfs on this draft PR (still lots of things to be done there): - https://github.com/apache/arrow/pull/50104 From a Python perspective all tests except one were successful when launching seaweedfs. The one that failed is a specific behavior change on directory deletion which seems that seaweedfs does not behave as S3/Minio, specifically: ``` ====================================================== FAILURES ======================================================= _________________________________ test_delete_dir_with_explicit_subdir[S3FileSystem] __________________________________ fs = <pyarrow._s3fs.S3FileSystem object at 0x7f917845a800> pathfn = <method-wrapper '__add__' of str object at 0x7f91d92e9f00> def test_delete_dir_with_explicit_subdir(fs, pathfn): # GH-38618: regression with AWS failing to delete directories, # depending on whether they were created explicitly. Note that # Minio doesn't reproduce the issue, so this test is not a regression # test in itself. skip_fsspec_s3fs(fs) d = pathfn('directory/') nd = pathfn('directory/nested/') # deleting dir with explicit subdir fs.create_dir(d) fs.create_dir(nd) fs.delete_dir(d) dir_info = fs.get_file_info(d) assert dir_info.type == FileType.NotFound # deleting dir with blob in explicit subdir d = pathfn('directory2') nd = pathfn('directory2/nested') f = pathfn('directory2/nested/target-file') fs.create_dir(d) fs.create_dir(nd) with fs.open_output_stream(f) as s: s.write(b'data') fs.delete_dir(d) dir_info = fs.get_file_info(d) > assert dir_info.type == FileType.NotFound E assert <FileType.Directory: 3> == <FileType.NotFound: 0> E E Use -v to get more diff opt/conda/envs/arrow/lib/python3.13/site-packages/pyarrow/tests/test_fs.py:895: AssertionError ``` There's also a couple more tests that fail to set minio/mc specific policies and are just skipped at the moment. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
