Tom-Newton commented on issue #40025: URL: https://github.com/apache/arrow/issues/40025#issuecomment-1939289270
> I would be grateful if you ran these tests manually against a real storage account with HNS support to catch any issues with the tests themselves. Turns out this was definitely a good idea. I was expecting everything to pass but there where actually a lot of failures ``` FAILED pyarrow/tests/test_fs.py::test_filesystem_is_functional_after_pickling[builtin_pickle-AzureFileSystem] - OSError: Failed to create directory '156d07e6-c9d2-11ee-989a-71cec6336ac8/a/aa/aaa/': https://tomtesthns.blob.core.windows.net/156d07e6-c9d2-11ee-989a-71cec6336ac8 Azure Error: [InvalidUri] 400 The request URI is invalid. FAILED pyarrow/tests/test_fs.py::test_filesystem_is_functional_after_pickling[cloudpickle-AzureFileSystem] - OSError: Failed to create directory '165e703f-c9d2-11ee-989a-71cec6336ac8/a/aa/aaa/': https://tomtesthns.blob.core.windows.net/165e703f-c9d2-11ee-989a-71cec6336ac8 Azure Error: [InvalidUri] 400 The request URI is invalid. FAILED pyarrow/tests/test_fs.py::test_get_file_info[AzureFileSystem] - OSError: Failed to create directory '165e7042-c9d2-11ee-989a-71cec6336ac8/a/aa/aaa/': https://tomtesthns.blob.core.windows.net/165e7042-c9d2-11ee-989a-71cec6336ac8 Azure Error: [InvalidUri] 400 The request URI is invalid. FAILED pyarrow/tests/test_fs.py::test_get_file_info_with_selector[AzureFileSystem] - OSError: Failed to delete a directory: selector-dir/: https://tomtesthns.blob.core.windows.net/17d5de34-c9d2-11ee-989a-71cec6336ac8/selector-dir/ Azure Error: [InvalidUri] 400 The request URI is invalid. FAILED pyarrow/tests/test_fs.py::test_create_dir[AzureFileSystem] - OSError: Failed to create directory '17d5de35-c9d2-11ee-989a-71cec6336ac8/test-directory/': https://tomtesthns.blob.core.windows.net/17d5de35-c9d2-11ee-989a-71cec6336ac8 Azure Error: [InvalidUri] 400 The request URI is invalid. FAILED pyarrow/tests/test_fs.py::test_delete_dir[AzureFileSystem] - OSError: Failed to create directory '1925cd58-c9d2-11ee-989a-71cec6336ac8/directory/nested/': https://tomtesthns.blob.core.windows.net/1925cd58-c9d2-11ee-989a-71cec6336ac8 Azure Error: [InvalidUri] 400 The request URI is invalid. FAILED pyarrow/tests/test_fs.py::test_delete_dir_with_explicit_subdir[AzureFileSystem] - OSError: Failed to create directory '1925cd59-c9d2-11ee-989a-71cec6336ac8/directory/': https://tomtesthns.blob.core.windows.net/1925cd59-c9d2-11ee-989a-71cec6336ac8 Azure Error: [InvalidUri] 400 The request URI is invalid. FAILED pyarrow/tests/test_fs.py::test_delete_dir_contents[AzureFileSystem] - OSError: Failed to create directory '1a1e807e-c9d2-11ee-989a-71cec6336ac8/directory/nested/': https://tomtesthns.blob.core.windows.net/1a1e807e-c9d2-11ee-989a-71cec6336ac8 Azure Error: [InvalidUri] 400 The request URI is invalid. FAILED pyarrow/tests/test_fs.py::test_copy_file[AzureFileSystem] - OSError: Failed to copy a blob. (https://tomtesthns.blob.core.windows.net/1a1e807f-c9d2-11ee-989a-71cec6336ac8/test-copy-source-file -> https://tomtesthns.blob.core.windows.net/1a1e807f-c9d2-11ee-989a-71cec6336ac8/test-copy-target-file) Azure Error: [CannotVerifyCopySource] 401 Server ... FAILED pyarrow/tests/test_fs.py::test_move_directory[AzureFileSystem] - OSError: Failed to create directory '1b30faf0-c9d2-11ee-989a-71cec6336ac8/source-dir/': https://tomtesthns.blob.core.windows.net/1b30faf0-c9d2-11ee-989a-71cec6336ac8 Azure Error: [InvalidUri] 400 The request URI is invalid. FAILED pyarrow/tests/test_fs.py::test_move_file[AzureFileSystem] - OSError: Failed to delete a file: test-move-target-file: https://tomtesthns.blob.core.windows.net/1b30faf1-c9d2-11ee-989a-71cec6336ac8/test-move-target-file Azure Error: [LeaseIdMissing] 412 There is currently a lease on the resource and no lease ID was specified in the request. FAILED pyarrow/tests/test_fs.py::test_open_output_stream_metadata[AzureFileSystem] - OSError: CommitBlockList failed for 'https://tomtesthns.blob.core.windows.net/283b1457-c9d2-11ee-989a-71cec6336ac8/open-output-stream-metadata'. Committing is required to flush an output/append stream. Azure Error: [InvalidMetadata] 400 The metadata specified is invalid. It has charact... ``` Most of them are failed to create directory which I think boils down to `fs->CreateDir("directory/")` failing on hierarchical namespace. I will make an issue and PR about this case. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
