jorisvandenbossche commented on code in PR #37768:
URL: https://github.com/apache/arrow/pull/37768#discussion_r1347240273
##########
python/pyarrow/tests/test_fs.py:
##########
@@ -699,12 +699,12 @@ def test_get_file_info_with_selector(fs, pathfn):
infos = fs.get_file_info(selector)
if fs.type_name == "py::fsspec+s3":
- # s3fs only lists directories if they are not empty, but depending
- # on the s3fs/fsspec version combo, it includes the base_dir
+ # s3fs only lists directories if they are not empty
# (https://github.com/dask/s3fs/issues/393)
Review Comment:
You can remove this issue link as well, because it is about the base dir
issue (not the empty directories)
##########
python/pyarrow/tests/test_fs.py:
##########
@@ -730,9 +726,10 @@ def test_get_file_info_with_selector(fs, pathfn):
# + for s3fs 0.5.2 all directories are dropped because of buggy
# side-effect of previous find() call
# (https://github.com/dask/s3fs/issues/410)
Review Comment:
This comment is for old s3fs version we also don't test anymore, so let's
remove it altogether
```suggestion
```
##########
python/pyarrow/tests/test_fs.py:
##########
@@ -699,12 +699,12 @@ def test_get_file_info_with_selector(fs, pathfn):
infos = fs.get_file_info(selector)
if fs.type_name == "py::fsspec+s3":
- # s3fs only lists directories if they are not empty, but depending
- # on the s3fs/fsspec version combo, it includes the base_dir
+ # s3fs only lists directories if they are not empty
# (https://github.com/dask/s3fs/issues/393)
- assert (len(infos) == 4) or (len(infos) == 5)
+ len(infos) == 4
else:
assert len(infos) == 5
+ assert base_dir not in infos
Review Comment:
I am not sure this would ever give false (because `infos` is a list of
FileInfo objects, and `base_dir` is a path. So you would have to do something
like `base_dir not in [info.path for info in infos]`, but in hindsight that's
actually ocvered already by the loop below (by removing the check for the base
"selector-dir", we are now actually already ensuring that it is not in the list
of `infos`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
