tolleybot commented on code in PR #39438:
URL: https://github.com/apache/arrow/pull/39438#discussion_r1473125211
##########
python/pyarrow/parquet/core.py:
##########
@@ -1327,17 +1327,15 @@ def __init__(self, path_or_paths, filesystem=None,
schema=None, *, filters=None,
except ValueError:
filesystem = LocalFileSystem(use_mmap=memory_map)
finfo = filesystem.get_file_info(path_or_paths)
- if finfo.is_file:
- single_file = path_or_paths
Review Comment:
Certainly,
Originally, the logic differentiated between handling a single file and
multiple files, which led to a somewhat fragmented approach. If a single file
was provided, it would create a fragment and then use FileSystemDataset to load
that file. However, the dataset class is well-equipped to handle single files
seamlessly, aligning with the handling of multiple files. This prompted the
revision of the logic to utilize the dataset class uniformly for both cases.
During this process, I observed that the only object passed into the
make_fragment function was consistently a BufferReader. This realization led me
to rename the variable to more accurately reflect its actual usage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
