kimotorc opened a new issue #10965: URL: https://github.com/apache/arrow/issues/10965
When I read in a parquet file using `pyarrow.parquet.read_table`, the files opened for read don't seem to close. Is there a way to specifically close these opened files? I looked at `pyarrow.parquet.ParquetDataset` as well and there doesn't seem to be a way to force the closure of the files opened for read. Here's my use case: I have a custom fsspec filesystem that I've created to interface with an S3 like API. When calling `open` the filesystem downloads the remote file locally and returns a custom file handle like [ this one](https://github.com/intake/filesystem_spec/blob/3cdfd6cf8386857815535a32f747cd2033b30da7/fsspec/implementations/cached.py#L713). It looks for __exit__ or close to clean up the local file which doesn't ever happen after reading in with pyarrow.parquet. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
