danepitkin commented on issue #36962: URL: https://github.com/apache/arrow/issues/36962#issuecomment-1660937397
Hi @shou123, The `SkyhookFileFormat` was implemented by the authors as an extension to pyarrow. It is not part of the pyarrow codebase. We can consider this ticket as an enhancement request instead of as a bug. From the research paper: `"We extend the FileFormat API in Arrow to create a SkyhookFileFormat API that leverages the DOA API to enable offloading fragment scan operations to Ceph OSDs."`[1] [1] https://arxiv.org/pdf/2204.06074.pdf -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
