westonpace commented on PR #13830: URL: https://github.com/apache/arrow/pull/13830#issuecomment-1231682922
> The set_bounds is not a method for Fragment. It is a method for FileFragment which is not an abstract class. Are there other Fragments that are not FileFragments? And does a FileFragment have to be immutable as well? Yes, my wording was a bit incorrect there. There are non-file fragments such as in-memory fragment and fragments backed by python iterators. However, yes, FileFragment needs to be immutable as well so I think we can continue with this change. > What I can do is probably make a new MakeFragment method in the abstract FileFormat class which will make a FileFragment with a range. The FileFragment object has a protected attribute format_ which we can use for this purpose. When you call slice on a FileFragment it will just call return format_->MakeFragment(...., range). That seems workable. This would be fine and avoids the need for Clone. > However the FileFragment object doesn't have an open method. OpenReaderForRangeAsync and OpenReaderAsync are called by the CsvFileFormat, who has access to the FIleFragment. So the CsvFileFormat needs to decide which method to call on the FileSource, by looking at the other field in the FileFragment. I missed entirely that `OpenRange` was part of `FileSource` and not `FileFragment`. You are correct, the API you have for `OpenRange` is fine. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
