vustef commented on PR #7307: URL: https://github.com/apache/arrow-rs/pull/7307#issuecomment-3446831909
> > One concern I have with the approach here is how to provide exact row numbers if we start selectively reading row group metadata. If we don't have metadata for all preceding row groups, we can't know the starting row number. This at least argues for reverting back to using an `Option` for the start index. > > I don't think we will be able to provide row numbers if we don't have all the preceding row group metadata > > Given the main usecase I have heard so far is indexing and delete vectors, which require exact and accurate row numbers, I think it would be better if the reader simply returned an error if it was configured to read row numbers but didn't have enough information to do so Sounds good, I'll bring back the `Option`, and make row number array reader error out if it lacks that information. I'll hold off with pushing changes here until we finish discussion on the GH issue. And most likely will create a new PR, because I'm not able to update description of this one nor move it between draft and active. (If jkylling doesn't object. Will add due credits to him in there ofc, and keep his commits). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
