alamb commented on PR #5041: URL: https://github.com/apache/arrow-rs/pull/5041#issuecomment-1795911107
> > Can you explain where this assumption is used in the code > > From the linked ticket #5036 > > > The async reader determines what data to fetch based on what rows are selected, however, when reading the data it performs each operation in turn. In order to perform the first skip, the reader must set up the decoders to the relevant position within the pages (as it doesn't know that the next operation is another skip). This in turn causes it to request data that wasn't fetched, and the reader bails out with an offset index error. I see -- so another potential fix would be to ignore zero length selections in the reader. I am not saying we should do this, only observing it might be a possiblility -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
