keksmd opened a new pull request, #982: URL: https://github.com/apache/incubator-graphar/pull/982
### Reason for this PR Closes #981. Parquet files without an Offset Index, including the Arrow C++ edge chunks in `testing/ldbc_sample`, could not be read by a partial row range. ### What changes are included in this PR? - When a partly selected row group has an Offset Index for every read column, it is still read page by page with `readFilteredRowGroup`. - When it does not, the row group is read whole and sliced to the half-open range. Row groups outside the range are still skipped. - `ROW_RANGE` is reported as applied only when every partly touched row group was read through its Offset Index, and as declined otherwise. The rows returned are identical either way. ### Are these changes tested? Yes. `ParquetPhysicalReaderWithoutOffsetIndexTest` reads a canonical `ldbc_sample` edge chunk that has no Offset Index (range, projection and limit). It also compares a multi-row-group file with an Offset Index against the same file with the page indexes stripped: the rows are identical and only the applied/declined sets differ. `mvn -pl io-parquet -am clean verify`: io-parquet 15 tests, io-api 85, 0 failures. ### Are there any user-facing changes? Reads that used to throw now succeed and report `ROW_RANGE` as declined. No API change. ## Checklist - [x] I have performed a self-review of my own code. - [ ] I have formatted my own code using `make cpplint` before submitting when changed files are in the `cpp` directory. - [x] I have performed `pre-commit run` before commit the changed files. - [x] I have added tests to prove my changes are effective. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
