xiaoh1024 opened a new pull request, #3890:
URL: https://github.com/apache/parquet-java/pull/3890
### Rationale for this change
`ParquetFileReader` recursively advances past empty row groups and groups
with no matching column-index-filtered rows. A long run of skipped groups grows
the call stack until reading fails with `StackOverflowError`.
### What changes are included in this PR?
Replace recursive skipping with iteration in `readNextRowGroup()` and
`readNextFilteredRowGroup()`. Each skipped group advances the reader position,
and each call returns the next matching group or `null` at EOF. The existing
group-reading, dictionary-reader and buffer-release logic is retained.
### Are these changes tested?
Extend `TestParquetReaderEmptyBlock` with runs of 50,000 skipped groups,
covering leading, intermediate and trailing empty groups, all-empty input, both
sequential reader entry points, and non-empty groups excluded by column-index
filtering. The assertions check returned row counts, group indices and EOF.
```bash
./mvnw -pl parquet-hadoop -am test \
-Dtest=TestParquetReaderEmptyBlock \
-Dsurefire.failIfNoSpecifiedTests=false
```
### Are there any user-facing changes?
Sequential readers can skip long runs of empty groups or groups with no
matching column-index-filtered rows without the stack overflow caused by
recursive skipping.
Closes #3889
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
