alamb commented on PR #27022:
URL: https://github.com/apache/datafusion/pull/27022#issuecomment-6662517274
> 1. The test files are small enough that "Read Parquet footer" is the whole
file (offsets `0..1402` / `0..838`). In `page_pruning_io` the page index is
clearly used (only page 1 is read for `id = 5`) but never shows up as its own
request, while `bloom_filter_io` shows the OffsetIndex as a separate one. A
short note (or a file where the footer read does not cover everything) would
help readers not assume the page index is free.
This is an excellent point -- I actually adjusted the test so that it sets
the metadata hint so that it only covers the metadata and not also the bloom
filters and offset indexes
> 2. `describe_range` returns on the first overlapping column chunk, so a
coalesced request spanning several chunks would only show the first one.
Fixex
> 3. `record_request` and `record_range` take the lock separately, so with
more than one partition the `Request N` lines could interleave with another
request's reads. Fine with `target_partitions = 1` today.
Fixex
> Nits: "acess" in the title, and double spaces in the `topk_io.rs` docs
("TopK filter is", "which proves").
Fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
