gripleaf opened a new pull request, #211:
URL: https://github.com/apache/paimon-cpp/pull/211
### Purpose
Linked issue: N/A
Expose scan metrics through the public `TableScan` API and improve
observability for scan planning, prefetch, read-ahead cache, and asynchronous
I/O.
This change adds:
- Public access to scan metrics through `TableScan::GetMetrics()`.
- Scan planning metrics for:
- Manifest read latency.
- Snapshot manifest cache hits, misses, load latency, and store latency.
- Lazy-decode scanned and materialized row counts.
- Prefetch metrics for queue activity, produced and consumed ranges,
discarded ranges, errors, adaptive disabling, producer read latency, and
consumer wait latency.
- Read-ahead cache metrics for cache hits, misses, waits, prefetch
operations, evictions, cached bytes, pending bytes, and latency.
- Physical synchronous and asynchronous I/O request, byte, failure,
pending, and latency metrics.
- Metrics forwarding through delegating, realtime, snapshot, and system
table scan implementations.
### Tests
Added and updated unit tests covering:
- Public `TableScan` metric access.
- Scan duration and manifest read metrics.
- Snapshot manifest cache hit and rebuild paths.
- Lazy-decode scanned and materialized row metrics.
- Prefetch metrics, including adaptive prefetch disabling.
- Read-ahead cache reuse and eviction metrics.
- Physical synchronous and asynchronous I/O metrics.
- Repeated metric collection without double counting.
- Parquet and ORC prefetch paths with predicate pushdown.
Validation performed:
- `paimon-core-test`: 1667 tests passed.
- `paimon-common-test --gtest_filter='-StatusDeathTest.*'`: 1433 tests
passed.
- Post-rebase focused prefetch and read-ahead cache tests: 37 tests passed.
- Post-rebase focused table scan and scan planning tests: 10 tests passed.
- `git diff origin/main...HEAD --check` passed.
The unfiltered common test run was not completed because the existing
`StatusDeathTest.TestAbort` hung in the local environment.
### API and Format
This change affects the public API under `include/paimon/`:
- Adds `TableScan::GetMetrics()`.
- Exposes prefetch metric names.
- Adds `ReadAheadCacheMetrics` and `ReadAheadCache::GetMetrics()`.
There are no storage format or protocol changes.
### Documentation
This change introduces new metrics APIs and metric names. No standalone
documentation files are added; the public declarations and tests describe the
available metrics and their behavior.
### Generative AI tooling
Generated-by: OpenAI Codex (GPT-5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
