github-actions[bot] commented on code in PR #66802:
URL: https://github.com/apache/doris/pull/66802#discussion_r3792132568
##########
be/src/exec/scan/file_scanner_v2.cpp:
##########
@@ -478,7 +535,8 @@ Status FileScannerV2::_get_block_impl(RuntimeState* state,
Block* block, bool* e
if (_table_reader_rf_num != _applied_rf_num) {
VExprContextSPtrs refreshed_conjuncts;
RETURN_IF_ERROR(_build_table_conjuncts(&refreshed_conjuncts));
-
RETURN_IF_ERROR(_table_reader->refresh_conjuncts(std::move(refreshed_conjuncts)));
+ RETURN_IF_ERROR(_table_reader->refresh_conjuncts(
+ std::move(refreshed_conjuncts),
_current_condition_cache_digest()));
Review Comment:
[P1] Restore COUNT(*) placeholder eligibility after the final pre-read RF
This refresh runs only after `_open_impl()` has already prepared the first
source/child, and one-row-group planning-reader reuse can even call
`open_reader()` before the hook. If the final Runtime Filter becomes ready at
this boundary, the split keeps `_all_runtime_filters_applied_for_split ==
false`; for generated children this call updates conjuncts and digest, but not
that one-way completeness snapshot. A `COUNT(*)` with the RF on supported
column B therefore leaves an unrelated planner carrier A unmarked. For valid
Parquet where A is adjusted `TIME_MILLIS` (the existing placeholder path
deliberately permits such unused carriers), `ParquetReader::open()` validates A
and returns `NotSupported` instead of filtering B and counting the survivors.
Synchronize ready filters before eager prepare/open, or defer that work until
after the hook; if the generated-child refresh remains, also propagate
completeness through it. Cover both generated-child and one-row-group reuse
paths with an
unsupported carrier plus a supported late RF.
##########
be/src/exec/scan/file_scanner_v2.cpp:
##########
@@ -448,16 +471,50 @@ Status FileScannerV2::_open_impl(RuntimeState* state) {
DORIS_CHECK(_table_reader != nullptr);
RETURN_IF_ERROR(_init_expr_ctxes());
RETURN_IF_ERROR(_init_table_reader(_current_range));
+ // Refine the first source split before yielding the scanner worker.
Other scanners may be
+ // waiting for its row-group children, so deferring publication until
a later get_block()
+ // turn could let those waiters occupy the scan thread pool ahead of
the producer.
+ bool eos = false;
+ RETURN_IF_ERROR(_prepare_next_split(&eos));
}
return Status::OK();
}
Status FileScannerV2::_get_next_scan_range(bool* has_next) {
SCOPED_TIMER(_get_next_range_timer);
DORIS_CHECK(has_next != nullptr);
- RETURN_IF_ERROR(_split_source->get_next(has_next, &_current_range));
+ RETURN_IF_ERROR(_split_source->get_next_split(has_next, &_current_split));
if (*has_next) {
+ _current_range = _current_split.materialize_range();
RETURN_IF_ERROR(_validate_scan_range(*_params, _current_range));
+ } else {
+ _current_split = {};
+ _current_range = {};
Review Comment:
[P2] Preserve the last range identity through final accounting
When the split source reports EOS, this clears the canonical range-level
`file_type` before `ScannerScheduler` performs its final
`update_realtime_counters()` and before close publishes load counters. Since
`TFileScanRangeParams.file_type` is deprecated and may be unset, an uncached
range-only S3/HDFS/HTTP scan then falls back to `FILE_LOCAL` and charges the
final task's bytes as local; a range-only `FILE_STREAM` TVF load also makes
`_should_update_load_counters()` false, dropping its accumulated
filtered/unselected rows. Keep the last materialized range until final
publication, or snapshot its accounting file type separately before clearing
the scheduling envelope. Add EOS regressions with the common file type unset
and range-level `FILE_S3` and `FILE_STREAM` values.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
