github-actions[bot] commented on code in PR #65218:
URL: https://github.com/apache/doris/pull/65218#discussion_r3527292498
##########
be/src/format_v2/parquet/parquet_reader.cpp:
##########
@@ -482,13 +491,25 @@ Status ParquetReader::get_block(Block* file_block,
size_t* rows, bool* eof) {
}
const auto predicate_filtered_rows_before =
_state->scheduler.predicate_filtered_rows();
- RETURN_IF_ERROR(_state->scheduler.read_next_batch(_state->file_context,
_state->file_schema,
- *request_snapshot,
file_block, rows, eof));
+ const auto raw_rows_read_before = _state->scheduler.raw_rows_read();
+ Status st = _state->scheduler.read_next_batch(_state->file_context,
_state->file_schema,
+ *request_snapshot,
file_block, rows, eof);
+ if (!st.ok()) {
+ if (_io_ctx != nullptr && _io_ctx->should_stop) {
+ *rows = 0;
+ *eof = true;
+ return Status::OK();
+ }
+ return st;
+ }
_sync_page_cache_profile();
if (_io_ctx != nullptr) {
_io_ctx->predicate_filtered_rows +=
_state->scheduler.predicate_filtered_rows() -
predicate_filtered_rows_before;
}
+ const auto raw_rows_read = _state->scheduler.raw_rows_read();
+ DORIS_CHECK(raw_rows_read >= raw_rows_read_before);
+ _record_scan_rows(raw_rows_read - raw_rows_read_before);
_eof = *eof;
return Status::OK();
}
Review Comment:
The read just above this accounting still misses the new stop conversion.
`DorisRandomAccessFile::ReadAt()` now returns an Arrow `IOError("stop")` when
`io_ctx->should_stop` is set, and `ParquetReader::get_block()` converts a
failure from `scheduler.read_next_batch()` to EOF in that case. But
`COUNT(col)` pushdown calls `get_aggregate_result()` directly; for complex
columns this path can hit the same Arrow reader through
`shape_reader->load_nested_levels_batch(batch_rows)`, and the `RETURN_IF_ERROR`
propagates the resulting IOError through
`TableReader::_try_materialize_aggregate_pushdown_rows()` as a scan failure. A
LIMIT/cancel that calls `FileScannerV2::try_stop()` while this aggregate path
is loading levels can therefore fail the query instead of stopping cleanly.
Please add the same `io_ctx->should_stop` conversion around the aggregate-level
reads, and cover COUNT(col) aggregate pushdown cancellation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
