github-actions[bot] commented on code in PR #65548:
URL: https://github.com/apache/doris/pull/65548#discussion_r3575676853
##########
be/src/format_v2/parquet/parquet_reader.cpp:
##########
@@ -598,6 +658,19 @@ Status ParquetReader::get_aggregate_result(const
format::FileAggregateRequest& r
request.agg_type);
}
Review Comment:
The aggregate validation still misses one requested-column shape: primitive
`COUNT(col)`. Nereids can push `COUNT(unsupported_time)` into
`PhysicalFileScan` by default when the counted slot is non-nullable, but
`TableReader::_build_file_aggregate_request()` only adds `request.columns` for
complex `COUNT(col)`. For a primitive leaf it sends an empty COUNT request, so
this loop has no column to validate and the COUNT branch below returns the
row-group count from metadata. A required Parquet `TIME_MILLIS` leaf can
therefore return a count instead of the promised `NotSupported` error. Please
either pass the counted primitive projection through the COUNT aggregate
request or disable this pushdown when the requested Parquet leaf carries
`unsupported_reason`.
##########
be/src/exec/scan/file_scanner_v2.cpp:
##########
@@ -384,6 +390,20 @@ Status FileScannerV2::_get_block_impl(RuntimeState* state,
Block* block, bool* e
*eof = false;
continue;
Review Comment:
This still treats too many EOF statuses as valid empty splits. A Native file
that has a valid header followed by an explicit zero-length first block is
malformed: `NativeReader::_read_next_pblock()` consumes the block length, sets
`eof` when `block_len == 0` at file end, and `_ensure_schema_loaded()` turns
the empty buffer into `Status::EndOfFile`. The existing
`RejectsZeroLengthBlockAndInvalidPBlock` unit builds exactly `header +
uint64(0)` and expects `get_schema()` to fail. When TableReader lazily opens
that reader from `get_block()`, this branch would instead abort the split,
increment `EmptyFileNum`, and continue as if the file were a valid header-only
empty file. Please make the Native reader return a hard error for an explicit
zero-length block, or have the scanner skip only a dedicated empty-file status
that cannot also represent malformed input.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
