github-actions[bot] commented on code in PR #69333:
URL: https://github.com/apache/doris/pull/69333#discussion_r4336309439
##########
be/src/exec/operator/file_scan_operator.cpp:
##########
@@ -149,6 +149,36 @@ bool
FileScanLocalState::TEST_should_use_file_scanner_v2(const TQueryOptions& qu
bool FileScanLocalState::should_use_file_scanner_v2(const TQueryOptions&
query_options,
bool is_load,
const
TFileScanRangeParams& scan_params) {
+ if (is_load) {
+ // ADBC and Fluss are query-only connectors. Their default Parquet
format must not
+ // admit them to the ordinary file-load conversion path.
+ if (scan_params.__isset.table_format_params &&
+ (scan_params.table_format_params.table_format_type == "adbc" ||
+ scan_params.table_format_params.table_format_type == "fluss")) {
+ return false;
+ }
+ // Load planners omit query session options. Thrift's default makes an
omitted V2
+ // option indistinguishable from false, so ingestion selects by format
instead.
+ switch (scan_params.format_type) {
+ case TFileFormatType::FORMAT_PARQUET:
Review Comment:
[P2] Preserve year-zero timestamps when enabling V2 Parquet loads
Selecting V2 for every Parquet load turns supported year-zero local
timestamps into NULL. For example, an optional INT64 `TIMESTAMP(MICROS,
isAdjustedToUTC=false)` value `-62167219200000000` loaded into a nullable
STRING column previously produced `0000-01-01 00:00:00.000000` through V1's
`Int64ToTimestamp` and CAST. V2's `parquet_timestamp_micros` rejects values
before 0001, and its non-strict native decoder marks this non-NULL physical
value as NULL. `LoadTableReader::_convert` then sees an already-NULL source, so
`strict_mode=true` still accepts/stores NULL without counting a rejected row; a
required destination instead rejects the formerly valid row. Preserve the prior
load behavior for Doris' supported year-zero values before routing these loads
unconditionally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
