Gabriel39 commented on code in PR #66206:
URL: https://github.com/apache/doris/pull/66206#discussion_r3674822131
##########
be/src/exec/scan/file_scanner_v2.cpp:
##########
@@ -758,6 +809,22 @@ Status FileScannerV2::_build_projected_columns(const
format::TableReader& table_
const bool prefer_exact_name_match =
!_params->__isset.history_schema_info ||
supports_iceberg_scan_semantics_v1(_params);
+ const auto format_type = get_range_format_type(*_params, _current_range);
+ if (table_format_name(_current_range) == "iceberg" &&
+ format_type != TFileFormatType::FORMAT_PARQUET) {
+ const bool projects_variant =
Review Comment:
Fixed. ScannerV2 now uses a shared semantic COUNT(*) predicate based on an
explicitly empty push_down_count_slot_ids list and excludes the retained
placeholder from Variant format validation. COUNT(col) remains a real
projection.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergUtils.java:
##########
@@ -713,7 +713,11 @@ public static Type
icebergTypeToDorisType(org.apache.iceberg.types.Type type, bo
.collect(Collectors.toCollection(ArrayList::new));
return new StructType(nestedTypes);
case VARIANT:
- return Type.UNSUPPORTED;
+ // Iceberg Variant uses the Parquet Variant encoding directly.
Mark it compute-only
+ // so BE scanners materialize ColumnVariantV2 without changing
persisted Doris
+ // table metadata semantics.
+ return new org.apache.doris.catalog.VariantType(
Review Comment:
Fixed. Iceberg scans with a semantic root or nested Variant projection now
reject smooth-upgrade source backends, while metadata-only COUNT(*) remains
allowed. Added FE coverage for the gate.
##########
be/src/format_v2/parquet/reader/native_column_reader.cpp:
##########
@@ -114,6 +185,12 @@ void collect_projected_ids(const ParquetColumnSchema&
schema,
const format::LocalColumnIndex* projection,
const NativeFieldSchema& native_field,
std::set<uint64_t>* ids) {
DORIS_CHECK(ids != nullptr);
+ if (schema.kind == ParquetColumnSchemaKind::VARIANT) {
Review Comment:
Fixed by the typed-leaf projection implementation: lossless single-key
object access retains only the requested shredded physical leaf instead of
expanding the full Variant subtree; root Variant projection still requests the
full tree. Mapper and statistics tests verify the retained projection.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
