Gabriel39 commented on code in PR #66941:
URL: https://github.com/apache/doris/pull/66941#discussion_r3826773175
##########
be/src/format_v2/parquet/reader/variant_column_reader.cpp:
##########
@@ -879,58 +1153,88 @@ class ParquetVariantShreddedState final : public
VariantShreddedState {
const ParquetColumnSchema* wrapper_schema = nullptr;
ColumnPtr wrapper = struct_child(*typed_schema, typed, key,
&wrapper_schema);
if (!wrapper) {
- return path_miss();
+ return seek_residual();
+ }
+ ColumnPtr residual = struct_child(*wrapper_schema, wrapper,
"value", nullptr);
+ typed = struct_child(*wrapper_schema, wrapper, "typed_value",
&typed_schema);
+ if (!typed) {
+ // The wrapper stores this field unshredded. Its residual
holds the whole value, so
+ // the remainder of the path resolves inside those bytes.
+ add_level(std::move(residual), nullptr);
+ return seek_residual();
+ }
+ add_level(std::move(residual), typed);
+
+ if (position + 1 < path.size()) {
+ // Every intermediate typed_value must be an object. A legacy
untyped path
+ // produced through an array/explode operation cannot cross a
repeated node, so
+ // those rows resolve from the residual beside it instead.
+ if (typed_schema->kind != ParquetColumnSchemaKind::STRUCT) {
+ levels.pop_back();
Review Comment:
[P1] Keep the intermediate field residual when its typed schema is primitive
This removes the level that was just added for the current wrapper. Valid
shredded data can declare `a.typed_value` as a primitive while a particular row
contains `{"a":{"b":7}}`; that row has a null typed value and stores the object
in `a.value`. An `a.b` lookup reaches this branch, drops `a.value`, and seeks
from the parent/root residual instead. Because an object residual is disjoint
from its shredded fields, the parent residual cannot contain `a`, so the query
returns SQL NULL instead of 7. Please retain the current level and resolve the
remaining path from its residual for typed-null rows, and add a test mixing
primitive typed rows, fallback object rows, and missing rows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
