hubgeter commented on code in PR #66758:
URL: https://github.com/apache/doris/pull/66758#discussion_r3782340759
##########
be/src/format_v2/parquet/reader/variant_column_reader.cpp:
##########
@@ -851,18 +1048,32 @@ class ParquetVariantShreddedState final : public
VariantShreddedState {
std::lock_guard lock(_materialization_lock);
_materialized.reset();
_serialized.reset();
+ _unshredded_metadata_cache.reset();
return true;
}
std::optional<VariantShreddedTypedValue> find_typed_value(
std::span<const VariantShreddedPathSegment> path) const override {
+ auto residual_seek_fallback = [&]() ->
std::optional<VariantShreddedTypedValue> {
+ // Complete mixed shredded states still need canonical
reconstruction when neither a
+ // typed leaf nor the pure unshredded direct-seek path can answer
the request.
+ if (_complete && !unshredded_child_indices(*_schema).has_value() &&
+ find_child(*_schema, "value", nullptr) != nullptr) {
+
update_counter(_profile.variant_direct_residual_seek_fallbacks, 1);
+ }
+ return std::nullopt;
+ };
auto path_miss = [&]() -> std::optional<VariantShreddedTypedValue> {
update_counter(_profile.variant_direct_leaf_path_misses, 1);
- return std::nullopt;
+ return residual_seek_fallback();
};
if (path.empty()) {
return path_miss();
}
+ if (auto normalized = find_unshredded_normalized_value(path);
normalized.has_value()) {
Review Comment:
Fixed in 715e82f7eea. Accessed containers now use reusable structural lookup
state. Canonical objects use adjacent offsets, reused noncanonical objects
lazily promote a sorted index under a 4 MiB budget, and retained state is
capped at 16K entries and four ancestors with traversal-local overflow. Append
and selection invalidation plus wide, deep, and budget tests were added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
