github-actions[bot] commented on code in PR #69321:
URL: https://github.com/apache/doris/pull/69321#discussion_r4322062722
##########
be/src/exprs/function/array/function_array_flatten.cpp:
##########
@@ -68,22 +68,42 @@ class FunctionArrayFlatten : public IFunction {
.data();
while (src_data_type_array->get_nested_type()->get_primitive_type() ==
TYPE_ARRAY) {
- nested_src_column_array_ptr = assert_cast<const ColumnArray*>(
-
remove_nullable(src_column_array_ptr->get_data_ptr()).get());
+ const auto* nullable =
+
check_and_get_column<ColumnNullable>(src_column_array_ptr->get_data());
+ auto nested_array =
remove_nullable(src_column_array_ptr->get_data_ptr());
+ const auto& array = assert_cast<const ColumnArray&>(*nested_array);
+
+ // IF can leave [101, 102] beneath a NULL subarray. Remove that
range before
+ // composing offsets, so [NULL, [9]] flattens to [9] at every
nesting level.
+ if (nullable && nullable->has_null()) {
Review Comment:
[P2] Avoid copying descendants for empty NULL ranges
`has_null()` also selects ordinary NULL arrays whose nested ranges are
empty. For example, with an `a ARRAY<INT>` table column,
`array_flatten(array(CAST(NULL AS ARRAY<INT>), a))` now copies every value and
null byte of `a`, although there is no hidden payload to remove. The previous
final `ColumnNullable::clone()` only copied the wrapper and shared its child
buffers. Nested empty NULL arrays repeat this recursive copying at each level
(up to eight full leaf copies at the supported depth nine), adding substantial
CPU and allocation traffic for large arrays. Skip rebuilding when every NULL
subarray has an empty range, or defer pruning so the surviving leaf data is
copied once.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
