github-actions[bot] commented on code in PR #65715:
URL: https://github.com/apache/doris/pull/65715#discussion_r3636263760
##########
be/src/core/data_type_serde/arrow_validation.h:
##########
@@ -262,6 +302,47 @@ inline void check_arrow_list_offsets(const
arrow::ListArray& array, int64_t star
}
}
+// Validate LargeList offsets before reading offsets or recursing into values.
+inline void check_arrow_large_list_offsets(const arrow::LargeListArray& array,
int64_t start,
+ int64_t end) {
+ check_arrow_array_range(array, start, end);
+ const auto offsets =
arrow_validation_detail::get_int64_offsets_array(array);
+ const int64_t last_offset =
+ arrow_validation_detail::check_arrow_offsets_range(*offsets,
start, end);
+ const int64_t values_length = array.values() ? array.values()->length() :
0;
+ if (UNLIKELY(last_offset > values_length)) {
+ arrow_validation_detail::throw_invalid_arrow(
+ array, "offsets exceed values length: last_offset={},
values_length={}",
+ last_offset, values_length);
+ }
+}
+
+inline int64_t checked_fixed_size_list_offset(const arrow::FixedSizeListArray&
array,
+ int64_t index) {
+ const int64_t list_size = array.value_length();
+ if (UNLIKELY(index < 0 || list_size < 0 ||
+ (list_size != 0 && index >
std::numeric_limits<int64_t>::max() / list_size))) {
+ arrow_validation_detail::throw_invalid_arrow(
+ array, "fixed-size list offset overflows: index={},
list_size={}", index,
+ list_size);
+ }
+ return index * list_size;
Review Comment:
[P1] Preserve the FixedSizeList slice offset
This checked replacement no longer matches Arrow's `value_offset(i)`, which
computes `(i + array.offset()) * list_size`. When
`enable_arrow_input_validation=false`, valid sliced arrays are accepted, so
slicing `[[10,11],[20,21]]` to the second row still produces child range
`[0,2)` here and silently returns `[10,11]`. Include `array.offset()` in the
overflow-checked addition/multiplication and add a disabled-validation slice
test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
