github-actions[bot] commented on code in PR #68301:
URL: https://github.com/apache/doris/pull/68301#discussion_r4063449574
##########
be/src/core/field.cpp:
##########
@@ -91,6 +91,38 @@ bool decimal_less_or_equal(Decimal128V3 x, Decimal128V3 y,
UInt32 xs, UInt32 ys)
return dec_less_or_equal<TYPE_DECIMAL128I>(x, y, xs, ys);
}
+namespace {
+// Expression literals can outlive decoder pages and source columns.
+// Keep the view first for Field::get(), and fit ownership into the existing
Field storage.
+struct OwnedBinaryField {
+ StringView view;
+ char* bytes = nullptr;
+
+ explicit OwnedBinaryField(const StringView& value) {
+ // Inline views already own their bytes; preserve their
allocation-free representation.
+ if (value.isInline()) {
+ view = value;
+ return;
+ }
+ // The Field must remain valid after the source column or decoder page
is released.
+ bytes = new char[value.size()];
+ memcpy(bytes, value.data(), value.size());
+ view = StringView(bytes, value.size());
+ }
+ OwnedBinaryField(const OwnedBinaryField&) = delete;
+ OwnedBinaryField& operator=(const OwnedBinaryField&) = delete;
Review Comment:
[P1] Allocate this retained payload through Doris's checked allocator. Every
non-inline VARBINARY value materialized as a Field now duplicates value.size()
bytes with raw new[]; this is reached by ordinary column extraction plus
Iceberg defaults and Parquet metadata, and the declared length can be
Integer.MAX_VALUE. The current jemalloc hook only routes allocation calls and
does not consume/release the task tracker, while raw new[] also skips
Allocator<false>::memory_check and its controlled MEM_ALLOC_FAILED path. A
large value (and the temporary/destination deep-copy peak) can therefore exceed
a query or process limit without Doris charging it or rejecting it at the
configured limit. Please use DorisUniqueBufferPtr<char> or an equivalent
allocator-backed owner that still fits Field::storage, and cover
allocation/destruction under a limited tracker.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
