This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-4.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-4.1 by this push:
new b97d8040fec [fix](function) size from_base64/to_base64 output buffer
correctly on branch-4.1 (#65141)
b97d8040fec is described below
commit b97d8040fecc8f154411c2b578524b93fc7638ad
Author: Steven Pall <[email protected]>
AuthorDate: Fri Jul 3 05:42:17 2026 -0700
[fix](function) size from_base64/to_base64 output buffer correctly on
branch-4.1 (#65141)
On branch-4.1, `FromBase64Impl::vector` and `ToBase64Impl::vector` size
the output scratch buffer as `cipher_len = srclen / 2`. base64 decode
writes up to `srclen*3/4` bytes and encode writes up to
`4*ceil(srclen/3)`, both larger than `srclen/2`. When `srclen/2` falls
at or below `MAX_STACK_CIPHER_LEN` (64 KiB) while the real output
exceeds 64 KiB, the write overflows the 64 KiB `stack_buf` and corrupts
the stack frame, causing a delayed SIGSEGV inside
`StringOP::push_value_string` (the output `ColumnString` PODArray
reference is clobbered). It reproduces on valid, correctly padded base64
(input length a multiple of 4), so the length guard in #64788 does not
prevent it.
master is not affected. It was refactored to pre-reserve the output
column at the true size (`total_size += len/4*3` for decode,
`4*((len+2)/3)` for encode) and write directly, which removed the
`srclen/2` scratch path. That change is not in any released 4.1.x, and
the auto cherry-pick of #64788 to branch-4.1 is marked
`dev/4.1.x-conflict`. This PR is the minimal sizing fix for the 4.1
line.
Fix (matches `FromBase64BinaryImpl`, which already sizes correctly):
```cpp
// FromBase64Impl::vector
auto cipher_len = srclen / 4 * 3;
// ToBase64Impl::vector
auto cipher_len = (srclen + 2) / 3 * 4;
```
Verification: built stock be-4.1.2 with only this change and ran it in a
production disaggregated cluster. A query decoding a 120000-character
valid base64 value (90 KB output, inside the 64 KiB stack window)
crashed the BE before and returns correctly after. Confirmed across all
rows wider than 100 KB in a real table (19382 rows, max decode 612 KB)
with no BE restarts.
Repro on stock 4.1.2:
```sql
SELECT length(from_base64(v)) FROM t; -- v: a 120000-char base64 string of
a repeated byte
```
Signed-off-by: Steven Pall <[email protected]>
---
be/src/exprs/function/function_string.cpp | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/be/src/exprs/function/function_string.cpp
b/be/src/exprs/function/function_string.cpp
index b080f22622e..4ce5e484b86 100644
--- a/be/src/exprs/function/function_string.cpp
+++ b/be/src/exprs/function/function_string.cpp
@@ -1124,7 +1124,7 @@ struct ToBase64Impl {
continue;
}
- auto cipher_len = srclen / 2;
+ auto cipher_len = (srclen + 2) / 3 * 4;
char* dst = nullptr;
if (cipher_len <= stack_buf.size()) {
dst = stack_buf.data();
@@ -1167,7 +1167,7 @@ struct FromBase64Impl {
continue;
}
- auto cipher_len = srclen / 2;
+ auto cipher_len = srclen / 4 * 3;
char* dst = nullptr;
if (cipher_len <= stack_buf.size()) {
dst = stack_buf.data();
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
