morrySnow opened a new pull request, #67841:
URL: https://github.com/apache/doris/pull/67841
## Problem
`URL_DECODE` returned different bytes depending on whether the expression
was folded by FE. For example:
```sql
SELECT HEX(URL_DECODE('%C0%AF'));
```
The folded path returned `EFBFBDEFBFBD`, while evaluation in BE returned
`C0AF`.
## Cause
The FE executable function delegates to Java `URLDecoder`, whose UTF-8
conversion silently replaces malformed byte sequences with U+FFFD. The BE URL
decoder has byte-level semantics and preserves each decoded `%HH` byte. A Java
string literal therefore cannot represent the BE result losslessly when the
decoded bytes are invalid UTF-8.
## Fix
- Reconstruct the byte stream produced by URL decoding and validate it with
a strict UTF-8 decoder before FE constant evaluation.
- If the decoded bytes are invalid UTF-8, abort FE evaluation so the
original function remains in the plan and BE preserves the bytes.
- Keep valid UTF-8 folding unchanged, including a legitimately encoded
U+FFFD value.
- Preserve the existing handling of malformed percent escapes.
## Tests
- Added executable-function unit coverage for `%80`, `%C0%AF`, `%E0%80%80`,
`%ED%A0%80`, `%FF`, valid multibyte text, and encoded U+FFFD.
- Added regression coverage comparing constant and dynamic execution paths
by their hexadecimal result.
- `StringArithmeticTest`: 7 tests passed.
- Full FE build and checkstyle passed.
- Targeted regression suite: 1/1 passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
