surawut-jirasaktavee opened a new pull request, #69927: URL: https://github.com/apache/doris/pull/69927
### What problem does this PR solve? Issue Number: close #69645 Related PR: None Problem Summary: Reading an Azure location with a plus sign in its key could silently read a different object. With both `a+b/1/a.csv` (content `21`) and `a b/1/a.csv` (content `91`) stored, `https://acct.blob.core.windows.net/c/a%2Bb/1/a.csv` returned `91` without any error. A literal `+` in `https://`, `wasb[s]://`, `abfs[s]://` and `s3://` locations resolved to the same wrong key. Root cause: `AzureUri.decodeKey` used `URLDecoder`, which follows form encoding and turns `+` into a space, although `+` in a URI path is a literal plus sign. For `https://` locations, `StorageUriUtils.convertAzureToS3Style` has already decoded `%2B` to `+` through `URI.getPath()`, so the escaped form was affected as well. Fix: protect `+` before calling `URLDecoder`, so only `%XX` escapes are decoded. | Location | Before | After | | --- | --- | --- | | `https://…/a%2Bb/1/a.csv` | `a b/1/a.csv` (`91`) | `a+b/1/a.csv` (`21`) | | `https://…/a+b/1/a.csv` | `a b/1/a.csv` (`91`) | `a+b/1/a.csv` (`21`) | | `abfss://…/a+b/1/a.csv` | `a b/1/a.csv` (`91`) | `a+b/1/a.csv` (`21`) | | `s3://c/a+b/1/a.csv` | `a b/1/a.csv` (`91`) | `a+b/1/a.csv` (`21`) | | `https://…/a%20b/1/a.csv` | `a b/1/a.csv` (`91`) | `a b/1/a.csv` (`91`) | Why `convertAzureToS3Style` keeps `getPath()`: the issue also suggests `getRawPath()`, but BE uses keys literally and catalog scans send the normalized location to BE (`FileQueryScanNode`). With `getRawPath()`, an `https://` location containing `%20` would start reading the literal key `a%20b`. Known limitation: `https://` paths containing `%25` are still decoded twice in FE, as they were before this change, so `100%25/…` still fails with `IllegalArgumentException`. Fixing that means making FE and BE agree on whether a normalized location is encoded, which is left for a separate change. ### Release note Fix Azure locations with a plus sign in the object key reading a different object. ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [x] Unit Test - [x] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> Unit tests: `AzureUriTest` and `AzureFileSystemTest`, 65 passed. Without the fix, the two new literal-plus tests fail. They were run with Maven on the `fe-filesystem-azure` module, not with `run-fe-ut.sh`. Manual test: compiled `StorageUriUtils` and `AzureUri`, then read the resolved keys back from an Azurite emulator holding both objects. All nine location forms tried returned the expected object. Not tested: a full `s3()` query through a Doris cluster, or real Azure storage. There is no regression test because one would need Azure credentials. - Behavior changed: - [ ] No. - [x] Yes. <!-- Explain the behavior change --> A literal `+` in an Azure object key is now kept instead of being read as a space. - Does this need documentation? - [x] No. - [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into --> 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
