PDGGK opened a new pull request, #9238:
URL: https://github.com/apache/paimon/pull/9238
### Purpose
Closes #8852.
`OSSFileIO` gives no way to switch off the OSS SDK's CNAME heuristic, and on
an OSS-compatible endpoint that is not an Aliyun domain that heuristic drops
the bucket out of the host being signed. Every request then fails with
`SignatureDoesNotMatch`.
The path, in `aliyun-sdk-oss` 3.17.4 — the version this module pins:
```java
// OSSUtils.buildCanonicalHost
boolean isCname = false;
if (clientConfig.isSupportCname()) {
isCname = cnameExcludeFilter(host, clientConfig.getCnameExcludeList());
}
if (bucket != null && !isCname && !clientConfig.isSLDEnabled()) {
cannonicalHost.append(bucket).append(".").append(host);
} else {
cannonicalHost.append(host); // <- bucket never reaches the
signed host
}
```
`cnameExcludeFilter` returns `false` only when the host ends with an entry
of the exclude list, which defaults to `aliyuncs.com, aliyun-inc.com,
aliyun.com`. Any other host — a private-cloud or Apsara Stack deployment, a
MinIO-style gateway — therefore takes the `isCname = true` branch.
`supportCname` defaults to `true` in `ClientConfiguration:90`, and
`hadoop-aliyun` 3.3.4 builds a bare `new ClientConfiguration()`
(`AliyunOSSFileSystemStore:99`) with no reference to cname anywhere in that
artifact, so there is currently nothing a user can set to opt out.
### What changes
One new key, `fs.oss.cname.enabled`, handled the same way
`fs.oss.sld.enabled` already is:
```java
if (!hadoopOptions.getBoolean(OSS_CNAME_ENABLED, true)) {
disableCname(fs);
}
```
`disableCname` reaches the client's `ClientConfiguration` through the
existing `getOssClient(fs)` +
`ReflectionUtils.getPrivateFieldValue(ossClient, "serviceClient")` pair that
`enableSecondLevelDomain` uses, and calls `setSupportCname(false)`.
Setting it after `fs.initialize(...)` is enough because the flag is read per
request, not captured at construction: `OSSRequestMessageBuilder:170` fetches
`innerClient.getClientConfiguration()` while building each request and hands it
to `determineFinalEndpoint` on line 177. That is the same object —
`ServiceClient.getClientConfiguration()` returns the instance it was
constructed with, and `OSSClient.getClientConfiguration()` (`:426`) is a
straight delegation to it. The neighbouring `isSLDEnabled()` is consumed one
line later, on 178, so the existing `fs.oss.sld.enabled` path already depends
on exactly this.
### Blast radius
Unset means unchanged. The default is `true`, matching the SDK's own
default, so a catalog that does not set the key builds precisely the client it
built before, and a public-cloud endpoint is unaffected in either position of
the flag — its host is on the exclude list, so the heuristic never fires. The
second test below pins that.
### Tests
Two, both asserting the host the signer actually computes rather than that a
setter was called — `signedHost` calls `OSSUtils.determineFinalEndpoint` with
the client's own configuration, which is the call `OSSRequestMessageBuilder`
makes for every request.
| endpoint | before | after `setSupportCname(false)` |
|---|---|---|
| `oss-cn-x.inter.env99.example.com` | `oss-cn-x.inter.env99.example.com` |
`my-bucket.oss-cn-x.inter.env99.example.com` |
| `oss-cn-hangzhou.aliyuncs.com` | `my-bucket.oss-cn-hangzhou.aliyuncs.com`
| `my-bucket.oss-cn-hangzhou.aliyuncs.com` |
The first row is the bug — a bucket that never reaches the host, so the
client signs one string and the server another. The second is the blast-radius
check.
Making `setSupportCname` a no-op fails exactly one of the 17 tests in the
class, and it is the first row:
```
[ERROR] OSSFileIOTest.testDisableCnameRestoresTheBucketInTheSignedHost:563
expected: "my-bucket.oss-cn-x.inter.env99.example.com"
but was: "oss-cn-x.inter.env99.example.com"
[ERROR] Tests run: 17, Failures: 1, Errors: 0, Skipped: 0
```
The public-cloud test stays green under that mutation, which is what makes
it a blast-radius check rather than a second copy of the first.
**One thing worth knowing, because it caught me out while writing the
test.** My first version built the client with `new
OSSClientBuilder().build(endpoint, "ak", "sk")` and the first assertion failed
— the bucket was *already* in the host. `ClientBuilderConfiguration extends
ClientConfiguration` and sets `supportCname = false` in its constructor, so the
SDK's own builder opts out of this heuristic by default. It is specifically the
bare `new ClientConfiguration()` that `hadoop-aliyun` uses which leaves it on.
The test therefore constructs the client the way `AliyunOSSFileSystemStore`
does; a test written against `OSSClientBuilder` would pass without the fix and
prove nothing.
I have no private-cloud OSS deployment to point at, so what is verified here
is the host string the SDK builds for signing, not an end-to-end request
against such an endpoint. That is the step the reporter identified as the
failure, and it is the step this change moves.
### API and Format
No change to any existing key, signature or on-disk format. One added
configuration key, inert unless set to `false`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
