Hi all, I'd like to propose adding built-in Alibaba Cloud (Aliyun) authentication support to the REST Catalog, in the same spirit as the existing sigv4 auth type for AWS.
*Background* MaxCompute (ODPS) now exposes a REST Catalog compatible service for managed Iceberg tables: https://www.alibabacloud.com/help/en/maxcompute/user-guide/maxcompute-managed-iceberg-tables-beta Clients must sign each request using Alibaba Cloud's AK/SK signature protocol (V2/V4), with a service-specific canonical request layout — for MaxCompute that means the ODPS variant. *Motivation* Today, Spark / Flink / Trino users connecting to MaxCompute have to ship a custom authentication jar and wire it in via rest.auth.type=<fully qualified class name>. Every engine deployment has to package, distribute, and version-align that jar separately, which is the same friction that motivated sigv4 being brought in-tree. A built-in aliyun type removes that friction for a non-trivial user base on Alibaba Cloud. *Design* The change is purely additive: no spec changes, no public API breaks, default behavior unchanged. Because Alibaba Cloud products (MaxCompute today, potentially OSS and others later) share AK/SK + HMAC but differ in canonical request construction and signing version, we introduce a small two-layer abstraction: - AliyunAuthManager / AliyunAuthSession handle credential resolution, STS token, and request dispatch. - AliyunRequestSigner is a pluggable interface; a signing-name config selects the concrete signer. - OdpsRequestSigner is the first implementation, covering V2 and V4 signing. This keeps the auth type name stable (aliyun) while leaving room to add more product signers without re-litigating naming later. *Scope* - core: register "aliyun" as a built-in auth type in AuthProperties / AuthManagers. - Under the existing iceberg-aliyun module: AliyunAuthManager, AliyunAuthSession, AliyunRequestSigner (generic layer), and OdpsRequestSigner (odps subpackage). - AccessKey/SecretKey with env var fallback (ALIBABA_CLOUD_ACCESS_KEY_ID / ALIBABA_CLOUD_ACCESS_KEY_SECRET, matching the official Alibaba Cloud SDK), optional STS token, optional region to enable V4. *Configuration* rest.auth.type=aliyun aliyun.auth.signing-name=odps aliyun.auth.access-key-id=<id> aliyun.auth.access-key-secret=<key> aliyun.auth.region=cn-hangzhou # optional, enables V4 aliyun.auth.sts-token=<token> # optional *Alternatives considered* - OAuth2 / bearer tokens: MaxCompute's REST Catalog endpoint currently authenticates via AK/SK signatures only, so OAuth2 is not an option on the server side yet. - Keep it as an external jar: works, but pushes packaging and version alignment onto every engine integrator — the same reason sigv4 was brought in-tree. - A single OdpsAuthManager without the signer interface: simpler today, but would force a second auth type (and another round of naming discussion) the moment another Alibaba Cloud product needs signing. *Naming* I went with "aliyun" because it matches the short name used across Alibaba Cloud's official SDKs and CLI. Happy to bikeshed — "alibabacloud" and "alibaba-cloud" are also reasonable. Same for the module name (iceberg-aliyun already exists). *Maintenance* Alibaba will own ongoing maintenance of the iceberg-aliyun module and respond to issues, in line with the expectations for vendor-specific code in the project. I have a working implementation with tests ready and can open a draft PR alongside this discussion if the direction looks reasonable. Thanks, Guoliang Yang
