liwuhen opened a new pull request, #540:
URL: https://github.com/apache/paimon-rust/pull/540
### Purpose
`normalize_storage_config` currently applies mirrored key pairs from a single
snapshot of the normalized properties. This cannot resolve aliases that
require
more than one mirror step.
For Azure, underscore aliases are declared through a dotted intermediate key:
- `azure.account_name` → `azure.account.name`
- `azure.account.name` → `azure.account-name`
However, `AzdlsConfig` reads the hyphenated canonical key
`azure.account-name`. Because newly mirrored keys are not processed again,
`azure.account_name` is silently ignored. The same issue affects the other
declared Azure underscore aliases.
This change replaces symmetric mirroring with direct, ordered
`alias → canonical` mappings. It also defines deterministic precedence when
multiple forms are supplied:
1. An explicitly supplied canonical key
2. The first matching alias in declaration order
For Azure, dotted aliases are declared before underscore aliases, resulting
in:
`canonical > dotted alias > underscore alias`
### Brief change log
- Change `normalize_storage_config` from symmetric one-pass mirroring to
direct
`alias → canonical` resolution.
- Preserve an explicitly supplied canonical value.
- Resolve multiple aliases in declaration order.
- Map every supported Azure dotted and underscore alias directly to the
hyphenated key read by `AzdlsConfig`:
- `account_name`
- `account_key`
- `sas_token`
- `client_id`
- `client_secret`
- `tenant_id`
- `authority_host`
- Update the S3, COS, OBS, and GCS alias tables to follow the same
directional
mapping contract.
### Tests
- Unit test `test_azdls_config_parse_underscore_aliases`
(`storage_azdls.rs`): data-driven coverage for every declared Azure
underscore alias and verification that each value reaches the corresponding
`AzdlsConfig` field. **Fails on the pre-fix code.**
- Unit test `test_azdls_config_alias_priority` (`storage_azdls.rs`): verifies
canonical keys take precedence over aliases and dotted aliases take
precedence over underscore aliases.
- Existing storage configuration tests verify the directional alias
conversion
does not regress S3, COS, OBS, or GCS configuration parsing.
- Commands run locally:
- `CARGO_INCREMENTAL=0 cargo test -p paimon --features storage-azdls --lib
storage_azdls`
- `CARGO_INCREMENTAL=0 cargo test -p paimon --no-default-features
--features storage-s3,storage-cos,storage-azdls,storage-obs,storage-gcs --lib
io::storage_`
- `cargo fmt --all -- --check`
- `git diff --check`
### API and Format
No public API or persisted data format changes.
Supported external storage configuration aliases now resolve directly to the
canonical keys consumed by each backend. When multiple forms of the same
setting are supplied, precedence is deterministic: the canonical key wins,
followed by aliases in declaration order.
### Documentation
No documentation changes required.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
