lucasfang opened a new pull request, #274:
URL: https://github.com/apache/paimon-cpp/pull/274
### Purpose
Linked issue: close #xxx
A `RestCatalog` authenticates its own requests, but the data of a table was
still read and written with the storage credentials configured in the catalog
options, which are scoped to the whole warehouse. This change lets the catalog
server be the authority for data access as well, following the
`RESTTokenFileIO` design of Java Paimon.
- `RestApi::LoadTableToken` calls `GET
/v1/{prefix}/databases/{db}/tables/{table}/token`. A 403 is reported to the
caller as an error status carrying a `RestErrorDetail` instead of being retried.
- `RestTokenFileSystem` loads the token of one table on first use, rebuilds
its delegate once the token expires in less than
`kTokenExpirationSafeTimeMillis` (1 hour, matching the Java constant), and
forwards every operation to the delegate. The token overrides the catalog file
system options, matching `RESTUtil.merge(catalogOptions, token.token())`;
`dlf.oss-endpoint` is applied on top as `fs.oss.endpoint` when it is not empty.
- `Catalog::GetTableFileSystem(identifier)` returns the file system to use
for a table. The default implementation returns `GetFileSystem()`, so
`FileSystemCatalog` and the other catalogs keep their current behavior.
`RestCatalog` overrides it and returns a `RestTokenFileSystem` only when
`data-token.enabled` is set; it normalizes a system table or the main branch to
the identifier of the table the credentials belong to, so `db.t$snapshots` and
`db.t$branch_main` share the credentials of `db.t`.
- The instances are held in a `GenericLruCache` bounded to 1000 entries,
matching the `maximumSize` of the Java Caffeine cache. An evicted table is
served by a new instance that loads a token of its own, so the bound trades a
token request for a bounded footprint.
- `sys.partitions` lists the files of every table, so it now asks the
catalog for the file system of each table instead of using the catalog-level
one. A table that disappears while the query runs is skipped, like the existing
handling of its location.
Two new catalog options: `data-token.enabled` (default `false`) and
`dlf.oss-endpoint`.
Callers opt in explicitly:
```cpp
PAIMON_ASSIGN_OR_RAISE(std::shared_ptr<FileSystem> fs,
catalog->GetTableFileSystem(identifier));
PAIMON_ASSIGN_OR_RAISE(ReadContext ctx,
ReadContextBuilder(...).WithFileSystem(fs).Build());
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
