JingsongLi commented on code in PR #1014:
URL: https://github.com/apache/paimon-rust/pull/1014#discussion_r4371278651
##########
crates/paimon/src/table/data_evolution_reader.rs:
##########
@@ -1397,6 +1397,8 @@ async fn resolve_descriptor_columns(
file_io: &FileIO,
limiter: &BlobReadLimiter,
) -> crate::Result<RecordBatch> {
+ // Descriptor fields reference objects the table does not own, which may
be replaced.
+ let file_io = file_io.origin_only();
Review Comment:
[P2] Apply external-descriptor origin routing to PK reads too
This guard covers data-evolution descriptor fields, but primary-key tables
also legally support `blob-descriptor-field=payload`. They keep the external
descriptor inline and read it through `table_read::read_pk_with_blob` →
`managed_blob_reader::resolve_batch`, which still dereferences it using the
original routed table FileIO. With the documented `meta,read,write` / `data`
policy, I created and committed such a PK table referencing
`oss://bkt/import/data-<uuid>-1.png`; origin holds NEW and the target retains
OLD. Ordinary TableRead returns OLD and the source GET reaches the target. The
same persisted table read with routing disabled returns NEW. No local block
cache is enabled. Give schema-declared external descriptor fields the
origin-only view in the PK payload/predicate resolver as well, while preserving
target reads for the table-owned `.managed.blob` references. The append-only
descriptor tests do not exercise this path.
##########
crates/paimon/src/table/mod.rs:
##########
@@ -1125,6 +1167,7 @@ impl Table {
} else {
schema_manager.with_branch(&branch)
};
+ table.file_io = with_schema_history(&table.file_io,
&table.schema_manager);
Review Comment:
[P2] Restore the branch's current prefixes after retaining its FileIO
The REST branch path replaces the newly loaded branch FileIO with the source
table's FileIO at line 1163, losing the branch schema's prefixes. Adding only
the history loader here does not restore them when a REST catalog owns the
schema without a storage `schema/` directory, a supported case documented above
this block. I reproduced a main schema with `main-` and a REST branch schema
with `branch-`, static OSS credentials and data routing: directly loading the
branch sends GET `branch-<uuid>-1.parquet` to the target, but
`main.copy_with_branch("dev")` has the same schema/path/identity and sends that
GET only to origin after an empty schema listing. This silently loses
configured cache/accelerator allocation for the branch's current files. Reapply
`table_file_io` with the loaded branch schema after retaining the original
provider/FUSE FileIO, then attach the branch history loader; cover catalog-only
branch switching.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
