sundapeng opened a new pull request, #10820:
URL: https://github.com/apache/paimon/pull/10820
### Purpose
With `scan.primary-branch`, `FallbackReadFileStoreTable` puts the other
branch first, so `Read.mainRead` is the other branch's read.
`newSnapshotReader()` is delegated to the wrapped table and returns plain
`DataSplit`s of the wrapped table's files, but `Read.createReader` sends every
split that is not a `FallbackSplit` to `mainRead`.
Those files are then decoded with the other branch's schema manager. As soon
as the wrapped table has a schema id the other branch lacks (for example after
an `ALTER TABLE` on the main branch), reading fails with:
```
FileNotFoundException: .../branch/branch-<name>/schema/schema-N
at FileSystemSchemaManager.schema
at RawFileSplitRead.createFileReader
at FallbackReadFileStoreTable$Read.createReader
```
Jobs that plan with `table.newSnapshotReader()` and read with
`table.newRead()` hit this, e.g. maintenance jobs that rewrite the main
branch's files. Even when the schema ids happen to exist in both branches, the
main branch's files are read with the other branch's read path.
This change keeps a read for the wrapped table and uses it for plain splits.
With `scan.fallback-branch` and chain tables the wrapped table is already
first, so their behavior is unchanged.
### Tests
- Added
`FallbackReadFileStoreTableTest#testReadPlainSplitWithPrimaryBranch`: evolves
only the main branch's schema, writes with the new schema, then reads the
splits from `newSnapshotReader()` through `newRead()` with the primary branch
first. Without the fix it fails with `FileNotFoundException` on
`branch/branch-bc/schema/schema-1`; with the fix it returns the main branch's
rows.
- `FallbackReadFileStoreTableTest`: 26 tests pass.
### API and Format
No.
### Documentation
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
