jackylee-ch opened a new pull request, #10174:
URL: https://github.com/apache/paimon/pull/10174
## Purpose
pypaimon supports streaming reads with a `consumer-id` (#7349) and
persists each consumer's progress under `{table}/consumer/`, but unlike
Java it did not expose that progress as the standard `db.t$consumers`
system table — so under a REST catalog the internal consumer API was the
only way to inspect consumption progress.
## Change
- Add `ConsumersTable`, mirroring Java `ConsumersTable`: columns
`consumer_id STRING NOT NULL` and `next_snapshot_id BIGINT NOT NULL`,
primary key `consumer_id`. It reads the base table's `ConsumerManager`
(already exposed via `FileStoreTable.consumer_manager()`, branch-aware)
and materializes one row per consumer, sorted by id, keeping the
declared `(string, int64)` schema even when empty.
- Register `consumers` in the system-table loader and drop it from the
not-yet-registered list.
## Tests
- End-to-end via the catalog: `db.t$consumers` loads, schema/primary-key
layout, read back recorded consumers, empty-table schema, projection.
- Update the loader registry test to expect `consumers`.
Written with Claude Code; verification is mine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
