Hi devs, Recently, I have been working on the Fluss YAML Source to support full-database synchronization (whole-database CDC). During the implementation, I ran into two fundamental limitations in the current LogScanner API:
1. LogScanner can only subscribe to a single table. When a business needs to consume change logs from multiple tables simultaneously, users must create a separate LogScanner for each table and manually manage multiplexing at the application layer. This leads to thread proliferation, resource waste (duplicate connections to the same TabletServer), and the inability to unify consumption into a single poll call. 2. LogScanner truncates logs by expected schema, losing schemaId and raw data. The current implementation projects records according to the schema provided at scanner creation time, which strips the schemaId metadata and discards columns added by subsequent schema changes. For CDC scenarios, we need to faithfully forward the original record data along with its schema version to downstream systems. To address these issues, I'd like to propose FIP-43: Multi-Table Subscription API[1] to solve both problems. Looking forward to your feedback and suggestions! Best, Hongshun [1] https://cwiki.apache.org/confluence/display/FLUSS/FIP-43++Fluss+Multi-Table+Subscription+API
