nzw921rx commented on PR #13239:
URL: https://github.com/apache/seatunnel/pull/13239#issuecomment-6815582866
Please reconsider the API design around two abstractions:
**`SourceEnumeratorMetrics`** and **`SourceReaderMetrics`**.
1. **`SourceEnumeratorMetrics`**
- Describes enumerator-owned progress, such as split discovery, creation,
assignment and known completion.
- Reports facts available to the Enumerator without inferring Reader
state.
2. **`SourceReaderMetrics`**
- Describes Reader-owned progress, such as actual reading progress,
current processing state and consumed positions where applicable.
- Reports facts maintained by the Reader and its split state.
These should be general Source-level contracts that can support both **Batch
and CDC**. Different source/split implementations may provide different
concrete progress types. Shared implementations can provide reusable progress
models, while CDC snapshot and incremental details should remain specific
implementations of the common contracts.
Please keep the public extension points within these two responsibilities
and avoid additional capability markers or empty report-interface hierarchies
unless there is a concrete requirement.
Before continuing with the implementation, please propose a simplified API
using these names and show one Batch example and one CDC example. The proposal
should clarify:
- What each component maintains and exposes.
- Which progress semantics are common and which are implementation-specific.
- How unsupported or temporarily unavailable information is represented.
- How the engine collects the information without blocking processing.
The exact methods and type parameters can be determined from these examples.
We should settle the responsibilities and extensibility first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]