ss666 opened a new pull request, #11544:
URL: https://github.com/apache/seatunnel/pull/11544
### Purpose of this pull request
Closes #8816.
This PR adds the **source** side of the Fluss connector. It lets SeaTunnel
read from Fluss log tables, both as a bounded batch read and as an unbounded
streaming read.
Main additions:
1. `FlussSource` / `FlussSourceFactory` — `SeaTunnelSource` implementation
and factory wiring (`SupportColumnProjection`), reusing the existing
`connector-fluss` module.
2. `FlussSourceSplitEnumerator` — discovers table buckets, assigns them as
splits, and supports snapshot/restore with stable split allocation across
restarts.
3. `FlussSourceSplitReader` / `FlussSourceReader` / `FlussRecordEmitter` —
poll records per bucket, honor the checkpoint lock, and handle
bounded/unbounded end conditions.
4. `StartMode` + `FlussSourceOptions` — `earliest` / `latest` startup offset
options.
5. `FlussTypeConverter` — maps Fluss types to SeaTunnel `SeaTunnelDataType`.
6. English & Chinese connector docs under
`docs/{en,zh}/connectors/source/Fluss.md`, and registration in
`plugin-mapping.properties` / `config/plugin_config`.
### Does this PR introduce _any_ user-facing change?
Yes — a new `Fluss` source connector becomes available (previously only the
Fluss sink existed). It only adds a new capability; no existing option key,
default, or behavior is changed.
### How was this patch tested?
Scoped to the `connector-fluss` / `connector-fluss-e2e` modules (the modules
this PR changes):
* `./mvnw spotless:apply` pass.
* `./mvnw test -pl seatunnel-connectors-v2/connector-fluss` → 26/26, 0
failures
(FlussSourceSplitEnumeratorTest 8, FlussSourceFactoryTest 12,
FlussSourceSplitReaderTest 6)
* Built the dist + connector for e2e:
`./mvnw install -DskipTests -Dskip.spotless=true -pl
seatunnel-dist,seatunnel-e2e/seatunnel-connector-v2-e2e/connector-fluss-e2e
-am` → BUILD SUCCESS
* `./mvnw verify -pl
seatunnel-e2e/seatunnel-connector-v2-e2e/connector-fluss-e2e
-Dit.test=FlussSourceIT -DskipUT=true -DskipIT=false -DfailIfNoTests=false` →
4/4, 0 failures
(testFlussSource, testFlussSourceAllTypes, testFlussSourceStreaming,
testFlussSourceStreamingLatest)
### Check list
* [ ] If any new Jar binary package adding in your PR, please add License
Notice according
[New License
Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
* [x] If necessary, please update the documentation to describe the new
feature. https://github.com/apache/seatunnel/tree/dev/docs
* [ ] If necessary, please update `incompatible-changes.md` to describe the
incompatibility caused by this PR.
* [x] If you are contributing the connector code, please check that the
following files are updated:
1. Update
[plugin-mapping.properties](https://github.com/apache/seatunnel/blob/dev/plugin-mapping.properties)
and add new connector information in it
2. Update the pom file of
[seatunnel-dist](https://github.com/apache/seatunnel/blob/dev/seatunnel-dist/pom.xml)
3. Add ci label in
[label-scope-conf](https://github.com/apache/seatunnel/blob/dev/.github/workflows/labeler/label-scope-conf.yml)
4. Add e2e testcase in
[seatunnel-e2e](https://github.com/apache/seatunnel/tree/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/)
5. Update connector
[plugin_config](https://github.com/apache/seatunnel/blob/dev/config/plugin_config)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
