assokhi opened a new pull request, #10575:
URL: https://github.com/apache/seatunnel/pull/10575
### Purpose of this pull request
**Resolves #9566**
This pull request introduces a new **MQTT Sink Connector** for Apache
SeaTunnel V2. It enables high-performance, distributed data integration with
IoT endpoints and message brokers using the MQTT 3.1.1 protocol.
**Core Engineering Highlights**
* **Subtask Multiplexing:** Programmatically generates unique `clientId`
values by appending the engine's subtask index. This prevents broker-side
connection hijacking during parallel execution.
* **Memory Efficiency:** Uses `MemoryPersistence` from the Eclipse Paho
client to maximize throughput and avoid disk I/O overhead in containerized
environments.
* **Network Resilience:** Implements localized retry loops and
`MqttCallback` triggers to handle transient network interruptions gracefully.
* **Serialization Support:** Native integration with SeaTunnel's JSON and
Text serialization schemas.
**Registry & Integration Updates**
The following project configuration files were updated to properly register
the new connector:
* `seatunnel-connectors-v2/pom.xml` — Registered the `connector-mqtt` module.
* `seatunnel-e2e/seatunnel-connector-v2-e2e/pom.xml` — Registered the E2E
test module.
* `plugin-mapping.properties` — Added `seatunnel.sink.MQTT` mapping.
* `seatunnel-dist/pom.xml` — Added dependency with `provided` scope for the
distribution build.
* `.github/workflows/labeler/label-scope-conf.yml` — Configured automated
GitHub labeling.
* `config/plugin_config` — Registered the MQTT sink for startup scripts and
plugin loading.
### Does this PR introduce _any_ user-facing change?
Yes. This PR introduces a new MQTT Sink Connector that allows users to
publish SeaTunnel pipeline output to MQTT brokers.
Example configuration:
```hocon
sink {
MQTT {
url = "tcp://localhost:1883"
topic = "seatunnel/telemetry"
username = "admin"
password = "password"
qos = 1
format = "json"
}
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
