lizhimins opened a new issue, #133: URL: https://github.com/apache/rocketmq-flink/issues/133
## Motivation The project is currently a single Maven module mixing the FLIP-27 connector, the legacy `SourceFunction`/`SinkFunction` API and the SQL packaging. A code review plus verification against a real Aliyun commercial 5.x instance surfaced a batch of critical issues: - `RocketMQRowDeserializationSchema.deserialize()` is a no-op, so the SQL source emits empty rows (the SQL path is effectively broken) - `RocketMQWriter` never closes its producer (connection/thread leak) and `write()` blocks on every send; non-transactional sends NPE on `getTransactionId()` - `RocketMQSplitReader.fetch()` swallows Throwable, leaving the source in a zombie busy-loop on errors - offset lookup fails on cloud instances whose route table only exposes proxy broker names (`The broker[...] not exist`), breaking partition discovery - source/sink builders have empty `sanityCheck()`; several documented options are never applied (ghost parameters: send timeout/retry, pull batch size, namespace, filters, TLS, client timeouts) ## Proposed changes 1. Restructure into a multi-module layout: `flink-connector-rocketmq` + `flink-sql-connector-rocketmq`, moving legacy classes under `org.apache.flink.streaming.connectors.rocketmq` 2. Fix the issues above; add a consumer-side fallback for offset lookup when admin broker-name resolution fails on cloud instances 3. Wire the declared-but-unused config options and drop dead ones 4. Add GitHub Actions CI (spotless + build/tests on JDK 11/17) replacing the stale `.travis.yml`, rewrite the README following the official Flink connector layout, and split option references into `docs/` ## Verification - 101 unit tests pass - Manual integration tests against a real commercial instance: FLIP-27 DataStream pipeline (20/20), EXACTLY_ONCE two-phase commit (10/10), SQL DDL end-to-end (10/10) A follow-up PR will add a gRPC connector track based on `LiteSimpleConsumer` for RocketMQ 5.x lite topics. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
