leizhiyuan opened a new issue, #1412: URL: https://github.com/apache/rocketmq-clients/issues/1412
### Before Creating the Enhancement Request - [x] I have confirmed that this should be classified as an enhancement rather than a bug/feature. ### Programming Language of the Client Rust ### Summary This is a tracking issue for the Rust client. It brings together: - the defects found by an end-to-end test of `LitePushConsumer` / `PushConsumer` against a managed RocketMQ 5 server, with the Java client 5.2.2 run against the same server, group and topic as a control; - the Rust issues and PRs that are already open. The goal is that reviewers can see what each PR fixes, where they overlap, and in which order they can be merged with the least rebase work. I will keep the lists below up to date. ### Motivation - Six Rust PRs are open, and most of them touch the same few files (`push_consumer.rs`, `lite_subscription_manager.rs`, `client.rs`, `lib.rs`). Several pairs conflict with each other, and two of them implement overlapping logic (see below). - On servers that return the assigned topic name in a decorated form, `LitePushConsumer` on `master` cannot receive any message (#1406). Unsubscribing never works (fixed in #1381). The lite subscription quota is not enforced (#1408). These should be fixed before a release that advertises LiteTopic support (#1400). - #1361 (Support LiteSimpleConsumer) was closed by the stale bot while its PR #1381 is still open. It may be worth reopening it. ### Describe the Solution You'd Like #### Open Rust issues and PRs | Issue | PR | Topic | Files | |---|---|---|---| | #1406 | #1407 | PushConsumer takes the filter expression from the subscription, not the assigned queue's topic name; backoff on receive failures | `push_consumer.rs` | | #1408 | #1409 | LitePushConsumer applies server `Settings` / `NotifyUnsubscribeLiteCommand`; `start()` waits for the first `Settings`; quota `0` no longer means unlimited | `push_consumer.rs`, `lite_push_consumer.rs`, `lite_subscription_manager.rs`, `lib.rs` | | #1410 | #1411 | A panic in the message listener is treated as `FAILURE` instead of killing the worker | `push_consumer.rs`, `client.rs` (test lock only) | | #1403 | #1404 | Telemetry session recovery after proxy rolling update, bounded session map, keep-alive, receive deadline, server-pushed settings | `session.rs`, `client.rs`, `conf.rs`, `push_consumer.rs`, `simple_consumer.rs`, `Cargo.toml` | | #1361 (closed by stale bot) | #1381 | LiteSimpleConsumer; also fixes `lite_topic` missing in ack / change-invisible-duration and `unsubscribe_lite` always failing; cancellable periodic sync | `lite_*`, `client.rs`, `simple_consumer.rs`, `util.rs`, `lib.rs`, `Cargo.toml`, ... | | #1400 | — | Request a new release with LiteTopic support | — | | #488 | — | End-to-end tests for the Rust client | — | #### Defects found by the end-to-end test | # | Defect | Java behavior | Status | |---|---|---|---| | 1 | PushConsumer / LitePushConsumer never receives messages when the assigned queue's topic name differs from the subscription key (e.g. `<instance>%<topic>`); the receive loop then spins every 1 ms with no backoff (8.9M error lines in 8 min) | filter expression comes from the subscription; 1 s / 20 ms backoff | #1406 / #1407 | | 2 | `unsubscribe_lite` always fails with `ClientIsNotRunning` (`check_started` on the cloned client) | works | fixed in #1381 | | 3 | LitePushConsumer never applies server lite settings or `NotifyUnsubscribeLiteCommand`; quota not enforced (2010 subscriptions accepted with quota 2000) | 2001st rejected with `42902` | #1408 / #1409 | | 4a | `shutdown()` takes 20–345 s: in-flight long polls are not cancelled, and actors are shut down one by one | ~6 s | not filed yet | | 4b | the periodic lite sync task is not cancelled on shutdown and re-creates a session afterwards | cancelled | fixed in #1381 (`stop_scheduler`) | | 5 | FIFO local retry: `MessageView::delivery_attempt()` seen by the listener stays at 1 | `incrementAndGetDeliveryAttempt()` | not filed yet (next) | | 6 | `batch_send` of lite messages returns `Ok` with one receipt holding comma-joined ids, and the messages are never consumed | no public batch API | investigating (server or client not yet known) | | 7 | a panic in the message listener kills the consumer worker for good | `catch (Throwable)` → `FAILURE` | #1410 / #1411 | | 8 | receive gRPC deadline equals the long-polling timeout, so every idle long poll ends with an ERROR | `awaitDuration + requestTimeout` | fixed in #1404 (also #1403) | | 9 | FIFO vs. standard worker is chosen from the local option, not from the server settings | `getSettings().isFifo()` | partly in #1404 (PushConsumer); LitePushConsumer to be verified once #1404 and #1409 are in | | 10 | lite examples use `http://localhost:8080`, which fails with a DNS error | — | not filed yet | | 11 | `MessageBuilder` accepts a blank lite topic; the error only comes from the server (`40014` / `40020`) | `isNotBlank` check | not filed yet | | 12 | LitePushConsumer pre-fetches the route before the client is started, logging a WARN on every start | — | not filed yet | Other observations, not filed: an intermittent `h2 ... frame with invalid size` when the telemetry stream starts (about 2 in 20 starts, cause unknown). The `tail_n` and `timestamp` offset options have no effect either, but Java behaves the same against this server, so this is not a Rust client issue. #### Overlaps and suggested merge order Pairwise conflicts, checked with `git merge-tree` on the current PR heads: | | #1381 | #1404 | #1407 | #1409 | #1411 | |---|---|---|---|---|---| | **#1381** | | `Cargo.toml`, `client.rs`, `simple_consumer.rs` | clean | `lib.rs`, `lite_subscription_manager.rs` | clean | | **#1404** | | | clean | `push_consumer.rs` (1 hunk) | clean | | **#1407** | | | | clean | clean\* | | **#1409** | | | | | clean | \* #1407 + #1411 merge cleanly as text, but the two tests added by #1411 then need the extra `filter_expression` argument introduced by #1407. Suggested order: 1. **#1407.** It is small and merges cleanly with every other PR. LitePushConsumer is unusable on such servers without it. 2. **#1411.** It merges cleanly with every other PR. I will push the test fix-up right after #1407 lands. 3. **#1404**, then **#1381** rebased on it. These two conflict in `Cargo.toml`, `client.rs` and `simple_consumer.rs`. 4. **#1409**, which I will rebase onto #1404 and #1381: - With #1404 there is a one-hunk conflict in the telemetry loop. The fix is to keep both `apply_settings` and the forwarding. - With #1381 the logic overlaps. Both add a "wait for the initial `Settings`" signal to `LiteSubscriptionManager`. #1409 will reuse the one from #1381 and keep the Java behavior for LitePushConsumer, which is to fail `start()` on timeout (`ClientSessionImpl#settingsInitFuture`). #1381 currently logs a warning and continues for LiteSimpleConsumer. It may be worth aligning that as well. Fixes for the remaining items (4a, 5, 10–12) will be opened as separate issue/PR pairs and linked here. ### Describe Alternatives You've Considered One large PR with all the fixes. I decided against it because the fixes are independent and much easier to review one by one, and a single PR would conflict with #1381 and #1404 even more. ### Additional Context - **Test setup.** About 60 scenarios across producer, LitePushConsumer subscribe / unsubscribe / offset options, ordering, retry, FIFO, quota, restart, multiple instances and shutdown. They ran against a managed RocketMQ 5 service, which pushes `lite_subscription_quota=2000`, `max_lite_topic_size=64`, `fifo=true` and `max_attempts=17`. Every finding above was compared with the Java client on the same server. I'm happy to contribute these scenarios towards #488. - **Clippy on `master`.** With the current stable toolchain locally (rustc 1.98), `cargo clippy --all-features -- -D warnings` already fails on `master` with 2 `for_kv_map` errors in `rust/src/client.rs`. If CI's stable clippy is at that level, the Clippy job will be red on every Rust PR, independent of its content. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
