goto456 opened a new issue, #16573:
URL: https://github.com/apache/dubbo/issues/16573

   ### Pre-check
   
   - [x] I am sure that all the content I provide is in English.
   
   
   ### Search before asking
   
   - [x] I had searched in the 
[issues](https://github.com/apache/dubbo/issues?q=is%3Aissue) and found no 
similar issues.
   
   
   ### Apache Dubbo Component
   
   Java SPI Extensions (apache/dubbo-spi-extensions)
   
   ### Dubbo Version
   
   - Reviewed published artifact: 
`org.apache.dubbo.extensions:dubbo-registry-consul:3.2.0` (JAR and sources).
   - The same control flow is present in the extension repository at commit 
`705910bd9bdd9e8f42c436c2a5d1927d5f7a2876`.
   - This is a source/artifact-based defect report. I have not yet run an 
independent upstream end-to-end reproduction, so no specific JDK, OS, or Consul 
server version is asserted as a tested runtime environment.
   
   ### Steps to reproduce this issue
   
   ### Summary
   
   A recoverable exception from a background Consul blocking query can 
terminate the subscription's `ConsulNotifier` task. There is no automatic 
continuation/resubmission for that failed task, so the consumer can retain a 
stale provider list after Consul becomes reachable again.
   
   This concerns recovery of the **registry watch**, not RPC invocation retries 
or provider TTL heartbeat recovery.
   
   ### Proposed fault-injection reproduction
   
   The following is a proposed integration fault-injection procedure, **not a 
claim that it has already been executed end-to-end**:
   
   1. Start a provider and consumer using the Consul registry. Route the 
consumer's Consul HTTP requests through a local test proxy that initially 
forwards normally.
   2. Wait until the initial subscription and provider-list notification 
succeed. Keep that subscription active.
   3. For one subsequent background query to `/v1/health/service/<service>`, 
have the proxy return HTTP 503 instead of forwarding it. Let the resulting 
Consul-client exception escape the query call.
   4. Immediately restore normal forwarding, then change the service's provider 
registrations.
   5. Observe whether further queries and provider-list notifications occur for 
the existing subscription.
   
   A deterministic unit test can instead provide a Consul-client stub that 
returns an initial valid response, throws one recoverable runtime exception on 
the first watch query, and would return updated data on later calls. Submit the 
actual notifier to an executor and inspect the returned Future in the test. The 
current control flow predicts exceptional task completion rather than a later 
query.
   
   ### What you expected to happen
   
   After a transient query failure, the active subscription should resume 
querying with bounded retry/backoff (or equivalent task recovery), and 
subsequent provider additions/removals should reach the consumer without 
restarting it.
   
   The watch failure should be observable in logs or task-health metrics.
   
   ### Anything else
   
   ### Actual behavior indicated by the implementation
   
   The following is confirmed by source/artifact inspection; the integration 
scenario above remains to be independently run:
   
   - 
[`doSubscribe()`](https://github.com/apache/dubbo-spi-extensions/blob/705910bd9bdd9e8f42c436c2a5d1927d5f7a2876/dubbo-registry-extensions/dubbo-registry-consul/src/main/java/org/apache/dubbo/registry/consul/ConsulRegistry.java#L149-L166)
 submits the notifier but discards the returned Future.
   - 
[`ConsulNotifier.run()`](https://github.com/apache/dubbo-spi-extensions/blob/705910bd9bdd9e8f42c436c2a5d1927d5f7a2876/dubbo-registry-extensions/dubbo-registry-consul/src/main/java/org/apache/dubbo/registry/consul/ConsulRegistry.java#L340-L349)
 directly loops over the processing methods without an exception boundary.
   - [The processing 
methods](https://github.com/apache/dubbo-spi-extensions/blob/705910bd9bdd9e8f42c436c2a5d1927d5f7a2876/dubbo-registry-extensions/dubbo-registry-consul/src/main/java/org/apache/dubbo/registry/consul/ConsulRegistry.java#L351-L376)
 invoke Consul queries and notifications without handling an escaping exception.
   
   Therefore an exception reaching `run()` ends that task. The executor 
captures the failure in its Future, but this caller does not inspect it. The 
executor's worker thread may survive; that does **not** mean the failed 
Runnable will automatically run again. A later explicit re-subscription may 
submit work again, but no autonomous recovery is visible in this path.
   
   ### Possible recovery considerations
   
   - Handle recoverable failures inside the watch lifecycle, with bounded 
backoff and failure logging; honor shutdown/interruption.
   - If supervising/resubmitting tasks, avoid duplicate watchers for the same 
subscription.
   - A blanket catch-and-continue is not sufficient for notification failures: 
the current methods advance `consulIndex` before conversion/listener delivery. 
Recovery should retain failed updates or otherwise ensure they are not skipped 
when the same index is returned.
   
   Could maintainers confirm whether another branch/release already addresses 
this failure mode, or whether recovery should be added to this Consul registry 
implementation?
   
   ### Related but different reports
   
   I searched open and closed issues and PRs in both `apache/dubbo` and 
`apache/dubbo-spi-extensions` and found no report of this specific watcher-task 
termination failure. [dubbo#8902](https://github.com/apache/dubbo/issues/8902) 
concerns provider TTL re-registration, and 
[dubbo-spi-extensions#183](https://github.com/apache/dubbo-spi-extensions/issues/183)
 concerns duplicate notifications.
   
   ### Do you have a (mini) reproduction demo?
   
   - [ ] Yes, I have a minimal reproduction demo to help resolve this issue 
more effectively!
   
   ### Are you willing to submit a pull request to fix on your own?
   
   - [ ] Yes I am willing to submit a pull request on my own!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

  • [I] [Bug] Consul registry wa... goto456 (via GitHub) via notifications

Reply via email to