[ 
https://issues.apache.org/jira/browse/CXF-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andriy Redko updated CXF-9272:
------------------------------
    Fix Version/s: 3.6.14
                   4.1.10

> HttpClientHTTPConduit: async call to a refused connection sometimes never 
> calls the ClientCallback (regression from CXF-9250)
> -----------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CXF-9272
>                 URL: https://issues.apache.org/jira/browse/CXF-9272
>             Project: CXF
>          Issue Type: Bug
>          Components: Transports
>    Affects Versions: 4.2.4
>            Reporter: Federico Mariani
>            Priority: Major
>             Fix For: 3.6.14, 4.2.5, 4.1.10
>
>
> Since 4.2.4, an asynchronous invocation through {{HttpClientHTTPConduit}} to 
> an address that refuses the connection sometimes never completes. Neither 
> {{ClientCallback.handleResponse}} nor {{handleException}} is called, so a 
> caller waiting on the callback (or on the future) waits forever. The receive 
> timeout does not help, because no response handling is ever scheduled.
> We found this through Apache Camel's camel-cxf async producer, where an 
> exchange hangs forever when the target SOAP service is down (CAMEL-26696, 
> CAMEL-26600). The reproducer below uses plain CXF only.
> *Reproducer* (JUnit 5, plain JAX-WS client, a fresh client per call, nothing 
> listening on the port):
> {code:java}
> @WebService(targetNamespace = "http://example.org/echo";)
> public interface Echo {
>     String echo(String text);
> }
> @Test
> void asyncCallToClosedPortAlwaysCallsBack() throws Exception {
>     int port;
>     try (ServerSocket s = new ServerSocket(0)) {
>         port = s.getLocalPort(); // nothing listens on it after close
>     }
>     String address = "http://localhost:"; + port + "/echo";
>     int calls = 2400;
>     AtomicInteger stuck = new AtomicInteger();
>     ExecutorService pool = Executors.newFixedThreadPool(16);
>     CountDownLatch all = new CountDownLatch(calls);
>     for (int i = 0; i < calls; i++) {
>         pool.execute(() -> {
>             try {
>                 JaxWsProxyFactoryBean factory = new JaxWsProxyFactoryBean();
>                 factory.setServiceClass(Echo.class);
>                 factory.setAddress(address);
>                 Client client = ClientProxy.getClient(factory.create());
>                 HTTPClientPolicy policy = new HTTPClientPolicy();
>                 policy.setConnectionTimeout(3000);
>                 policy.setReceiveTimeout(3000);
>                 ((HTTPConduit) client.getConduit()).setClient(policy);
>                 CountDownLatch callback = new CountDownLatch(1);
>                 client.invoke(new ClientCallback() {
>                     @Override
>                     public void handleResponse(Map<String, Object> ctx, 
> Object[] res) {
>                         super.handleResponse(ctx, res);
>                         callback.countDown();
>                     }
>                     @Override
>                     public void handleException(Map<String, Object> ctx, 
> Throwable ex) {
>                         super.handleException(ctx, ex);
>                         callback.countDown();
>                     }
>                 }, "echo", "hello");
>                 if (!callback.await(8, TimeUnit.SECONDS)) {
>                     stuck.incrementAndGet();
>                 }
>                 client.destroy();
>             } catch (Exception e) {
>                 // a synchronous failure from invoke() counts as completed
>             } finally {
>                 all.countDown();
>             }
>         });
>     }
>     all.await();
>     pool.shutdown();
>     System.out.println(Version.getCurrentVersion() + ": stuck (no callback 
> within 8s) = " + stuck.get());
> }
> {code}
> *Results* (2 runs of 2400 calls per version, 16 threads, JDK 25.0.2 Temurin, 
> macOS aarch64):
> ||CXF||calls||no callback within 8 s||
> |4.2.3|4800|0|
> |4.2.4|4800|22 (18 + 4)|
> |4.2.5-SNAPSHOT (20261110.085644)|4800|42 (22 + 20)|
> Sequential calls rarely hit it locally (0 in 3300 calls through the Camel 
> producer). Concurrency widens the race window, as does a loaded CI machine.
> *What the stuck exchanges look like*
> We instrumented the Camel-based test, which uses the same async 
> {{ClientImpl.invoke(ClientCallback, ...)}} path, with an out interceptor at 
> {{Phase.SETUP}} that keeps the out message. Every stuck exchange shows the 
> same state:
> * {{outMessage.getContent(Exception.class)}} is 
> {{java.util.concurrent.CompletionException: java.net.ConnectException}}
> * {{exchange.getInMessage()}} is null
> * {{exchange.get(ClientCallback.class)}} is still set, so {{ClientImpl}} 
> never dispatched it
> The calls that do complete report a {{Fault}} caused by {{ConnectException}} 
> through {{handleException}}.
> *Suspected cause* (our reading of the code; we did not instrument inside CXF)
> CXF-9250 added this to the {{future.exceptionally(...)}} callback in 
> {{HttpClientHTTPConduit.HttpClientWrappedOutputStream#setProtocolHeaders}}:
> {code:java}
> outMessage.setContent(Exception.class, ex instanceof Exception ? (Exception) 
> ex : new IOException(ex));
> ...
> close();
> {code}
> {{HTTPConduit.WrappedOutputStream#handleResponseOnWorkqueue}} only schedules 
> the runnable that calls {{handleResponseInternal()}}, and the fault observer 
> if that fails, when no exception is recorded:
> {code:java}
> boolean exceptionSet = outMessage.getContent(Exception.class) != null;
> if (!exceptionSet) {
>     ... execute(runnable) ...
> }
> {code}
> If the {{exceptionally}} callback records the {{ConnectException}} before the 
> caller thread's {{close()}} reaches {{handleResponse()}}, both {{close()}} 
> calls (the caller's and the one from {{exceptionally}}) skip scheduling. 
> Nothing then calls the fault observer, so the {{ClientCallback}} is never 
> invoked. The raw {{CompletionException}} left on the out message of the stuck 
> exchanges matches this path. The 4.2.3 to 4.2.4 difference also points at 
> CXF-9250: between the {{cxf-4.2.3}} and {{cxf-4.2.4}} tags it is the only 
> commit touching {{HttpClientHTTPConduit}} or {{HTTPConduit}}.
> CXF-9250 also went into 4.1.9, 4.0.12 and 3.6.13. We did not test those 
> versions.
> Possibly related, but a different cause: CXF-9257 (the callback throws) and 
> CXF-9267 (a race in {{HttpClientBodyPublisher}} hides the connect timeout). 
> Both are fixed on main, and this issue still reproduces on the 4.2.5-SNAPSHOT 
> above.
> _Claude Code on behalf of Croway_



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to