Federico Mariani created CAMEL-27016:
----------------------------------------
Summary: camel-mina-sftp - CamelContext.stop() waits until a hung
upload ends: the producer stop waits for the MinaSftpOperations lock that the
upload holds, also after the shutdown timeout
Key: CAMEL-27016
URL: https://issues.apache.org/jira/browse/CAMEL-27016
Project: Camel
Issue Type: Bug
Components: camel-mina-sftp
Affects Versions: 4.23.0
Reporter: Federico Mariani
Fix For: 4.24.0
The same as CAMEL-27001 (camel-sftp), for camel-mina-sftp, which CAMEL-27001
did not change.
{{MinaSftpOperations}} runs every operation under one {{ReentrantLock}}
({{lock}}, line 118 of
{{components/camel-mina-sftp/src/main/java/org/apache/camel/component/file/remote/mina/MinaSftpOperations.java}}
on main). {{storeFile()}} (line 1409) holds it for the whole upload, including
the close of the stream of {{SftpClient.write()}} (line 1481), which waits for
the acknowledgement of each write by the server, up to the idle timeout of the
Apache MINA SSHD session (10 minutes by default) for each one. When the server
stops answering during the upload (the SFTP subsystem hangs while the SSH
transport stays alive), the upload holds the lock that long.
When Camel stops, the shutdown strategy forces the shutdown after its timeout
and stops the routes. Stopping the producer, {{RemoteFileProducer.doStop()}}
({{components/camel-ftp-common/.../RemoteFileProducer.java}}, line 192) calls
{{RemoteFileOperations.disconnectOnStop()}} (added by CAMEL-27001), which
{{MinaSftpOperations}} does not override: the default implementation (line 66
of {{RemoteFileOperations.java}}) calls {{isConnected()}} (line 820), which
waits for the lock without a timeout. Nothing interrupts the thread that runs
the upload (the thread of the caller), so {{CamelContext.stop()}} does not
return before the upload ends.
h3. Steps to reproduce
As in {{SftpProducerStopDuringHungUploadIT}} of camel-ftp: an embedded Apache
SSHD server whose SFTP subsystem blocks on the first write request
({{SftpEventListener.writing()}}). Route
{{from("direct:upload").to("mina-sftp://...")}}, shutdown timeout 1 s. Send a
64 KB body from another thread; once the upload reached the server, call
{{CamelContext.stop()}}.
h3. Actual (4.24.0-SNAPSHOT)
{{stop()}} is still blocked 20 s after the shutdown timeout, and returns only
when the server answers again. The same with a route whose consumer cannot be
suspended (its producer stops before the shutdown is forced).
h3. Expected
As for camel-sftp since CAMEL-27001: once the shutdown is forced, the stop of
the producer does not wait for the lock without bound, but disconnects, and the
hung upload fails. Outside a forced shutdown (a producer cache that evicts the
producer, {{ProducerTemplate.stop()}}, a graceful shutdown), the stop still
waits for the upload, which can be healthy but slow.
h3. Suggested fix
Override {{disconnectOnStop()}} in {{MinaSftpOperations}} as {{SftpOperations}}
does: wait for the lock while the shutdown is not forced, checking
{{ShutdownStrategy.isForceShutdown()}} every second; once it is forced, close
the session at once ({{ClientSession.close(true)}}) without the lock, which
closes the SFTP channel and cancels the requests that wait for an answer, so
the upload fails and releases the lock.
_Claude Code on behalf of Croway_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)