Federico Mariani created CAMEL-27018:
----------------------------------------
Summary: camel-sftp - CamelContext.stop() can wait until the
server answers again when a download hangs: after the interrupt, the poll
changes back to its directory with the SftpOperations lock, and the consumer
stop waits for that lock
Key: CAMEL-27018
URL: https://issues.apache.org/jira/browse/CAMEL-27018
Project: Camel
Issue Type: Bug
Components: camel-ftp
Affects Versions: 4.18.5, 4.23.0
Reporter: Federico Mariani
Fix For: 4.24.0
A follow-up of CAMEL-27001, which fixed the stop of the producer and noted that
the consumer was not affected the same way: the forced shutdown interrupts the
poll thread, JSch's read of the channel pipe is interruptible, and the download
fails. It does, but the poll is not over.
The poll thread holds the lock of {{SftpOperations}}
({{components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/SftpOperations.java}}
on main) for the whole {{retrieveFile()}} (line 1158). With {{stepwise=true}}
(the default) the retrieval first changes to the directory of the file, and in
its {{finally}} changes back to the directory it started from
({{changeBackToDirectory()}}, line 1246), which sends requests to the server
({{ChannelSftp.cd()}}: {{realpath}}, {{stat}}). When the server stopped
answering, that change of directory waits as well, with the lock, once the
interrupt has been spent on the download.
When Camel stops, the forced shutdown stops the consumer:
{{ScheduledPollConsumer.doStop()}} stops the scheduler, which interrupts the
poll ({{future.cancel(true)}}, line 229 of
{{DefaultScheduledPollConsumerScheduler}}); then
{{RemoteFileConsumer.doStop()}}
({{components/camel-ftp-common/.../RemoteFileConsumer.java}}, line 215) calls
{{disconnect()}}, whose {{SftpOperations.isConnected()}} (line 623) waits for
the lock without a timeout. So {{CamelContext.stop()}} waits until the server
answers again.
Whether it happens depends on the interrupts: a consumer on its own scheduled
thread pool is interrupted a second time, when the scheduler shuts that pool
down ({{shutdownNow}}, line 235), which usually ends the change of directory as
well, but not when both interrupts reach the thread before it wakes up (they
are then one). A consumer on a shared scheduled thread pool
({{scheduledExecutorService}} option) is interrupted once, and the stop always
waits. Found with fault-injection tests, where the stop hung in about one run
in three.
h3. Steps to reproduce
An embedded Apache SSHD server whose SFTP subsystem blocks on the first read
request of file data ({{SftpEventListener.reading()}}), as in
{{SftpProducerStopDuringHungUploadIT}}. Route
{{from("sftp://...?scheduledExecutorService=#sharedPool")}} with a file to
download, shutdown timeout 1 s. Once the download reached the server, call
{{CamelContext.stop()}}.
h3. Actual (4.24.0-SNAPSHOT)
{{stop()}} is still blocked 20 s after the shutdown timeout; it returns when
the server answers again, and the poll then logs the {{InterruptedIOException}}
of the download.
h3. Expected
Once the shutdown is forced, the stop of the consumer does not wait for the
lock without bound, but disconnects, as the stop of the producer does since
CAMEL-27001: {{RemoteFileConsumer.doStop()}} can call
{{RemoteFileOperations.disconnectOnStop()}}, which {{SftpOperations}}
implements. Outside a forced shutdown it waits as before.
_Claude Code on behalf of Croway_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)