Federico Mariani created CAMEL-27018:
----------------------------------------

             Summary: camel-sftp - CamelContext.stop() can wait until the 
server answers again when a download hangs: after the interrupt, the poll 
changes back to its directory with the SftpOperations lock, and the consumer 
stop waits for that lock
                 Key: CAMEL-27018
                 URL: https://issues.apache.org/jira/browse/CAMEL-27018
             Project: Camel
          Issue Type: Bug
          Components: camel-ftp
    Affects Versions: 4.18.5, 4.23.0
            Reporter: Federico Mariani
             Fix For: 4.24.0


A follow-up of CAMEL-27001, which fixed the stop of the producer and noted that 
the consumer was not affected the same way: the forced shutdown interrupts the 
poll thread, JSch's read of the channel pipe is interruptible, and the download 
fails. It does, but the poll is not over.

The poll thread holds the lock of {{SftpOperations}} 
({{components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/SftpOperations.java}}
 on main) for the whole {{retrieveFile()}} (line 1158). With {{stepwise=true}} 
(the default) the retrieval first changes to the directory of the file, and in 
its {{finally}} changes back to the directory it started from 
({{changeBackToDirectory()}}, line 1246), which sends requests to the server 
({{ChannelSftp.cd()}}: {{realpath}}, {{stat}}). When the server stopped 
answering, that change of directory waits as well, with the lock, once the 
interrupt has been spent on the download.

When Camel stops, the forced shutdown stops the consumer: 
{{ScheduledPollConsumer.doStop()}} stops the scheduler, which interrupts the 
poll ({{future.cancel(true)}}, line 229 of 
{{DefaultScheduledPollConsumerScheduler}}); then 
{{RemoteFileConsumer.doStop()}} 
({{components/camel-ftp-common/.../RemoteFileConsumer.java}}, line 215) calls 
{{disconnect()}}, whose {{SftpOperations.isConnected()}} (line 623) waits for 
the lock without a timeout. So {{CamelContext.stop()}} waits until the server 
answers again.

Whether it happens depends on the interrupts: a consumer on its own scheduled 
thread pool is interrupted a second time, when the scheduler shuts that pool 
down ({{shutdownNow}}, line 235), which usually ends the change of directory as 
well, but not when both interrupts reach the thread before it wakes up (they 
are then one). A consumer on a shared scheduled thread pool 
({{scheduledExecutorService}} option) is interrupted once, and the stop always 
waits. Found with fault-injection tests, where the stop hung in about one run 
in three.

h3. Steps to reproduce

An embedded Apache SSHD server whose SFTP subsystem blocks on the first read 
request of file data ({{SftpEventListener.reading()}}), as in 
{{SftpProducerStopDuringHungUploadIT}}. Route 
{{from("sftp://...?scheduledExecutorService=#sharedPool";)}} with a file to 
download, shutdown timeout 1 s. Once the download reached the server, call 
{{CamelContext.stop()}}.

h3. Actual (4.24.0-SNAPSHOT)

{{stop()}} is still blocked 20 s after the shutdown timeout; it returns when 
the server answers again, and the poll then logs the {{InterruptedIOException}} 
of the download.

h3. Expected

Once the shutdown is forced, the stop of the consumer does not wait for the 
lock without bound, but disconnects, as the stop of the producer does since 
CAMEL-27001: {{RemoteFileConsumer.doStop()}} can call 
{{RemoteFileOperations.disconnectOnStop()}}, which {{SftpOperations}} 
implements. Outside a forced shutdown it waits as before.

_Claude Code on behalf of Croway_



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to