Croway opened a new pull request, #30093:
URL: https://github.com/apache/camel/pull/30093

   [CAMEL-27001](https://issues.apache.org/jira/browse/CAMEL-27001)
   
   ### What was wrong
   
   `SftpOperations` runs every operation under one lock (CAMEL-8887; a 
`ReentrantLock` since CAMEL-20199), and `storeFile()` holds it for the whole 
upload. When the server stops answering, `ChannelSftp.put()` blocks in the 
thread of the caller, which nothing interrupts. When Camel stops, the shutdown 
strategy forces the shutdown after its timeout and stops the producer: 
`RemoteFileProducer.doStop()` → `disconnect()` → `SftpOperations.isConnected()` 
waits for that lock without bound. So `CamelContext.stop()` does not return 
before the upload ends: about twice `soTimeout` (10 minutes with the defaults) 
over a dead network path, never when the SSH transport is alive but the SFTP 
subsystem of the server hangs. Found with fault-injection tests.
   
   ### Fix
   
   - `RemoteFileOperations.disconnectOnStop()`: a new default method, 
`disconnect()` when connected, which `RemoteFileProducer.doStop()` now calls 
instead of `disconnect()`. FTP/FTPS, mina-sftp, scp and azure-files keep the 
default, so their behaviour does not change.
   - `SftpOperations.disconnectOnStop()` waits up to 1 s for the lock. If 
another thread still holds it, it disconnects the JSch session without the 
lock: `Session.disconnect()` can be called from another thread (JSch's own 
session thread does), and closes the channel and its pipe, so the blocked 
`put()` fails with "Cannot store file" and releases the lock. The exchange 
fails, it does not report success.
   - When the upload is blocked writing to the socket (a half-open connection, 
TCP send buffer full), it holds the write lock of the JSch session, and 
`Session.disconnect()` waits for it as well, as it first sends 
`SSH_MSG_CHANNEL_CLOSE` (JSch 2.28.8: `Channel.close()` → `Session.write()` → 
`_write()`, `synchronized (lock)`). So the disconnect runs in a Camel daemon 
thread, and the stop waits for it at most 1 s; it completes when the write 
fails or the connection recovers.
   - A connect in progress (with its reconnect attempts) is waited for, not 
cut: `connectTimeout` and `maximumReconnectAttempts` bound it, and cutting it 
would let its next attempt open a new session for a stopped producer.
   
   ### Tests
   
   `SftpProducerStopDuringHungUploadIT` (embedded SSHD from test-infra, no 
container), shutdown timeout 1 s, the upload sent from a thread Camel does not 
manage:
   
   - `stopDoesNotWaitForUploadWhenServerStopsAnswering`: the SFTP subsystem 
blocks on the first write (`SftpEventListener.writing()`) while the transport 
stays alive.
   - `stopDoesNotWaitForUploadBlockedWritingToHalfOpenConnection`: a test-local 
TCP proxy stops reading after the first MB, the upload blocks in the socket 
write.
   
   Results:
   
   | | server stops answering | half-open, write blocked |
   |---|---|---|
   | `main` | fails: `stop()` still blocked after 20 s | fails: still blocked 
after 20 s |
   | lock change only, `Session.disconnect()` in the stopping thread | passes 
(2.1 s) | fails: the stop waits in `Channel.close()` → `Session._write()` |
   | this PR | passes (2.1 s) | passes (3.7 s), the disconnect finishes in the 
background |
   
   Both tests also passed 5 and 3 times in a row while the SFTP ITs of the 
module ran in parallel. camel-ftp unit tests (190) and the ITs `*Sftp*IT`, 
`*Shutdown*IT`, `*Disconnect*IT`, `FtpProducer*IT` pass. In one parallel run 
`SftpChangedReadLockIT` failed with 40 lines instead of 20: it passes alone, 
and it consumes with `readLock=changed` without stopping a producer during an 
upload.
   
   ### For reviewers
   
   - No upgrade-guide entry: a producer is stopped while one of its uploads 
still runs when a forced shutdown gave up waiting for it, or when an 
application stops a `ProducerTemplate` that another thread still sends with 
(stopping the `CamelContext` does not stop the producers of a 
`ProducerTemplate`). Otherwise the in-flight exchanges are done before the 
producer stops, the lock is free, and the stop disconnects as before.
   - The two 1 s waits are constants, not endpoint options.
   - `RemoteFileProducer` (camel-ftp-common) changes only in `doStop()`. The 
consumers' stop path is unchanged.
   - camel-mina-sftp has the same lock pattern in `MinaSftpOperations`; it 
keeps the default here.
   
   _Claude Code on behalf of Croway_
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to