tyskjohan opened a new pull request, #28195: URL: https://github.com/apache/camel/pull/28195
# Description [CAMEL-25896](https://issues.apache.org/jira/browse/CAMEL-25896) Before every poll, `RemoteFileConsumer.connectIfNecessary()` sends a noop to check the connection. The `sendNoop` option cannot turn that off, because only the producer reads it. For the JSch-based sftp component the noop is `Session.sendKeepAliveMsg()`, a `[email protected]` global request that does not wait for a reply, so it cannot tell a live server from a dead one. Servers that do not support the request log each one. With the default `delay=500`, that is about two log lines a second for every polling route. ### Change - `sendNoop` now also applies to the consumer. With `sendNoop=false`, `connectIfNecessary()` calls `getOperations().isConnected()` instead of `sendNoop()`, and reconnects when it returns false. The default stays `true`, so nothing changes unless the option is set. - **Recovery without the noop.** An ftp/ftps client does not notice a connection that the server has closed, because `Socket.isConnected()` stays true. A failure while retrieving a file goes through `handleException()` and not out of `poll()`, so the poll strategy never disconnects. As a result, a consumer with `useList=false&stepwise=false` would never recover. So with `sendNoop=false`, a failure caused by an I/O error makes the consumer send one noop before the next poll, and connect again if that noop fails. `FtpConsumer.ignoreCannotRetrieveFile()` does the same, because a closed connection can still carry the reply code 550 of an earlier command and be ignored as a missing file. - **Why not disconnect at once.** mina-sftp reports server status replies (`SSH_FX_NO_SUCH_FILE`, `SSH_FX_PERMISSION_DENIED`) as `SftpException`, which extends `IOException`, and a local file that cannot be written also fails with an `IOException`. Disconnecting on those would reconnect on every missing file and leave the rest of the batch on a closed client. A noop on the next poll only reconnects when the connection really is gone. - **camel-azure-files.** `FilesConfiguration` turns `sendNoop` off for its producer and does not expose the option. A new protected `RemoteFileConsumer.isSendNoop()` hook lets `FilesConsumer` keep the noop before each poll, as before. - **Option metadata.** The `@UriParam` label changes from `producer,advanced` to `common,advanced` (as `fastExistsCheck` uses), and the description covers both checks and the per-protocol trade-off. The component JSON, catalog and endpoint DSL are regenerated. - **Upgrade guide.** A new 4.24 entry covers consumer URIs that already set `sendNoop=false`, for example one shared by `from` and `to`. The option was ignored there before; those consumers now skip the noop. ### Trade-off with `sendNoop=false` - **ftp/ftps:** a connection that the server has closed is found only when the next operation fails. That poll or file fails and is logged, and the consumer reconnects on the poll after. - **sftp/mina-sftp:** the client notices a closed connection itself. A connection that dropped silently is found as it is today, when an operation times out. - **JSch sftp:** the keepalive never detected a dead server, so nothing is lost there. ### Tests New tests: - `RemoteFileConsumerSendNoopTest` (camel-ftp, 33 cases over ftp/ftps/sftp) covers: - which check runs; - reconnecting when the check fails; - the noop on the next poll after an I/O failure, and only then; - no noop after a non-I/O failure; - the ignored 550 case; - the default being unchanged. - `FtpConsumerSendNoopFalseIT` counts NOOP and USER commands on the embedded FTP server. It includes a recovery case with `useList=false&stepwise=false` after `disconnectAllSessions()`, with and without `ignoreFileNotFoundOrPermissionError`. - `SftpConsumerSendNoopFalseIT` counts `keepalive@` global requests and logins on the embedded SSHD. - `MinaSftpConsumerSendNoopTest`, including a status reply that triggers a noop and no reconnect. - `FilesConsumerSendNoopTest`. Existing tests that also pass: - camel-ftp: `RemoteFileProducerPreWriteCheckTest`, `FtpProducerSendNoopFalseIT`, `FromFtpNoopIT`, `FromFtpUseListFalseIT`, `FtpConsumerTemplateUseListFalseIT`, `FtpConsumerDisconnectIT`, `FtpReconnectAttemptServerStoppedIT`, `FtpConsumerPostProcessingOnDisconnectIT`, `FtpConsumerDeleteNoWritePermissionIT`, and the Sftp simple/template/use-list ITs. - camel-mina-sftp: `MinaSftpErrorHandlingIT`, `MinaSftpServerAliveIT`, `MinaSftpSimpleConsumeIT`, `MinaSftpSimpleProduceIT`. - All camel-azure-files tests. Negative check: I disabled the noop on the next poll, and 8 unit cases and both recovery ITs failed. Build: `formatter:format impsort:sort` and `-Psourcecheck install -DskipTests` on camel-ftp-common, camel-ftp, camel-mina-sftp and camel-azure-files, then regenerated the catalog and endpoint DSL. I did not run `mvn clean install -DskipTests` from the root. # Target - [x] I checked that the commit is targeting the correct branch (Camel 4 uses the `main` branch) # Tracking - [x] If this is a large change, bug fix, or code improvement, I checked there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for the change (usually before you start working on it). # Apache Camel coding standards and style - [x] I checked that each commit in the pull request has a meaningful subject line and body. - [ ] I have run `mvn clean install -DskipTests` locally from root folder and I have committed all auto-generated changes. (Built and regenerated per module as described above, not from the root.) # AI-assisted contributions - [x] If this PR includes AI-generated code, commits have proper co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR description identifies the AI tool used. _Claude Code on behalf of tyskjohan_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01SonJJYXyt16gNRQkSu72AH -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
