[ 
https://issues.apache.org/jira/browse/CAMEL-26998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Federico Mariani updated CAMEL-26998:
-------------------------------------
    Fix Version/s: 4.24.0

> camel-core - Polling consumers: the consumer health check stays UP while a 
> poll hangs
> -------------------------------------------------------------------------------------
>
>                 Key: CAMEL-26998
>                 URL: https://issues.apache.org/jira/browse/CAMEL-26998
>             Project: Camel
>          Issue Type: Improvement
>          Components: camel-core
>            Reporter: Federico Mariani
>            Assignee: Federico Mariani
>            Priority: Major
>             Fix For: 4.24.0
>
>
> The health check of a polling consumer ({{ScheduledPollConsumerHealthCheck}}, 
> which every consumer that extends {{ScheduledPollConsumer}} gets: file, 
> ftp/sftp, sql, jpa, mail, aws2-s3, ...) reports from what a poll leaves 
> behind once it is done: the error counter and the last error, which 
> {{ScheduledPollConsumer.doRun}} updates after {{poll()}} returned or threw, 
> and whether the first poll is done ({{isConsumerReady()}}). A poll that does 
> not return is not seen: readiness stays UP for as long as it blocks.
> A poll blocks for long when a client has no read timeout, or a long one, on a 
> connection that went silent (a half-open TCP connection, a NAT or firewall 
> that drops the flow). Found with fault-injection tests (Toxiproxy blackhole 
> and silent drop in front of a real PostgreSQL and SFTP server): the consumer 
> thread waits in a socket read, with no error, no WARN, and readiness UP for 
> the whole fault:
> * camel-sql with pgjdbc and its default {{socketTimeout=0}}: the poll waits 
> until TCP gives up (~15 min), or for ever;
> * sftp with the default {{soTimeout}} (300 s), which is the JSch session 
> timeout: JSch sends a keepalive at the first timeout and drops the connection 
> at the second, so ~600 s.
> A client timeout ({{socketTimeout}}, {{soTimeout}}) is the fix for the hang 
> itself, but the health check cannot tell a hung poll from a quiet one either 
> way. It is also UP when the very first poll hangs with 
> {{camel.health.initialState=UP}}, and when a consumer marks itself ready 
> before its poll is done ({{forceConsumerAsReady()}}: sql after the query, ftp 
> after the listing, CAMEL-20189).
> h3. Steps to reproduce
> A {{ScheduledPollConsumer}} whose {{poll()}} waits on a latch: a first poll 
> that succeeds, then a poll that blocks. Call the consumer health check as 
> readiness while the poll blocks.
> h3. Actual (4.23.0 and 4.24.0-SNAPSHOT)
> UP for as long as the poll blocks.
> h3. Expected
> A way to have the consumer health check DOWN, with a message that says so, 
> once a poll has been running longer than the user expects a poll to take; UP 
> again when the poll is done.
> h3. Proposal
> * {{ScheduledPollConsumer}} keeps when the poll in progress started (one 
> volatile time, set when begin/poll/commit starts, as {{isPolling()}} counts 
> them).
> * A new consumer option {{healthCheckMaxPollDuration}} on 
> {{ScheduledPollEndpoint}}, so every polling component has it. 0 (the default) 
> turns it off. When set, the consumer health check is DOWN while the poll in 
> progress has been running longer than that, with a message such as "Consumer 
> has been polling for 5m10s, longer than healthCheckMaxPollDuration (5m)". The 
> poll is not interrupted.
> Opt-in, because a poll includes routing what it polled (a batch of files, a 
> big file, a slow query): a default would turn readiness DOWN for polls that 
> are only slow (CAMEL-20189 is about readiness and a long first poll). This is 
> not a threshold on results that a monitoring system could apply itself 
> (CAMEL-17587 removed those): while a poll hangs, the health check has no 
> failure to report at all.
> _Claude Code on behalf of Croway_



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to