[
https://issues.apache.org/jira/browse/CAMEL-19491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Simon Rasmussen updated CAMEL-19491:
------------------------------------
Description:
update: First observed on SQS component, but other aws polling consumers
affected as well.
If a Sqs2ConsumerHealthCheck returns DOWN, then it will not recover back to UP,
despite the consumer polling messages and processing them successfully.
Detected on SQS, but likely all aws2 components are affected by this.
We have experienced this a few times in production now (on various camel 2.20.x
versions), including 3.20.5.
Actual output from our readiness check:
{noformat}
{"status":"DOWN","components":{"camelHealth":{"status":"DOWN","details":{"name":"camel-health-check","consumer:queue_name":"DOWN"}},"db":{"status":"UP","details":{"database":"MariaDB","validationQuery":"isValid()"}},"diskSpace":{"status":"UP","details":{"total":64411906048,"free":13194371072,"threshold":10485760,"exists":true}},"ping":{"status":"UP"},"readinessState":{"status":"UP"}}}{noformat}
Notice how the health check prefix is absent: aws2-sqs-consumer-
I noticed that the tests of this functionality are manually plumbing the setup.
I also see that Sqs2ConsumerHealthCheck extends AbstractHealthCheck, but
shouldn't this be ConsumerHealthCheck instead?
My availability does not allow for attempting to fix this myself, thus I've
just created this ticket for now, maybe someone else is up for grabbing it.
was:
If a Sqs2ConsumerHealthCheck returns DOWN, then it will not recover back to UP,
despite the consumer polling messages and processing them successfully.
Detected on SQS, but likely all aws2 components are affected by this.
We have experienced this a few times in production now (on various camel 2.20.x
versions), including 3.20.5.
Actual output from our readiness check:
{noformat}
{"status":"DOWN","components":{"camelHealth":{"status":"DOWN","details":{"name":"camel-health-check","consumer:queue_name":"DOWN"}},"db":{"status":"UP","details":{"database":"MariaDB","validationQuery":"isValid()"}},"diskSpace":{"status":"UP","details":{"total":64411906048,"free":13194371072,"threshold":10485760,"exists":true}},"ping":{"status":"UP"},"readinessState":{"status":"UP"}}}{noformat}
Notice how the health check prefix is absent: aws2-sqs-consumer-
I noticed that the tests of this functionality are manually plumbing the setup.
I also see that Sqs2ConsumerHealthCheck extends AbstractHealthCheck, but
shouldn't this be ConsumerHealthCheck instead?
My availability does not allow for attempting to fix this myself, thus I've
just created this ticket for now, maybe someone else is up for grabbing it.
> Failing healthcheck on aws2-* polling consumers causes readiness check to be
> stuck
> ----------------------------------------------------------------------------------
>
> Key: CAMEL-19491
> URL: https://issues.apache.org/jira/browse/CAMEL-19491
> Project: Camel
> Issue Type: Bug
> Components: camel-aws2
> Affects Versions: 3.20.5
> Reporter: Simon Rasmussen
> Priority: Major
> Labels: easy, help-wanted
>
> update: First observed on SQS component, but other aws polling consumers
> affected as well.
> If a Sqs2ConsumerHealthCheck returns DOWN, then it will not recover back to
> UP, despite the consumer polling messages and processing them successfully.
> Detected on SQS, but likely all aws2 components are affected by this.
> We have experienced this a few times in production now (on various camel
> 2.20.x versions), including 3.20.5.
> Actual output from our readiness check:
> {noformat}
> {"status":"DOWN","components":{"camelHealth":{"status":"DOWN","details":{"name":"camel-health-check","consumer:queue_name":"DOWN"}},"db":{"status":"UP","details":{"database":"MariaDB","validationQuery":"isValid()"}},"diskSpace":{"status":"UP","details":{"total":64411906048,"free":13194371072,"threshold":10485760,"exists":true}},"ping":{"status":"UP"},"readinessState":{"status":"UP"}}}{noformat}
> Notice how the health check prefix is absent: aws2-sqs-consumer-
> I noticed that the tests of this functionality are manually plumbing the
> setup.
> I also see that Sqs2ConsumerHealthCheck extends AbstractHealthCheck, but
> shouldn't this be ConsumerHealthCheck instead?
> My availability does not allow for attempting to fix this myself, thus I've
> just created this ticket for now, maybe someone else is up for grabbing it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
