[ https://issues.apache.org/jira/browse/CASSSIDECAR-353 ]
Carl Sandland deleted comment on CASSSIDECAR-353:
-------------------------------------------
was (Author: JIRAUSER311163):
I'm having some issues with the 'cassinstancesdown' and 'cassinstancesup'
metrics I'm collecting using 'jmx_enabled' config option. I'm getting values
fine, they just don't seem to be correctly updating when I stop/start the
associated cassandra server (ie the one in my sidecar.yaml).
*My expectations where:*
Assuming a sidecar config with one attached cassandra instance, all started up
and running happily:
cassinstancesdown = 0, cassinstancesup = 1
manually stop cassandra:
cassinstancesdown = 1, cassinstancesup = 0
I was seeing a constant : cassinstancesdown=0, cassinstancesup=1
This seems to be because the update to cassinstancesdown in
sidecar/tasks/HealthCheckPeriodicTask.java::healthCheck() in the onFailure()
block is never called. The exceptions that would trigger it are swallowed
inside the call to instanceMetadata.delegate().healthCheck()
(CassandraAdapterDelegate).
I've confirmed this locally by modifying the delegate code to rethrow the
exceptions and it all seems to update fine. I'm not familiar enough with the
code to know if that is a sane approach. My brief thought was that swallowing
exceptions in the delegate doesn't fit with the idea of the higher level
promise chain.
I think this would be a helpful metric for me to build some basic observability
around. Actively monitoring the health of sidecar itself might seem an odd
focus, but we need some basic observability around stuff we offer as managed
options. The vertx metrics seem solid. Am using sidecar JMX port to collect.
> Review logback configuration for Sidecar
> ----------------------------------------
>
> Key: CASSSIDECAR-353
> URL: https://issues.apache.org/jira/browse/CASSSIDECAR-353
> Project: Sidecar for Apache Cassandra
> Issue Type: Bug
> Components: Observability
> Reporter: Francisco Guerrero
> Priority: Major
> Fix For: 0.2
>
>
> The logback configuration used by Sidecar is currently not properly
> configured for production. There are configurations set to debug level which
> don't make it suitable to start using in production. We need to revise the
> configuration and update it to make it suitable for production grade setups.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
