Re: JMX metric for dropped hints?
Hi Thomas, I misread the question. The metrics indicated here are dropped due to dropped messages. The value you see in the log is dynamically generated so unfortunately the metrics you are looking for is not available through JMX. However, the Cassandra logs the value in system.peer_events table so you should be able to retrieve this information. FYI the inserted values have a hard coded 30 day TTL and get overwritten by new values. Hayato On Tue, 22 Jan 2019 at 12:04, Steinmaurer, Thomas < thomas.steinmau...@dynatrace.com> wrote: > Hi, > > > > thanks. I have seen that, but I’m not sure if this maps 1:1 to dropped > hints in the Cassandra log. > > > > Dropped “message” could also mean that something went wrong during a hint > replay for a recovering node, but not necessarily dropped hints due to a > node being offline longer than the default 3hr hinted handoff window. > > > > Am I wrong? > > > > Thanks again, > > Thomas > > > > *From:* Hayato Shimizu > *Sent:* Dienstag, 22. Jänner 2019 10:16 > *To:* user@cassandra.apache.org > *Subject:* Re: JMX metric for dropped hints? > > > > Hi Thomas, > > > > The dropped hints count and various rates are available in the following > location: > > > > > > org.apache.cassandra.metrics:type=DroppedMessage,scope=HINT,name=Dropped,* > > > > > > org.apache.cassandra.metrics/DroppedMessage/HINT/Attributes/Count > > org.apache.cassandra.metrics/DroppedMessage/HINT/Attributes/MeanRate > > org.apache.cassandra.metrics/DroppedMessage/HINT/Attributes/OneMinuteRate > > org.apache.cassandra.metrics/DroppedMessage/HINT/Attributes/FiveMinuteRate > > > org.apache.cassandra.metrics/DroppedMessage/HINT/Attributes/FifteenMinuteRate > > > > > > Hayato > > > > > > On Tue, 22 Jan 2019 at 07:45, Steinmaurer, Thomas < > thomas.steinmau...@dynatrace.com> wrote: > > Hello, > > > > is there a JMX metric for monitoring dropped hints as a counter/rate, > equivalent to what we see in Cassandra log, e.g.: > > > > WARN [HintedHandoffManager:1] 2018-11-13 13:28:46,991 > HintedHandoffMetrics.java:79 - /XXX has 18180 dropped hints, because node > is down past configured hint window. > > WARN [HintedHandoffManager:1] 2018-11-13 13:27:29,305 > HintedHandoffMetrics.java:79 - /XXX has 1191 dropped hints, because node is > down past configured hint window. > > WARN [HintedHandoffManager:1] 2018-11-13 13:23:09,393 > HintedHandoffMetrics.java:79 - /XXX has 135531 dropped hints, because node > is down past configured hint window. > > > > > > Thanks, > > Thomas > > > > The contents of this e-mail are intended for the named addressee only. It > contains information that may be confidential. Unless you are the named > addressee or an authorized designee, you may not copy or use it, or > disclose it to anyone else. If you received it in error please notify us > immediately and then destroy it. Dynatrace Austria GmbH (registration > number FN 91482h) is a company registered in Linz whose registered office > is at 4040 Linz, Austria, Freistädterstraße 313 > > > The contents of this e-mail are intended for the named addressee only. It > contains information that may be confidential. Unless you are the named > addressee or an authorized designee, you may not copy or use it, or > disclose it to anyone else. If you received it in error please notify us > immediately and then destroy it. Dynatrace Austria GmbH (registration > number FN 91482h) is a company registered in Linz whose registered office > is at 4040 Linz, Austria, Freistädterstraße 313 >
RE: JMX metric for dropped hints?
Hi, thanks. I have seen that, but I’m not sure if this maps 1:1 to dropped hints in the Cassandra log. Dropped “message” could also mean that something went wrong during a hint replay for a recovering node, but not necessarily dropped hints due to a node being offline longer than the default 3hr hinted handoff window. Am I wrong? Thanks again, Thomas From: Hayato Shimizu Sent: Dienstag, 22. Jänner 2019 10:16 To: user@cassandra.apache.org Subject: Re: JMX metric for dropped hints? Hi Thomas, The dropped hints count and various rates are available in the following location: org.apache.cassandra.metrics:type=DroppedMessage,scope=HINT,name=Dropped,* org.apache.cassandra.metrics/DroppedMessage/HINT/Attributes/Count org.apache.cassandra.metrics/DroppedMessage/HINT/Attributes/MeanRate org.apache.cassandra.metrics/DroppedMessage/HINT/Attributes/OneMinuteRate org.apache.cassandra.metrics/DroppedMessage/HINT/Attributes/FiveMinuteRate org.apache.cassandra.metrics/DroppedMessage/HINT/Attributes/FifteenMinuteRate Hayato On Tue, 22 Jan 2019 at 07:45, Steinmaurer, Thomas mailto:thomas.steinmau...@dynatrace.com>> wrote: Hello, is there a JMX metric for monitoring dropped hints as a counter/rate, equivalent to what we see in Cassandra log, e.g.: WARN [HintedHandoffManager:1] 2018-11-13 13:28:46,991 HintedHandoffMetrics.java:79 - /XXX has 18180 dropped hints, because node is down past configured hint window. WARN [HintedHandoffManager:1] 2018-11-13 13:27:29,305 HintedHandoffMetrics.java:79 - /XXX has 1191 dropped hints, because node is down past configured hint window. WARN [HintedHandoffManager:1] 2018-11-13 13:23:09,393 HintedHandoffMetrics.java:79 - /XXX has 135531 dropped hints, because node is down past configured hint window. Thanks, Thomas The contents of this e-mail are intended for the named addressee only. It contains information that may be confidential. Unless you are the named addressee or an authorized designee, you may not copy or use it, or disclose it to anyone else. If you received it in error please notify us immediately and then destroy it. Dynatrace Austria GmbH (registration number FN 91482h) is a company registered in Linz whose registered office is at 4040 Linz, Austria, Freistädterstraße 313 The contents of this e-mail are intended for the named addressee only. It contains information that may be confidential. Unless you are the named addressee or an authorized designee, you may not copy or use it, or disclose it to anyone else. If you received it in error please notify us immediately and then destroy it. Dynatrace Austria GmbH (registration number FN 91482h) is a company registered in Linz whose registered office is at 4040 Linz, Austria, Freistädterstraße 313
Re: JMX metric for dropped hints?
Hi Thomas, The dropped hints count and various rates are available in the following location: org.apache.cassandra.metrics:type=DroppedMessage,scope=HINT,name=Dropped,* org.apache.cassandra.metrics/DroppedMessage/HINT/Attributes/Count org.apache.cassandra.metrics/DroppedMessage/HINT/Attributes/MeanRate org.apache.cassandra.metrics/DroppedMessage/HINT/Attributes/OneMinuteRate org.apache.cassandra.metrics/DroppedMessage/HINT/Attributes/FiveMinuteRate org.apache.cassandra.metrics/DroppedMessage/HINT/Attributes/FifteenMinuteRate Hayato On Tue, 22 Jan 2019 at 07:45, Steinmaurer, Thomas < thomas.steinmau...@dynatrace.com> wrote: > Hello, > > > > is there a JMX metric for monitoring dropped hints as a counter/rate, > equivalent to what we see in Cassandra log, e.g.: > > > > WARN [HintedHandoffManager:1] 2018-11-13 13:28:46,991 > HintedHandoffMetrics.java:79 - /XXX has 18180 dropped hints, because node > is down past configured hint window. > > WARN [HintedHandoffManager:1] 2018-11-13 13:27:29,305 > HintedHandoffMetrics.java:79 - /XXX has 1191 dropped hints, because node is > down past configured hint window. > > WARN [HintedHandoffManager:1] 2018-11-13 13:23:09,393 > HintedHandoffMetrics.java:79 - /XXX has 135531 dropped hints, because node > is down past configured hint window. > > > > > > Thanks, > > Thomas > > > The contents of this e-mail are intended for the named addressee only. It > contains information that may be confidential. Unless you are the named > addressee or an authorized designee, you may not copy or use it, or > disclose it to anyone else. If you received it in error please notify us > immediately and then destroy it. Dynatrace Austria GmbH (registration > number FN 91482h) is a company registered in Linz whose registered office > is at 4040 Linz, Austria, Freistädterstraße 313 >
JMX metric for dropped hints?
Hello, is there a JMX metric for monitoring dropped hints as a counter/rate, equivalent to what we see in Cassandra log, e.g.: WARN [HintedHandoffManager:1] 2018-11-13 13:28:46,991 HintedHandoffMetrics.java:79 - /XXX has 18180 dropped hints, because node is down past configured hint window. WARN [HintedHandoffManager:1] 2018-11-13 13:27:29,305 HintedHandoffMetrics.java:79 - /XXX has 1191 dropped hints, because node is down past configured hint window. WARN [HintedHandoffManager:1] 2018-11-13 13:23:09,393 HintedHandoffMetrics.java:79 - /XXX has 135531 dropped hints, because node is down past configured hint window. Thanks, Thomas The contents of this e-mail are intended for the named addressee only. It contains information that may be confidential. Unless you are the named addressee or an authorized designee, you may not copy or use it, or disclose it to anyone else. If you received it in error please notify us immediately and then destroy it. Dynatrace Austria GmbH (registration number FN 91482h) is a company registered in Linz whose registered office is at 4040 Linz, Austria, Freist?dterstra?e 313
