[
https://issues.apache.org/jira/browse/ARTEMIS-5438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17945715#comment-17945715
]
Ryan Yeats edited comment on ARTEMIS-5438 at 4/18/25 8:25 PM:
--------------------------------------------------------------
Mostly looking at broker performance but also in the context of how long its
taking messaging clients to actually send and receive message from the broker
too. Specifically the time at which Artemis receives a message to when its sent
to a specific consumer and then the time it takes that consumer to send the
acknowledgement. Normally that would be expensive to capture in logs and hard
to string together from multiple metrics to get an accurate picture. So i am
looking at sampling traces of messages using open telemetry and looking at the
ones that take a long time to see if there is some sort of resource constraint
that coincide with those times in our metrics.
Yes, i was referring to the coarse-grained network interceptors, I only looked
at the AMQP and Core ones though but didn't implement anything because I
couldn't get the trace information to them using thread local variables or
piggy backing on protocolData.
was (Author: ryeats):
The broker performance but also in the context of how long its taking messaging
clients to actually send and receive message from the broker also. Specifically
the time Artemis receives a message to when its sent to a specific consumer and
then the time it takes that consumer to actually acknowledge are of interest.
Which would be expensive to capture in logs and hard to string together
multiple metrics to get an accurate picture so i want to sample messages using
open telemetry tracing and then look at the trace samples that took a long time
to see if there is some sort of resource constraint that correspond to those
times in our metrics that I can fix.
Yes, i was referring to the coarse-grained network interceptors, I only looked
at the AMQP and Core ones though but didn't implement anything because I
couldn't get the trace information to them using thread local variables or
piggy backing on protocolData.
> Tracing and OpenTelemetry in Artemis
> ------------------------------------
>
> Key: ARTEMIS-5438
> URL: https://issues.apache.org/jira/browse/ARTEMIS-5438
> Project: ActiveMQ Artemis
> Issue Type: Improvement
> Components: Broker
> Affects Versions: 2.40.0
> Reporter: Ryan Yeats
> Priority: Major
> Attachments: Screenshot 2025-04-17 at 3.24.43 PM.png
>
>
> If a consumer is slow to process a messages it looks like artemis is slow
> delivering them. I would like to be able to trace a request all the way
> through the entire system and get a comprehensive idea of what is going on
> and where latency is being added.
>
> To do this I recently took a look at the example to add tracing support to
> Artemis through the plugins API and the ARTEMIS-2028 ticket related to
> tracing. The example only would work for anycast messaging its not properly
> handling spans for multicast. Resolving that doesn't solve the issues because
> there isn't a way to pass the correct traceparent unless your using AMQP
> specific DeliveryAnnotation but those aren't in the location or even named
> the way QPID expects so in the end that doesn't work either.
> Issues:
> * Interceptors don't have access to enough context about which consumer is
> receiving the message in order to add a the traceparent info specific for
> that delivery.
> * It's not trivial to implement, I made use of Message.userContext and
> MessageReference.protocolData to pass around spans between plugin points. Its
> not really how they are intended to be used but it wasn't safe to use Thread
> local between the various plugin calls. Its also not clear when or if it
> makes sense to include message routing in traces.
> * It should not require a specific messaging protocol code to be able to add
> trace information we should be able to use the Jms or application properties
> and if a protocols has support for traces Artemis should also be able to
> support that.
> If it makes sense for the interceptors to be passed protocolData then I could
> make the tracing work using Artemis plugins but it may be better to modify
> Artemis to add support for tracing using micrometer tracing library too but
> thats is a much bigger change.
>
> Here are the changes i made to the example for my attempt to add tracing
> [https://github.com/apache/activemq-artemis-examples/compare/main...ryeats:activemq-artemis-examples:trace_example]
>
> Attached is an image showing the spans that aren't attached to the correct
> parent
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
For further information, visit: https://activemq.apache.org/contact
