hey Ashok, Have you tried enabling debug logs to check if the tracer is getting initialized and generating traces?
You can do this by running: ceph config set global debug_trace 20 Then, try grepping the logs for relevant trace information. You can also refer to tracer.cc <https://github.com/ceph/ceph/blob/main/src/common/tracer.cc> to see where the debug points are. If you are using Rook-Ceph, you’ll need to inject a Jaeger sidecar into the OSD and RGW pods. Yuval and I worked on using Rook-Ceph with Jaeger for our Cephalocon presentation last year [1], and you can take inspiration from that approach if you’re using Rook-Ceph [2]. [1] https://www.youtube.com/watch?v=Yo9yB3KYBwo [2] https://gist.github.com/ideepika/42f87206cbde8a4bff78f93af72ee642 Feel free to reach out if you have any questions. Best, Deepika On Thu, Sep 25, 2025 at 11:41 AM Ashok Kumar M <[email protected]> wrote: > Hi Ceph Community, > > I'm currently running a Ceph cluster inside a Kubernetes environment > using *Rook-Ceph*, and I'm working on enabling *distributed tracing for > Ceph RGW*. > As per the https://docs.ceph.com/en/latest/jaegertracing/ , I used the > following docker run command to deploy a Jaeger all-in-one instance for > testing: > > docker run -d --name jaeger \ > -e COLLECTOR_ZIPKIN_HOST_PORT=:9411 \ > -e COLLECTOR_OTLP_ENABLED=true \ > -p 6799:6799/udp \ > -p 6832:6832/udp \ > -p 5778:5778 \ > -p 16686:16686 \ > -p 4317:4317 \ > -p 4318:4318 \ > -p 14250:14250 \ > -p 14268:14268 \ > -p 14269:14269 \ > -p 9411:9411 \ > jaegertracing/all-in-one:latest \ > --processor.jaeger-compact.server-host-port=6799 > > However, when using the *latest Jaeger version*, I get the following > error: > unknown flag: --processor.jaeger-compact.server-host-port > > After researching, I discovered that this flag is only available in *older > versions of Jaeger (e.g., v1.27)* and is no longer supported in Jaeger > v2+. > > I then tried using a specific older Jaeger version ( > jaegertracing/all-in-one:1.27) that supports the flag. The container runs > fine and listens on port 6799 as expected, but I *still don’t see any RGW > traces appearing in the Jaeger UI*. > My Setup: > > - > > Ceph cluster deployed via Rook in Kubernetes > - > > Ceph RGW is running as a pod inside the cluster > - > > Jaeger all-in-one is running externally (for testing) > - > > Jaeger port 6799/udp is exposed and forwarded > - > > Tracing is enabled in Ceph config > > > Questions: > > 1. > > Could the issue be related to the Ceph RGW pod not being able to send > UDP traces to the Jaeger container? > 2. > > Is there a specific configuration needed inside the RGW pod or > Rook-Ceph to make sure tracing data is being sent correctly? > 3. > > Should I consider using the default Jaeger port 6831/udp instead of > 6799/udp, and configure Ceph accordingly? > 4. > > Are there any known compatibility issues between Ceph RGW tracing and > Jaeger v1.x? > 5. > > Could you suggest some *troubleshooting steps* to help verify whether > traces are actually being sent from RGW and received by Jaeger? > > Any advice or working examples from a similar Kubernetes-based setup would > be greatly appreciated. > > > > ------------------------------ > *Disclaimer :** The content of this email and any files transmitted with > it are confidential and intended solely for the use of the individual or > entity to which they are addressed. If you have received this email in > error, please notify the sender and remove the messages from your system. > If you are not the named addressee, it is strictly forbidden for you to > share, circulate, distribute or copy any part of this e-mail to any third > party without the written consent of the sender.* > > > > *E-mail transmission cannot be guaranteed to be secured or error free as > information could be intercepted, corrupted, lost, destroyed, arrive late, > incomplete, or may contain viruses. Therefore, we do not accept liability > for any errors or omissions in the contents of this message, which arise as > a result of e-mail transmission. The recipient should check this e-mail and > any attachments for the presence of viruses. The company accepts no > liability for any damage caused by any virus transmitted by this email."* > > _______________________________________________ > Dev mailing list -- [email protected] > To unsubscribe send an email to [email protected] > _______________________________________________ ceph-users mailing list -- [email protected] To unsubscribe send an email to [email protected]
