[
https://issues.apache.org/jira/browse/CAMEL-23708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen resolved CAMEL-23708.
---------------------------------
Fix Version/s: 4.21.0
Resolution: Fixed
> OpenTelemetry2 - Missing RECEIVED spans for routes consuming from stub/seda
> in nested async paths
> -------------------------------------------------------------------------------------------------
>
> Key: CAMEL-23708
> URL: https://issues.apache.org/jira/browse/CAMEL-23708
> Project: Camel
> Issue Type: Bug
> Components: camel-opentelemetry
> Reporter: Claus Ibsen
> Priority: Major
> Fix For: 4.21.0
>
>
> When using the stub component to simulate Kafka (e.g. with {{--observe}} /
> dev profile), route consumer RECEIVED spans are not exported for routes that
> consume from stub endpoints when the producer side is inside a multicast on a
> seda consumer thread.
> h3. Steps to reproduce
> Run the route-topology example with OpenTelemetry dev mode:
> {code}
> camel run route-topology.camel.yaml --observe
> {code}
> The example has this flow:
> {code}
> timer -> order-generator -> direct:process-order -> process-order ->
> kafka:orders
> -> order-dispatcher (multicast)
> -> kafka:fulfillment -> fulfillment route -> kafka:warehouse-shipments
> -> kafka:notifications -> notification route -> kafka:email-outbox
> {code}
> h3. Expected behavior
> Each trace should have 27 spans including EVENT_RECEIVED spans for all 6
> routes:
> - timer (order-generator) - OK
> - direct:process-order (process-order) - OK
> - direct:validate-order (validate-order) - OK
> - kafka:orders via stub (order-dispatcher) - OK
> - kafka:fulfillment via stub (fulfillment) - MISSING
> - kafka:notifications via stub (notification) - MISSING
> h3. Actual behavior
> Each trace has exactly 25 spans. The EVENT_RECEIVED spans for the fulfillment
> and notification routes are never exported to the DevSpanExporter.
> The RECEIVED spans ARE created (processor spans log4, to7, log5, to8
> correctly reference them as parentSpanId), but {{span.end()}} is apparently
> never called, so {{SimpleSpanProcessor.onEnd()}} never fires and they never
> reach the exporter.
> h3. Analysis
> The issue is specific to nested async routing: the fulfillment and
> notification routes consume from stub endpoints whose producers are inside a
> multicast within the order-dispatcher route, which is itself on a seda
> consumer thread. The {{onExchangeDone}} route policy callback
> ({{TracingRoutePolicy}} in {{camel-telemetry}}) does not fire for these
> exchanges, so the RECEIVED span is started but never ended/exported.
> The kafka:orders RECEIVED span (order-dispatcher route) works correctly -
> this is a direct stub producer to seda consumer path without the nested
> multicast.
> Key code path:
> - {{camel-telemetry}}: {{Tracer.TracingRoutePolicy.onExchangeDone()}} ->
> {{endEventSpan()}} -> {{spanLifecycleManager.deactivate(span)}} ->
> {{otelSpan.end()}}
> - {{camel-seda}}: {{SedaProducer.process()}} (InOnly path) ->
> {{addToQueue(exchange, true)}} -> async consumer pickup
> - {{camel-stub}}: extends seda, same behavior
> h3. Impact
> - Orphan spans in trace visualization (Jaeger, TUI waterfall) - processor
> spans reference non-existent parent
> - Missing routeId propagation for orphaned spans (dev console cannot walk
> parent chain)
> - Affects any scenario with stub/seda consumers inside multicast or other
> async EIPs
> h3. How this was discovered
> Using the TUI MCP {{tui_get_spans}} tool to fetch raw span JSON data from a
> running Camel TUI session, analyzing parent chain integrity across 20
> consecutive traces (500 spans). Every trace showed the identical orphan
> pattern.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
