GitHub user weiqingy added a comment to the discussion: [Discuss][Observability] OpenTelemetry GenAI export for Agent Traces (follow-up to #900)
Thanks for making the mapping explicit rather than leaving it implicit in #900. OTLP export is worth pursuing, and a standalone exporter looks like the right direction. Could the proposal include a short rejected-alternatives note for Flink's `OpenTelemetryTraceReporterFactory`? Although it is available with the pinned Flink 2.3.0, it cannot preserve caller-derived IDs or span status, requires related spans to be reported together, and lacks consistent support across the 1.20, 2.0, and 2.1 distributions shipped here. That seems enough to explain why the native trace pipeline cannot represent an agent trace. Before implementation, could we clarify the architecture and recording contract on which the exporter would depend? **1. What durable source would the standalone exporter consume?** Today, Event Log output is either SLF4J or TaskManager-local files under `java.io.tmpdir`. Both are local, and the logger type is a closed enum that does not support adding a durable sink alongside the existing one. A separate consumer would therefore amount to tailing files that may disappear during failover. Could Phase 1 define durability, partition discovery, checkpoints, replay, retention, failover, and handling of incomplete or out-of-order lifecycle events? Would `event-log.trace.otel.*` also be clearer as consumer configuration, since an external process cannot access AgentPlan? **2. Could #924 capture the telemetry-neutral metadata while its contract is still open?** Several proposed OTel attributes currently have no reliable source: - The LLM entity name is a resource alias such as `_default_chat_model`, not necessarily the exact requested model. - Provider identity exists only indirectly in the resource descriptor. - Token counts are cumulative Flink counters keyed by model, rather than per-execution values. The existing `entityMetadata` channel is already serialized and used for tool executions, while both Java and Python LLM paths currently leave it empty. Would recording the resolved model, provider, and per-call token usage there keep the exporter focused on conversion rather than inference and metric joins? Could the contract also define how agent, action, LLM, parser, and tool executions become spans? It currently records no agent entity, while action roots have no parent and parser executions have no proposed mapping. Without explicit run, root, and parent semantics, could chained actions appear as unrelated roots or recorded executions be omitted? **3. Could the contract remain language-neutral from the start?** Python actions already report through Java, and Java's `EventLogWriter` serializes their events. Could "Java first, Python parity later" therefore refer only to implementation work, with one serialized contract and one exporter serving both runtimes? **4. Would an optional module provide a safer dependency boundary?** The Elasticsearch integration already brings an unrelocated OTel API 1.29.0 into the uber JAR, and the default OTLP sender would add a second okhttp package alongside the one already there. Introducing the 1.54.1 SDK/exporter can produce incompatible dependency mediation, including the observed missing `ComponentLoader` failure. Could a standalone optional artifact align its OTel dependencies through the BOM and avoid exposing that conflict to every distribution? **5. Could delivery semantics be defined separately from ID derivation?** Stable derived IDs would help with reconstruction, but they would not make OTLP delivery idempotent. Could the proposal describe consumer checkpoints, retries, and replay behavior for complete and partial spans? Would it make sense to frame Phase 1 as: 1. Finalize a language-neutral execution and metadata contract. 2. Define a durable, replayable Event Log source and consumer-checkpoint model. 3. Provide an optional standalone OTLP converter on top of that contract. In-process export, content capture, and further enrichment could then build on that foundation in later phases. GitHub link: https://github.com/apache/flink-agents/discussions/929#discussioncomment-17915174 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
