pjfanning opened a new issue, #3472: URL: https://github.com/apache/pekko/issues/3472
This is informational rather than a bug report. The OpenTelemetry Java agent propagates trace context through pekko actors, streams and remoting by attaching bytecode advice to specific methods. Most of those methods are internal to pekko, so a refactor that is invisible to users can silently stop context propagation. Filing this so the list is written down somewhere the pekko project can see it. A companion issue covers pekko-http: apache/pekko-http#1241. Why "silently": the agent's muzzle checks verify the classes and methods its *advice code calls*, and disable the instrumentation when they no longer exist. They do not verify the method *matchers*. If a matched method is renamed, inlined, or restructured, the advice simply never applies — no error, no log, just broken propagation. We hit that during this work: `EndpointReader.dispatchMessage` reads like an ideal hook in the sources, but there is no such method in the bytecode because scala inlined it. Only `javap` showed that. The instrumentation now uses `DefaultMessageDispatcher.dispatch` instead. ### pekko-actor and pekko-stream | Symbol | Declared | Used for | | --- | --- | --- | | `Dispatcher.dispatch(ActorCell, Envelope)` | `protected[pekko]` | attaches the current context to the envelope of a message | | `ActorCell.invoke`, `ActorCell.systemInvoke` | `private[pekko] class` | restores that context while the actor handles the message | | `DefaultSystemMessageQueue.systemEnqueue` | `private[pekko] trait` | same for system messages | | `LightArrayRevolverScheduler.schedule*` | public methods, implementation class | propagating context through scheduled tasks | | `GraphInterpreter.processPush` | **`private def`** in `@InternalApi private[pekko] final class` | makes the context current when a stream stage passes an element to user code | `GraphInterpreter.processPush` is the most exposed of these: a genuinely private method, matched by name. It is load-bearing for pekko-http server context propagation. ### pekko-remote, artery | Symbol | Declared | Used for | | --- | --- | --- | | `RemoteInstruments.create` | `@InternalStableApi` | registering a `RemoteInstrument` without requiring users to edit `pekko.remote.artery.advanced.instruments` | | `RemoteInstruments.serialize`, `.deserialize` | `private[remote] final class` | associating the envelope being (de)serialized with the instrument, which is called with the message only | | `ReusableOutboundEnvelope.init`, `.copy`, `.clear` | `private[remote] final class` | capturing the sender's context, carrying it to a copy, clearing it when the pooled envelope is recycled | | `ReusableInboundEnvelope.init`, `.clear` | same | clearing a pooled envelope's previous context | | `artery.MessageDispatcher.dispatch` | `private[remote] class` | making the received context current while the message is delivered | ### pekko-remote, classic | Symbol | Declared | Used for | | --- | --- | --- | | `EndpointManager.Send` constructor and `copy` | `private[remote] object` | capturing the sender's context, and carrying it to the copy made to add a sequence number | | `EndpointWriter.writeSend` | `private[remote] class` | restoring that context while the message is serialized, which happens later than the send when the endpoint buffers | | `PekkoPduProtobufCodec.constructMessage`, `.decodeMessage` | `private[remote] object` | writing and reading the context | | `DefaultMessageDispatcher.dispatch` | `private[remote] class` | making the received context current while the message is delivered | ### One small documentation request `RemoteInstrument.identifier` says values 1 to 7 are reserved for pekko internal use, and `LoggingRemoteInstrument` carries the comment `// Cinnamon is using 0`. Those two comments are the only record of which identifiers are taken. In practice: 0 Lightbend Telemetry, 1 pekko's own `LoggingRemoteInstrument`, 8 Kamon, and 9 is what the OpenTelemetry agent now uses. Listing the known third-party identifiers in that scaladoc would stop the next implementer from having to grep other projects to find a free one, and would make collisions less likely. ### What would help otherwise No API change is being requested. Mainly awareness that these are load-bearing for an out-of-tree consumer, so a refactor of any of them can be mentioned in release notes. If any are considered stable in practice, `@InternalStableApi` would say so explicitly, as it already does for `RemoteInstruments.create`. Related OpenTelemetry work: open-telemetry/opentelemetry-java-instrumentation#19823 (remoting context propagation). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
