Hi all, I’m not sure if I misread some of the comments earlier - but I was under the impression that
> We could probably (additionally) store it [referring to: OTel Trace ID] in > the request_id column if the Polaris-specific request ID header is not set. meant that we would only persist one or the other, not both. Personally, the answer for the question I posed earlier still remains unclear to me: what is the use case for keeping both within events persistence? The reason for having them originally was to correlate events to requests that the user is sending. Is there any other use cases known (or on the horizon) at this time? Based on what I’ve read, I believe we should only persist the single request ID that the server returned back to the client - regardless of whether it is the OTel Trace ID or the `x-request-id` header. As per Dmitri’s comment, I’m okay if we put in the additional_properties section whether the request ID was user-input or OTel, although I’m still not sure what that helps with? Best, Adnan Hemani > On Oct 29, 2025, at 10:04 AM, Yufei Gu <[email protected]> wrote: > > I think the first question is whether any downstream use cases need to > interpret the semantics of a request ID or OTel context. As I mentioned, > the original purpose of a request ID was event correlation, e.g., tying a > BeforeUpdateTable event to the corresponding AfterUpdateTable event. If > that’s the only requirement, an opaque correlation ID is good enough. We > could change the field name(request-id) if needed. > > If we do have use cases that depend on the meaning of the request ID or > OTel context, we can store them in additional_properties ( > https://www.google.com/url?q=https://github.com/polaris-catalog/polaris/blob/main/persistence/relational-jdbc/src/main/resources/postgres/schema-v3.sql%23L134-L134&source=gmail-imap&ust=1762362311000000&usg=AOvVaw04HH7BS9bFY4FOCh2ER0Pw). > This is more flexible because it allows us to: > 1. Record a key that indicates the format (e.g., x-request-id vs. > otel-context). > 2. Preserve both when Polaris receives both headers. > 3. Add future tracing fields without a schema change. > > Yufei > > > On Wed, Oct 29, 2025 at 9:20 AM Dmitri Bourlatchkov <[email protected]> > wrote: > >> Hi All, >> >> From my POV the use cases for x-request-id and OTel are slightly different. >> OTel is an observability use case where cross-linking of various pieces of >> data is desired but not required in all cases (data loss is acceptable). >> The x-request-id use case appears to have some very specific custom >> applications without any obvious relationship to OTel. >> >> Given the current state of this discussion, I think it makes sense to >> handle and persist them separately. >> >> However, if we intend x-request-id to act as an observability aid >> alternative to OTel, then it might be preferable to have a config flag >> indicating which one of them should be added to persistent event data. In >> that case we should reuse the same column for them (and possibly add a >> "type" prefix, e.g. "x-request-id:ABCDEF" or "traceparent:00-12345...") >> >> WDYT? >> >> Thanks, >> Dmitri. >> >> >> On Wed, Oct 29, 2025 at 9:38 AM Alexandre Dutra <[email protected]> wrote: >> >>> Hi Yufei, >>> >>>> I’d suggest persisting a single opaque correlation ID instead >>> >>> The problem I see with this is that readers cannot know the syntax of >>> the opaque id, and will have to "probe" using some heuristics, e.g. if >>> the id matches the OTel trace context syntax, asume it's OTel, >>> otherwise... assume it's something else. >>> >>> That's not unfeasible, but I wonder if it isn't better to store the 2 >>> ids separately. >>> >>> Thanks, >>> Alex >>> >>> On Tue, Oct 28, 2025 at 7:14 PM Yufei Gu <[email protected]> wrote: >>>> >>>> Thanks Alex for raising this discussion. Thanks everyone for chiming >> in. >>>> The plan looks good to me overall. >>>> >>>>> Update PolarisEvent, expose request ID if available, expose OTel >>> context >>>> if available. >>>> >>>> https://www.google.com/url?q=https://github.com/apache/polaris/pull/2914&source=gmail-imap&ust=1762362311000000&usg=AOvVaw11MsqGqjbsQVqTPLDZiOT2 >>>> persists both request id >> and >>>> otel context. >>>> >>>> Do we actually need to persist both? >>>> From the original discussion, the intent of a request ID was to >> correlate >>>> multiple events generated by the same request, for example, linking a >>>> BeforeUpdateTable and an AfterUpdateTable event. In that sense, what we >>>> really need is a single opaque identifier that ties related events >>>> together, regardless of whether it comes from a request ID or an OTel >>>> context. >>>> >>>> If we persist both fields (requestId and otelContext), it can lead to >>>> consumer confusion, since downstream systems would need to decide which >>> one >>>> to rely on, potentially causing inconsistent event correlation. Unless >>>> there are concrete use cases where event consumers need both, I’d >> suggest >>>> persisting a single opaque correlation ID instead. This keeps the >> schema >>>> simpler and ensures consistent event linking. >>>> >>>> Yufei >>>> >>>> >>>> On Tue, Oct 28, 2025 at 10:08 AM Alexandre Dutra <[email protected]> >>> wrote: >>>> >>>>> Hi all, >>>>> >>>>> Here is the PR: >>>>> https://www.google.com/url?q=https://github.com/apache/polaris/pull/2914&source=gmail-imap&ust=1762362311000000&usg=AOvVaw11MsqGqjbsQVqTPLDZiOT2 >>>>> >>>>> Thanks, >>>>> Alex >>>>> >>>>> On Sun, Oct 26, 2025 at 5:49 PM Jean-Baptiste Onofré < >> [email protected]> >>>>> wrote: >>>>>> >>>>>> Hi >>>>>> >>>>>> Yeah it looks good to me and I agree we have a consensus. >>>>>> >>>>>> Thanks ! >>>>>> Regards >>>>>> JB >>>>>> >>>>>> Le dim. 26 oct. 2025 à 14:29, Alexandre Dutra <[email protected]> >> a >>>>> écrit : >>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> Thanks for the good discussion so far. I think we have a broad >>>>> consensus. >>>>>>> >>>>>>> Unless anybody disagrees I am going to move forward with the the >>>>>>> following revised plan, incorporating the latest feedback: >>>>>>> >>>>>>> 1. Restore the original functionality for request IDs, change the >>>>>>> default header name back to x-request-id >>>>>>> >>>>>>> 2. Remove RequestIdGenerator and related functionality. >>>>>>> >>>>>>> 3. Update PolarisEvent, expose request ID if available, expose >> OTel >>>>>>> context if available. >>>>>>> >>>>>>> 4. Update events table SQL schema: insert request ID if >> available, >>>>>>> insert OTel context if available. >>>>>>> >>>>>>> Thanks, >>>>>>> Alex >>>>>>> >>>>>>> On Sat, Oct 25, 2025 at 11:12 AM Jean-Baptiste Onofré < >>> [email protected] >>>>>> >>>>>>> wrote: >>>>>>>> >>>>>>>> Hi >>>>>>>> >>>>>>>> I think it's better to use w3c_trace_context (better than two >>>>> fields). >>>>>>>> >>>>>>>> Regarding x-request-id, I think it's for a different purpose, >>>>>>>> specifically for persistence. Personally, I think we can >> achieve >>> the >>>>>>>> same with span and events. >>>>>>>> If we want to "distinguish" the two layers, it makes sense to >>> have >>>>>>>> x-request-id (not sure it's super helpful). Else, I think we >> can >>>>>>>> achieve the same with span/event. >>>>>>>> >>>>>>>> Just my $0.01 >>>>>>>> >>>>>>>> Regards >>>>>>>> JB >>>>>>>> >>>>>>>> On Sat, Oct 25, 2025 at 10:42 AM Adnan Hemani >>>>>>>> <[email protected]> wrote: >>>>>>>>> >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> +1 to all of Alex’s AIs with Dmitri’s suggested changes as >>> well. >>>>> Great >>>>>>> find, Dmitri! >>>>>>>>> >>>>>>>>> I’m still debating with myself on whether we need to store >> the >>>>>>> `x-request-id` field as part of the Events persistence. Can we >>> think >>>>> of a >>>>>>> good use case where this is more helpful to the user than the >> OTel >>>>>>> Trace/Span IDs? I am making the assumption here that those are >>> still >>>>> being >>>>>>> returned back to the client. >>>>>>>>> >>>>>>>>> Best, >>>>>>>>> Adnan Hemani >>>>>>>>> >>>>>>>>>> On Oct 24, 2025, at 9:12 AM, Alexandre Dutra < >>> [email protected]> >>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Hi Dmitri, >>>>>>>>>> >>>>>>>>>> Yes, I think your suggestion to use just one field >>>>> w3c_trace_context >>>>>>>>>> makes more sense than two fields (span_id and trace_id). >>>>>>>>>> >>>>>>>>>> With that, I also think that we are slowly drifting into >>>>>>>>>> implementation considerations; let's get consensus on the >>> general >>>>>>>>>> design first, and we can certainly fine-tune the actual >> Java >>>>> methods >>>>>>>>>> and SQL table columns in the future PR. WDYT? >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Alex >>>>>>>>>> >>>>>>>>>> On Fri, Oct 24, 2025 at 5:14 PM Dmitri Bourlatchkov < >>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>> Hi All, >>>>>>>>>>> >>>>>>>>>>> Many thanks for the background info, Adnan! >>>>>>>>>>> >>>>>>>>>>> +1 to action items proposed by Alex! >>>>>>>>>>> >>>>>>>>>>> Re: (3) Can we abstract request/trace info into a separate >>>>> object >>>>>>> without >>>>>>>>>>> exposing those accessors on the Event class directly? OTel >>>>> defines >>>>>>>>>>> trace/span concepts, but in request ID is a bit foreign to >>> OTel. >>>>>>> Having >>>>>>>>>>> tracing / request ID isolated in java could help with >>>>> maintaining >>>>>>> it and >>>>>>>>>>> potentially supporting other (custom) tracing methods. >>>>>>>>>>> >>>>>>>>>>> Re: (4) I'd like to propose storing OTel correlation data >>> in the >>>>>>> form of a >>>>>>>>>>> standard context propagation string, e.g. W3C >> trace-context >>> [1] >>>>>>> (same value >>>>>>>>>>> as its HTTP header), so the column could be called >>>>>>> w3c_trace_context or >>>>>>>>>>> simply trace_context. >>>>>>>>>>> >>>>>>>>>>> Open question: do we need to write a separate, individual >>> trace >>>>> ID >>>>>>> field in >>>>>>>>>>> SQL? I suppose it is not very useful since correlating it >> to >>>>> other >>>>>>> trace >>>>>>>>>>> data already requires understanding OTel context >> propagation >>>>> and a >>>>>>> query >>>>>>>>>>> against trace_context can still be made using >>> string-matching >>>>>>> clauses. We >>>>>>>>>>> could probably (additionally) store it in the request_id >>> column >>>>> if >>>>>>> the >>>>>>>>>>> Polaris-specific request ID header is not set. >>>>>>>>>>> >>>>>>>>>>> As for span ID, I do not really see a use case for >>> persisting it >>>>>>>>>>> individually. It is very specific to OTel trace data >>>>> construction. >>>>>>>>>>> >>>>>>>>>>> Actually, using the W3C trace context [1] encoding >> probably >>>>> makes >>>>>>> sense in >>>>>>>>>>> the java event representation too. Interested callers can >>> easily >>>>>>> decode >>>>>>>>>>> this information since the format is well-defined. As a >> side >>>>>>> benefit, this >>>>>>>>>>> opens opportunities for downstream event consumers to >>> connect >>>>>>> (propagate >>>>>>>>>>> context) to traces that produced events based on the event >>> data >>>>>>> itself, >>>>>>>>>>> without relying on the intermediate frameworks. This may >> be >>>>>>> desirable since >>>>>>>>>>> the current Polaris event persistence impl. writes events >> in >>>>>>> batches, so >>>>>>>>>>> the association to individual requests that produced those >>>>> events >>>>>>> is lost. >>>>>>>>>>> Whether to perform this kind of context propagation will >> be >>> at >>>>> the >>>>>>>>>>> discretion of the event consumer, of course (outside of >>> Polaris >>>>>>> code). >>>>>>>>>>> >>>>>>>>>>> [1] >>>>>>> >>>>> >>> >> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.w3.org/TR/trace-context/%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw2fn0lRsTx-f9r4PCz8wmJK&source=gmail-imap&ust=1762362311000000&usg=AOvVaw2dvj8jJ6YcldfV2g-Xc27v >>>>>>>>>>> >>>>>>>>>>> WDYT? >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Dmitri. >>>>>>>>>>> >>>>>>>>>>> On Fri, Oct 24, 2025 at 7:18 AM Alexandre Dutra < >>>>> [email protected]> >>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi all, >>>>>>>>>>>> >>>>>>>>>>>> Thank you for chiming in; the context around request IDs >>> is now >>>>>>> clear. >>>>>>>>>>>> >>>>>>>>>>>> Trying to summarize this thread into actionable items, >>> here's >>>>> what >>>>>>> I >>>>>>>>>>>> propose: >>>>>>>>>>>> >>>>>>>>>>>> 1. Restore the original functionality for request IDs. >>>>>>>>>>>> - Change the default header name back to x-request-id >>>>> (despite >>>>>>> the >>>>>>>>>>>> x- prefix being deprecated), but keep it configurable as >>> today. >>>>>>>>>>>> 2. Remove RequestIdGenerator and related functionality. >>>>>>>>>>>> - Do not generate a request ID if the REST client >>> doesn't >>>>>>> provide one. >>>>>>>>>>>> 3. Update PolarisEvent: >>>>>>>>>>>> - Expose new requestId(), traceId(), spanId() methods, >>> all >>>>>>> nullable. >>>>>>>>>>>> - This would align with the emerging consensus around >>>>> including >>>>>>>>>>>> contextual information in PolarisEvent [1]. >>>>>>>>>>>> 4. Update events table SQL schema: >>>>>>>>>>>> - Insert the client-provided request ID into the >>> request_id >>>>>>>>>>>> column; otherwise, insert null. >>>>>>>>>>>> - Add two new nullable columns, trace_id and span_id, >>> and >>>>>>> populate >>>>>>>>>>>> them if OTel is enabled. >>>>>>>>>>>> >>>>>>>>>>>> From our discussions, I think it's important to not >>> conflate >>>>> OTel >>>>>>>>>>>> tracing fields with Envoy tracing fields, which is why I >>>>> suggest we >>>>>>>>>>>> use separate fields / columns for them. >>>>>>>>>>>> >>>>>>>>>>>> Would the above plan work for everyone? >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Alex >>>>>>>>>>>> >>>>>>>>>>>> [1]: >>>>>>> >>>>> >>> >> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://lists.apache.org/thread/rl5cpcft16sn5n00mfkmx9ldn3gsqtfy%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw02wPvb0qxRzYAKEP0h8l9T&source=gmail-imap&ust=1762362311000000&usg=AOvVaw2T5pohrqE8tGS2t3tyP2CP >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Fri, Oct 24, 2025 at 9:33 AM Adnan Hemani >>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi all, >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks to Alex for starting this thread - because of >>> this, I’m >>>>>>> just >>>>>>>>>>>> coming to the realization that OTel Trace and Span IDs >> are >>>>> coming >>>>>>> built-in >>>>>>>>>>>> with Quarkus and my previous work to generate a >>>>>>> Request/Correlation ID is >>>>>>>>>>>> likely not needed as a result. My original motivation for >>>>>>> generation of a >>>>>>>>>>>> Request/Correlation ID was to ensure that any client can >>>>> uniquely >>>>>>> identify >>>>>>>>>>>> a request made to Polaris, which would be especially >>> useful for >>>>>>> debugging >>>>>>>>>>>> failing requests or identifying call patterns. >>>>>>>>>>>>> >>>>>>>>>>>>> As a result, I’m a +1 on Michael’s opinion: we should >>> remove >>>>> the >>>>>>>>>>>> Request/Correlation ID generation and always use the OTel >>>>>>> trace/span IDs >>>>>>>>>>>> (which come for free with Quarkus) instead for the >>> Correlation >>>>> ID >>>>>>> unless a >>>>>>>>>>>> valid header is present, which would take over as the >>>>> Correlation >>>>>>> ID >>>>>>>>>>>> instead. >>>>>>>>>>>>> >>>>>>>>>>>>> — >>>>>>>>>>>>> >>>>>>>>>>>>> To answer Dmitri’s question re Polaris Events: The >>> intended >>>>> use >>>>>>> case is >>>>>>>>>>>> to provide some sort of correlation between events that >>> have >>>>>>> occurred as >>>>>>>>>>>> part of the same request. For example, if a user makes an >>>>>>> CommitTransaction >>>>>>>>>>>> request, it would be helpful to see all UpdateTable calls >>> that >>>>>>> were made as >>>>>>>>>>>> part of that one user request. >>>>>>>>>>>>> >>>>>>>>>>>>> Best, >>>>>>>>>>>>> Adnan Hemani >>>>>>>>>>>>> >>>>>>>>>>>>>> On Oct 23, 2025, at 12:15 PM, Dmitri Bourlatchkov < >>>>>>> [email protected]> >>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Michael, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Logging x-request-id headers makes sense. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Just to confirm: if / when we restore that, Polaris >> will >>> NOT >>>>>>> generate >>>>>>>>>>>> new >>>>>>>>>>>>>> IDs in case the header is not present in the request, >>>>> correct? >>>>>>>>>>>>>> >>>>>>>>>>>>>> I believe x-request-id can co-exist with OTel. >>>>>>>>>>>>>> >>>>>>>>>>>>>> What about adding request IDs to events [1][2]? What's >>> the >>>>>>> intended use >>>>>>>>>>>>>> case for that? Could you share some context here too? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Side note: I proposed [2877] flagging event persistence >>> as >>>>>>> "beta" in >>>>>>>>>>>>>> 1.2.0... This discussion adds another point towards >>> that, I >>>>>>> think. >>>>>>>>>>>>>> >>>>>>>>>>>>>> [1] >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>> >>>>> >>> >> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%252523L97%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw1WfUaXLp6z_M87iAXEqSUw%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw1Pioys4ROm8vYM_mdx4ygH&source=gmail-imap&ust=1762362311000000&usg=AOvVaw1ZMsu0wq2eWDF4OVIWaywJ >>>>>>>>>>>>>> [2] >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>> >>>>> >>> >> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%252523L66%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw12-7e3ahm2sLSkLSNqTecm%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw1PGW6uFwtUN8F1dOlJwrpm&source=gmail-imap&ust=1762362311000000&usg=AOvVaw385xLbQwQJRS5fmXlOPG8k >>>>>>>>>>>>>> >>>>>>>>>>>>>> [2877] >>>>>>>>>>>> >>>>>>> >>>>> >>> >> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://github.com/apache/polaris/pull/2877%252523discussion_r2456300613%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw3TuYbkzwnLx3QEVIM8oDda%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw2Tfk7wAM1MaB5z9Dvw5X5H&source=gmail-imap&ust=1762362311000000&usg=AOvVaw1uAmRmbITVSySHFKIYJQ-f >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Dmitri. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 2:23 PM Michael Collado < >>>>>>>>>>>> [email protected]> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hey Dmitri >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The generating a request id is new code that was added >>>>> after the >>>>>>>>>>>> original >>>>>>>>>>>>>>> x-request-id support. You can see the state from ~1 >> year >>>>> ago, we >>>>>>>>>>>> hard-coded >>>>>>>>>>>>>>> request_id as the header we used for the MDC - >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>> >>>>> >>> >> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%252523L415-L416%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw35Q1A_2avAiSYlYVAnZxBb%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw34DksOw7DSHJu8PfQQ5bT6&source=gmail-imap&ust=1762362311000000&usg=AOvVaw2UoTrfPw_IJnFET_H3G_-M >>>>>>>>>>>>>>> . At some point, it was changed to be configurable, >>> then the >>>>>>>>>>>>>>> ContextResolverFilter filter was refactored/eliminated >>> and >>>>> the >>>>>>>>>>>>>>> RequestIdFilter took its responsibility, but lost some >>> of >>>>> its >>>>>>> original >>>>>>>>>>>>>>> functionality. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> My personal opinion is that restoring support for the >>>>>>> x-request-id >>>>>>>>>>>> header >>>>>>>>>>>>>>> is something that we should do, but if the header >> isn't >>>>> present, >>>>>>>>>>>> falling >>>>>>>>>>>>>>> back on simply using OTel trace ids is good enough >>> (better, >>>>>>> even) than >>>>>>>>>>>>>>> generating another random request id. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Mike >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 10:47 AM Dmitri Bourlatchkov < >>>>>>>>>>>> [email protected]> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi Michael, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks for the info! >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Working with Envoy's tracing headers makes sense to >> me. >>>>>>> However, I >>>>>>>>>>>>>>> wonder: >>>>>>>>>>>>>>>> why would Polaris need to generate a new request ID >>> inside >>>>> its >>>>>>>>>>>> code?.. >>>>>>>>>>>>>>>> and return it in response headers? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> How important is it to propagate this ID to Polaris >>> Events? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I'm just trying to understand the full context :) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> Dmitri. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 1:29 PM Michael Collado < >>>>>>>>>>>> [email protected]> >>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I think the original intention for this requestId >>> field >>>>> was to >>>>>>>>>>>> support >>>>>>>>>>>>>>>>> request propagation from load balancers, like Envoy >> ( >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>> >>>>> >>> >> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw1RnuM8mViV-j7jvuxq74Aw%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw3LNqtilZ2OoJfE7yEwraZa&source=gmail-imap&ust=1762362311000000&usg=AOvVaw0Rnn1QlkkrAmt4j8rz0gNl >>>>>>>>>>>>>>>>> ), which is distinct from OTEL. Don't ask me why the >>>>> default >>>>>>>>>>>>>>>>> is Polaris-Request-Id - I think it was originally a >>> custom >>>>>>> thing, >>>>>>>>>>>> but >>>>>>>>>>>>>>>> then >>>>>>>>>>>>>>>>> we changed to integrate with existing conventions. >>>>>>> Unfortunately, >>>>>>>>>>>>>>> looking >>>>>>>>>>>>>>>>> through the code, I think that the actual functional >>>>> plumbing >>>>>>> has >>>>>>>>>>>> been >>>>>>>>>>>>>>>> lost >>>>>>>>>>>>>>>>> in the course of multiple refactors around the call >>>>> context >>>>>>> and >>>>>>>>>>>>>>>> resolver. I >>>>>>>>>>>>>>>>> don't see references to that property or the >>> underlying >>>>>>> header. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Support for the unofficial x-request-id header feels >>> like >>>>>>> something >>>>>>>>>>>> we >>>>>>>>>>>>>>>>> should definitely keep, especially when Polaris is >> one >>>>>>> service in a >>>>>>>>>>>>>>> mesh >>>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>> services that maybe don't have OTel integration. >> I'm a >>>>> fan of >>>>>>> the >>>>>>>>>>>> OTel >>>>>>>>>>>>>>>>> standard, but it's not entirely ubiquitous and there >>> are >>>>> many >>>>>>>>>>>>>>> middleware >>>>>>>>>>>>>>>>> layers that know how to forward on the x-request-id >>>>> header. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Mike >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Thu, Oct 23, 2025 at 3:00 AM Robert Stupp < >>>>> [email protected]> >>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Yes, we should aim for interoperability with the >>> existing >>>>>>> de-facto >>>>>>>>>>>>>>>>>> standard OTel and make it easy for users to >> integrate >>>>> into >>>>>>> their >>>>>>>>>>>>>>>>>> observability platforms. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri >> Bourlatchkov < >>>>>>>>>>>>>>> [email protected]> >>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Hi Alex and All, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I certainly support the idea of following OTel >>>>> standards for >>>>>>>>>>>>>>>> achieving >>>>>>>>>>>>>>>>>>> "correlation" wrt Polaris requests and/or events. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> As to what form the correlation data should take, >> I >>>>> believe >>>>>>> it is >>>>>>>>>>>>>>>>>>> conceptually what the OTel "context" represents. >>> So, I >>>>>>> believe it >>>>>>>>>>>>>>>> makes >>>>>>>>>>>>>>>>>>> sense for Polaris to support standard context >>>>> propagators >>>>>>> at the >>>>>>>>>>>>>>> API >>>>>>>>>>>>>>>>>> layer. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> If the incoming request has OTel context >>> information, >>>>> then >>>>>>>>>>>>>>> returning >>>>>>>>>>>>>>>>> any >>>>>>>>>>>>>>>>>>> other "correlation" data in the response is >>> redundant, I >>>>>>> think. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> If the incoming request does not have OTel context >>> info, >>>>>>> what is >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>> purpose of generating a Polaris-specific >>> "correlation >>>>> ID"? >>>>>>> How is >>>>>>>>>>>>>>> it >>>>>>>>>>>>>>>>>>> envisioned to be used? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> If the intention is to correlate a Polaris >> response >>>>>>> (operation) >>>>>>>>>>>>>>> with >>>>>>>>>>>>>>>>>> events >>>>>>>>>>>>>>>>>>> that might have resulted from its execution, I >>> believe a >>>>>>> more >>>>>>>>>>>>>>> robust >>>>>>>>>>>>>>>>>>> approach would be to propagate the OTel trace info >>>>>>> (starting a new >>>>>>>>>>>>>>>>> trace >>>>>>>>>>>>>>>>>> if >>>>>>>>>>>>>>>>>>> necessary) into event data. Then, Polaris can also >>>>> return >>>>>>> the >>>>>>>>>>>> trace >>>>>>>>>>>>>>>>>> context >>>>>>>>>>>>>>>>>>> in the API response (top span). It's a bit awkward >>> from >>>>> the >>>>>>> OTel >>>>>>>>>>>>>>>>>>> perspective, but might be an option for supporting >>>>> custom >>>>>>>>>>>>>>>> correlators. >>>>>>>>>>>>>>>>> It >>>>>>>>>>>>>>>>>>> could be covered by a feature flag. The header >> name >>>>> could be >>>>>>>>>>>>>>>>>>> "polaris-traceparent" for W3C Trace Context. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Custom correlation code will be able to extract >> the >>>>> trace >>>>>>> ID from >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>> response and from events and find related data. >>>>> Granted, it >>>>>>> will >>>>>>>>>>>>>>>>> require >>>>>>>>>>>>>>>>>> a >>>>>>>>>>>>>>>>>>> bit more effort for the custom code to decode >> trace >>> IDs >>>>>>> from the >>>>>>>>>>>>>>> OTel >>>>>>>>>>>>>>>>>>> context, but the format is standard and not >>> complex. The >>>>>>> benefit >>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>>>> Polaris, though, is that it can easily integrate >>> with >>>>>>>>>>>>>>> OTel-compatible >>>>>>>>>>>>>>>>>>> observability platforms regardless of whether any >>>>> particular >>>>>>>>>>>>>>>> deployment >>>>>>>>>>>>>>>>>>> uses custom correlators or not. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> WDYT? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> Dmitri. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On Wed, Oct 22, 2025 at 6:03 AM Alexandre Dutra < >>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Today, Polaris has the notion of "request ID", >> but >>> its >>>>>>> purpose is >>>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>>>>>> entirely clear. It seems to serve as an >>> observability >>>>>>> feature to >>>>>>>>>>>>>>>>>>>> facilitate correlation. A pending PR aims to >>> rename it >>>>> to >>>>>>>>>>>>>>>>> "correlation >>>>>>>>>>>>>>>>>>>> ID" for better alignment with industry standards >>> [1]. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> However, this PR has brought to light overlaps >> with >>>>> core >>>>>>>>>>>>>>> telemetry >>>>>>>>>>>>>>>>>>>> features: when OpenTelemetry (OTel) is enabled in >>>>> Polaris, >>>>>>> each >>>>>>>>>>>>>>>>>>>> request already has a trace ID and span ID, >> making >>> a >>>>>>> separate >>>>>>>>>>>>>>>>>>>> correlation ID redundant. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Moreover, using the OTel trace ID and span ID in >>>>> Polaris >>>>>>> events, >>>>>>>>>>>>>>>>>>>> rather than the generated correlation ID, would >>>>>>> significantly >>>>>>>>>>>>>>>>> simplify >>>>>>>>>>>>>>>>>>>> correlation of events with other traces. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Therefore, I propose the following changes: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 1. If OTel is enabled, use the trace ID and span >>> ID as >>>>> the >>>>>>>>>>>>>>>>> correlation >>>>>>>>>>>>>>>>>>>> ID for the request, instead of generating a >> random >>>>>>> correlation >>>>>>>>>>>>>>> ID. >>>>>>>>>>>>>>>>>>>> 2. Otherwise, if a (Polaris-specific) correlation >>> ID >>>>>>> header is >>>>>>>>>>>>>>>>> present >>>>>>>>>>>>>>>>>>>> in the request, use it. >>>>>>>>>>>>>>>>>>>> 3. If neither of the above conditions is met, >>> generate >>>>> a >>>>>>> random >>>>>>>>>>>>>>>>>>>> correlation ID. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I am somewhat undecided on the best approach >> when a >>>>>>> correlation >>>>>>>>>>>>>>> ID >>>>>>>>>>>>>>>>>>>> header is present in the request. However, I >>> believe it >>>>>>> would be >>>>>>>>>>>>>>>> more >>>>>>>>>>>>>>>>>>>> sensible to disregard it if OTel is enabled, as >>> OTel >>>>>>> offers a >>>>>>>>>>>>>>> more >>>>>>>>>>>>>>>>>>>> robust solution for client-to-server trace >>> propagation, >>>>>>> e.g. W3C >>>>>>>>>>>>>>>>> Trace >>>>>>>>>>>>>>>>>>>> Context propagation [2]. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Please share your thoughts! >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> Alex >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> [1]: >>>>>>>>>>>> >>>>>>> >>>>> >>> >> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://github.com/apache/polaris/pull/2757%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw1-kAWfEk4tmsEg0q0GZBCn%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw1Oe-25vtt4gLVSMFMtSVNg&source=gmail-imap&ust=1762362311000000&usg=AOvVaw0oKBg5hY3-80AeZXaXcgmo >>>>>>>>>>>>>>>>>>>> [2]: >>>>>>>>>>>> >>>>>>> >>>>> >>> >> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.google.com/url?q%253Dhttps://www.w3.org/TR/trace-context%2526source%253Dgmail-imap%2526ust%253D1761851831000000%2526usg%253DAOvVaw22nMyOS7pbJ69XrBo5kHQS%26source%3Dgmail-imap%26ust%3D1761927167000000%26usg%3DAOvVaw1TRdkzABc_7U-_KZ1MZ59v&source=gmail-imap&ust=1762362311000000&usg=AOvVaw2dn5YK73cKTciJwSnNt8pd >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >>> >>
