Thanks for the nice summary, Alex! To All:
I'm not sure an INDEX on correlation_id is that useful... Polaris itself does not use that index, right? Do we expect end users to query events directly from the database? This will work only with RDBMS, but probably not with the proposed NoSQL persistence. I believe we still need to clarify / confirm whether returning any request / correlation ID in _response_ headers is required. >From my POV, I do not see a use case for it, but I wonder what other people think / need. As for persisting correlation ID type, I agree that we do not need a separate column for that. Still, it could be useful to embed some type data into the value to help with interpretation, specifically for telemetry use cases (both x-request-id and OTel). For example, we could persist "traceparent:00-12345-...". Then, the reader would be able to identify this as information following the W3C spec [1], parse it and link to other traces (if desired). That processing is optional and at the discretion of the event data reader, but making the value no-so-opaque upfront can enable a lot of interesting options for users without much extra overhead in Polaris. WDYT? [1] https://www.w3.org/TR/2021/REC-trace-context-1-20211123/ Thanks, Dmitri. On Thu, Oct 30, 2025 at 6:04 AM Alexandre Dutra <[email protected]> wrote: > Hi all, > > > we would only persist one or the other, not both. > > My apologies, I confess that this wasn't my understanding of the desired > design. > > Let's try to gather the requirements again: > > 1. Only one correlation ID is enough > 2. The correlation ID is an opaque string > 3. The main use case is to find events with matching correlation IDs > 4. The only query pattern is by exact match > > Regarding *where* to store the correlation ID, and expanding on what > Dmitri said: the requirement to find matching records by correlation > ID clearly suggests a separate column, and with an INDEX on it. > Storing it in additional_properties will be inefficient. > > If we all agree on the above, here is what I propose: > > 1. Rename the existing column request_id to correlation_id; > 2. Populate from OTel context if available; the exact syntax is > implementation-specific; > 3. Otherwise, populate from X-Request-Id header if available; > 4. Otherwise, don't populate. > 5. Create an INDEX on correlation_id. > > You'll notice that we don't store the ID type, but that doesn't seem > necessary. > > What do you all think? > > Thanks, > Alex > > > > > > > On Wed, Oct 29, 2025 at 8:58 PM Dmitri Bourlatchkov <[email protected]> > wrote: > > > > Hi Yufei, > > > > [...] tying a BeforeUpdateTable event to the corresponding > AfterUpdateTable > > event. > > > > > > It looks like this calls for a strong and always present ID in > events.This > > ID is obviously distincts from OTel context information and should be > > persisted separately, IMHO. > > > > I believe there's still an open question whether the request ID in the > > events should be populated from the x-request-id in REST API headers. > > > > Yufei: Is that required in your use cases? > > > > Thanks, > > Dmitri. > > > > > > > > On Wed, Oct 29, 2025 at 1:05 PM Yufei Gu <[email protected]> wrote: > > > > > I think the first question is whether any downstream use cases need to > > > interpret the semantics of a request ID or OTel context. As I > mentioned, > > > the original purpose of a request ID was event correlation, e.g., > tying a > > > BeforeUpdateTable event to the corresponding AfterUpdateTable event. If > > > that’s the only requirement, an opaque correlation ID is good enough. > We > > > could change the field name(request-id) if needed. > > > > > > If we do have use cases that depend on the meaning of the request ID or > > > OTel context, we can store them in additional_properties ( > > > > > > > https://github.com/polaris-catalog/polaris/blob/main/persistence/relational-jdbc/src/main/resources/postgres/schema-v3.sql#L134-L134 > > > ). > > > This is more flexible because it allows us to: > > > 1. Record a key that indicates the format (e.g., x-request-id vs. > > > otel-context). > > > 2. Preserve both when Polaris receives both headers. > > > 3. Add future tracing fields without a schema change. > > > > > > Yufei > > > > > > > > > On Wed, Oct 29, 2025 at 9:20 AM Dmitri Bourlatchkov <[email protected]> > > > wrote: > > > > > > > Hi All, > > > > > > > > From my POV the use cases for x-request-id and OTel are slightly > > > different. > > > > OTel is an observability use case where cross-linking of various > pieces > > > of > > > > data is desired but not required in all cases (data loss is > acceptable). > > > > The x-request-id use case appears to have some very specific custom > > > > applications without any obvious relationship to OTel. > > > > > > > > Given the current state of this discussion, I think it makes sense to > > > > handle and persist them separately. > > > > > > > > However, if we intend x-request-id to act as an observability aid > > > > alternative to OTel, then it might be preferable to have a config > flag > > > > indicating which one of them should be added to persistent event > data. In > > > > that case we should reuse the same column for them (and possibly add > a > > > > "type" prefix, e.g. "x-request-id:ABCDEF" or > "traceparent:00-12345...") > > > > > > > > WDYT? > > > > > > > > Thanks, > > > > Dmitri. > > > > > > > > > > > > On Wed, Oct 29, 2025 at 9:38 AM Alexandre Dutra <[email protected]> > > > wrote: > > > > > > > > > Hi Yufei, > > > > > > > > > > > I’d suggest persisting a single opaque correlation ID instead > > > > > > > > > > The problem I see with this is that readers cannot know the syntax > of > > > > > the opaque id, and will have to "probe" using some heuristics, > e.g. if > > > > > the id matches the OTel trace context syntax, asume it's OTel, > > > > > otherwise... assume it's something else. > > > > > > > > > > That's not unfeasible, but I wonder if it isn't better to store > the 2 > > > > > ids separately. > > > > > > > > > > Thanks, > > > > > Alex > > > > > > > > > > On Tue, Oct 28, 2025 at 7:14 PM Yufei Gu <[email protected]> > wrote: > > > > > > > > > > > > Thanks Alex for raising this discussion. Thanks everyone for > chiming > > > > in. > > > > > > The plan looks good to me overall. > > > > > > > > > > > > > Update PolarisEvent, expose request ID if available, expose > OTel > > > > > context > > > > > > if available. > > > > > > > > > > > > https://github.com/apache/polaris/pull/2914 persists both > request id > > > > and > > > > > > otel context. > > > > > > > > > > > > Do we actually need to persist both? > > > > > > From the original discussion, the intent of a request ID was to > > > > correlate > > > > > > multiple events generated by the same request, for example, > linking a > > > > > > BeforeUpdateTable and an AfterUpdateTable event. In that sense, > what > > > we > > > > > > really need is a single opaque identifier that ties related > events > > > > > > together, regardless of whether it comes from a request ID or an > OTel > > > > > > context. > > > > > > > > > > > > If we persist both fields (requestId and otelContext), it can > lead to > > > > > > consumer confusion, since downstream systems would need to decide > > > which > > > > > one > > > > > > to rely on, potentially causing inconsistent event correlation. > > > Unless > > > > > > there are concrete use cases where event consumers need both, I’d > > > > suggest > > > > > > persisting a single opaque correlation ID instead. This keeps the > > > > schema > > > > > > simpler and ensures consistent event linking. > > > > > > > > > > > > Yufei > > > > > > > > > > > > > > > > > > On Tue, Oct 28, 2025 at 10:08 AM Alexandre Dutra < > [email protected]> > > > > > wrote: > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > Here is the PR: https://github.com/apache/polaris/pull/2914 > > > > > > > > > > > > > > Thanks, > > > > > > > Alex > > > > > > > > > > > > > > On Sun, Oct 26, 2025 at 5:49 PM Jean-Baptiste Onofré < > > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > > Hi > > > > > > > > > > > > > > > > Yeah it looks good to me and I agree we have a consensus. > > > > > > > > > > > > > > > > Thanks ! > > > > > > > > Regards > > > > > > > > JB > > > > > > > > > > > > > > > > Le dim. 26 oct. 2025 à 14:29, Alexandre Dutra < > [email protected] > > > > > > > > a > > > > > > > écrit : > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > Thanks for the good discussion so far. I think we have a > broad > > > > > > > consensus. > > > > > > > > > > > > > > > > > > Unless anybody disagrees I am going to move forward with > the > > > the > > > > > > > > > following revised plan, incorporating the latest feedback: > > > > > > > > > > > > > > > > > > 1. Restore the original functionality for request IDs, > change > > > the > > > > > > > > > default header name back to x-request-id > > > > > > > > > > > > > > > > > > 2. Remove RequestIdGenerator and related functionality. > > > > > > > > > > > > > > > > > > 3. Update PolarisEvent, expose request ID if available, > expose > > > > OTel > > > > > > > > > context if available. > > > > > > > > > > > > > > > > > > 4. Update events table SQL schema: insert request ID if > > > > available, > > > > > > > > > insert OTel context if available. > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > Alex > > > > > > > > > > > > > > > > > > On Sat, Oct 25, 2025 at 11:12 AM Jean-Baptiste Onofré < > > > > > [email protected] > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > Hi > > > > > > > > > > > > > > > > > > > > I think it's better to use w3c_trace_context (better > than two > > > > > > > fields). > > > > > > > > > > > > > > > > > > > > Regarding x-request-id, I think it's for a different > purpose, > > > > > > > > > > specifically for persistence. Personally, I think we can > > > > achieve > > > > > the > > > > > > > > > > same with span and events. > > > > > > > > > > If we want to "distinguish" the two layers, it makes > sense to > > > > > have > > > > > > > > > > x-request-id (not sure it's super helpful). Else, I > think we > > > > can > > > > > > > > > > achieve the same with span/event. > > > > > > > > > > > > > > > > > > > > Just my $0.01 > > > > > > > > > > > > > > > > > > > > Regards > > > > > > > > > > JB > > > > > > > > > > > > > > > > > > > > On Sat, Oct 25, 2025 at 10:42 AM Adnan Hemani > > > > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > > > +1 to all of Alex’s AIs with Dmitri’s suggested > changes as > > > > > well. > > > > > > > Great > > > > > > > > > find, Dmitri! > > > > > > > > > > > > > > > > > > > > > > I’m still debating with myself on whether we need to > store > > > > the > > > > > > > > > `x-request-id` field as part of the Events persistence. > Can we > > > > > think > > > > > > > of a > > > > > > > > > good use case where this is more helpful to the user than > the > > > > OTel > > > > > > > > > Trace/Span IDs? I am making the assumption here that those > are > > > > > still > > > > > > > being > > > > > > > > > returned back to the client. > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > Adnan Hemani > > > > > > > > > > > > > > > > > > > > > > > On Oct 24, 2025, at 9:12 AM, Alexandre Dutra < > > > > > [email protected]> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > Hi Dmitri, > > > > > > > > > > > > > > > > > > > > > > > > Yes, I think your suggestion to use just one field > > > > > > > w3c_trace_context > > > > > > > > > > > > makes more sense than two fields (span_id and > trace_id). > > > > > > > > > > > > > > > > > > > > > > > > With that, I also think that we are slowly drifting > into > > > > > > > > > > > > implementation considerations; let's get consensus > on the > > > > > general > > > > > > > > > > > > design first, and we can certainly fine-tune the > actual > > > > Java > > > > > > > methods > > > > > > > > > > > > and SQL table columns in the future PR. WDYT? > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Alex > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 24, 2025 at 5:14 PM Dmitri Bourlatchkov < > > > > > > > > > [email protected]> wrote: > > > > > > > > > > > >> > > > > > > > > > > > >> Hi All, > > > > > > > > > > > >> > > > > > > > > > > > >> Many thanks for the background info, Adnan! > > > > > > > > > > > >> > > > > > > > > > > > >> +1 to action items proposed by Alex! > > > > > > > > > > > >> > > > > > > > > > > > >> Re: (3) Can we abstract request/trace info into a > > > separate > > > > > > > object > > > > > > > > > without > > > > > > > > > > > >> exposing those accessors on the Event class > directly? > > > OTel > > > > > > > defines > > > > > > > > > > > >> trace/span concepts, but in request ID is a bit > foreign > > > to > > > > > OTel. > > > > > > > > > Having > > > > > > > > > > > >> tracing / request ID isolated in java could help > with > > > > > > > maintaining > > > > > > > > > it and > > > > > > > > > > > >> potentially supporting other (custom) tracing > methods. > > > > > > > > > > > >> > > > > > > > > > > > >> Re: (4) I'd like to propose storing OTel correlation > > > data > > > > > in the > > > > > > > > > form of a > > > > > > > > > > > >> standard context propagation string, e.g. W3C > > > > trace-context > > > > > [1] > > > > > > > > > (same value > > > > > > > > > > > >> as its HTTP header), so the column could be called > > > > > > > > > w3c_trace_context or > > > > > > > > > > > >> simply trace_context. > > > > > > > > > > > >> > > > > > > > > > > > >> Open question: do we need to write a separate, > > > individual > > > > > trace > > > > > > > ID > > > > > > > > > field in > > > > > > > > > > > >> SQL? I suppose it is not very useful since > correlating > > > it > > > > to > > > > > > > other > > > > > > > > > trace > > > > > > > > > > > >> data already requires understanding OTel context > > > > propagation > > > > > > > and a > > > > > > > > > query > > > > > > > > > > > >> against trace_context can still be made using > > > > > string-matching > > > > > > > > > clauses. We > > > > > > > > > > > >> could probably (additionally) store it in the > request_id > > > > > column > > > > > > > if > > > > > > > > > the > > > > > > > > > > > >> Polaris-specific request ID header is not set. > > > > > > > > > > > >> > > > > > > > > > > > >> As for span ID, I do not really see a use case for > > > > > persisting it > > > > > > > > > > > >> individually. It is very specific to OTel trace data > > > > > > > construction. > > > > > > > > > > > >> > > > > > > > > > > > >> Actually, using the W3C trace context [1] encoding > > > > probably > > > > > > > makes > > > > > > > > > sense in > > > > > > > > > > > >> the java event representation too. Interested > callers > > > can > > > > > easily > > > > > > > > > decode > > > > > > > > > > > >> this information since the format is well-defined. > As a > > > > side > > > > > > > > > benefit, this > > > > > > > > > > > >> opens opportunities for downstream event consumers > to > > > > > connect > > > > > > > > > (propagate > > > > > > > > > > > >> context) to traces that produced events based on the > > > event > > > > > data > > > > > > > > > itself, > > > > > > > > > > > >> without relying on the intermediate frameworks. > This may > > > > be > > > > > > > > > desirable since > > > > > > > > > > > >> the current Polaris event persistence impl. writes > > > events > > > > in > > > > > > > > > batches, so > > > > > > > > > > > >> the association to individual requests that produced > > > those > > > > > > > events > > > > > > > > > is lost. > > > > > > > > > > > >> Whether to perform this kind of context propagation > will > > > > be > > > > > at > > > > > > > the > > > > > > > > > > > >> discretion of the event consumer, of course > (outside of > > > > > Polaris > > > > > > > > > code). > > > > > > > > > > > >> > > > > > > > > > > > >> [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://www.google.com/url?q=https://www.w3.org/TR/trace-context/&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2fn0lRsTx-f9r4PCz8wmJK > > > > > > > > > > > >> > > > > > > > > > > > >> WDYT? > > > > > > > > > > > >> > > > > > > > > > > > >> Thanks, > > > > > > > > > > > >> Dmitri. > > > > > > > > > > > >> > > > > > > > > > > > >> On Fri, Oct 24, 2025 at 7:18 AM Alexandre Dutra < > > > > > > > [email protected]> > > > > > > > > > wrote: > > > > > > > > > > > >> > > > > > > > > > > > >>> Hi all, > > > > > > > > > > > >>> > > > > > > > > > > > >>> Thank you for chiming in; the context around > request > > > IDs > > > > > is now > > > > > > > > > clear. > > > > > > > > > > > >>> > > > > > > > > > > > >>> Trying to summarize this thread into actionable > items, > > > > > here's > > > > > > > what > > > > > > > > > I > > > > > > > > > > > >>> propose: > > > > > > > > > > > >>> > > > > > > > > > > > >>> 1. Restore the original functionality for request > IDs. > > > > > > > > > > > >>> - Change the default header name back to > > > x-request-id > > > > > > > (despite > > > > > > > > > the > > > > > > > > > > > >>> x- prefix being deprecated), but keep it > configurable > > > as > > > > > today. > > > > > > > > > > > >>> 2. Remove RequestIdGenerator and related > functionality. > > > > > > > > > > > >>> - Do not generate a request ID if the REST > client > > > > > doesn't > > > > > > > > > provide one. > > > > > > > > > > > >>> 3. Update PolarisEvent: > > > > > > > > > > > >>> - Expose new requestId(), traceId(), spanId() > > > methods, > > > > > all > > > > > > > > > nullable. > > > > > > > > > > > >>> - This would align with the emerging consensus > > > around > > > > > > > including > > > > > > > > > > > >>> contextual information in PolarisEvent [1]. > > > > > > > > > > > >>> 4. Update events table SQL schema: > > > > > > > > > > > >>> - Insert the client-provided request ID into the > > > > > request_id > > > > > > > > > > > >>> column; otherwise, insert null. > > > > > > > > > > > >>> - Add two new nullable columns, trace_id and > > > span_id, > > > > > and > > > > > > > > > populate > > > > > > > > > > > >>> them if OTel is enabled. > > > > > > > > > > > >>> > > > > > > > > > > > >>> From our discussions, I think it's important to not > > > > > conflate > > > > > > > OTel > > > > > > > > > > > >>> tracing fields with Envoy tracing fields, which is > why > > > I > > > > > > > suggest we > > > > > > > > > > > >>> use separate fields / columns for them. > > > > > > > > > > > >>> > > > > > > > > > > > >>> Would the above plan work for everyone? > > > > > > > > > > > >>> > > > > > > > > > > > >>> Thanks, > > > > > > > > > > > >>> Alex > > > > > > > > > > > >>> > > > > > > > > > > > >>> [1]: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://www.google.com/url?q=https://lists.apache.org/thread/rl5cpcft16sn5n00mfkmx9ldn3gsqtfy&source=gmail-imap&ust=1761927167000000&usg=AOvVaw02wPvb0qxRzYAKEP0h8l9T > > > > > > > > > > > >>> > > > > > > > > > > > >>> > > > > > > > > > > > >>> On Fri, Oct 24, 2025 at 9:33 AM Adnan Hemani > > > > > > > > > > > >>> <[email protected]> wrote: > > > > > > > > > > > >>>> > > > > > > > > > > > >>>> Hi all, > > > > > > > > > > > >>>> > > > > > > > > > > > >>>> Thanks to Alex for starting this thread - because > of > > > > > this, I’m > > > > > > > > > just > > > > > > > > > > > >>> coming to the realization that OTel Trace and Span > IDs > > > > are > > > > > > > coming > > > > > > > > > built-in > > > > > > > > > > > >>> with Quarkus and my previous work to generate a > > > > > > > > > Request/Correlation ID is > > > > > > > > > > > >>> likely not needed as a result. My original > motivation > > > for > > > > > > > > > generation of a > > > > > > > > > > > >>> Request/Correlation ID was to ensure that any > client > > > can > > > > > > > uniquely > > > > > > > > > identify > > > > > > > > > > > >>> a request made to Polaris, which would be > especially > > > > > useful for > > > > > > > > > debugging > > > > > > > > > > > >>> failing requests or identifying call patterns. > > > > > > > > > > > >>>> > > > > > > > > > > > >>>> As a result, I’m a +1 on Michael’s opinion: we > should > > > > > remove > > > > > > > the > > > > > > > > > > > >>> Request/Correlation ID generation and always use > the > > > OTel > > > > > > > > > trace/span IDs > > > > > > > > > > > >>> (which come for free with Quarkus) instead for the > > > > > Correlation > > > > > > > ID > > > > > > > > > unless a > > > > > > > > > > > >>> valid header is present, which would take over as > the > > > > > > > Correlation > > > > > > > > > ID > > > > > > > > > > > >>> instead. > > > > > > > > > > > >>>> > > > > > > > > > > > >>>> — > > > > > > > > > > > >>>> > > > > > > > > > > > >>>> To answer Dmitri’s question re Polaris Events: The > > > > > intended > > > > > > > use > > > > > > > > > case is > > > > > > > > > > > >>> to provide some sort of correlation between events > that > > > > > have > > > > > > > > > occurred as > > > > > > > > > > > >>> part of the same request. For example, if a user > makes > > > an > > > > > > > > > CommitTransaction > > > > > > > > > > > >>> request, it would be helpful to see all UpdateTable > > > calls > > > > > that > > > > > > > > > were made as > > > > > > > > > > > >>> part of that one user request. > > > > > > > > > > > >>>> > > > > > > > > > > > >>>> Best, > > > > > > > > > > > >>>> Adnan Hemani > > > > > > > > > > > >>>> > > > > > > > > > > > >>>>> On Oct 23, 2025, at 12:15 PM, Dmitri > Bourlatchkov < > > > > > > > > > [email protected]> > > > > > > > > > > > >>> wrote: > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>>> Hi Michael, > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>>> Logging x-request-id headers makes sense. > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>>> Just to confirm: if / when we restore that, > Polaris > > > > will > > > > > NOT > > > > > > > > > generate > > > > > > > > > > > >>> new > > > > > > > > > > > >>>>> IDs in case the header is not present in the > request, > > > > > > > correct? > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>>> I believe x-request-id can co-exist with OTel. > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>>> What about adding request IDs to events [1][2]? > > > What's > > > > > the > > > > > > > > > intended use > > > > > > > > > > > >>>>> case for that? Could you share some context here > too? > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>>> Side note: I proposed [2877] flagging event > > > persistence > > > > > as > > > > > > > > > "beta" in > > > > > > > > > > > >>>>> 1.2.0... This discussion adds another point > towards > > > > > that, I > > > > > > > > > think. > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>>> [1] > > > > > > > > > > > >>>>> > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%2523L97%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1WfUaXLp6z_M87iAXEqSUw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Pioys4ROm8vYM_mdx4ygH > > > > > > > > > > > >>>>> [2] > > > > > > > > > > > >>>>> > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%2523L66%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw12-7e3ahm2sLSkLSNqTecm&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1PGW6uFwtUN8F1dOlJwrpm > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>>> [2877] > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2877%2523discussion_r2456300613%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw3TuYbkzwnLx3QEVIM8oDda&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2Tfk7wAM1MaB5z9Dvw5X5H > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>>> Thanks, > > > > > > > > > > > >>>>> Dmitri. > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>>> On Thu, Oct 23, 2025 at 2:23 PM Michael Collado < > > > > > > > > > > > >>> [email protected]> > > > > > > > > > > > >>>>> wrote: > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>>>> Hey Dmitri > > > > > > > > > > > >>>>>> > > > > > > > > > > > >>>>>> The generating a request id is new code that was > > > added > > > > > > > after the > > > > > > > > > > > >>> original > > > > > > > > > > > >>>>>> x-request-id support. You can see the state > from ~1 > > > > year > > > > > > > ago, we > > > > > > > > > > > >>> hard-coded > > > > > > > > > > > >>>>>> request_id as the header we used for the MDC - > > > > > > > > > > > >>>>>> > > > > > > > > > > > >>>>>> > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%2523L415-L416%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw35Q1A_2avAiSYlYVAnZxBb&source=gmail-imap&ust=1761927167000000&usg=AOvVaw34DksOw7DSHJu8PfQQ5bT6 > > > > > > > > > > > >>>>>> . At some point, it was changed to be > configurable, > > > > > then the > > > > > > > > > > > >>>>>> ContextResolverFilter filter was > > > refactored/eliminated > > > > > and > > > > > > > the > > > > > > > > > > > >>>>>> RequestIdFilter took its responsibility, but > lost > > > some > > > > > of > > > > > > > its > > > > > > > > > original > > > > > > > > > > > >>>>>> functionality. > > > > > > > > > > > >>>>>> > > > > > > > > > > > >>>>>> My personal opinion is that restoring support > for > > > the > > > > > > > > > x-request-id > > > > > > > > > > > >>> header > > > > > > > > > > > >>>>>> is something that we should do, but if the > header > > > > isn't > > > > > > > present, > > > > > > > > > > > >>> falling > > > > > > > > > > > >>>>>> back on simply using OTel trace ids is good > enough > > > > > (better, > > > > > > > > > even) than > > > > > > > > > > > >>>>>> generating another random request id. > > > > > > > > > > > >>>>>> > > > > > > > > > > > >>>>>> Mike > > > > > > > > > > > >>>>>> > > > > > > > > > > > >>>>>> On Thu, Oct 23, 2025 at 10:47 AM Dmitri > > > Bourlatchkov < > > > > > > > > > > > >>> [email protected]> > > > > > > > > > > > >>>>>> wrote: > > > > > > > > > > > >>>>>> > > > > > > > > > > > >>>>>>> Hi Michael, > > > > > > > > > > > >>>>>>> > > > > > > > > > > > >>>>>>> Thanks for the info! > > > > > > > > > > > >>>>>>> > > > > > > > > > > > >>>>>>> Working with Envoy's tracing headers makes > sense to > > > > me. > > > > > > > > > However, I > > > > > > > > > > > >>>>>> wonder: > > > > > > > > > > > >>>>>>> why would Polaris need to generate a new > request ID > > > > > inside > > > > > > > its > > > > > > > > > > > >>> code?.. > > > > > > > > > > > >>>>>>> and return it in response headers? > > > > > > > > > > > >>>>>>> > > > > > > > > > > > >>>>>>> How important is it to propagate this ID to > Polaris > > > > > Events? > > > > > > > > > > > >>>>>>> > > > > > > > > > > > >>>>>>> I'm just trying to understand the full context > :) > > > > > > > > > > > >>>>>>> > > > > > > > > > > > >>>>>>> Thanks, > > > > > > > > > > > >>>>>>> Dmitri. > > > > > > > > > > > >>>>>>> > > > > > > > > > > > >>>>>>> On Thu, Oct 23, 2025 at 1:29 PM Michael > Collado < > > > > > > > > > > > >>> [email protected]> > > > > > > > > > > > >>>>>>> wrote: > > > > > > > > > > > >>>>>>> > > > > > > > > > > > >>>>>>>> I think the original intention for this > requestId > > > > > field > > > > > > > was to > > > > > > > > > > > >>> support > > > > > > > > > > > >>>>>>>> request propagation from load balancers, like > > > Envoy > > > > ( > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > >>>>>>> > > > > > > > > > > > >>>>>> > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1RnuM8mViV-j7jvuxq74Aw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw3LNqtilZ2OoJfE7yEwraZa > > > > > > > > > > > >>>>>>>> ), which is distinct from OTEL. Don't ask me > why > > > the > > > > > > > default > > > > > > > > > > > >>>>>>>> is Polaris-Request-Id - I think it was > originally > > > a > > > > > custom > > > > > > > > > thing, > > > > > > > > > > > >>> but > > > > > > > > > > > >>>>>>> then > > > > > > > > > > > >>>>>>>> we changed to integrate with existing > conventions. > > > > > > > > > Unfortunately, > > > > > > > > > > > >>>>>> looking > > > > > > > > > > > >>>>>>>> through the code, I think that the actual > > > functional > > > > > > > plumbing > > > > > > > > > has > > > > > > > > > > > >>> been > > > > > > > > > > > >>>>>>> lost > > > > > > > > > > > >>>>>>>> in the course of multiple refactors around the > > > call > > > > > > > context > > > > > > > > > and > > > > > > > > > > > >>>>>>> resolver. I > > > > > > > > > > > >>>>>>>> don't see references to that property or the > > > > > underlying > > > > > > > > > header. > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > >>>>>>>> Support for the unofficial x-request-id header > > > feels > > > > > like > > > > > > > > > something > > > > > > > > > > > >>> we > > > > > > > > > > > >>>>>>>> should definitely keep, especially when > Polaris is > > > > one > > > > > > > > > service in a > > > > > > > > > > > >>>>>> mesh > > > > > > > > > > > >>>>>>> of > > > > > > > > > > > >>>>>>>> services that maybe don't have OTel > integration. > > > > I'm a > > > > > > > fan of > > > > > > > > > the > > > > > > > > > > > >>> OTel > > > > > > > > > > > >>>>>>>> standard, but it's not entirely ubiquitous and > > > there > > > > > are > > > > > > > many > > > > > > > > > > > >>>>>> middleware > > > > > > > > > > > >>>>>>>> layers that know how to forward on the > > > x-request-id > > > > > > > header. > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > >>>>>>>> Mike > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > >>>>>>>> On Thu, Oct 23, 2025 at 3:00 AM Robert Stupp < > > > > > > > [email protected]> > > > > > > > > > > > >>> wrote: > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > >>>>>>>>> Yes, we should aim for interoperability with > the > > > > > existing > > > > > > > > > de-facto > > > > > > > > > > > >>>>>>>>> standard OTel and make it easy for users to > > > > integrate > > > > > > > into > > > > > > > > > their > > > > > > > > > > > >>>>>>>>> observability platforms. > > > > > > > > > > > >>>>>>>>> > > > > > > > > > > > >>>>>>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri > > > > Bourlatchkov < > > > > > > > > > > > >>>>>> [email protected]> > > > > > > > > > > > >>>>>>>>> wrote: > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > >>>>>>>>>> Hi Alex and All, > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > >>>>>>>>>> I certainly support the idea of following > OTel > > > > > > > standards for > > > > > > > > > > > >>>>>>> achieving > > > > > > > > > > > >>>>>>>>>> "correlation" wrt Polaris requests and/or > > > events. > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > >>>>>>>>>> As to what form the correlation data should > > > take, > > > > I > > > > > > > believe > > > > > > > > > it is > > > > > > > > > > > >>>>>>>>>> conceptually what the OTel "context" > represents. > > > > > So, I > > > > > > > > > believe it > > > > > > > > > > > >>>>>>> makes > > > > > > > > > > > >>>>>>>>>> sense for Polaris to support standard > context > > > > > > > propagators > > > > > > > > > at the > > > > > > > > > > > >>>>>> API > > > > > > > > > > > >>>>>>>>> layer. > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > >>>>>>>>>> If the incoming request has OTel context > > > > > information, > > > > > > > then > > > > > > > > > > > >>>>>> returning > > > > > > > > > > > >>>>>>>> any > > > > > > > > > > > >>>>>>>>>> other "correlation" data in the response is > > > > > redundant, I > > > > > > > > > think. > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > >>>>>>>>>> If the incoming request does not have OTel > > > context > > > > > info, > > > > > > > > > what is > > > > > > > > > > > >>>>>> the > > > > > > > > > > > >>>>>>>>>> purpose of generating a Polaris-specific > > > > > "correlation > > > > > > > ID"? > > > > > > > > > How is > > > > > > > > > > > >>>>>> it > > > > > > > > > > > >>>>>>>>>> envisioned to be used? > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > >>>>>>>>>> If the intention is to correlate a Polaris > > > > response > > > > > > > > > (operation) > > > > > > > > > > > >>>>>> with > > > > > > > > > > > >>>>>>>>> events > > > > > > > > > > > >>>>>>>>>> that might have resulted from its > execution, I > > > > > believe a > > > > > > > > > more > > > > > > > > > > > >>>>>> robust > > > > > > > > > > > >>>>>>>>>> approach would be to propagate the OTel > trace > > > info > > > > > > > > > (starting a new > > > > > > > > > > > >>>>>>>> trace > > > > > > > > > > > >>>>>>>>> if > > > > > > > > > > > >>>>>>>>>> necessary) into event data. Then, Polaris > can > > > also > > > > > > > return > > > > > > > > > the > > > > > > > > > > > >>> trace > > > > > > > > > > > >>>>>>>>> context > > > > > > > > > > > >>>>>>>>>> in the API response (top span). It's a bit > > > awkward > > > > > from > > > > > > > the > > > > > > > > > OTel > > > > > > > > > > > >>>>>>>>>> perspective, but might be an option for > > > supporting > > > > > > > custom > > > > > > > > > > > >>>>>>> correlators. > > > > > > > > > > > >>>>>>>> It > > > > > > > > > > > >>>>>>>>>> could be covered by a feature flag. The > header > > > > name > > > > > > > could be > > > > > > > > > > > >>>>>>>>>> "polaris-traceparent" for W3C Trace Context. > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > >>>>>>>>>> Custom correlation code will be able to > extract > > > > the > > > > > > > trace > > > > > > > > > ID from > > > > > > > > > > > >>>>>> the > > > > > > > > > > > >>>>>>>>>> response and from events and find related > data. > > > > > > > Granted, it > > > > > > > > > will > > > > > > > > > > > >>>>>>>> require > > > > > > > > > > > >>>>>>>>> a > > > > > > > > > > > >>>>>>>>>> bit more effort for the custom code to > decode > > > > trace > > > > > IDs > > > > > > > > > from the > > > > > > > > > > > >>>>>> OTel > > > > > > > > > > > >>>>>>>>>> context, but the format is standard and not > > > > > complex. The > > > > > > > > > benefit > > > > > > > > > > > >>>>>> for > > > > > > > > > > > >>>>>>>>>> Polaris, though, is that it can easily > integrate > > > > > with > > > > > > > > > > > >>>>>> OTel-compatible > > > > > > > > > > > >>>>>>>>>> observability platforms regardless of > whether > > > any > > > > > > > particular > > > > > > > > > > > >>>>>>> deployment > > > > > > > > > > > >>>>>>>>>> uses custom correlators or not. > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > >>>>>>>>>> WDYT? > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > >>>>>>>>>> Thanks, > > > > > > > > > > > >>>>>>>>>> Dmitri. > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > >>>>>>>>>> On Wed, Oct 22, 2025 at 6:03 AM Alexandre > Dutra > > > < > > > > > > > > > > > >>> [email protected] > > > > > > > > > > > >>>>>>> > > > > > > > > > > > >>>>>>>>> wrote: > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > >>>>>>>>>>> Hi all, > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > >>>>>>>>>>> Today, Polaris has the notion of "request > ID", > > > > but > > > > > its > > > > > > > > > purpose is > > > > > > > > > > > >>>>>>> not > > > > > > > > > > > >>>>>>>>>>> entirely clear. It seems to serve as an > > > > > observability > > > > > > > > > feature to > > > > > > > > > > > >>>>>>>>>>> facilitate correlation. A pending PR aims > to > > > > > rename it > > > > > > > to > > > > > > > > > > > >>>>>>>> "correlation > > > > > > > > > > > >>>>>>>>>>> ID" for better alignment with industry > > > standards > > > > > [1]. > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > >>>>>>>>>>> However, this PR has brought to light > overlaps > > > > with > > > > > > > core > > > > > > > > > > > >>>>>> telemetry > > > > > > > > > > > >>>>>>>>>>> features: when OpenTelemetry (OTel) is > enabled > > > in > > > > > > > Polaris, > > > > > > > > > each > > > > > > > > > > > >>>>>>>>>>> request already has a trace ID and span ID, > > > > making > > > > > a > > > > > > > > > separate > > > > > > > > > > > >>>>>>>>>>> correlation ID redundant. > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > >>>>>>>>>>> Moreover, using the OTel trace ID and span > ID > > > in > > > > > > > Polaris > > > > > > > > > events, > > > > > > > > > > > >>>>>>>>>>> rather than the generated correlation ID, > would > > > > > > > > > significantly > > > > > > > > > > > >>>>>>>> simplify > > > > > > > > > > > >>>>>>>>>>> correlation of events with other traces. > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > >>>>>>>>>>> Therefore, I propose the following changes: > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > >>>>>>>>>>> 1. If OTel is enabled, use the trace ID and > > > span > > > > > ID as > > > > > > > the > > > > > > > > > > > >>>>>>>> correlation > > > > > > > > > > > >>>>>>>>>>> ID for the request, instead of generating a > > > > random > > > > > > > > > correlation > > > > > > > > > > > >>>>>> ID. > > > > > > > > > > > >>>>>>>>>>> 2. Otherwise, if a (Polaris-specific) > > > correlation > > > > > ID > > > > > > > > > header is > > > > > > > > > > > >>>>>>>> present > > > > > > > > > > > >>>>>>>>>>> in the request, use it. > > > > > > > > > > > >>>>>>>>>>> 3. If neither of the above conditions is > met, > > > > > generate > > > > > > > a > > > > > > > > > random > > > > > > > > > > > >>>>>>>>>>> correlation ID. > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > >>>>>>>>>>> I am somewhat undecided on the best > approach > > > > when a > > > > > > > > > correlation > > > > > > > > > > > >>>>>> ID > > > > > > > > > > > >>>>>>>>>>> header is present in the request. However, > I > > > > > believe it > > > > > > > > > would be > > > > > > > > > > > >>>>>>> more > > > > > > > > > > > >>>>>>>>>>> sensible to disregard it if OTel is > enabled, as > > > > > OTel > > > > > > > > > offers a > > > > > > > > > > > >>>>>> more > > > > > > > > > > > >>>>>>>>>>> robust solution for client-to-server trace > > > > > propagation, > > > > > > > > > e.g. W3C > > > > > > > > > > > >>>>>>>> Trace > > > > > > > > > > > >>>>>>>>>>> Context propagation [2]. > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > >>>>>>>>>>> Please share your thoughts! > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > >>>>>>>>>>> Thanks, > > > > > > > > > > > >>>>>>>>>>> Alex > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > >>>>>>>>>>> [1]: > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2757%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1-kAWfEk4tmsEg0q0GZBCn&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Oe-25vtt4gLVSMFMtSVNg > > > > > > > > > > > >>>>>>>>>>> [2]: > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.w3.org/TR/trace-context%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw22nMyOS7pbJ69XrBo5kHQS&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1TRdkzABc_7U-_KZ1MZ59v > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > >>>>>>>>> > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > >>>>>>> > > > > > > > > > > > >>>>>> > > > > > > > > > > > >>>> > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
