Hi Dmitri, > I believe we still need to clarify / confirm whether returning any request / > correlation ID in _response_ headers is required.
Indeed, that specific item was still pending. FYI I'm going to *remove* the echoing of the request ID in the response headers, unless someone objects. > it could be useful to embed some type data into the value to help with > interpretation I'm not sure we need this at this moment. I'd suggest that we leave this for further improvements in order to not over-specify the requirements. Given that the events feature is beta, this shouldn't be a problem. Thanks, Alex On Thu, Oct 30, 2025 at 5:22 PM Dmitri Bourlatchkov <[email protected]> wrote: > > Thanks for the nice summary, Alex! > > To All: > > I'm not sure an INDEX on correlation_id is that useful... Polaris itself > does not use that index, right? Do we expect end users to query events > directly from the database? This will work only with RDBMS, but probably > not with the proposed NoSQL persistence. > > I believe we still need to clarify / confirm whether returning any request > / correlation ID in _response_ headers is required. > > From my POV, I do not see a use case for it, but I wonder what other people > think / need. > > As for persisting correlation ID type, I agree that we do not need a > separate column for that. Still, it could be useful to embed some type data > into the value to help with interpretation, specifically for telemetry use > cases (both x-request-id and OTel). For example, we could > persist "traceparent:00-12345-...". Then, the reader would be able to > identify this as information following the W3C spec [1], parse it and link > to other traces (if desired). That processing is optional and at the > discretion of the event data reader, but making the value no-so-opaque > upfront can enable a lot of interesting options for users without much > extra overhead in Polaris. WDYT? > > [1] https://www.w3.org/TR/2021/REC-trace-context-1-20211123/ > > Thanks, > Dmitri. > > On Thu, Oct 30, 2025 at 6:04 AM Alexandre Dutra <[email protected]> wrote: > > > Hi all, > > > > > we would only persist one or the other, not both. > > > > My apologies, I confess that this wasn't my understanding of the desired > > design. > > > > Let's try to gather the requirements again: > > > > 1. Only one correlation ID is enough > > 2. The correlation ID is an opaque string > > 3. The main use case is to find events with matching correlation IDs > > 4. The only query pattern is by exact match > > > > Regarding *where* to store the correlation ID, and expanding on what > > Dmitri said: the requirement to find matching records by correlation > > ID clearly suggests a separate column, and with an INDEX on it. > > Storing it in additional_properties will be inefficient. > > > > If we all agree on the above, here is what I propose: > > > > 1. Rename the existing column request_id to correlation_id; > > 2. Populate from OTel context if available; the exact syntax is > > implementation-specific; > > 3. Otherwise, populate from X-Request-Id header if available; > > 4. Otherwise, don't populate. > > 5. Create an INDEX on correlation_id. > > > > You'll notice that we don't store the ID type, but that doesn't seem > > necessary. > > > > What do you all think? > > > > Thanks, > > Alex > > > > > > > > > > > > > > On Wed, Oct 29, 2025 at 8:58 PM Dmitri Bourlatchkov <[email protected]> > > wrote: > > > > > > Hi Yufei, > > > > > > [...] tying a BeforeUpdateTable event to the corresponding > > AfterUpdateTable > > > event. > > > > > > > > > It looks like this calls for a strong and always present ID in > > events.This > > > ID is obviously distincts from OTel context information and should be > > > persisted separately, IMHO. > > > > > > I believe there's still an open question whether the request ID in the > > > events should be populated from the x-request-id in REST API headers. > > > > > > Yufei: Is that required in your use cases? > > > > > > Thanks, > > > Dmitri. > > > > > > > > > > > > On Wed, Oct 29, 2025 at 1:05 PM Yufei Gu <[email protected]> wrote: > > > > > > > I think the first question is whether any downstream use cases need to > > > > interpret the semantics of a request ID or OTel context. As I > > mentioned, > > > > the original purpose of a request ID was event correlation, e.g., > > tying a > > > > BeforeUpdateTable event to the corresponding AfterUpdateTable event. If > > > > that’s the only requirement, an opaque correlation ID is good enough. > > We > > > > could change the field name(request-id) if needed. > > > > > > > > If we do have use cases that depend on the meaning of the request ID or > > > > OTel context, we can store them in additional_properties ( > > > > > > > > > > https://github.com/polaris-catalog/polaris/blob/main/persistence/relational-jdbc/src/main/resources/postgres/schema-v3.sql#L134-L134 > > > > ). > > > > This is more flexible because it allows us to: > > > > 1. Record a key that indicates the format (e.g., x-request-id vs. > > > > otel-context). > > > > 2. Preserve both when Polaris receives both headers. > > > > 3. Add future tracing fields without a schema change. > > > > > > > > Yufei > > > > > > > > > > > > On Wed, Oct 29, 2025 at 9:20 AM Dmitri Bourlatchkov <[email protected]> > > > > wrote: > > > > > > > > > Hi All, > > > > > > > > > > From my POV the use cases for x-request-id and OTel are slightly > > > > different. > > > > > OTel is an observability use case where cross-linking of various > > pieces > > > > of > > > > > data is desired but not required in all cases (data loss is > > acceptable). > > > > > The x-request-id use case appears to have some very specific custom > > > > > applications without any obvious relationship to OTel. > > > > > > > > > > Given the current state of this discussion, I think it makes sense to > > > > > handle and persist them separately. > > > > > > > > > > However, if we intend x-request-id to act as an observability aid > > > > > alternative to OTel, then it might be preferable to have a config > > flag > > > > > indicating which one of them should be added to persistent event > > data. In > > > > > that case we should reuse the same column for them (and possibly add > > a > > > > > "type" prefix, e.g. "x-request-id:ABCDEF" or > > "traceparent:00-12345...") > > > > > > > > > > WDYT? > > > > > > > > > > Thanks, > > > > > Dmitri. > > > > > > > > > > > > > > > On Wed, Oct 29, 2025 at 9:38 AM Alexandre Dutra <[email protected]> > > > > wrote: > > > > > > > > > > > Hi Yufei, > > > > > > > > > > > > > I’d suggest persisting a single opaque correlation ID instead > > > > > > > > > > > > The problem I see with this is that readers cannot know the syntax > > of > > > > > > the opaque id, and will have to "probe" using some heuristics, > > e.g. if > > > > > > the id matches the OTel trace context syntax, asume it's OTel, > > > > > > otherwise... assume it's something else. > > > > > > > > > > > > That's not unfeasible, but I wonder if it isn't better to store > > the 2 > > > > > > ids separately. > > > > > > > > > > > > Thanks, > > > > > > Alex > > > > > > > > > > > > On Tue, Oct 28, 2025 at 7:14 PM Yufei Gu <[email protected]> > > wrote: > > > > > > > > > > > > > > Thanks Alex for raising this discussion. Thanks everyone for > > chiming > > > > > in. > > > > > > > The plan looks good to me overall. > > > > > > > > > > > > > > > Update PolarisEvent, expose request ID if available, expose > > OTel > > > > > > context > > > > > > > if available. > > > > > > > > > > > > > > https://github.com/apache/polaris/pull/2914 persists both > > request id > > > > > and > > > > > > > otel context. > > > > > > > > > > > > > > Do we actually need to persist both? > > > > > > > From the original discussion, the intent of a request ID was to > > > > > correlate > > > > > > > multiple events generated by the same request, for example, > > linking a > > > > > > > BeforeUpdateTable and an AfterUpdateTable event. In that sense, > > what > > > > we > > > > > > > really need is a single opaque identifier that ties related > > events > > > > > > > together, regardless of whether it comes from a request ID or an > > OTel > > > > > > > context. > > > > > > > > > > > > > > If we persist both fields (requestId and otelContext), it can > > lead to > > > > > > > consumer confusion, since downstream systems would need to decide > > > > which > > > > > > one > > > > > > > to rely on, potentially causing inconsistent event correlation. > > > > Unless > > > > > > > there are concrete use cases where event consumers need both, I’d > > > > > suggest > > > > > > > persisting a single opaque correlation ID instead. This keeps the > > > > > schema > > > > > > > simpler and ensures consistent event linking. > > > > > > > > > > > > > > Yufei > > > > > > > > > > > > > > > > > > > > > On Tue, Oct 28, 2025 at 10:08 AM Alexandre Dutra < > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > Here is the PR: https://github.com/apache/polaris/pull/2914 > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Alex > > > > > > > > > > > > > > > > On Sun, Oct 26, 2025 at 5:49 PM Jean-Baptiste Onofré < > > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > Hi > > > > > > > > > > > > > > > > > > Yeah it looks good to me and I agree we have a consensus. > > > > > > > > > > > > > > > > > > Thanks ! > > > > > > > > > Regards > > > > > > > > > JB > > > > > > > > > > > > > > > > > > Le dim. 26 oct. 2025 à 14:29, Alexandre Dutra < > > [email protected] > > > > > > > > > > a > > > > > > > > écrit : > > > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > Thanks for the good discussion so far. I think we have a > > broad > > > > > > > > consensus. > > > > > > > > > > > > > > > > > > > > Unless anybody disagrees I am going to move forward with > > the > > > > the > > > > > > > > > > following revised plan, incorporating the latest feedback: > > > > > > > > > > > > > > > > > > > > 1. Restore the original functionality for request IDs, > > change > > > > the > > > > > > > > > > default header name back to x-request-id > > > > > > > > > > > > > > > > > > > > 2. Remove RequestIdGenerator and related functionality. > > > > > > > > > > > > > > > > > > > > 3. Update PolarisEvent, expose request ID if available, > > expose > > > > > OTel > > > > > > > > > > context if available. > > > > > > > > > > > > > > > > > > > > 4. Update events table SQL schema: insert request ID if > > > > > available, > > > > > > > > > > insert OTel context if available. > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > Alex > > > > > > > > > > > > > > > > > > > > On Sat, Oct 25, 2025 at 11:12 AM Jean-Baptiste Onofré < > > > > > > [email protected] > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > Hi > > > > > > > > > > > > > > > > > > > > > > I think it's better to use w3c_trace_context (better > > than two > > > > > > > > fields). > > > > > > > > > > > > > > > > > > > > > > Regarding x-request-id, I think it's for a different > > purpose, > > > > > > > > > > > specifically for persistence. Personally, I think we can > > > > > achieve > > > > > > the > > > > > > > > > > > same with span and events. > > > > > > > > > > > If we want to "distinguish" the two layers, it makes > > sense to > > > > > > have > > > > > > > > > > > x-request-id (not sure it's super helpful). Else, I > > think we > > > > > can > > > > > > > > > > > achieve the same with span/event. > > > > > > > > > > > > > > > > > > > > > > Just my $0.01 > > > > > > > > > > > > > > > > > > > > > > Regards > > > > > > > > > > > JB > > > > > > > > > > > > > > > > > > > > > > On Sat, Oct 25, 2025 at 10:42 AM Adnan Hemani > > > > > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > > > > > +1 to all of Alex’s AIs with Dmitri’s suggested > > changes as > > > > > > well. > > > > > > > > Great > > > > > > > > > > find, Dmitri! > > > > > > > > > > > > > > > > > > > > > > > > I’m still debating with myself on whether we need to > > store > > > > > the > > > > > > > > > > `x-request-id` field as part of the Events persistence. > > Can we > > > > > > think > > > > > > > > of a > > > > > > > > > > good use case where this is more helpful to the user than > > the > > > > > OTel > > > > > > > > > > Trace/Span IDs? I am making the assumption here that those > > are > > > > > > still > > > > > > > > being > > > > > > > > > > returned back to the client. > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > Adnan Hemani > > > > > > > > > > > > > > > > > > > > > > > > > On Oct 24, 2025, at 9:12 AM, Alexandre Dutra < > > > > > > [email protected]> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Dmitri, > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, I think your suggestion to use just one field > > > > > > > > w3c_trace_context > > > > > > > > > > > > > makes more sense than two fields (span_id and > > trace_id). > > > > > > > > > > > > > > > > > > > > > > > > > > With that, I also think that we are slowly drifting > > into > > > > > > > > > > > > > implementation considerations; let's get consensus > > on the > > > > > > general > > > > > > > > > > > > > design first, and we can certainly fine-tune the > > actual > > > > > Java > > > > > > > > methods > > > > > > > > > > > > > and SQL table columns in the future PR. WDYT? > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > Alex > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 24, 2025 at 5:14 PM Dmitri Bourlatchkov < > > > > > > > > > > [email protected]> wrote: > > > > > > > > > > > > >> > > > > > > > > > > > > >> Hi All, > > > > > > > > > > > > >> > > > > > > > > > > > > >> Many thanks for the background info, Adnan! > > > > > > > > > > > > >> > > > > > > > > > > > > >> +1 to action items proposed by Alex! > > > > > > > > > > > > >> > > > > > > > > > > > > >> Re: (3) Can we abstract request/trace info into a > > > > separate > > > > > > > > object > > > > > > > > > > without > > > > > > > > > > > > >> exposing those accessors on the Event class > > directly? > > > > OTel > > > > > > > > defines > > > > > > > > > > > > >> trace/span concepts, but in request ID is a bit > > foreign > > > > to > > > > > > OTel. > > > > > > > > > > Having > > > > > > > > > > > > >> tracing / request ID isolated in java could help > > with > > > > > > > > maintaining > > > > > > > > > > it and > > > > > > > > > > > > >> potentially supporting other (custom) tracing > > methods. > > > > > > > > > > > > >> > > > > > > > > > > > > >> Re: (4) I'd like to propose storing OTel correlation > > > > data > > > > > > in the > > > > > > > > > > form of a > > > > > > > > > > > > >> standard context propagation string, e.g. W3C > > > > > trace-context > > > > > > [1] > > > > > > > > > > (same value > > > > > > > > > > > > >> as its HTTP header), so the column could be called > > > > > > > > > > w3c_trace_context or > > > > > > > > > > > > >> simply trace_context. > > > > > > > > > > > > >> > > > > > > > > > > > > >> Open question: do we need to write a separate, > > > > individual > > > > > > trace > > > > > > > > ID > > > > > > > > > > field in > > > > > > > > > > > > >> SQL? I suppose it is not very useful since > > correlating > > > > it > > > > > to > > > > > > > > other > > > > > > > > > > trace > > > > > > > > > > > > >> data already requires understanding OTel context > > > > > propagation > > > > > > > > and a > > > > > > > > > > query > > > > > > > > > > > > >> against trace_context can still be made using > > > > > > string-matching > > > > > > > > > > clauses. We > > > > > > > > > > > > >> could probably (additionally) store it in the > > request_id > > > > > > column > > > > > > > > if > > > > > > > > > > the > > > > > > > > > > > > >> Polaris-specific request ID header is not set. > > > > > > > > > > > > >> > > > > > > > > > > > > >> As for span ID, I do not really see a use case for > > > > > > persisting it > > > > > > > > > > > > >> individually. It is very specific to OTel trace data > > > > > > > > construction. > > > > > > > > > > > > >> > > > > > > > > > > > > >> Actually, using the W3C trace context [1] encoding > > > > > probably > > > > > > > > makes > > > > > > > > > > sense in > > > > > > > > > > > > >> the java event representation too. Interested > > callers > > > > can > > > > > > easily > > > > > > > > > > decode > > > > > > > > > > > > >> this information since the format is well-defined. > > As a > > > > > side > > > > > > > > > > benefit, this > > > > > > > > > > > > >> opens opportunities for downstream event consumers > > to > > > > > > connect > > > > > > > > > > (propagate > > > > > > > > > > > > >> context) to traces that produced events based on the > > > > event > > > > > > data > > > > > > > > > > itself, > > > > > > > > > > > > >> without relying on the intermediate frameworks. > > This may > > > > > be > > > > > > > > > > desirable since > > > > > > > > > > > > >> the current Polaris event persistence impl. writes > > > > events > > > > > in > > > > > > > > > > batches, so > > > > > > > > > > > > >> the association to individual requests that produced > > > > those > > > > > > > > events > > > > > > > > > > is lost. > > > > > > > > > > > > >> Whether to perform this kind of context propagation > > will > > > > > be > > > > > > at > > > > > > > > the > > > > > > > > > > > > >> discretion of the event consumer, of course > > (outside of > > > > > > Polaris > > > > > > > > > > code). > > > > > > > > > > > > >> > > > > > > > > > > > > >> [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://www.google.com/url?q=https://www.w3.org/TR/trace-context/&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2fn0lRsTx-f9r4PCz8wmJK > > > > > > > > > > > > >> > > > > > > > > > > > > >> WDYT? > > > > > > > > > > > > >> > > > > > > > > > > > > >> Thanks, > > > > > > > > > > > > >> Dmitri. > > > > > > > > > > > > >> > > > > > > > > > > > > >> On Fri, Oct 24, 2025 at 7:18 AM Alexandre Dutra < > > > > > > > > [email protected]> > > > > > > > > > > wrote: > > > > > > > > > > > > >> > > > > > > > > > > > > >>> Hi all, > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> Thank you for chiming in; the context around > > request > > > > IDs > > > > > > is now > > > > > > > > > > clear. > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> Trying to summarize this thread into actionable > > items, > > > > > > here's > > > > > > > > what > > > > > > > > > > I > > > > > > > > > > > > >>> propose: > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> 1. Restore the original functionality for request > > IDs. > > > > > > > > > > > > >>> - Change the default header name back to > > > > x-request-id > > > > > > > > (despite > > > > > > > > > > the > > > > > > > > > > > > >>> x- prefix being deprecated), but keep it > > configurable > > > > as > > > > > > today. > > > > > > > > > > > > >>> 2. Remove RequestIdGenerator and related > > functionality. > > > > > > > > > > > > >>> - Do not generate a request ID if the REST > > client > > > > > > doesn't > > > > > > > > > > provide one. > > > > > > > > > > > > >>> 3. Update PolarisEvent: > > > > > > > > > > > > >>> - Expose new requestId(), traceId(), spanId() > > > > methods, > > > > > > all > > > > > > > > > > nullable. > > > > > > > > > > > > >>> - This would align with the emerging consensus > > > > around > > > > > > > > including > > > > > > > > > > > > >>> contextual information in PolarisEvent [1]. > > > > > > > > > > > > >>> 4. Update events table SQL schema: > > > > > > > > > > > > >>> - Insert the client-provided request ID into the > > > > > > request_id > > > > > > > > > > > > >>> column; otherwise, insert null. > > > > > > > > > > > > >>> - Add two new nullable columns, trace_id and > > > > span_id, > > > > > > and > > > > > > > > > > populate > > > > > > > > > > > > >>> them if OTel is enabled. > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> From our discussions, I think it's important to not > > > > > > conflate > > > > > > > > OTel > > > > > > > > > > > > >>> tracing fields with Envoy tracing fields, which is > > why > > > > I > > > > > > > > suggest we > > > > > > > > > > > > >>> use separate fields / columns for them. > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> Would the above plan work for everyone? > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> Thanks, > > > > > > > > > > > > >>> Alex > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> [1]: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://www.google.com/url?q=https://lists.apache.org/thread/rl5cpcft16sn5n00mfkmx9ldn3gsqtfy&source=gmail-imap&ust=1761927167000000&usg=AOvVaw02wPvb0qxRzYAKEP0h8l9T > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> On Fri, Oct 24, 2025 at 9:33 AM Adnan Hemani > > > > > > > > > > > > >>> <[email protected]> wrote: > > > > > > > > > > > > >>>> > > > > > > > > > > > > >>>> Hi all, > > > > > > > > > > > > >>>> > > > > > > > > > > > > >>>> Thanks to Alex for starting this thread - because > > of > > > > > > this, I’m > > > > > > > > > > just > > > > > > > > > > > > >>> coming to the realization that OTel Trace and Span > > IDs > > > > > are > > > > > > > > coming > > > > > > > > > > built-in > > > > > > > > > > > > >>> with Quarkus and my previous work to generate a > > > > > > > > > > Request/Correlation ID is > > > > > > > > > > > > >>> likely not needed as a result. My original > > motivation > > > > for > > > > > > > > > > generation of a > > > > > > > > > > > > >>> Request/Correlation ID was to ensure that any > > client > > > > can > > > > > > > > uniquely > > > > > > > > > > identify > > > > > > > > > > > > >>> a request made to Polaris, which would be > > especially > > > > > > useful for > > > > > > > > > > debugging > > > > > > > > > > > > >>> failing requests or identifying call patterns. > > > > > > > > > > > > >>>> > > > > > > > > > > > > >>>> As a result, I’m a +1 on Michael’s opinion: we > > should > > > > > > remove > > > > > > > > the > > > > > > > > > > > > >>> Request/Correlation ID generation and always use > > the > > > > OTel > > > > > > > > > > trace/span IDs > > > > > > > > > > > > >>> (which come for free with Quarkus) instead for the > > > > > > Correlation > > > > > > > > ID > > > > > > > > > > unless a > > > > > > > > > > > > >>> valid header is present, which would take over as > > the > > > > > > > > Correlation > > > > > > > > > > ID > > > > > > > > > > > > >>> instead. > > > > > > > > > > > > >>>> > > > > > > > > > > > > >>>> — > > > > > > > > > > > > >>>> > > > > > > > > > > > > >>>> To answer Dmitri’s question re Polaris Events: The > > > > > > intended > > > > > > > > use > > > > > > > > > > case is > > > > > > > > > > > > >>> to provide some sort of correlation between events > > that > > > > > > have > > > > > > > > > > occurred as > > > > > > > > > > > > >>> part of the same request. For example, if a user > > makes > > > > an > > > > > > > > > > CommitTransaction > > > > > > > > > > > > >>> request, it would be helpful to see all UpdateTable > > > > calls > > > > > > that > > > > > > > > > > were made as > > > > > > > > > > > > >>> part of that one user request. > > > > > > > > > > > > >>>> > > > > > > > > > > > > >>>> Best, > > > > > > > > > > > > >>>> Adnan Hemani > > > > > > > > > > > > >>>> > > > > > > > > > > > > >>>>> On Oct 23, 2025, at 12:15 PM, Dmitri > > Bourlatchkov < > > > > > > > > > > [email protected]> > > > > > > > > > > > > >>> wrote: > > > > > > > > > > > > >>>>> > > > > > > > > > > > > >>>>> Hi Michael, > > > > > > > > > > > > >>>>> > > > > > > > > > > > > >>>>> Logging x-request-id headers makes sense. > > > > > > > > > > > > >>>>> > > > > > > > > > > > > >>>>> Just to confirm: if / when we restore that, > > Polaris > > > > > will > > > > > > NOT > > > > > > > > > > generate > > > > > > > > > > > > >>> new > > > > > > > > > > > > >>>>> IDs in case the header is not present in the > > request, > > > > > > > > correct? > > > > > > > > > > > > >>>>> > > > > > > > > > > > > >>>>> I believe x-request-id can co-exist with OTel. > > > > > > > > > > > > >>>>> > > > > > > > > > > > > >>>>> What about adding request IDs to events [1][2]? > > > > What's > > > > > > the > > > > > > > > > > intended use > > > > > > > > > > > > >>>>> case for that? Could you share some context here > > too? > > > > > > > > > > > > >>>>> > > > > > > > > > > > > >>>>> Side note: I proposed [2877] flagging event > > > > persistence > > > > > > as > > > > > > > > > > "beta" in > > > > > > > > > > > > >>>>> 1.2.0... This discussion adds another point > > towards > > > > > > that, I > > > > > > > > > > think. > > > > > > > > > > > > >>>>> > > > > > > > > > > > > >>>>> [1] > > > > > > > > > > > > >>>>> > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/2f0c7a43d446452004ea51196b618de9bdf0e25b/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/inmemory/InMemoryBufferEventListener.java%2523L97%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1WfUaXLp6z_M87iAXEqSUw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Pioys4ROm8vYM_mdx4ygH > > > > > > > > > > > > >>>>> [2] > > > > > > > > > > > > >>>>> > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/19742cc20f4bc0b7e5a315a62f89c6085ad81b7d/runtime/service/src/main/java/org/apache/polaris/service/events/listeners/PolarisPersistenceEventListener.java%2523L66%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw12-7e3ahm2sLSkLSNqTecm&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1PGW6uFwtUN8F1dOlJwrpm > > > > > > > > > > > > >>>>> > > > > > > > > > > > > >>>>> [2877] > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2877%2523discussion_r2456300613%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw3TuYbkzwnLx3QEVIM8oDda&source=gmail-imap&ust=1761927167000000&usg=AOvVaw2Tfk7wAM1MaB5z9Dvw5X5H > > > > > > > > > > > > >>>>> > > > > > > > > > > > > >>>>> Thanks, > > > > > > > > > > > > >>>>> Dmitri. > > > > > > > > > > > > >>>>> > > > > > > > > > > > > >>>>> On Thu, Oct 23, 2025 at 2:23 PM Michael Collado < > > > > > > > > > > > > >>> [email protected]> > > > > > > > > > > > > >>>>> wrote: > > > > > > > > > > > > >>>>> > > > > > > > > > > > > >>>>>> Hey Dmitri > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > >>>>>> The generating a request id is new code that was > > > > added > > > > > > > > after the > > > > > > > > > > > > >>> original > > > > > > > > > > > > >>>>>> x-request-id support. You can see the state > > from ~1 > > > > > year > > > > > > > > ago, we > > > > > > > > > > > > >>> hard-coded > > > > > > > > > > > > >>>>>> request_id as the header we used for the MDC - > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/blob/a6197bd7d8cb5551253fa427e4373897205ecece/polaris-service/src/main/java/org/apache/polaris/service/PolarisApplication.java%2523L415-L416%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw35Q1A_2avAiSYlYVAnZxBb&source=gmail-imap&ust=1761927167000000&usg=AOvVaw34DksOw7DSHJu8PfQQ5bT6 > > > > > > > > > > > > >>>>>> . At some point, it was changed to be > > configurable, > > > > > > then the > > > > > > > > > > > > >>>>>> ContextResolverFilter filter was > > > > refactored/eliminated > > > > > > and > > > > > > > > the > > > > > > > > > > > > >>>>>> RequestIdFilter took its responsibility, but > > lost > > > > some > > > > > > of > > > > > > > > its > > > > > > > > > > original > > > > > > > > > > > > >>>>>> functionality. > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > >>>>>> My personal opinion is that restoring support > > for > > > > the > > > > > > > > > > x-request-id > > > > > > > > > > > > >>> header > > > > > > > > > > > > >>>>>> is something that we should do, but if the > > header > > > > > isn't > > > > > > > > present, > > > > > > > > > > > > >>> falling > > > > > > > > > > > > >>>>>> back on simply using OTel trace ids is good > > enough > > > > > > (better, > > > > > > > > > > even) than > > > > > > > > > > > > >>>>>> generating another random request id. > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > >>>>>> Mike > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > >>>>>> On Thu, Oct 23, 2025 at 10:47 AM Dmitri > > > > Bourlatchkov < > > > > > > > > > > > > >>> [email protected]> > > > > > > > > > > > > >>>>>> wrote: > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > >>>>>>> Hi Michael, > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > >>>>>>> Thanks for the info! > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > >>>>>>> Working with Envoy's tracing headers makes > > sense to > > > > > me. > > > > > > > > > > However, I > > > > > > > > > > > > >>>>>> wonder: > > > > > > > > > > > > >>>>>>> why would Polaris need to generate a new > > request ID > > > > > > inside > > > > > > > > its > > > > > > > > > > > > >>> code?.. > > > > > > > > > > > > >>>>>>> and return it in response headers? > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > >>>>>>> How important is it to propagate this ID to > > Polaris > > > > > > Events? > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > >>>>>>> I'm just trying to understand the full context > > :) > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > >>>>>>> Thanks, > > > > > > > > > > > > >>>>>>> Dmitri. > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > >>>>>>> On Thu, Oct 23, 2025 at 1:29 PM Michael > > Collado < > > > > > > > > > > > > >>> [email protected]> > > > > > > > > > > > > >>>>>>> wrote: > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > >>>>>>>> I think the original intention for this > > requestId > > > > > > field > > > > > > > > was to > > > > > > > > > > > > >>> support > > > > > > > > > > > > >>>>>>>> request propagation from load balancers, like > > > > Envoy > > > > > ( > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1RnuM8mViV-j7jvuxq74Aw&source=gmail-imap&ust=1761927167000000&usg=AOvVaw3LNqtilZ2OoJfE7yEwraZa > > > > > > > > > > > > >>>>>>>> ), which is distinct from OTEL. Don't ask me > > why > > > > the > > > > > > > > default > > > > > > > > > > > > >>>>>>>> is Polaris-Request-Id - I think it was > > originally > > > > a > > > > > > custom > > > > > > > > > > thing, > > > > > > > > > > > > >>> but > > > > > > > > > > > > >>>>>>> then > > > > > > > > > > > > >>>>>>>> we changed to integrate with existing > > conventions. > > > > > > > > > > Unfortunately, > > > > > > > > > > > > >>>>>> looking > > > > > > > > > > > > >>>>>>>> through the code, I think that the actual > > > > functional > > > > > > > > plumbing > > > > > > > > > > has > > > > > > > > > > > > >>> been > > > > > > > > > > > > >>>>>>> lost > > > > > > > > > > > > >>>>>>>> in the course of multiple refactors around the > > > > call > > > > > > > > context > > > > > > > > > > and > > > > > > > > > > > > >>>>>>> resolver. I > > > > > > > > > > > > >>>>>>>> don't see references to that property or the > > > > > > underlying > > > > > > > > > > header. > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > >>>>>>>> Support for the unofficial x-request-id header > > > > feels > > > > > > like > > > > > > > > > > something > > > > > > > > > > > > >>> we > > > > > > > > > > > > >>>>>>>> should definitely keep, especially when > > Polaris is > > > > > one > > > > > > > > > > service in a > > > > > > > > > > > > >>>>>> mesh > > > > > > > > > > > > >>>>>>> of > > > > > > > > > > > > >>>>>>>> services that maybe don't have OTel > > integration. > > > > > I'm a > > > > > > > > fan of > > > > > > > > > > the > > > > > > > > > > > > >>> OTel > > > > > > > > > > > > >>>>>>>> standard, but it's not entirely ubiquitous and > > > > there > > > > > > are > > > > > > > > many > > > > > > > > > > > > >>>>>> middleware > > > > > > > > > > > > >>>>>>>> layers that know how to forward on the > > > > x-request-id > > > > > > > > header. > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > >>>>>>>> Mike > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > >>>>>>>> On Thu, Oct 23, 2025 at 3:00 AM Robert Stupp < > > > > > > > > [email protected]> > > > > > > > > > > > > >>> wrote: > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > >>>>>>>>> Yes, we should aim for interoperability with > > the > > > > > > existing > > > > > > > > > > de-facto > > > > > > > > > > > > >>>>>>>>> standard OTel and make it easy for users to > > > > > integrate > > > > > > > > into > > > > > > > > > > their > > > > > > > > > > > > >>>>>>>>> observability platforms. > > > > > > > > > > > > >>>>>>>>> > > > > > > > > > > > > >>>>>>>>> On Wed, Oct 22, 2025 at 7:05 PM Dmitri > > > > > Bourlatchkov < > > > > > > > > > > > > >>>>>> [email protected]> > > > > > > > > > > > > >>>>>>>>> wrote: > > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>> Hi Alex and All, > > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>> I certainly support the idea of following > > OTel > > > > > > > > standards for > > > > > > > > > > > > >>>>>>> achieving > > > > > > > > > > > > >>>>>>>>>> "correlation" wrt Polaris requests and/or > > > > events. > > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>> As to what form the correlation data should > > > > take, > > > > > I > > > > > > > > believe > > > > > > > > > > it is > > > > > > > > > > > > >>>>>>>>>> conceptually what the OTel "context" > > represents. > > > > > > So, I > > > > > > > > > > believe it > > > > > > > > > > > > >>>>>>> makes > > > > > > > > > > > > >>>>>>>>>> sense for Polaris to support standard > > context > > > > > > > > propagators > > > > > > > > > > at the > > > > > > > > > > > > >>>>>> API > > > > > > > > > > > > >>>>>>>>> layer. > > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>> If the incoming request has OTel context > > > > > > information, > > > > > > > > then > > > > > > > > > > > > >>>>>> returning > > > > > > > > > > > > >>>>>>>> any > > > > > > > > > > > > >>>>>>>>>> other "correlation" data in the response is > > > > > > redundant, I > > > > > > > > > > think. > > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>> If the incoming request does not have OTel > > > > context > > > > > > info, > > > > > > > > > > what is > > > > > > > > > > > > >>>>>> the > > > > > > > > > > > > >>>>>>>>>> purpose of generating a Polaris-specific > > > > > > "correlation > > > > > > > > ID"? > > > > > > > > > > How is > > > > > > > > > > > > >>>>>> it > > > > > > > > > > > > >>>>>>>>>> envisioned to be used? > > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>> If the intention is to correlate a Polaris > > > > > response > > > > > > > > > > (operation) > > > > > > > > > > > > >>>>>> with > > > > > > > > > > > > >>>>>>>>> events > > > > > > > > > > > > >>>>>>>>>> that might have resulted from its > > execution, I > > > > > > believe a > > > > > > > > > > more > > > > > > > > > > > > >>>>>> robust > > > > > > > > > > > > >>>>>>>>>> approach would be to propagate the OTel > > trace > > > > info > > > > > > > > > > (starting a new > > > > > > > > > > > > >>>>>>>> trace > > > > > > > > > > > > >>>>>>>>> if > > > > > > > > > > > > >>>>>>>>>> necessary) into event data. Then, Polaris > > can > > > > also > > > > > > > > return > > > > > > > > > > the > > > > > > > > > > > > >>> trace > > > > > > > > > > > > >>>>>>>>> context > > > > > > > > > > > > >>>>>>>>>> in the API response (top span). It's a bit > > > > awkward > > > > > > from > > > > > > > > the > > > > > > > > > > OTel > > > > > > > > > > > > >>>>>>>>>> perspective, but might be an option for > > > > supporting > > > > > > > > custom > > > > > > > > > > > > >>>>>>> correlators. > > > > > > > > > > > > >>>>>>>> It > > > > > > > > > > > > >>>>>>>>>> could be covered by a feature flag. The > > header > > > > > name > > > > > > > > could be > > > > > > > > > > > > >>>>>>>>>> "polaris-traceparent" for W3C Trace Context. > > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>> Custom correlation code will be able to > > extract > > > > > the > > > > > > > > trace > > > > > > > > > > ID from > > > > > > > > > > > > >>>>>> the > > > > > > > > > > > > >>>>>>>>>> response and from events and find related > > data. > > > > > > > > Granted, it > > > > > > > > > > will > > > > > > > > > > > > >>>>>>>> require > > > > > > > > > > > > >>>>>>>>> a > > > > > > > > > > > > >>>>>>>>>> bit more effort for the custom code to > > decode > > > > > trace > > > > > > IDs > > > > > > > > > > from the > > > > > > > > > > > > >>>>>> OTel > > > > > > > > > > > > >>>>>>>>>> context, but the format is standard and not > > > > > > complex. The > > > > > > > > > > benefit > > > > > > > > > > > > >>>>>> for > > > > > > > > > > > > >>>>>>>>>> Polaris, though, is that it can easily > > integrate > > > > > > with > > > > > > > > > > > > >>>>>> OTel-compatible > > > > > > > > > > > > >>>>>>>>>> observability platforms regardless of > > whether > > > > any > > > > > > > > particular > > > > > > > > > > > > >>>>>>> deployment > > > > > > > > > > > > >>>>>>>>>> uses custom correlators or not. > > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>> WDYT? > > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>> Thanks, > > > > > > > > > > > > >>>>>>>>>> Dmitri. > > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>> On Wed, Oct 22, 2025 at 6:03 AM Alexandre > > Dutra > > > > < > > > > > > > > > > > > >>> [email protected] > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > >>>>>>>>> wrote: > > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>> Hi all, > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>> Today, Polaris has the notion of "request > > ID", > > > > > but > > > > > > its > > > > > > > > > > purpose is > > > > > > > > > > > > >>>>>>> not > > > > > > > > > > > > >>>>>>>>>>> entirely clear. It seems to serve as an > > > > > > observability > > > > > > > > > > feature to > > > > > > > > > > > > >>>>>>>>>>> facilitate correlation. A pending PR aims > > to > > > > > > rename it > > > > > > > > to > > > > > > > > > > > > >>>>>>>> "correlation > > > > > > > > > > > > >>>>>>>>>>> ID" for better alignment with industry > > > > standards > > > > > > [1]. > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>> However, this PR has brought to light > > overlaps > > > > > with > > > > > > > > core > > > > > > > > > > > > >>>>>> telemetry > > > > > > > > > > > > >>>>>>>>>>> features: when OpenTelemetry (OTel) is > > enabled > > > > in > > > > > > > > Polaris, > > > > > > > > > > each > > > > > > > > > > > > >>>>>>>>>>> request already has a trace ID and span ID, > > > > > making > > > > > > a > > > > > > > > > > separate > > > > > > > > > > > > >>>>>>>>>>> correlation ID redundant. > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>> Moreover, using the OTel trace ID and span > > ID > > > > in > > > > > > > > Polaris > > > > > > > > > > events, > > > > > > > > > > > > >>>>>>>>>>> rather than the generated correlation ID, > > would > > > > > > > > > > significantly > > > > > > > > > > > > >>>>>>>> simplify > > > > > > > > > > > > >>>>>>>>>>> correlation of events with other traces. > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>> Therefore, I propose the following changes: > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>> 1. If OTel is enabled, use the trace ID and > > > > span > > > > > > ID as > > > > > > > > the > > > > > > > > > > > > >>>>>>>> correlation > > > > > > > > > > > > >>>>>>>>>>> ID for the request, instead of generating a > > > > > random > > > > > > > > > > correlation > > > > > > > > > > > > >>>>>> ID. > > > > > > > > > > > > >>>>>>>>>>> 2. Otherwise, if a (Polaris-specific) > > > > correlation > > > > > > ID > > > > > > > > > > header is > > > > > > > > > > > > >>>>>>>> present > > > > > > > > > > > > >>>>>>>>>>> in the request, use it. > > > > > > > > > > > > >>>>>>>>>>> 3. If neither of the above conditions is > > met, > > > > > > generate > > > > > > > > a > > > > > > > > > > random > > > > > > > > > > > > >>>>>>>>>>> correlation ID. > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>> I am somewhat undecided on the best > > approach > > > > > when a > > > > > > > > > > correlation > > > > > > > > > > > > >>>>>> ID > > > > > > > > > > > > >>>>>>>>>>> header is present in the request. However, > > I > > > > > > believe it > > > > > > > > > > would be > > > > > > > > > > > > >>>>>>> more > > > > > > > > > > > > >>>>>>>>>>> sensible to disregard it if OTel is > > enabled, as > > > > > > OTel > > > > > > > > > > offers a > > > > > > > > > > > > >>>>>> more > > > > > > > > > > > > >>>>>>>>>>> robust solution for client-to-server trace > > > > > > propagation, > > > > > > > > > > e.g. W3C > > > > > > > > > > > > >>>>>>>> Trace > > > > > > > > > > > > >>>>>>>>>>> Context propagation [2]. > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>> Please share your thoughts! > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>> Thanks, > > > > > > > > > > > > >>>>>>>>>>> Alex > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>>>> [1]: > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/polaris/pull/2757%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw1-kAWfEk4tmsEg0q0GZBCn&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1Oe-25vtt4gLVSMFMtSVNg > > > > > > > > > > > > >>>>>>>>>>> [2]: > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://www.w3.org/TR/trace-context%26source%3Dgmail-imap%26ust%3D1761851831000000%26usg%3DAOvVaw22nMyOS7pbJ69XrBo5kHQS&source=gmail-imap&ust=1761927167000000&usg=AOvVaw1TRdkzABc_7U-_KZ1MZ59v > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > >>>>>>>>> > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > >>>> > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
