Sorry, I meant Adnan, of course :) On Tue, Nov 4, 2025 at 2:08 PM Dmitri Bourlatchkov <[email protected]> wrote:
> Hi Prashant, > > Thanks for clarifying that pair-wise correlation of multiple before/after > event pairs per request is _not_ required. > > That was my only concern in this thread. I do not have any additional use > cases in mind. I was just trying to achieve clarity on what was actually > required and what was collateral :) > > Cheers, > Dmitri. > > > On Mon, Nov 3, 2025 at 10:48 PM Adnan Hemani > <[email protected]> wrote: > >> Hi Dmitri, >> >> Thanks for creating this thread! If what has sparked this thought was my >> input earlier that users would like to correlate their events together, >> I'd >> like to make a clarification: I believe that users would like to correlate >> events coming from the same request to Polaris together. In which case, I >> think Request ID is sufficient to do what we need. If there is other input >> that we should be able to correlate pairs of Before/After events, then I >> think we should discuss either adding "Correlation ID" (which I really >> don't like, as it makes things quite confusing IMO) or putting a "Span ID" >> inside the `additionalProperties` of the event, like Yufei suggested (and >> we may have to generate that "Span ID"). >> >> As to Yufei's point, I don't think we need to explicitly store OTel >> context >> information. If it is present through Quarkus' integration with OTel, I >> still believe the OTel Trace ID should be the Request ID, if the user has >> not defined a Request ID (sorry to mix the lines with the other threads' >> topics on this). >> >> WDYT? >> >> Best, >> Adnan Hemani >> >> On Fri, Oct 31, 2025 at 11:47 AM Yufei Gu <[email protected]> wrote: >> >> > Agreed with Dmitri. >> > >> > X-Request-Id (or Request-Id) is not a standard HTTP header but is widely >> > used across systems, including Polaris, for logging, events, and >> metrics. >> > Its primary purpose is debuggability. For example, it lets us correlate >> > logs and events for the same HTTP call. >> > Example: >> > Log: {timestamp: ..., request-id: 123, message: "Update table >> > successfully!"} >> > Event: {type: BeforeUpdateTable, request-id: 123}, {type: >> AfterUpdateTable, >> > request-id: 123} >> > >> > *Request ID* should remain the canonical identifier for every request >> > handled by Polaris. >> > >> > - If missing, Polaris generates one (e.g., UUIDv7 or brings back the >> one >> > Adnan built). >> > - All logs, events, and metrics include it for consistent internal >> and >> > downstream correlation. No need to invent another concept like >> > "correlated ID" for event only, which introduce inconsistency and >> make >> > correlation harder. >> > >> > *OTel context* is OPTIONAL. >> > >> > - If present, persist it alongside for tracing. Put it in >> > `additionalProperties` for events if people need it. >> > - If absent, users decide to start a new trace (nice to have) or not. >> > >> > >> > Yufei >> > >> > >> > On Fri, Oct 31, 2025 at 10:37 AM Dmitri Bourlatchkov <[email protected]> >> > wrote: >> > >> > > Hi All, >> > > >> > > I'd like to broaden the events / OTel discussion [1][2] a bit. >> Apologies >> > > for forking the email threads even more, but I think this aspect >> > deserves a >> > > focused discussion. >> > > >> > > From previous messages my impression is that there are use cases for >> > > strongly correlating "before" and "after" events. In that situation >> the >> > > full OTel context may be too volatile. For example, nothing guarantees >> > that >> > > the "before" and "after" events are produced under the same OTel span >> in >> > > Polaris. >> > > >> > > Another aspect of this is multiple before/after event pairs may be >> > produced >> > > under the same API request. Assuming event consumers do want to >> > > re-establish correct event pairs, I believe using only the request ID >> is >> > > not sufficient for that purpose. >> > > >> > > We probably need to restore the Polaris request ID generator but use >> it >> > to >> > > produce an ID per group of events (before/after) and publish / persist >> > that >> > > ID with the event data. >> > > >> > > WDYT? >> > > >> > > [1] https://lists.apache.org/thread/bb1qyxjt827t3tomv2xp0s1kovxjsp94 >> > > [2] https://lists.apache.org/thread/5dpyo0nn2jbnjtkgv0rm1dz8mpt132j9 >> > > >> > > Thanks, >> > > Dmitri. >> > > >> > >> >
