Hi Dmitri, Thank you for taking the time to review. I had pulled out the schema in its own PR (based on comments) here: https://github.com/apache/polaris/pull/3523
I will update the document with the SPI and carve out another PR that focuses on the SPI. — Anand From: Dmitri Bourlatchkov <[email protected]> Date: Thursday, January 29, 2026 at 10:43 AM To: [email protected] <[email protected]> Cc: Anand Kumar Sankaran <[email protected]> Subject: Re: Polaris Telemetry and Audit Trail This Message Is From an External Sender This message came from outside your organization. Report Suspicious<https://us-phishalarm-ewt.proofpoint.com/EWT/v1/Iz9xO38YGHZK!YhNDZAGr2cumY1cAlUeoPrU2eNUPz9Wxq2PaG1t99vEN3v60gucxC3Qsk-xK09RndcW46WtNjMMjQZbaQ495h1D1g293F_uqLhPM8gDMlA6xRmo5b5uqrSh2v7QKHrhQ$> Hi Anand, Thanks for making a proposal doc and starting this thread. Apologies for late comments. As you know I already reviewed a couple of related PRs, which had a smaller impact on the codebase, but now, I believe, we're coming to the core of this feature and it might need a deeper discussion. For the sake of awareness of other project members, I'd like to highlight some aspects by email. * Storing scan metrics in RDBMS in general looks like a reasonable approach to me. However, Polaris Persistence is an extensible mechanism. Other backends are possible, including privately developed backends. From this POV, I believe it might be preferable to start not with an RDBMS schema, but with a java SPI outlining expected read/write operations and the data model. This will make it easier to assess impact on other Persistence implementations. Naturally, Polaris service code will have to be able to work seamlessly regardless of the backend impl. (including the do-nothing impl.). I see that some connection between the SQL schema and java code is made in PR [3385], but it does not appear to offer a clean SPI that could be implemented by different Persistence backends. Would it be ok from your perspective to make another PR with just SPI code? * Regarding ad-hoc SQL queries - it is certainly a valid use case for custom code to query the Polaris database directly. However, those ad-hoc queries may require certain indexes, which may not have a relevant use case in OSS code. In that case, I believe it would be preferable to manage the extra indexes in custom code (not in Polaris). Whether this is the case or not currently, I cannot say with certainty, so apologies if this is a false alarm... Having the java SPI available would be a huge aid to reasoning about the proposed SQL schema, I think. That said, I do not mean to complicate the development of this feature, just trying to structure it in a way that is hopefully easy to understand (including myself), maintain and expand later. If I missed something, please feel free to point it out. [3385] https://github.com/apache/polaris/pull/3385<https://urldefense.com/v3/__https://github.com/apache/polaris/pull/3385__;!!Iz9xO38YGHZK!_dbBfwEnZEuPp_8qIuBDoYfWRgCZKiPDMg9ouDltqsHhr9hqCHVlF9IedSkNVBDWupbneU7inC9YK-THL-lTjWkkyZhI0OE$> Thanks, Dmitri. On Wed, Jan 14, 2026 at 10:05 PM Anand Kumar Sankaran via dev <[email protected]<mailto:[email protected]>> wrote: Hi Yufei, Gave commenting privileges to all. Thanks. — Anand From: Yufei Gu <[email protected]<mailto:[email protected]>> Date: Wednesday, January 14, 2026 at 6:13 PM To: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> Cc: Anand Kumar Sankaran <[email protected]<mailto:[email protected]>> Subject: Re: Polaris Telemetry and Audit Trail This Message Is From an External Sender This message came from outside your organization. Report Suspicious<https://us-phishalarm-ewt.proofpoint.com/EWT/v1/Iz9xO38YGHZK!YhNDZABkHi1B6hyPVYUK0T-K5j4Aoqdrlu07UjBpJcpjOsz7Ie7d_DhPl-ywi50dfQz4O6LL-AHLCOQTQ0X7Le8yD2cF0ogquGFB8Aw8uQLgB0RL6Ezpa-pLDPA4ziXA$> Hi Anand, thanks for the design doc and PR. Can you allow comments in the doc so that people can chime in? Thanks! Yufei On Sat, Jan 10, 2026 at 9:09 AM Anand Kumar Sankaran via dev <[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>> wrote: Hi all My first PR for adding AWS STS Session Tags support for credential vending was merged. https://github.com/apache/polaris/pull/3327<https://urldefense.com/v3/__https://github.com/apache/polaris/pull/3327__;!!Iz9xO38YGHZK!_dbBfwEnZEuPp_8qIuBDoYfWRgCZKiPDMg9ouDltqsHhr9hqCHVlF9IedSkNVBDWupbneU7inC9YK-THL-lTjWkklJ4Xf9Y$><https://urldefense.com/v3/__https://github.com/apache/polaris/pull/3327__;!!Iz9xO38YGHZK!8vlQ2_OqjmZNIpbrTTluFFNKFCEeM87nr95MP5pMpMu5fFWVSoBZ3eb6-C2xbAGCG-ry5eaCO-pFLxl3IZAJGqY$> I have been working with Prashant on an end-to-end telemetry and audit trail tracking for Polaris. It is documented here. https://docs.google.com/document/d/1Ehzvi5RNPs4hChkBFI6VD23myEqm-7sWW3d2kjmuYj8/edit?tab=t.0<https://urldefense.com/v3/__https://docs.google.com/document/d/1Ehzvi5RNPs4hChkBFI6VD23myEqm-7sWW3d2kjmuYj8/edit?tab=t.0__;!!Iz9xO38YGHZK!_dbBfwEnZEuPp_8qIuBDoYfWRgCZKiPDMg9ouDltqsHhr9hqCHVlF9IedSkNVBDWupbneU7inC9YK-THL-lTjWkk8MVJPlY$><https://urldefense.com/v3/__https://docs.google.com/document/d/1Ehzvi5RNPs4hChkBFI6VD23myEqm-7sWW3d2kjmuYj8/edit?tab=t.0__;!!Iz9xO38YGHZK!8vlQ2_OqjmZNIpbrTTluFFNKFCEeM87nr95MP5pMpMu5fFWVSoBZ3eb6-C2xbAGCG-ry5eaCO-pFLxl3hYqTlcg$> Based on Prashant’s initial feedback (parity with Apache Gravitino for metrics reports), I have an initial PR here. https://github.com/apache/polaris/pull/3385<https://urldefense.com/v3/__https://github.com/apache/polaris/pull/3385__;!!Iz9xO38YGHZK!_dbBfwEnZEuPp_8qIuBDoYfWRgCZKiPDMg9ouDltqsHhr9hqCHVlF9IedSkNVBDWupbneU7inC9YK-THL-lTjWkkyZhI0OE$><https://urldefense.com/v3/__https://github.com/apache/polaris/pull/3385__;!!Iz9xO38YGHZK!8vlQ2_OqjmZNIpbrTTluFFNKFCEeM87nr95MP5pMpMu5fFWVSoBZ3eb6-C2xbAGCG-ry5eaCO-pFLxl39opCznI$> I will be reaching out to various folks for review. I am open to breaking down the PR into smaller logical groups of PR if it helps. I request you to look at the Google doc and let me know what you think. — Anand -- Dmitri Bourlatchkov Senior Staff Software Engineer, Dremio Dremio.com<https://urldefense.com/v3/__https://www.dremio.com/?utm_medium=email&utm_source=signature&utm_term=na&utm_content=email-signature&utm_campaign=email-signature__;!!Iz9xO38YGHZK!_dbBfwEnZEuPp_8qIuBDoYfWRgCZKiPDMg9ouDltqsHhr9hqCHVlF9IedSkNVBDWupbneU7inC9YK-THL-lTjWkkiOwjJc0$> / Follow Us on LinkedIn<https://urldefense.com/v3/__https://www.linkedin.com/company/dremio__;!!Iz9xO38YGHZK!_dbBfwEnZEuPp_8qIuBDoYfWRgCZKiPDMg9ouDltqsHhr9hqCHVlF9IedSkNVBDWupbneU7inC9YK-THL-lTjWkkCLjBQ2M$> / Get Started<https://urldefense.com/v3/__https://www.dremio.com/get-started/__;!!Iz9xO38YGHZK!_dbBfwEnZEuPp_8qIuBDoYfWRgCZKiPDMg9ouDltqsHhr9hqCHVlF9IedSkNVBDWupbneU7inC9YK-THL-lTjWkkoKQUMII$> [https://lh3.googleusercontent.com/d/1Ta6MlHHnksAk0fuUEm2yxgiTJbWMh6YM] The Agentic Lakehouse The only lakehouse built for agents, managed by agents
