Hi Yufei, I do not think this discussion is about API/SPI changes per se. It is rather about the approach to handling downstream dependencies as the project evolves.
The "existing code paths" affected by PR 3200 are not conceptually different from existing code paths affected by the configuration rename in PR 3135. Those downstream dependencies "exist" in the same way because configuration is naturally part of the generalized API surface. Formalizing API/SPI definitions is not enough since any change to a formalized API/SPI is going to expose the same issues WRT downstream projects as a change to a non-formalized API (which is polaris-core ATM). As JB noted, and I agree, downstream projects should be prepared for change. As PR 3200 illustrates, an absolute block on any effects or restrictions downstream projects may have on the current Polaris code is not feasible. I would like this discussion to lead to a consensus in practical attitudes for ensuring that the Polaris project can evolve, without causing undue hardship downstream. Cheers, Dmitri. On Thu, Dec 4, 2025 at 3:08 PM Yufei Gu <[email protected]> wrote: > Hi Dmitri, > > Thanks for following up. > > On the Java side, I believe Michael already highlighted the implications > around the existing code paths. The community also has an ongoing > discussion about SPI and Java interface stability in Polaris. One thing I > want to emphasize is that, at least in the core module, we generally lean > toward treating all public classes and methods as part of the public > interface. Even if they aren’t formally annotated as API, they are used as > extension points by downstream systems, and removing them can introduce > subtle breakages. > > Yufei > > > On Thu, Dec 4, 2025 at 11:34 AM Dmitri Bourlatchkov <[email protected]> > wrote: > > > Hi Yufei, > > > > As I previously mentioned, your statement about avoiding downstream > > projects dictating OSS choices resonates with me. > > > > I created PR [3200] as a practical illustration of some of these > concerns > > in the java code. > > > > I wonder what your and other people's thoughts on this might be. > > > > [3200] https://github.com/apache/polaris/pull/3200 > > > > Thanks, > > Dmitri. > > > > > > > > On Tue, Dec 2, 2025 at 8:24 PM Yufei Gu <[email protected]> wrote: > > > > > I don’t think the Java code is ever a concern. Contributors will more > > > likely be comfortable following the upstream conventions, even if their > > > private downstream setup differs. > > > > > > Yufei > > > > > > On Tue, Dec 2, 2025 at 4:31 PM Dmitri Bourlatchkov <[email protected]> > > > wrote: > > > > > > > Thanks for your thoughts, Yufei! > > > > > > > > What you wrote resonates with me. > > > > > > > > I'd like to clarify, though, whether your points apply only to > > > > configuration or to java code as well. WDYT? > > > > > > > > Thanks, > > > > Dmitri. > > > > > > > > On Tue, Dec 2, 2025 at 4:06 PM Yufei Gu <[email protected]> > wrote: > > > > > > > > > Thanks Dmitri for the detailed context. I want to offer a different > > > > > perspective. > > > > > > > > > > While I understand the desire to avoid churn for a private > downstream > > > > > deployment, I’m concerned that merging the PR as-is purely to > > > > accommodate a > > > > > non-OSS downstream project sets a precedent that is risky for > > Polaris. > > > If > > > > > we adopt this pattern, any private downstream project could > > effectively > > > > > promote its internal configuration choices into upstream simply by > > > > > upstreaming code without aligning to Polaris conventions first. > > > > > > > > > > For the long-term health of the project, I think we need to > maintain > > > > > consistency and avoid creating special cases that diverge from > > > > established > > > > > user-faced conventions(e.g., config naming patterns here). The > > upstream > > > > > should remain the place where common conventions are defined, not > the > > > > place > > > > > where downstream decisions dictate upstream behavior. > > > > > > > > > > To avoid blocking this PR while still maintaining our standards, > one > > > > > possible approach is: > > > > > 1. Align the config names now to the Polaris conventions, as > > reviewers > > > > > requested. > > > > > 2. Let the downstream project map or override its own existing > config > > > > names > > > > > suggested by Eric. > > > > > > > > > > This keeps Polaris consistent, avoids setting an undesirable > > precedent, > > > > and > > > > > still gives the downstream project a clear migration path without > > > > requiring > > > > > OSS to inherit private naming decisions. > > > > > > > > > > Just my two cents. Happy to discuss further. > > > > > > > > > > Yufei > > > > > > > > > > > > > > > On Thu, Nov 27, 2025 at 5:12 PM Eric Maynard < > > [email protected] > > > > > > > > > wrote: > > > > > > > > > > > I see. By extending the configuration store, I believe that > project > > > > > should > > > > > > be able to quite easily map between its own configs and the ones > > that > > > > > > eventually get merged in Polaris. Or to add its own configs. > > > > > > > > > > > > On Thu, Nov 27, 2025 at 5:04 PM Dmitri Bourlatchkov < > > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > Hi Eric, > > > > > > > > > > > > > > This situation here is that some code moves from a private > > > downstream > > > > > > > project to Polaris. Some of the related config names are > already > > in > > > > use > > > > > > > downstream. > > > > > > > > > > > > > > Cheers, > > > > > > > Dmitri. > > > > > > > > > > > > > > On Thu, Nov 27, 2025 at 7:35 PM Eric Maynard < > > > > [email protected] > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > I read through the PR, but I’m a little confused what the > > > > dependency > > > > > in > > > > > > > > question is here. Surely, the project should be mindful not > to > > > > break > > > > > > > > downstream dependencies of Polaris. But it sounds like the > > > > contention > > > > > > > here > > > > > > > > is around a dependency on unmerged code? > > > > > > > > > > > > > > > > If this is the case, that deployment is not depending on > > Polaris. > > > > In > > > > > my > > > > > > > > view if it depends on particular configs not present in OSS > > > > Polaris, > > > > > > > > it would be best served by taking advantage of some of the > > > > mechanisms > > > > > > > > available for adding custom configs on top of the ones in OSS > > > > > Polaris. > > > > > > > > > > > > > > > > —EM > > > > > > > > > > > > > > > > On Thu, Nov 27, 2025 at 10:53 AM Dmitri Bourlatchkov < > > > > > [email protected] > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Hi All, > > > > > > > > > > > > > > > > > > This discussion stems from PR [3135], where having > particular > > > > > config > > > > > > > > names > > > > > > > > > was flagged as a concern [1]. > > > > > > > > > > > > > > > > > > While I agree that aligning configuration names across all > > > > Polaris > > > > > > code > > > > > > > > > modules is a worthy goal, in this particular case it is > > > > complicated > > > > > > by > > > > > > > > > downstream dependencies that already exist on those config > > > names. > > > > > > > > > > > > > > > > > > The NoSQL code contributed in this PR is already in use in > > > > private > > > > > > > > > environments. At the same time it is being contributed to > > > Polaris > > > > > in > > > > > > > the > > > > > > > > > hole that it would be beneficial to a wide community. > > > > > > > > > > > > > > > > > > Making the requested config changes up front would > complicate > > > > > > > > > re-integration of the contributed code from Polaris `main` > > into > > > > > > > existing > > > > > > > > > downstream projects. > > > > > > > > > > > > > > > > > > Therefore, I ask the community to allow the existing config > > > names > > > > > to > > > > > > be > > > > > > > > > merged. > > > > > > > > > > > > > > > > > > If there are strong concerns about the existing config > > names, I > > > > > > propose > > > > > > > > to > > > > > > > > > open GH issues for them and address later when backward > > > > > compatibility > > > > > > > can > > > > > > > > > be handled in OSS code for all use cases. This can probably > > be > > > > done > > > > > > > after > > > > > > > > > some reasonable delay to allow us to first re-integrate the > > > code > > > > > from > > > > > > > > > `main` downstream. > > > > > > > > > > > > > > > > > > Note that addressing config renames with backward > > compatibility > > > > > > > directly > > > > > > > > in > > > > > > > > > [3135] would complicate the code and make reviewing it > > harder. > > > > The > > > > > > > > current > > > > > > > > > PR reflects the code that has already been put to practice. > > > > > > > > > > > > > > > > > > We have an existing situation where a downstream project's > > > > > > requirements > > > > > > > > for > > > > > > > > > keeping `polaris-code` java compatibility at version 11 are > > > > > respected > > > > > > > [2] > > > > > > > > > despite having no reasons for holding that module's java > > > version > > > > > back > > > > > > > > from > > > > > > > > > the perspective of binary Polaris Server distributions. > > > > > > > > > > > > > > > > > > In general, I believe it is important for the success of > the > > > > > project > > > > > > to > > > > > > > > > take downstream builds into consideration as well as users > of > > > the > > > > > > > binary > > > > > > > > > distribution. > > > > > > > > > > > > > > > > > > Please share your thoughts on this matter. > > > > > > > > > > > > > > > > > > [1] > > > > > > > https://github.com/apache/polaris/pull/3135#discussion_r2558515915 > > > > > > > > > > > > > > > > > > [2] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/polaris/discussions/100#discussioncomment-10267097 > > > > > > > > > > > > > > > > > > [3135] https://github.com/apache/polaris/pull/3135 > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > Dmitri. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
