Thanks for raising this, Dmitri. I agree we should be careful about coupling new features into core and runtime in ways that are hard to unwind, and I'm on board with feature flags and keeping optional behavior off default paths where that makes sense.
I'm less convinced we need a blanket policy of isolated Gradle modules, feature-specific SPIs, and staged entity-then-REST PRs for every new proposal. A big reason is contributor experience: most of the existing server code lives in a small number of well-understood modules (polaris-core, polaris-runtime-service, the generated API jars). Someone proposing scan metrics or a semantic layer API can find similar REST handlers, persistence patterns, and tests without first learning a module taxonomy or deciding which of several new jars their change belongs in. I'd rather optimize for single-module (or few-module) contributions by default: add the endpoint, service logic, and tests alongside existing similar code. I can understand bundling sets of functionality together like if we did want to separate Polaris specific from Iceberg specific modules but I think the codebase is actually more usable with fewer modules rather than more. If a feature later proves it needs independent deployment, a pluggable backend, or a separate schema lifecycle, we can extract it then. Most deployments run the standard server release anyway; downstream custom assembly is possible but doesn't seem to be a widespread pattern today. Given that, I'd support lighter guidelines: 1. Don't entangle new features into core call paths unnecessarily 2. Use flags for optional capabilities 3. Default to adding code where contributors already look 4. Split into a separate module only when there's a demonstrated need Happy to discuss where that line should be. On Wed, Jun 24, 2026 at 9:06 AM Dmitri Bourlatchkov <[email protected]> wrote: > Hi All, > > Polaris has been getting many new and interesting proposals lately. This is > certainly good for the project. > > On the other hand, we need to think about the stability and usability of > the system as new features are introduced. > > Polaris is currently used in two modes: a) as a ready-made server for the > default set of features (source or binary releases) and/or b) as a basis > for custom downstream builds (from Maven artifacts). > > I'd like to propose the following general principles, which I hope will > allow quick feature development without adding risks to either of the usage > avenues. > > 1) Put code for new REST API services in isolated Gradle modules. > > 2) Wire those services into the runtime/server explicitly, behind feature > flags where appropriate. > > 3) Do not add hard dependencies from runtime/service or polaris-core to > REST API modules. > > 4) If a feature requires new Polaris entity types, add those core model > changes in a dedicated PR so the entity and persistence contract can be > reviewed on its own. New REST service modules can depend on those core > entities, but existing core call paths should not depend on > feature-specific entities. > > 5) Add new Persistence SPI(s) for non-entity storage (e.g. Scan/Commit > Metrics). Keep new SPI classes in feature-specific Gradle modules. > > 6) Use isolated SQL schema definition files for each feature involving > non-entity JDBC persistence. > > That is: separate .sql files for Metrics, Events, etc. > > Thoughts? > > Cheers, > Dmitri. >
