> In practice though it is possible that different realms get bootstrapped with different database schemas and I think we should try to make the service resilient to situations like this
+1 on this. Each realm could have its own schema, or even different Postgres server. A bit more context on it, we were trying to implement this while doing JDBC implementation. The only reason we didn't do that is the Quarkus configuration limits, it can only support one data source. Yufei On Fri, Aug 1, 2025 at 10:56 PM Eric Maynard <eric.w.mayn...@gmail.com> wrote: > +1 to everything Dmitri said, well put. > > Ideally, I would say that "schema" ought not to be a first-class concept > within Polaris administration, just whether a realm has been bootstrapped > or not. In practice though it is possible that different realms get > bootstrapped with different database schemas and I think we should try to > make the service resilient to situations like this. > > As far as the PR itself, I agree that giving admins the *option* to refine > bootstrapping down into substeps seems useful. Beyond separating the schema > creation as the PR proposes, one could imagine being able to just re-create > a certain table or something like that. Maybe there could be options, such > as whether to include some particular table (e.g. they aren't using events, > so don't make an events table). However, I agree with Dmitri that the > default should just be a plain & complete bootstrap. > > --EM > > On Sat, Aug 2, 2025 at 8:51 AM Dmitri Bourlatchkov <di...@apache.org> > wrote: > > > Hi Alex, > > > > Thanks for starting this thread! My opinions below. > > > > > 1) Should schema setup be separate from realm bootstrapping? > > > > I think it should be possible to execute the two actions separately. > Yet, I > > do not mind running them together by default. > > > > > 2) Should the server perform schema setup at all? > > > > Only if the server is configured to bootstrap automatically (which should > > be off by default for any real persistence). > > > > > 3) different schemas per realm? > > > > I cannot imagine how this might work... Do you mean different schemas in > > different realms? That feels like a super narrow use case to me, > although I > > do not mind supporting it. > > > > > 4) options [...] > > > > I believe the primary schema setup case (with many options, etc.) should > be > > through the admin tool. > > > > If the server bootstraps automatically, it should use only defaults for > the > > schema setup. I consider this case only as a "test" or "getting started" > > use case (i.e. not "production"). > > > > Cheers, > > Dmitri. > > > > > > > > On Fri, Aug 1, 2025 at 6:43 AM Alexandre Dutra <adu...@apache.org> > wrote: > > > > > Hi all, > > > > > > A nice recent contribution [1], still under review, proposes to create > > > a separate admin tool command for setting up the database schema. > > > > > > Currently, schema setup is done as part of realm bootstrapping [2], > > > which can happen at server bootstrap, when running the tool's > > > bootstrap command, or on-demand, depending on the RealmContextResolver > > > in use. For this reason, the setup script must be idempotent. > > > > > > But the PR raised several questions. I will try to summarize them here: > > > > > > 1) Should schema setup be separate from realm bootstrapping? > > > > > > 2) Should the server perform schema setup at all? If so, when? At > > > startup, or when a new realm is resolved? > > > - Side question: if the server doesn't perform schema setup, how are > > > tests going to do it? > > > > > > 3) Do we want/need to support – or at least leave the door open for – > > > different schemas per realm? > > > > > > 4) If we introduce an option to control how schema setup is done: > > > - Should it be a configuration option in application.properties? It > > > would then be available to both server and tool. > > > - Should it be an option of the admin tool's `bootstrap` command? > > > Opt-in or opt-out? > > > - Should it be an admin tool's separate command like `setup` ? > > > > > > For 4) I would be in favor of a new configuration option, e.g.: > > > > > > polaris.schema.setup-mode=NEVER|STARTUP|PER_REALM > > > > > > It would be accessible to both server and tool, and the default could > > > be NEVER for production, and AT_STARTUP for tests. PER_REALM could be > > > introduced later. > > > > > > I'm curious to see what others think. > > > > > > Thanks, > > > Alex > > > > > > [1] https://github.com/apache/polaris/pull/2196 > > > [2] > > > > > > https://github.com/apache/polaris/blob/2117dbd08e8352b32a2c948ed6c166d7c77da50a/persistence/relational-jdbc/src/main/java/org/apache/polaris/persistence/relational/jdbc/JdbcMetaStoreManagerFactory.java#L148-L158 > > > > > >
