Hi Romain, Thanks for the info!
Metrics requests generally read from the MetaStore but write only in the Metrics datasource. The latest thinking about Events is to isolate them from the initial request via the Quarkus Event Bus (async delivery), IIRC. The new Idempotency feature [3] might be the first case where we (could) do writes into multiple datasources. However, the way it is designed does not assume or require coordination of changes / commits between the MetaStore and the Idempotency data store, AFAIK. In general, this is a very interesting topic. It has implications for our (MetaStore) Persistence interface contracts too. IIRC, previous [1] discussions generally settled on the idea that each Persistence API call forms one atomic (and durable) change. Yet, I do not think we have completely ironed out all wrinkles [2] there. The tricky aspect is that Persistence may be backed by a non-transactional database. Currently we have JDBC and MongoDB, which could be said to be transactional, still the NoSQL Persistence in Polaris is built without assuming Tx capabilities, so it will also work on non-transactional databases. This is just for context :) [1] https://lists.apache.org/thread/fh6t3mgzdb5ft0dt5q7qxm1pj1x5po75 [2] https://lists.apache.org/thread/rf5orxs815zs4h64p4rwp03q3pbgxb5r [3] https://lists.apache.org/thread/ksf5b5y3jt8dmft5kgblbwms8dhs4fn9 Cheers, Dmitri. On Thu, Jun 18, 2026 at 3:03 PM Romain Manni-Bucau <[email protected]> wrote: > Hi Dmitri, > > Assuming you put metadata and metrics/events (maybe metrics but these ones > where not under my radar) in different databases to leverage different > tuning and aggregation level but still have exactly once guarantee. > I assume at some point there are or will be some writes in the same request > (stat updates, meta refresh/audit etc). > > While a single request/background task is guaranteed to be single > datasource backed there is no real topic if it is what you have in mind. > > Romain Manni-Bucau > @rmannibucau <https://x.com/rmannibucau> | .NET Blog > <https://dotnetbirdie.github.io/> | Blog <https://rmannibucau.github.io/> > | Old > Blog <http://rmannibucau.wordpress.com> | Github > <https://github.com/rmannibucau> | LinkedIn > <https://www.linkedin.com/in/rmannibucau> | Book > < > https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064 > > > Javaccino founder (Java/.NET service - contact via linkedin) > > > Le jeu. 18 juin 2026 à 19:58, Dmitri Bourlatchkov <[email protected]> a > écrit : > > > Hi Romain, > > > > In what cases / API calls do you envision XA to be used in Polaris? > > > > I'm asking just for my education, as I do not see how / where it could be > > relevant ATM. > > > > Thanks, > > Dmitri. > > > > On Thu, Jun 18, 2026 at 1:11 PM Romain Manni-Bucau < > [email protected]> > > wrote: > > > > > Ok, good it was already thought. > > > > > > Think worse case an user can set the 3 same or enable XA using agroal > > > capabilities ([1]) > > > > > > Thanks > > > > > > [1] > > > > > > > > > https://quarkus.io/guides/datasource#quarkus-agroal_quarkus-datasource-jdbc-transactions > > > > > > Romain Manni-Bucau > > > @rmannibucau <https://x.com/rmannibucau> | .NET Blog > > > <https://dotnetbirdie.github.io/> | Blog < > https://rmannibucau.github.io/ > > > > > > | Old > > > Blog <http://rmannibucau.wordpress.com> | Github > > > <https://github.com/rmannibucau> | LinkedIn > > > <https://www.linkedin.com/in/rmannibucau> | Book > > > < > > > > > > https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064 > > > > > > > Javaccino founder (Java/.NET service - contact via linkedin) > > > > > > > > > Le jeu. 18 juin 2026 à 18:17, Alexandre Dutra <[email protected]> a > > écrit > > > : > > > > > > > Hi Romain, > > > > > > > > You raise good points. > > > > > > > > Datasource multitenancy in Quarkus would have helped a lot during > this > > > > old discussion: [1] where we eventually decided to go with one > > > > datasource for N realms, out of better choices by then. But that ship > > > > has sailed. > > > > > > > > And good question about XA: the proposal in [3960] does not include > > > > any transactional coordination between datasources, afaict, and the > > > > outbox pattern doesn't fit anymore in the design we picked for events > > > > (and probably for metrics). This pattern was considered initially > > > > iirc, as it would have allowed exactly-once semantics for events; but > > > > in the end, we went with separate tables updated independently (~= > > > > at-most-once semantics). > > > > > > > > Thanks, > > > > Alex > > > > > > > > [1]: > https://lists.apache.org/thread/kvt3gd29hyxlhnwxr9b674s555jlvjjs > > > > [3960]: https://github.com/apache/polaris/pull/3960 > > > > > > > > On Thu, Jun 18, 2026 at 8:38 AM Romain Manni-Bucau > > > > <[email protected]> wrote: > > > > > > > > > > Hi Alexandre, > > > > > > > > > > Can be worth shiming in the quarkus thread about it: > > > > > https://github.com/quarkusio/quarkus/issues/11861 before merging a > > new > > > > way > > > > > to do it (even if it delays from a few weeks the feature) I think. > > > > > > > > > > Technically one thing the 1 database -> N databases (not speaking > > about > > > > > tenants where there is no challenge but meta+metrics+events) brings > > the > > > > > transactional challenge, do you go XA with this shift or do you > plan > > > some > > > > > recovery (kind of outbox pattern with some challenges)? > > > > > > > > > > Romain Manni-Bucau > > > > > @rmannibucau <https://x.com/rmannibucau> | .NET Blog > > > > > <https://dotnetbirdie.github.io/> | Blog < > > > https://rmannibucau.github.io/> > > > > | Old > > > > > Blog <http://rmannibucau.wordpress.com> | Github > > > > > <https://github.com/rmannibucau> | LinkedIn > > > > > <https://www.linkedin.com/in/rmannibucau> | Book > > > > > < > > > > > > > > > > https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064 > > > > > > > > > > Javaccino founder (Java/.NET service - contact via linkedin) > > > > > > > > > > > > > > > Le mer. 17 juin 2026 à 21:37, Alexandre Dutra <[email protected]> > a > > > > écrit : > > > > > > > > > > > Hi all, > > > > > > > > > > > > I'd like to elaborate on the "pluggable persistence" concept > > defined > > > > > > my previous message, and how it might evolve in the future: > > > > > > > > > > > > The proposal from PR [3960] originally suggested three distinct > > > "store > > > > > > types": metastore, metrics, and events. Although that PR was > closed > > > > > > because of inactivity, the concept remains relevant. If we decide > > to > > > > > > support multiple datasources for each store type, I envision the > > > > > > following approach: > > > > > > > > > > > > Polaris would provide datasource definitions for every > combination > > of > > > > > > store type and JDBC driver (such as "postgresql-metastore" or > > > > > > "h2-metrics"). The current "enabler property" would be replaced > by > > a > > > > > > map of properties keyed by the store type. For instance: > > > > > > > > > > > > > > > > > > > > > > > > > > > > polaris.persistence.relational.jdbc.metastore.datasource=postgresql-metastore > > > > > > > > > > > > polaris.persistence.relational.jdbc.metrics.datasource=postgresql-metrics > > > > > > > > > polaris.persistence.relational.jdbc.events.datasource=postgresql-events > > > > > > > > > > > > In this configuration, three separate PostgreSQL datasources > would > > be > > > > > > enabled, each managing its own connection pool and settings. > > > > > > > > > > > > Alternatively, if a user wants to use a single datasource for all > > > > > > data, they could simply map every store type to the same name > > (e.g., > > > > > > "postgresql-metastore"). This would result in only one active > > > > > > datasource and a single connection pool for the metastore, > metrics, > > > > > > and events. > > > > > > > > > > > > It is still open for debate whether the default setup should > > > > > > prioritize a single shared datasource or separate ones for each > > store > > > > > > type. > > > > > > > > > > > > While this is primarily speculative at this point, since the > > feature > > > > > > hasn't been finalized, it demonstrates that we have a viable path > > > > > > forward should we choose to implement this architecture. > > > > > > > > > > > > Thanks, > > > > > > Alex > > > > > > > > > > > > [3960]: https://github.com/apache/polaris/pull/3960 > > > > > > > > > > > > On Wed, Jun 17, 2026 at 9:26 PM Alexandre Dutra < > [email protected] > > > > > > > wrote: > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > I have recently submitted a PR [4812] that introduces support > for > > > > > > > multiple datasources with possibility of runtime activation. > > > > > > > > > > > > > > I would like to open a discussion regarding the intent behind > > this > > > > > > > proposal and the advantages it offers. > > > > > > > > > > > > > > While Quarkus is a powerful platform, it presents certain > > > challenges, > > > > > > > particularly with JDBC datasources. Currently, the datasource > > type > > > > > > > must be defined at build time using the "db-kind" property. > This > > > > makes > > > > > > > managing multiple datasources difficult, especially for > Polaris, > > > > where > > > > > > > users should ideally be able to select the appropriate > datasource > > > > (and > > > > > > > driver) based on their specific requirements. > > > > > > > > > > > > > > The core of this proposal relies on a key fact: while "db-kind" > > is > > > > > > > fixed at build time, the "active" property can be toggled at > > > runtime. > > > > > > > Combined with named datasources, this property becomes the > > missing > > > > > > > lever to achieve runtime datasource activation. > > > > > > > > > > > > > > Under this change, Polaris must include predefined named > > > datasources > > > > > > > for every supported driver. Users simply choose which ones to > > > > activate > > > > > > > at runtime using an enabler property: > > > > > > > "polaris.persistence.relational.jdbc.datasource", which > > identifies > > > > the > > > > > > > named datasource for activation. (Worth noting: an inactive > > > > datasource > > > > > > > does not consume any resources at runtime.) > > > > > > > > > > > > > > As a result, the server will now include the H2 driver by > > default, > > > as > > > > > > > it is a prerequisite for selecting an H2 datasource at runtime. > > We > > > > > > > will need to follow a similar approach for any other JDBC > drivers > > > we > > > > > > > intend to support in the future. > > > > > > > > > > > > > > This architectural shift is necessary for several reasons: > > > > > > > > > > > > > > 1. Flexible defaults: It allows us to use JDBC + H2 as the > > default > > > > for > > > > > > > server images (see these related discussions for details: [1] > > [2]) > > > > > > > while enabling users to switch to PostgreSQL or other drivers > > *via > > > > > > > configuration alone*, thus eliminating the need to rebuild > > Polaris > > > > > > > entirely. > > > > > > > > > > > > > > 2. Pluggable persistence: It enables the persistence layer to > > > connect > > > > > > > to multiple datasources simultaneously. For instance, one could > > > use a > > > > > > > specific datasource for main persistence and another for > metrics > > – > > > a > > > > > > > concept previously discussed in [3960] that lacked an easy > > > > > > > implementation path in my opinion. > > > > > > > > > > > > > > I look forward to hearing your thoughts on this proposal. > > > > > > > > > > > > > > Thanks, > > > > > > > Alex > > > > > > > > > > > > > > [4812]: https://github.com/apache/polaris/pull/4812 > > > > > > > [3960]: https://github.com/apache/polaris/pull/3960 > > > > > > > > > > > > > > [1]: > > > > https://lists.apache.org/thread/yw8l026g2smdk7gdg7k61tdcvdwcncqw > > > > > > > [2]: > > > > https://lists.apache.org/thread/nzoljc1ohnsq4f5o28dh4opqkqw3p09h > > > > > > > > > > > > > > > >
