Thanks! On Tue, Nov 19, 2024, 9:13 PM Zheng Feng <zf...@redhat.com> wrote:
> Answer inline, > > On Wed, Nov 20, 2024 at 10:49 AM David Hoffer <dhoff...@gmail.com> wrote: > > > Thanks for your quick response. > > > > As I think about this more I do not think MongoDB and Athena are ever > used > > together in the same REST transaction. As we use those DBs for different > > REST services. > > > > So it seems we are okay with all the other DB options. > > > > However I have a few questions. > > > > - How does Quarkus JTA transaction manager support any non XA > datasource > > in an XA transaction? > > > It marks the non-XA datasource as the Last Resource which is committed in > the last one, so if it fails, rollback all of the other XA datasources. > > > - When you said above that Athena is OK did you mean OK using our > > current CData JDBC driver? Or do we have to use the Camel AWS Athena > > SDK > > to get the behavior you stated? > > > Yeah, using the CData JDBC driver. > > > - What are the pros/cons of using the Camel AWS Athena SDK vs our > > currency cdata JDBC approach? > > > Sorry, I know nothing about Athena and just thinking from the JDBC and XA > transactions. Just reading from the camel doc > https://camel.apache.org/components/4.8.x/aws2-athena-component.html, it > looks like that does not support transactions at all. > > > > > > > Thanks, > > -David > > > > > > On Tue, Nov 19, 2024 at 5:57 PM Zheng Feng <zf...@redhat.com> wrote: > > > > > OK, I see. So you are using PG and MongoDB and Ahtena in one service, > > but I > > > think only PG supports the XA transaction currently. > > > > > > And yeah, Quarkus JTA transaction manager only supports using one > non-XA > > > datasource in a XA transaction. > > > PG + MongoDB is OK > > > PG + Athena is OK > > > PG + MongoDB + Athena is not recommended because there are some risks > to > > > lost the data consistent > > > > > > Anyway, unsafe-multiple-last-resources can make it working but you will > > > face the risks. So the answer to your question is "No, it is not > > possible". > > > > > > I don't know any business about your REST services, not sure if they > > really > > > need the strong ACID transaction. Or maybe you can consider LRA (Long > > > Running Activities) which is guaranteed to eventually consistent. See > > > https://quarkus.io/blog/using-lra/ > > > > > > > > > > > > > > > On Wed, Nov 20, 2024 at 8:30 AM David Hoffer <dhoff...@gmail.com> > wrote: > > > > > > > Hi, > > > > > > > > Yes we actually have 4 datasources. > > > > > > > > This is the main one, always exists. > > > > > > > > db-kind: postgresql > > > > > > > > emrDS: > > > > db-kind: other > > > > jdbc: > > > > driver: com.facebook.presto.jdbc.PrestoDriver > > > > > > > > dialect: Derby > > > > > > > > athenaDS: > > > > db-kind: other > > > > jdbc: > > > > driver: cdata.jdbc.amazonathena.AmazonAthenaDriver > > > > > > > > dialect: PostgreSQL > > > > > > > > EMR is used on some networks and Athena on others. We are trying to > > > > replace EMR with Athena everywhere once it's available. > > > > > > > > We also use MongoDB. > > > > > > > > Yes our apps are Quarkus and we are upgrading to the latest 3.16.x > > > version. > > > > > > > > So we use these 3 dependencies for JPA & JTA. > > > > > > > > <dependency> > > > > <groupId>io.quarkus</groupId> > > > > <artifactId>quarkus-hibernate-orm</artifactId> > > > > </dependency> > > > > <dependency> > > > > <groupId>io.quarkus</groupId> > > > > <artifactId>quarkus-narayana-jta</artifactId> > > > > </dependency> > > > > <dependency> > > > > <groupId>io.quarkus</groupId> > > > > <artifactId>quarkus-agroal</artifactId> > > > > </dependency> > > > > > > > > <dependency> > > > > <groupId>io.quarkus</groupId> > > > > <artifactId>quarkus-jdbc-postgresql</artifactId> > > > > </dependency> > > > > > > > > <dependency> > > > > <groupId>org.mongodb</groupId> > > > > <artifactId>mongodb-driver-sync</artifactId> > > > > </dependency> > > > > > > > > Our app is a large JAX-RS application using the above DBs. Some of > > > > the operations require usage of two of the JDBC DBs above. > > > > > > > > Plus some use MongoDB. So could be 3 DBs. (We never use EMR & > Athena > > > > at the same time.) > > > > > > > > We would like each REST service to use XA for any DB that uses any of > > > > the DBs above. It works fine for Postgres & MongoDB but latest > > > > version of > > > > > > > > Quarkus gives errors when Athena is used as it's not XA. We found > > > > that adding: unsafe-multiple-last-resources: allow to the > > > > transaction-manager > > > > > > > > works but it's deprecated and warns it may/will be removed in the > > future. > > > > > > > > > > > > Is this possible for all to be XA? > > > > > > > > Thanks, > > > > > > > > -David > > > > > > > > > > > > On Tue, Nov 19, 2024 at 4:53 PM Zheng Feng <zf...@redhat.com> wrote: > > > > > > > > > Hi David, > > > > > > > > > > Can you describe more about your scenario? Do you want to run > Athean > > > and > > > > > other data sources in an XA transaction? What transaction manager > did > > > you > > > > > use, is it quarkus-narayana-jta? > > > > > > > > > > On Wed, Nov 20, 2024 at 7:19 AM David Hoffer <dhoff...@gmail.com> > > > wrote: > > > > > > > > > > > > > https://camel.apache.org/components/4.8.x/aws2-athena-component.html > > > > > > > > > > > > We use multiple data sources in our Quarkus application and > Athena > > is > > > > one > > > > > > of them. Is there a way to make this XA compliant? The latest > > > Quarkus > > > > > is > > > > > > failing with our current CData driver as its not XA, so looking > for > > > > > > something that is XA. > > > > > > > > > > > > Thanks, > > > > > > -David > > > > > > > > > > > > > > > > > > > > >
