Hi, I've taken a look at Trafodion (and read some about TiDB as well) and will try to state what I perceive as some differences in the properties this will have, as compared with the architecture I was thinking about.
There is so much to say in this area so that I will only mention a couple of things that I think bring out some differences. Only pointing at differences is not of much interest, after all no one has said that there are no differences. But I do think it can be illuminating to state the important ones. So the outset was my question about using one and the same database for all microservices. (I'm not talking about the Cassandra db, if I understand it correctly it only stores commands. Please correct me if I'm wrong.) Then Eric challenged the position of one database per service and also that CQRS is not needed now that NewSQL databases are available. >From the Trafodion web site: "Distributed ACID transaction protection across multiple statements, tables and/or rows". I argue that if you have distributed transactions then no matter what technology you put to use, by the CAP theorem this makes you take another stance then the maximally scale out CQRS architecture you get with DDD style aggregates as the transaction boundary. This architecture opts for AP and sacrifices strong consistency. Note that when I say CQRS I mean separate models for the read and write sides, this is something different than just persisting the commands as documented in https://cwiki.apache.org/confluence/display/FINERACT/CQRS+in+Fineract+CN . >From the Trafodion FAQ: "Where would you position Trafodion according to the CAP theorem? Is it CP (consistent and partition tolerant as HBase) or CA (consistent and highly available)? Trafodion is hosted on top of HBase and HDFS. HBase is generally viewed as being CA (Consistent, Available) in the context of the CAP theorem. Unlike regular HBase, Trafodion extends the definition of consistency to provide ACID protection across transactions comprised of multiple SQL statements, tables, and rows." This further illuminates the fundamental difference between the CQRS architecture that simply opts that in practice you will choose between consistency and availability, you can't have both since you can't guarantee network partition will not happen. I am not arguing that Trafodion (or similar) is not a good fit for Fineract, but I am saying that it does not bring the same scalability properties to the table as state of the art CQRS where the write side is isolated and allows for no transacation boundaries extending the aggregate. But really the original question touches other things that are not as advanced as the ones discussed above. If a service shares the database with other services, then by necessity there will be a dependency relation between them. If not the database scheme could be changed arbitrarily without impact on other services and this if of course not true if they use the same database. Best regards Niklas Resurs Bank AB Niklas Uhrberg Konsult IT Development | Loans & Credit Box 22209, SE-250 24 Helsingborg Direkt Tfn: Mobil: Vxl: +46 42 382000 Fax: E-post: [email protected] Webb: http://www.resursbank.se ___________________________ From: Eric Owhadi [[email protected]] Sent: Wednesday, October 23, 2019 5:21 PM To: [email protected] Subject: RE: Fineract CN and its database Yes it is indeed, There was some follow up reactions on it too. Eric From: Niklas Uhrberg <[email protected]> Sent: Wednesday, October 23, 2019 10:19 AM To: [email protected] Subject: RE: Fineract CN and its database External Hi Eric, yes I do make an assumption about the properties of the database. I think I have found the mail thread, is it the one where you write: " Hello Fineracteers, I discovered Fineract and specially Fineract-cn last week, and am very impressed by the future potential of the project, both to deliver on its original social mission but also to revolutionize commercial FinTech sector the same way the big.. " ? Before I say anything, I will read up on this, it is new stuff for me! Best regards Niklas [cid:[email protected]] Niklas Uhrberg Konsult IT Development | Loans & Credit Resurs Bank AB Växel: +46 42 38 20 00 E-post: [email protected]<mailto:[email protected]> Webb: www.resursbank.se<http://www.resursbank.se> ________________________________ From: Eric Owhadi [[email protected]] Sent: Wednesday, October 23, 2019 4:51 PM To: [email protected]<mailto:[email protected]> Subject: RE: Fineract CN and its database Hi Niklas, Your assumption about the impossibility to do both use case with a shared database assumes the use of regular RDBMS. Not the use of New SQL DB like Trafodion, EsgynDB or TiDB (I am not expert in TiDB). These DBs are horizontally scalable, with shared nothing architecture. Eventual consistency and CQRS comes at a price (see former post on the topic when I discussed the NewSQL trend), and is not needed given the availability of NewSQL DB. The original design was great at time NewSQL did not exist, but it is not currently the best approach given you can get a full RDBMS capability, fully ACID and horizontally scalable. I don't know if you red my original email on Trafodion option for fineract cn? Hope this makes sense, Best regards, Eric From: Niklas Uhrberg <[email protected]<mailto:[email protected]>> Sent: Wednesday, October 23, 2019 9:36 AM To: [email protected]<mailto:[email protected]> Subject: RE: Fineract CN and its database External Hi Eric! I'm taking the perspective of service isolation and scalability here. For services to be isolated with all that this brings they cannot share its database with other services. Scalability is also a central issue where one shared database for all services is the worst case and an event sourced system with CQRS and eventual consistency is the most scalable. Now, the nice thing is that we can have it both ways: In a microservice architecture fully embracing scalabilty by not sharing state we can still have read models serving the analytical functionality you refer to by constructing a database that serves these use cases well. Referencing the discussion about load testing initiated by Joseph Cabral where processing 1.2 million accounts took hours, I would like to point out that in a DDD design where the aggregates contain all or nearly all the data needed for the calculation it is straightforward to scale this type of operation to take no more than a couple of seconds. I recently did a proof of concept using the Akka stack doing exactly this type of demo. The operation carried out in the PoC is probably a great deal simpler, but the important thing is to use a suitable model and allow scaling by not sharing data. This is totally impossible if you use a single shared database for both write and read purposes. Best regards Niklas [cid:[email protected]] Niklas Uhrberg Konsult IT Development | Loans & Credit Resurs Bank AB Växel: +46 42 38 20 00 E-post: [email protected]<mailto:[email protected]> Webb: www.resursbank.se<http://www.resursbank.se> ________________________________ From: Eric Owhadi [[email protected]] Sent: Tuesday, October 22, 2019 4:32 PM To: [email protected]<mailto:[email protected]> Subject: RE: Fineract CN and its database Hi Niklas, I have not followed or seen the email thread on this topic, but I am curious if the adverse draw back of doing so was discussed: When analytical queries are needed to help in business decisions/metrics, crossing several domain, performing joins between tables belonging to different domain can only be done efficiently using database joins. If each micro service owns its database, we explicitly forbid using DB joins, therefore by high price doing adhoc application level joins. Does that make sense? Regards, Eric Owhadi From: Niklas Uhrberg <[email protected]<mailto:[email protected]>> Sent: Tuesday, October 22, 2019 4:43 AM To: [email protected]<mailto:[email protected]> Subject: Fineract CN and its database External Hi! About Fineract CN and its database, I guess you have discussed splitting the database so that each microservice "owns" its database and does not share the same database with the other microservices. Is this setup an explicit goal for Fineract CN? Best regards Niklas [cid:[email protected]] Niklas Uhrberg Konsult IT Development | Loans & Credit Resurs Bank AB Växel: +46 42 38 20 00 E-post: [email protected]<mailto:[email protected]> Webb: www.resursbank.se<http://www.resursbank.se>
