Hi James, Let me update on the work we did (Rohit Jain and I): After realizing that there was not only a migration away from Maria DB to Postgres, but also from Hibernate to EclipseLink, I postponed the prototyping work until I could use a migrated code base, that was not readily available. But waiting for that, and also having discovered the dependency on Flyway, that did not support trafodion, Rohit Jain, CTO of Esgyn, took on the task of making Flyway and EclipseLink trafodion/EsgynDB compatible. I personally was side tracked on another project and might be able to resume working on Fineract depending on how we staff internally across our various initiatives, or we may find another Esgyn resource to deal with the task. Thanks for resurrecting the topic, and accept my apologies for not having updated on our current work earlier. Best regards, Eric Owhadi
From: James Dailey <[email protected]> Sent: Sunday, October 20, 2019 12:08 PM To: Dev <[email protected]> Cc: Eric Owhadi <[email protected]> Subject: Upstream, database decisions, newSQL, postgres ? External Devs - (fineract-CN) Please see below on trafodion and subsequent discussion by Victor Romero's team on NewSQL (analysis) as well as back and forth on separation of concerns (design pattern and implication in DB). What was the community and expert consensus on this rich discussion? I think it relates to postgresql migration. Am I missing something? Assuming other users are going to maintain their own runtime versions of the code using different back end resources, then shouldn't the community release include a way to simplify up stream compatibility? Should this be part of "move MariaDB to archive" discussion? @Jdailey67 ---------- Forwarded message --------- From: Eric Owhadi <[email protected]<mailto:[email protected]>> Date: Wed, Jun 5, 2019, 11:39 AM Subject: RE: Would it be interesting to use Apache "Trafodion" as an option for persistence layer? To: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> Hi Markus, Thanks for your message and for sharing your experience with a large retail German bank. You may be familiar with this: https://martinfowler.com/bliki/CQRS.html The author explains the pro/cons of CQRS design pattern/ when to use vs when to avoid. Also, from fineract documentation, the following has been identified https://cwiki.apache.org/confluence/display/FINERACT/CQRS+in+Fineract+CN “The advantage to this approach is that it's really fast. The drawback is that some kinds of errors can only be detected while the command handler is executing and this class of errors will not be surfaced appropriately to the user. The basic sanity checks are intended to ameliorate this.” You can find from feedback published on fineract wiki https://cwiki.apache.org/confluence/display/FINERACT/Hosting+Experiences+from+Partners#HostingExperiencesfromPartners-Mines.io “Concern about using Cassandra for journal: noSQL is not a typical choice for strongly consistent use case” And the message bellow from Victor Romero having his company investigate a NewSQL alternative to the NoSQL + MariaDB/MySQL/Postgres There is an operational cost associated to running a database (backup/restore/upgrade/patch/admin etc). So the least amount of DB instances, the cheaper the TCO of the solution. Last point that comes to mind: having the RDBMS leg horizontally scalable allows the “Query” side of CQRS to be also horizontally scalable. Hope this makes sense? Best regards, Eric From: Markus Geiss <[email protected]<mailto:[email protected]>> Sent: Wednesday, June 5, 2019 1:36 AM To: Dev <[email protected]<mailto:[email protected]>> Subject: Re: Would it be interesting to use Apache "Trafodion" as an option for persistence layer? Hey all, trust this finds you well. (; The whole architecture of CN is build around a clean separation of concerns, e.g. commands vs queries. Based on the tools we are using (e.g. Spring data, Spring messaging) and have built on top (e.g. Command gateway) on a day to day base writing business value code will not interfere with technical details or writing SQL statements. I'm not sure about what kind of complexity Eric is talking about, and would be happy to hear about these concerns. PS: We are talking to large retail banks here in Germany, and neither from an internal objection nor from a regulatory point of view NoSQL is an issue. Markus On Wed, Jun 5, 2019 at 7:33 AM Victor Romero <[email protected]<mailto:[email protected]>> wrote: Hi Eric, You have bring a good topic on the table, the use of NewSQL in Core Banking. I think that is important to put in the center of the discussion concept of "NewSQL" instead of naming a solution before researching, evaluating and sharing experiences. Recently we have done some research for supporting a request from an enterprise user, because the customer have told us the concerns that the institution has about MariaDB (SQL) and Cassandra (NoSQL) used in Fineract. Then we did the leg-work comparing NewSQL options: Google Spanner (Not an option because data regulation issues), CockroachDB (Time consuming for changing the code), VoltDB (Time consuming for changing the code) & Trafodion (Time consuming for changing the code) and TiDB (now time to explain very briefly) In our case we have decided to do Proof of Concepts using TiDB (NewSQL) because of these drivers: - Scalability - High Availability - ACID - Ease of use ---- It is compatible with the MySQL ecosystem tools ---- Reuse the knowledge of our data engineering team ---- Reduce the time to port/close the SQL gaps ( https://pingcap.com/docs/dev/reference/mysql-compatibility/ ) - Existing customer experience published by the "vendor" of the DB ---- We have contacted companies to gather information about the use/migraction experience of TiDB (https://github.com/pingcap/docs/blob/master/adopters.md and https://www.pingcap.com/success-stories/tidb-in-shopee/ ) We will share the experience by the end of this month about the use if Fineract CN and TiDB for each driver. Just to give more information about the DBs, I will take a table and a image from the paper "TOP NEWSQL DATABASES AND FEATURES CLASSIFICATION" - International Journal of Database Management Systems ( IJDMS ) Vol.10, No.2, April 201, for summarizing the concepts about SQL, NoSQL and NewSQL [cid:[email protected]] [cid:[email protected]] By the way, the document has evaluated Trafodion :) and the authors give their opinions about it, it is worth reading. Regards Victor El 04/06/19 a las 20:12, Eric Owhadi escribió: Hi Graham, Thanks you for your comment and question. Trafodion does have a light version that we use for dev. We call it local_hadoop, and it fits in single dev boxes. This will perfectly fit this use cases, Hope this helps, Eric From: Ebenezer Graham <[email protected]><mailto:[email protected]> Sent: Tuesday, June 4, 2019 7:53 AM To: Dev <[email protected]><mailto:[email protected]> Subject: Re: Would it be interesting to use Apache "Trafodion" as an option for persistence layer? External Hi Eric, Thanks for this interesting and insightful proposal. Please expound on how to use Trafodion in component tests and CI/CD. If you noticed, all the services have a component-test package aside from the unit tests. This is used to test the service during development as well as for CI/CD - the data from the component tests is persisted to an embedded Cassandra and embedded MariaDB/PostgreSQL helping to mimic a similar CQRS pattern as will be used in production. How will this work when using Trafodion? Does Trafodion have a light-weight version for this use case? If not, what is a typical approach for this? I have looked at the specs for a basic Trafodion setup and its quite resource intensive. Looking forward to your response At your service, Ebenezer Graham On Fri, 31 May 2019 at 02:45, Eric Owhadi <[email protected]<mailto:[email protected]>> wrote: Hi James, Thanks for taking the time to answer, Please find bellow inline answers to your questions. What would be the impact on current efforts to remove current dependencies on non Apache license compatible data persistent layers/libraries? >>> I am not yet familiar with the code, so can’t comment on this. What I can >>> say is that Trafodion uses a standard jdbc driver, and other than dealing >>> with minor SQL dialect differences that may exist (like any popular RDBMS >>> dialect minor differences), the adaptation to Trafodion should be minimal >>> (depending on how the code is structured – understand if the SQL statement >>> are well separated vs have to hunt for them in many places in the code). Having said that, the statement is valid for just replacing RDBMS calls to trafodion. Then replacing Cassandra calls to trafodion should also be easy. However, if we want to replace some of current CQRS commands from Async to Sync, the code change will be more involved, as I assume the whole thing about doing “light check” before performing the async command, and the un-necessary error handling involved at the use case level (what happen if an async command fail, and UI thought it went OK) should be optionally removed. Instead classic RDBMS transaction commit/rollback would be done. Once implemented, is this more, less, same to maintain and add functionality? Would it require a dev to map the data logical level (ORM) more intensively? Is there a different skill set needed with this than current concept and are those skills generally available? >>> general DB skill. Easier than current solution, as standard SQL is easier >>> to find than standard SQL + Cassandra special. Is the trajectory of this technology vs noSql clear? >>> Yes. It’s purpose is to add back everything that was given up (ACID) from >>> classic RDBMS to achieve horizontal scalability. Now you have a full >>> classic RDBMS, but capable of handling huge volume, with scale out share >>> nothing architecture. You've made a good case, what's the counter? >>> The counter argument would be maturity. MySQL, Postgres or MariaDB, given >>> their wide adoption, benefit from more battle testing than Trafodion. >>> Having said that, and given the recent commitment to core banking, EsgynDB >>> will be battle tested for core Banking, while the other DB are not >>> typically seen in the “classic” Core Banking sector. By the time >>> fineract-cn will be feature complete, I think the maturity argument will >>> have seriously faded out. What about Apache Ignite? >>> Apache Ignite is a different beast. Classified as “in memory DB”. I have >>> no experience with it, but can say that what is needed is a persistence >>> layer, not an in memory db to speed up analytical queries. Therefore I have >>> hard time seeing a fit with Ignite. Is there an independent review of Trafodion? >>> Not that I know of. Is the open source version useable or does this require use of the EsgynDB in production? Are other providers using the tech? >>> I would not use Trafodion in production, given I would never go in >>> production without support contract on the database. Currently only Esgyn >>> Corporation provide a Trafodion based production ready product. If the project devs can come to some conclusion on this, what's the level of effort? How should we do think about the size of this effort. & do you think... is there someone available to work on this project, perhaps from one of those banks? >>> On my free cycles, I will study fineract-cn code and analyze effort / >>> prototype something. Best regards, Eric Owhadi From: James Dailey <[email protected]<mailto:[email protected]>> Sent: Thursday, May 30, 2019 4:26 PM To: Dev <[email protected]<mailto:[email protected]>> Cc: Eric Owhadi <[email protected]<mailto:[email protected]>> Subject: Re: Would it be interesting to use Apache "Trafodion" as an option for persistence layer? External Eric, First, thank you very much. This is a really interesting concept and proposal for a shift in architecture. I have some questions: What would be the impact on current efforts to remove current dependencies on non Apache license compatible data persistent layers/libraries? Once implemented, is this more, less, same to maintain and add functionality? Would it require a dev to map the data logical level (ORM) more intensively? Is there a different skill set needed with this than current concept and are those skills generally available? Is the trajectory of this technology vs noSql clear? What about Apache Ignite? Is there an independent review of Trafodion? You've made a good case, what's the counter? Is the open source version useable or does this require use of the EsgynDB in production? Are other providers using the tech? If the project devs can come to some conclusion on this, what's the level of effort? How should we do think about the size of this effort. & do you think... is there someone available to work on this project, perhaps from one of those banks? Thanks, Jdailey67 On Thu, May 30, 2019, 1:48 AM Eric Owhadi <[email protected]<mailto:[email protected]>> wrote: Hello Fineracteers, I discovered Fineract and specially Fineract-cn last week, and am very impressed by the future potential of the project, both to deliver on its original social mission but also to revolutionize commercial FinTech sector the same way the big data wave caught up when Hadoop was open sourced. The architecture [Cloud Native/micro-services] is at the forefront of what a modern application should look like, and the decision to start from scratch and don’t carry the burden of legacy was a bold but very smart move imho. There is however, one area that might be interesting to challenge: the CQRS and the dual storage engine needed to provide capability for async command execution (and its draw back on error handling from front end) and sync query accessing the traditional DBMS with its scalability shortfall. The current implementation looks optimal when it was picked (I assume when fineract-cn was still mifos io), because there was no open source “NewSQL” solution available. Using classic RDBMS with known scalability limitation, coupled with NoSQL eventual consistency, no ACID, no multicolumn/table transaction, no join or grouping, given up to gain stellar horizontal scalability was a smart move. However, in 2017, Apache released Trafodion as Top Level Project. It could replace both Postgress/MariaDb/MySQL and Cassandra. It is an MPP RDBMS built on top of Hadoop Hbase (the Hadoop equivalent of Cassandra). It is fully ACID, supports multicolumn/multi table transactions, joins, group by, is fully horizontally scalable and support all the bells and whistle you would expect from a classic RDBMS. It can be classified as NewSQL, basically claiming back all the things given up by NoSql in exchange for scalability. In addition, the company backing it up (Esgyn Corp) have recently announced that its commercial product (EsgynDB, derived from Trafodion) is being used for Core Banking in 2 major Banks in China, struggling with the ever growing mobile payment volume that traditional scale up databases cannot keep up with. In addition Esgyn provides a DB as a service offering, cloud hosted fully managed (Esgyn Strato), giving the option to potential fineract-cn hosting company to “outsource” the persistence layer management and focus on the rest. Wouldn’t it be an interesting idea to hook Trafodion to Fineract-cn? I can think of the following advantages: * Simplify persistence layer ecosystem from 2 DB to 1 -> back to classic RDBMS but with full scale out capability MPP style. * Still Apache license compatible * Persistence layer ready to support challenging analytical workload (reporting use cases? - thanks for reclaiming Join/group-by at scale). * Remove the reticence of decision maker to use NoSQL in Core Banking * Performance over Cassandra [on paper, even if Trafodion was used only as a drop in replacement to Cassandra for async commands, the use of Trafodion “Align Format” feature should demonstrate higher performance for journaling, as Trafodion would write 1 BigTable cell per row, while Cassandra or Native Hbase would write 1 X nb Column cells per row]. * With a solid company already showing its commitment to core banking backing the Apache project Hope this makes sense? Eric Owhadi, https://www.linkedin.com/in/eric-owhadi-3173058/ Principal Engineer at Esgyn Corporation. message resent from personal email, the first tentative did not make it...
