Petri - Thank you so much. Some responses inline. I look forward to having you engaged when you can!
On Fri, Oct 15, 2021 at 10:15 PM Petri Tuomola <[email protected]> wrote: > James - thanks for sharing this > > I think there would be a lot of interesting “mini-projects” that could be > undertaken to refactor / modernise Fineract 1.x. The benefit of such an > approach would be backward compatibility, allowing us to bring improvements > to all Fineract 1.x users without a migration. > I think Aleksander had a set of smaller efforts that he wanted to push forward. His talk is at YouTube <https://www.youtube.com/watch?v=fn9uhpOWrHk&list=PLU2OcwpQkYCwZfTRhACzyM6QT1qPijPFw&index=149>. He shares his ideas there. As you state, performance is definitely one of the key areas that we should > look to address. I’m personally not certain that PostgresQL / EclipseLink > would give significantly better performance than MySQL / OpenJPA, but it > would be interesting to performance test this. However, I’m pretty sure we > can get significant performance improvements simply by tuning the existing > SQL queries / schema / business logic. > So, I think the group is in agreement that there are definitely improvements possible from tuning existing queries... which relates to the above topic of refactoring. > > But to undertake this, we’d first need a robust performance test suite in > place to establish the baseline and then to measure the improvements. > Writing the tests themselves may not be that much of a challenge - but we’d > need the infrastructure to run these in a consistent environment on a > regular basis. AWS / GCP would be one option, though it requires at least > some funding… but perhaps there are some free options out there? > We could check with Infra @ Apache to see if some perf testing suites can be implemented as part of CI/CD. I think there was some previous discussion with Infra about this, so we should revisit that before asking again... Alternatively, outside of the project, Community members may be able to get results and share them here. That may be the easier path to establish some baselines. We have some old data, but nothing from release 1.5. For example, I'm looking at Gatling, which can be implemented within Jenkins and uses the Apache2.0 license (with some bundling issue): https://gatling.io/docs/gatling/reference/current/project/licenses/. ... so, maybe that can be done outside of the project now and then code brought to the infra. The migration to PostgresQL would be interesting due to the licensing > issues with MySQL JDBC connector though. > By licensing issue, I assume you mean the fact that JDBC is GPLv2 which is a CategoryX license in Apache (not allowed), right? I started working on this earlier but didn’t get a chance to finish. Will > see if I can pick this up when I have some spare time again. > > Another possible improvement would be to see if we could rework the code > to reduce the amount of “boilerplate” code for serialisation / validation / > data access. In a typical pull request today, a large percentage of the > code is there just to add the new parameter(s) to the APIs / database / > function signatures. Perhaps with some of the modern libraries we could > reduce the amount of code that is required to simply pass data around, > making the Fineract code more focused on business functionality (and hence > easier to manage/refactor/maintain). > Can you say more about this? This would be useful to understand. Can you point to some potential new options? > > Anyway - just some thoughts. I’m unfortunately at the moment completely > swamped with work, but will see if I can pick up some of these ideas when I > have some free time again. @Aleksandar Vidakovic - it would be great to see > your list of improvement / modernisation ideas? > > Thanks > > Regards > Petri > > > On 7 Oct 2021, at 02:52, James Dailey <[email protected]> wrote: > > Devs - > > During ApacheCON, a few of the Fineract participants gathered in the usual > "Birds of a Feather" - where people of like interests flock together - and > one topic we took up was some general roadmapping for Fineract1.x > > A key discussion was around componentization of Fineract1.x, and here is > my attempt to bring this rich discussion back to the list. @Edward Cable > <[email protected]> , @Victor Manuel Romero Rodriguez > <[email protected]> , @Javier Borkenztain <[email protected]> > > Aleksandar had proposed, during his ApacheCON @home2021 talk that day, an > effort to REVISE and RATIONALIZE the underlying components to be more > "rational", "streamlined", "better at high volumes", which amounted to some > fairly substantial restructuring of the existing components. > This idea may help with the code debt that has crept into the code over > the last 10 years. It would make the project more easily maintained. > It may also allow fineract1.x to evolve into possibly a better behaving > "microservice" (fineract1.x acting as a microservice itself) or (again > maybe) a composition of microservices. > In particular, the class structure around products and portfolio was > highlighted as being highly complex to deal with. Currently, code changes > in one area could have unanticipated impacts in other areas and test > coverage is not sufficient to catch all of these issues. > So, in summary, we admired the problem a bit, and agreed to start on some > of the more manageable smaller changes, whilst still looking to a more > intensive re-write. @Aleksandar Vidakovic <[email protected]> : > are you able to put some high level Tickets in Jira for this? This will > take some serious effort and needs to be scoped out. > > Secondly, this rewrite/revision of Fineract1.x was then compared to the > FineractCN codebase strategy which began life as a rewrite of the Fineract > structure from the ground up. Some devs are currently proposing PRs > related to the core microservices (e.g. Identity) based on their > experiences using the existing code base in production. The evolution of > FineractCN into a full release will need some attention from the Community > to make happen. The proposal for how to get to a release is still at ==> > https://cwiki.apache.org/confluence/display/FINERACT/Apache+Fineract+CN+Community+Roadmap > . > Let's start a separate thread on that. > > Thirdly, going back to Fineract1.x, one of the key issues is around the > scalability and performance of the platform. A number of participants > mentioned that fineract1.x is starting to be used in increasingly > large-scale scenarios. As discussed on list previously, there are potential > strategies, including replacing OpenJPA with EclipseLink ==> > https://issues.apache.org/jira/browse/FINERACT-849, (thank you @Yemdjih > Kaze Nasser <[email protected]> ) and migrating to a Postgres option > (whilst maintaining backwards compatibility?)... > https://issues.apache.org/jira/browse/FINERACT-984. There was some > interest in moving that forward. > > There were a lot of good discussions at Birds of a Feather and some fun > moments. If we missed you this time, there's always next. > > Thank you. > James > > >
