Re: Logical dependency tree for ofbiz components OFBIZ-12309
Hello Taher, Thanks for getting back. I think we are very much aligned on where we want to go but we have different perspectives on where we are in regards to OFBiz. I will motivate my reply inline: On 29.11.2021 10:36, Taher Alkhateeb wrote: Hello, I may be misunderstanding your point, but ofbiz is already broken down to sub-projects and the sub-projects are defined from the framework itself (check common.gradle). And it is possible to publish ofbiz parts separately, and also possible to isolate dependencies in each project and it is already done for some components. We also do already have a plugin mechanism that allows you to publish things separately. I disagree with your statement above. From my experience ofbiz is not split up in separate modules. If you read https://issues.apache.org/jira/browse/OFBIZ-12308 and comments from https://github.com/apache/ofbiz-framework/pull/319 you will see what I mean. There are circular dependencies between modules - so they are not split up. Also if you look at https://issues.apache.org/jira/browse/OFBIZ-12263 you can't build plugins that use other repositories, unless you add those repos (and I believe also dependencies) to OFBIz. Basically you can't publish binary plugins easily IMO. In order to fix these I believe we need to use gradle sub-projects and have a full fledged build.gradle in each directory (full fledged gradle project) complete with dependencies. Maybe there is little value now in working on Gradle beyond basic maintenance. The real hard-work that is probably badly needed is in breaking down the components themselves into layers that makes it easy to extend the framework at multiple levels. And to get rid of many things that piled up over the years. If you can get the system to work well at each of the below layers then extensibility might become an easier problem: 1. framework 2. data model -> framework 3. data-model services -> data -model 5. data-model UI -> data-model & data-model services 6. applications (I don't mean accounting, party, but rather a full application that utilizes the above layers) 7. plugins (pick and choose whatever layers needed) My 2 cents. I do agree we should allow multiple aggregation of these modules in different configurations. That will allow us to build custom OFBiz distributions. As I stated in https://github.com/apache/ofbiz-framework/pull/319 , this is not possible since modules are interconnected (circular dependencies) and you can't split them apart. You can't publish and independently use framework/entities or services or base because they use classes from each other. Also the gradle build system is highly customized and hence not very friendly to new developers because - since they need to understand the 1000+ lines of custom build code. My proposal on this part is to: * Drop dynamic project add via activeComponents (maybe keep for plugins ?!) * Use standard build.gradle in every project - static project composition * Gradually refactor code -> move dependencies into each sub-project * Deprecate framework/start and replace with top level ofbiz-start gradle project that will depend and aggregate ofbiz modules and build the app. This will be the deliverable and also example on how to build your own ofbiz distribution. People should be able to customize or create their own copy of this project to have a custom OFBiz distribution with whatever modules and configuration they desire. If we publish jars for modules, this project could be anywhere on git(hub). Cheers, Taher Alkhateeb Regards, -- Eugen Stan +40770 941 271 / https://www.netdava.combegin:vcard fn:Eugen Stan n:Stan;Eugen email;internet:eugen.s...@netdava.com tel;cell:+40720898747 x-mozilla-html:FALSE url:https://www.netdava.com version:2.1 end:vcard
Re: Logical dependency tree for ofbiz components OFBIZ-12309
Hello, I may be misunderstanding your point, but ofbiz is already broken down to sub-projects and the sub-projects are defined from the framework itself (check common.gradle). And it is possible to publish ofbiz parts separately, and also possible to isolate dependencies in each project and it is already done for some components. We also do already have a plugin mechanism that allows you to publish things separately. Maybe there is little value now in working on Gradle beyond basic maintenance. The real hard-work that is probably badly needed is in breaking down the components themselves into layers that makes it easy to extend the framework at multiple levels. And to get rid of many things that piled up over the years. If you can get the system to work well at each of the below layers then extensibility might become an easier problem: 1. framework 2. data model -> framework 3. data-model services -> data -model 5. data-model UI -> data-model & data-model services 6. applications (I don't mean accounting, party, but rather a full application that utilizes the above layers) 7. plugins (pick and choose whatever layers needed) My 2 cents. Cheers, Taher Alkhateeb On 11/29/21 03:28, Eugen Stan wrote: Hi, I think I made some progress while reverse engineering how ofbiz is built and I decided to share this so on the issue https://issues.apache.org/jira/browse/OFBIZ-3500 My train of thought is to split the current setup first into 4 static gradle sub-projects: * framework * application * themes * plugins - we should add plugins dynamically first. Each sub-project should have it's own dependencies {}, source-sets {} etc. We can then depend these sub-projects in the main project. Benefits we get from this change: * we get closer to publish ofbiz parts as libraries . * we get closer to smaller components - which are easier to maintain * we get better dependency management - and closer into splitting the ofbiz monolith into smaller services Please let me know how you feel about this.
Re: Logical dependency tree for ofbiz components OFBIZ-12309
Hi, I think I made some progress while reverse engineering how ofbiz is built and I decided to share this so on the issue https://issues.apache.org/jira/browse/OFBIZ-3500 My train of thought is to split the current setup first into 4 static gradle sub-projects: * framework * application * themes * plugins - we should add plugins dynamically first Each sub-project should have it's own dependencies {}, source-sets {} etc. We can then depend these sub-projects in the main project. Benefits we get from this change: * we get closer to publish ofbiz parts as libraries . * we get closer to smaller components - which are easier to maintain * we get better dependency management - and closer into splitting the ofbiz monolith into smaller services Please let me know how you feel about this. -- Eugen Stan +40770 941 271 / https://www.netdava.combegin:vcard fn:Eugen Stan n:Stan;Eugen email;internet:eugen.s...@netdava.com tel;cell:+40720898747 x-mozilla-html:FALSE url:https://www.netdava.com version:2.1 end:vcard
Re: Logical dependency tree for ofbiz components OFBIZ-12309
Hello Eugen, [...] > > I have on my mind to elaborate a framework-core an acceptable minimal > > circular dependencies and some other present as three dependency > > > >* core : common - base - start - entity - service - security - webapp > >* entityext : core > >* datafile: core > >* testtools: core > >* catalina: core > >* widget: core > >* webtools: widget, testtools, datafile, catalina > > > > I know this isn't your final objective but can it be an acceptable spot ? > > Thanks. It's a very good place to start. > We might be able to introduce a 'core' gradle project that contains those > and depend on that one in the others. > > I would move out start module. > It seems to be responsible for starting up the app. > I imagine components have an interface (the Container). > All ofbiz plugins/modules implement this so there should not be a dependency > on implementations. > Perhaps this is a good point to start - separating the container and > depending on it in other modules. > > start will start ofbiz and load modules list from xml + look them up on the > classpath. > > WDYT? How would the list above change with a container API module? About dependency between base and start. Today, Start is packaged in `org.apache.ofbiz.base`, it seems to have been designed to be the only alternative for OFBiz server launch (no other implementation possible). Indeed, this component is the one that stock the launch information (portoffset etc.). To make it happen, there have to be defined interfaces from which actual codebase would be one implementation. Is that what you envision, if so this could be a good first example of circular dependency removal. Other example, regarding common component, that contains common elements applicable to any other component like Unit of measure, lookups, emails. For those, the services can be based upon screen to generate the desired results inducing a dependency toward the widget component. Moreover, lots of display information are propagated by Visual theme, (during login, emails, freemarker engine preparation). We might need to integrate all of these into base component using interface OR split into the good/new component OR decide that OFBiz nature is to live with widgets in its core. > > Another helpful thing is to establish the boundaries of each project. > What should each module do and what it should NOT do? * service : contains all soa configuration and engine to manage services * entity : contains all database / entity management * common : contains functional widget, data, services that can be used by all other components * base : contains technical low level code for all OFBiz components * start : launching, stopping OFBiz server instance, managing start configuration * security : as is named, tools to manage user security * webapp : engine to manage exchange between servlet and controller definition * entityext : container that extend the entity engine on some element not vital for OFBiz to run like entity sync, cache distribution, ... * datafile: tools to create/read big data file base on xml definition * testtools: integration test engine * catalina: container for embedded tomcat * widget: screen rendering based on xml definition * webtools: web interface to analyse and manage different OFBiz engines Thanks Eugen, Nicolas and Gil signature.asc Description: PGP signature
Re: Logical dependency tree for ofbiz components OFBIZ-12309
Hello Nicolas, Thank you for looking into this. I know it's not an easy task. I believe that where is a will, there is a way. Now let's see if the OFBiz community has a strong will in this direction. Let's discuss about your proposal and let's see where it goes. See my comments inline: On 02.09.2021 17:43, Nicolas Malin wrote: Hello Eugen, On 30/08/2021 11:48, Eugen Stan wrote: Hello, I've opened a new issue https://issues.apache.org/jira/browse/OFBIZ-12309 . I need community help with this. I analyzed at fresh head, It's a hard task to remove the circular dependency. Yes it is :(. We have some dependencies that would be easily to remove (and some introduce by myself I realized by misunderstanding or weakness of mind) and some other that have their own reason. If we want to go on increase the scalability, I search if the effort to scale at each component is coherent with the most-valuable to win. I'm mostly concerned with project maintainability and innovation - both are hard to do in a large project with unclear dependencies. I have on my mind to elaborate a framework-core an acceptable minimal circular dependencies and some other present as three dependency * core : common - base - start - entity - service - security - webapp * entityext : core * datafile: core * testtools: core * catalina: core * widget: core * webtools: widget, testtools, datafile, catalina I know this isn't your final objective but can it be an acceptable spot ? Thanks. It's a very good place to start. We might be able to introduce a 'core' gradle project that contains those and depend on that one in the others. I would move out start module. It seems to be responsible for starting up the app. I imagine components have an interface (the Container). All ofbiz plugins/modules implement this so there should not be a dependency on implementations. Perhaps this is a good point to start - separating the container and depending on it in other modules. start will start ofbiz and load modules list from xml + look them up on the classpath. WDYT? How would the list above change with a container API module? Another helpful thing is to establish the boundaries of each project. What should each module do and what it should NOT do? I think a lot of things will clear up once that is established. The functionality that does not fit in the module/project should be moved elsewhere. Can you (or anyone else) (please) provide some lines as to what should each module in framework do and NOT do? Eugen Nicolas From the exploratory branch https://github.com/apache/ofbiz-framework/pull/319 I discovered there are a lot of circular dependencies between components (components depending on each other in order to build). I believe it would be very useful to have a logical dependency tree between components. As a developer working to make OFBiz usable as a framework I need to solve issues like circular dependencies between components (see https://issues.apache.org/jira/browse/OFBIZ-12308 ) . This should serve as a guide to help me decide how to solve the circular dependency issue. This is the current list of dependencies in framework (applications should be in another issue IMO). Related work: * https://issues.apache.org/jira/browse/OFBIZ-3500 * https://cwiki.apache.org/confluence/display/OFBIZ/Ofbiz+as+a+development+framework+-+release+9.04 * https://cwiki.apache.org/confluence/display/OFBIZ/Ofbiz+as+a+development+framework+-+release+10.04 Looking forward to feedback on this. Regards, -- Eugen Stan +40720 898 747 / netdava.com
Re: Logical dependency tree for ofbiz components OFBIZ-12309
Hello Eugen, On 30/08/2021 11:48, Eugen Stan wrote: > Hello, > > I've opened a new issue > https://issues.apache.org/jira/browse/OFBIZ-12309 . > > I need community help with this. I analyzed at fresh head, It's a hard task to remove the circular dependency. We have some dependencies that would be easily to remove (and some introduce by myself I realized by misunderstanding or weakness of mind) and some other that have their own reason. If we want to go on increase the scalability, I search if the effort to scale at each component is coherent with the most-valuable to win. I have on my mind to elaborate a framework-core an acceptable minimal circular dependencies and some other present as three dependency * core : common - base - start - entity - service - security - webapp * entityext : core * datafile: core * testtools: core * catalina: core * widget: core * webtools: widget, testtools, datafile, catalina I know this isn't your final objective but can it be an acceptable spot ? Nicolas > > From the exploratory > branch https://github.com/apache/ofbiz-framework/pull/319 I > discovered there are a lot of circular dependencies between components > (components depending on each other in order to build). > > I believe it would be very useful to have a logical dependency tree > between components. > > As a developer working to make OFBiz usable as a framework I need to > solve issues like circular dependencies between components > (see https://issues.apache.org/jira/browse/OFBIZ-12308 ) . > This should serve as a guide to help me decide how to solve the > circular dependency issue. > > This is the current list of dependencies in framework (applications > should be in another issue IMO). > > > > > > > > > > > > > > > > > Related work: > > * https://issues.apache.org/jira/browse/OFBIZ-3500 > * > https://cwiki.apache.org/confluence/display/OFBIZ/Ofbiz+as+a+development+framework+-+release+9.04 > * > https://cwiki.apache.org/confluence/display/OFBIZ/Ofbiz+as+a+development+framework+-+release+10.04 > > > Looking forward to feedback on this. > > Regards,
Logical dependency tree for ofbiz components OFBIZ-12309
Hello, I've opened a new issue https://issues.apache.org/jira/browse/OFBIZ-12309 . I need community help with this. From the exploratory branch https://github.com/apache/ofbiz-framework/pull/319 I discovered there are a lot of circular dependencies between components (components depending on each other in order to build). I believe it would be very useful to have a logical dependency tree between components. As a developer working to make OFBiz usable as a framework I need to solve issues like circular dependencies between components (see https://issues.apache.org/jira/browse/OFBIZ-12308 ) . This should serve as a guide to help me decide how to solve the circular dependency issue. This is the current list of dependencies in framework (applications should be in another issue IMO). Related work: * https://issues.apache.org/jira/browse/OFBIZ-3500 * https://cwiki.apache.org/confluence/display/OFBIZ/Ofbiz+as+a+development+framework+-+release+9.04 * https://cwiki.apache.org/confluence/display/OFBIZ/Ofbiz+as+a+development+framework+-+release+10.04 Looking forward to feedback on this. Regards, -- Eugen Stan +40720 898 747 / netdava.com