Hi Suresh, > > I do not think bundling all core into one module is a good idea. When > there is distinct enough functionality why combine? There are advantages > with micro-service architecture both in terms of rapid evolution as well as > ease of light weight deployments. For instance lets take API server and > Orchestrator example. API server gets hit for all the calls, but only a > fraction of them are intended for Orchestrator. Even in your approach of > starting different services from one module, the dependencies are all class > loaded (unless we put sophistication in startup scripts). >
> If two modules have significant overlapping functionality then + 1 to > discuss those as case by case and combine them as necessary. But can you > explain a little more on why you want to combine distinct functional > modules? > A good example is Apache Spark, it is a rapid evolving project, and it also has different major components ( Sprak SQL, Spark Streaming, MLlib and GraphX) but all share one core component[1]. No doubt they need light weight deployments more than us, as they have higher user community. So why do we make our repository bulky with modules unless it doesn't provide any considerable advantage. If we really w ant to separate distribution bundle for each component ( apiServer , Orchestrator and Gfac ) let ' s use different bin.xml file s to do it instead of using different modules. But reality is we only use all in one distribution. Thanks, Shameera. > > Suresh > > On May 29, 2015, at 11:29 AM, Suresh Marru <[email protected]> wrote: > > + 1. > > I was planning to bring up this issue also. Probably it will not address > what you are raising, but here is a tree output from airavata labs code I > was toying with locally. I did not yet compare it with what you proposed, I > will do so later today. > > ├── airavata-api > │ ├── airavata-api-interface-descriptions > │ ├── airavata-api-java-stubs > │ ├── airavata-api-server > │ ├── airavata-data-models > │ ├── api-security-manager > ├── clients > │ ├── airavata-client-cpp-sdk > │ ├── airavata-client-java-sdk > │ ├── airavata-client-php-sdk > │ ├── airavata-client-python-sdk > │ ├── airavata-sample-examples > │ └── airavata-xbaya-gui > ├── components > │ ├── commons > │ ├── component-interface-descriptions > │ ├── component-services > │ │ ├── credential-store-service > │ │ ├── orchestrator-service > │ │ ├── task-executor-service > │ │ └── workflow-interpreter-service > │ ├── component-clients > │ │ ├── credential-store-client > │ │ ├── orchestrator-client > │ │ ├── task-executor-client > │ │ ├── workflow-interpreter-client > │ │ └── messaging > │ ├── task-adaptors > │ │ ├── compute > │ │ └── data-movement > │ ├── registry > │ │ ├── app-catalog > │ │ ├── experiment-catalog > │ │ └── resource-catalog > │ └── workflow-interpreter > ├── distribution > ├── integration-tests > > > > On May 29, 2015, at 10:15 AM, Shameera Rathnayaka <[email protected]> > wrote: > > Hi Devs, > > As we are using different modules to package different type of > functionalities, which will help us to maintain loosely couple codes. Now > the project has 49 leaf module ( one to hit half century :) ). If we allow > project to grow this way, having too fine grain modules will be huge > headache in future. IMO we should clean this ASAP before it become really > mess. Actually we half way there, I experienced cyclic dependency issues > when I was writing workflow implementation and email monitoring. Please see > the modules in current repo below. > > <module-name> ( <num of child modules> ) > > modules ( 43 ) > app-catalog ( 2 ) > commons ( 1 ) > configurations ( 2 ) > credential-store ( 3 ) > distribution ( 8 ) > gfac ( 10 ) > integration test ( 1 ) > messaging ( 2 ) > orchestrator ( 3 ) > registry ( 3 ) > security ( 1 ) > server ( 1 ) > test-suit ( 1 ) > workflow ( 1 ) > workflow-modal ( 3 ) > xbaya ( 1 ) > airavata-api ( 5 ) > tools ( 1 ) > > Most of the current modules have interfaces and implementations together, > but this violate our main goal which reduce inter module dependencies. > Following is what I am suggesting, WDYS? > > core { has all core interfaces and basic classes of gfac-core , > orchestrator-core , message-core , monitor core, registry core, > workflow-core} > service - all thrift services and service handlers > orchestrator - orchestrator specific classes > gfac > SSH > BES > Local > message - amqp implemention > distribution > XBaya > server - { use different mode input to start server as orchestrator , > Gfac or/and api-server } > commons > registry > app-catalog > security > Workflow > XBaya-gui > Integration-test > > Thanks, > Shameera. > > > > -- Best Regards, Shameera Rathnayaka. email: shameera AT apache.org , shameerainfo AT gmail.com Blog : http://shameerarathnayaka.blogspot.com/
