HI devs,
To align with architectural refactoring suggestion, I would like to modify
the module structure as below. We no longer need different type of Gfac
providers with that.
core { has all core interfaces and basic classes of gfac-core ,
orchestrator-core , message-core , monitor core, registry core,
workflow-core}
service - all thrift services and service handlers
orchestrator - orchestrator specific classes
*task-adapter *- *replace gfac nested module with this.*
message - amqp implemention
distribution
XBaya
server - { use different mode input to start server as orchestrator ,
Gfac or/and api-server }
commons
registry
app-catalog
security
workflow
XBaya-gui
Integration-test
Thanks,
Shameera.
On Sat, May 30, 2015 at 12:34 PM, Shameera Rathnayaka <
[email protected]> wrote:
> Hi Suresh,
>
>>
>> I do not think bundling all core into one module is a good idea. When
>> there is distinct enough functionality why combine? There are advantages
>> with micro-service architecture both in terms of rapid evolution as well as
>> ease of light weight deployments. For instance lets take API server and
>> Orchestrator example. API server gets hit for all the calls, but only a
>> fraction of them are intended for Orchestrator. Even in your approach of
>> starting different services from one module, the dependencies are all class
>> loaded (unless we put sophistication in startup scripts).
>>
>
>> If two modules have significant overlapping functionality then + 1 to
>> discuss those as case by case and combine them as necessary. But can you
>> explain a little more on why you want to combine distinct functional
>> modules?
>>
>
> A good example is Apache Spark, it is a rapid evolving project, and it
> also has different major components ( Sprak SQL, Spark Streaming, MLlib and
> GraphX)
>
> but all share one core component[1]. No doubt they need light weight
> deployments more than us, as they have higher user community. So why do we
> make our repository bulky with modules unless it doesn't provide any
> considerable advantage.
>
> If we really w
> ant to
> separate distribution bundle for each component ( apiServer , Orchestrator
> and Gfac ) let
> '
> s use different bin.xml file
> s
> to do it instead of using different modules. But reality is we only use
> all in one distribution.
>
> Thanks,
> Shameera.
>
>
>>
>> Suresh
>>
>> On May 29, 2015, at 11:29 AM, Suresh Marru <[email protected]> wrote:
>>
>> + 1.
>>
>> I was planning to bring up this issue also. Probably it will not address
>> what you are raising, but here is a tree output from airavata labs code I
>> was toying with locally. I did not yet compare it with what you proposed, I
>> will do so later today.
>>
>> ├── airavata-api
>> │ ├── airavata-api-interface-descriptions
>> │ ├── airavata-api-java-stubs
>> │ ├── airavata-api-server
>> │ ├── airavata-data-models
>> │ ├── api-security-manager
>> ├── clients
>> │ ├── airavata-client-cpp-sdk
>> │ ├── airavata-client-java-sdk
>> │ ├── airavata-client-php-sdk
>> │ ├── airavata-client-python-sdk
>> │ ├── airavata-sample-examples
>> │ └── airavata-xbaya-gui
>> ├── components
>> │ ├── commons
>> │ ├── component-interface-descriptions
>> │ ├── component-services
>> │ │ ├── credential-store-service
>> │ │ ├── orchestrator-service
>> │ │ ├── task-executor-service
>> │ │ └── workflow-interpreter-service
>> │ ├── component-clients
>> │ │ ├── credential-store-client
>> │ │ ├── orchestrator-client
>> │ │ ├── task-executor-client
>> │ │ ├── workflow-interpreter-client
>> │ │ └── messaging
>> │ ├── task-adaptors
>> │ │ ├── compute
>> │ │ └── data-movement
>> │ ├── registry
>> │ │ ├── app-catalog
>> │ │ ├── experiment-catalog
>> │ │ └── resource-catalog
>> │ └── workflow-interpreter
>> ├── distribution
>> ├── integration-tests
>>
>>
>>
>> On May 29, 2015, at 10:15 AM, Shameera Rathnayaka <[email protected]>
>> wrote:
>>
>> Hi Devs,
>>
>> As we are using different modules to package different type of
>> functionalities, which will help us to maintain loosely couple codes. Now
>> the project has 49 leaf module ( one to hit half century :) ). If we allow
>> project to grow this way, having too fine grain modules will be huge
>> headache in future. IMO we should clean this ASAP before it become really
>> mess. Actually we half way there, I experienced cyclic dependency issues
>> when I was writing workflow implementation and email monitoring. Please see
>> the modules in current repo below.
>>
>> <module-name> ( <num of child modules> )
>>
>> modules ( 43 )
>> app-catalog ( 2 )
>> commons ( 1 )
>> configurations ( 2 )
>> credential-store ( 3 )
>> distribution ( 8 )
>> gfac ( 10 )
>> integration test ( 1 )
>> messaging ( 2 )
>> orchestrator ( 3 )
>> registry ( 3 )
>> security ( 1 )
>> server ( 1 )
>> test-suit ( 1 )
>> workflow ( 1 )
>> workflow-modal ( 3 )
>> xbaya ( 1 )
>> airavata-api ( 5 )
>> tools ( 1 )
>>
>> Most of the current modules have interfaces and implementations together,
>> but this violate our main goal which reduce inter module dependencies.
>> Following is what I am suggesting, WDYS?
>>
>> core { has all core interfaces and basic classes of gfac-core ,
>> orchestrator-core , message-core , monitor core, registry core,
>> workflow-core}
>> service - all thrift services and service handlers
>> orchestrator - orchestrator specific classes
>> gfac
>> SSH
>> BES
>> Local
>> message - amqp implemention
>> distribution
>> XBaya
>> server - { use different mode input to start server as orchestrator
>> , Gfac or/and api-server }
>> commons
>> registry
>> app-catalog
>> security
>> Workflow
>> XBaya-gui
>> Integration-test
>>
>> Thanks,
>> Shameera.
>>
>>
>>
>>
>
>
> --
> Best Regards,
> Shameera Rathnayaka.
>
> email: shameera AT apache.org , shameerainfo AT gmail.com
> Blog : http://shameerarathnayaka.blogspot.com/
>
--
Best Regards,
Shameera Rathnayaka.
email: shameera AT apache.org , shameerainfo AT gmail.com
Blog : http://shameerarathnayaka.blogspot.com/
