Hi Amila, Thank you very much for your comments and really helpful for further enhancements. please check my inline comments. Let me know your thoughts.
On Sat, Nov 30, 2019 at 4:33 PM Thejaka Amila J Kanewala < [email protected]> wrote: > Hi Isuru, > > First, sorry for my lack of understanding about some of the bus-words used > in this design/discussion (e.g., gRPC, Saga, Axon, Camunda etc.) > Few questions and comments. > > 1. As per the design, we have two services (integration and atomic) and > atomic services have their own database. My assumption is you will have all > tenant information in the tenant db and all user information in the iam db. > When you say two dbs, did you mean two MySQL instances or two databases in > a single MySQL instance? Why we need two databases (even if it is running > in a single MySQL instance)? > The atomic services represents a set of functions which can be grouped together to do some independent work . Tenant related functions are grouped to tenant service and IAM related functions are grouped to IAM service. Data related to tenant service are located in tenant db and all the db operations of tenant db is only done through tenant service. If tenant services requires an information related to a tenant which resides in iam db, should call to IAM service and fetch. Tenant service is not allowed to call to iam db directly. The main reasons for this kind of decoupling is for independent development, deployment and release of each versions of services. For instance, if tenant schema is changed and want to do some change in tenant db, data migration or rollback it should only affect tenant db. it should not affect iam db. I think we can use single MySQL instance. Moreover, we can use different database technologies for different services, for e.g if tenant service only has direct insertions and retrievals of data with less interdependencies among data we can use mongoDB over mySQL. If iam service deal with data having interdependencies and transactions we can use MySQL over mongoDB. This may help agile development as well like adding new services and removing services easily. > 2. Can you conceptually explain how transaction management works (Saga, > Axon, Camunda) in this solution? Preferably using some examples (e.g., > tenant registration, user registration, use deletion, etc.) > Let's consider tenant registration message flow, firstly, tenant registration integration service will talk to tenant registration core service to create tenant id and save tenant profile information in tenant db and subsequently tenant registration integration service will call to iam service with tenant id to create keycloak realm and save credentials. Let's say call to iam service is failed. Then we have two options to do roll back tenant info from tenant db or updated tenant info with tenant creation failed status. As in monolithic architecture we cannot perform two phase commits here. We can only achieved eventual consistency. Saga is architectural pattern which keeps track of all passed service calls and failed service calls. So we can rollback passed service calls using saga logs. Axon is a framework which supports saga pattern. > 3. At first, I thought "API gateway" refers to "Science Gateway" in your > document, apparently that is not the case (as per description API gateway > is for API management capabilities....). In that case, what is "Consumer" > in your diagram? Is that the actual "science gateway" (e.g., SeaGrid) or > "science gateway middleware" (e.g., Airavata)? > > Actual consumers would be integration services. API gateway is not doing any data related operation. It is just for handle SSL termination, service discovery, throttling, ..etc. 4. Is this design only for provisioning? In the sense, are users also > authenticated/authorized through this architecture? > Users are authenticated and authorized. Actually, issuing access tokens or client credentials are done through custos services. > 5. Regarding the asynchronous model: You probably need to consider a way to > avoid duplicate requests and should have an API call to check the status of > an asynchronous request. > Yes we need to handle it. May be we may need some background tasks to retry for failure and scenarios and update status for each task. > I have a few more questions but let me wait for answers to the above > questions. > > Thank you. > Best Regards, > Thejaka Amila Kanewala, PhD > https://github.com/thejkane/agm > > On Wed, Nov 27, 2019 at 4:36 PM Isuru Ranawaka <[email protected]> wrote: > > > Hi Dimuthu, > > > > For instance, in tenant registration when client requests to register a > > tenant, the api should return with 200 OK saying tenant requested and > > subsequently calling to keycloak, vault and iam services should be run in > > background and update the status. > > > > thanks > > > > > > On Wed, Nov 27, 2019 at 4:07 PM DImuthu Upeksha < > > [email protected]> > > wrote: > > > > > Hi Isuru, > > > > > > What are the use cases for asynchronous communication? > > > > > > Thanks > > > Dimuthu > > > > > > On Wed, Nov 27, 2019 at 3:39 PM Isuru Ranawaka <[email protected]> > > wrote: > > > > > > > Hi all, > > > > > > > > Just to give an update on the project, > > > > > > > > As per the discussion had with Suresh, Custos has synchronous > > > communication > > > > as well as asynchronous communication. For synchronous, there is one > to > > > one > > > > mapping between integration service endpoint to core service > endpoint. > > > For > > > > asynchronous communication there are many services communicate > > > > asynchronously and get work done. > > > > > > > > I have modeled and implemented asynchronous communication as a > service > > > > chain and each core service is invoked via service task and > integration > > > > service builds a service task pipeline which is modeled as service > > chain. > > > > Responses are communicated back via callbacks. > > > > > > > > Moreover, gRPC zipkin interceptors for tracing and Prometheus related > > > > configurations for metrics are added. Docker images for each service > is > > > > built and published using docker maven plugin and helm charts are > > > packaged > > > > using helm maven plugin. > > > > > > > > thanks > > > > > > > > Isuru > > > > > > > > > > > > On Wed, Nov 20, 2019 at 3:12 PM Isuru Ranawaka <[email protected]> > > > wrote: > > > > > > > > > Hi all, > > > > > > > > > > I have implemented a basic prototype [1] according to the > > architecture > > > > > mentioned above. > > > > > > > > > > custos > > > > > | > > > > > |_ custos-core-services > > > > > | | > > > > > | | _ tenant-profile-core-service > > > > > | > > > > > |_ java > > > > > |_proto > > > > > |_ resources > > > > > | > > > > > |_custos-core-services-client-stubs > > > > > | > > > > > |_ tenant-profile-core-service-client-stub > > > > > | > > > > > |_custos-integration-services > > > > > | > > > > > | _ tenant-registration-service > > > > > > > > > > > > > > > Basically core services are gRPC services and their stubs and > clients > > > are > > > > > generated in client-stubs module. Integration services use those > > stubs > > > to > > > > > talk to core services. This is currently working locally and as > next > > > > step, > > > > > need to develop docker images, k8s artifacts and run on K8s cluster > > to > > > > > check whether integration services resolves core services > addresses > > > > > seamlessly. > > > > > > > > > > [1]https://github.com/apache/airavata-custos/pull/14 > > > > > > > > > > thanks > > > > > Isuru > > > > > > > > > > > > > > > > > > > > On Mon, Nov 18, 2019 at 9:49 PM Isuru Ranawaka <[email protected] > > > > > > wrote: > > > > > > > > > >> Hi Suresh, > > > > >> > > > > >> Thanks for your feedback. I have updated the doc [1]. For > > > observability > > > > >> purposes we can use spring-cloud-sleuth and servers such as Zipkin > > and > > > > >> those are capable of distributed tracing. For vault operations we > > have > > > > >> separate audit feature. > > > > >> > > > > >> [1] > > > > >> > > > > > > > > > > https://docs.google.com/document/d/1QtUKTfAnIrErvtYnKo3FizCRL2GUiWikEzvBnHPXfWw/edit?usp=sharing > > > > >> > > > > >> thanks > > > > >> > > > > >> On Mon, Nov 18, 2019 at 6:47 PM Suresh Marru <[email protected]> > > > wrote: > > > > >> > > > > >>> Hi Isuru, > > > > >>> > > > > >>> Looks like your embedded architecture diagram did not make it > with > > > the > > > > >>> email. I will request a wiki space for Custos so you can upload > > there > > > > for > > > > >>> future discussions. For now, can you add it to the google doc? > > > > >>> > > > > >>> Your overall architecture proposal seems to be good. Can you also > > > > >>> consider adding logging and detailed instrumentation as part of > the > > > > core > > > > >>> architecture? This will help to have a detailed audit of each > and > > > > every > > > > >>> action on who did what and when. > > > > >>> > > > > >>> Cheers, > > > > >>> Suresh > > > > >>> > > > > >>> > On Nov 18, 2019, at 3:41 PM, Isuru Ranawaka < > [email protected]> > > > > >>> wrote: > > > > >>> > > > > > >>> > Hi all, > > > > >>> > > > > > >>> > I am initiating this thread to provide Custos initial version > > > > >>> > implementation details. Since we are moving into cloud native > > > > >>> development > > > > >>> > of Custos. I have come up with the following structure for > micro > > > > >>> services > > > > >>> > based development of Custos. > > > > >>> > > > > > >>> > Mainly codebase composed of the atomic services and > integration > > > > >>> services. > > > > >>> > > > > > >>> > Atomic services are independent services with minimal network > > > > >>> communication > > > > >>> > and they have their own database with local data. > > > > >>> > > > > > >>> > Integration services represents the actual business use case > and > > it > > > > >>> acts as > > > > >>> > an orchestrator between core services and external services. > This > > > is > > > > >>> > exposed as a rest api via API Gateway. Integration services > talk > > to > > > > >>> atomic > > > > >>> > services via gRPC protocol. Integration services includes gRPC > > stub > > > > of > > > > >>> > core services. Each integration service and core service is > > > deployed > > > > to > > > > >>> > Kubernetes cluster as services and have their own service > > > addresses. > > > > >>> > > > > > >>> > Main problem with this architecture is transaction management > > and > > > it > > > > >>> is > > > > >>> > planned to handle with Saga using Axon Server or Camunda like > > > > >>> framework. > > > > >>> > All the services are written using Spring Boot and Spring > Cloud. > > > > >>> > > > > > >>> > All the Integration services are registered with the service > > > registry > > > > >>> such > > > > >>> > as Consul. API Gateway is used for API Management capabilities > > such > > > > as > > > > >>> SSL > > > > >>> > Termination, throttling..etc. There are Open Source API > Gateways > > > such > > > > >>> as > > > > >>> > Ambassador, Envoy which are compatible with Kubernetes and > > service > > > > >>> > registries. API Gateway is responsible for service discovery. > > > > >>> > > > > > >>> > Vault and Keycloak are deployed in Kubernetes with HA[1]. > > > > >>> > > > > > >>> > > > > > >>> > Let me know your thoughts. > > > > >>> > > > > > >>> > > > > > >>> > > > > > >>> > [1] > > > > >>> > > > > > >>> > > > > > > > > > > https://docs.google.com/document/d/1zGXlstt9w1ge8oRWjiUv3LdQETcSYtwDngm9grB5sbk/edit?usp=sharing > > > > >>> > > > > > >>> > > > > > >>> > Thanks > > > > >>> > > > > > >>> > Isuru > > > > >>> > > > > >>> > > > > >> > > > > >> -- > > > > >> Senior Engineer > > > > >> WSO2 Lanka (pvt) Ltd > > > > >> Blog:- http://isurur.blogspot.com/ > > > > >> Phone:- +94714629880 > > > > >> > > > > > > > > > > > > > > > -- > > > > > Senior Engineer > > > > > WSO2 Lanka (pvt) Ltd > > > > > Blog:- http://isurur.blogspot.com/ > > > > > Phone:- +94714629880 > > > > > > > > > > > > > > > > > -- > > > > Senior Engineer > > > > WSO2 Lanka (pvt) Ltd > > > > Blog:- http://isurur.blogspot.com/ > > > > Phone:- +94714629880 > > > > > > > > > > > > > -- > > Senior Engineer > > WSO2 Lanka (pvt) Ltd > > Blog:- http://isurur.blogspot.com/ > > Phone:- +94714629880 > > > > > -- > <https://github.com/thejkane/agm> > -- Senior Engineer WSO2 Lanka (pvt) Ltd Blog:- http://isurur.blogspot.com/ Phone:- +94714629880
