Hi Dimuthu, For instance, in tenant registration when client requests to register a tenant, the api should return with 200 OK saying tenant requested and subsequently calling to keycloak, vault and iam services should be run in background and update the status.
thanks On Wed, Nov 27, 2019 at 4:07 PM DImuthu Upeksha <[email protected]> wrote: > Hi Isuru, > > What are the use cases for asynchronous communication? > > Thanks > Dimuthu > > On Wed, Nov 27, 2019 at 3:39 PM Isuru Ranawaka <[email protected]> wrote: > > > Hi all, > > > > Just to give an update on the project, > > > > As per the discussion had with Suresh, Custos has synchronous > communication > > as well as asynchronous communication. For synchronous, there is one to > one > > mapping between integration service endpoint to core service endpoint. > For > > asynchronous communication there are many services communicate > > asynchronously and get work done. > > > > I have modeled and implemented asynchronous communication as a service > > chain and each core service is invoked via service task and integration > > service builds a service task pipeline which is modeled as service chain. > > Responses are communicated back via callbacks. > > > > Moreover, gRPC zipkin interceptors for tracing and Prometheus related > > configurations for metrics are added. Docker images for each service is > > built and published using docker maven plugin and helm charts are > packaged > > using helm maven plugin. > > > > thanks > > > > Isuru > > > > > > On Wed, Nov 20, 2019 at 3:12 PM Isuru Ranawaka <[email protected]> > wrote: > > > > > Hi all, > > > > > > I have implemented a basic prototype [1] according to the architecture > > > mentioned above. > > > > > > custos > > > | > > > |_ custos-core-services > > > | | > > > | | _ tenant-profile-core-service > > > | > > > |_ java > > > |_proto > > > |_ resources > > > | > > > |_custos-core-services-client-stubs > > > | > > > |_ tenant-profile-core-service-client-stub > > > | > > > |_custos-integration-services > > > | > > > | _ tenant-registration-service > > > > > > > > > Basically core services are gRPC services and their stubs and clients > are > > > generated in client-stubs module. Integration services use those stubs > to > > > talk to core services. This is currently working locally and as next > > step, > > > need to develop docker images, k8s artifacts and run on K8s cluster to > > > check whether integration services resolves core services addresses > > > seamlessly. > > > > > > [1]https://github.com/apache/airavata-custos/pull/14 > > > > > > thanks > > > Isuru > > > > > > > > > > > > On Mon, Nov 18, 2019 at 9:49 PM Isuru Ranawaka <[email protected]> > > wrote: > > > > > >> Hi Suresh, > > >> > > >> Thanks for your feedback. I have updated the doc [1]. For > observability > > >> purposes we can use spring-cloud-sleuth and servers such as Zipkin and > > >> those are capable of distributed tracing. For vault operations we have > > >> separate audit feature. > > >> > > >> [1] > > >> > > > https://docs.google.com/document/d/1QtUKTfAnIrErvtYnKo3FizCRL2GUiWikEzvBnHPXfWw/edit?usp=sharing > > >> > > >> thanks > > >> > > >> On Mon, Nov 18, 2019 at 6:47 PM Suresh Marru <[email protected]> > wrote: > > >> > > >>> Hi Isuru, > > >>> > > >>> Looks like your embedded architecture diagram did not make it with > the > > >>> email. I will request a wiki space for Custos so you can upload there > > for > > >>> future discussions. For now, can you add it to the google doc? > > >>> > > >>> Your overall architecture proposal seems to be good. Can you also > > >>> consider adding logging and detailed instrumentation as part of the > > core > > >>> architecture? This will help to have a detailed audit of each and > > every > > >>> action on who did what and when. > > >>> > > >>> Cheers, > > >>> Suresh > > >>> > > >>> > On Nov 18, 2019, at 3:41 PM, Isuru Ranawaka <[email protected]> > > >>> wrote: > > >>> > > > >>> > Hi all, > > >>> > > > >>> > I am initiating this thread to provide Custos initial version > > >>> > implementation details. Since we are moving into cloud native > > >>> development > > >>> > of Custos. I have come up with the following structure for micro > > >>> services > > >>> > based development of Custos. > > >>> > > > >>> > Mainly codebase composed of the atomic services and integration > > >>> services. > > >>> > > > >>> > Atomic services are independent services with minimal network > > >>> communication > > >>> > and they have their own database with local data. > > >>> > > > >>> > Integration services represents the actual business use case and it > > >>> acts as > > >>> > an orchestrator between core services and external services. This > is > > >>> > exposed as a rest api via API Gateway. Integration services talk to > > >>> atomic > > >>> > services via gRPC protocol. Integration services includes gRPC stub > > of > > >>> > core services. Each integration service and core service is > deployed > > to > > >>> > Kubernetes cluster as services and have their own service > addresses. > > >>> > > > >>> > Main problem with this architecture is transaction management and > it > > >>> is > > >>> > planned to handle with Saga using Axon Server or Camunda like > > >>> framework. > > >>> > All the services are written using Spring Boot and Spring Cloud. > > >>> > > > >>> > All the Integration services are registered with the service > registry > > >>> such > > >>> > as Consul. API Gateway is used for API Management capabilities such > > as > > >>> SSL > > >>> > Termination, throttling..etc. There are Open Source API Gateways > such > > >>> as > > >>> > Ambassador, Envoy which are compatible with Kubernetes and service > > >>> > registries. API Gateway is responsible for service discovery. > > >>> > > > >>> > Vault and Keycloak are deployed in Kubernetes with HA[1]. > > >>> > > > >>> > > > >>> > Let me know your thoughts. > > >>> > > > >>> > > > >>> > > > >>> > [1] > > >>> > > > >>> > > > https://docs.google.com/document/d/1zGXlstt9w1ge8oRWjiUv3LdQETcSYtwDngm9grB5sbk/edit?usp=sharing > > >>> > > > >>> > > > >>> > Thanks > > >>> > > > >>> > Isuru > > >>> > > >>> > > >> > > >> -- > > >> Senior Engineer > > >> WSO2 Lanka (pvt) Ltd > > >> Blog:- http://isurur.blogspot.com/ > > >> Phone:- +94714629880 > > >> > > > > > > > > > -- > > > Senior Engineer > > > WSO2 Lanka (pvt) Ltd > > > Blog:- http://isurur.blogspot.com/ > > > Phone:- +94714629880 > > > > > > > > > -- > > Senior Engineer > > WSO2 Lanka (pvt) Ltd > > Blog:- http://isurur.blogspot.com/ > > Phone:- +94714629880 > > > -- Senior Engineer WSO2 Lanka (pvt) Ltd Blog:- http://isurur.blogspot.com/ Phone:- +94714629880
