Hi all, @Marcus,
Thanks for your input. It answered some issues I had. Tenant Registration, User Registration, and Authentication Flow is as follows. Tenant Registration POST https://custos.scigap.org:32036/tenant-management/v1.0.0/tenant { "tenantName": "sampletenant", "requesterEmail": "[email protected]", "requesterUsername": "isjarana", "adminFirstName": "Janith", "adminLastName": "Ranawaka", "adminEmail": "[email protected]", "contacts": [ "+1812345386" ], "redirectURIs": [ "http://tenantsample.com/callback"; ], "tenantURI": "http://tenantsample.com";, "scope": "email", "domain": "tenantsample.com", "adminPassword": "1234", "adminUsername": "[email protected]" } Response { { "tenantId": "10000001", "clientId": "custos/HUGxHVZbFgY8fPiM5QeJ/10000001", "clientSecret": "403Rhpy4Z0LKoFa6Nt3uk1EsQ93h6kJy51JCZ9Ra", "msg": "Use Base64 encoded clientId:clientSecret as auth token for authorization, Credentials are activated after admin approval" } } Tenant Approval POST https://custos.scigap.org:32036/tenant-management/v1.0.0/status { "tenantId":10000001, "status":"APPROVED", "updatedBy":"Custosadmin" } { "tenantId": "10000001", "status": "APPROVED" } Add User to Tenant POST https://custos.scigap.org:32036/user-management/v1.0.0/user Header :- authorization base64Encode(custosClientId:CustosClientSec) { "username": "janith123", "firstName": "Janith", "lastName": "Ranawaka", "password": "janith123", "email": "[email protected]" } { "isRegistered": true } Enable User POST https://custos.scigap.org:32036//user-management/v1.0.0/user/activation Header :- authorization base64Encode(custosClientId:CustosClientSec) { "username": "janith123" } { "internalUserId": "janith123@10000002", "username": "janith123", "tenantId": "10000002", "firstName": "Janith", "lastName": "Ranawaka", "email": "[email protected]", "creationTime": 1.5790367e+12, "state": "ACTIVE" } Authenticate POST https://custos.scigap.org:32036/identity-management/v1.0.0/authenticate Header :- authorization base64Encode(custosClientId:CustosClientSec) { "username":"janith123", "password":"janith123" } { "accessToken": "eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJlOHY0cVpYWVVMbWNsZDQ0OGplUW9EMzE0NUppSVREYm45TTJ1bHVnSlFRIn0.eyJqdGkiOiI4OWRhNjc4Ny1iZjBmLTQ4OGYtYTJhNy01YWVhZjBmOWE3NmQiLCJleHAiOjE1NzkwMzg3NTAsIm5iZiI6MCwiaWF0IjoxNTc5MDM2OTUwLCJpc3MiOiJodHRwczovL2tleWNsb2FrLmN1c3Rvcy5zY2lnYXAub3JnOjMxMDAwL2F1dGgvcmVhbG1zLzEwMDAwMDAyIiwiYXVkIjoiYWNjb3VudCIsInN1YiI6ImQxZmVkMTQxLTAzMjktNDFiMy1hZWFkLTJmNGZlYmI0ZjQ3YyIsInR5cCI6IkJlYXJlciIsImF6cCI6InBnYSIsImF1dGhfdGltZSI6MCwic2Vzc2lvbl9zdGF0ZSI6IjAyYTkxMjUxLTQxMDEtNGU2Yi1iN2UxLTU2ZmIxMjFlNTllZiIsImFjciI6IjEiLCJhbGxvd2VkLW9yaWdpbnMiOlsiaHR0cDovL3RlbmFudHNhbXBsZS5jb20iXSwicmVhbG1fYWNjZXNzIjp7InJvbGVzIjpbIm9mZmxpbmVfYWNjZXNzIiwidW1hX2F1dGhvcml6YXRpb24iXX0sInJlc291cmNlX2FjY2VzcyI6eyJhY2NvdW50Ijp7InJvbGVzIjpbIm1hbmFnZS1hY2NvdW50IiwibWFuYWdlLWFjY291bnQtbGlua3MiLCJ2aWV3LXByb2ZpbGUiXX19LCJzY29wZSI6ImVtYWlsIHByb2ZpbGUiLCJlbWFpbF92ZXJpZmllZCI6dHJ1ZSwibmFtZSI6Ikphbml0aCBSYW5hd2FrYSIsInByZWZlcnJlZF91c2VybmFtZSI6Imphbml0aDEyMyIsImdpdmVuX25hbWUiOiJKYW5pdGgiLCJmYW1pbHlfbmFtZSI6IlJhbmF3YWthIiwiZW1haWwiOiJmZXJuYWRvQGl1LmVkdSJ9.c6NwgtzMeG33_K-JQo9ijOBPkOc-el-DB4aSSuKi_V7on8FttTvfzoYrnnwoxGXFocUVeJMdSsOVsriXu_YtPzUgYueVf0bYpPgoNQWltzGPC3jmaq1OLKlydDYbojwCoN8IhqlXjTPvKB_0L3lycaTLKFFssaZXBa0C2_cs2hnZGcE8li8Pu8FyWGB6oJMsny4sv1z_KuAYmRwou-dGPeVzTKFFiywqXRGxRqvSv_Nqc_ol54q1baS8HJWHvz98DGrhShr4EV0XUQJgo3iULQdRWvo_hmVxYJhWMBAZwmuwfJY1E2Sr_fc_IxOeHwSaPkTfdZ2fx4u-3HEHJvTNkg" , "claims": [ { "key": "username", "value": "janith123" }, { "key": "tenantId", "value": "10000002" } ] } On Mon, Jan 6, 2020 at 4:51 PM Christie, Marcus Aaron <[email protected]> wrote: > Hi Isuru, > > Sorry for the late reply. Answers inline. > > > On Dec 16, 2019, at 5:17 PM, Isuru Ranawaka <[email protected]> wrote: > > > > Hi Aarushi, Suresh > > > > I am trying to bring back the functionalities of Tenant Profile Service, > > IAM Service , User Profile Service and Group\ Sharing Service into > Custos. > > Once I am going through the existing code in airavata, I came across few > > unclear parts. I have created some ER diagrams and sequence diagrams to > > further realize the design. See DB Layer Design Part of [1]. > > > > > > 1. In tenant creation flow, tenant creation function of the > > IAMAdminService creates an admin user at keycloak and enable the user, > but > > that user is not saved in UserProfileRespository, where all keycloak > > enabled users are saved. Is this a bug ? > > In Airavata the user profile is typically created when the user first logs > in. For Custos though, it is perhaps a bug to not create the user profile > for the admin user. > > > > > 2. How do we get tenant admin password in above case ? > > > > Not quite sure what you are asking for. The tenant admin password > shouldn't be stored in plain text, but maybe you are asking for something > else. Why do you want to get the tenant admin password? > > > 3. Does tenant requester need to be registered with custos before > > requesting a tenant. (Otherwise how to access custos API without access > > token) > > > > In Airavata, a tenant requester effectively has a user account with the > super-tenant. I think something similar could be done for Custos. > > > 3. The users in UserProfileRepository (Keycloak enabled users) are not > sync > > with UserRepository and they are only synced when creating a group for a > > particular domain. Is this ok ? > > > > In Airavata, user profile is synced via event-based replication. When you > say they are only synced when creating a group, I'm not familiar with that, > so I'm not sure. > > > 4. What is the relationship between domainId and tenantId ? According to > > the current code tenantId is treated as domainId. See > > (GroupManagerServiceHandler -> createGroup) > > > > Yeah they are the same thing. Historically these identifiers are called > gatewayId or domainId or tenantId in Airavata. In Custos they should > probably be standardized to just "tenantId". > > > > > > > thanks > > Isuru > > > > [1] > > > https://docs.google.com/document/d/1QtUKTfAnIrErvtYnKo3FizCRL2GUiWikEzvBnHPXfWw/edit?usp=sharing > > > > > > > > > > > > On Sat, Nov 30, 2019 at 8:13 PM Isuru Ranawaka <[email protected]> > wrote: > > > >> > >> Hi Amila, > >> > >> Thank you very much for your comments and really helpful for further > >> enhancements. please check my inline comments. Let me know your > thoughts. > >> > >> On Sat, Nov 30, 2019 at 4:33 PM Thejaka Amila J Kanewala < > >> [email protected]> wrote: > >> > >>> Hi Isuru, > >>> > >>> First, sorry for my lack of understanding about some of the bus-words > used > >>> in this design/discussion (e.g., gRPC, Saga, Axon, Camunda etc.) > >>> Few questions and comments. > >>> > >>> 1. As per the design, we have two services (integration and atomic) and > >>> atomic services have their own database. My assumption is you will have > >>> all > >>> tenant information in the tenant db and all user information in the iam > >>> db. > >>> When you say two dbs, did you mean two MySQL instances or two > databases in > >>> a single MySQL instance? Why we need two databases (even if it is > running > >>> in a single MySQL instance)? > >>> > >> > >> The atomic services represents a set of functions which can be grouped > >> together to do some independent work . Tenant related functions are > >> grouped to > >> tenant service and IAM related functions are grouped to IAM service. > Data > >> related to tenant service are located in tenant db and all the db > >> operations of tenant db is only done through tenant service. > >> If tenant services requires an information related to a tenant which > >> resides in iam db, should call to IAM service and fetch. Tenant service > is > >> not allowed to call to iam db directly. The main reasons for this kind > of > >> decoupling is for independent development, deployment and release of > each > >> versions of services. For instance, if tenant schema is changed and > want > >> to do some change in tenant db, data migration or rollback it should > only > >> affect tenant db. it should not affect iam db. I think we can use single > >> MySQL instance. Moreover, we can use different database technologies for > >> different services, for e.g if tenant service only has direct > insertions > >> and retrievals of data with less interdependencies among data we can use > >> mongoDB over mySQL. If iam service deal with data having > interdependencies > >> and transactions we can use MySQL over mongoDB. This may help agile > >> development as well like adding new services and removing services > easily. > >> > >> > >> > >> > >>> 2. Can you conceptually explain how transaction management works (Saga, > >>> Axon, Camunda) in this solution? Preferably using some examples (e.g., > >>> tenant registration, user registration, use deletion, etc.) > >>> > >> > >> Let's consider tenant registration message flow, firstly, tenant > >> registration integration service will talk to tenant registration core > >> service to create tenant id and save tenant profile information in > tenant > >> db and subsequently tenant registration integration service will call > to > >> iam service with tenant id to create keycloak realm and save > credentials. > >> Let's say call to iam service is failed. Then we have two options to do > >> roll back tenant info from tenant db or updated tenant info with tenant > >> creation failed status. As in monolithic architecture we cannot perform > two > >> phase commits here. We can only achieved eventual consistency. Saga is > >> architectural pattern which keeps track of all passed service calls and > >> failed service calls. So we can rollback passed service calls using saga > >> logs. Axon is a framework which supports saga pattern. > >> > >> > >>> 3. At first, I thought "API gateway" refers to "Science Gateway" in > your > >>> document, apparently that is not the case (as per description API > gateway > >>> is for API management capabilities....). In that case, what is > "Consumer" > >>> in your diagram? Is that the actual "science gateway" (e.g., SeaGrid) > or > >>> "science gateway middleware" (e.g., Airavata)? > >>> > >>> Actual consumers would be integration services. API gateway is not > doing > >> any data related operation. It is just for handle SSL termination, > >> service discovery, throttling, ..etc. > >> > >> 4. Is this design only for provisioning? In the sense, are users also > >>> authenticated/authorized through this architecture? > >>> > >> > >> Users are authenticated and authorized. Actually, issuing access tokens > or > >> client credentials are done through custos services. > >> > >> > >>> 5. Regarding the asynchronous model: You probably need to consider a > way > >>> to > >>> avoid duplicate requests and should have an API call to check the > status > >>> of > >>> an asynchronous request. > >>> > >> > >> Yes we need to handle it. May be we may need some background tasks to > >> retry for failure and scenarios and update status for each task. > >> > >> > >>> I have a few more questions but let me wait for answers to the above > >>> questions. > >>> > >>> Thank you. > >>> Best Regards, > >>> Thejaka Amila Kanewala, PhD > >>> https://github.com/thejkane/agm > >>> > >>> On Wed, Nov 27, 2019 at 4:36 PM Isuru Ranawaka <[email protected]> > >>> wrote: > >>> > >>>> Hi Dimuthu, > >>>> > >>>> For instance, in tenant registration when client requests to register > a > >>>> tenant, the api should return with 200 OK saying tenant requested and > >>>> subsequently calling to keycloak, vault and iam services should be run > >>> in > >>>> background and update the status. > >>>> > >>>> thanks > >>>> > >>>> > >>>> On Wed, Nov 27, 2019 at 4:07 PM DImuthu Upeksha < > >>>> [email protected]> > >>>> wrote: > >>>> > >>>>> Hi Isuru, > >>>>> > >>>>> What are the use cases for asynchronous communication? > >>>>> > >>>>> Thanks > >>>>> Dimuthu > >>>>> > >>>>> On Wed, Nov 27, 2019 at 3:39 PM Isuru Ranawaka <[email protected]> > >>>> wrote: > >>>>> > >>>>>> Hi all, > >>>>>> > >>>>>> Just to give an update on the project, > >>>>>> > >>>>>> As per the discussion had with Suresh, Custos has synchronous > >>>>> communication > >>>>>> as well as asynchronous communication. For synchronous, there is > >>> one to > >>>>> one > >>>>>> mapping between integration service endpoint to core service > >>> endpoint. > >>>>> For > >>>>>> asynchronous communication there are many services communicate > >>>>>> asynchronously and get work done. > >>>>>> > >>>>>> I have modeled and implemented asynchronous communication as a > >>> service > >>>>>> chain and each core service is invoked via service task and > >>> integration > >>>>>> service builds a service task pipeline which is modeled as service > >>>> chain. > >>>>>> Responses are communicated back via callbacks. > >>>>>> > >>>>>> Moreover, gRPC zipkin interceptors for tracing and Prometheus > >>> related > >>>>>> configurations for metrics are added. Docker images for each > >>> service is > >>>>>> built and published using docker maven plugin and helm charts are > >>>>> packaged > >>>>>> using helm maven plugin. > >>>>>> > >>>>>> thanks > >>>>>> > >>>>>> Isuru > >>>>>> > >>>>>> > >>>>>> On Wed, Nov 20, 2019 at 3:12 PM Isuru Ranawaka <[email protected]> > >>>>> wrote: > >>>>>> > >>>>>>> Hi all, > >>>>>>> > >>>>>>> I have implemented a basic prototype [1] according to the > >>>> architecture > >>>>>>> mentioned above. > >>>>>>> > >>>>>>> custos > >>>>>>> | > >>>>>>> |_ custos-core-services > >>>>>>> | | > >>>>>>> | | _ tenant-profile-core-service > >>>>>>> | > >>>>>>> |_ java > >>>>>>> |_proto > >>>>>>> |_ resources > >>>>>>> | > >>>>>>> |_custos-core-services-client-stubs > >>>>>>> | > >>>>>>> |_ tenant-profile-core-service-client-stub > >>>>>>> | > >>>>>>> |_custos-integration-services > >>>>>>> | > >>>>>>> | _ tenant-registration-service > >>>>>>> > >>>>>>> > >>>>>>> Basically core services are gRPC services and their stubs and > >>> clients > >>>>> are > >>>>>>> generated in client-stubs module. Integration services use those > >>>> stubs > >>>>> to > >>>>>>> talk to core services. This is currently working locally and as > >>> next > >>>>>> step, > >>>>>>> need to develop docker images, k8s artifacts and run on K8s > >>> cluster > >>>> to > >>>>>>> check whether integration services resolves core services > >>> addresses > >>>>>>> seamlessly. > >>>>>>> > >>>>>>> [1]https://github.com/apache/airavata-custos/pull/14 > >>>>>>> > >>>>>>> thanks > >>>>>>> Isuru > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> On Mon, Nov 18, 2019 at 9:49 PM Isuru Ranawaka < > >>> [email protected]> > >>>>>> wrote: > >>>>>>> > >>>>>>>> Hi Suresh, > >>>>>>>> > >>>>>>>> Thanks for your feedback. I have updated the doc [1]. For > >>>>> observability > >>>>>>>> purposes we can use spring-cloud-sleuth and servers such as > >>> Zipkin > >>>> and > >>>>>>>> those are capable of distributed tracing. For vault operations we > >>>> have > >>>>>>>> separate audit feature. > >>>>>>>> > >>>>>>>> [1] > >>>>>>>> > >>>>>> > >>>>> > >>>> > >>> > https://docs.google.com/document/d/1QtUKTfAnIrErvtYnKo3FizCRL2GUiWikEzvBnHPXfWw/edit?usp=sharing > >>>>>>>> > >>>>>>>> thanks > >>>>>>>> > >>>>>>>> On Mon, Nov 18, 2019 at 6:47 PM Suresh Marru <[email protected]> > >>>>> wrote: > >>>>>>>> > >>>>>>>>> Hi Isuru, > >>>>>>>>> > >>>>>>>>> Looks like your embedded architecture diagram did not make it > >>> with > >>>>> the > >>>>>>>>> email. I will request a wiki space for Custos so you can upload > >>>> there > >>>>>> for > >>>>>>>>> future discussions. For now, can you add it to the google doc? > >>>>>>>>> > >>>>>>>>> Your overall architecture proposal seems to be good. Can you > >>> also > >>>>>>>>> consider adding logging and detailed instrumentation as part of > >>> the > >>>>>> core > >>>>>>>>> architecture? This will help to have a detailed audit of each > >>> and > >>>>>> every > >>>>>>>>> action on who did what and when. > >>>>>>>>> > >>>>>>>>> Cheers, > >>>>>>>>> Suresh > >>>>>>>>> > >>>>>>>>>> On Nov 18, 2019, at 3:41 PM, Isuru Ranawaka < > >>> [email protected]> > >>>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>> Hi all, > >>>>>>>>>> > >>>>>>>>>> I am initiating this thread to provide Custos initial version > >>>>>>>>>> implementation details. Since we are moving into cloud native > >>>>>>>>> development > >>>>>>>>>> of Custos. I have come up with the following structure for > >>> micro > >>>>>>>>> services > >>>>>>>>>> based development of Custos. > >>>>>>>>>> > >>>>>>>>>> Mainly codebase composed of the atomic services and > >>> integration > >>>>>>>>> services. > >>>>>>>>>> > >>>>>>>>>> Atomic services are independent services with minimal network > >>>>>>>>> communication > >>>>>>>>>> and they have their own database with local data. > >>>>>>>>>> > >>>>>>>>>> Integration services represents the actual business use case > >>> and > >>>> it > >>>>>>>>> acts as > >>>>>>>>>> an orchestrator between core services and external services. > >>> This > >>>>> is > >>>>>>>>>> exposed as a rest api via API Gateway. Integration services > >>> talk > >>>> to > >>>>>>>>> atomic > >>>>>>>>>> services via gRPC protocol. Integration services includes gRPC > >>>> stub > >>>>>> of > >>>>>>>>>> core services. Each integration service and core service is > >>>>> deployed > >>>>>> to > >>>>>>>>>> Kubernetes cluster as services and have their own service > >>>>> addresses. > >>>>>>>>>> > >>>>>>>>>> Main problem with this architecture is transaction management > >>>> and > >>>>> it > >>>>>>>>> is > >>>>>>>>>> planned to handle with Saga using Axon Server or Camunda like > >>>>>>>>> framework. > >>>>>>>>>> All the services are written using Spring Boot and Spring > >>> Cloud. > >>>>>>>>>> > >>>>>>>>>> All the Integration services are registered with the service > >>>>> registry > >>>>>>>>> such > >>>>>>>>>> as Consul. API Gateway is used for API Management capabilities > >>>> such > >>>>>> as > >>>>>>>>> SSL > >>>>>>>>>> Termination, throttling..etc. There are Open Source API > >>> Gateways > >>>>> such > >>>>>>>>> as > >>>>>>>>>> Ambassador, Envoy which are compatible with Kubernetes and > >>>> service > >>>>>>>>>> registries. API Gateway is responsible for service discovery. > >>>>>>>>>> > >>>>>>>>>> Vault and Keycloak are deployed in Kubernetes with HA[1]. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> Let me know your thoughts. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> [1] > >>>>>>>>>> > >>>>>>>>> > >>>>>> > >>>>> > >>>> > >>> > https://docs.google.com/document/d/1zGXlstt9w1ge8oRWjiUv3LdQETcSYtwDngm9grB5sbk/edit?usp=sharing > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> Thanks > >>>>>>>>>> > >>>>>>>>>> Isuru > >>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>>> -- > >>>>>>>> Senior Engineer > >>>>>>>> WSO2 Lanka (pvt) Ltd > >>>>>>>> Blog:- http://isurur.blogspot.com/ > >>>>>>>> Phone:- +94714629880 > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> -- > >>>>>>> Senior Engineer > >>>>>>> WSO2 Lanka (pvt) Ltd > >>>>>>> Blog:- http://isurur.blogspot.com/ > >>>>>>> Phone:- +94714629880 > >>>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> Senior Engineer > >>>>>> WSO2 Lanka (pvt) Ltd > >>>>>> Blog:- http://isurur.blogspot.com/ > >>>>>> Phone:- +94714629880 > >>>>>> > >>>>> > >>>> > >>>> > >>>> -- > >>>> Senior Engineer > >>>> WSO2 Lanka (pvt) Ltd > >>>> Blog:- http://isurur.blogspot.com/ > >>>> Phone:- +94714629880 > >>>> > >>> > >>> > >>> -- > >>> <https://github.com/thejkane/agm> > >>> > >> > >> > >> -- > >> Senior Engineer > >> WSO2 Lanka (pvt) Ltd > >> Blog:- http://isurur.blogspot.com/ > >> Phone:- +94714629880 > >> > > > > > > -- > > Senior Engineer > > WSO2 Lanka (pvt) Ltd > > Blog:- http://isurur.blogspot.com/ > > Phone:- +94714629880 > > -- Senior Engineer WSO2 Lanka (pvt) Ltd Blog:- http://isurur.blogspot.com/ Phone:- +94714629880
