Hi Dimuthu, I am + 0 if you choose to go with REST.
For Airavata the data models are complex and gateways use cases were very diverse which made thrift a good choice. For Custos in general and the credential store API in particular, I agree REST might be a good choice. Suresh > On Jul 8, 2019, at 12:52 PM, DImuthu Upeksha <[email protected]> > wrote: > > Hi Amila, Aarushi, All, > > Thanks for the clarification. I'm working on revamping current credential > store to support multiple credential types with the Vault [1] as the > backend. I will start a new thread to discuss more on that with a PR. > However do we have an idea on how our external APIs should be? Would that > be Thrift based or do we have any other reasons to look for an alternative > approach. I personally +0 for Thrift and +1 for a REST based approach > because users can easily understand the usage by looking at the > documentation. What do you think? > > [1] https://www.vaultproject.io/ > > Thanks > Dimuthu > > On Wed, Jun 26, 2019 at 3:27 AM Thejaka Amila J Kanewala < > [email protected]> wrote: > >> Hi Aarushi, >> >> Well thought questions. >> Some answers/comments inline. >> >> Thanks >> -Thejaka >> >> On Tue, Jun 25, 2019 at 6:38 AM Bisht, Aarushi <[email protected]> wrote: >> >>> Hi Thejaka, >>> >>> Thank you for the project structure. Besides the above-mentioned modules, >>> I have also added >>> >>> 1. custos-commons - for common utility methods used across custos >> services >>> 2. custos-profile-service - which has profile service APIs, handlers and >>> other utility function for user profile service, tenant profile service >> and >>> iamadminservice >>> 3. thrift-interfaces - thrift files for all models and custos APIs >>> >>> Please let me know if any reorganization is required for the above >> modules. >>> >> >> This is good. We need these. >> >> >>> >>> I also have a couple of doubts related to some of the points mentioned in >>> the mail. >>> >>> 1. As you mentioned that Custos will provide multi-tenancy >>> functionalities. If I understand correctly, this means multiple tenants >> can >>> be initialized in Custos using Tenant Profile Service(which is currently >>> present in Airavata). Since I am migrating this service to custos, I want >>> to understand what does it mean for Airavata. Since Airavata will use >>> custos profile services, does this mean all tenant operation to Airavata >>> will be delegated to Custos? >>> >> >> Yes (Assuming "tenant operation" is referring to add/delete/modify tenants, >> add users to tenants, etc.). Airavata should only work with a reference to >> the tenant (e.g., tenant id). However, we need to make sure tenant >> isolation is properly implemented in Airavata (i.e., users in tenant1 >> should not be able to access any resource belong to a user in tenant2 >> (unless if they are shared)). >> >> >>> >>> Also, I am a little confused with the terminology. Airavata is a gateway, >>> what does allowing multi-tenants in airavata mean? When "public Gateway >>> addGateway(AuthzToken authzToken, Gateway gateway)" of >> TenantProfileService >>> is invoked, does gateway here refers to tenants like PGA, Seagrid? >>> >> >> This is a very good question and I believe Marlon already answered this. As >> Marlon described, Airavata is a gateway middleware. Scientists can use >> Airavata to build gateways. Each tenant is a separate gateway. >> >> "does gateway here refer to tenants like PGA, Seagrid?" -- correct. >> >> >>> 2. There are modules related to authentication and authorization in >>> Custos. I can understand the new authenticate flow but I am not very >> clear >>> about authorization. When a user invokes an API call, its authentication >>> will be delegated by Airavata to Custos which will authenticate the user >>> using the authztoken with Keycloak and return the result to Airavata. >>> >>> In case of authorization, every Airavata API call is authorized based on >>> the group user belongs to, which is either AdminGroup, >> ReadsOnlyAdminGroup >>> or a gateway user. The user group information and what groups are >>> authorized for which API call is kept with Airavata. I do not understand >>> how will authorization in Custos work? >>> >> >> Good question and I am not 100% sure whether I know the answer for this. I >> have not touched Airavata code base for a while and I don't clearly >> understand why we maintain groups within Airavata (why not in Keycloak >> groups and roles?). Maybe Suresh/Dimuthu or Marlon can explain why we >> maintain groups in Airavata but not in Keycloak. >> >> The way I envision the authorization functionality should be implemented is >> as follows: >> >> The authorization is performed on a particular resource. A resource can be >> represented using a unique string (sometimes called "permission string") >> and this string can be associated with an API function. Sometimes these >> resources hierarchical (e.g., a directory structure) in such cases those >> string can also be hierarchical ("api/addUser" etc.). Then, there is an >> action that can be performed on a resource (e.g., "read/write/invoke" >> etc.). In an API call, we need to make sure the current user has "invoke" >> permission on the resource associated with the API. So, in summary, Custos >> need to provide an API like following for authorization: >> >> "bool isAuthorize(tenantId, userId, permissionString, action)" >> >> Airavata should invoke above Custos API upon invoking the Airavata API. The >> underlying authorization method is irrelevant to Airavata (i.e., whether >> Custos is using RBAC or fine-grained authorization etc.) and possibly above >> method should inspect all inheriting permissions through groups, roles and >> sharing as well. >> >> >>> Also, I am currently following the first approach to integrate Keycloak >>> with Custos ie using keycloak as an external service as Airavata was >> using >>> this approach. Let me know if this needs to be changed. >>> >> >> +1, let's use this approach. >> >> >>> >>> Thanks & Regards, >>> Aarushi Bisht >>> ________________________________________ >>> From: Thejaka Amila J Kanewala <[email protected]> >>> Sent: Monday, June 24, 2019 1:07 PM >>> To: [email protected] >>> Subject: Re: Module structure >>> >>> Hi Dimuthu, >>> >>> I am sorry for the late reply. I thought I play around this first and >> reply >>> to your email but I got busy during the weekend and did not get a chance >> to >>> experiment around this. Therefore, I do not want to hold you on this >>> anymore. >>> >>> Here is what I had in mind: >>> >>> Custos provides the following functionalities: >>> >>> 1. Delegate science gateway authentication >>> 2. Provide transparent resource authentication to science gateways >>> 3. Provide sharing/multi-tenancy functionalities (what is already in >>> Airavata) >>> >>> To provide some of these functionalities we need an identity server like >>> Keycloak. So there are two ways we can integrate Keycloak with Custos: 1. >>> As an external service, 2. As an embedded server running inside Custos >>> server. >>> We had different opinions about these two approaches, however, I favored >>> the second approach as it simplifies the deployment complexity (and I >> guess >>> Suresh favored the first approach). The issue with the second approach is >>> there is no straightforward way to run Keycloak inside Tomcat as an >>> embedded server. There are a couple of blog posts saying that it is >>> possible however, I did not get enough time to successfully run them >>> (theoretically this is possible). >>> >>> So, back to your question about the structure. Assuming we take the first >>> approach (deploy Keycloak as an external server), I would imagine the >>> directory structure would be something similar to the following: >>> >>> modules >>> - custos-connectors (Connectors to connect to Keycloak or any other >>> identity server) >>> - custos-authentication (Delegated authentication) >>> - custos-authorisation >>> - custos-resource-authentication (Resource authentication e.g., Amazon, >>> Google clouds, SSH etc) >>> - custos-sharing (Airavata sharing) >>> - custos-credential-store >>> - custos-dist (Distribution + tomcat server) >>> - custos-client (Custos client library which will be used by science >>> gateways) >>> - custos-test (Integration tests) >>> >>> If we use the second approach, in addition to above modules, we will >> have a >>> module called "custos-server" which includes changes to embed keycloak >>> within Tomcat. >>> >>> In either case, the above-described structure is not static, it's just a >>> starting point. We definitely need to re-organize as we develop >>> functionality. >>> >>> Thanks >>> Best Regards, >>> Thejaka Amila Kanewala, PhD >>> https://github.com/thejkane/agm >>> >>> On Sun, Jun 16, 2019 at 9:17 PM DImuthu Upeksha < >>> [email protected]> >>> wrote: >>> >>>> Hi Folks, >>>> >>>> Is there a decided module structure for the project? If you have any >>>> suggestion, please let me know. I'm working on refactoring credential >>> store >>>> to support multiple protocols (SSH, S3, Azure, Box and etc) and I >> believe >>>> that this might me useful for Custos as well. >>>> >>>> Thanks >>>> Dimuthu >>>> >>> >>> >>> -- >>> >> >> >> -- >> Best Regards, >> Thejaka Amila Kanewala, PhD >> https://github.com/thejkane/agm >>
