I am indifferent about the choice. Honestly, i dont have much recent experience with either, though i see lot of people use REST. So chose whichever you are comfortable with.
Thejaka On Mon, Jul 8, 2019, 8:11 PM Suresh Marru <[email protected]> wrote: > Hi Dimuthu, > > I am + 0 if you choose to go with REST. > > For Airavata the data models are complex and gateways use cases were very > diverse which made thrift a good choice. For Custos in general and the > credential store API in particular, I agree REST might be a good choice. > > Suresh > > > On Jul 8, 2019, at 12:52 PM, DImuthu Upeksha <[email protected]> > wrote: > > > > Hi Amila, Aarushi, All, > > > > Thanks for the clarification. I'm working on revamping current credential > > store to support multiple credential types with the Vault [1] as the > > backend. I will start a new thread to discuss more on that with a PR. > > However do we have an idea on how our external APIs should be? Would that > > be Thrift based or do we have any other reasons to look for an > alternative > > approach. I personally +0 for Thrift and +1 for a REST based approach > > because users can easily understand the usage by looking at the > > documentation. What do you think? > > > > [1] https://www.vaultproject.io/ > > > > Thanks > > Dimuthu > > > > On Wed, Jun 26, 2019 at 3:27 AM Thejaka Amila J Kanewala < > > [email protected]> wrote: > > > >> Hi Aarushi, > >> > >> Well thought questions. > >> Some answers/comments inline. > >> > >> Thanks > >> -Thejaka > >> > >> On Tue, Jun 25, 2019 at 6:38 AM Bisht, Aarushi <[email protected]> wrote: > >> > >>> Hi Thejaka, > >>> > >>> Thank you for the project structure. Besides the above-mentioned > modules, > >>> I have also added > >>> > >>> 1. custos-commons - for common utility methods used across custos > >> services > >>> 2. custos-profile-service - which has profile service APIs, handlers > and > >>> other utility function for user profile service, tenant profile service > >> and > >>> iamadminservice > >>> 3. thrift-interfaces - thrift files for all models and custos APIs > >>> > >>> Please let me know if any reorganization is required for the above > >> modules. > >>> > >> > >> This is good. We need these. > >> > >> > >>> > >>> I also have a couple of doubts related to some of the points mentioned > in > >>> the mail. > >>> > >>> 1. As you mentioned that Custos will provide multi-tenancy > >>> functionalities. If I understand correctly, this means multiple tenants > >> can > >>> be initialized in Custos using Tenant Profile Service(which is > currently > >>> present in Airavata). Since I am migrating this service to custos, I > want > >>> to understand what does it mean for Airavata. Since Airavata will use > >>> custos profile services, does this mean all tenant operation to > Airavata > >>> will be delegated to Custos? > >>> > >> > >> Yes (Assuming "tenant operation" is referring to add/delete/modify > tenants, > >> add users to tenants, etc.). Airavata should only work with a reference > to > >> the tenant (e.g., tenant id). However, we need to make sure tenant > >> isolation is properly implemented in Airavata (i.e., users in tenant1 > >> should not be able to access any resource belong to a user in tenant2 > >> (unless if they are shared)). > >> > >> > >>> > >>> Also, I am a little confused with the terminology. Airavata is a > gateway, > >>> what does allowing multi-tenants in airavata mean? When "public Gateway > >>> addGateway(AuthzToken authzToken, Gateway gateway)" of > >> TenantProfileService > >>> is invoked, does gateway here refers to tenants like PGA, Seagrid? > >>> > >> > >> This is a very good question and I believe Marlon already answered > this. As > >> Marlon described, Airavata is a gateway middleware. Scientists can use > >> Airavata to build gateways. Each tenant is a separate gateway. > >> > >> "does gateway here refer to tenants like PGA, Seagrid?" -- correct. > >> > >> > >>> 2. There are modules related to authentication and authorization in > >>> Custos. I can understand the new authenticate flow but I am not very > >> clear > >>> about authorization. When a user invokes an API call, its > authentication > >>> will be delegated by Airavata to Custos which will authenticate the > user > >>> using the authztoken with Keycloak and return the result to Airavata. > >>> > >>> In case of authorization, every Airavata API call is authorized based > on > >>> the group user belongs to, which is either AdminGroup, > >> ReadsOnlyAdminGroup > >>> or a gateway user. The user group information and what groups are > >>> authorized for which API call is kept with Airavata. I do not > understand > >>> how will authorization in Custos work? > >>> > >> > >> Good question and I am not 100% sure whether I know the answer for > this. I > >> have not touched Airavata code base for a while and I don't clearly > >> understand why we maintain groups within Airavata (why not in Keycloak > >> groups and roles?). Maybe Suresh/Dimuthu or Marlon can explain why we > >> maintain groups in Airavata but not in Keycloak. > >> > >> The way I envision the authorization functionality should be > implemented is > >> as follows: > >> > >> The authorization is performed on a particular resource. A resource can > be > >> represented using a unique string (sometimes called "permission string") > >> and this string can be associated with an API function. Sometimes these > >> resources hierarchical (e.g., a directory structure) in such cases those > >> string can also be hierarchical ("api/addUser" etc.). Then, there is an > >> action that can be performed on a resource (e.g., "read/write/invoke" > >> etc.). In an API call, we need to make sure the current user has > "invoke" > >> permission on the resource associated with the API. So, in summary, > Custos > >> need to provide an API like following for authorization: > >> > >> "bool isAuthorize(tenantId, userId, permissionString, action)" > >> > >> Airavata should invoke above Custos API upon invoking the Airavata API. > The > >> underlying authorization method is irrelevant to Airavata (i.e., whether > >> Custos is using RBAC or fine-grained authorization etc.) and possibly > above > >> method should inspect all inheriting permissions through groups, roles > and > >> sharing as well. > >> > >> > >>> Also, I am currently following the first approach to integrate Keycloak > >>> with Custos ie using keycloak as an external service as Airavata was > >> using > >>> this approach. Let me know if this needs to be changed. > >>> > >> > >> +1, let's use this approach. > >> > >> > >>> > >>> Thanks & Regards, > >>> Aarushi Bisht > >>> ________________________________________ > >>> From: Thejaka Amila J Kanewala <[email protected]> > >>> Sent: Monday, June 24, 2019 1:07 PM > >>> To: [email protected] > >>> Subject: Re: Module structure > >>> > >>> Hi Dimuthu, > >>> > >>> I am sorry for the late reply. I thought I play around this first and > >> reply > >>> to your email but I got busy during the weekend and did not get a > chance > >> to > >>> experiment around this. Therefore, I do not want to hold you on this > >>> anymore. > >>> > >>> Here is what I had in mind: > >>> > >>> Custos provides the following functionalities: > >>> > >>> 1. Delegate science gateway authentication > >>> 2. Provide transparent resource authentication to science gateways > >>> 3. Provide sharing/multi-tenancy functionalities (what is already in > >>> Airavata) > >>> > >>> To provide some of these functionalities we need an identity server > like > >>> Keycloak. So there are two ways we can integrate Keycloak with Custos: > 1. > >>> As an external service, 2. As an embedded server running inside Custos > >>> server. > >>> We had different opinions about these two approaches, however, I > favored > >>> the second approach as it simplifies the deployment complexity (and I > >> guess > >>> Suresh favored the first approach). The issue with the second approach > is > >>> there is no straightforward way to run Keycloak inside Tomcat as an > >>> embedded server. There are a couple of blog posts saying that it is > >>> possible however, I did not get enough time to successfully run them > >>> (theoretically this is possible). > >>> > >>> So, back to your question about the structure. Assuming we take the > first > >>> approach (deploy Keycloak as an external server), I would imagine the > >>> directory structure would be something similar to the following: > >>> > >>> modules > >>> - custos-connectors (Connectors to connect to Keycloak or any other > >>> identity server) > >>> - custos-authentication (Delegated authentication) > >>> - custos-authorisation > >>> - custos-resource-authentication (Resource authentication e.g., Amazon, > >>> Google clouds, SSH etc) > >>> - custos-sharing (Airavata sharing) > >>> - custos-credential-store > >>> - custos-dist (Distribution + tomcat server) > >>> - custos-client (Custos client library which will be used by science > >>> gateways) > >>> - custos-test (Integration tests) > >>> > >>> If we use the second approach, in addition to above modules, we will > >> have a > >>> module called "custos-server" which includes changes to embed keycloak > >>> within Tomcat. > >>> > >>> In either case, the above-described structure is not static, it's just > a > >>> starting point. We definitely need to re-organize as we develop > >>> functionality. > >>> > >>> Thanks > >>> Best Regards, > >>> Thejaka Amila Kanewala, PhD > >>> https://github.com/thejkane/agm > >>> > >>> On Sun, Jun 16, 2019 at 9:17 PM DImuthu Upeksha < > >>> [email protected]> > >>> wrote: > >>> > >>>> Hi Folks, > >>>> > >>>> Is there a decided module structure for the project? If you have any > >>>> suggestion, please let me know. I'm working on refactoring credential > >>> store > >>>> to support multiple protocols (SSH, S3, Azure, Box and etc) and I > >> believe > >>>> that this might me useful for Custos as well. > >>>> > >>>> Thanks > >>>> Dimuthu > >>>> > >>> > >>> > >>> -- > >>> > >> > >> > >> -- > >> Best Regards, > >> Thejaka Amila Kanewala, PhD > >> https://github.com/thejkane/agm > >> > >
