Re: [External] Re: How to generate the structured js:es6 client stub for airavata api
Not sure if it could help to look at the SEAGrid desktop client. Thanks, Sudhakar. From: Isuru Ranawaka Reply-To: "us...@airavata.apache.org" Date: Monday, April 6, 2020 at 11:11 PM To: Airavata Users , "Christie, Marcus Aaron" Cc: Airavata Dev Subject: [External] Re: How to generate the structured js:es6 client stub for airavata api This message was sent from a non-IU address. Please exercise caution when clicking links or opening attachments from external sources. Hi Dinuka, Adding Marcus to the thread. He may also have good ideas on this. On Mon, Apr 6, 2020 at 6:03 PM Dinuka Desilva mailto:l.dinukadesi...@gmail.com>> wrote: Hi Isuru & Suresh, Few concerns I have are, 1. Existing login implementation is a form submission and a server rendered html. And it's session based. 2. The endpoints are also session based and goes through CSRF verification. So, I'm not quite seeing any clear direction than electron app directly accessing the app by url. Any advice is much appreciated. what I have understood from your concerns, is that you are worrying about session management between the backend server and the frontend. Basically, where to decouple frontend view management logic from the frontend server API management layer. Is that your concern? Can you explain a bit more about how your electron app design decouples view management components (including HTML, CSS, JS) from the server access API layer?. Does it have any state management mechanism? Anyhow, we need CSRF verifications at least for authentication requests between frontend and backend. But, there should be a CSRF verification process between browser requests and frontend server. thanks Isuru Regards, Dinuka On Mon, Apr 6, 2020 at 6:31 PM Isuru Ranawaka mailto:irjan...@gmail.com>> wrote: Hi Dinuka, On Mon, Apr 6, 2020 at 3:11 AM Dinuka Desilva mailto:l.dinukadesi...@gmail.com>> wrote: On Mon, 6 Apr 2020, 06:36 Suresh Marru, mailto:sma...@apache.org>> wrote: Hi Dinuka, We have not successfully used Thrift generated JS previously (its possible but do not have that experience within Airavata). Django portal uses the python generated code and exposes them as REST API’s using DRF (https://www.django-rest-framework.org/). The Vue.js UI components communicate to these REST API’s. I wonder if you can have electronJS talk to the same API’s instead of directly to Airavata API. Yes. Since airavata APIs doesn't have any authentication or authorization layer, I have to use the Django API. My only worry is then this become only a copy of the same application. Is that the only purpose of this? Airavata APIs do not have an authentication layer. But it has an authorization layer. You can refer to AiravataAPIServer class there it engages a security interceptor for authorization. Anyhow, I guess using same APIs that used by Vue.js will enhanced code reusability otherwise there will be two code bases for the same functionality. Also, we would like to move from Thrift to Protobuf and gRPC. I wonder if REST support can be more seamless once the migration is done. Suresh On Apr 5, 2020, at 4:17 PM, Dinuka Desilva mailto:l.dinukadesi...@gmail.com>> wrote: Hi, I'm trying to generate the es6 client stub for airavata api using the following script. thrift -r --gen js:es6 ../../airavata/thrift-interface-descriptions/airavata-apis/airavata_api.thrift But, I'm not getting it correctly I guess. I'm getting a list of files in a folder called gen-js. Instead what I need is a structured code as there in the airavata-django-portal. I'm also not sure whether what's on the portal is a generated code. Please advise. Can you help me? Regards, Dinuka -- Research Software Engineer Indiana University, IN -- Research Software Engineer Indiana University, IN
Re: How to generate the structured js:es6 client stub for airavata api
Hi Dinuka, Adding Marcus to the thread. He may also have good ideas on this. On Mon, Apr 6, 2020 at 6:03 PM Dinuka Desilva wrote: > Hi Isuru & Suresh, > > Few concerns I have are, > >1. Existing login implementation is a form submission and a server >rendered html. And it's session based. >2. The endpoints are also session based and goes through CSRF >verification. > > So, I'm not quite seeing any clear direction than electron app directly > accessing the app by url. Any advice is much appreciated. > what I have understood from your concerns, is that you are worrying about session management between the backend server and the frontend. Basically, where to decouple frontend view management logic from the frontend server API management layer. Is that your concern? Can you explain a bit more about how your electron app design decouples view management components (including HTML, CSS, JS) from the server access API layer?. Does it have any state management mechanism? Anyhow, we need CSRF verifications at least for authentication requests between frontend and backend. But, there should be a CSRF verification process between browser requests and frontend server. thanks Isuru > > Regards, > Dinuka > > On Mon, Apr 6, 2020 at 6:31 PM Isuru Ranawaka wrote: > >> Hi Dinuka, >> >> On Mon, Apr 6, 2020 at 3:11 AM Dinuka Desilva >> wrote: >> >>> >>> >>> On Mon, 6 Apr 2020, 06:36 Suresh Marru, wrote: >>> Hi Dinuka, We have not successfully used Thrift generated JS previously (its possible but do not have that experience within Airavata). Django portal uses the python generated code and exposes them as REST API’s using DRF ( https://www.django-rest-framework.org/). The Vue.js UI components communicate to these REST API’s. I wonder if you can have electronJS talk to the same API’s instead of directly to Airavata API. >>> >>> Yes. Since airavata APIs doesn't have any authentication or >>> authorization layer, I have to use the Django API. My only worry is then >>> this become only a copy of the same application. Is that the only purpose >>> of this? >>> >>> Airavata APIs do not have an authentication layer. But it has an >> authorization layer. You can refer to AiravataAPIServer class there it >> engages a security interceptor for authorization. Anyhow, I guess using >> same APIs that used by Vue.js will enhanced code reusability otherwise >> there will be two code bases for the same functionality. >> >> >> >> >> Also, we would like to move from Thrift to Protobuf and gRPC. I wonder if REST support can be more seamless once the migration is done. Suresh On Apr 5, 2020, at 4:17 PM, Dinuka Desilva wrote: Hi, I'm trying to generate the es6 client stub for airavata api using the following script. thrift -r --gen js:es6 ../../airavata/thrift-interface-descriptions/airavata-apis/airavata_api.thrift But, I'm not getting it correctly I guess. I'm getting a list of files in a folder called gen-js. Instead what I need is a structured code as there in the airavata-django-portal. I'm also not sure whether what's on the portal is a generated code. Please advise. Can you help me? Regards, Dinuka >> >> -- >> Research Software Engineer >> Indiana University, IN >> >> -- Research Software Engineer Indiana University, IN
[GitHub] [airavata-custos] isururanawaka merged pull request #35: Agent based token manipulation
isururanawaka merged pull request #35: Agent based token manipulation URL: https://github.com/apache/airavata-custos/pull/35 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
Re: How to generate the structured js:es6 client stub for airavata api
Hi Isuru & Suresh, Few concerns I have are, 1. Existing login implementation is a form submission and a server rendered html. And it's session based. 2. The endpoints are also session based and goes through CSRF verification. So, I'm not quite seeing any clear direction than electron app directly accessing the app by url. Any advice is much appreciated. Regards, Dinuka On Mon, Apr 6, 2020 at 6:31 PM Isuru Ranawaka wrote: > Hi Dinuka, > > On Mon, Apr 6, 2020 at 3:11 AM Dinuka Desilva > wrote: > >> >> >> On Mon, 6 Apr 2020, 06:36 Suresh Marru, wrote: >> >>> Hi Dinuka, >>> >>> We have not successfully used Thrift generated JS previously (its >>> possible but do not have that experience within Airavata). Django portal >>> uses the python generated code and exposes them as REST API’s using DRF ( >>> https://www.django-rest-framework.org/). The Vue.js UI components >>> communicate to these REST API’s. I wonder if you can have electronJS talk >>> to the same API’s instead of directly to Airavata API. >>> >> >> Yes. Since airavata APIs doesn't have any authentication or authorization >> layer, I have to use the Django API. My only worry is then this become >> only a copy of the same application. Is that the only purpose of this? >> >> Airavata APIs do not have an authentication layer. But it has an > authorization layer. You can refer to AiravataAPIServer class there it > engages a security interceptor for authorization. Anyhow, I guess using > same APIs that used by Vue.js will enhanced code reusability otherwise > there will be two code bases for the same functionality. > > > > > Also, we would like to move from Thrift to Protobuf and gRPC. I wonder if >>> REST support can be more seamless once the migration is done. >>> >>> Suresh >>> >>> On Apr 5, 2020, at 4:17 PM, Dinuka Desilva >>> wrote: >>> >>> Hi, >>> >>> I'm trying to generate the es6 client stub for airavata api using the >>> following script. >>> >>> thrift -r --gen js:es6 >>> ../../airavata/thrift-interface-descriptions/airavata-apis/airavata_api.thrift >>> >>> >>> But, I'm not getting it correctly I guess. I'm getting a list of files >>> in a folder called gen-js. Instead what I need is a structured code as >>> there in the airavata-django-portal. >>> >>> I'm also not sure whether what's on the portal is a generated code. >>> Please advise. >>> >>> >>> >>> Can you help me? >>> >>> Regards, >>> Dinuka >>> >>> >>> > > -- > Research Software Engineer > Indiana University, IN > >
Re: Apache Airavata MFT - AWS/GCS support
Hi Dimuthu, Thank you for the update. We look into it and get an idea about how the system works. We were hoping to try an implementation for GCS, we will also look into Azure. Thank you Aravind Ramalingam On Mon, Apr 6, 2020 at 4:44 PM DImuthu Upeksha wrote: > Aravind, > > Here [2] is the complete commit for S3 transport implementation but don't > get confused by the amount of changes as this includes both transport > implementation and the service backend implementations. If you need to > implement a new transport, you need to implement a Receiver, Sender and a > MetadataCollector like this [3]. Then you need to add that resource support > to Resource service and Secret service [4] [5]. You can similarly do that > for Azure. A sample SCP -> S3 transfer request is like below. Hope that > helps. > > String sourceId = "remote-ssh-resource"; > String sourceToken = "local-ssh-cred"; > String sourceType = "SCP"; > String destId = "s3-file"; > String destToken = "s3-cred"; > String destType = "S3"; > > TransferApiRequest request = TransferApiRequest.newBuilder() > .setSourceId(sourceId) > .setSourceToken(sourceToken) > .setSourceType(sourceType) > .setDestinationId(destId) > .setDestinationToken(destToken) > .setDestinationType(destType) > .setAffinityTransfer(false).build(); > > > [2] > https://github.com/apache/airavata-mft/commit/62fae3d0ab2921fa8bf0bea7970e233f842e6948 > [3] > https://github.com/apache/airavata-mft/tree/master/transport/s3-transport/src/main/java/org/apache/airavata/mft/transport/s3 > [4] > https://github.com/apache/airavata-mft/blob/master/services/resource-service/stub/src/main/proto/ResourceService.proto#L90 > [5] > https://github.com/apache/airavata-mft/blob/master/services/secret-service/stub/src/main/proto/SecretService.proto#L45 > > Thanks > Dimuthu > > > On Sun, Apr 5, 2020 at 12:10 AM DImuthu Upeksha < > dimuthu.upeks...@gmail.com> wrote: > >> There is a working on S3 transport in my local copy. Will commit it once >> I test it out properly. You can follow the same pattern for any cloud >> provider which has clients with streaming IO. Streaming among different >> transfer protocols inside an Agent has been discussed in the last part of >> this [1] document. Try to get the conceptual idea from that and reverse >> engineer SCP transport. >> >> [1] >> https://docs.google.com/document/d/1zrO4Z1dn7ENhm1RBdVCw-dDpWiebaZEWy66ceTWoOlo >> >> Dimuthu >> >> On Sat, Apr 4, 2020 at 9:22 PM Aravind Ramalingam >> wrote: >> >>> Hello, >>> >>> We were looking at the existing code in the project. We could find >>> implementations only for local copy and SCP. >>> We were confused on how to go about with an external provider like S3 or >>> Azure? Since it would require integrating with their respective clients. >>> >>> Thank you >>> Aravind Ramalingam >>> >>> > On Apr 4, 2020, at 21:15, Suresh Marru wrote: >>> > >>> > Hi Aravind, >>> > >>> > I have to catch up with the code, but you may want to look at the S3 >>> implementation and extend it to Azure, GCP or other cloud services like >>> Box, Dropbox and so on. >>> > >>> > There could be many use cases, here is an idea: >>> > >>> > * Compute a job on a supercomputer with SCP access and push the >>> outputs to a Cloud storage. >>> > >>> > Suresh >>> > >>> >> On Apr 4, 2020, at 8:09 PM, Aravind Ramalingam >>> wrote: >>> >> >>> >> Hello, >>> >> >>> >> We set up the MFT project on local system and tested out SCP transfer >>> between JetStream VMs, we were wondering how the support can be extended >>> for AWS/GCS. >>> >> >>> >> As per our understanding, the current implementation has support for >>> two protocols i.e. local-transport and scp-transport. Would we have to >>> modify/add to the code base to extend support for AWS/GCS clients? >>> >> >>> >> Could you please provide suggestions for this use case. >>> >> >>> >> Thank you >>> >> Aravind Ramalingam >>> > >>> >>
Re: Apache Airavata MFT - AWS/GCS support
Aravind, Here [2] is the complete commit for S3 transport implementation but don't get confused by the amount of changes as this includes both transport implementation and the service backend implementations. If you need to implement a new transport, you need to implement a Receiver, Sender and a MetadataCollector like this [3]. Then you need to add that resource support to Resource service and Secret service [4] [5]. You can similarly do that for Azure. A sample SCP -> S3 transfer request is like below. Hope that helps. String sourceId = "remote-ssh-resource"; String sourceToken = "local-ssh-cred"; String sourceType = "SCP"; String destId = "s3-file"; String destToken = "s3-cred"; String destType = "S3"; TransferApiRequest request = TransferApiRequest.newBuilder() .setSourceId(sourceId) .setSourceToken(sourceToken) .setSourceType(sourceType) .setDestinationId(destId) .setDestinationToken(destToken) .setDestinationType(destType) .setAffinityTransfer(false).build(); [2] https://github.com/apache/airavata-mft/commit/62fae3d0ab2921fa8bf0bea7970e233f842e6948 [3] https://github.com/apache/airavata-mft/tree/master/transport/s3-transport/src/main/java/org/apache/airavata/mft/transport/s3 [4] https://github.com/apache/airavata-mft/blob/master/services/resource-service/stub/src/main/proto/ResourceService.proto#L90 [5] https://github.com/apache/airavata-mft/blob/master/services/secret-service/stub/src/main/proto/SecretService.proto#L45 Thanks Dimuthu On Sun, Apr 5, 2020 at 12:10 AM DImuthu Upeksha wrote: > There is a working on S3 transport in my local copy. Will commit it once I > test it out properly. You can follow the same pattern for any cloud > provider which has clients with streaming IO. Streaming among different > transfer protocols inside an Agent has been discussed in the last part of > this [1] document. Try to get the conceptual idea from that and reverse > engineer SCP transport. > > [1] > https://docs.google.com/document/d/1zrO4Z1dn7ENhm1RBdVCw-dDpWiebaZEWy66ceTWoOlo > > Dimuthu > > On Sat, Apr 4, 2020 at 9:22 PM Aravind Ramalingam > wrote: > >> Hello, >> >> We were looking at the existing code in the project. We could find >> implementations only for local copy and SCP. >> We were confused on how to go about with an external provider like S3 or >> Azure? Since it would require integrating with their respective clients. >> >> Thank you >> Aravind Ramalingam >> >> > On Apr 4, 2020, at 21:15, Suresh Marru wrote: >> > >> > Hi Aravind, >> > >> > I have to catch up with the code, but you may want to look at the S3 >> implementation and extend it to Azure, GCP or other cloud services like >> Box, Dropbox and so on. >> > >> > There could be many use cases, here is an idea: >> > >> > * Compute a job on a supercomputer with SCP access and push the outputs >> to a Cloud storage. >> > >> > Suresh >> > >> >> On Apr 4, 2020, at 8:09 PM, Aravind Ramalingam >> wrote: >> >> >> >> Hello, >> >> >> >> We set up the MFT project on local system and tested out SCP transfer >> between JetStream VMs, we were wondering how the support can be extended >> for AWS/GCS. >> >> >> >> As per our understanding, the current implementation has support for >> two protocols i.e. local-transport and scp-transport. Would we have to >> modify/add to the code base to extend support for AWS/GCS clients? >> >> >> >> Could you please provide suggestions for this use case. >> >> >> >> Thank you >> >> Aravind Ramalingam >> > >> >
Re: How to generate the structured js:es6 client stub for airavata api
Hi Dinuka, On Mon, Apr 6, 2020 at 3:11 AM Dinuka Desilva wrote: > > > On Mon, 6 Apr 2020, 06:36 Suresh Marru, wrote: > >> Hi Dinuka, >> >> We have not successfully used Thrift generated JS previously (its >> possible but do not have that experience within Airavata). Django portal >> uses the python generated code and exposes them as REST API’s using DRF ( >> https://www.django-rest-framework.org/). The Vue.js UI components >> communicate to these REST API’s. I wonder if you can have electronJS talk >> to the same API’s instead of directly to Airavata API. >> > > Yes. Since airavata APIs doesn't have any authentication or authorization > layer, I have to use the Django API. My only worry is then this become > only a copy of the same application. Is that the only purpose of this? > > Airavata APIs do not have an authentication layer. But it has an authorization layer. You can refer to AiravataAPIServer class there it engages a security interceptor for authorization. Anyhow, I guess using same APIs that used by Vue.js will enhanced code reusability otherwise there will be two code bases for the same functionality. Also, we would like to move from Thrift to Protobuf and gRPC. I wonder if >> REST support can be more seamless once the migration is done. >> >> Suresh >> >> On Apr 5, 2020, at 4:17 PM, Dinuka Desilva >> wrote: >> >> Hi, >> >> I'm trying to generate the es6 client stub for airavata api using the >> following script. >> >> thrift -r --gen js:es6 >> ../../airavata/thrift-interface-descriptions/airavata-apis/airavata_api.thrift >> >> >> But, I'm not getting it correctly I guess. I'm getting a list of files in >> a folder called gen-js. Instead what I need is a structured code as there >> in the airavata-django-portal. >> >> I'm also not sure whether what's on the portal is a generated code. >> Please advise. >> >> >> >> Can you help me? >> >> Regards, >> Dinuka >> >> >> -- Research Software Engineer Indiana University, IN
Re: How to generate the structured js:es6 client stub for airavata api
On Mon, 6 Apr 2020, 06:36 Suresh Marru, wrote: > Hi Dinuka, > > We have not successfully used Thrift generated JS previously (its possible > but do not have that experience within Airavata). Django portal uses the > python generated code and exposes them as REST API’s using DRF ( > https://www.django-rest-framework.org/). The Vue.js UI components > communicate to these REST API’s. I wonder if you can have electronJS talk > to the same API’s instead of directly to Airavata API. > Yes. Since airavata APIs doesn't have any authentication or authorization layer, I have to use the Django API. My only worry is then this become only a copy of the same application. Is that the only purpose of this? > Also, we would like to move from Thrift to Protobuf and gRPC. I wonder if > REST support can be more seamless once the migration is done. > > Suresh > > On Apr 5, 2020, at 4:17 PM, Dinuka Desilva > wrote: > > Hi, > > I'm trying to generate the es6 client stub for airavata api using the > following script. > > thrift -r --gen js:es6 > ../../airavata/thrift-interface-descriptions/airavata-apis/airavata_api.thrift > > > But, I'm not getting it correctly I guess. I'm getting a list of files in > a folder called gen-js. Instead what I need is a structured code as there > in the airavata-django-portal. > > I'm also not sure whether what's on the portal is a generated code. Please > advise. > > > > Can you help me? > > Regards, > Dinuka > > >
