Re: [External] Re: Airavata MFT- Gdrive transport implementation
Hi Shivali, This looks great. This is a good habit for other transport developers also. Once we have some quality content, we can directly put into the repository wiki. I requested some changes to your PR and please update relevant parts of the document once you addressed them. Thanks Dimuthu On Fri, May 1, 2020 at 1:33 PM Jejurkar, Shivali wrote: > Hello, > > > > We have created a document for setting up service account for Google Drive > transport setup in MFT. Kindly review and share your comments/feedback. > > > https://docs.google.com/document/d/1ql3megjxd_4_t9ID23se9Lb8NFpY83e175XHoKYpYdg/edit?usp=sharing > > > > > > Thanks, > > Shivali Jejurkar > > > > Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for > Windows 10 > > > > *From: *Bafna, Nikita > *Sent: *Thursday, April 30, 2020 12:43 PM > *To: *dev@airavata.apache.org > *Subject: *RE: [External] Re: Airavata MFT- Gdrive transport > implementation > > > > Hi Dimuthu, > > > > We have now addressed all the issues raised. > > Could you please verify and provide your inputs. > > > > Regards, > > Nikita > > > > *From: *DImuthu Upeksha > *Sent: *Wednesday, April 29, 2020 1:49 AM > *To: *Airavata Dev > *Subject: *Re: [External] Re: Airavata MFT- Gdrive transport > implementation > > > Still there are some unaddressed reviews available. Please have a look at > them. > > [2] https://github.com/apache/airavata-mft/pull/11/files > > On Wed, Apr 29, 2020 at 1:33 AM Patil, Aniruddha Vivek > wrote: > >> Hello Dimuthu, >> >> We addressed the changes suggested in your review. Please let us know >> what should be addressed further. >> >> Thanks and regards, >> Aniruddha Patil >> >> On Apr 28, 2020, at 10:05 PM, DImuthu Upeksha >> wrote: >> >> Thanks. I did the first pass of you PR. Please have a look. Once you >> address the issues, I will do more detailed reviews. >> >> Dimuthu >> >> On Tue, Apr 28, 2020 at 6:22 PM Bafna, Nikita wrote: >> >>> Hello Dimuthu, >>> >>> >>> Sorry I forgot to mention the PR [1] link. Here it is. >>> >>> >>> [1] https://github.com/apache/airavata-mft/pull/11 >>> >>> >>> Regards, >>> >>> Nikita >>> >>> >>> *From: *Bafna, Nikita >>> *Sent: *Tuesday, April 28, 2020 4:39 PM >>> *To: *dev@airavata.apache.org >>> *Subject: *Re: [External] Re: Airavata MFT- Gdrive transport >>> implementation >>> >>> >>> Hello Dimuthu, >>> >>> We have implemented transport system for Google drive. >>> >>> We have build our transport in a way where if a file with that name >>> already exists it will update that file instead of creating a new one on >>> Google drive. >>> >>> For now we have tested our implementation between endpoints below. >>> >>> S3 to Google drive >>> >>> Google drive to S3 >>> >>> Local to Google drive (Screenshot attached) >>> >>> Google drive to Local (Screenshot attached) >>> >>> Could you please have a look and let us know your feedback. >>> >>> Thank you, >>> Team NSA >>> (Nikita Bafna, Shivali Jejurkar, Aniruddha Patil) >>> >>> >>> -- >>> *From:* Bafna, Nikita >>> *Sent:* Tuesday, April 28, 2020 5:20 AM >>> *To:* dev@airavata.apache.org >>> *Subject:* Re: [External] Re: Airavata MFT- Gdrive transport >>> implementation >>> >>> >>> Hi Dimuthu, >>> >>> >>> During testing of the GDrive transport, we came across a situation where >>> if a file is being uploaded on the Gdrive and if the file with same name >>> already exists, then Gdrive creates a new file with the same name (it >>> creates a new version ). So how should this be handled? Should it create a >>> new file or update the existing file? >>> >>> >>> Let us know your inputs. >>> >>> >>> Regards, >>> >>> Nikita >>> -- >>> *From:* Bafna, Nikita >>> *Sent:* Tuesday, April 28, 2020 12:48 AM >>> *To:* dev@airavata.apache.org >>> *Subject:* RE: [External] Re: Airavata MFT- Gdrive transport >>> implementation >>> >>> >>> Okay. >>> >>> Thanks for the inputs Dimuthu. >>> >&
RE: [External] Re: Airavata MFT- Gdrive transport implementation
Hello, We have created a document for setting up service account for Google Drive transport setup in MFT. Kindly review and share your comments/feedback. https://docs.google.com/document/d/1ql3megjxd_4_t9ID23se9Lb8NFpY83e175XHoKYpYdg/edit?usp=sharing Thanks, Shivali Jejurkar Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10 From: Bafna, Nikita<mailto:niba...@iu.edu> Sent: Thursday, April 30, 2020 12:43 PM To: dev@airavata.apache.org<mailto:dev@airavata.apache.org> Subject: RE: [External] Re: Airavata MFT- Gdrive transport implementation Hi Dimuthu, We have now addressed all the issues raised. Could you please verify and provide your inputs. Regards, Nikita From: DImuthu Upeksha<mailto:dimuthu.upeks...@gmail.com> Sent: Wednesday, April 29, 2020 1:49 AM To: Airavata Dev<mailto:dev@airavata.apache.org> Subject: Re: [External] Re: Airavata MFT- Gdrive transport implementation Still there are some unaddressed reviews available. Please have a look at them. [2] https://github.com/apache/airavata-mft/pull/11/files On Wed, Apr 29, 2020 at 1:33 AM Patil, Aniruddha Vivek mailto:anipa...@iu.edu>> wrote: Hello Dimuthu, We addressed the changes suggested in your review. Please let us know what should be addressed further. Thanks and regards, Aniruddha Patil On Apr 28, 2020, at 10:05 PM, DImuthu Upeksha mailto:dimuthu.upeks...@gmail.com>> wrote: Thanks. I did the first pass of you PR. Please have a look. Once you address the issues, I will do more detailed reviews. Dimuthu On Tue, Apr 28, 2020 at 6:22 PM Bafna, Nikita mailto:niba...@iu.edu>> wrote: Hello Dimuthu, Sorry I forgot to mention the PR [1] link. Here it is. [1] https://github.com/apache/airavata-mft/pull/11 Regards, Nikita From: Bafna, Nikita<mailto:niba...@iu.edu> Sent: Tuesday, April 28, 2020 4:39 PM To: dev@airavata.apache.org<mailto:dev@airavata.apache.org> Subject: Re: [External] Re: Airavata MFT- Gdrive transport implementation Hello Dimuthu, We have implemented transport system for Google drive. We have build our transport in a way where if a file with that name already exists it will update that file instead of creating a new one on Google drive. For now we have tested our implementation between endpoints below. S3 to Google drive Google drive to S3 Local to Google drive (Screenshot attached) Google drive to Local (Screenshot attached) Could you please have a look and let us know your feedback. Thank you, Team NSA (Nikita Bafna, Shivali Jejurkar, Aniruddha Patil) From: Bafna, Nikita Sent: Tuesday, April 28, 2020 5:20 AM To: dev@airavata.apache.org<mailto:dev@airavata.apache.org> Subject: Re: [External] Re: Airavata MFT- Gdrive transport implementation Hi Dimuthu, During testing of the GDrive transport, we came across a situation where if a file is being uploaded on the Gdrive and if the file with same name already exists, then Gdrive creates a new file with the same name (it creates a new version ). So how should this be handled? Should it create a new file or update the existing file? Let us know your inputs. Regards, Nikita From: Bafna, Nikita mailto:niba...@iu.edu>> Sent: Tuesday, April 28, 2020 12:48 AM To: dev@airavata.apache.org<mailto:dev@airavata.apache.org> Subject: RE: [External] Re: Airavata MFT- Gdrive transport implementation Okay. Thanks for the inputs Dimuthu. From: DImuthu Upeksha mailto:dimuthu.upeks...@gmail.com>> Sent: Monday, April 27, 2020 3:05 AM To: Airavata Dev mailto:dev@airavata.apache.org>> Subject: Re: [External] Re: Airavata MFT- Gdrive transport implementation Go with service account approach as this is service to service authentication Dimuthu On Sat, Apr 25, 2020 at 7:11 PM Bafna, Nikita mailto:niba...@iu.edu>> wrote: Hi Dimuthu, While working for Google drive transport, we are wondering what kind of credential account should we use for our setup. Attaching the options available. Could you please provide your feedback. Regards, Nikita From: Suresh Marru mailto:sma...@apache.org>> Sent: Friday, April 24, 2020 11:36 PM To: Airavata Dev Subject: [External] Re: Airavata MFT- Gdrive transport implementation This message was sent from a non-IU address. Please exercise caution when clicking links or opening attachments from external sources. A big kudos for all the work on transports, these are nicely evolving. We do not need to drive these with IU decisions, but good point these will lead to potential use cases. On that note, looks like Google Drive will be used for non-sensitive data and one drive for PHI data. So an eventual plan for MFT to support both will be nice. I added it to the spreadsheet. Suresh On Apr 24, 2020, at 12:45 PM, DImuthu Upeksha mailto:dimuthu.upeks...@gmail.com>> wrote: Beca
RE: [External] Re: Airavata MFT- Gdrive transport implementation
Hi Dimuthu, We have now addressed all the issues raised. Could you please verify and provide your inputs. Regards, Nikita From: DImuthu Upeksha<mailto:dimuthu.upeks...@gmail.com> Sent: Wednesday, April 29, 2020 1:49 AM To: Airavata Dev<mailto:dev@airavata.apache.org> Subject: Re: [External] Re: Airavata MFT- Gdrive transport implementation Still there are some unaddressed reviews available. Please have a look at them. [2] https://github.com/apache/airavata-mft/pull/11/files On Wed, Apr 29, 2020 at 1:33 AM Patil, Aniruddha Vivek mailto:anipa...@iu.edu>> wrote: Hello Dimuthu, We addressed the changes suggested in your review. Please let us know what should be addressed further. Thanks and regards, Aniruddha Patil On Apr 28, 2020, at 10:05 PM, DImuthu Upeksha mailto:dimuthu.upeks...@gmail.com>> wrote: Thanks. I did the first pass of you PR. Please have a look. Once you address the issues, I will do more detailed reviews. Dimuthu On Tue, Apr 28, 2020 at 6:22 PM Bafna, Nikita mailto:niba...@iu.edu>> wrote: Hello Dimuthu, Sorry I forgot to mention the PR [1] link. Here it is. [1] https://github.com/apache/airavata-mft/pull/11 Regards, Nikita From: Bafna, Nikita<mailto:niba...@iu.edu> Sent: Tuesday, April 28, 2020 4:39 PM To: dev@airavata.apache.org<mailto:dev@airavata.apache.org> Subject: Re: [External] Re: Airavata MFT- Gdrive transport implementation Hello Dimuthu, We have implemented transport system for Google drive. We have build our transport in a way where if a file with that name already exists it will update that file instead of creating a new one on Google drive. For now we have tested our implementation between endpoints below. S3 to Google drive Google drive to S3 Local to Google drive (Screenshot attached) Google drive to Local (Screenshot attached) Could you please have a look and let us know your feedback. Thank you, Team NSA (Nikita Bafna, Shivali Jejurkar, Aniruddha Patil) From: Bafna, Nikita Sent: Tuesday, April 28, 2020 5:20 AM To: dev@airavata.apache.org<mailto:dev@airavata.apache.org> Subject: Re: [External] Re: Airavata MFT- Gdrive transport implementation Hi Dimuthu, During testing of the GDrive transport, we came across a situation where if a file is being uploaded on the Gdrive and if the file with same name already exists, then Gdrive creates a new file with the same name (it creates a new version ). So how should this be handled? Should it create a new file or update the existing file? Let us know your inputs. Regards, Nikita From: Bafna, Nikita mailto:niba...@iu.edu>> Sent: Tuesday, April 28, 2020 12:48 AM To: dev@airavata.apache.org<mailto:dev@airavata.apache.org> Subject: RE: [External] Re: Airavata MFT- Gdrive transport implementation Okay. Thanks for the inputs Dimuthu. From: DImuthu Upeksha mailto:dimuthu.upeks...@gmail.com>> Sent: Monday, April 27, 2020 3:05 AM To: Airavata Dev mailto:dev@airavata.apache.org>> Subject: Re: [External] Re: Airavata MFT- Gdrive transport implementation Go with service account approach as this is service to service authentication Dimuthu On Sat, Apr 25, 2020 at 7:11 PM Bafna, Nikita mailto:niba...@iu.edu>> wrote: Hi Dimuthu, While working for Google drive transport, we are wondering what kind of credential account should we use for our setup. Attaching the options available. Could you please provide your feedback. Regards, Nikita From: Suresh Marru mailto:sma...@apache.org>> Sent: Friday, April 24, 2020 11:36 PM To: Airavata Dev Subject: [External] Re: Airavata MFT- Gdrive transport implementation This message was sent from a non-IU address. Please exercise caution when clicking links or opening attachments from external sources. A big kudos for all the work on transports, these are nicely evolving. We do not need to drive these with IU decisions, but good point these will lead to potential use cases. On that note, looks like Google Drive will be used for non-sensitive data and one drive for PHI data. So an eventual plan for MFT to support both will be nice. I added it to the spreadsheet. Suresh On Apr 24, 2020, at 12:45 PM, DImuthu Upeksha mailto:dimuthu.upeks...@gmail.com>> wrote: Because of the IUs decision of migration of Box data to Google Drive, I expect the use cases will come into MFT to handle drive transports. So from the usability perspective, I'd prefer gdrive over one drive. But it's up to you to decide. Once you have finalized on one, please update it in [1] [1] https://docs.google.com/spreadsheets/d/1M7-Reda-pCi1l-TSSstI6Yi1pSbtINUqlBFcy5UrOW0/edit#gid=0 On Fri, Apr 24, 2020 at 11:43 AM Jejurkar, Shivali mailto:sjeju...@iu.edu>> wrote: Hello Dimuthu, I have set up MFT locally and was able to transfer file from S3 to S3 and S3 to local. I am int
Re: [External] Re: Airavata MFT- Gdrive transport implementation
Still there are some unaddressed reviews available. Please have a look at them. [2] https://github.com/apache/airavata-mft/pull/11/files On Wed, Apr 29, 2020 at 1:33 AM Patil, Aniruddha Vivek wrote: > Hello Dimuthu, > > We addressed the changes suggested in your review. Please let us know what > should be addressed further. > > Thanks and regards, > Aniruddha Patil > > On Apr 28, 2020, at 10:05 PM, DImuthu Upeksha > wrote: > > Thanks. I did the first pass of you PR. Please have a look. Once you > address the issues, I will do more detailed reviews. > > Dimuthu > > On Tue, Apr 28, 2020 at 6:22 PM Bafna, Nikita wrote: > >> Hello Dimuthu, >> >> >> Sorry I forgot to mention the PR [1] link. Here it is. >> >> >> [1] https://github.com/apache/airavata-mft/pull/11 >> >> >> Regards, >> >> Nikita >> >> >> *From: *Bafna, Nikita >> *Sent: *Tuesday, April 28, 2020 4:39 PM >> *To: *dev@airavata.apache.org >> *Subject: *Re: [External] Re: Airavata MFT- Gdrive transport >> implementation >> >> >> Hello Dimuthu, >> >> We have implemented transport system for Google drive. >> >> We have build our transport in a way where if a file with that name >> already exists it will update that file instead of creating a new one on >> Google drive. >> >> For now we have tested our implementation between endpoints below. >> >> S3 to Google drive >> >> Google drive to S3 >> >> Local to Google drive (Screenshot attached) >> >> Google drive to Local (Screenshot attached) >> >> Could you please have a look and let us know your feedback. >> >> Thank you, >> Team NSA >> (Nikita Bafna, Shivali Jejurkar, Aniruddha Patil) >> >> >> -- >> *From:* Bafna, Nikita >> *Sent:* Tuesday, April 28, 2020 5:20 AM >> *To:* dev@airavata.apache.org >> *Subject:* Re: [External] Re: Airavata MFT- Gdrive transport >> implementation >> >> >> Hi Dimuthu, >> >> >> During testing of the GDrive transport, we came across a situation where >> if a file is being uploaded on the Gdrive and if the file with same name >> already exists, then Gdrive creates a new file with the same name (it >> creates a new version ). So how should this be handled? Should it create a >> new file or update the existing file? >> >> >> Let us know your inputs. >> >> >> Regards, >> >> Nikita >> -- >> *From:* Bafna, Nikita >> *Sent:* Tuesday, April 28, 2020 12:48 AM >> *To:* dev@airavata.apache.org >> *Subject:* RE: [External] Re: Airavata MFT- Gdrive transport >> implementation >> >> >> Okay. >> >> Thanks for the inputs Dimuthu. >> >> >> *From:* DImuthu Upeksha >> *Sent:* Monday, April 27, 2020 3:05 AM >> *To:* Airavata Dev >> *Subject:* Re: [External] Re: Airavata MFT- Gdrive transport >> implementation >> >> >> Go with service account approach as this is service to service >> authentication >> >> >> Dimuthu >> >> >> On Sat, Apr 25, 2020 at 7:11 PM Bafna, Nikita wrote: >> >> Hi Dimuthu, >> >> >> While working for Google drive transport, we are wondering what kind of >> credential account should we use for our setup. >> >> >> Attaching the options available. >> >> >> Could you please provide your feedback. >> >> >> Regards, >> >> Nikita >> >> -- >> >> *From:* Suresh Marru >> *Sent:* Friday, April 24, 2020 11:36 PM >> *To:* Airavata Dev >> *Subject:* [External] Re: Airavata MFT- Gdrive transport implementation >> >> >> This message was sent from a non-IU address. Please exercise caution when >> clicking links or opening attachments from external sources. >> >> >> A big kudos for all the work on transports, these are nicely evolving. >> >> >> We do not need to drive these with IU decisions, but good point these >> will lead to potential use cases. On that note, looks like Google Drive >> will be used for non-sensitive data and one drive for PHI data. So an >> eventual plan for MFT to support both will be nice. I added it to the >> spreadsheet. >> >> >> Suresh >> >> >> >> On Apr 24, 2020, at 12:45 PM, DImuthu Upeksha >> wrote: >> >> >> Because of the IUs decision of migration of Box data to Google Drive, I >> expect the use cases will come into MFT to handle drive transports. So from >> the usability perspective, I'd prefer gdrive over one drive. But it's up to >> you to decide. Once you have finalized on one, please update it in [1] >> >> >> [1] >> https://docs.google.com/spreadsheets/d/1M7-Reda-pCi1l-TSSstI6Yi1pSbtINUqlBFcy5UrOW0/edit#gid=0 >> >> >> >> On Fri, Apr 24, 2020 at 11:43 AM Jejurkar, Shivali >> wrote: >> >> Hello Dimuthu, >> >> >> I have set up MFT locally and was able to transfer file from S3 to S3 and >> S3 to local. I am interested in implementing Gdrive transfers. Would MFT >> also extend support for other cloud storage platforms like OneDrive? Can >> you share your feedback or any challenges that you see if my team pick >> Gdrive and Onedrive up? >> >> >> >> Thanks, >> >> Shivali >> >> >> >> >> >
Re: [External] Re: Airavata MFT- Gdrive transport implementation
Hello Dimuthu, We addressed the changes suggested in your review. Please let us know what should be addressed further. Thanks and regards, Aniruddha Patil On Apr 28, 2020, at 10:05 PM, DImuthu Upeksha mailto:dimuthu.upeks...@gmail.com>> wrote: Thanks. I did the first pass of you PR. Please have a look. Once you address the issues, I will do more detailed reviews. Dimuthu On Tue, Apr 28, 2020 at 6:22 PM Bafna, Nikita mailto:niba...@iu.edu>> wrote: Hello Dimuthu, Sorry I forgot to mention the PR [1] link. Here it is. [1] https://github.com/apache/airavata-mft/pull/11 Regards, Nikita From: Bafna, Nikita<mailto:niba...@iu.edu> Sent: Tuesday, April 28, 2020 4:39 PM To: dev@airavata.apache.org<mailto:dev@airavata.apache.org> Subject: Re: [External] Re: Airavata MFT- Gdrive transport implementation Hello Dimuthu, We have implemented transport system for Google drive. We have build our transport in a way where if a file with that name already exists it will update that file instead of creating a new one on Google drive. For now we have tested our implementation between endpoints below. S3 to Google drive Google drive to S3 Local to Google drive (Screenshot attached) Google drive to Local (Screenshot attached) Could you please have a look and let us know your feedback. Thank you, Team NSA (Nikita Bafna, Shivali Jejurkar, Aniruddha Patil) From: Bafna, Nikita Sent: Tuesday, April 28, 2020 5:20 AM To: dev@airavata.apache.org<mailto:dev@airavata.apache.org> Subject: Re: [External] Re: Airavata MFT- Gdrive transport implementation Hi Dimuthu, During testing of the GDrive transport, we came across a situation where if a file is being uploaded on the Gdrive and if the file with same name already exists, then Gdrive creates a new file with the same name (it creates a new version ). So how should this be handled? Should it create a new file or update the existing file? Let us know your inputs. Regards, Nikita From: Bafna, Nikita mailto:niba...@iu.edu>> Sent: Tuesday, April 28, 2020 12:48 AM To: dev@airavata.apache.org<mailto:dev@airavata.apache.org> Subject: RE: [External] Re: Airavata MFT- Gdrive transport implementation Okay. Thanks for the inputs Dimuthu. From: DImuthu Upeksha mailto:dimuthu.upeks...@gmail.com>> Sent: Monday, April 27, 2020 3:05 AM To: Airavata Dev mailto:dev@airavata.apache.org>> Subject: Re: [External] Re: Airavata MFT- Gdrive transport implementation Go with service account approach as this is service to service authentication Dimuthu On Sat, Apr 25, 2020 at 7:11 PM Bafna, Nikita mailto:niba...@iu.edu>> wrote: Hi Dimuthu, While working for Google drive transport, we are wondering what kind of credential account should we use for our setup. Attaching the options available. Could you please provide your feedback. Regards, Nikita From: Suresh Marru mailto:sma...@apache.org>> Sent: Friday, April 24, 2020 11:36 PM To: Airavata Dev Subject: [External] Re: Airavata MFT- Gdrive transport implementation This message was sent from a non-IU address. Please exercise caution when clicking links or opening attachments from external sources. A big kudos for all the work on transports, these are nicely evolving. We do not need to drive these with IU decisions, but good point these will lead to potential use cases. On that note, looks like Google Drive will be used for non-sensitive data and one drive for PHI data. So an eventual plan for MFT to support both will be nice. I added it to the spreadsheet. Suresh On Apr 24, 2020, at 12:45 PM, DImuthu Upeksha mailto:dimuthu.upeks...@gmail.com>> wrote: Because of the IUs decision of migration of Box data to Google Drive, I expect the use cases will come into MFT to handle drive transports. So from the usability perspective, I'd prefer gdrive over one drive. But it's up to you to decide. Once you have finalized on one, please update it in [1] [1] https://docs.google.com/spreadsheets/d/1M7-Reda-pCi1l-TSSstI6Yi1pSbtINUqlBFcy5UrOW0/edit#gid=0 On Fri, Apr 24, 2020 at 11:43 AM Jejurkar, Shivali mailto:sjeju...@iu.edu>> wrote: Hello Dimuthu, I have set up MFT locally and was able to transfer file from S3 to S3 and S3 to local. I am interested in implementing Gdrive transfers. Would MFT also extend support for other cloud storage platforms like OneDrive? Can you share your feedback or any challenges that you see if my team pick Gdrive and Onedrive up? Thanks, Shivali
Re: [External] Re: Airavata MFT- Gdrive transport implementation
Thanks. I did the first pass of you PR. Please have a look. Once you address the issues, I will do more detailed reviews. Dimuthu On Tue, Apr 28, 2020 at 6:22 PM Bafna, Nikita wrote: > Hello Dimuthu, > > > > Sorry I forgot to mention the PR [1] link. Here it is. > > > > [1] https://github.com/apache/airavata-mft/pull/11 > > > > Regards, > > Nikita > > > > *From: *Bafna, Nikita > *Sent: *Tuesday, April 28, 2020 4:39 PM > *To: *dev@airavata.apache.org > *Subject: *Re: [External] Re: Airavata MFT- Gdrive transport > implementation > > > > Hello Dimuthu, > > We have implemented transport system for Google drive. > > We have build our transport in a way where if a file with that name > already exists it will update that file instead of creating a new one on > Google drive. > > For now we have tested our implementation between endpoints below. > > S3 to Google drive > > Google drive to S3 > > Local to Google drive (Screenshot attached) > > Google drive to Local (Screenshot attached) > > Could you please have a look and let us know your feedback. > > Thank you, > Team NSA > (Nikita Bafna, Shivali Jejurkar, Aniruddha Patil) > > > -------------- > *From:* Bafna, Nikita > *Sent:* Tuesday, April 28, 2020 5:20 AM > *To:* dev@airavata.apache.org > *Subject:* Re: [External] Re: Airavata MFT- Gdrive transport > implementation > > > Hi Dimuthu, > > > During testing of the GDrive transport, we came across a situation where > if a file is being uploaded on the Gdrive and if the file with same name > already exists, then Gdrive creates a new file with the same name (it > creates a new version ). So how should this be handled? Should it create a > new file or update the existing file? > > > Let us know your inputs. > > > Regards, > > Nikita > -- > *From:* Bafna, Nikita > *Sent:* Tuesday, April 28, 2020 12:48 AM > *To:* dev@airavata.apache.org > *Subject:* RE: [External] Re: Airavata MFT- Gdrive transport > implementation > > > Okay. > > Thanks for the inputs Dimuthu. > > > > *From:* DImuthu Upeksha > *Sent:* Monday, April 27, 2020 3:05 AM > *To:* Airavata Dev > *Subject:* Re: [External] Re: Airavata MFT- Gdrive transport > implementation > > > > Go with service account approach as this is service to service > authentication > > > > Dimuthu > > > > On Sat, Apr 25, 2020 at 7:11 PM Bafna, Nikita wrote: > > Hi Dimuthu, > > > > While working for Google drive transport, we are wondering what kind of > credential account should we use for our setup. > > > > Attaching the options available. > > > > Could you please provide your feedback. > > > > Regards, > > Nikita > > > -- > > *From:* Suresh Marru > *Sent:* Friday, April 24, 2020 11:36 PM > *To:* Airavata Dev > *Subject:* [External] Re: Airavata MFT- Gdrive transport implementation > > > > This message was sent from a non-IU address. Please exercise caution when > clicking links or opening attachments from external sources. > > > > A big kudos for all the work on transports, these are nicely evolving. > > > > We do not need to drive these with IU decisions, but good point these will > lead to potential use cases. On that note, looks like Google Drive will be > used for non-sensitive data and one drive for PHI data. So an eventual plan > for MFT to support both will be nice. I added it to the spreadsheet. > > > > Suresh > > > > On Apr 24, 2020, at 12:45 PM, DImuthu Upeksha > wrote: > > > > Because of the IUs decision of migration of Box data to Google Drive, I > expect the use cases will come into MFT to handle drive transports. So from > the usability perspective, I'd prefer gdrive over one drive. But it's up to > you to decide. Once you have finalized on one, please update it in [1] > > > > [1] > https://docs.google.com/spreadsheets/d/1M7-Reda-pCi1l-TSSstI6Yi1pSbtINUqlBFcy5UrOW0/edit#gid=0 > > > > > On Fri, Apr 24, 2020 at 11:43 AM Jejurkar, Shivali > wrote: > > Hello Dimuthu, > > > > I have set up MFT locally and was able to transfer file from S3 to S3 and > S3 to local. I am interested in implementing Gdrive transfers. Would MFT > also extend support for other cloud storage platforms like OneDrive? Can > you share your feedback or any challenges that you see if my team pick > Gdrive and Onedrive up? > > > > > > Thanks, > > Shivali > > > > > >
RE: [External] Re: Airavata MFT- Gdrive transport implementation
Hello Dimuthu, Sorry I forgot to mention the PR [1] link. Here it is. [1] https://github.com/apache/airavata-mft/pull/11 Regards, Nikita From: Bafna, Nikita<mailto:niba...@iu.edu> Sent: Tuesday, April 28, 2020 4:39 PM To: dev@airavata.apache.org<mailto:dev@airavata.apache.org> Subject: Re: [External] Re: Airavata MFT- Gdrive transport implementation Hello Dimuthu, We have implemented transport system for Google drive. We have build our transport in a way where if a file with that name already exists it will update that file instead of creating a new one on Google drive. For now we have tested our implementation between endpoints below. S3 to Google drive Google drive to S3 Local to Google drive (Screenshot attached) Google drive to Local (Screenshot attached)? Could you please have a look and let us know your feedback. Thank you, Team NSA (Nikita Bafna, Shivali Jejurkar, Aniruddha Patil) From: Bafna, Nikita Sent: Tuesday, April 28, 2020 5:20 AM To: dev@airavata.apache.org Subject: Re: [External] Re: Airavata MFT- Gdrive transport implementation Hi Dimuthu, During testing of the GDrive transport, we came across a situation where if a file is being uploaded on the Gdrive and if the file with same name already exists, then Gdrive creates a new file with the same name (it creates a new version ). So how should this be handled? Should it create a new file or update the existing file? Let us know your inputs. Regards, Nikita From: Bafna, Nikita Sent: Tuesday, April 28, 2020 12:48 AM To: dev@airavata.apache.org Subject: RE: [External] Re: Airavata MFT- Gdrive transport implementation Okay. Thanks for the inputs Dimuthu. From: DImuthu Upeksha Sent: Monday, April 27, 2020 3:05 AM To: Airavata Dev Subject: Re: [External] Re: Airavata MFT- Gdrive transport implementation Go with service account approach as this is service to service authentication Dimuthu On Sat, Apr 25, 2020 at 7:11 PM Bafna, Nikita mailto:niba...@iu.edu>> wrote: Hi Dimuthu, While working for Google drive transport, we are wondering what kind of credential account should we use for our setup. Attaching the options available. Could you please provide your feedback. Regards, Nikita ? From: Suresh Marru mailto:sma...@apache.org>> Sent: Friday, April 24, 2020 11:36 PM To: Airavata Dev Subject: [External] Re: Airavata MFT- Gdrive transport implementation This message was sent from a non-IU address. Please exercise caution when clicking links or opening attachments from external sources. A big kudos for all the work on transports, these are nicely evolving. We do not need to drive these with IU decisions, but good point these will lead to potential use cases. On that note, looks like Google Drive will be used for non-sensitive data and one drive for PHI data. So an eventual plan for MFT to support both will be nice. I added it to the spreadsheet. Suresh On Apr 24, 2020, at 12:45 PM, DImuthu Upeksha mailto:dimuthu.upeks...@gmail.com>> wrote: Because of the IUs decision of migration of Box data to Google Drive, I expect the use cases will come into MFT to handle drive transports. So from the usability perspective, I'd prefer gdrive over one drive. But it's up to you to decide. Once you have finalized on one, please update it in [1] [1] https://docs.google.com/spreadsheets/d/1M7-Reda-pCi1l-TSSstI6Yi1pSbtINUqlBFcy5UrOW0/edit#gid=0 On Fri, Apr 24, 2020 at 11:43 AM Jejurkar, Shivali mailto:sjeju...@iu.edu>> wrote: Hello Dimuthu, I have set up MFT locally and was able to transfer file from S3 to S3 and S3 to local. I am interested in implementing Gdrive transfers. Would MFT also extend support for other cloud storage platforms like OneDrive? Can you share your feedback or any challenges that you see if my team pick Gdrive and Onedrive up?? Thanks, Shivali
Re: [External] Re: Airavata MFT- Gdrive transport implementation
Hi Dimuthu, During testing of the GDrive transport, we came across a situation where if a file is being uploaded on the Gdrive and if the file with same name already exists, then Gdrive creates a new file with the same name (it creates a new version ). So how should this be handled? Should it create a new file or update the existing file? Let us know your inputs. Regards, Nikita From: Bafna, Nikita Sent: Tuesday, April 28, 2020 12:48 AM To: dev@airavata.apache.org Subject: RE: [External] Re: Airavata MFT- Gdrive transport implementation Okay. Thanks for the inputs Dimuthu. From: DImuthu Upeksha Sent: Monday, April 27, 2020 3:05 AM To: Airavata Dev Subject: Re: [External] Re: Airavata MFT- Gdrive transport implementation Go with service account approach as this is service to service authentication Dimuthu On Sat, Apr 25, 2020 at 7:11 PM Bafna, Nikita mailto:niba...@iu.edu>> wrote: Hi Dimuthu, While working for Google drive transport, we are wondering what kind of credential account should we use for our setup. Attaching the options available. Could you please provide your feedback. Regards, Nikita ? From: Suresh Marru mailto:sma...@apache.org>> Sent: Friday, April 24, 2020 11:36 PM To: Airavata Dev Subject: [External] Re: Airavata MFT- Gdrive transport implementation This message was sent from a non-IU address. Please exercise caution when clicking links or opening attachments from external sources. A big kudos for all the work on transports, these are nicely evolving. We do not need to drive these with IU decisions, but good point these will lead to potential use cases. On that note, looks like Google Drive will be used for non-sensitive data and one drive for PHI data. So an eventual plan for MFT to support both will be nice. I added it to the spreadsheet. Suresh On Apr 24, 2020, at 12:45 PM, DImuthu Upeksha mailto:dimuthu.upeks...@gmail.com>> wrote: Because of the IUs decision of migration of Box data to Google Drive, I expect the use cases will come into MFT to handle drive transports. So from the usability perspective, I'd prefer gdrive over one drive. But it's up to you to decide. Once you have finalized on one, please update it in [1] [1] https://docs.google.com/spreadsheets/d/1M7-Reda-pCi1l-TSSstI6Yi1pSbtINUqlBFcy5UrOW0/edit#gid=0 On Fri, Apr 24, 2020 at 11:43 AM Jejurkar, Shivali mailto:sjeju...@iu.edu>> wrote: Hello Dimuthu, I have set up MFT locally and was able to transfer file from S3 to S3 and S3 to local. I am interested in implementing Gdrive transfers. Would MFT also extend support for other cloud storage platforms like OneDrive? Can you share your feedback or any challenges that you see if my team pick Gdrive and Onedrive up?? Thanks, Shivali
RE: [External] Re: Airavata MFT- Gdrive transport implementation
Okay. Thanks for the inputs Dimuthu. From: DImuthu Upeksha Sent: Monday, April 27, 2020 3:05 AM To: Airavata Dev Subject: Re: [External] Re: Airavata MFT- Gdrive transport implementation Go with service account approach as this is service to service authentication Dimuthu On Sat, Apr 25, 2020 at 7:11 PM Bafna, Nikita mailto:niba...@iu.edu>> wrote: Hi Dimuthu, While working for Google drive transport, we are wondering what kind of credential account should we use for our setup. Attaching the options available. Could you please provide your feedback. Regards, Nikita From: Suresh Marru mailto:sma...@apache.org>> Sent: Friday, April 24, 2020 11:36 PM To: Airavata Dev Subject: [External] Re: Airavata MFT- Gdrive transport implementation This message was sent from a non-IU address. Please exercise caution when clicking links or opening attachments from external sources. A big kudos for all the work on transports, these are nicely evolving. We do not need to drive these with IU decisions, but good point these will lead to potential use cases. On that note, looks like Google Drive will be used for non-sensitive data and one drive for PHI data. So an eventual plan for MFT to support both will be nice. I added it to the spreadsheet. Suresh On Apr 24, 2020, at 12:45 PM, DImuthu Upeksha mailto:dimuthu.upeks...@gmail.com>> wrote: Because of the IUs decision of migration of Box data to Google Drive, I expect the use cases will come into MFT to handle drive transports. So from the usability perspective, I'd prefer gdrive over one drive. But it's up to you to decide. Once you have finalized on one, please update it in [1] [1] https://docs.google.com/spreadsheets/d/1M7-Reda-pCi1l-TSSstI6Yi1pSbtINUqlBFcy5UrOW0/edit#gid=0 On Fri, Apr 24, 2020 at 11:43 AM Jejurkar, Shivali mailto:sjeju...@iu.edu>> wrote: Hello Dimuthu, I have set up MFT locally and was able to transfer file from S3 to S3 and S3 to local. I am interested in implementing Gdrive transfers. Would MFT also extend support for other cloud storage platforms like OneDrive? Can you share your feedback or any challenges that you see if my team pick Gdrive and Onedrive up? Thanks, Shivali
Re: [External] Re: Airavata MFT- Gdrive transport implementation
Go with service account approach as this is service to service authentication Dimuthu On Sat, Apr 25, 2020 at 7:11 PM Bafna, Nikita wrote: > Hi Dimuthu, > > > While working for Google drive transport, we are wondering what kind of > credential account should we use for our setup. > > > Attaching the options available. > > > Could you please provide your feedback. > > > Regards, > > Nikita > > > -- > *From:* Suresh Marru > *Sent:* Friday, April 24, 2020 11:36 PM > *To:* Airavata Dev > *Subject:* [External] Re: Airavata MFT- Gdrive transport implementation > > This message was sent from a non-IU address. Please exercise caution when > clicking links or opening attachments from external sources. > > A big kudos for all the work on transports, these are nicely evolving. > > We do not need to drive these with IU decisions, but good point these will > lead to potential use cases. On that note, looks like Google Drive will be > used for non-sensitive data and one drive for PHI data. So an eventual plan > for MFT to support both will be nice. I added it to the spreadsheet. > > Suresh > > On Apr 24, 2020, at 12:45 PM, DImuthu Upeksha > wrote: > > Because of the IUs decision of migration of Box data to Google Drive, I > expect the use cases will come into MFT to handle drive transports. So from > the usability perspective, I'd prefer gdrive over one drive. But it's up to > you to decide. Once you have finalized on one, please update it in [1] > > [1] > https://docs.google.com/spreadsheets/d/1M7-Reda-pCi1l-TSSstI6Yi1pSbtINUqlBFcy5UrOW0/edit#gid=0 > > > On Fri, Apr 24, 2020 at 11:43 AM Jejurkar, Shivali > wrote: > >> Hello Dimuthu, >> >> I have set up MFT locally and was able to transfer file from S3 to S3 and >> S3 to local. I am interested in implementing Gdrive transfers. Would MFT >> also extend support for other cloud storage platforms like OneDrive? Can >> you share your feedback or any challenges that you see if my team pick >> Gdrive and Onedrive up? >> >> >> Thanks, >> Shivali >> >> >> >