Gateways 2020 Call for Participation (Tutorials deadline extended to May 5)

[Pierce, Marlon]

Hi all, 

 

Please see the announcement below. If you are developing a science gateway, 
please consider submitting a paper, tutorial, or poster.

 

Thanks,


Marlon

 

---

 

Call for Participation: Gateways 2020 (October 19–21, Bethesda, Maryland) is 
now accepting submissions. 

What’s a gateway? Gateways are user-friendly interfaces to computing, data, 
instruments, and other domain-specific resources to support research and 
education. They may also be known as portals, virtual research environments, 
eScience, virtual labs, eResearch, digital repositories, or research 
cyberinfrastructure.

Deadlines:
Tutorials and Workshops: extended to May 5 
Short Papers, Demos, and Panels: May 11
Posters (open to all attendees): September 11
Read more details in the Call for Participation: 
http://sciencegateways.org/gateways2020/call 

COVID-19 UPDATE:
Because the COVID-19 outbreak is changing rapidly, we are continuing to plan 
our October conference while monitoring the situation. In case we have to 
cancel, we will strive to offer an alternative format, and peer-reviewed, 
accepted submissions would still be published in the Gateways 2020 Proceedings. 
Questions, concerns, and suggestions may be sent to h...@sciencegateways.org. 

 

 



smime.p7s
Description: S/MIME cryptographic signature

Re: Airavata MFT- Gdrive transport implementation

[Suresh Marru]

A big kudos for all the work on transports, these are nicely evolving. 

We do not need to drive these with IU decisions, but good point these will lead 
to potential use cases. On that note, looks like Google Drive will be used for 
non-sensitive data and one drive for PHI data. So an eventual plan for MFT to 
support both will be nice. I added it to the spreadsheet. 

Suresh

> On Apr 24, 2020, at 12:45 PM, DImuthu Upeksha  
> wrote:
> 
> Because of the IUs decision of migration of Box data to Google Drive, I 
> expect the use cases will come into MFT to handle drive transports. So from 
> the usability perspective, I'd prefer gdrive over one drive. But it's up to 
> you to decide. Once you have finalized on one, please update it in [1]
> 
> [1] 
> https://docs.google.com/spreadsheets/d/1M7-Reda-pCi1l-TSSstI6Yi1pSbtINUqlBFcy5UrOW0/edit#gid=0
>  
> 
>  
> 
> On Fri, Apr 24, 2020 at 11:43 AM Jejurkar, Shivali  > wrote:
> Hello Dimuthu,
> 
> I have set up MFT locally and was able to transfer file from S3 to S3 and S3 
> to local. I am interested in implementing Gdrive transfers. Would MFT also 
> extend support for other cloud storage platforms like OneDrive? Can you share 
> your feedback or any challenges that you see if my team pick Gdrive and 
> Onedrive up?​
> 
> 
> Thanks,
> Shivali
> 
> 

Re: [External] Re: MFT- Dropbox Transport Implementation

[Aravind Ramalingam]

Hi Dimuthu,

Thank you for the response. We have incorporated the changes and raised a
PR [1] for Dropbox transport implementation.

Please do review and let us know your feedback.

[1] https://github.com/apache/airavata-mft/pull/9

Thank you
Aravind Ramalingam

On Fri, Apr 24, 2020 at 11:21 AM DImuthu Upeksha 
wrote:

> I would say support for both. Do a small validation in the transport. If
> user has provided /, use it as it is and if he didn't provide, add a / to
> the beginning.
>
> Dimuthu
>
> On Thu, Apr 23, 2020 at 11:46 PM Rajvanshi, Akshay 
> wrote:
>
>> Hello,
>>
>>
>>
>> We had a doubt regarding Dropbox transport implementation. In the
>> resources.json file, the Dropbox API requires the resource path to be
>> appended with a “/” in the starting as a part of the Dropbox file system
>> convention.
>>
>>
>>
>> We wanted to ask if we should add the “/” manually or expect the user to
>> add this on their own? Also, as a test, we added a “/” on our own while
>> parsing the resourcePath, but if the user adds a “/” on their own in the
>> resources.json, you get the same error again.
>>
>>
>>
>> [image: A screenshot of a computer Description automatically generated]
>>
>>
>>
>> Kind Regards
>>
>> Akshay Rajvanshi
>>
>>
>>
>> *From: *DImuthu Upeksha 
>> *Reply-To: *"dev@airavata.apache.org" 
>> *Date: *Thursday, April 23, 2020 at 14:47
>> *To: *Airavata Dev 
>> *Subject: *[External] Re: MFT- Dropbox Transport Implementation
>>
>>
>>
>> This message was sent from a non-IU address. Please exercise caution when
>> clicking links or opening attachments from external sources.
>>
>>
>>
>> If you set the md5 to null, it will not be evaluated. So you should be
>> fine
>>
>>
>>
>> Dimuthu
>>
>>
>>
>> On Thu, Apr 23, 2020 at 2:41 PM Ravichandran, Sharanya 
>> wrote:
>>
>> Hi,
>>
>>
>>
>> We were taking a look at dropbox's documentation and dropbox's API
>> doesnt provide the md5checksum.
>>
>>
>>
>> Since we have md5 integrity checks whenever a transfer takes place, can
>> we skip the md5 check instead by setting the metadata to null ?
>>
>>
>>
>> Thanks,
>>
>> Sharanya R.
>>
>>

Re: Airavata MFT- Gdrive transport implementation

[DImuthu Upeksha]

Because of the IUs decision of migration of Box data to Google Drive, I
expect the use cases will come into MFT to handle drive transports. So from
the usability perspective, I'd prefer gdrive over one drive. But it's up to
you to decide. Once you have finalized on one, please update it in [1]

[1]
https://docs.google.com/spreadsheets/d/1M7-Reda-pCi1l-TSSstI6Yi1pSbtINUqlBFcy5UrOW0/edit#gid=0


On Fri, Apr 24, 2020 at 11:43 AM Jejurkar, Shivali  wrote:

> Hello Dimuthu,
>
> I have set up MFT locally and was able to transfer file from S3 to S3 and
> S3 to local. I am interested in implementing Gdrive transfers. Would MFT
> also extend support for other cloud storage platforms like OneDrive? Can
> you share your feedback or any challenges that you see if my team pick
> Gdrive and Onedrive up?​
>
>
> Thanks,
> Shivali
>
>
>

Re: [External] Re: Apache Airavata MFT : File Autotransfer

[DImuthu Upeksha]

Only prerequisite is understanding message paths between API, Controller
and Agents. For technology pieces like consul and grpc messaging, you can
easily figure out as there are enough code examples in the code itself.
Once you grab the architecture properly, implementation might be easy or
rather straightforward. I account most of the estimated time for
understanding the core of MFT rather than the implementation part. For
implementation I expect following top level milestones

1. Defining a new transfer type for API [2]
2. Parsing the new transfer type in controller [3]
3. Implementing data structures and timers to handle scheduled transfers in
the controller. This includes handling fault tolerance in case of
controller crashes. This is the place I need a proper design form you on
how you are going to do it

[2]
https://github.com/apache/airavata-mft/blob/master/api/stub/src/main/proto/MFTApi.proto#L9
[3]
https://github.com/apache/airavata-mft/blob/master/controller/src/main/java/org/apache/airavata/mft/controller/MFTController.java#L95

Thanks
Dimuthu


On Thu, Apr 23, 2020 at 8:12 PM Patil, Aniruddha Vivek 
wrote:

> Since you mentioned that the project involves complexity, we’d like to know
> 1) If you have a plan in mind that would help us estimate if we could be
> able to realistically complete the project in time.
> 2) A list of prerequisite concepts relevant to this project.
> 3) Our biggest concern is the time constraint due to the end-semester time
> period and hence we wanted to be sure before we commit.
>
> Regards,
> Aniruddha
>
> On Apr 23, 2020, at 5:20 PM, DImuthu Upeksha 
> wrote:
>
> I would say this project is an interesting one to work on if you have some
> quality time to put on.  And I can provide some significant amount of help
> if the time is the critical factor. But I guess narrowing down the scope
> might not work for both the project and you as it would be hard for a new
> one to continue from the point where you have stopped given the complexity
> of the project. It's up to you to decide.
>
> Dimuthu
>
> On Thu, Apr 23, 2020 at 4:20 PM Bafna, Nikita  wrote:
>
>> Hi Dimuthu,
>>
>>
>> Given the short deadline, do you think we can pick up and do small part
>> of out proposal.
>>
>> Or is it okay to take up Google Drive implementation as a transport type?
>>
>>
>> Regards,
>>
>> Nikita
>>
>>
>> *From: *Patil, Aniruddha Vivek 
>> *Sent: *Thursday, April 23, 2020 2:59 PM
>> *To: *dev@airavata.apache.org
>> *Subject: *Re: [External] Re: Apache Airavata MFT : File Autotransfer
>>
>> Hello,
>>
>> As part of our course, we have a soft deadline of 30th April, 2020. We
>> can adjust the scope of this proposal to fit the deadline. However, we
>> would like to use a few more days in order to incorporate this feature
>> completely.
>>
>> Regards,
>> Aniruddha
>>
>> On Apr 23, 2020, at 2:40 PM, DImuthu Upeksha 
>> wrote:
>>
>> Hi,
>>
>> This looks interesting. This is a feature going into the controller of
>> MFT. And you have to define a new Transfer request type for scheduled
>> transfers. Basically it will affect to MFT Controller and MFT API modules.
>> You should have a deep understanding on how messages are processed in
>> controller so I suggest you to setup MFT locally and try out
>> different transfers first. Then you can send a design diagram with some
>> text on how you are going to do it and the timeline for that. Do you have
>> an absolute deadline for this project?
>>
>> [1]
>> https://github.com/apache/airavata-mft/blob/master/controller/src/main/java/org/apache/airavata/mft/controller/MFTController.java
>>
>> Thanks
>> Dimuthu
>>
>> On Thu, Apr 23, 2020 at 2:24 PM Patil, Aniruddha Vivek 
>> wrote:
>>
>>> Hello Dimuthu,
>>>
>>> Here is a link to our proposal (
>>> https://github.com/airavata-courses/NSA/wiki/Assignment-3-Proposal)
>>> It has more detailed information about this feature.
>>>
>>> Regards,
>>> Aniruddha
>>>
>>> On Apr 22, 2020, at 10:42 PM, DImuthu Upeksha <
>>> dimuthu.upeks...@gmail.com> wrote:
>>>
>>> This message was sent from a non-IU address. Please exercise caution
>>> when clicking links or opening attachments from external sources.
>>>
>>> Hi Aniruddha,
>>>
>>> Can you note down more fine grained goals of this feature. May be add
>>> some use cases as well. Then I will be able to help in figuring out where
>>> to implement it.
>>>
>>> Thanks
>>> Dimuthu
>>>
>>> On Wed, Apr 22, 2020 at 9:37 PM Patil, Aniruddha Vivek 
>>> wrote:
>>>
 Hello,
 As per discussions with Prof. Suresh, we are planning on contributing
 the feature of scheduling file transfers to Airavata's MFT (
 https://www.mail-archive.com/dev@airavata.apache.org/msg12564.html).
 We were considering delivering certain API endpoints that could easily
 be integrated into the dashboard in the future.

 Are there any pointers we could follow that would help us with this?

 Regards,
 Aniruddha Patil
>>>
>>>
>>>
>>
>

Re: [External] Re: MFT- Dropbox Transport Implementation

[DImuthu Upeksha]

I would say support for both. Do a small validation in the transport. If
user has provided /, use it as it is and if he didn't provide, add a / to
the beginning.

Dimuthu

On Thu, Apr 23, 2020 at 11:46 PM Rajvanshi, Akshay  wrote:

> Hello,
>
>
>
> We had a doubt regarding Dropbox transport implementation. In the
> resources.json file, the Dropbox API requires the resource path to be
> appended with a “/” in the starting as a part of the Dropbox file system
> convention.
>
>
>
> We wanted to ask if we should add the “/” manually or expect the user to
> add this on their own? Also, as a test, we added a “/” on our own while
> parsing the resourcePath, but if the user adds a “/” on their own in the
> resources.json, you get the same error again.
>
>
>
> [image: A screenshot of a computer Description automatically generated]
>
>
>
> Kind Regards
>
> Akshay Rajvanshi
>
>
>
> *From: *DImuthu Upeksha 
> *Reply-To: *"dev@airavata.apache.org" 
> *Date: *Thursday, April 23, 2020 at 14:47
> *To: *Airavata Dev 
> *Subject: *[External] Re: MFT- Dropbox Transport Implementation
>
>
>
> This message was sent from a non-IU address. Please exercise caution when
> clicking links or opening attachments from external sources.
>
>
>
> If you set the md5 to null, it will not be evaluated. So you should be
> fine
>
>
>
> Dimuthu
>
>
>
> On Thu, Apr 23, 2020 at 2:41 PM Ravichandran, Sharanya 
> wrote:
>
> Hi,
>
>
>
> We were taking a look at dropbox's documentation and dropbox's API doesnt
> provide the md5checksum.
>
>
>
> Since we have md5 integrity checks whenever a transfer takes place, can we
> skip the md5 check instead by setting the metadata to null ?
>
>
>
> Thanks,
>
> Sharanya R.
>
>