RE: HTCondor Extension Help

[Daniel Shchur]

Hi Dimuthu,

I did try the steps mentioned in [10]. I forgot to mention that when doing so, 
I wouldn’t get any connection at all. However, if I were to simply leave it as:

docker-compose exec pga getent hosts

I could then copy the last address (which wasn’t a loopback) and use the next 
`exec` function to set the container’s hosts file, allowing me to connect to 
the web interface presented by the container. I should note that we have one 
developer doing this on native Ubuntu Linux and he faced the same issue.

Thank you!

- Daniel Shchur

From: DImuthu Upeksha
Sent: Monday, October 19, 2020 5:29 PM
To: Airavata Dev
Cc: daniellthibod...@gmail.com; Evan 
Palmer
Subject: Re: HTCondor Extension Help

Hi Daniel,

Did you follow the steps mentioned in [10]?. I recently added couple of 
instructions that is specific to windows deployments

docker-compose exec pga getent hosts host.docker.internal

docker-compose exec pga /bin/sh -c "echo ' airavata.host' >> 
/etc/hosts"

[10] 
https://github.com/apache/airavata/tree/develop/modules/ide-integration#optional-starting-super-admin-portal-pga

Thanks
Dimuthu

On Sun, Oct 18, 2020 at 7:26 PM Daniel Shchur 
mailto:daniel.shc...@huskers.unl.edu>> wrote:
Hi Dimuthu,

We had some trouble initially getting the changes pulled in, but we eventually 
did and were able to run the portal. However, when running as recommended, the 
PGA portal is not able to communicate with the database stack with Keystore and 
so on, so we weren’t able to get to the login. The error we receive is:

curl_exec() failed. Error1: Failed to connect to airavata.host port 8443: 
Connection refused

One theory I had was that the containers aren’t communicating with each other 
so I added an external network within their respective docker-compose files 
might mitigate the issue, and it did. I was able to get into the login portal 
without the issue above given the resources stack and pga container were all on 
the same docker internal network. However, now when trying to log in, the 
portal redirects to the home page noting that the “Airavata servers are 
currently down”.

Any help with this would be greatly appreciated. Thanks!

- Daniel Shchur

From: DImuthu Upeksha
Sent: Wednesday, October 14, 2020 12:33 AM
To: Airavata Dev
Cc: daniellthibod...@gmail.com; Evan 
Palmer
Subject: Re: HTCondor Extension Help

Hi Daniel,

We only have tested ide integration on MacOS and Ubuntu. However I just managed 
to reproduce your issue in Windows 10 with WSL2 enabled and commits [5-9] done 
to the develop branch should fix those issues. You might have to merge those 
changes to your fork to make it effective and I recommend getting a fresh clone 
to avoid unexpected behaviors due to temp files. Let us know if it works for 
you.

[5] 
https://github.com/apache/airavata/commit/f9b9e203fc268db468977680ab511f88a3eadc99
[6] 
https://github.com/apache/airavata/commit/3794421f01fdb4ef5c0199fdfc6cfc15c25d2e7a
[7] 
https://github.com/apache/airavata/commit/f581b61d44eb9a848b866456acd63f29e2bbfc8e
[8] 

Re: Experiment Outputs -- Display Link?

[Christie, Marcus Aaron]

Hi Luke,

Glad you were able to get it working. Yes that issue has been pesky for us, but 
I have the code fixed on a branch. When we have bandwidth to do the testing, 
I'll get it merged.

> On Oct 18, 2020, at 2:49 PM, Luke Peyralans  wrote:
> 
> Hi Marcus--
> 
> Thank you! The settings file you sent worked perfectly. The only problem I 
> ran into was:
> 
> https://issues.apache.org/jira/browse/AIRAVATA-3376 
> 
> 
> ... which I was able to resolve following:
> 
> https://issues.apache.org/jira/browse/AIRAVATA-3376 
> 
> 
> Now that I have a local django-portal dev environment connected to the Phasta 
> SciGap I will work with my team on the Custom Output View Provider.
> 
> Thanks!
> Luke
> 
> 
> 
> From: Christie, Marcus Aaron
> Sent: Wednesday, October 14, 2020 3:25 PM
> To: Airavata Dev
> Subject: Re: Experiment Outputs -- Display Link?
> 
> Hi Luke,
> 
> I followed up with you via a separate email with a settings_local.py file you 
> can use to setup a local Phasta Django portal dev environment.
> 
> Please use the airavata-django-portal README to get setup. Then see 
> https://apache-airavata-django-portal.readthedocs.io/en/latest/dev/custom_output_view_provider/
>  
> 
>  for detailed documentation on creating a custom output view provider.
> 
> Let me know if you run into any issues. And yes we can definitely do a screen 
> share if you run into issues.
> 
> Thanks,
> 
> Marcus
> 
>> On Oct 12, 2020, at 5:28 PM, Luke Peyralans > > wrote:
>> 
>> Hi Marcus, that would be awesome if I could have some help setting up a 
>> local dev env. Would you recommend I just follow the instructions on 
>> airavata-django-portal github page? If I run into issues can we do a screen 
>> share? 
>> 
>> Thank you!
>> 
>>> On Oct 12, 2020, at 2:15 PM, Christie, Marcus Aaron >> > wrote:
>>> 
>>> Hi Eugene,
>>> 
>>> Yes it can. The way it works is that the custom output view is packaged as 
>>> a Python package and then it can be installed into the hosted Django portal 
>>> virtual environment.
>>> 
>>> See 
>>> https://apache-airavata-django-portal.readthedocs.io/en/latest/dev/custom_output_view_provider/
>>>  
>>> 
>>>  for more details.
>>> 
>>> I can also help you setup a local development environment that mirrors 
>>> phasta.scigap.org  so you can quickly iterate on 
>>> the custom output view.
>>> 
>>> 
>>> - Marcus
>>> 
>>> 
 On Oct 9, 2020, at 5:14 PM, Eugene Walker >>> > wrote:
 
 Can this be utilized on the airavata hosted service, ie phasta.scigap.org 
  ?
 
> On Oct 9, 2020, at 1:49 PM, Christie, Marcus Aaron  > wrote:
> 
> Hi Luke,
> 
> Not as an Airavata output, but our Django Portal supports a link type 
> custom output view. [1]  Essentially this involves creating a Python 
> function that takes the experiment metadata and the output file and 
> returns a label and a URL.
> 
> Let me know if you have any questions.
> 
> Thanks,
> 
> Marcus
> 
> [1] 
> https://apache-airavata-django-portal.readthedocs.io/en/latest/dev/custom_output_view_provider/#display-type-link
>  
> 
> 
>> On Oct 6, 2020, at 12:49 PM, Luke Peyralans > > wrote:
>> 
>> Is it possible to include a link as an experiment output? I would like 
>> to upload some experiment data to a third-party hosted service that 
>> creates interactive visualizations, and then provide a *unique link* 
>> that would let a user access these visualizations upon conclusion of the 
>> experiment.



smime.p7s
Description: S/MIME cryptographic signature

Re: HTCondor Extension Help

[DImuthu Upeksha]

Hi Daniel,

Did you follow the steps mentioned in [10]?. I recently added couple of
instructions that is specific to windows deployments

docker-compose exec pga getent hosts host.docker.internal

docker-compose exec pga /bin/sh -c "echo '
airavata.host' >> /etc/hosts"


[10]
https://github.com/apache/airavata/tree/develop/modules/ide-integration#optional-starting-super-admin-portal-pga

Thanks
Dimuthu

On Sun, Oct 18, 2020 at 7:26 PM Daniel Shchur 
wrote:

> Hi Dimuthu,
>
>
>
> We had some trouble initially getting the changes pulled in, but we
> eventually did and were able to run the portal. However, when running as
> recommended, the PGA portal is not able to communicate with the database
> stack with Keystore and so on, so we weren’t able to get to the login. The
> error we receive is:
>
>
>
> curl_exec() failed. Error1: Failed to connect to airavata.host port 8443:
> Connection refused
>
>
>
> One theory I had was that the containers aren’t communicating with each
> other so I added an external network within their respective docker-compose
> files might mitigate the issue, and it did. I was able to get into the
> login portal without the issue above given the resources stack and pga
> container were all on the same docker internal network. However, now when
> trying to log in, the portal redirects to the home page noting that the
> “Airavata servers are currently down”.
>
> Any help with this would be greatly appreciated. Thanks!
>
>
>
> - Daniel Shchur
>
>
>
> *From: *DImuthu Upeksha 
> *Sent: *Wednesday, October 14, 2020 12:33 AM
> *To: *Airavata Dev 
> *Cc: *daniellthibod...@gmail.com; Evan Palmer
> 
> *Subject: *Re: HTCondor Extension Help
>
>
>
> Hi Daniel,
>
>
>
> We only have tested ide integration on MacOS and Ubuntu. However I just
> managed to reproduce your issue in Windows 10 with WSL2 enabled and
> commits [5-9] done to the develop branch should fix those issues. You might
> have to merge those changes to your fork to make it effective and I
> recommend getting a fresh clone to avoid unexpected behaviors due to temp
> files. Let us know if it works for you.
>
>
>
> [5]
> https://github.com/apache/airavata/commit/f9b9e203fc268db468977680ab511f88a3eadc99
> 
>
> [6]
> https://github.com/apache/airavata/commit/3794421f01fdb4ef5c0199fdfc6cfc15c25d2e7a
> 
>
> [7]
> https://github.com/apache/airavata/commit/f581b61d44eb9a848b866456acd63f29e2bbfc8e
> 
>
> [8]
> https://github.com/apache/airavata/commit/9e30529d7545a1df0b9d94c2783d7d11c32d301d
> 
>
> [9]
> https://github.com/apache/airavata/commit/b11d81182f915491187dcf9f3ede5beeb5ea94a2
> 
>
>
>
> Thanks
>
> Dimuthu
>
>
>
> On Tue, Oct 13, 2020 at 6:37 PM Daniel Shchur <
> daniel.shc...@huskers.unl.edu> wrote:
>
> Hi Dimuthu,
>
>
>
> Windows 10 2004 19041.508
>
> Docker v19.03.13 running on WSL2 - Ubuntu 20.04
>
>
>
> - Daniel Shchur
>
>
>
> *From: *DImuthu Upeksha 
> *Sent: *Tuesday, October 13, 2020 5:34 PM
> *To: *Airavata Dev 
> *Cc: *daniellthibod...@gmail.com; Evan Palmer
> 
> *Subject: *Re: HTCondor Extension Help
>
>
>
> Hi Daniel,
>
>
>
> I'm trying to reproduce the error locally. Can you please send me the
> following information of your local setup.
>
>
>
> OS / version
>
> Docker version
>
>
>
> Thanks
>
> Dimuthu
>
>
>
> On Tue, Oct 13, 2020 at 5:32 PM Daniel Shchur <
> daniel.shc...@huskers.unl.edu> wrote:
>
> Hi Dimunth,
>
>
>
> We have been trying to get the PGA running; however, it seems to 

RE: [External] Overall architecture and deployment of Custos Admin Portal

[Narvekar, Bhakti Sunil]

Hi Dinuka,

Thank you for your email. In regards to your questions:

1) Why are we developing a separate application for the admin features. I think 
we could append these admin features also to the same code base [1]. As per my 
understanding, it'll be just about adding some javascript code to enable these 
features. More than that, it's a way to reuse all the other common components 
(Login, sign up, home etc.)

So, the reason the Admin portal is a separate application because it was 
decided lately. But, now we are planning on merging it.


2) Is there a reason for using the Django service layer between the client and 
the custom apis? Just curious to know.

The reason we are using Django as a service layer because we have a hybrid 
codebase. Again, we are planning on improving the architecture overall.


3) Should the tenant creation be allowed for both admin and normal users? I can 
see there's a tenant approval process allowed only for admin users. I wonder if 
there is a reason for having an approval process?

Yes, we have tenant creation process for both the admin and the normal users. 
After creating the parent tenant, custos admin needs to check the redirect URI 
to check if the request is made from the actual client. Also, we can create 
child tenants that does not require Custos Admin approval.


Please let us know if you have any other questions.


Sincerely,
Bhakti Narvekar

From: Dinuka Desilva
Sent: Sunday, October 18, 2020 9:54 AM
To: Narvekar, Bhakti Sunil
Cc: Airavata Dev; Ranawaka, Isuru 
Janith
Subject: [External] Overall architecture and deployment of Custos Admin Portal

This message was sent from a non-IU address. Please exercise caution when 
clicking links or opening attachments from external sources.

Hi Bhakti,

I have few questions here. Correct me if I am wrong please.

1) Why are we developing a separate application for the admin features. I think 
we could append these admin features also to the same code base [1]. As per my 
understanding, it'll be just about adding some javascript code to enable these 
features. More than that, it's a way to reuse all the other common components 
(Login, sign up, home etc.)

2) Is there a reason for using the Django service layer between the client and 
the custom apis? Just curious to know.

3) Should the tenant creation be allowed for both admin and normal users? I can 
see there's a tenant approval process allowed only for admin users. I wonder if 
there is a reason for having an approval process?

Regards,
Dinuka

[1] https://github.com/apache/airavata-custos-portal

On Sat, Oct 17, 2020 at 9:07 PM Isuru Ranawaka 
mailto:irjan...@gmail.com>> wrote:


-- Forwarded message -
From: Narvekar, Bhakti Sunil mailto:bhn...@iu.edu>>
Date: Fri, Oct 16, 2020 at 9:29 PM
Subject: Overall architecture and deployment of Custos Admin Portal
To: dev@airavata.apache.org 
mailto:dev@airavata.apache.org>>
Cc: Ranawaka, Isuru Janith mailto:isjar...@iu.edu>>, Isuru 
Ranawaka mailto:irjan...@gmail.com>>


Hello Everyone,

We are planning to integrate Custos Portal with Custos Admin portal. I would 
like to share the overall architecture for Custos Admin Portal and the 
architecture for Custos Portal will be explained by Isuru.

Overall Archirecture: The Custos Admin portal is implemented using Django and 
Vue.js.
Below is the working for Custos Admin Portal:


  1.  A user/admin can register/login into the system.
  2.  If a user logs in, he can perform the below specified actions:
 *   Request new tenant - The user enters details like client name, email, 
redirect URL, etc to create a new tenant. Once a tenant is created 
successfully, Custos ID and Custos Secret are displayed for the client.
 *   List all tenants - The user can view all the tenants that he has 
created so far. If he clicks on the clickable link, he can view all the details 
for a tenant.
  3.  If an admin logs in, he can perform the below specified actions:
 *   Request new tenant - The admin enters details like client name, email, 
redirect URL, etc to create a new tenant. Once a tenant is created 
successfully, Custos ID and Custos Secret are displayed for the client.
 *   List all tenants - The admin can view all the tenants that he has 
created so far. On clicking the clickable link, admin can perform the three 
functions - edit, approve or decline.




[cid:17537355117d497bc331]

Deployment:
The portal is deployed on centos using Apache and mod_wsgi. The link to view 
the admin portal is -> https://dev.portal.custos.scigap.org/



Sincerely,
Bhakti Narvekar





--
Research Software Engineer
Indiana University, IN

[em-center-gateway] MFT requirements for EMC portal

[Dinuka Desilva]

Hi Dimuthu,

Let me bring back the questions we discussed at the call to a discussion
thread. We currently see the following requirements from MFT. Let's discuss
further here.

1) Transferring files from the microscopes to the users storage. In this,
there's no authentication layer between the users other than a root level
authentication (if there's anything possible). The user and folder
structure has to be understood from the folder or file path specified at
the microscope.

Eg: if the file path at the microscope is
"./abce-microscope/dinuka/session-36/bacteria-sample.png", It should be
uploaded to the user "dinuka" to the folder "./abce-microscope/session-36"
in his user drive.

@Marru, Suresh  @Kuruvilla, Tanya Elizabeth  I
would +1 for above. I believe it's good to let the users share the files to
the groups manually instead of taking that also from the microscope file
path. In my perspective, I would like not to share directly. Instead I
would like to go back to my computer and see the files and do any
modifications if needed and then share with the PI groups.

2) For file management, there's the need of managing folder structure for
individual users. And the users should be able to share the files or
folders to individual users or groups.

@Christie, Marcus Aaron  I think for the user storage
management within the diravata-django-portal also, we could use MFT if
these requirements are satisfied from MFT.

3) Form the microscopes, since there's no way of getting individual user
authentication processed and it has to be deeply coupled with the path of
the files created (1) and also since the creation of the files is
continuous ,there's a need of having a folder syncing capability which has
to keep watching the file changes.

Regards,
Dinuka