Hi Marcus,
Thanks for the reply. Earlier, I did observe that all the experiments are being
created in ‘/tmp’ directory using
this<https://github.com/apache/airavata-django-portal/blob/5ca39e986fe2fe53020ed45cff9c0a8a62262976/django_airavata/apps/api/datastore.py#L195>
code since, the GATEWAY_DATA_STORE_DIR in the settings_local.py.sample
reflects ‘/tmp’ in my local.
What I’m confused about is, when does the location in ‘Storage Preferences’
come into picture? I was under the impression that all the user data is stored
at the place defined by an admin through the Storage Preferences in the Gateway
Resource Profile page. Debugging the code proved otherwise.
Now, according to my current understanding, a user experiment’s input and
output files are stored in the
‘GATEWAY_DATA_STORE_DIR/{username}/{ProjectId}/{ExperimentId}/’ directory. But
what is the role of the File system location mentioned in a Storage Preferences?
If in my local, I was able to run any experiment which either generated an
output file or needed an input file, I would have probably understood the this
better. But the only experiment I’m able to run in my local is ‘Echo’, which
doesn’t need any input file and sadly for some reason is not generating any
output and is always in ‘Launched’ state.
How can I create other experiments which are available on
https://testdrive.airavata.org/ in my local?
Regards,
Vivek.
From: "Christie, Marcus Aaron" <[email protected]>
Reply-To: <[email protected]>
Date: Thursday, July 23, 2020 at 6:35 PM
To: "[email protected]" <[email protected]>
Subject: Re: User storage
Hi Vivek,
The File System Location is used by Airavata to know the location of the
gateway storage directory on the user portal so it can stage output files back
to the portal. In practice however, the File System Location is rarely used
because when an experiment is created, the portal sets an experimentDataDir
property on it to specify exactly where output files should be put.
The Django portal has settings that determine the directory where the gateway
storage. [0]
[0]
https://github.com/apache/airavata-django-portal/blob/master/django_airavata/settings_local.py.sample#L84-L86
On Jul 21, 2020, at 2:45 AM, Bandaru, Vivek Shresta
<[email protected]<mailto:[email protected]>> wrote:
Hi All,
I’m confused on when the File System Location specified in StoragePreferences
in ‘Gateway Resource Profile’ come into picture.
Upon login, a storage directory (ex: /tmp/default-admin for the user
default-admin) is created for every user where all his experiments and the
respective input and output data is stored.
Every experiment has its own directory with the same name as that of the
experiment created under the above mentioned directory.
<image001.png>
Not sure where does the File System Location –
‘/var/www/portals/gateway-user-data’ mentioned in the Storage Preferences come
into picture. Is this specific to Airavata and earlier mentioned file system
location specific to Django-portal gateway?
<image002.png>
During one of my very earlier interactions regarding the GSoC project, I was
told about File System location in the Storage preferences which has all the
gateways data. Please refer to the below mentioned screenshot shared (Note:
This screenshot isn’t from my local).
<image003.png>
But in my local, the directory var, doesn’t have any directory called ‘www’.
Please refer to the below mentioned screenshot.
<image004.png>
I would like to know a little more on how file system storage is used in
Django-Portal and Airavata and any pointers are appreciated.
Thanks for reading.
Regards,
Vivek.
