So, we’ve talked about this since and I am taking a crack at including the SFTP feature into PGA. This seems like an ideal framework to include: https://github.com/thephpleague/flysystem-sftp <https://github.com/thephpleague/flysystem-sftp>
I’m looking for the part of the application where the input and output types are defined — I’m guessing we would do this as another “type” like “URI” or “STRING.” Can someone give me a good place to start? Thanks, J > On Sep 1, 2016, at 4:32 PM, Suresh Marru <[email protected]> wrote: > > Hi Jarett, > > We have some interest from who were looking for task to contribute to > Airavata. We can probably motivate interest on this task. Can you please > write a detailed use cases clearly articulating the user sequence of actions > you would like to see? We can then over lay that with Airavata architecture, > break it down into small sprints and work through it. > > Thanks, > Suresh > >> On Sep 1, 2016, at 4:06 PM, Jarett DeAngelis <[email protected] >> <mailto:[email protected]>> wrote: >> >> Hi Suresh, >> >> Just wanted to follow up on this: I see what you mean re: the file picker >> connected to a remote node. So, given that what we want to do is allow users >> to both access shared data (like databases, for example, for applications >> like blastn) and read and write data to their home folders, what do you >> think is the best course of action? >> >> I’m also curious what existing use cases you’re seeing that make the gateway >> useful currently, *without* access to permission-controlled user >> directories. How are people using it now? >> >> Ultimately, fixes for this situation that would help us would include: >> 1) A file picker for directories *local* to the PGA gateway (because we >> could put shared resources like databases in a directory like this) >> 2) The ability to SCP/SFTP, from the gateway, in and out of user-permission >> space, using the credentials entered into the gateway to log in. >> >> So, some questions: >> Do you see either of these as being implementable in the short term? >> Are these on your radar for the Airavata team to implement themselves? >> >> Thanks! >> >> Jarett >> >> >>> On Aug 12, 2016, at 4:17 PM, Suresh Marru <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> >>>> On Aug 12, 2016, at 3:23 PM, Jarett DeAngelis <[email protected] >>>> <mailto:[email protected]>> wrote: >>>> >>>> Hey Suresh, >>>> >>>> Thanks for starting this conversation! >>>> >>>> What do you think the users’ “extra one time configuration” would look >>>> like? >>>> >>>> One of the things I was thinking about was how the gateway can do things >>>> like pass a path to the compute node in order to find files. Is it >>>> possible for the gateway to produce a “file picker” sort of dialog based >>>> on what it can see in a path you give it? It runs jobs as its service >>>> account when Airavata interacts with it. You might not have to have it run >>>> *as* the user in order to produce a directory listing for acquiring and >>>> depositing files, if the user changes the permissions on a certain set of >>>> directories such that the service account can read/write to them. >>>> >>>> Does this make sense? Would it be a large development effort to get this >>>> additional bit of UI functionality? Obviously part of the purpose of a >>>> science gateway is to avoid having to dig around on the command line >>>> sorting out what files are where. >>> >>> Hi Jaratt, >>> >>> After thinking through this scenario, I think users providing a path to a >>> permissible users space is a low hanging fruit. Doing a file picker (which >>> implies reading remote files) will be an involving task since PGA does not >>> talk to remote server directly but has to broker the file listings through >>> Airavata. Make sense? >>> >>> Suresh >>> >>>> >>>> Thanks! >>>> Jarett >>>> >>>>> On Aug 8, 2016, at 2:17 PM, Suresh Marru <[email protected] >>>>> <mailto:[email protected]>> wrote: >>>>> >>>>> Hi Jarett, >>>>> >>>>> To give some context, let me start refreshing the security(authentication >>>>> and authorization aspects) within Airavata. >>>>> >>>>> There are clearly two distinct layers, users securely interacting with >>>>> Airavata and Airavata able to security interact with storage and compute >>>>> resources. The former interactions are facilitated with WSO2 IS and later >>>>> managed by Airavata Credential Store. For your use case we need to >>>>> brainstorm on how we bridge the two. You can read more about Credential >>>>> Store here [1]. >>>>> >>>>> If I am understanding your scenario, users authenticate on PGA with the >>>>> credentials in LDAP/FreeIPA (brokered through WSO2 IS). Airavata then >>>>> allows them to use compute resources managed by roles in the IS (in the >>>>> future we might replace this with more fine grained group management). >>>>> All authenticated and authorized users can interact with storage and >>>>> compute resources which the gateway administers have enabled them by >>>>> retrieving the credentials stored in the credential store. Currently only >>>>> administrators generate and configure these credentials. We have >>>>> considered power users (with appropriate role privileges also using >>>>> credential store directly). >>>>> >>>>> If your use case is for every user, then we need to essentially have them >>>>> delegate access to their personal storage space. Access protocol is not >>>>> not too much of an issue to support, how seamlessly and security we can >>>>> have users delegate their storage credentials is a challenge. Its not a >>>>> major development effort, but its more of a usability issue. If we need >>>>> to do it in a good secure way, users should be prepared to do extra one >>>>> time configuration. If we need to have users do nothing, then that will >>>>> lead to less secure options. >>>>> >>>>> Can we outline from a usability standout what will be a acceptable >>>>> credential delegation (access to their storage space) and build from >>>>> there? >>>>> >>>>> Cheers, >>>>> Suresh >>>>> >>>>> [1] - >>>>> https://scholarworks.iu.edu/dspace/bitstream/handle/2022/17379/ccgrid_2014_credential_store.pdf >>>>> >>>>> <https://scholarworks.iu.edu/dspace/bitstream/handle/2022/17379/ccgrid_2014_credential_store.pdf> >>>>> >>>>> >>>>>> On Aug 2, 2016, at 5:08 PM, Jarett DeAngelis <[email protected] >>>>>> <mailto:[email protected]>> wrote: >>>>>> >>>>>> Hi all, >>>>>> >>>>>> When the PGA gateway is dealing with something like getting files to and >>>>>> from user space (like an NFS share, for example), what is the canonical >>>>>> way of doing this? Since the gateway is running as the Apache service >>>>>> account, it does not have permissions on the user's directories. Is >>>>>> there something standard you can put into an application interface to >>>>>> let it get files back and forth? Like, some kind of spot for ssh/scp >>>>>> URIs, for example? I dunno, putting >>>>>> scp://user:password@host:~/path/to/directory >>>>>> <scp://user:password@host:~/path/to/directory> in for a path? Or some >>>>>> more nicely-designed file picker-type interface that uses the PGA user’s >>>>>> logged in credentials (in our case, from LDAP/FreeIPA)? >>>>>> >>>>>> Thanks in advance, >>>>>> Jarett >>>>> >>>> >>> >> >
