[GitHub] [airavata-django-portal] machristie opened a new pull request #46: AIRAVATA-3365 Increase max number of request parameters to 10000 for …

[GitBox]

machristie opened a new pull request #46:
URL: https://github.com/apache/airavata-django-portal/pull/46


   …large wagtail pages



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [airavata-django-portal] machristie merged pull request #46: AIRAVATA-3365 Increase max number of request parameters to 10000 for …

[GitBox]

machristie merged pull request #46:
URL: https://github.com/apache/airavata-django-portal/pull/46


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: Establishing an SSH connection to StorageResource

[DImuthu Upeksha]

Hi Vivek,

The error you are getting is self explanatory. You are trying to SSH into a
storage resource with a ssh key that is not available in the storage's
authorized_keys file. I can see that you are trying to create a new ssh
credential.
token = generateAndRegisterSSHKeys(authzToken, "");
Issue is, the public key of this credential is not copied to the
authorized_keys file of the target storage resource. You need to fetch the
SSH credential from the storage preference as we are doing in Helix [1]
[2]. These keys are already registered in the storage endpoint.

[1]
https://github.com/apache/airavata/blob/master/modules/airavata-helix/helix-spectator/src/main/java/org/apache/airavata/helix/impl/task/staging/DataStagingTask.java#L83
[2]
https://github.com/apache/airavata/blob/7a9b27aac395883afcf9f5bf08a3eb4c6ff577a5/modules/airavata-helix/helix-spectator/src/main/java/org/apache/airavata/helix/impl/task/TaskContext.java#L477

Thanks
Dimuthu

On Wed, Aug 19, 2020 at 4:06 AM Bandaru, Vivek Shresta 
wrote:

> Hi All,
>
>
>
> Though the previous approach worked, it had a problem. Every gateway would
> need to implement their own user storage tracking mechanism.  As discussed
> earlier in my previous mail, the better approach would be to establish an
> SSH connection with the Storage Resource and track the storage size in
> Airavata. After a little debugging and discussions, I found that
> StorageResourceAdaptor and AgentAdaptor’s (which uses pooling) are the way
> to establish a connection and execute commands on remote machines instead
> of creating a new SSHConnection, so my new implementation uses
> StorageResourceAdaptor to establish a connection and execute the du
> 
>  commands
> to get the size of the user directory.
>
>
>
> Questions:
>
>
>
>1. Currently, in case of Airavata-Django-Portal, experimentDataDir is
>always inside the project directiory which in turn is inside the user
>directory. Will the same behavior be followed across all the gateways?
>2. I was trying to establish a connection with the gateway using
>StorageResourceAdaptor (I believe AgentAdaptor’s are used for compute
>resources) but I’m stuck at this step. I was successfully able to create a
>connection using AgentAdaptor (only for testing purposes) and execute
>commands on it. But I’m not able to do the same with
>StorageResourceAdaptor.
>The only difference I can think of is, here I’m creating a new
>authToken for establishing a connection with the Storage resource (and of
>course deleting it at the end of execution), and I’m assuming this might be
>the issue. While testing with AgentAdaptor, I hardcoded the authToken value
>that’s already generated for the Jetstream compute resources, so I’m
>guessing that might be the reason why it was working in this case.
>
>This is the error stacktrace that I’ve encountered while executing the
>‘executeCommand()’ on StorageResourceAdaptor after initiating it.
>
>
> 2020-08-18 19:31:07,152 [SSH-Pool-Monitor-airavata.host-2] INFO
> o.a.a.h.a.PoolingSSHJClient Current active connections for  null @
> airavata.host : 2 are 0 []
>
> 2020-08-18 19:31:07,173 [pool-24-thread-6] INFO
> n.s.s.transport.TransportImpl Client identity string: SSH-2.0-SSHJ_0.23.0 []
>
> 2020-08-18 19:31:07,180 [pool-24-thread-6] INFO
> n.s.s.transport.TransportImpl Server identity string: SSH-2.0-OpenSSH_7.2p2
> Ubuntu-4ubuntu2.7 []
>
> 2020-08-18 19:31:07,488 [pool-24-thread-6] ERROR
> o.a.a.a.s.h.AiravataServerHandler
> net.schmizz.sshj.userauth.UserAuthException: Exhausted available
> authentication methods []
>
> org.apache.airavata.agents.api.AgentException:
> net.schmizz.sshj.userauth.UserAuthException: Exhausted available
> authentication methods
>
> at
> org.apache.airavata.helix.adaptor.SSHJAgentAdaptor.executeCommand(SSHJAgentAdaptor.java:190)
>
> at
> org.apache.airavata.helix.adaptor.SSHJStorageAdaptor.executeCommand(SSHJStorageAdaptor.java:94)
>
> at
> org.apache.airavata.api.server.handler.AiravataServerHandler.validateStorageLimit(AiravataServerHandler.java:726)
>
> at
> org.apache.airavata.api.Airavata$Processor$validateStorageLimit.getResult(Airavata.java:18296)
>
> at
> org.apache.airavata.api.Airavata$Processor$validateStorageLimit.getResult(Airavata.java:18280)
>
> at
> org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>
> at
> org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
>
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>
> at
> 

Establishing an SSH connection to StorageResource

[Bandaru, Vivek Shresta]

Hi All,

Though the previous approach worked, it had a problem. Every gateway would need 
to implement their own user storage tracking mechanism.  As discussed earlier 
in my previous mail, the better approach would be to establish an SSH 
connection with the Storage Resource and track the storage size in Airavata. 
After a little debugging and discussions, I found that StorageResourceAdaptor 
and AgentAdaptor’s (which uses pooling) are the way to establish a connection 
and execute commands on remote machines instead of creating a new 
SSHConnection, so my new implementation uses StorageResourceAdaptor to 
establish a connection and execute the 
du
 commands to get the size of the user directory.

Questions:


  1.  Currently, in case of Airavata-Django-Portal, experimentDataDir is always 
inside the project directiory which in turn is inside the user directory. Will 
the same behavior be followed across all the gateways?
  2.  I was trying to establish a connection with the gateway using 
StorageResourceAdaptor (I believe AgentAdaptor’s are used for compute 
resources) but I’m stuck at this step. I was successfully able to create a 
connection using AgentAdaptor (only for testing purposes) and execute commands 
on it. But I’m not able to do the same with StorageResourceAdaptor.
The only difference I can think of is, here I’m creating a new authToken for 
establishing a connection with the Storage resource (and of course deleting it 
at the end of execution), and I’m assuming this might be the issue. While 
testing with AgentAdaptor, I hardcoded the authToken value that’s already 
generated for the Jetstream compute resources, so I’m guessing that might be 
the reason why it was working in this case.

This is the error stacktrace that I’ve encountered while executing the 
‘executeCommand()’ on StorageResourceAdaptor after initiating it.

2020-08-18 19:31:07,152 [SSH-Pool-Monitor-airavata.host-2] INFO  
o.a.a.h.a.PoolingSSHJClient Current active connections for  null @ 
airavata.host : 2 are 0 []

2020-08-18 19:31:07,173 [pool-24-thread-6] INFO  n.s.s.transport.TransportImpl 
Client identity string: SSH-2.0-SSHJ_0.23.0 []

2020-08-18 19:31:07,180 [pool-24-thread-6] INFO  n.s.s.transport.TransportImpl 
Server identity string: SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.7 []

2020-08-18 19:31:07,488 [pool-24-thread-6] ERROR 
o.a.a.a.s.h.AiravataServerHandler net.schmizz.sshj.userauth.UserAuthException: 
Exhausted available authentication methods []

org.apache.airavata.agents.api.AgentException: 
net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication 
methods

at 
org.apache.airavata.helix.adaptor.SSHJAgentAdaptor.executeCommand(SSHJAgentAdaptor.java:190)

at 
org.apache.airavata.helix.adaptor.SSHJStorageAdaptor.executeCommand(SSHJStorageAdaptor.java:94)

at 
org.apache.airavata.api.server.handler.AiravataServerHandler.validateStorageLimit(AiravataServerHandler.java:726)

at 
org.apache.airavata.api.Airavata$Processor$validateStorageLimit.getResult(Airavata.java:18296)

at 
org.apache.airavata.api.Airavata$Processor$validateStorageLimit.getResult(Airavata.java:18280)

at 
org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)

at 
org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)

at 
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)

at 
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)

at 
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)

at java.base/java.lang.Thread.run(Thread.java:834)

Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available 
authentication methods

at net.schmizz.sshj.SSHClient.auth(SSHClient.java:231)

at 
org.apache.airavata.helix.adaptor.PoolingSSHJClient.createNewSSHClient(PoolingSSHJClient.java:273)

at 
org.apache.airavata.helix.adaptor.PoolingSSHJClient.newClientWithSessionValidation(PoolingSSHJClient.java:101)

at 
org.apache.airavata.helix.adaptor.PoolingSSHJClient.leaseSSHClient(PoolingSSHJClient.java:141)

at 
org.apache.airavata.helix.adaptor.PoolingSSHJClient.startSessionWrapper(PoolingSSHJClient.java:281)

at 
org.apache.airavata.helix.adaptor.SSHJAgentAdaptor.executeCommand(SSHJAgentAdaptor.java:173)

... 10 common frames omitted

This
 is the new validation api in AiravataServerHandler I’ve written, where I’m 
encountering the