Hi Bhanu & Suresh, In PGA we assume that PGA and Airavata are located in two different hosts. So when creating experiments from pga for input files we provide the path in remote format (i.e user@host:/<path>) and assume that AdvancedSCPInputHandler is enabled. AdvancedSCPInputHandler will first copy the files to Airavata host and change the file paths to local file paths in the job execution context.
For the SSHInputHandler to copy the files the file paths should be in local format. I think in Bhanu's setting this fails because we haven't enabled AdvancedSCPInputHandler so the file paths are in remote format and SSHInputHandler fails to copy the files. To verify this issue we can try enabling AdvancedSCPInputHandler to copy files to a different folder. If that is the case then we can provide a fix from PGA or side or Airavata side. On Wed, Aug 26, 2015 at 12:30 AM, Bhanu Rekapalli <[email protected]> wrote: > Hi Suresh, > Caught up in meetings and demo till now. > Do you want both AA and PGA folders? > Bhanu > > On Tue, Aug 25, 2015 at 12:34 PM, Suresh Marru <[email protected]> > wrote: > >> Yes we tried that and it works ok. I think this has a small issue >> somewhere in parsing the URL’s and handshake between PGA and Airavata on >> setting these file paths. >> >> Suresh >> >> On Aug 25, 2015, at 12:26 PM, Pierce, Marlon <[email protected]> wrote: >> >> Is there a way to check SCP on the command line? >> >> From: Suresh Marru <[email protected]> >> Reply-To: Airavata Users <[email protected]> >> Date: Tuesday, August 25, 2015 at 11:49 AM >> To: Airavata Users <[email protected]> >> Subject: Re: Issues moving data from my PGA server to stampede >> >> Hi Supun, Shameera, >> >> We can reproduce this issue the following way: >> >> Install a PGA and Airavata locally, upload a file and have Airavata stage >> it to XSEDE resource. When the hostname name is localhost, something is not >> picking up the file path. Can we try and reproduce it on one of the VM? >> >> Hi Bhanu, >> >> Is it possible to save the Amazon VM with all the installation you have >> and share it with us? We can then deploy that on EC2 instance and see if we >> can get it working. >> >> Suresh >> >> >> On Aug 25, 2015, at 11:19 AM, Bhanu Rekepalli <[email protected]> wrote: >> >> Hi: >> I am hosting PGA on EC2 instance and I currently cannot move data from >> EC2 to Stampede. >> On the contrary I can get data files from Stampede to EC2 instance. >> >> Following is the error I am getting. >> org.apache.airavata.gfac.core.handler.GFacHandlerException: Error while >> input File Staging at >> org.apache.airavata.gfac.ssh.handler.SSHInputHandler.stageInputFiles(SSHInputHandler.java:191) >> at >> org.apache.airavata.gfac.ssh.handler.SSHInputHandler.invoke(SSHInputHandler.java:105) >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.invokeInFlowHandlers(BetterGfacImpl.java:895) >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.launch(BetterGfacImpl.java:686) >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.submitJob(BetterGfacImpl.java:492) >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.submitJob(BetterGfacImpl.java:183) >> at >> org.apache.airavata.gfac.core.utils.InputHandlerWorker.run(InputHandlerWorker.java:47) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >> at java.lang.Thread.run(Thread.java:745) Caused by: >> org.apache.airavata.gfac.core.handler.GFacHandlerException: Error while >> input File Staging at >> org.apache.airavata.gfac.ssh.handler.SSHInputHandler.stageInputFiles(SSHInputHandler.java:183) >> ... 9 more Caused by: org.apache.airavata.gsi.ssh.api.SSHApiException: >> Failed during scping local >> file://[email protected]:/var/www/html/experimentData/rekepalli7b79a4a79919d9cd100656fadf897e89/testFasta >> to remote file stampede.tacc.xsede.org:rFile : >> /scratch/01437/ogce/usda-workdirs/mt_68c0b9ee-7ea9-4d5b-bff8-e151091a07c0/testFasta >> at >> org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster.scpTo(GSISSHAbstractCluster.java:470) >> at >> org.apache.airavata.gfac.ssh.handler.SSHInputHandler.stageInputFiles(SSHInputHandler.java:177) >> ... 9 more Caused by: java.io.FileNotFoundException: >> /[email protected]:/var/www/html/experimentData/rekepalli7b79a4a79919d9cd100656fadf897e89/testFasta >> (No such file or directory) at java.io.FileInputStream.open0(Native Method) >> at java.io.FileInputStream.open(FileInputStream.java:195) at >> java.io.FileInputStream.(FileInputStream.java:138) at >> java.io.FileInputStream.(FileInputStream.java:93) at >> org.apache.airavata.gsi.ssh.util.SSHUtils.scpTo(SSHUtils.java:293) at >> org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster.scpTo(GSISSHAbstractCluster.java:463) >> ... 10 more org.apache.airavata.gfac.GFacException: Error Invoking >> Handlers:Error Executing a InFlow Handler >> org.apache.airavata.gfac.GFacException: Error launching the Job at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.submitJob(BetterGfacImpl.java:505) >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.submitJob(BetterGfacImpl.java:183) >> at >> org.apache.airavata.gfac.core.utils.InputHandlerWorker.run(InputHandlerWorker.java:47) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >> at java.lang.Thread.run(Thread.java:745) Caused by: >> org.apache.airavata.gfac.GFacException: Error Invoking Handlers:Error >> Executing a InFlow Handler at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.launch(BetterGfacImpl.java:743) >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.submitJob(BetterGfacImpl.java:492) >> ... 5 more Caused by: org.apache.airavata.gfac.GFacException: Error >> Invoking Handlers:Error Executing a InFlow Handler at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.invokeInFlowHandlers(BetterGfacImpl.java:910) >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.launch(BetterGfacImpl.java:686) >> ... 6 more Caused by: org.apache.airavata.gfac.GFacException: Error >> Executing a InFlow Handler at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.invokeInFlowHandlers(BetterGfacImpl.java:899) >> ... 7 more Caused by: >> org.apache.airavata.gfac.core.handler.GFacHandlerException: Error while >> input File Staging at >> org.apache.airavata.gfac.ssh.handler.SSHInputHandler.stageInputFiles(SSHInputHandler.java:191) >> at >> org.apache.airavata.gfac.ssh.handler.SSHInputHandler.invoke(SSHInputHandler.java:105) >> at >> org.apache.airavata.gfac.core.cpi.BetterGfacImpl.invokeInFlowHandlers(BetterGfacImpl.java:895) >> ... 7 more Caused by: >> org.apache.airavata.gfac.core.handler.GFacHandlerException: Error while >> input File Staging at >> org.apache.airavata.gfac.ssh.handler.SSHInputHandler.stageInputFiles(SSHInputHandler.java:183) >> ... 9 more Caused by: org.apache.airavata.gsi.ssh.api.SSHApiException: >> Failed during scping local >> file://[email protected]:/var/www/html/experimentData/rekepalli7b79a4a79919d9cd100656fadf897e89/testFasta >> to remote file stampede.tacc.xsede.org:rFile : >> /scratch/01437/ogce/usda-workdirs/mt_68c0b9ee-7ea9-4d5b-bff8-e151091a07c0/testFasta >> at >> org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster.scpTo(GSISSHAbstractCluster.java:470) >> at >> org.apache.airavata.gfac.ssh.handler.SSHInputHandler.stageInputFiles(SSHInputHandler.java:177) >> ... 9 more Caused by: java.io.FileNotFoundException: >> /[email protected]:/var/www/html/experimentData/rekepalli7b79a4a79919d9cd100656fadf897e89/testFasta >> (No such file or directory) at java.io.FileInputStream.open0(Native Method) >> at java.io.FileInputStream.open(FileInputStream.java:195) at >> java.io.FileInputStream.(FileInputStream.java:138) at >> java.io.FileInputStream.(FileInputStream.java:93) at >> org.apache.airavata.gsi.ssh.util.SSHUtils.scpTo(SSHUtils.java:293) at >> org.apache.airavata.gsi.ssh.impl.GSISSHAbstractCluster.scpTo(GSISSHAbstractCluster.java:463) >> ... 10 more >> >> >> >> > > > -- > Bhanu Rekepalli, PhD > Sr. Scientific Consultant & Principal Investigator > BioTeam <http://bioteam.net/>: Enabling Science > > 7 Derosier Drive, Middleton, MA 01949 > Off. Phones: 800-293-7090, 865-240-2910 > Cell: 865-230-1605 > -- Thank you Supun Nakandala Dept. Computer Science and Engineering University of Moratuwa
