Hi John,

     Thanks for your reply.
Question 1:
      I used the relative path (e.g. ../../../applications/executable) which 
still does not work. I have managed to put the script in the same folder which 
works though against the initial design.
Question 2:
      I pass the $output to the perl script within which a file is generated 
with the filename contained in $output. The Galaxy server can recognize the 
output. However when I generate another file with the name 
"primary_$job.id_additional_visible_txt" in the same folder as the $output. In 
theory, it should also be recognized by Galaxy as being in the 
job_working_directory. But it is not. Could you reckon the reason? 
Question 3:
      Your explanation is very clear. Thank you again for this. The initial 
purpose of asking this question is related to question 2: I was wondering which 
"correct" folder I should use to put the additional optional outputs. Now as 
you suggested, I should not worry about it at all.
Question 4: 
     I have managed to convert a RAW file (900M) to mzML (2.3G). But if there 
is a limit on the file size, the spectral file maybe need to be split into 
multiple mgf files. 

Best regards!

-----Original Message-----
From: John Chilton [mailto:jmchil...@gmail.com] 
Sent: Monday, March 3, 2014 2:27 PM
To: Jun Fan
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] LWR questions

Hey Jun,

Thanks for your continued interest in the LWR. Responses inline.

On Sun, Mar 2, 2014 at 7:00 PM, Jun Fan <j....@qmul.ac.uk> wrote:
> Hi all,
>       LWR sounds very attractive and promising. However how to program 
> on galaxy side to utilize LWR server is not thoroughly documented.

Critique noted, I have tried to make it seamless but when you move a job from 
Linux to Windows there are bound to be severe limitations and I should document 
these more clearly and provide a guide for writing Windows compatible tools 

> I have
> several questions to ask:
> 1.       I separate the tool wrappers (.xml) and the applications (.pl, .py
> etc) in two different folders. In the traditional way it works fine. 
> When I use the legacy way of using LWR (adding galaxy:tool_runners 
> section in the universal_wsgi.ini file), the error message says that 
> the application cannot be found. Does the executable have to be in the 
> same folder with the wrapper?

Do you hard code absolute paths to your scripts in the tool XML files then? The 
LWR does not support that. If you place a single script in the same folder as 
the XML (a simple light-weight wrapper as is fairly
common) - that is supported - otherwise you will need to place all the required 
dependencies on the LWR's path on the remote Windows server.
This could get awkward if used with interpreter tag for instance - if you send 
me examples of your tools I can provide some guidance.

I guess in general my advice for Windows tools would be if you can use a simple 
one-file wrapper script placed next the tool you are fine - just make sure the 
underlying application is loaded from the path instead of via absolute paths - 
and then make sure all dependencies for the wrapper script are installed on the 
remote server and the underlying application is on the LWR's path.

(If your remote server is a *nix server there is new support for using 
tool_dependency_dir style Galaxy dependencies that resolve on the remote server 
- but this will not work with Windows (enhancements to provide this would be 
welcomed though) ).

> 2.       In the thread
> http://dev.list.galaxyproject.org/Managing-Data-Locality-td4662438.htm
> l#a4662570 John only mentioned $GALAXY_ROOT/databases/files, does this 
> mean that the dynamic multiple output is not supported as it uses 
> $GALAXY_ROOT/databases/tmp folder?


My understanding is that writing those files to the databases/tmp is somewhat 
deprecated - ideally these should be written to the working directory of the 
job. The newest version of the LWR *should* attempt to transfer various kinds 
of working directory files back to the Galaxy server after the job is complete 
- these files *should* include such variable output files. This hasn't been 
tested on Windows though
- if there are problems please let me know and I can try to address them -


> 3.       What types of files are supposed to be stored under tools directory
> (tool_files), working directory (working), inputs (inputs), configs 
> directory (configs), all of which are subfolders of 
> ${staging_directory}/${job_identifier}?

In theory, you shouldn't need to be worried about these implementations details 
as an application programmer, it should feel just like Galaxy (again with 
limitations).  But for what it is worth - the LWR client will transfer tool 
wrappers it infers are necessary to run the job to `tool_files`, the contents 
of any `configfile` templates defined for the job to `configs` 
and any inputs it infers are nessecary to `inputs`.

`working` is will be the working directory for the job - so any files your 
script or application writes to relative directories at runtime will be 
relative to this directory and finally all output paths (say you have $output 
defined in your Cheetah) will be rewritten so they target the outputs directory.

> 4.       Is there file size limit on the LWR server?

It is quite possible there is. Before I had left MSI we had transferred 
multi-gigabyte files to and from the LWR - but I don't know if we had ever 
actually transferred a file over 4Gb (an obvious potential limit). I get the 
sense that the underlying paste server is kind of taxed by these. Like Galaxy 
itself, it could presumably be hosted in a more robust web server. I have done 
some looking at chaussette for instance which sees to have support for Windows.

Hope this helps somewhat,

> Best regards!
> Jun
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this and other 
> Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/

Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

To search Galaxy mailing lists use the unified search at:

Reply via email to