[galaxy-user] Question about scripting

2013-04-19 Thread leconte

hello,

My subject is at the border between Galaxy and Condor

I'found method to use Galaxy with condor cluster solution.

In fact I have wrote a script when Galaxy call clustalw2 , it calls in 
fact a script called clustalw2.


All run exactly as I want excepting  ... but Galaxy get in error and 
said there was a problem



I turn my script to debug mode and it puts followings output

+ prog=/usr/bin/clustalw2
+ err=0
+ infile=-infile=/home/galaxy/galaxy-dist/database/files/000/dataset_11.dat
+ 
outfile=-outfile=/home/galaxy/galaxy-dist/database/job_working_directory/000/46/galaxy_dataset_80.dat
+ outorder=-OUTORDER=ALIGNED
+ seqnos=-SEQNOS=OFF
+ typeseq=-TYPE=DNA
+ var6=
+ var7=
+ var8=
+ var9=
+ echo

++ echo -infile=/home/galaxy/galaxy-dist/database/files/000/dataset_11.dat
++ sed -e s/-infile=//
+ infile2=/home/galaxy/galaxy-dist/database/files/000/dataset_11.dat
++ sed -e 's/galaxy_dataset_\(.*\)\.dat//'
++ sed -e s/-outfile=//
++ echo 
-outfile=/home/galaxy/galaxy-dist/database/job_working_directory/000/46/galaxy_dataset_80.dat
+ WorkingDir=/home/galaxy/galaxy-dist/database/job_working_directory/000/46/
+ mkdir -p /home/galaxy/galaxy-dist/database/job_working_directory/000/46/
+ mkdir -p /home/galaxy/galaxy-dist/database/job_working_directory/000/46/
+ mkdir -p /home/galaxy/galaxy-dist/database/job_working_directory/000/46/
+ [[ -n /usr/bin/clustalw2==clustalw2 ]]
+ echo 'universe= vanilla'
+ echo 'executable  = /usr/bin/mpirun '
+ echo 'arguments   = -np 10 -hostfile /home/galaxy/hostfile 
/usr/bin/clustalw-mpi -infile=/home/galaxy/galaxy-dist/database/files/000/dataset_11.dat 
-outfile=/home/galaxy/galaxy-dist/database/job_working_directory/000/46/galaxy_dataset_80.dat
  -OUTORDER=ALIGNED  -SEQNOS=OFF -TYPE=DNA '
+ echo 'Getenv  = True
machine_count   = 1
transfer_input_files= /home/galaxy/hostfile,/usr/bin/clustalw-mpi, 
/home/galaxy/galaxy-dist/database/files/000/dataset_11.dat
should_transfer_files   = yes
Log = /home/galaxy/logs/normal.$(cluster).$(process).log
Output  = /home/galaxy/logs/normal.$(cluster).$(process).out
Error   = /home/galaxy/logs/normal.$(cluster).$(process).error
notification= never
queue 1'
+ /usr/bin/condor_submit /home/galaxy/submit
Submitting job(s).
1 job(s) submitted to cluster 179.




I suspect the method I use, doesn't return properly information to 
galaxy ... but I don't know what I can do ...


Any Idea ...
___
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org.  Please keep all replies on the list by
using reply all in your mail client.  For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:

 http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists,
please use the interface at:

 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:

 http://galaxyproject.org/search/mailinglists/


Re: [galaxy-user] Regarding a cuffdiff output

2013-04-19 Thread Jennifer Hillman-Jackson

Hi Yona Kim,

On 4/18/13 9:54 PM, Yona Kim wrote:

Dear Jennifer

Thank you very much for your help for my analysis.
I'm still stuck on getting the final data from Cuffdiff analysis.
As you have mentioned, I've obtained the correct GTF file (mm9 gene
annotation), and also made sure that the reference genome and GTF file
are an exact match -  they both are mm9.


Are you using the iGenomes version of the GTF file? With the attributes 
Cuffdiff requires for generating all of the additional statistics? It 
appears that this is the case, but I just wanted to double check. If not 
using it, you can find a copy to load and use on the public server (if 
this is where you are working) in Shared Data - Data Libraries - 
iGenomes. Otherwise, it can be found at the Cufflinks web site.


These are the two attributes that are important to have, when available:
http://cufflinks.cbcb.umd.edu/manual.html#cuffdiff_input



When I view the output of transcript differential expression testing
(one of the outputs of cuffdiff) in excel, the names of the genes seem
to be properly annotated according to their location on chromosome, but
I have no values recorded for any of the calculations (I'm attaching
this file just in case you want to take a look at it).

The results you are getting indicate the data coverage is sparse, which 
aligns with your thoughts about this mapping not being as successful as 
prior runs:


NOTEST and LOWDATA are explained here with advice about parameter tuning:
   http://wiki.galaxyproject.org/Support#Tools_on_the_Main_server
follow links to Cufflinks FAQ to find:
   http://cufflinks.cbcb.umd.edu/faq.html#notest


Do you think that the problem might have been originated from fastq
files itself?

And also I was wondering about the reduce in the size of the files.
Comparing with one of my other analysis in galaxy, I realized that the
size of the file was significantly reduced from 6.1GB (fastq groomer) to
1006.9KB (Tophat accepted hits), whereas in my other analysis, the size
was reduced from 5.9GB (fastq groomer) only to 1.6 GB(Tophat accepted
hits).

Do you think there might have been an error occurred when Tophat was
running on the groomed data, and thus, providing an erroneous data to
Cufflinks, and eventually to Cuffdiff?


This could be the source of the problem. Making sure that the data was 
groomed correctly would be a good place the start. The comments from the 
first run will note the detected input type (but there can be some 
overlap), so also use the tool FastQC to help determine the proper 
settings for FASTQ Groomer. And if necessary, re-run from this step to 
see if that improves the mapping.


http://wiki.galaxyproject.org/Support#Dataset_special_cases
See the second bullet under FASTQ

If your query data is short (less than around 40 bases), then tuning 
Tophat could also improve mapping, see the tool's web page for advice 
regarding mapping shorter sequences. Then test out a few different 
parameter options to see what produces the best results for your 
particular datasets/samples. There is a balance between being too 
sensitive and too stringent - and this is a judgement call in most cases.


Trimming the reads may help if quality is an issue (FastQC will also 
give information about this). The RNA-seq example tutorial has an 
example of how to do basic QC:

https://main.g2.bx.psu.edu/u/jeremy/p/galaxy-rna-seq-analysis-exercise


Hopefully this helps to give some new options to test out that improve 
the result!


Jen
Galaxy team



Thank you very very much for your time and help

Sincerely yours,

Yona Kim
Department of Genetics
Rutgers University




On Mon, Apr 8, 2013 at 4:54 PM, Jennifer Jackson j...@bx.psu.edu
mailto:j...@bx.psu.edu wrote:

Hi Yona,

Yes, the GTF file is most likely the problem due to it lacking
certain attributes that Cuffdiff requires to perform these
calculations. You will also want to double check that the reference
genome and GTF file (where you source it next) are an exact match -
both the genome build and the identifier format. If either are not a
match, you will not get the expected or full results that Cuffdiff
can produce.

This wiki has some help;
http://wiki.galaxyproject.org/Support#Interpreting_scientific_results
See Tools on the Main server: Example → RNA-seq analysis tools.

The links to the Cufflinks web site explains the attributes that
Cuffdiff is looking for, links to the iGenomes datasets available
(best to use if your genome is represented), and a pointer to the
tool's user group. Two iGenomes GTF files are also already available
in Galaxy (hg19, mm9) in Shared Data - Data Libraries -
iGenomes. The link to our tutorial and FAQ has help about how the
GTF files are used along with troubleshooting advice.

Best,

Jen
Galaxy team


On 4/3/13 8:28 AM, Yona Kim wrote:

Dear galaxy users

Hello. I have a quick question about Cuffdiff analysis.