[galaxy-dev] Galaxy web interface falls over when running TopHat
Hi,
We seem to have sorted out the problem of TopHat failing to run but now we have
a new problem. When TopHat runs with a large genome (but not with small
genomes), it finishes the run fine and all the data is there but the web
interface falls over about 8 hours into the run and when we try to access the
web based interface we get a Proxy Error. When we restart it all looks fine.
This is the sort of errors our HPC people find:
Exception happened during processing of request from ('XXX.XXX.XXX.XXX', 32960)
Traceback (most recent call last):
File /gpfs/cluster/isys/galaxy/Galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/pa
ste/httpserver.py, line 1053, in process_request_in_thread
Unexpected exception in worker function lambda at 0x201caed8
Traceback (most recent call last):
File /gpfs/cluster/isys/galaxy/Galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/pa
ste/httpserver.py, line 863, in worker_thread_callback
Unhandled exception in thread started by bound method Thread.__bootstrap of Th
read(worker 118, stopped 1127647552)
Traceback (most recent call last):
Do you have any suggestions for what is happening?
Best Wishes,
David.
__
Dr David A. Matthews
Senior Lecturer in Virology
Room E49
Department of Cellular and Molecular Medicine,
School of Medical Sciences
University Walk,
University of Bristol
Bristol.
BS8 1TD
U.K.
Tel. +44 117 3312058
Fax. +44 117 3312091
d.a.matth...@bristol.ac.uk
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
[galaxy-dev] How and where to install tool dependencies
Hello, I have a question which I have not seen specifically addressed in the online Galaxy wiki documentation about how to integrate tools (dependencies) into Galaxy. I have implemented a locally managed instance of Galaxy that my business is using with our cluster and now have a freshly installed and configured instance of Galaxy running. It is bare-bones right now and I did not use mercurial to sync any existing files/directory structures. I have seen the page on external tool dependencies ( http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Dependencies) needed for Galaxy, but I am somewhat unsure where to place the tools to utilize them as intended (other than through trial error). It appears that there are shell directories for the tools under ~/galaxy-dist/tools/ with basic wrapper scripts but without the corresponding executables (very few that I've noticed have the tools already in them). Is the intent to download the dependency tools and (building from source if necessary) take the binaries in those directories and copy them to their corresponding directory under ~/galaxy-dist/tools/? This seems to have worked with an error I first got when clipping a FASTQ file which reported that fastx_clipper was not a recognized command. So I downloaded the FASTX Toolkit, compiled the binaries, and copied only the binaries into the corresponding fastx tools directory. Would I do the same thing for TopHat and Cufflinks by taking all their binaries (combined) and copying them into ~/galaxy-dist/tools/ngs_rna/? Even if that is the case though, I have occasionally gotten errors about tools missing in completely different directories. One was for the FASTQ Groomer. One user saw this error in their browser (which for now is the only way I know to figure out where tools are *expected* to be): *File /home/galaxy/galaxy-dist/tools/rgenetics/rgFastQC.py, line 141, in assert os.path.isfile(opts.executable),'##rgFastQC.py error - cannot find executable %s' % opts.executable AssertionError: ##rgFastQC.py error - cannot find executable /home/galaxy/galaxy-dist/tool-data/shared/jars/FastQC/fastqc * To fix this I downloaded the FastQC tar file from its webpage, unzipped it, and copied the fastqc binary/script to the home/galaxy/galaxy -dist/tool-data/shared/jars/FastQC/** directory. I also had to mkdir FastQC/ under jars/ to place it there since it didn't already exist. Had I not been told the specific directory by the error I'm not sure how I would have intuitively known to place the binary there (unless I'm overlooking some critical documentation). And how do I know that other similar things are not missing which should be there? Can anyone shed some light on this please? Adding a brief page on the Galaxy wiki site under the Admin section about this would really help, even if it only showed an example for one or two specific tools. Thanks, Josh Nielsen ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Galaxy web interface falls over when running TopHat
David,
What is the full stack trace that you're seeing? The stack trace is the text
below Traceback and identifies the exact location where the problem occurs.
Are you following these guidelines:
http://wiki.g2.bx.psu.edu/Admin/NGS%20Local%20Setup
for setting up large genomes in Galaxy?
Also, it would be ideal if you could upload the problematic data--genome +
reads--to our public instance (main.g2.bx.psu.edu) and see if you can reproduce
the problem that you're seeing.
Thanks,
J.
On Dec 9, 2011, at 12:20 PM, David Matthews wrote:
Hi,
We seem to have sorted out the problem of TopHat failing to run but now we
have a new problem. When TopHat runs with a large genome (but not with small
genomes), it finishes the run fine and all the data is there but the web
interface falls over about 8 hours into the run and when we try to access the
web based interface we get a Proxy Error. When we restart it all looks fine.
This is the sort of errors our HPC people find:
Exception happened during processing of request from ('XXX.XXX.XXX.XXX',
32960)
Traceback (most recent call last):
File
/gpfs/cluster/isys/galaxy/Galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/pa
ste/httpserver.py, line 1053, in process_request_in_thread
Unexpected exception in worker function lambda at 0x201caed8
Traceback (most recent call last):
File
/gpfs/cluster/isys/galaxy/Galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/pa
ste/httpserver.py, line 863, in worker_thread_callback
Unhandled exception in thread started by bound method Thread.__bootstrap of
Th
read(worker 118, stopped 1127647552)
Traceback (most recent call last):
Do you have any suggestions for what is happening?
Best Wishes,
David.
__
Dr David A. Matthews
Senior Lecturer in Virology
Room E49
Department of Cellular and Molecular Medicine,
School of Medical Sciences
University Walk,
University of Bristol
Bristol.
BS8 1TD
U.K.
Tel. +44 117 3312058
Fax. +44 117 3312091
d.a.matth...@bristol.ac.uk
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] How and where to install tool dependencies
Josh, It appears that there are shell directories for the tools under ~/galaxy-dist/tools/ with basic wrapper scripts but without the corresponding executables (very few that I've noticed have the tools already in them). Is the intent to download the dependency tools and (building from source if necessary) take the binaries in those directories and copy them to their corresponding directory under ~/galaxy-dist/tools/? This seems to have worked with an error I first got when clipping a FASTQ file which reported that fastx_clipper was not a recognized command. So I downloaded the FASTX Toolkit, compiled the binaries, and copied only the binaries into the corresponding fastx tools directory. Would I do the same thing for TopHat and Cufflinks by taking all their binaries (combined) and copying them into ~/galaxy-dist/tools/ngs_rna/? You'll want to read about Galaxy Tool files a bit to understand the files in ~/galaxy-dist/tools: http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#Admin.2BAC8-Tools.2BAC8-Tool_Config_Syntax.Galaxy_Tool_XML_File These are not shell directories; instead, they include tool config files + additional wrapper scripts to run a tool in Galaxy. To answer your question, executables for tools need to be in your path but do not need to be in the config/wrapper directories. For example, in an SGE cluster, we suggest setting the PATH environment var in ~/.sge_request Even if that is the case though, I have occasionally gotten errors about tools missing in completely different directories. One was for the FASTQ Groomer. One user saw this error in their browser (which for now is the only way I know to figure out where tools are *expected* to be): File /home/galaxy/galaxy-dist/tools/rgenetics/rgFastQC.py, line 141, in assert os.path.isfile(opts.executable),'##rgFastQC.py error - cannot find executable %s' % opts.executable AssertionError: ##rgFastQC.py error - cannot find executable /home/galaxy/galaxy-dist/tool-data/shared/jars/FastQC/fastqc The exception to the above is Java-based tools. For these tools, you'll need to use the ~/galaxy-dist/shared/jars directory. This is a limitation of Galaxy that will likely be addressed in the future. Adding a brief page on the Galaxy wiki site under the Admin section about this would really help, even if it only showed an example for one or two specific tools. I looked a bit but couldn't find it; I suspect it is out on the wiki somewhere, though clearly it needs to be easier to find. Good luck, J. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
