Re: [galaxy-dev] No module named galaxy !!
On 06/19/2012 07:11 PM, Chebbi Mohamed Amine wrote: Hi Galaxy team ! I'am trying to remove userless histories from my galaxy database as mentoned in this link ( by using the command above : python cleanup_datasets.py universe_wsgi.ini -d 60 -1 However I get an error message like this : File cleanup_datasets.py, line 13, in module from galaxy import eggs ImportError: No module named galaxy looks like you are calling the script from: ~/galaxy_dist/scripts/cleanup_datasets/ try calling it with: python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -d 60 -1 from: ~/galaxy_dist/ Regards, Hans Although I set cleanup_job = never in my Galaxy config and check the PBS script like mentioned in the mailing list discussions as above , i have always the same problem of galaxy module : GALAXY_LIB=/path/to/galaxy-dist/lib if [ $GALAXY_LIB != None ]; then if [ -n $PYTHONPATH ]; then export PYTHONPATH=$GALAXY_LIB:$PYTHONPATH else export PYTHONPATH=$GALAXY_LIB fi fi Could someone help me please ! Thanks Amine ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] No module named galaxy !!
Hi Hans-Rudolf; Yes!! I didn't pay attention and i was really calling the script fom ~/galaxy_dist/scripts/cleanup_datasets/. Now when I call it from ~/galaxy_dist/ it works very well. Thanks :) Amine 2012/6/20 Hans-Rudolf Hotz h...@fmi.ch On 06/19/2012 07:11 PM, Chebbi Mohamed Amine wrote: Hi Galaxy team ! I'am trying to remove userless histories from my galaxy database as mentoned in this link ( by using the command above : python cleanup_datasets.py universe_wsgi.ini -d 60 -1 However I get an error message like this : File cleanup_datasets.py, line 13, in module from galaxy import eggs ImportError: No module named galaxy looks like you are calling the script from: ~/galaxy_dist/scripts/cleanup_**datasets/ try calling it with: python scripts/cleanup_datasets/**cleanup_datasets.py universe_wsgi.ini -d 60 -1 from: ~/galaxy_dist/ Regards, Hans Although I set cleanup_job = never in my Galaxy config and check the PBS script like mentioned in the mailing list discussions as above , i have always the same problem of galaxy module : GALAXY_LIB=/path/to/galaxy-**dist/lib if [ $GALAXY_LIB != None ]; then if [ -n $PYTHONPATH ]; then export PYTHONPATH=$GALAXY_LIB:$**PYTHONPATH else export PYTHONPATH=$GALAXY_LIB fi fi Could someone help me please ! Thanks Amine __**_ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] samtools view region
Hi all, I was working on the integration of the samtools view to work with Galaxy. The idea is to filter an indexed bam file on a chromosome with samtools view -bh in.bam.bai chr1. After some research I noted that samtools will only execute this command when it can find the corresponding sorted bam file, so the in.bam. The problem is that Galaxy renames all the files. So in.bam will be for instance dataset_1.dat. When I index the bam file I will get an indexed bam file named dataset_2.dat. Samtools can't detect then that dataset_1.dat is the sorted bam file and dataset_2.dat is the bai file. Now I was wondering if it is possible to rename files in the ~/galaxy-dist/database/files/000 with a command line. So that dataset_2.dat will be moved to dataset_1.dat.bai. Does anyone know how to do that? Kind Regards, Jaap ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] samtools view region
On Wed, Jun 20, 2012 at 2:59 PM, j.w.f.van_der_heij...@lumc.nl wrote: Hi all, I was working on the integration of the samtools view to work with Galaxy. The idea is to filter an indexed bam file on a chromosome with samtools view -bh in.bam.bai chr1. After some research I noted that samtools will only execute this command when it can find the corresponding sorted bam file, so the in.bam. The problem is that Galaxy renames all the files. So in.bam will be for instance dataset_1.dat. When I index the bam file I will get an indexed bam file named dataset_2.dat. Samtools can't detect then that dataset_1.dat is the sorted bam file and dataset_2.dat is the bai file. Now I was wondering if it is possible to rename files in the ~/galaxy-dist/database/files/000 with a command line. So that dataset_2.dat will be moved to dataset_1.dat.bai. Does anyone know how to do that? Kind Regards, Jaap The simple answer is a wrapper script which makes symlinks for the BAM file and its BAI file using the normal naming conventions. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] samtools view region
Hi Peter, Thanks for you quick answer! But I am very new to Galaxy. Could you explain something about those symlinks? Regards, Jaap From: Peter Cock [p.j.a.c...@googlemail.com] Sent: Wednesday, June 20, 2012 4:01 PM To: Heijden, J.W.F. van der (HG) Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] samtools view region On Wed, Jun 20, 2012 at 2:59 PM, j.w.f.van_der_heij...@lumc.nl wrote: Hi all, I was working on the integration of the samtools view to work with Galaxy. The idea is to filter an indexed bam file on a chromosome with samtools view -bh in.bam.bai chr1. After some research I noted that samtools will only execute this command when it can find the corresponding sorted bam file, so the in.bam. The problem is that Galaxy renames all the files. So in.bam will be for instance dataset_1.dat. When I index the bam file I will get an indexed bam file named dataset_2.dat. Samtools can't detect then that dataset_1.dat is the sorted bam file and dataset_2.dat is the bai file. Now I was wondering if it is possible to rename files in the ~/galaxy-dist/database/files/000 with a command line. So that dataset_2.dat will be moved to dataset_1.dat.bai. Does anyone know how to do that? Kind Regards, Jaap The simple answer is a wrapper script which makes symlinks for the BAM file and its BAI file using the normal naming conventions. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] tool_data_table_config.xml.sample
I have to refine that statement about the error message I'm getting now: On upload I get the same error message Metadata was defined for some items in revision 'e183f52194a5'. Correct the following problems if necessary and reset metadata. join.xml - This file refers to a missing file cg_crr_files.loc. Upload a file named cg_crr_files.loc.sample to the repository to correct this error. junctiondiff.xml - This file refers to a missing file cg_crr_files.loc. Upload a file named cg_crr_files.loc.sample to the repository to correct this error. listtestvariants.xml - This file refers to a missing file cg_crr_files.loc. Upload a file named cg_crr_files.loc.sample to the repository to correct this error. listvariants.xml - This file refers to a missing file cg_crr_files.loc. Upload a file named cg_crr_files.loc.sample to the repository to correct this error. snpdiff.xml - This file refers to a missing file cg_crr_files.loc. Upload a file named cg_crr_files.loc.sample to the repository to correct this error. testvariants.xml - This file refers to a missing file cg_crr_files.loc. Upload a file named cg_crr_files.loc.sample to the repository to correct this error. varfilter.xml - This file refers to a missing file cg_crr_files.loc. Upload a file named cg_crr_files.loc.sample to the repository to correct this error. One tool (calldiff.xml) shows as valid tool, but clicking on that tool I get the message: In green box: Error loading tool: [Errno 2] No such file or directory: '/var/opt/galaxy/g2cmnty/galaxy_toolshed/shed-tool-data/cg_crr_files.loc'. Below that: Tool not properly loaded. The tools displayed as invalid tools just show the 'Tool not properly loaded' message. This is what I got in the main toolshed. I went back to the test toolshed and uploaded the same tar ball there, none of the tools loaded properly. Hope this helps. Birgit Crain, Ph.D. | Sr. Professional Services Scientist | Complete Genomics, Inc. (650) 428-6023 office | (408) 605-3938 mobile bcr...@completegenomics.commailto:bcr...@completegenomics.com From: Birgit Crain bcr...@completegenomics.commailto:bcr...@completegenomics.com Date: Tuesday, June 19, 2012 8:25 AM To: Greg Von Kuster g...@bx.psu.edumailto:g...@bx.psu.edu Cc: galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] tool_data_table_config.xml.sample Same file structure, but updated files and added the executable and tool_config.xml.sample for users who install manually. Here's the tar ball. Birgit Crain, Ph.D. | Sr. Professional Services Scientist | Complete Genomics, Inc. (650) 428-6023 office | (408) 605-3938 mobile bcr...@completegenomics.commailto:bcr...@completegenomics.com From: Greg Von Kuster g...@bx.psu.edumailto:g...@bx.psu.edu Date: Monday, June 18, 2012 5:29 PM To: Birgit Crain bcr...@completegenomics.commailto:bcr...@completegenomics.com Cc: galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] tool_data_table_config.xml.sample Hi Birgit, Did you upload the same tarball you sent me previously, or was it a different tarball. If different, can you send it to me? Thanks On Jun 18, 2012, at 8:18 PM, Birgit Crain wrote: Hi Greg I just created a repository on the main toolshed and upload a tar ball. I got the same error messages again as described in the email thread below. Regards Birgit Crain, Ph.D. | Sr. Professional Services Scientist | Complete Genomics, Inc. (650) 428-6023 office | (408) 605-3938 mobile bcr...@completegenomics.commailto:bcr...@completegenomics.com From: Greg Von Kuster g...@bx.psu.edumailto:g...@bx.psu.edu Date: Thursday, June 14, 2012 11:44 AM To: Birgit Crain bcr...@completegenomics.commailto:bcr...@completegenomics.com Cc: galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] tool_data_table_config.xml.sample Hello Birgit, This issue has been resolved in changeset 7272:b761471d7590, which is currently available from our Galaxy central repository. Both the Galaxy test and Galaxy main tool sheds are running this latest changeset revision. Thanks very much for reporting this problem! Greg Von Kuster On Jun 14, 2012, at 12:45 PM, Birgit Crain wrote: Hi Greg I was using the Galaxy test toolshed to upload the files into a repository. On my Mac where I develop the tools I have a local instance that I loaded from zipped file (downloaded April galaxy-galaxy-dist-40f1816d6857.zip) running python 2.7. When I download the repository to a local instance on our server the tools and .loc file install fine (except that I don't have my datatypes defined right, working on that). Birgit Crain, Ph.D. | Sr. Professional Services Scientist | Complete Genomics, Inc. (650) 428-6023 office | (408) 605-3938 mobile
[galaxy-dev] circular import in tool_shed code gives unit test failure
Dear, I've been running unit tests on the latest galaxy-central 7294:663e03e40c86 that have me this error: [localhost] out: == [localhost] out: ERROR: Failure: ImportError (cannot import name install_manager) [localhost] out: -- [localhost] out: Traceback (most recent call last): [localhost] out: File /Users/pajon01/galaxy-test/galaxy-central/eggs/nose-0.11.1-py2.7.egg/nose/loader.py, line 379, in loadTestsFromName [localhost] out: addr.filename, addr.module) [localhost] out: File /Users/pajon01/galaxy-test/galaxy-central/eggs/nose-0.11.1-py2.7.egg/nose/importer.py, line 39, in importFromPath [localhost] out: return self.importFromDir(dir_path, fqname) [localhost] out: File /Users/pajon01/galaxy-test/galaxy-central/eggs/nose-0.11.1-py2.7.egg/nose/importer.py, line 86, in importFromDir [localhost] out: mod = load_module(part_fqname, fh, filename, desc) [localhost] out: File /Users/pajon01/galaxy-test/galaxy-central/lib/galaxy/tool_shed/install_manager.py, line 10, in module [localhost] out: from galaxy.tool_shed.migrate.common import * [localhost] out: File /Users/pajon01/galaxy-test/galaxy-central/lib/galaxy/tool_shed/migrate/common.py, line 9, in module [localhost] out: from galaxy.tool_shed import install_manager [localhost] out: ImportError: cannot import name install_manager [localhost] out: -- I fixed it by commenting out the circular import in tool_shed/install_manager.py hg diff diff -r 663e03e40c86 lib/galaxy/tool_shed/install_manager.py --- a/lib/galaxy/tool_shed/install_manager.py Wed Jun 20 03:39:05 2012 -0400 +++ b/lib/galaxy/tool_shed/install_manager.py Wed Jun 20 16:37:06 2012 +0100 @@ -7,7 +7,7 @@ from galaxy.util.json import from_json_string, to_json_string from galaxy.util.shed_util import * from galaxy.util.odict import odict -from galaxy.tool_shed.migrate.common import * +#from galaxy.tool_shed.migrate.common import * Hope it would be useful to someone else. Kind regards, Anne. -- Anne Pajon, Ph.D. Cancer Research UK - Cambridge Research Institute Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE anne.pa...@cancer.org.uk | +44 (0)1223 404 334 NOTICE AND DISCLAIMER This e-mail (including any attachments) is intended for the above-named person(s). If you are not the intended recipient, notify the sender immediately, delete this email from your system and do not disclose or use for any purpose. We may monitor all incoming and outgoing emails in line with current legislation. We have taken steps to ensure that this email and attachments are free from any virus, but it remains your responsibility to ensure that viruses do not adversely affect you. Cancer Research UK Registered in England and Wales Company Registered Number: 4325234. Registered Charity Number: 1089464 and Scotland SC041666 Registered Office Address: Angel Building, 407 St John Street, London EC1V 4AD. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] circular import in tool_shed code gives unit test failure
Thanks Anne, I'll make sure this fix makes it into my next commit to Galaxy central. Greg On Jun 20, 2012, at 11:42 AM, Anne Pajon wrote: Dear, I've been running unit tests on the latest galaxy-central 7294:663e03e40c86 that have me this error: [localhost] out: == [localhost] out: ERROR: Failure: ImportError (cannot import name install_manager) [localhost] out: -- [localhost] out: Traceback (most recent call last): [localhost] out: File /Users/pajon01/galaxy-test/galaxy-central/eggs/nose-0.11.1-py2.7.egg/nose/loader.py, line 379, in loadTestsFromName [localhost] out: addr.filename, addr.module) [localhost] out: File /Users/pajon01/galaxy-test/galaxy-central/eggs/nose-0.11.1-py2.7.egg/nose/importer.py, line 39, in importFromPath [localhost] out: return self.importFromDir(dir_path, fqname) [localhost] out: File /Users/pajon01/galaxy-test/galaxy-central/eggs/nose-0.11.1-py2.7.egg/nose/importer.py, line 86, in importFromDir [localhost] out: mod = load_module(part_fqname, fh, filename, desc) [localhost] out: File /Users/pajon01/galaxy-test/galaxy-central/lib/galaxy/tool_shed/install_manager.py, line 10, in module [localhost] out: from galaxy.tool_shed.migrate.common import * [localhost] out: File /Users/pajon01/galaxy-test/galaxy-central/lib/galaxy/tool_shed/migrate/common.py, line 9, in module [localhost] out: from galaxy.tool_shed import install_manager [localhost] out: ImportError: cannot import name install_manager [localhost] out: -- I fixed it by commenting out the circular import in tool_shed/install_manager.py hg diff diff -r 663e03e40c86 lib/galaxy/tool_shed/install_manager.py --- a/lib/galaxy/tool_shed/install_manager.py Wed Jun 20 03:39:05 2012 -0400 +++ b/lib/galaxy/tool_shed/install_manager.py Wed Jun 20 16:37:06 2012 +0100 @@ -7,7 +7,7 @@ from galaxy.util.json import from_json_string, to_json_string from galaxy.util.shed_util import * from galaxy.util.odict import odict -from galaxy.tool_shed.migrate.common import * +#from galaxy.tool_shed.migrate.common import * Hope it would be useful to someone else. Kind regards, Anne. -- Anne Pajon, Ph.D. Cancer Research UK - Cambridge Research Institute Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE anne.pa...@cancer.org.uk | +44 (0)1223 404 334 NOTICE AND DISCLAIMER This e-mail (including any attachments) is intended for the above-named person(s). If you are not the intended recipient, notify the sender immediately, delete this email from your system and do not disclose or use for any purpose. We may monitor all incoming and outgoing emails in line with current legislation. We have taken steps to ensure that this email and attachments are free from any virus, but it remains your responsibility to ensure that viruses do not adversely affect you. Cancer Research UK Registered in England and Wales Company Registered Number: 4325234. Registered Charity Number: 1089464 and Scotland SC041666 Registered Office Address: Angel Building, 407 St John Street, London EC1V 4AD. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] tool_data_table_config.xml.sample
Thanks very much for the information Birgit - I'll be taking a look at this shortly and get back to you when I have the fix committed. On Jun 20, 2012, at 11:41 AM, Birgit Crain wrote: I have to refine that statement about the error message I'm getting now: On upload I get the same error message Metadata was defined for some items in revision 'e183f52194a5'. Correct the following problems if necessary and reset metadata. join.xml - This file refers to a missing file cg_crr_files.loc. Upload a file named cg_crr_files.loc.sample to the repository to correct this error. junctiondiff.xml - This file refers to a missing file cg_crr_files.loc. Upload a file named cg_crr_files.loc.sample to the repository to correct this error. listtestvariants.xml - This file refers to a missing file cg_crr_files.loc. Upload a file named cg_crr_files.loc.sample to the repository to correct this error. listvariants.xml - This file refers to a missing file cg_crr_files.loc. Upload a file named cg_crr_files.loc.sample to the repository to correct this error. snpdiff.xml - This file refers to a missing file cg_crr_files.loc. Upload a file named cg_crr_files.loc.sample to the repository to correct this error. testvariants.xml - This file refers to a missing file cg_crr_files.loc. Upload a file named cg_crr_files.loc.sample to the repository to correct this error. varfilter.xml - This file refers to a missing file cg_crr_files.loc. Upload a file named cg_crr_files.loc.sample to the repository to correct this error. One tool (calldiff.xml) shows as valid tool, but clicking on that tool I get the message: In green box: Error loading tool: [Errno 2] No such file or directory: '/var/opt/galaxy/g2cmnty/galaxy_toolshed/shed-tool-data/cg_crr_files.loc'. Below that: Tool not properly loaded. The tools displayed as invalid tools just show the 'Tool not properly loaded' message. This is what I got in the main toolshed. I went back to the test toolshed and uploaded the same tar ball there, none of the tools loaded properly. Hope this helps. Birgit Crain, Ph.D. | Sr. Professional Services Scientist | Complete Genomics, Inc. (650) 428-6023 office | (408) 605-3938 mobile bcr...@completegenomics.com From: Birgit Crain bcr...@completegenomics.com Date: Tuesday, June 19, 2012 8:25 AM To: Greg Von Kuster g...@bx.psu.edu Cc: galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] tool_data_table_config.xml.sample Same file structure, but updated files and added the executable and tool_config.xml.sample for users who install manually. Here's the tar ball. Birgit Crain, Ph.D. | Sr. Professional Services Scientist | Complete Genomics, Inc. (650) 428-6023 office | (408) 605-3938 mobile bcr...@completegenomics.com From: Greg Von Kuster g...@bx.psu.edu Date: Monday, June 18, 2012 5:29 PM To: Birgit Crain bcr...@completegenomics.com Cc: galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] tool_data_table_config.xml.sample Hi Birgit, Did you upload the same tarball you sent me previously, or was it a different tarball. If different, can you send it to me? Thanks On Jun 18, 2012, at 8:18 PM, Birgit Crain wrote: Hi Greg I just created a repository on the main toolshed and upload a tar ball. I got the same error messages again as described in the email thread below. Regards Birgit Crain, Ph.D. | Sr. Professional Services Scientist | Complete Genomics, Inc. (650) 428-6023 office | (408) 605-3938 mobile bcr...@completegenomics.com From: Greg Von Kuster g...@bx.psu.edu Date: Thursday, June 14, 2012 11:44 AM To: Birgit Crain bcr...@completegenomics.com Cc: galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] tool_data_table_config.xml.sample Hello Birgit, This issue has been resolved in changeset 7272:b761471d7590, which is currently available from our Galaxy central repository. Both the Galaxy test and Galaxy main tool sheds are running this latest changeset revision. Thanks very much for reporting this problem! Greg Von Kuster On Jun 14, 2012, at 12:45 PM, Birgit Crain wrote: Hi Greg I was using the Galaxy test toolshed to upload the files into a repository. On my Mac where I develop the tools I have a local instance that I loaded from zipped file (downloaded April galaxy-galaxy-dist-40f1816d6857.zip) running python 2.7. When I download the repository to a local instance on our server the tools and .loc file install fine (except that I don't have my datatypes defined right, working on that). Birgit Crain, Ph.D. | Sr. Professional Services Scientist | Complete Genomics, Inc. (650) 428-6023 office | (408) 605-3938 mobile bcr...@completegenomics.com From: Greg Von Kuster g...@bx.psu.edu Date: Wednesday, June 13, 2012 6:05 PM To: Birgit Crain bcr...@completegenomics.com
Re: [galaxy-dev] Set job_working_directory to a cluster compute node /tmp
Hi Derrick, If you use outputs_to_working_directory = True and put the working directory on the SSD pool, writing during job execution will use the fast space. Once the job completes, the outputs will be moved back to the larger, slower pool configured for file_path. --nate On Jun 19, 2012, at 8:08 AM, Derrick Lin wrote: Another relevant question, my institute has configured a NFS volume that based on a SSD disk pool on the file server. I want to use it for improving galaxy's job execution on big dataset. However, the SSD volume has only 2.5TB (they are very expensive...). So migrate the entire database folder to there is impossible. Any recommendation for the galaxy to have a good use of the SSD? Thanks Derrick On Tue, Jun 19, 2012 at 9:55 PM, Derrick Lin klin...@gmail.com wrote: I think my question has been answered: http://gmod.827538.n3.nabble.com/NFS-Cluster-and-working-directories-questions-td3784165.html Hopefully can see the enhancements in the near future. Cheers. Derrick On Tue, Jun 19, 2012 at 5:01 PM, Derrick Lin klin...@gmail.com wrote: Hi guys, I have deploy a galaxy on a cluster (so I installed it on a NFS share that it's accessible by all cluster compute nodes). Everything is running fine. Now I am looking for a way such that every job dispatched to a compute node uses that node's local /tmp as working directory. I know galaxy config provides job_working_directory for the similar purpose. My question really is, while all my compute nodes can access the NFS share where galaxy installed, but the galaxy host cannot access each compute node's /tmp. Is there a way that for the galaxy to collect job results back to the data directory? Regards, Derrick ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] using x11 or freenx to connect to ASW cloud instance of CloudMan / Galaxy
On Jun 15, 2012, at 4:55 PM, Dellwo, Martin J. [JRDUS] wrote: Hi there, I am piloting my own instance of CloudMan in Amazon Web Services, using the current public AMI (according to your Wiki), AMI: ami-da58aab3 and Name: 861460482541/galaxy-cloudman-2011-03-22. I’d like to either tunnel X11 through SSH (either PuTTY on Windows, or just at a Linux prompt with “ssh –X”), or use an NX client to connect. I’m very familiar with how to do both, and have tested both locally. I have a local X server (latest XMing) on my Windows box, latest PuTTY, latestnomachine.com NX client. When I first start up CloudMan and login as user ‘ubuntu’, it asks if I would like to set up NX, and I’ve added myself as a user there. I can log in as that user via normal SSH connection. I can’t seem to either tunnel X11 over SSH or connect w/ NX. NX connects up through the initial stage and asks me for my password, but doesn’t start a desktop. Trying to tunnel X11 (xclock command) just sits forever (either via PuTTY or just normal “ssh –X” at a Linux command line). I see that Xvfb is running, and I can use ‘xwd’ to see an image of what is in the virtual frame buffer, ie the xclock window. It does seem like not all the required packages are installed, or possibly I just need to start a different service. Hi Martin, Xvfb is a fake display, the fact that your xclock is appearing on the virtual framebuffer rather than over your tunneled SSH connection suggests that $DISPLAY is being overwritten after you log in, or ssh is incorrectly choosing a display ID that is identical to the ID that Xvfb is using. I can't say much about NX, unfortunately, since I have never used it. --nate Should I expect that either tunneling or NX works ‘out of the box’, or are there more packages I need to install and configure? Any tutorials or pointers out on the web? Thanks. I did search mail archives. :) Martin J. Dellwo, Ph.D. Manager, Translational Informatics image001.jpg Janssen Research Development, LLC Information Technology Welsh McKean Roads, Spring House, PA 19477 mdel...@its.jnj.com ** Confidentiality Notice: This e-mail transmission may contain confidential or legally privileged information that is intended only for the individual or entity named in the e-mail address. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or reliance upon the contents of this e-mail is strictly prohibited. If you have received this e-mail transmission in error, please reply to the sender, so that Johnson Johnson can arrange for proper delivery, and then please delete the message from your inbox. Thank you. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Set job_working_directory to a cluster compute node /tmp
Hi Nate, It seems very promising. Will try it out! Thanks! Derrick On Thu, Jun 21, 2012 at 2:16 AM, Nate Coraor n...@bx.psu.edu wrote: Hi Derrick, If you use outputs_to_working_directory = True and put the working directory on the SSD pool, writing during job execution will use the fast space. Once the job completes, the outputs will be moved back to the larger, slower pool configured for file_path. --nate On Jun 19, 2012, at 8:08 AM, Derrick Lin wrote: Another relevant question, my institute has configured a NFS volume that based on a SSD disk pool on the file server. I want to use it for improving galaxy's job execution on big dataset. However, the SSD volume has only 2.5TB (they are very expensive...). So migrate the entire database folder to there is impossible. Any recommendation for the galaxy to have a good use of the SSD? Thanks Derrick On Tue, Jun 19, 2012 at 9:55 PM, Derrick Lin klin...@gmail.com wrote: I think my question has been answered: http://gmod.827538.n3.nabble.com/NFS-Cluster-and-working-directories-questions-td3784165.html Hopefully can see the enhancements in the near future. Cheers. Derrick On Tue, Jun 19, 2012 at 5:01 PM, Derrick Lin klin...@gmail.com wrote: Hi guys, I have deploy a galaxy on a cluster (so I installed it on a NFS share that it's accessible by all cluster compute nodes). Everything is running fine. Now I am looking for a way such that every job dispatched to a compute node uses that node's local /tmp as working directory. I know galaxy config provides job_working_directory for the similar purpose. My question really is, while all my compute nodes can access the NFS share where galaxy installed, but the galaxy host cannot access each compute node's /tmp. Is there a way that for the galaxy to collect job results back to the data directory? Regards, Derrick ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Configuring Data Library Usage for Non-Admins
Thank you all for the answers! CL ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Do input and output files get modified in Galaxy?
Hi all, This is going to be a very long message, so please bear with me. Anyway... I have been asked by my colleagues to integrate modified versions of velveth, velvetg and oases into our Galaxy instance. Of course, the first step was to upload the would-be input files. Since these are stored as .dat files in the database/files/000 folder, I symlinked the uploads to another folder. When I run the commands of the aforementioned steps, they access these symlinked files and perform the needed operations using them. I then just copy the data from the generated files onto the output files that would be visible in Galaxy. Is everything still clear? I hope so. Moving on... Now, they have asked me to integrate another script, and this time it is running rsem. It requires corrected left and right reads, and OASES' transcript output. But when I ran the rsem, I got an error concerning the input transcript file. I found two probable reasons why I was getting an error. One was that the transcript file might be empty. I've already checked this, and the file was not empty. Another reason was that the transcript FASTA file might not be following the required format. And so we arrive at my question: Do input and output files go through some minor format modifications when used in Galaxy? I have already manually ran the above commands, and everything works well. But when I downloaded the transcript file from Galaxy to use it via the konsole, I got the same error. Thank you in advance for any help, CL ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
