Re: [galaxy-dev] How to speed up job execution
On 08/24/2012 07:37 PM, kauerb...@comcast.net wrote: Hi Hans, I added my email address to the admin_users in universe_wsgi.ini , logged out of Galaxy, and logged back in but I don't see any 'admin' tab. Could you tell me what the problem might be? any change in the universe_wsgi.ini file requires a restart of the Galaxy server. Regards, Hans PS: Please keep all replies on the list by using reply all in your mail client. Thank you! -Ken. *From: *Hans-Rudolf Hotz h...@fmi.ch *To: *kauerb...@comcast.net *Cc: *Galaxy Development galaxy-...@bx.psu.edu *Sent: *Friday, August 24, 2012 1:07:20 PM *Subject: *Re: [galaxy-dev] How to speed up job execution On 08/24/2012 06:59 PM, kauerb...@comcast.net wrote: That's very helpful. Thank you. I'm new to Galaxy. Can you tell me how to become an 'admin' user and/or change the current admin user? again, change the settings in universe_wsgi.ini by adding your e-mail (the one you use to log into Galaxy): #admin_users = f...@bar.com Regards, Hans Thank you. -Ken. *From: *Hans-Rudolf Hotz h...@fmi.ch *To: *kauerb...@comcast.net *Cc: *galaxy-dev@lists.bx.psu.edu *Sent: *Friday, August 24, 2012 12:39:07 PM *Subject: *Re: [galaxy-dev] How to speed up job execution On 08/24/2012 05:22 PM, kauerb...@comcast.net wrote: Hello, On our local instance of Galaxy jobs are taking an extremely long time to run, or they are not run at all. In the history it always says job is waiting to run, even simple jobs like reformatting text files to be tab delimited. Is there a way to 1) check the Galaxy job queue, and 2) perhaps modify some parameters to speed up processing? Any suggestions would be greatly appreciated. Hi Kenneth Usually, Galaxy is not slow, but the tools executed by Galaxy are slow (eg due to bad IO performance of the host server). job is waiting to run indicates there are unfinished jobs in the queue. There are two ways to check the queue: If you are an 'Admin', select the Admin tab - Manage jobs. This will list all running jobs. And you can kill jobs as well. Or use the galaxy report tools and have a look at Today's jobs Depending on your set-up, you might consider changing universe_wsgi.ini: # Number of concurrent jobs to run (local job runner) local_job_queue_workers = 5 to allow more parallel jobs Hope this helps Hans Thank you. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Changing the location of toolsheddb fails
Hi all, I want to change the location of the toolshed database. First I copied all contents to the new location, and next, I adjusted community_wsgi.ini: file_path = /mnt/toolsheddb/community_files However, the toolshed fails to start now, throwing this error: RepoError: repository /home/galaxy/toolshed/database/community_files/000/repo_1 not found It still checks the old location apparently. Thanks for the assistance, Joachim -- Joachim Jacob, PhD Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Changing the location of toolsheddb fails
Hello Joachim, You may be the first that has tried this, so you're covering new territory. You'll have to manually change the entries in the hgweb.config file located in your Galaxy install directory. If your tool shed then starts up and you can point your browser to it, you should reset the metadata on all repositories using the Reset selected metadata option on the tool shed's Admin menu. Let us know if this does not work. Greg Von Kuster On Aug 27, 2012, at 6:22 AM, Joachim Jacob wrote: Hi all, I want to change the location of the toolshed database. First I copied all contents to the new location, and next, I adjusted community_wsgi.ini: file_path = /mnt/toolsheddb/community_files However, the toolshed fails to start now, throwing this error: RepoError: repository /home/galaxy/toolshed/database/community_files/000/repo_1 not found It still checks the old location apparently. Thanks for the assistance, Joachim -- Joachim Jacob, PhD Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] problems with labels in the tool box and order of the tools
Hello Hans-Rudolph, If I remember correctly, the release in March did not properly handle the scenario where new tool entries were made within an existing section of the tool panel or new tool panel sections were added to specific locations of the tool panel. In both these cases, the new entry (tool or tool section) were appended to the end of the section or panel rather than inserted into the desired location. This is probably the cause of the behavior you are seeing. This issue was corrected in the next Galaxy release after the March release. Greg Von Kuster On Aug 21, 2012, at 2:03 PM, Peter Cock wrote: On Tue, Aug 21, 2012 at 4:18 PM, Hans-Rudolf Hotz h...@fmi.ch wrote: HI We have been struggling with this problem since we updated to changeset:6799:40f1816d6857 (March 12, 2012 release). We never had the time to figure out whether it was a bug or whether we made a mistake. Unfortunately, it now has reached a point where we have a serious problem with our production server. I saw something funny like your screenshot of the funny with labels on our machine when I last modified our tool XML file. It seemed to be caching something from the old XML... Note so far we've not installed anything automatically from the toolshed (just manually). I've not dug into the issue - just mentioning this as a clue that this is probably not unique to your Galaxy installation. Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Changing the location of toolsheddb fails
Thanks Greg, this works! Joachim @bitsatvib On 08/27/2012 02:10 PM, Greg Von Kuster wrote: Hello Joachim, You may be the first that has tried this, so you're covering new territory. You'll have to manually change the entries in the hgweb.config file located in your Galaxy install directory. If your tool shed then starts up and you can point your browser to it, you should reset the metadata on all repositories using the Reset selected metadata option on the tool shed's Admin menu. Let us know if this does not work. Greg Von Kuster On Aug 27, 2012, at 6:22 AM, Joachim Jacob wrote: Hi all, I want to change the location of the toolshed database. First I copied all contents to the new location, and next, I adjusted community_wsgi.ini: file_path = /mnt/toolsheddb/community_files However, the toolshed fails to start now, throwing this error: RepoError: repository /home/galaxy/toolshed/database/community_files/000/repo_1 not found It still checks the old location apparently. Thanks for the assistance, Joachim -- Joachim Jacob, PhD Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Declaring dependencies on file format ToolShed entries
Hi Peter, This feature has been on my list for some time. There was discussion about this at the Galaxy Community Conference in July, and the priority for providing this was raised at that time. However, my next chunk of work will be in the area of enhancing the Galaxy framework to better handle locating certain tool-related items at tool execution time (e.g., Java jar files and other files on which tools depend). When this is finished, the next major chunk of work on my list is this feature you're requesting below. I'm hopeful that I can get started on this within the next couple of weeks. I have the implementation worked out in my head, and I don't think it will take too long to finish once I can get started. I'll keep you and the community informed as progress is made. Greg Von Kuster On Aug 23, 2012, at 9:12 AM, Peter Cock wrote: Dear all, Is there any way for a Tool Shed bundle (or individual tool) to declare a dependency on another Tool Shed bundle (and ideally a minimum version) which provides a datatype? For example, the 'emboss_5' suite depends on the 'emboss_datatypes' entry. Likewise the ncbi_blast_plus suite depends on 'blast_datatypes' to define the 'blastxml' format (and soon nucleotide and protein BLAST databases). As a specific case, I'd like to update my 'blast2go' wrapper to say it now also depends on blast_datatypes because it uses the 'blastxml' format. Similarly some of my sequence filtering/selection tools could be expanded to handle other formats like 'genbank', which is defined in the 'emboss_datatypes' repository. Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] apache external auth and proftpd
On Aug 24, 2012, at 8:11 AM, Laure wrote: Hi galaxy users, I configured galaxy with external ldap authentification using apache proxy, so my galaxy users are automatically created in galaxy database. When I want to configure proftpd to work with galaxy as it is mentionned in galaxy wiki, authentification fails on the ftp because registered password in galaxy_user table mismatchs ldap user password encrypted in SHA1. It seems to be a randomly created password. If I manually translate the ldap password to SHA1, and modify it in galaxy_user table, ftp login works correctly. I wonder how passwords are inserted in galaxy database when users are automatically created ? Is there a way to go through this ? Hi Laure, Since your Galaxy instance authenticates users via LDAP, your ProFTPD server should be configured to do the same. See ProFTPD's mod_ldap for details. --nate Thanks, Laure ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Tool development without a local Tool Shed?
Hi Peter, I assume you are using a Galaxy instance within your development environment when developing new Galaxy tools so that you ultimately ensure your tool is functionally correct. Adding a local tool shed to your development environment will help you ensure that certain tool shed related processes (e.g., setting tool shed metadata, installing the repository into a local Galaxy, etc) are functionally correct. With regard to proprietary datatypes (these are datatypes that are included in a tool shed repository), the process is simple if the datatypes are all subclassed from datatypes defined in the Galaxy framework. The emboss_datatypes are an example of this category, where the type attribute is something like galaxy.datatypes.data:Text. You probably don't need to use a local tool shed to help in the development of tools that use this kind of proprietary datatype. The more complex category is when the proprietary datatypes subclass from proprietary classes defined in a code file in the repository. In this case, the class file's import statement definitions must be fully qualified. This is the category where using a local tool shed as a component of the development process may be beneficial as you can ensure the proprietary datatypes class file is written in such a way that the installation process into a local Galaxy instance is functionally correct. There is currently no way to define proprietary datatypes_conf.xml files in a configuration entry. Sorry for the long-winded answer - hopefully the context is helpful. Greg Von Kuster On Aug 23, 2012, at 10:11 AM, Peter Cock wrote: Dear all, Until recently I'd not needed to define new datatypes while developing tools for Galaxy - instead using (and sometimes modifying) existing datatypes in situ. I have been manually 'installing' my in-development tools by added their XML files to tool_conf.xml - and this has worked fine to date. However, I would now like to experiment with defining or modifying datatypes which would eventually be defined via a datatypes_conf.xml file in a ToolShed entry. My impression from reading the latest wiki documentation on http://wiki.g2.bx.psu.edu/Tool%20Shed is that I will have to install my own local tool shed. Is there a simpler alternative? e.g. Can I add the new datatypes_conf.xml file(s) to a configuration entry instead? Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Tool development without a local Tool Shed?
On Mon, Aug 27, 2012 at 1:58 PM, Greg Von Kuster g...@bx.psu.edu wrote: Hi Peter, I assume you are using a Galaxy instance within your development environment when developing new Galaxy tools so that you ultimately ensure your tool is functionally correct. Adding a local tool shed to your development environment will help you ensure that certain tool shed related processes (e.g., setting tool shed metadata, installing the repository into a local Galaxy, etc) are functionally correct. With regard to proprietary datatypes (these are datatypes that are included in a tool shed repository), the process is simple if the datatypes are all subclassed from datatypes defined in the Galaxy framework. The emboss_datatypes are an example of this category, where the type attribute is something like galaxy.datatypes.data:Text. You probably don't need to use a local tool shed to help in the development of tools that use this kind of proprietary datatype. The more complex category is when the proprietary datatypes subclass from proprietary classes defined in a code file in the repository. In this case, the class file's import statement definitions must be fully qualified. This is the category where using a local tool shed as a component of the development process may be beneficial as you can ensure the proprietary datatypes class file is written in such a way that the installation process into a local Galaxy instance is functionally correct. There is currently no way to define proprietary datatypes_conf.xml files in a configuration entry. Sorry for the long-winded answer - hopefully the context is helpful. Greg Von Kuster That is helpful Greg - thank you - although I worry the learning curve for developing Galaxy tools is getting steeper. You've implied 'simple' proprietary datatypes which depend only on Galaxy itself (and not other ToolShed entries) can be developed without a local ToolShed. That's the situation I'm interested in. How would I enable the 'blastxml' datatype (now in the ToolShed as 'blast_datatypes')? My guess is adding something to file datatypes_conf.xml, which used to include these lines: datatype extension=blastxml type=galaxy.datatypes.xml:BlastXml mimetype=application/xml display_in_upload=true/ and: sniffer type=galaxy.datatypes.xml:BlastXml/ Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] How to restart the Galaxy process
Hello, I modified the Galaxy configuration file and would like to stop and re-start the Galaxy process. Can you tell me how to do that? Thank you. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Restarting Galaxy
Hello, Can someone tell me if stopping and then restarting the Galaxy process would prevent currently-running jobs from completing after the restart? Will all the jobs remain in the queue after the restart? Also, what is the best way to restart the Galaxy server? Users are complaining that jobs are extremely slow to start running. Is there a way to speed things up? Any info or suggestions would be greatly appreciated. Thank you. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] [galaxy-user] Installing Galaxy and Hooking into a SGE Cluster
On Fri, Aug 24, 2012 at 4:30 PM, Peter Cock p.j.a.c...@googlemail.com wrote: Would anyone know about my question 1? If I install a local version of Galaxy and connect it to our cluster, where is each user's (uploaded?) data stored? How will the cluster jobs be able to access the data? By default, all the data is under a single folder, belonging to the galaxy Linux user account. We use /mnt/galaxy/galaxy-dist for this. In order that the cluster jobs can access the data, they must also be able to see this mount. So can one user see another's files in your setup? I'd prefer if each user could only see and use his own files. No - they only access their data via the website, which has it's own user controls, and prevents user A seeing the files of user B (unless explicitly shared). How do users get data into Galaxy in your system? Mostly via the uplad tool, or sometimes the remote data tools (eg from UCSC). Some commonly used files are also setup on our system as shared libraries within Galaxy. It seems weird to ask them to upload something that's already in their home directory and available to the cluster jobs otherwise. I wonder if there's a way they can copy files to the Galaxy data directory? I guess they'd have to let Galaxy know about the new data somehow? Thanks, Greg P.S. It is very confusing that your emails give your name as mailing list rather than Greg. I think I fixed this? Yes :) ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Cannot install freebayes with migration script
Any ideas anyone? Philip Mabon Senior Bioinformatician National Microbiology Laboratory Public Health Agency of Canada On Tue, Aug 21, 2012 at 10:34 AM, Philip Mabon philipma...@gmail.comwrote: I just upgrade our galaxy to the latest release : 7487:be81990d148a and ran the migration tool script for freebayes. I received the following error: No handlers could be found for logger galaxy.tools /opt/galaxy/galaxy_dist/eggs/pysam-0.4.2_kanwei_b10f6e722e9a-py2.6-linux-x86_64-ucs4.egg/pysam/__init__.py:1: RuntimeWarning: __builtin__.file size changed, may indicate binary incompatibility from csamtools import * Repositories will be installed into configured tool_path location ../shed_tools Adding new row (or updating an existing row) for repository 'freebayes' in the tool_shed_repository table. Traceback (most recent call last): File ./scripts/migrate_tools/migrate_tools.py, line 21, in module app = MigrateToolsApplication( sys.argv[ 1 ] ) File /opt/galaxy/galaxy_dist/lib/galaxy/tool_shed/migrate/common.py, line 150, in __init__ install_dependencies=install_dependencies ) File /opt/galaxy/galaxy_dist/lib/galaxy/tool_shed/install_manager.py, line 37, in __init__ self.install_repository( repository_elem, install_dependencies ) File /opt/galaxy/galaxy_dist/lib/galaxy/tool_shed/install_manager.py, line 262, in install_repository install_dependencies=install_dependencies ) File /opt/galaxy/galaxy_dist/lib/galaxy/tool_shed/install_manager.py, line 150, in handle_repository_contents repository_tools_tups = get_repository_tools_tups( self.app, metadata_dict ) File /opt/galaxy/galaxy_dist/lib/galaxy/util/shed_util.py, line 1077, in get_repository_tools_tups tool = app.toolbox.load_tool( os.path.abspath( relative_path ), guid=guid ) File /opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py, line 452, in load_tool return ToolClass( config_file, root, self.app, guid=guid ) File /opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py, line 790, in __init__ self.parse( root, guid=guid ) File /opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py, line 961, in parse self.parse_inputs( root ) File /opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py, line 1045, in parse_inputs display, inputs = self.parse_input_page( page, enctypes ) File /opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py, line 1447, in parse_input_page inputs = self.parse_input_elem( input_elem, enctypes ) File /opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py, line 1514, in parse_input_elem case.inputs = self.parse_input_elem( case_elem, enctypes, context ) File /opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py, line 1470, in parse_input_elem group.inputs = self.parse_input_elem( elem, enctypes, context ) File /opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py, line 1540, in parse_input_elem param = self.parse_param_elem( elem, enctypes, context ) File /opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py, line 1552, in parse_param_elem param = ToolParameter.build( self, input_elem ) File /opt/galaxy/galaxy_dist/lib/galaxy/tools/parameters/basic.py, line 176, in build return parameter_types[param_type]( tool, param ) File /opt/galaxy/galaxy_dist/lib/galaxy/tools/parameters/basic.py, line 1361, in __init__ ToolParameter.__init__( self, tool, elem ) File /opt/galaxy/galaxy_dist/lib/galaxy/tools/parameters/basic.py, line 43, in __init__ self.validators.append( validation.Validator.from_element( self, elem ) ) File /opt/galaxy/galaxy_dist/lib/galaxy/tools/parameters/validation.py, line 23, in from_element return validator_types[type].from_element( param, elem ) File /opt/galaxy/galaxy_dist/lib/galaxy/tools/parameters/validation.py, line 279, in from_element tool_data_table = param.tool.app.tool_data_tables[ table_name ] File /opt/galaxy/galaxy_dist/lib/galaxy/tools/data/__init__.py, line 25, in __getitem__ return self.data_tables.__getitem__( key ) KeyError: 'sam_fa_indexes' What happen is that the freebayes itself installs correctly into the shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/* but the dependency on samtools 1.1.8 fails. The value 'sam_fa_indexes' is present in the tool_data_table_conf.xml and I know it works since all my samtools are functional. The status column for the freebayes entry in the database is 'Cloning'. I get the same error if I try to install freebayes thru the web front end. ( Had to delete the row in the db and remove the freebayes install directory) Any ideas Greg? Thanks! Philip Mabon (Takadonet) Senior Bioinformatician National Microbiology Laboratory Public Health Agency of Canada ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the
Re: [galaxy-dev] problems with labels in the tool box and order of the tools
Hi Greg Thank you very much for your e-mail Unfortunately, we still see this behavior in changeset:7404:ec29ce8e27a1 (July 20, 2012 release). All screen shots in my original e-mail (21-Aug) were made with a server running the July release. Regards, Hans-Rudolf On 08/27/2012 02:18 PM, Greg Von Kuster wrote: Hello Hans-Rudolph, If I remember correctly, the release in March did not properly handle the scenario where new tool entries were made within an existing section of the tool panel or new tool panel sections were added to specific locations of the tool panel. In both these cases, the new entry (tool or tool section) were appended to the end of the section or panel rather than inserted into the desired location. This is probably the cause of the behavior you are seeing. This issue was corrected in the next Galaxy release after the March release. Greg Von Kuster On Aug 21, 2012, at 2:03 PM, Peter Cock wrote: On Tue, Aug 21, 2012 at 4:18 PM, Hans-Rudolf Hotz h...@fmi.ch wrote: HI We have been struggling with this problem since we updated to changeset:6799:40f1816d6857 (March 12, 2012 release). We never had the time to figure out whether it was a bug or whether we made a mistake. Unfortunately, it now has reached a point where we have a serious problem with our production server. I saw something funny like your screenshot of the funny with labels on our machine when I last modified our tool XML file. It seemed to be caching something from the old XML... Note so far we've not installed anything automatically from the toolshed (just manually). I've not dug into the issue - just mentioning this as a clue that this is probably not unique to your Galaxy installation. Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] problems with labels in the tool box and order of the tools
I responded to your original email, which I should have done initially. Sorry for the back-and-forth on this. On Aug 27, 2012, at 10:16 AM, Hans-Rudolf Hotz wrote: Hi Greg Thank you very much for your e-mail Unfortunately, we still see this behavior in changeset:7404:ec29ce8e27a1 (July 20, 2012 release). All screen shots in my original e-mail (21-Aug) were made with a server running the July release. Regards, Hans-Rudolf On 08/27/2012 02:18 PM, Greg Von Kuster wrote: Hello Hans-Rudolph, If I remember correctly, the release in March did not properly handle the scenario where new tool entries were made within an existing section of the tool panel or new tool panel sections were added to specific locations of the tool panel. In both these cases, the new entry (tool or tool section) were appended to the end of the section or panel rather than inserted into the desired location. This is probably the cause of the behavior you are seeing. This issue was corrected in the next Galaxy release after the March release. Greg Von Kuster On Aug 21, 2012, at 2:03 PM, Peter Cock wrote: On Tue, Aug 21, 2012 at 4:18 PM, Hans-Rudolf Hotz h...@fmi.ch wrote: HI We have been struggling with this problem since we updated to changeset:6799:40f1816d6857 (March 12, 2012 release). We never had the time to figure out whether it was a bug or whether we made a mistake. Unfortunately, it now has reached a point where we have a serious problem with our production server. I saw something funny like your screenshot of the funny with labels on our machine when I last modified our tool XML file. It seemed to be caching something from the old XML... Note so far we've not installed anything automatically from the toolshed (just manually). I've not dug into the issue - just mentioning this as a clue that this is probably not unique to your Galaxy installation. Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Tool development without a local Tool Shed?
On Mon, Aug 27, 2012 at 3:31 PM, Greg Von Kuster g...@bx.psu.edu wrote: If you install the blast_datatypes repository into your Galaxy instance using the tool shed installation process everything necessary is handled for you automatically, but I understand you are not yet installing from the tool shed. Exactly. If you are not installing, you'll have to manually download the repository contents from the tool shed using the Repository Actions pop-up menu and place the contained xml.py file in the appropriate location so it can be loaded when you start your Galaxy server. You'll then need to manually add the datatype entries in your datatypes_conf.xml file as they used to be. Assuming by proper place you mean under the Galaxy folder lib/galaxy/datatypes/ then that makes sense - thanks. In this particular case there would be a name clash with xml.py, since there is still a file of that name in the main repository to define the XML base class etc. That isn't critical as I can rename the ToolShed file to blast.py (or similar), which would make sense anyway as I want to include additional formats for BLAST databases. Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] [galaxy-user] Installing Galaxy and Hooking into a SGE Cluster
On Mon, Aug 27, 2012 at 2:25 PM, greg margeem...@gmail.com wrote: How do users get data into Galaxy in your system? Mostly via the upload tool, or sometimes the remote data tools (eg from UCSC). Some commonly used files are also setup on our system as shared libraries within Galaxy. It seems weird to ask them to upload something that's already in their home directory and available to the cluster jobs otherwise. I was answering about *our* Galaxy, where only a few of the users have a Linux account. In most cases their home directory is on Windows... and not easily accessed from our Linux cluster if at all. I wonder if there's a way they can copy files to the Galaxy data directory? I guess they'd have to let Galaxy know about the new data somehow? This is available to a Galaxy administrator for shared libraries of files made available within Galaxy, see: http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Uploading%20Library%20Files Perhaps something like Alban Lermine's upload_local_file or Edward Kirton's data_nfs tool on the ToolShed would work for you? Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Galaxy-France -- Communauté française de Galaxy (French Galaxy Community)
Bonjour à tous, Avec l'aide et le soutien de l'équipe Galaxy, nous venons de mettre en ligne la liste de diffusion Galaxy France. Cette liste a pour but de partager et communiquer sur nos différentes expériences avec Galaxy au sein de l'hexagone. La langue officielle de cette liste est le français. Vous avez besoin d'aide dans l'utilisation de Galaxy, Vous hébergez un serveur public Galaxy, Vous maintenez une instance locale de Galaxy, Vous venez de déposer un outil sur le toolshed, Vous avez des trucs et astuces pouvant interresser la communauté, Vous organisez des formations avec Galaxy, Rejoignez nous en vous inscrivant à http://lists.bx.psu.edu/listinfo/galaxy-france Les archives de cette liste sont disponibles à http://france.list.galaxyproject.org A très bientôt sur la liste Galaxy France, Les administrateurs: Alban Lermine, Olivier Inizan, Rémi Marenco, Dave Clements Jennifer Jackson -- - Hi everyone, With the help and the support of the Galaxy team, we just put online the Galaxy French mailing list. The aim of this list is to share and communicate on our different experiences with Galaxy in France. * The main language is French. * Topics include, but are not limited to: You need help in the use of Galaxy, You're hosting a Galaxy public server, You're maintaining a local instance of Galaxy, You just add a new tool on the toolshed, You have some tips that could interested the community, You organize workshops with Galaxy, Join us by subscribing at http://lists.bx.psu.edu/listinfo/galaxy-france The list's archives are also available at http://france.list.galaxyproject.org See you soon on the Galaxy-France list, The administrators: Alban Lermine, Olivier Inizan, Rémi Marenco, Dave Clements Jennifer Jackson -- http://galaxyproject.org/wiki/GCC2012http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://galaxyproject.org/wiki/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Galaxy wiki is throwing errors during edit commit
I found that the problem happened if I used the UI for editing text. What I did instead was edit the wiki using the text-based mode instead. If you have set your preferences to use the UI for editing by default, then you will need to change it back to use the text-based editing instead; I found that, once I used the UI for editing text on the wiki, I couldn't save a set of changes. Let me know if that doesn't work. -Scott - Original Message - Hi Devs, The galaxy wiki is throwing error whenever I try to commit an edit change to it. I have attached a screenshot for your reference. Cheers Tomithy ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Re-starting Galaxy
Hello, If I were to stop and re-start the Galaxy server with the run.sh script would jobs that are aleady running or are in the queue be lost when the server is re-started? If so, is there another method to use which would not lose the running or queued jobs? Thank you. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Re-starting Galaxy
Unless something has recently changed, you will need to set up a cluster to get jobs to persist across galaxy restarts. You can do this with just a single node acting as both queue master and job runner using sun grid engine (i think, I have not tried this myself) Brad On Aug 27, 2012, at 12:51 PM, kauerb...@comcast.netmailto:kauerb...@comcast.net wrote: Hello, If I were to stop and re-start the Galaxy server with the run.sh script would jobs that are aleady running or are in the queue be lost when the server is re-started? If so, is there another method to use which would not lose the running or queued jobs? Thank you. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Brad Langhorst langho...@neb.commailto:langho...@neb.com 978-380-7564 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Cannot install freebayes with migration script
I already had that entry to the tool_data_table_conf.xml but still getting the same error. After some debugging and print statement, I found that it initially finds the correct path to the 'sam_fa_indexes' when reading tool_data_table_conf.xml in Galaxy installation directory but re-reads another file in the shed_tools dir. It's the sample tool_data_table_conf.xml from the freebayes directory. ../shed_tools/ toolshed.g2.bx.psu.edu/repos/devteam/freebayes/213a3d6b579a/freebayes/tool_data_table_conf.xml.sample . Maybe the tool_data_table_conf.xml.sample is invalid? Philip Mabon Senior Bioinformatician National Microbiology Laboratory Public Health Agency of Canada On Mon, Aug 27, 2012 at 9:41 AM, Greg Von Kuster g...@bx.psu.edu wrote: Hi Phillip, I'm not able to reproduce this behavior, so it's difficult to determine what may be the cause. I have not seen others in the community encounter this problem, but that may simply be due to the fact that no one is yet installing the freebayes tool from the tool shed. A possible work-aound is to add the following entry into your local tool_data_table_conf.xml file located in your Galaxy installation directory. table name=sam_fa_indexes comment_char=# columnsline_type, value, path/columns file path=tool-data/sam_fa_indices.loc / /table Let us know if this still does not correct the problem. Greg Von Kuster On Aug 27, 2012, at 10:07 AM, Philip Mabon wrote: Any ideas anyone? Philip Mabon Senior Bioinformatician National Microbiology Laboratory Public Health Agency of Canada On Tue, Aug 21, 2012 at 10:34 AM, Philip Mabon philipma...@gmail.comwrote: I just upgrade our galaxy to the latest release : 7487:be81990d148a and ran the migration tool script for freebayes. I received the following error: No handlers could be found for logger galaxy.tools /opt/galaxy/galaxy_dist/eggs/pysam-0.4.2_kanwei_b10f6e722e9a-py2.6-linux-x86_64-ucs4.egg/pysam/__init__.py:1: RuntimeWarning: __builtin__.file size changed, may indicate binary incompatibility from csamtools import * Repositories will be installed into configured tool_path location ../shed_tools Adding new row (or updating an existing row) for repository 'freebayes' in the tool_shed_repository table. Traceback (most recent call last): File ./scripts/migrate_tools/migrate_tools.py, line 21, in module app = MigrateToolsApplication( sys.argv[ 1 ] ) File /opt/galaxy/galaxy_dist/lib/galaxy/tool_shed/migrate/common.py, line 150, in __init__ install_dependencies=install_dependencies ) File /opt/galaxy/galaxy_dist/lib/galaxy/tool_shed/install_manager.py, line 37, in __init__ self.install_repository( repository_elem, install_dependencies ) File /opt/galaxy/galaxy_dist/lib/galaxy/tool_shed/install_manager.py, line 262, in install_repository install_dependencies=install_dependencies ) File /opt/galaxy/galaxy_dist/lib/galaxy/tool_shed/install_manager.py, line 150, in handle_repository_contents repository_tools_tups = get_repository_tools_tups( self.app, metadata_dict ) File /opt/galaxy/galaxy_dist/lib/galaxy/util/shed_util.py, line 1077, in get_repository_tools_tups tool = app.toolbox.load_tool( os.path.abspath( relative_path ), guid=guid ) File /opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py, line 452, in load_tool return ToolClass( config_file, root, self.app, guid=guid ) File /opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py, line 790, in __init__ self.parse( root, guid=guid ) File /opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py, line 961, in parse self.parse_inputs( root ) File /opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py, line 1045, in parse_inputs display, inputs = self.parse_input_page( page, enctypes ) File /opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py, line 1447, in parse_input_page inputs = self.parse_input_elem( input_elem, enctypes ) File /opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py, line 1514, in parse_input_elem case.inputs = self.parse_input_elem( case_elem, enctypes, context ) File /opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py, line 1470, in parse_input_elem group.inputs = self.parse_input_elem( elem, enctypes, context ) File /opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py, line 1540, in parse_input_elem param = self.parse_param_elem( elem, enctypes, context ) File /opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py, line 1552, in parse_param_elem param = ToolParameter.build( self, input_elem ) File /opt/galaxy/galaxy_dist/lib/galaxy/tools/parameters/basic.py, line 176, in build return parameter_types[param_type]( tool, param ) File /opt/galaxy/galaxy_dist/lib/galaxy/tools/parameters/basic.py, line 1361, in __init__ ToolParameter.__init__( self, tool, elem ) File /opt/galaxy/galaxy_dist/lib/galaxy/tools/parameters/basic.py, line 43, in
Re: [galaxy-dev] Re-starting Galaxy
If you refer to using one machine as master AND host simultaneously, which you never tried: let me tell you it does work well with SGE/OGE. I have it running, and it just works out great. It enhances the load control on that machine, due to the great configuration possibilities on queue level (max. cores, queue ranking, user restrictions etc.). I did not connect it to Galaxy yet (which is also installed on the same machine), but I do not see a reason why that should not work out. Indeed, the reason for acting like this was the need to set up a modularized system, using defined interfaces. Currently everything is physically in on one machine, but that will change. Galaxy for example does not care, where and how a cluster is set up physically, it just connects to a given ressource. Regarding the initial question: at least jobs in the cluster queue will indeed finish, because after submission those jobs are decoupled from Galaxy, by definition; it's like submitting from a desktop machine, which can be switched off afterwards. Additionally, I would wonder if Galaxy (which has really great fail safe mechanisms) would not be able to reconnect to the cluster, asking for jobs, which were submitted before restart and not noted as finished or picked up after restart (noted internally within the DB). I would say test it and get back to the mailing list :). Best, Sebastian Langhorst, Brad schrieb: Unless something has recently changed, you will need to set up a cluster to get jobs to persist across galaxy restarts. You can do this with just a single node acting as both queue master and job runner using sun grid engine (i think, I have not tried this myself) Brad On Aug 27, 2012, at 12:51 PM, kauerb...@comcast.netmailto:kauerb...@comcast.net wrote: Hello, If I were to stop and re-start the Galaxy server with the run.sh script would jobs that are aleady running or are in the queue be lost when the server is re-started? If so, is there another method to use which would not lose the running or queued jobs? Thank you. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Brad Langhorst langho...@neb.commailto:langho...@neb.com 978-380-7564 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Sebastian Schaaf, M.Sc. Bioinformatics Chair of Biometry and Bioinformatics Department of Medical Information Sciences, Biometry and Epidemiology University of Munich Marchioninistr. 15, K U1 (postal) Marchioninistr. 17, U 006 (office) D-81377 Munich (Germany) Tel: +49 89 2180-78178 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
