Re: [galaxy-dev] Cluster jobs running as real user
Follow the Galaxy wiki https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#Submitting_Jobs_as_the_Real_User setup permission instructions. Follow these 2 steps to get Galaxy to run as real users with IBM Platform LSF: Step 1) Download the latest LSF DRMAA 1.1.1 from github. https://github.com/PlatformLSF/lsf-drmaa Level commit e4d5c1ab10fcba572eaede68fc2fc05332198bec or above This fixes the LSF job id not found issue. Step 2) LSF needs to disable message being printed out via the LSF environment variable "BSUB_QUIET". This can be done either in in Galaxy drmaa_external_runner.py file or in the galaxy admin environment profile Edit the drmaa_external_runner.py file to add in the LSF environment variable "BSUB_QUIET" to disable message print out. Line in blue below. scripts/drmaa_external_runner.py: def main(): userid, json_filename, assign_all_groups = validate_paramters() set_user(userid, assign_all_groups) json_file_exists(json_filename) os.environ['BSUB_QUIET'] = 'Y' s = drmaa.Session() s.initialize() jt = s.createJobTemplate() load_job_template_from_file(jt, json_filename) OR Disable LSF message in the galaxy admin profile e.g. export BSUB_QUIET=Y in .bash_profile Regards, Linda___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] toolshed repository down?
Hi. Getting error when trying to install the latest Galaxy on a new stand alone linux RH server. Is the repository down? migrate.versioning.repository DEBUG 2013-07-16 14:14:46,235 Config: {'db_settings': {'__name__': 'db_settings', 'required_dbs': '[]', 'version_table': 'migrate_tools', 'repository_id': ' GalaxyTools'}} tool_shed.galaxy_install.migrate.check DEBUG 2013-07-16 14:14:46,240 psycopg2 egg successfully loaded for postgres dialect The URL http://toolshed.g2.bx.psu.edu/repository/get_tool_dependencies?name=bowtie_wrappers&owner=devteam&changeset_revision=0c7e4eadfb3c&from_install_manager=True raised the exception: The URL http://toolshed.g2.bx.psu.edu/repository/get_tool_dependencies?name=bowtie_color_wrappers&owner=devteam&changeset_revision=fd0914e451c5&from_install_manager=True raised the exception: The URL http://toolshed.g2.bx.psu.edu/repository/get_tool_dependencies?name=lastz&owner=devteam&changeset_revision=0801f8207d30&from_install_manager=True raised the exception: The URL http://toolshed.g2.bx.psu.edu/repository/get_tool_dependencies?name=lastz_paired_reads&owner=devteam&changeset_revision=96825cee5c25&from_install_manager=True raised the exception: Traceback (most recent call last): ... return tool_shed_accessible, missing_tool_configs_dict UnboundLocalError: local variable 'missing_tool_configs_dict' referenced before assignment Regards, Linda___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Trackster set display mode - Output looks all the same
Thank you Jeremy. The new wiki page looks great! Regards, Linda From: Jeremy Goecks To: Linda Cham/Poughkeepsie/IBM@IBMUS Cc: Date: 06/11/2013 12:56 PM Subject:Re: [galaxy-dev] Trackster set display mode - Output looks all the same Hi Linda, 1- Should the "Set display mode" give out different output? Not yet. VCF visualization is new to Galaxy, and mode switching isn't implemented yet. (It will be implemented in the next release.) What you're seeing is the display of individual variants; of course, trying to view so many variants leads to the mismash that you noted. If you zoom in sufficiently, you should see variants without trouble. See the attached screenshot for example. 2- What might be the issue if all I see are the same output or maybe the output displayed is totally incorrect? Might there be some trackster setup that I might be missing for it to draw incorrectly? Everything looks OK so far. FYI, I've created a new page that lists the steps for setting up visualization for a Galaxy instance and updated wiki links/text to point to this page: http://wiki.galaxyproject.org/Visualization%20Setup Best, J.<><>___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Installation of 20130603 Galaxy image with MySQL fails
Thanks Dannon. I am going with postgres for now while mysql is being fixed. Regards, Linda From: Dannon Baker To: Linda Cham/Poughkeepsie/IBM@IBMUS Cc: Björn Grüning , Galaxy Dev Date: 06/06/2013 04:41 PM Subject:Re: [galaxy-dev] Installation of 20130603 Galaxy image with MySQL fails Hi Linda, Sorry I wasn't clear before. The fix for that first issue is in galaxy-central's stable branch, which will be migrated to galaxy-dist probably sometime next week. You can always pull directly from -central though, using 'hg pull -u -b stable http://bitbucket.org/galaxy/galaxy-central/' That said, this doesn't resolve the mysql index issues, which I'm currently working on. I hope to have a fix for them available soon, though. -Dannon On Wed, Jun 5, 2013 at 4:47 PM, Linda Cham wrote: Hi Bjorn, Currently seen $ hg tip changeset: 9850:cea3ddf6cdda branch: stable tag: tip user: Nate Coraor date: Mon Jun 03 16:17:31 2013 -0400 summary: Update tag for stable_2013.06.03 Removed galaxy-dist directory and rerun the hg clone command. $ hg clone https://bitbucket.org/galaxy/galaxy-dist#stable destination directory: galaxy-dist requesting all changes adding changesets adding manifests adding file changes added 9851 changesets with 36400 changes to 7175 files updating to branch stable 4132 files updated, 0 files merged, 0 files removed, 0 files unresolved $ cd galaxy-dist $ hg heads changeset: 9850:cea3ddf6cdda branch: stable tag: tip user: Nate Coraor date: Mon Jun 03 16:17:31 2013 -0400 summary: Update tag for stable_2013.06.03 $ hg pull ; hg update stable pulling from https://bitbucket.org/galaxy/galaxy-dist searching for changes no changes found 0 files updated, 0 files merged, 0 files removed, 0 files unresolved Check the script/manage_db.py: from galaxy import eggs <- No decorator line eggs.require( "Tempita" ) eggs.require( "SQLAlchemy" ) eggs.require( "sqlalchemy_migrate" ) $ hg update tip 0 files updated, 0 files merged, 0 files removed, 0 files unresolved $ hg heads changeset: 9850:cea3ddf6cdda branch: stable tag: tip user: Nate Coraor date: Mon Jun 03 16:17:31 2013 -0400 summary: Update tag for stable_2013.06.03 $ hg update default 82 files updated, 0 files merged, 2 files removed, 0 files unresolved $ hg heads changeset: 9850:cea3ddf6cdda branch: stable tag: tip user: Nate Coraor date: Mon Jun 03 16:17:31 2013 -0400 summary: Update tag for stable_2013.06.03 script/manage_db.py still the same as above. Inactive hide details for Björn Grüning ---06/05/2013 04:30:09 PM---Hi Linda, > Hi Dannon,Björn Grüning ---06/05/2013 04:30:09 PM---Hi Linda, > Hi Dannon, From: Björn Grüning To: Linda Cham/Poughkeepsie/IBM@IBMUS Cc: Dannon Baker , Galaxy Dev < galaxy-dev@lists.bx.psu.edu> Date: 06/05/2013 04:30 PM Subject: Re: [galaxy-dev] Installation of 20130603 Galaxy image with MySQL fails Hi Linda, > Hi Dannon, > > Thanks. I tried to do the hg pull and update and hg tip but I do not > see the changes in my working directory. So I added the one liner > eggs.require( "decorator" ) to scripts/manage_db.py. > Ran the sh manage_db.sh upgrade and get this error: that is really strange. Can you show us what the output of hg tip is? Do you mind to start with a fresh checkout and that particular change? Thanks, Björn <>___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Galaxy job will stay in waiting under DRMAA runner until manually trigger a command line submission
Hi. Running standalone Galaxy stable_2013.06.03 version on RH6.2 with postgresl db with GPFS filesystem. Setup via the directions in http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster to run all jobs through the LSF DRMAA runner. job_conf.xml: Scheduler and Galaxy were started. Submit a FASTQ groomer job on a FASTQ file under Galaxy. The groomer job in History will stay in GREY "Job is waiting to run" state until one physically go onto the node and submits a command line job through the scheduler. Once that command line job (e.g. sleep 5) is submitted then the groomer job will show up in the scheduler's job queue and galaxy's HISTORY will now show it as running. I can submit another groomer job after this and it will show up as running. At times, one will see this type of scenario again where one has to manually start a job under the scheduler on the command line to trigger the galaxy's jobs to start running under the scheduler. Is there some setup I am missing in the job_conf.xml above or a flag not set in universe_wsgi.ini? Searching the Galaxy web, I had read that one can set the track_jobs_in_database=True to trigger it to look in memory or database. But that didn't work either. Wonder if anyone seen this type of issue with any runners and have any suggestions on how to debug and fix this. Thanks. Linda___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] GATK
Hi Sridhar, About the error, you need to request a key from GATK http://gatkforums.broadinstitute.org/discussion/1250/what-is-phone-home-and-how-does-it-affect-me#latest See under "How to obtain and use a GATK key" Or by pass it by removing the -et "NO_ET" line from the xml files. From: sridhar srinivasan To: Galaxy Dev Date: 06/06/2013 06:30 AM Subject:[galaxy-dev] GATK Sent by:galaxy-dev-boun...@lists.bx.psu.edu Dear Developers, I am running GATK for first time. In galaxy i use Unified Genotyper. I am getting error as below mentioned. # ERROR -- # ERROR A USER ERROR has occurred (version 2.5-2-gf57256b): # ERROR The invalid arguments or inputs must be corrected before the GATK can proceed # ERROR Please do not post this error to the GATK forum # ERROR # ERROR See the documentation (rerun with -h) for this tool to view allowable command-line arguments. # ERROR Visit our website and forum for extensive documentation and answers to # ERROR commonly asked questions http://www.broadinstitute.org/gatk # ERROR # ERROR MESSAGE: Running with the -et NO_ET or -et STDOUT option requires a GATK Key file. Please see http://gatkforums.broadinstitute.org/discussion/1250/what-is-phone-home-and-how-does-it-affect-me#latest for more information and instructions on how to obtain a key. # ERROR -- also suggest me about the Rod file for dbSNP. Thanks Sridhar___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/<>___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Installation of 20130603 Galaxy image with MySQL fails
Hi Bjorn, Currently seen $ hg tip changeset: 9850:cea3ddf6cdda branch: stable tag: tip user:Nate Coraor date:Mon Jun 03 16:17:31 2013 -0400 summary: Update tag for stable_2013.06.03 Removed galaxy-dist directory and rerun the hg clone command. $ hg clone https://bitbucket.org/galaxy/galaxy-dist#stable destination directory: galaxy-dist requesting all changes adding changesets adding manifests adding file changes added 9851 changesets with 36400 changes to 7175 files updating to branch stable 4132 files updated, 0 files merged, 0 files removed, 0 files unresolved $ cd galaxy-dist $ hg heads changeset: 9850:cea3ddf6cdda branch: stable tag: tip user:Nate Coraor date:Mon Jun 03 16:17:31 2013 -0400 summary: Update tag for stable_2013.06.03 $ hg pull ; hg update stable pulling from https://bitbucket.org/galaxy/galaxy-dist searching for changes no changes found 0 files updated, 0 files merged, 0 files removed, 0 files unresolved Check the script/manage_db.py: from galaxy import eggs <- No decorator line eggs.require( "Tempita" ) eggs.require( "SQLAlchemy" ) eggs.require( "sqlalchemy_migrate" ) $ hg update tip 0 files updated, 0 files merged, 0 files removed, 0 files unresolved $ hg heads changeset: 9850:cea3ddf6cdda branch: stable tag: tip user:Nate Coraor date:Mon Jun 03 16:17:31 2013 -0400 summary: Update tag for stable_2013.06.03 $ hg update default 82 files updated, 0 files merged, 2 files removed, 0 files unresolved $ hg heads changeset: 9850:cea3ddf6cdda branch: stable tag: tip user:Nate Coraor date:Mon Jun 03 16:17:31 2013 -0400 summary: Update tag for stable_2013.06.03 script/manage_db.py still the same as above. From: Björn Grüning To: Linda Cham/Poughkeepsie/IBM@IBMUS Cc: Dannon Baker , Galaxy Dev Date: 06/05/2013 04:30 PM Subject:Re: [galaxy-dev] Installation of 20130603 Galaxy image with MySQL fails Hi Linda, > Hi Dannon, > > Thanks. I tried to do the hg pull and update and hg tip but I do not > see the changes in my working directory. So I added the one liner > eggs.require( "decorator" ) to scripts/manage_db.py. > Ran the sh manage_db.sh upgrade and get this error: that is really strange. Can you show us what the output of hg tip is? Do you mind to start with a fresh checkout and that particular change? Thanks, Björn <>___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Installation of 20130603 Galaxy image with MySQL fails
Hi Dannon, Thanks. I tried to do the hg pull and update and hg tip but I do not see the changes in my working directory. So I added the one liner eggs.require ( "decorator" ) to scripts/manage_db.py. Ran the sh manage_db.sh upgrade and get this error: File "build/bdist.linux-x86_64-ucs4/egg/MySQLdb/cursors.py", line 173, in execute File "build/bdist.linux-x86_64-ucs4/egg/MySQLdb/connections.py", line 36, in defaulterrorhandler sqlalchemy.exc.OperationalError: (OperationalError) (1060, "Duplicate column name 'slug'") '\nALTER TABLE history ADD slug TEXT' () So I thought maybe I should remove the column that was added before. ALTER TABLE history DROP COLUMN slug; Ran the sh manage_db.sh upgrade again, but now it gives back the original error: File "build/bdist.linux-x86_64-ucs4/egg/MySQLdb/connections.py", line 36, in defaulterrorhandler sqlalchemy.exc.OperationalError: (OperationalError) (1170, "BLOB/TEXT column 'slug' used in key specification without a key length") 'CREATE INDEX ix_history_slug ON history (slug)' () Interesting to note that if I ran the sh manage_db.sh again after the CREATE INDEX error above, I get back this error now: File "build/bdist.linux-x86_64-ucs4/egg/MySQLdb/connections.py", line 36, in defaulterrorhandler sqlalchemy.exc.OperationalError: (OperationalError) (1060, "Duplicate column name 'slug'") '\nALTER TABLE history ADD slug TEXT' () Hi Linda, The decorator issue should be resolved in galaxy-central now. I'm setting up a test environment to see if I can reproduce the mysql issue now, and will let you know what I come up with. Thanks! -Dannon On Wed, Jun 5, 2013 at 11:12 AM, Linda Cham wrote: Hi. I am new to the Galaxy forum and I am trying to setup a new local Galaxy instance with MySQL on RH6.2 x86_64. On a newly installed OS node, using the hg clone command (hg clone https://bitbucket.org/galaxy/galaxy-dist#stable) which gets the current Galaxy 20130603 image, I am getting the following errors when running run.sh: File "build/bdist.linux-x86_64-ucs4/egg/MySQLdb/cursors.py", line 173, in execute self.errorhandler(self, exc, value) File "build/bdist.linux-x86_64-ucs4/egg/MySQLdb/connections.py", line 36, in defaulterrorhandler raise errorclass, errorvalue OperationalError: (OperationalError) (1170, "BLOB/TEXT column 'slug' used in key specification without a key length") 'CREATE INDEX ix_history_slug ON history (slug)' () Reading the forum, someone stated it can be bypass by running the CREATE INDEX. mysql> CREATE INDEX ix_history_slug ON history (slug(1000)); Query OK, 0 rows affected (0.09 sec) Records: 0 Duplicates: 0 Warnings: 0 When trying to run.sh again after the CREATE INDEX, will get this error: File "/gpfs/gpfs2/home/galaxyadmin25/galaxy-dist/lib/galaxy/app.py", line 52, in __init__ create_or_verify_database( db_url, kwargs.get( 'global_conf', {} ).get( '__file__', None ), self.config.database_engine_options, app=self ) File "/gpfs/gpfs2/home/galaxyadmin25/galaxy-dist/lib/galaxy/model/migrate/check.py", line 107, in create_or_verify_database % ( db_schema.version, migrate_repository.versions.latest, config_arg ) ) Exception: Your database has version '29' but this code expects version '115'. Please backup your database and then migrate the schema by running 'sh manage_db.sh upgrade'. So try to run the "sh manage_db.sh upgrade' now will get this error: $ sh manage_db.sh upgrade Traceback (most recent call last): File "./scripts/manage_db.py", line 13, in from migrate.versioning.shell import main File "/gpfs/gpfs2/home/galaxyadmin25/galaxy-dist/eggs/sqlalchemy_migrate-0.7.2-py2.6.egg/migrate/versioning/shell.py", line 12, in from migrate.versioning import api File "/gpfs/gpfs2/home/galaxyadmin25/galaxy-dist/eggs/sqlalchemy_migrate-0.7.2-py2.6.egg/migrate/versioning/api.py", line 33, in from migrate.versioning import (repository, schema, version, File "/gpfs/gpfs2/home/galaxyadmin25/galaxy-dist/eggs/sqlalchemy_migrate-0.7.2-py2.6.egg/migrate/versioning/repository.py", line 13, in from migrate.versioning import version, pathed, cfgparse File "/gpfs/gpfs2/home/galaxyadmin25/galaxy-dist/eggs/sqlalchemy_migrate-0.7.2-py2.6.egg/migrate/versioning/version.py", line 10, in from migrate.versioning import pathed, script File "/gpfs/gpfs2/home/galaxyadmin25/galaxy-dist/eggs/sqlalchemy_migrate-0.7.2-py2.6.egg/migrate/versioning/pathed.py", line 11, in from migrate.versioning.util import KeyedInstance File "/gpfs/g
[galaxy-dev] Installation of 20130603 Galaxy image with MySQL fails
Hi. I am new to the Galaxy forum and I am trying to setup a new local Galaxy instance with MySQL on RH6.2 x86_64. On a newly installed OS node, using the hg clone command (hg clone https://bitbucket.org/galaxy/galaxy-dist#stable) which gets the current Galaxy 20130603 image, I am getting the following errors when running run.sh: File "build/bdist.linux-x86_64-ucs4/egg/MySQLdb/cursors.py", line 173, in execute self.errorhandler(self, exc, value) File "build/bdist.linux-x86_64-ucs4/egg/MySQLdb/connections.py", line 36, in defaulterrorhandler raise errorclass, errorvalue OperationalError: (OperationalError) (1170, "BLOB/TEXT column 'slug' used in key specification without a key length") 'CREATE INDEX ix_history_slug ON history (slug)' () Reading the forum, someone stated it can be bypass by running the CREATE INDEX. mysql> CREATE INDEX ix_history_slug ON history (slug(1000)); Query OK, 0 rows affected (0.09 sec) Records: 0 Duplicates: 0 Warnings: 0 When trying to run.sh again after the CREATE INDEX, will get this error: File "/gpfs/gpfs2/home/galaxyadmin25/galaxy-dist/lib/galaxy/app.py", line 52, in __init__ create_or_verify_database( db_url, kwargs.get( 'global_conf', {} ).get ( '__file__', None ), self.config.database_engine_options, app=self ) File "/gpfs/gpfs2/home/galaxyadmin25/galaxy-dist/lib/galaxy/model/migrate/check.py", line 107, in create_or_verify_database % ( db_schema.version, migrate_repository.versions.latest, config_arg ) ) Exception: Your database has version '29' but this code expects version '115'. Please backup your database and then migrate the schema by running 'sh manage_db.sh upgrade'. So try to run the "sh manage_db.sh upgrade' now will get this error: $ sh manage_db.sh upgrade Traceback (most recent call last): File "./scripts/manage_db.py", line 13, in from migrate.versioning.shell import main File "/gpfs/gpfs2/home/galaxyadmin25/galaxy-dist/eggs/sqlalchemy_migrate-0.7.2-py2.6.egg/migrate/versioning/shell.py", line 12, in from migrate.versioning import api File "/gpfs/gpfs2/home/galaxyadmin25/galaxy-dist/eggs/sqlalchemy_migrate-0.7.2-py2.6.egg/migrate/versioning/api.py", line 33, in from migrate.versioning import (repository, schema, version, File "/gpfs/gpfs2/home/galaxyadmin25/galaxy-dist/eggs/sqlalchemy_migrate-0.7.2-py2.6.egg/migrate/versioning/repository.py", line 13, in from migrate.versioning import version, pathed, cfgparse File "/gpfs/gpfs2/home/galaxyadmin25/galaxy-dist/eggs/sqlalchemy_migrate-0.7.2-py2.6.egg/migrate/versioning/version.py", line 10, in from migrate.versioning import pathed, script File "/gpfs/gpfs2/home/galaxyadmin25/galaxy-dist/eggs/sqlalchemy_migrate-0.7.2-py2.6.egg/migrate/versioning/pathed.py", line 11, in from migrate.versioning.util import KeyedInstance File "/gpfs/gpfs2/home/galaxyadmin25/galaxy-dist/eggs/sqlalchemy_migrate-0.7.2-py2.6.egg/migrate/versioning/util/__init__.py", line 7, in from decorator import decorator ImportError: No module named decorator In some searches, there were tips of deleting the eggs directory and rerun run.sh or use a python env. Neither option worked. Was not able to find anything meaningful about the decorator error. Note:, I was able to get Galaxy up with MySQL using the Galaxy 788cd3d06541 image on a different node. So I am thinking maybe it is the new 20130603 image that is giving me issues? Commands used to get Galaxy up: Install mysql from yum repository [MySQL 5.1.52-1.el6_0.1] python version [2.6.6] Grant ALL PRIVILEGES galaxy_db.* TO galaxy_user on localhostname with PWD Images tested: *Copy of the galaxy-galaxy-dist-788cd3d06541.tar.bz2 image Did not run the hg update command. [This worked] *hg clone stable of newest image [Failed] Update the universe_wsg.ini with the port, host, database_connection to mysql Ran run.sh Thanks. Linda___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/