Re: [galaxy-dev] Launching External functional tests fails with: local variable 'galaxy_test_proxy_port' referenced before assignment
Hi John, Thanks for the reply and the trello card. Since there's no mention of this in the documentation, do you know if there are any other way(s) (official or not) of running functional tests on production servers? Thanks again, Renato Quoting John Chilton on 05-02-2014 21:41: I asked around a while ago about this (GALAXY_TEST_EXTERNAL) and I believe the consensus was it does not work and has not worked for sometime - this is unfortunate because it was very useful. I have created a Trello card - please up vote it if you feel this should be a priority. https://trello.com/c/8K86l4Qk Thanks, -John On Wed, Feb 5, 2014 at 9:11 AM, Renato Alves rjal...@igc.gulbenkian.pt wrote: I'm trying to run functional tests on the stable branch of galaxy using: GALAXY_TEST_EXTERNAL=1 GALAXY_TEST_HOST=galaxy GALAXY_TEST_PORT=80 ./run_functional_tests.sh But I'm getting the error: Traceback (most recent call last): File ./scripts/functional_tests.py, line 457, in module sys.exit( main() ) File ./scripts/functional_tests.py, line 362, in main if galaxy_test_proxy_port: UnboundLocalError: local variable 'galaxy_test_proxy_port' referenced before assignment 'run_functional_tests.sh help' for help Am I doing something wrong? I'm following the instructions at https://wiki.galaxyproject.org/Admin/Running Tests Renato ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ signature.asc Description: OpenPGP digital signature ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Launching External functional tests fails with: local variable 'galaxy_test_proxy_port' referenced before assignment
I cannot think of another way to do this, sorry. -John On Thu, Feb 6, 2014 at 4:41 AM, Renato Alves rjal...@igc.gulbenkian.pt wrote: Hi John, Thanks for the reply and the trello card. Since there's no mention of this in the documentation, do you know if there are any other way(s) (official or not) of running functional tests on production servers? Thanks again, Renato Quoting John Chilton on 05-02-2014 21:41: I asked around a while ago about this (GALAXY_TEST_EXTERNAL) and I believe the consensus was it does not work and has not worked for sometime - this is unfortunate because it was very useful. I have created a Trello card - please up vote it if you feel this should be a priority. https://trello.com/c/8K86l4Qk Thanks, -John On Wed, Feb 5, 2014 at 9:11 AM, Renato Alves rjal...@igc.gulbenkian.pt wrote: I'm trying to run functional tests on the stable branch of galaxy using: GALAXY_TEST_EXTERNAL=1 GALAXY_TEST_HOST=galaxy GALAXY_TEST_PORT=80 ./run_functional_tests.sh But I'm getting the error: Traceback (most recent call last): File ./scripts/functional_tests.py, line 457, in module sys.exit( main() ) File ./scripts/functional_tests.py, line 362, in main if galaxy_test_proxy_port: UnboundLocalError: local variable 'galaxy_test_proxy_port' referenced before assignment 'run_functional_tests.sh help' for help Am I doing something wrong? I'm following the instructions at https://wiki.galaxyproject.org/Admin/Running Tests Renato ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Running Import History from File as local job
Hello Dr. Etherington, The id of the special tool is __IMPORT_HISTORY__ so if you have a local destination called local, the following tool entry SHOULD force local import - seemed to function the way I expected in a quick test anyway: tool id=__IMPORT_HISTORY__ destination=local / Hope this helps, -John On Thu, Feb 6, 2014 at 6:12 AM, graham etherington (TSL) graham.ethering...@sainsbury-laboratory.ac.uk wrote: Hi, I have two local instances of Galaxy running (both entirely independent of each other and running on separate clusters). I'm trying to export a history from Instance A to Instance B, but so far have not been able. I've attempted this in two ways. 1. Download the Galaxy_history.tar.gz file from Instance A (Options Export to File) and place it in a path on the filesystem which Galaxy B can see. Then Options Import from File and in the 'Archived History URL:' field place: '/full/path/to/Galaxy_history.tar.gz' (I've also tried 'file:///full/path/to/Galaxy_history.tar.gz') The job runs (submitted to LSF cluster) with no errors in paster.log, but the stdout file in the job_working_directory gives the error Exception getting file from URL: unknown url type:/full/path/to/Galaxy_history.tar.gz open file 'stderr', mode 'w' at 0x2ba2d4e9b1e0 Error unpacking tar/gz archive: nothing to open open file 'stderr', mode 'w' at 0x2ba2d4e9b1e0 2. Copy the url link for the History in Instance A (Options Share or Publish ), then in Instance B go Options Import from File and in the 'Archived History URL:' field place the url provided by Instance A. Again the job runs OK, but the stdout file has the error: Exception getting file from URL: urlopen error [Errno 111] Connection refused open file 'stderr', mode 'w' at 0x2b359790a1e0 Error unpacking tar/gz archive: nothing to open open file 'stderr', mode 'w' at 0x2b359790a1e0 I'm pretty sure method number 2 is failing because the Import History job is being run on the cluster which cannot see the outside world (and hence Instance A). I think I should be able to overcome this by specifying in job_conf.xml that the import tool be run locally. The problem is, that I've not been able to identify what the actual tool or process is that runs the Import Histories method. I know that it produces a bash script which runs lib/galaxy/tools/imp_exp/unpack_tar_gz_archive.py, but not being a standard tool (i.e. found in the ./tools/ directory with an xml wrapper), I can't find an ID for it. Naively I tried the following in job_conf.xml: (I have 'local' defined in 'destinations' and 'plugins') tools default=local !--make the import histories tool run locally-- tool id=lib/galaxy/tools/imp_exp/unpack_tar_gz_archive.py destination=local/ /tools But this made no difference. Does anyone know how I can either get the Import History tool to run locally or suggest another way to import my history? Many thanks, Graham Dr. Graham Etherington Bioinformatics Support Officer, The Sainsbury Laboratory, Norwich Research Park, Norwich NR4 7UH. UK Tel: +44 (0)1603 450601 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] SIFT error on local server
Hi, I am trying to get SIFT to run on my local server. I downloaded the sqlite files, and changed the sift_db.loc file to point to the directory I downloaded them to. The weird thing is that the output is green, as in completed successfully, but there is no output and the stdout is: failed: SIFT_exome_nssnvs.pl -i /home/alan/galaxy-dist/database/job_working_directory/000/101/sift_input.txt -d /home/alan/galaxy-dist/data/hg19/seq -o /home/alan/galaxy-dist/database/job_working_directory/000/101 -B 1 -J 1 -K 1 -L 1 Any help would be appreciated. Thanks, -- Alan Williams Life is short, art long, opportunity fleeting, experience deceptive, judgement difficult. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] running tools within tool
Hi John, Alex, All, Elaborating on the motivation behind my question of running tools within tool. First, running a tool in parallel at large-scale. For example, if I need to find a pattern from 1000 files via Galaxy Select tool from Text and Filter tool-group, I am limited by providing one file at a time to the tool which will take a long time to finish. Please correct me if there is a more sophisticated way to approach this problem. Second, related concern is running a tool in parallel on one or more HPC resources. We want to write a generic wrapper Galaxy tool, powered by Swift parallel framework such that it can run any arbitrary Galaxy tool in parallel on HPC resources. Currently, we have developed this capability but for external executables which is not a most secure way of using Galaxy as I understand from previous conversation. Having such a wrapper tool in a standard way is desirable so that it preserves the tool contract and binding within Galaxy environment. That is maintaining the history and metadata conventions of Galaxy. Thanks, Ketan On Wed, Feb 5, 2014 at 3:53 PM, John Chilton chil...@msi.umn.edu wrote: Galaxy has an API that is capable of running tools - certainly this is one path forward on something like this. I am not sure it is the best path forward though. Probably the best way to enhance Galaxy's execution capabilities is to extend the Galaxy core framework itself - this has its own downsides though. If you can offer more details about how you would like to enhance Galaxy - what it cannot do that you would like it to do - I or others may be able to provide more specific ideas. Otherwise, sorry I have not been or more help. -John On Tue, Feb 4, 2014 at 2:51 PM, Ketan Maheshwari ke...@mcs.anl.gov wrote: Hi, This is a question I posted to galaxy user mailing list a while back and was redirected to dev for possible answers: Is it possible in Galaxy to design a tool whose sole purpose is to run other tools. This is motivated by our desire to enhance execution capabilities of existing tools via a generic tool which acts as a wrapper. Thanks, Ketan ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Ketan ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Put Galaxy behind an index.php page
Hi Galaxy support, Currently we are using Apache proxy to Galaxy and followed the instructions on this webpage: https://wiki.galaxyproject.org/Admin/Config/Apache%20Proxy https://wiki.galaxyproject.org/Admin/Config/Apache%20Proxy. Now I am trying to put Galaxy behind a index.php webpage. The scenario would be: 1. user input the Galaxy URL; 2.directed to the php page and then verify their identity; 3. if approved, redirect to Galaxy. Does anyone knows how to configure this in Apache? The reason I am doing this is because the index.php page contains our two-factor authentication API. Any help is appreciated. Thanks. -Jing ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Running Import History from File as local job
Hi John, Thanks for your suggestion. I included the suggested line in my job_conf.xml file and sure enough the job was run locally. It exits without any errors, but unfortunately it still didn’t work – the imported history doesn’t appear in the history panel. Here’s the output from the paster log... galaxy.jobs DEBUG 2014-02-06 14:48:38,463 (714) Working directory for job is: /tsl/services/galaxy/dist/galaxy-dist/database/job_working_directory/000/714 galaxy.jobs.handler DEBUG 2014-02-06 14:48:38,471 (714) Dispatching to local runner galaxy.jobs DEBUG 2014-02-06 14:48:38,544 (714) Persisting job destination (destination id: local) galaxy.jobs.handler INFO 2014-02-06 14:48:38,595 (714) Job dispatched galaxy.jobs.runners.local DEBUG 2014-02-06 14:48:38,854 (714) executing: export GALAXY_SLOTS=1; python /tsl/services/galaxy/dist/galaxy-dist/lib/galaxy/tools/imp_exp/unpack_tar_gz_archive.py http://galaxy.tsl.ac.uk/u/ethering/h/galaxy-intro /tsl/services/galaxy/dist/galaxy-dist/database/tmp/tmp4yCMyk --url galaxy.jobs DEBUG 2014-02-06 14:48:38,936 (714) Persisting job destination (destination id: local) galaxy.jobs.runners.local DEBUG 2014-02-06 14:48:45,024 execution finished: export GALAXY_SLOTS=1; python /tsl/services/galaxy/dist/galaxy-dist/lib/galaxy/tools/imp_exp/unpack_tar_gz_archive.py http://galaxy.tsl.ac.uk/u/ethering/h/galaxy-intro /tsl/services/galaxy/dist/galaxy-dist/database/tmp/tmp4yCMyk --url galaxy.jobs DEBUG 2014-02-06 14:48:45,179 job 714 ended …and here’s a skimmed down version of my job_conf.xml file (TSL-Test128 is the default LSF queue that Galaxy jobs are sent to) ?xml version=1.0? job_conf plugins workers=8 plugin id=local type=runner load=galaxy.jobs.runners.local:LocalJobRunner workers=16/ plugin id=drmaa type=runner load=galaxy.jobs.runners.drmaa:DRMAAJobRunner workers=8/ /plugins handlers default=handlerLocal handler id=main tags=handlerLocal/ /handlers destinations default=TSL-Test128 destination id=local runner=local/ destination id=TSL-Test128 runner=drmaa tags=LSF,Test128 /destination /destinations tools default=local !--make the import histories tool run locally-- tool id=__IMPORT_HISTORY__ destination=local/ /tools /job_conf I’m presuming if there was something wrong with the xml that an error would be thrown, so I’m not sure if there’s something else that I’m missing. Many thanks, Graham From: John Chilton chil...@msi.umn.edumailto:chil...@msi.umn.edu Date: Thursday, 6 February 2014 14:32 To: graham etherington (TSL) graham.ethering...@sainsbury-laboratory.ac.ukmailto:graham.ethering...@sainsbury-laboratory.ac.uk Cc: Galaxy Dev galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Running Import History from File as local job Hello Dr. Etherington, The id of the special tool is __IMPORT_HISTORY__ so if you have a local destination called local, the following tool entry SHOULD force local import - seemed to function the way I expected in a quick test anyway: tool id=__IMPORT_HISTORY__ destination=local / Hope this helps, -John On Thu, Feb 6, 2014 at 6:12 AM, graham etherington (TSL) graham.ethering...@sainsbury-laboratory.ac.ukmailto:graham.ethering...@sainsbury-laboratory.ac.uk wrote: Hi, I have two local instances of Galaxy running (both entirely independent of each other and running on separate clusters). I'm trying to export a history from Instance A to Instance B, but so far have not been able. I've attempted this in two ways. 1. Download the Galaxy_history.tar.gz file from Instance A (Options Export to File) and place it in a path on the filesystem which Galaxy B can see. Then Options Import from File and in the 'Archived History URL:' field place: '/full/path/to/Galaxy_history.tar.gz' (I've also tried 'file:///full/path/to/Galaxy_history.tar.gz') The job runs (submitted to LSF cluster) with no errors in paster.log, but the stdout file in the job_working_directory gives the error Exception getting file from URL: unknown url type:/full/path/to/Galaxy_history.tar.gz open file 'stderr', mode 'w' at 0x2ba2d4e9b1e0 Error unpacking tar/gz archive: nothing to open open file 'stderr', mode 'w' at 0x2ba2d4e9b1e0 2. Copy the url link for the History in Instance A (Options Share or Publish ), then in Instance B go Options Import from File and in the 'Archived History URL:' field place the url provided by Instance A. Again the job runs OK, but the stdout file has the error: Exception getting file from URL: urlopen error [Errno 111] Connection refused open file 'stderr', mode 'w' at 0x2b359790a1e0 Error unpacking tar/gz archive: nothing to open open file 'stderr', mode 'w' at 0x2b359790a1e0 I'm pretty sure method number 2 is failing because the Import History job is being run on the cluster which cannot see the outside world (and hence Instance A). I think I should be able to overcome this by specifying in
Re: [galaxy-dev] Put Galaxy behind an index.php page
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Jing, I have never used this method of authenticating before, and have no experience with it, but it sounds like you're looking for http://httpd.apache.org/docs/current/mod/mod_auth_form.html I would imagine that apache will set a REMOTE_USER header which can then be used by galaxy but again, I do not know as I've never used it. Additionally, you'll have to configure REMOTE_USER=true in galaxy's universe_wsgi.ini. If you get this worked out, would you mind replying with your configuration information, so we can add that to the wiki? Cheers, Eric On 02/06/2014 10:01 AM, Jingchao Zhang wrote: Hi Galaxy support, Currently we are using Apache proxy to Galaxy and followed the instructions on this webpage: https://wiki.galaxyproject.org/Admin/Config/Apache%20Proxy https://wiki.galaxyproject.org/Admin/Config/Apache%20Proxy. Now I am trying to put Galaxy behind a index.php webpage. The scenario would be: 1. user input the Galaxy URL; 2.directed to the php page and then verify their identity; 3. if approved, redirect to Galaxy. Does anyone knows how to configure this in Apache? The reason I am doing this is because the index.php page contains our two-factor authentication API. Any help is appreciated. Thanks. -Jing ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ - -- Eric Rasche Programmer II Center for Phage Technology Texas AM University College Station, TX 77843 404-692-2048 tel:4046922048 e...@tamu.edu mailto:e...@tamu.edu -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJS87mSAAoJEMqDXdrsMcpVQPoP/3Ydbp9roIQ3M0dzfX22lOlg 9JX4D2g2ENVi7nlOY4/xxUqAK6jYSq/3y75hV5nAE4Mnb/GrbgKh4z9w232XIQL3 XLguNOpVfOJRvEcC7HeFfONz9aeRpO2igrAOGZwSjsqToJbPwyCl5zaiSjo1Ueyb YMXJIhoc2XX/967tlnvBSSgPhtsjNA6MSTdrBQfmqLWuml2Nz8saKzbYNm1iPXYy XzPdkC4UV+a4sX/Pk+uEwBOAMzjKldbmThG9VmN04KnG55oD4Z2+OjSDcowzOXfP Swx0NhhyDC5VePrYVLbc3PnZDJtnqcO5KGRakyCragcv9I7UiDhMJ1E2+u9hcox/ p/PdN7CTB0jgfBETC2KmJFzTyJg+Sgy2UXw73Z+kLlV2D9nJFAuaQFpEpPzI5WZW B9EMwQHPCkX5AspBUDQ1cIih4fFfSCWuXIpPpZvLUHaqne1Q7xXx0oQQOVgriZDv 9bh7fx1vZp+QRNXWqLl0J89lijnwizJHnttG5VaRDMJmlWyrMnCO+L6glaBmRkZp CV8QQ4BMlKdLT/Vk94mERjjFOrL2qYMbZWCW9aXz/laxJi6RVkaDwnpiZMxTe4By K619Qe7ZAPRqRDch8ezRSh5bodsF74Jmxbw41g4zTH+CQ4WrVY02hgQm5CxBGuRZ /kgejBhDOpvTh0jhkj5W =J8QL -END PGP SIGNATURE- ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Running Import History from File as local job
Any chance you have cleanup_job=never in universe_wsgi.ini? I recently pushed a bugfix to galaxy-central - history import wouldn't work if that is set. Otherwise you may want to review this thread for clues: http://lists.bx.psu.edu/pipermail/galaxy-dev/2013-December/017773.html -John On Thu, Feb 6, 2014 at 10:34 AM, graham etherington (TSL) graham.ethering...@sainsbury-laboratory.ac.uk wrote: Hi John, Thanks for your suggestion. I included the suggested line in my job_conf.xml file and sure enough the job was run locally. It exits without any errors, but unfortunately it still didn't work - the imported history doesn't appear in the history panel. Here's the output from the paster log... galaxy.jobs DEBUG 2014-02-06 14:48:38,463 (714) Working directory for job is: /tsl/services/galaxy/dist/galaxy-dist/database/job_working_directory/000/714 galaxy.jobs.handler DEBUG 2014-02-06 14:48:38,471 (714) Dispatching to local runner galaxy.jobs DEBUG 2014-02-06 14:48:38,544 (714) Persisting job destination (destination id: local) galaxy.jobs.handler INFO 2014-02-06 14:48:38,595 (714) Job dispatched galaxy.jobs.runners.local DEBUG 2014-02-06 14:48:38,854 (714) executing: export GALAXY_SLOTS=1; python /tsl/services/galaxy/dist/galaxy-dist/lib/galaxy/tools/imp_exp/unpack_tar_gz_archive.py http://galaxy.tsl.ac.uk/u/ethering/h/galaxy-intro /tsl/services/galaxy/dist/galaxy-dist/database/tmp/tmp4yCMyk --url galaxy.jobs DEBUG 2014-02-06 14:48:38,936 (714) Persisting job destination (destination id: local) galaxy.jobs.runners.local DEBUG 2014-02-06 14:48:45,024 execution finished: export GALAXY_SLOTS=1; python /tsl/services/galaxy/dist/galaxy-dist/lib/galaxy/tools/imp_exp/unpack_tar_gz_archive.py http://galaxy.tsl.ac.uk/u/ethering/h/galaxy-intro /tsl/services/galaxy/dist/galaxy-dist/database/tmp/tmp4yCMyk --url galaxy.jobs DEBUG 2014-02-06 14:48:45,179 job 714 ended ...and here's a skimmed down version of my job_conf.xml file (TSL-Test128 is the default LSF queue that Galaxy jobs are sent to) ?xml version=1.0? job_conf plugins workers=8 plugin id=local type=runner load=galaxy.jobs.runners.local:LocalJobRunner workers=16/ plugin id=drmaa type=runner load=galaxy.jobs.runners.drmaa:DRMAAJobRunner workers=8/ /plugins handlers default=handlerLocal handler id=main tags=handlerLocal/ /handlers destinations default=TSL-Test128 destination id=local runner=local/ destination id=TSL-Test128 runner=drmaa tags=LSF,Test128 /destination /destinations tools default=local !--make the import histories tool run locally-- tool id=__IMPORT_HISTORY__ destination=local/ /tools /job_conf I'm presuming if there was something wrong with the xml that an error would be thrown, so I'm not sure if there's something else that I'm missing. Many thanks, Graham From: John Chilton chil...@msi.umn.edu Date: Thursday, 6 February 2014 14:32 To: graham etherington (TSL) graham.ethering...@sainsbury-laboratory.ac.uk Cc: Galaxy Dev galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Running Import History from File as local job Hello Dr. Etherington, The id of the special tool is __IMPORT_HISTORY__ so if you have a local destination called local, the following tool entry SHOULD force local import - seemed to function the way I expected in a quick test anyway: tool id=__IMPORT_HISTORY__ destination=local / Hope this helps, -John On Thu, Feb 6, 2014 at 6:12 AM, graham etherington (TSL) graham.ethering...@sainsbury-laboratory.ac.uk wrote: Hi, I have two local instances of Galaxy running (both entirely independent of each other and running on separate clusters). I'm trying to export a history from Instance A to Instance B, but so far have not been able. I've attempted this in two ways. 1. Download the Galaxy_history.tar.gz file from Instance A (Options Export to File) and place it in a path on the filesystem which Galaxy B can see. Then Options Import from File and in the 'Archived History URL:' field place: '/full/path/to/Galaxy_history.tar.gz' (I've also tried 'file:///full/path/to/Galaxy_history.tar.gz') The job runs (submitted to LSF cluster) with no errors in paster.log, but the stdout file in the job_working_directory gives the error Exception getting file from URL: unknown url type:/full/path/to/Galaxy_history.tar.gz open file 'stderr', mode 'w' at 0x2ba2d4e9b1e0 Error unpacking tar/gz archive: nothing to open open file 'stderr', mode 'w' at 0x2ba2d4e9b1e0 2. Copy the url link for the History in Instance A (Options Share or Publish ), then in Instance B go Options Import from File and in the 'Archived History URL:' field place the url provided by Instance A. Again the job runs OK, but the stdout file has the error: Exception getting file from URL: urlopen error [Errno 111] Connection refused open file 'stderr', mode 'w' at 0x2b359790a1e0 Error
Re: [galaxy-dev] Running Import History from File as local job
Hi John, As the Galaxy instance I’m importing into is quite a new build, I indeed had 'cleanup_job=never’ for debugging purposes. I changed this (to ‘onsuccess’) and I was able to import a history from my account on the public Galaxy server, but still not my other local instance. I think there must be some sort of local networking issue going on here and that the history import is just failing silently. I have managed now though to download the Galaxy_history.tar.gz file, place it on a pubic server and import it from there, so at least that’s a little hack that I can use for now. Anyway, other than the silent fail, I don’t think this is a Galaxy problem. Thanks for all your help! Best wishes, Graham Dr. Graham Etherington Bioinformatics Support Officer, The Sainsbury Laboratory, Norwich Research Park, Norwich NR4 7UH. UK Tel: +44 (0)1603 450601 From: John Chilton chil...@msi.umn.edumailto:chil...@msi.umn.edu Date: Thursday, 6 February 2014 16:54 To: graham etherington (TSL) graham.ethering...@sainsbury-laboratory.ac.ukmailto:graham.ethering...@sainsbury-laboratory.ac.uk Cc: Galaxy Dev galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Running Import History from File as local job Any chance you have cleanup_job=never in universe_wsgi.ini? I recently pushed a bugfix to galaxy-central - history import wouldn't work if that is set. Otherwise you may want to review this thread for clues: http://lists.bx.psu.edu/pipermail/galaxy-dev/2013-December/017773.html -John On Thu, Feb 6, 2014 at 10:34 AM, graham etherington (TSL) graham.ethering...@sainsbury-laboratory.ac.ukmailto:graham.ethering...@sainsbury-laboratory.ac.uk wrote: Hi John, Thanks for your suggestion. I included the suggested line in my job_conf.xml file and sure enough the job was run locally. It exits without any errors, but unfortunately it still didn't work - the imported history doesn't appear in the history panel. Here's the output from the paster log... galaxy.jobs DEBUG 2014-02-06 14:48:38,463 (714) Working directory for job is: /tsl/services/galaxy/dist/galaxy-dist/database/job_working_directory/000/714 galaxy.jobs.handler DEBUG 2014-02-06 14:48:38,471 (714) Dispatching to local runner galaxy.jobs DEBUG 2014-02-06 14:48:38,544 (714) Persisting job destination (destination id: local) galaxy.jobs.handler INFO 2014-02-06 14:48:38,595 (714) Job dispatched galaxy.jobs.runners.local DEBUG 2014-02-06 14:48:38,854 (714) executing: export GALAXY_SLOTS=1; python /tsl/services/galaxy/dist/galaxy-dist/lib/galaxy/tools/imp_exp/unpack_tar_gz_archive.py http://galaxy.tsl.ac.uk/u/ethering/h/galaxy-intro /tsl/services/galaxy/dist/galaxy-dist/database/tmp/tmp4yCMyk --url galaxy.jobs DEBUG 2014-02-06 14:48:38,936 (714) Persisting job destination (destination id: local) galaxy.jobs.runners.local DEBUG 2014-02-06 14:48:45,024 execution finished: export GALAXY_SLOTS=1; python /tsl/services/galaxy/dist/galaxy-dist/lib/galaxy/tools/imp_exp/unpack_tar_gz_archive.py http://galaxy.tsl.ac.uk/u/ethering/h/galaxy-intro /tsl/services/galaxy/dist/galaxy-dist/database/tmp/tmp4yCMyk --url galaxy.jobs DEBUG 2014-02-06 14:48:45,179 job 714 ended ...and here's a skimmed down version of my job_conf.xml file (TSL-Test128 is the default LSF queue that Galaxy jobs are sent to) ?xml version=1.0? job_conf plugins workers=8 plugin id=local type=runner load=galaxy.jobs.runners.local:LocalJobRunner workers=16/ plugin id=drmaa type=runner load=galaxy.jobs.runners.drmaa:DRMAAJobRunner workers=8/ /plugins handlers default=handlerLocal handler id=main tags=handlerLocal/ /handlers destinations default=TSL-Test128 destination id=local runner=local/ destination id=TSL-Test128 runner=drmaa tags=LSF,Test128 /destination /destinations tools default=local !--make the import histories tool run locally-- tool id=__IMPORT_HISTORY__ destination=local/ /tools /job_conf I'm presuming if there was something wrong with the xml that an error would be thrown, so I'm not sure if there's something else that I'm missing. Many thanks, Graham From: John Chilton chil...@msi.umn.edumailto:chil...@msi.umn.edu Date: Thursday, 6 February 2014 14:32 To: graham etherington (TSL) graham.ethering...@sainsbury-laboratory.ac.ukmailto:graham.ethering...@sainsbury-laboratory.ac.uk Cc: Galaxy Dev galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Running Import History from File as local job Hello Dr. Etherington, The id of the special tool is __IMPORT_HISTORY__ so if you have a local destination called local, the following tool entry SHOULD force local import - seemed to function the way I expected in a quick test anyway: tool id=__IMPORT_HISTORY__ destination=local / Hope this helps, -John On Thu, Feb 6, 2014 at 6:12 AM, graham etherington (TSL)
[galaxy-dev] Galaxy in there server
Hi Galaxy Team, I want to install a local instance of Galaxy on our lab server. It is a Unix server with pretty good RAM and storage. Is there any wiki page or instruction for installation. Thanks, Bishwo ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Galaxy in there server
On 06/02/2014 19:11, Bishwo Adhikari wrote: Hi Galaxy Team, I want to install a local instance of Galaxy on our lab server. It is a Unix server with pretty good RAM and storage. Is there any wiki page or instruction for installation. Thanks, Bishwo ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ Hello, start here: https://wiki.galaxyproject.org/Admin/GetGalaxy Regards, Thomas -- Thomas Bellembois, network and system administrator ENS-Lyon/IGFL - 46, Allée d'Italie 69364 LYON CEDEX 07 FRANCE +33 4 26 73 13 67 .The IGFL is a member of the portail-labos project (https://portail-labos.ens-lyon.fr/site/). .IGFL IT documentation: http://itdoc.igfl.ens-lyon.fr/itdoc ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Galaxy in there server
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 http://getgalaxy.org/ On 02/06/2014 12:11 PM, Bishwo Adhikari wrote: Hi Galaxy Team, I want to install a local instance of Galaxy on our lab server. It is a Unix server with pretty good RAM and storage. Is there any wiki page or instruction for installation. Thanks, Bishwo ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ - -- Eric Rasche Programmer II Center for Phage Technology Texas AM University College Station, TX 77843 404-692-2048 tel:4046922048 e...@tamu.edu mailto:e...@tamu.edu -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJS896ZAAoJEMqDXdrsMcpVYnIP+wTfmgrzsy+64HALIj1YYUJG MCUlJ/NWlcY5jzM4nmbZgq3CBAXiui6qgvzwnCoooTQHg1No8t1qly0IxERLm0wW XGJOa2q6rKBUUiATPCv39D8g/nF33y+wMOsXQEdvta/z/ZktNnfwYeIg86GqfwhB AfQJ1GQ19I1fphuRSv48e9ijEqB56xEwtdCauPQjbgMlt+kcCzfOLdXZX1auvI0U 52vzlDAakJE2sBUrkGqo/wyJX7UkKcD7NKjSIT5/QsSiljtDRmMgvUqy8TkiLSPC EE29d8jGTI4iaR2ddnjie3ElSXvrJowP4FWusp1jLooLczlQ7TEfkMOmA5rxI5B9 DUTF1koWw70ePxcVcnYDYsLVaaqt7W884hGdfaynr7OXrNXmOu6YhTz8TN4gChuM LvYSTQY0jR4wk37Ocio7DPpPAkcsiLSIrZEchMtYK28ZqKsZSR91dWQeN8BIwkuT Npw01HYIH+KtFQMBoNzkf2Pvm6rYIYuB52LHVY21rc/59v1g4KBVsW9+i2aHCUQz K+8YSZQWVHKqxfNH5UnPTvkjv33R7pCDRI1GkqHnO05k+eI9eUls2YDTuUchYwz9 RASBp/yNa6ALh1PZ0LivFqgH1qIJhBbzMRea1BpILMS5PbeH62uvhT+9WY7FdqKZ OIqaBMbuXWbP3k1jXQBV =y2iq -END PGP SIGNATURE- ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] sysadmin help
I have a galaxy server set up, running off of our HPC cluster file system. I'd like to submit a simple job to test my settings with our moab scheduler. Could someone point me to the easiest possible example of uploading and running a job... possible an example a sysadmin with limited knowledge could follow ;) Thanks, Don FSU RCC ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] sysadmin help
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I believe there are select first and select last tools in the Text Utilites section (i.e., head and tail wrappers) you might try those... On 02/06/2014 01:37 PM, Shrum, Donald C wrote: I have a galaxy server set up, running off of our HPC cluster file system. I'd like to submit a simple job to test my settings with our moab scheduler. Could someone point me to the easiest possible example of uploading and running a job... possible an example a sysadmin with limited knowledge could follow ;) Thanks, Don FSU RCC ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ - -- Eric Rasche Programmer II Center for Phage Technology Texas AM University College Station, TX 77843 404-692-2048 tel:4046922048 e...@tamu.edu mailto:e...@tamu.edu -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJS8+cRAAoJEMqDXdrsMcpVR6IQAJLHsE5rbngBFAohv3n4mjDa A0ktXx6wsL9MKN6cqtr13DOn2ZNMMagxiHGEKxNN/Qs5i0P7Twtne257jO9dTuuo CIl7n+QeSmjvsnyXCQDNwhBUnpuc+QkdnL7uy5viP7X5E8mHOENTxFC7qlS/NTZE v9Gi09z4VWhFZ+QG33e1U9M1asqW1jxszApefHj21logaSTRiHyF0B924VTLTRl5 Cv+lvRqoKNn4c++8Op1UPsW2GsT7jQ5PGZTlnE2Ny1tGOmhFwQaPBB2M8ZKlokRa WS5W2zuo+aN8pvlKUGM2pEaRlBs8GJd4SChAr6rp0ykiYEtDuZCyphL0Pcm2tFUi ZtsrJYN11zDVrHAQt1rX6Ik4vaec8GPDFG1WOFrur/whDpWKGvDx9YyrwR3nN2nV 7e+QCnA6I2SOgjvB8XJy2EkuFxlavjAOXkWKrt6KIUpw593xSt9NXD2j1isqGlrL sSM7vxGyvNubQ3vQaIW0QX3HdWz80OV6neq4+HnGhhGYd8B31QJvbspjS0KQ5rz4 yyPB+Bt0F/EmGvCa4yW8Q9z97WNWup6q25lHCNfEJQR6KPHFYZ8FR41JcJW0EVXK XfqwsYr2mWo2BKeX5GAIehjd9GUAF+NsBJi/2L2uN7m09hwHMmhTk2e3yDQWWaRK 9AuKU/whhN4LEIuXYDrk =S5v7 -END PGP SIGNATURE- ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
