[galaxy-dev] shall us specify which tool run in cluster?
Hi Thon Deboer , I am a newer in Galaxy.I installed my Galaxy with Torque2.5.0 ,and Galaxy uses the pbs_modoule to interface with TORQUE.But I have some question of the job_conf.xml : 1.)In your job_conf.xml ,you use regularjobs,longjobs,shortjobs...to run different jobs ,how our Galaxy know which tool belongs to regularjobs or longjobs.And what is the meaning of nativeSpecification? 2.)Shall us use toolscollection of tool id=bwa_wrapper destination=multicorejobs4/to specify bwa ?Does it mean the bwa belong to multicorejobs4,and run in cluster? 3.)Does every tool need us to specify which job it belong to? I saw http://wiki.galaxyproject.org/Admin/Config/Jobs about this,but I am not sure above.Could you help me please? shenwiyn From: Thon Deboer Date: 2013-07-18 14:31 To: galaxy-dev Subject: [galaxy-dev] Jobs remain in queue until restart Hi, I have noticed that from time to time, the job queue seems to be “stuck” and can only be unstuck by restarting galaxy. The jobs seem to be in the queue state and the python job handler processes are hardly ticking over and the cluster is empty. When I restart, the startup procedure realizes all jobs are in the a “new state” and it then assigns a jobhandler after which the jobs start fine…. Any ideas? Torque Thon P.S I am using the june version of galaxy and I DO set limits on my users in job_conf.xml as so: (Maybe it is related? Before it went into dormant mode, this user had started lots of jobs and may have hit the limit, but I assumed this limit was the number of running jobs at one time, right?) ?xml version=1.0? job_conf plugins workers=4 !-- workers is the number of threads for the runner's work queue. The default from plugins is used if not defined for a plugin. -- plugin id=local type=runner load=galaxy.jobs.runners.local:LocalJobRunner workers=2/ plugin id=drmaa type=runner load=galaxy.jobs.runners.drmaa:DRMAAJobRunner workers=8/ plugin id=cli type=runner load=galaxy.jobs.runners.cli:ShellJobRunner workers=2/ /plugins handlers default=handlers !-- Additional job handlers - the id should match the name of a [server:id] in universe_wsgi.ini. -- handler id=handler0 tags=handlers/ handler id=handler1 tags=handlers/ handler id=handler2 tags=handlers/ handler id=handler3 tags=handlers/ !-- handler id=handler10 tags=handlers/ handler id=handler11 tags=handlers/ handler id=handler12 tags=handlers/ handler id=handler13 tags=handlers/ -- /handlers destinations default=regularjobs !-- Destinations define details about remote resources and how jobs should be executed on those remote resources. -- destination id=local runner=local/ destination id=regularjobs runner=drmaa tags=cluster !-- These are the parameters for qsub, such as queue etc. -- param id=nativeSpecification-V -q long.q -pe smp 1/param /destination destination id=longjobs runner=drmaa tags=cluster,long_jobs !-- These are the parameters for qsub, such as queue etc. -- param id=nativeSpecification-V -q long.q -pe smp 1/param /destination destination id=shortjobs runner=drmaa tags=cluster,short_jobs !-- These are the parameters for qsub, such as queue etc. -- param id=nativeSpecification-V -q short.q -pe smp 1/param /destination destination id=multicorejobs4 runner=drmaa tags=cluster,multicore_jobs !-- These are the parameters for qsub, such as queue etc. -- param id=nativeSpecification-V -q long.q -pe smp 4/param /destination !-- destination id=real_user_cluster runner=drmaa param id=galaxy_external_runjob_scriptscripts/drmaa_external_runner.py/param param id=galaxy_external_killjob_scriptscripts/drmaa_external_killer.py/param param id=galaxy_external_chown_scriptscripts/external_chown_script.py/param /destination -- destination id=dynamic runner=dynamic !-- A destination that represents a method in the dynamic runner. -- param id=typepython/param param id=functioninteractiveOrCluster/param /destination /destinations tools !-- Tools can be configured to use specific destinations or handlers, identified by either the id or tags attribute. If assigned to a tag, a handler or destination that matches that tag will be chosen at random. -- tool id=bwa_wrapper destination=multicorejobs4/ /tools limits !-- Certain limits can be defined. limit type=registered_user_concurrent_jobs500/limit limit type=unregistered_user_concurrent_jobs1/limit limit type=concurrent_jobs id=local1/limit
Re: [galaxy-dev] Build the Galaxy under Cluster
Hi Carlos, Thank you very much for your help.I use Galaxy with the torque2.5.0,pbs_python for a test.It seems to work successfully ! shenwiyn From: Carlos Borroto Date: 2013-07-10 01:49 To: shenwiyn CC: galaxy-...@bx.psu.edu Subject: Re: [galaxy-dev] Build the Galaxy under Cluster Hi shenwyn, This is a very generic question hard to answer. I can tell you I use Galaxy with the torque 4.1.x branch, maui scheduler and pbs-drmaa. I would suggest you take a look a this[1] wiki page. Please notice as it is mentioned in that page, using libdrmaa.so from torque won't work, I can confirm that. [1]http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster Hope it helps, Carlos On Tue, Jul 2, 2013 at 5:50 AM, shenwiyn shenw...@gmail.com wrote: Hi everyone, I want to install my local Galaxy under the Cluster with TORQUE.I have a question about how they work together successfully.In addition,after my local Galaxy and TORQUE install successfully,anything else should I do to the Cluster Galaxy?Thank you very much. shenwiyn ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] BWA missing index on reference genome
Hey, 1 I installed Galaxy locally. 2 I also have BWA installed and then used the 3 shed repository to get the bwa wrapper. 4 I have the reference genome: -rw-r--r-- 1 trr users 10509 2013-07-25 11:31 human_g1k_v37.dict -rw-r--r-- 1 trr users 3153506519 2013-07-25 11:28 human_g1k_v37.fasta -rw-r--r-- 1 trr users 6597 2013-07-25 19:41 human_g1k_v37.fasta.amb -rw-r--r-- 1 trr users 6844 2013-07-25 19:41 human_g1k_v37.fasta.ann -rw-r--r-- 1 trr users 3101804844 2013-07-25 19:41 human_g1k_v37.fasta.bwt -rw-r--r-- 1 trr users 2746 2013-07-25 11:32 human_g1k_v37.fasta.fai -rw-r--r-- 1 trr users 775451186 2013-07-25 19:41 human_g1k_v37.fasta.pac -rw-r--r-- 1 trr users 1550902424 2013-07-25 20:02 human_g1k_v37.fasta.sa 5 and I changed the bwa_index and bw_index_color.loc to the path /genedata/human_genome_GRCh37/hg19.fa 6 This is my *$PATH* trr@portalmoritz:~ echo $PATH /home/trr/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/home/trr/bpipe-0.9.8/bin:/home/trr/bwa-0.7.5a:/home/trr/samtools-0.1.19 Is there anything I am missing? I would be glad about some help. Best Moritz ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Galaxy ego fetching error (Mac OS X 10.8)
Hello Galaxy!
I can actually confirm this issue in my macbook air after fetching a fresh
clone from galaxy-dist (a few minutes ago) and running a:
$ rm -rf eggs/* ./run.sh
(… many correctly fetched eggs… )
Traceback (most recent call last):
File ./scripts/fetch_eggs.py, line 37, in module
c.resolve() # Only fetch eggs required by the config
File /Users/roman/dev/galaxy-dist/lib/galaxy/eggs/__init__.py, line 345, in
resolve
egg.resolve()
File /Users/roman/dev/galaxy-dist/lib/galaxy/eggs/__init__.py, line 195, in
resolve
return self.version_conflict( e.args[0], e.args[1] )
File /Users/roman/dev/galaxy-dist/lib/galaxy/eggs/__init__.py, line 226, in
version_conflict
r = pkg_resources.working_set.resolve( ( dist.as_requirement(), ), env,
egg.fetch )
File
/Users/roman/.venvburrito/lib/python/distribute-0.6.49-py2.7.egg/pkg_resources.py,
line 596, in resolve
raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: mercurial==2.2.3
Fetch failed.
Immediately after running the command above, re-running ./run.sh gives:
Some eggs are out of date, attempting to fetch...
Warning: MarkupSafe (a dependent egg of Mako) cannot be fetched
Warning: pycrypto (a dependent egg of Fabric) cannot be fetched
Warning: SQLAlchemy (a dependent egg of sqlalchemy-migrate) cannot be fetched
Warning: simplejson (a dependent egg of WebHelpers) cannot be fetched
Fetched http://eggs.galaxyproject.org/ssh/ssh-1.7.14-py2.7.egg
One of Galaxy's managed eggs depends on something which is missing, this is
almost certainly a bug in the egg distribution.
Dependency ssh requires pycrypto=2.1,!=2.4
Traceback (most recent call last):
File ./scripts/fetch_eggs.py, line 37, in module
c.resolve() # Only fetch eggs required by the config
File /Users/roman/dev/galaxy-dist/lib/galaxy/eggs/__init__.py, line 345, in
resolve
egg.resolve()
File /Users/roman/dev/galaxy-dist/lib/galaxy/eggs/__init__.py, line 168, in
resolve
dists = pkg_resources.working_set.resolve( (
self.distribution.as_requirement(), ), env, self.fetch )
File
/Users/roman/.venvburrito/lib/python/distribute-0.6.49-py2.7.egg/pkg_resources.py,
line 600, in resolve
raise VersionConflict(dist,req) # XXX put more info here
pkg_resources.VersionConflict: (ssh 1.7.14
(/Users/roman/dev/galaxy-dist/eggs/ssh-1.7.14-py2.7.egg),
Requirement.parse('pycrypto=2.1,!=2.4'))
Fetch failed.
So I tried to upgrade virtualenv-burrito, just in case it was a problem with
its pkg_resources being outdated?:
$ virtualenv-burrito update
Everything is up to date.
Then unsetted PYTHONPATH to use MacOSX's python base installation but same
error appeared:
$ unset PYTHONPATH
Traceback (most recent call last):
File ./scripts/fetch_eggs.py, line 37, in module
c.resolve() # Only fetch eggs required by the config
File /Users/roman/dev/galaxy-dist/lib/galaxy/eggs/__init__.py, line 345, in
resolve
egg.resolve()
File /Users/roman/dev/galaxy-dist/lib/galaxy/eggs/__init__.py, line 195, in
resolve
return self.version_conflict( e.args[0], e.args[1] )
File /Users/roman/dev/galaxy-dist/lib/galaxy/eggs/__init__.py, line 226, in
version_conflict
r = pkg_resources.working_set.resolve( ( dist.as_requirement(), ), env,
egg.fetch )
File
/Library/Python/2.7/site-packages/distribute-0.6.28-py2.7.egg/pkg_resources.py,
line 588, in resolve
raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: mercurial==2.2.3
Fetch failed.
Apparently the correct eggs are up there in the galaxy egg repo:
http://eggs.galaxyproject.org/mercurial/
I checked pull requests and bug reports first in trello, but there doesn't seem
to be a ticket for this one.
Cheers!
Roman
16 jul 2013 kl. 20:59 skrev Iry Witham iry.wit...@jax.org:
Hi Team,
I am attempting to rebuild my local instance of galaxy on my Mac since having
been upgraded to Mountain Lion. I have installed Mercurial, postgres,
python2.7. I have cloned the latest galaxy-dist. However, when I attempted
to launch galaxy with sh run.sh it fails to complete the fetch. Here is what
I get:
milkyway:galaxy-dist itw$ sh run.sh
Initializing datatypes_conf.xml from datatypes_conf.xml.sample
Initializing external_service_types_conf.xml from
external_service_types_conf.xml.sample
Initializing migrated_tools_conf.xml from migrated_tools_conf.xml.sample
Initializing reports_wsgi.ini from reports_wsgi.ini.sample
Initializing shed_tool_conf.xml from shed_tool_conf.xml.sample
Initializing tool_conf.xml from tool_conf.xml.sample
Initializing shed_tool_data_table_conf.xml from
shed_tool_data_table_conf.xml.sample
Initializing tool_data_table_conf.xml from tool_data_table_conf.xml.sample
Initializing tool_sheds_conf.xml from tool_sheds_conf.xml.sample
Initializing data_manager_conf.xml from data_manager_conf.xml.sample
Initializing shed_data_manager_conf.xml from shed_data_manager_conf.xml.sample
Initializing openid_conf.xml from openid_conf.xml.sample
Re: [galaxy-dev] BWA missing index on reference genome
Hi Moritz, Pls see below On Jul 26, 2013, at 6:37 AM, Moritz Juchler juch...@stud.uni-heidelberg.de wrote: Hey, 1 I installed Galaxy locally. 2 I also have BWA installed and then used the 3 shed repository to get the bwa wrapper. 4 I have the reference genome: -rw-r--r-- 1 trr users 10509 2013-07-25 11:31 human_g1k_v37.dict -rw-r--r-- 1 trr users 3153506519 2013-07-25 11:28 human_g1k_v37.fasta -rw-r--r-- 1 trr users 6597 2013-07-25 19:41 human_g1k_v37.fasta.amb -rw-r--r-- 1 trr users 6844 2013-07-25 19:41 human_g1k_v37.fasta.ann -rw-r--r-- 1 trr users 3101804844 2013-07-25 19:41 human_g1k_v37.fasta.bwt -rw-r--r-- 1 trr users 2746 2013-07-25 11:32 human_g1k_v37.fasta.fai -rw-r--r-- 1 trr users 775451186 2013-07-25 19:41 human_g1k_v37.fasta.pac -rw-r--r-- 1 trr users 1550902424 2013-07-25 20:02 human_g1k_v37.fasta.sa 5 and I changed the bwa_index and bw_index_color.loc to the path /genedata/human_genome_GRCh37/hg19.fa This is the problem - the path must point to the full name of the fasta file. In your case human_g1kv_v37.fasta. Using the name without the fasta as the base name in other fields as the unique dbkey is a good practice. Hopefully this helps, Jen Galaxy team 6 This is my $PATH trr@portalmoritz:~ echo $PATH /home/trr/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/home/trr/bpipe-0.9.8/bin:/home/trr/bwa-0.7.5a:/home/trr/samtools-0.1.19 Is there anything I am missing? I would be glad about some help. Best Moritz ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ Jennifer Hillman-Jackson Galaxy Support Training http://galaxyproject.org___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] BWA missing index on reference genome
I solved it so far :) Thanks for your help. On 26 July 2013 16:15, Jennifer Jackson j...@bx.psu.edu wrote: Hi Moritz, Pls see below On Jul 26, 2013, at 6:37 AM, Moritz Juchler juch...@stud.uni-heidelberg.de wrote: Hey, 1 I installed Galaxy locally. 2 I also have BWA installed and then used the 3 shed repository to get the bwa wrapper. 4 I have the reference genome: -rw-r--r-- 1 trr users 10509 2013-07-25 11:31 human_g1k_v37.dict -rw-r--r-- 1 trr users 3153506519 2013-07-25 11:28 human_g1k_v37.fasta -rw-r--r-- 1 trr users 6597 2013-07-25 19:41 human_g1k_v37.fasta.amb -rw-r--r-- 1 trr users 6844 2013-07-25 19:41 human_g1k_v37.fasta.ann -rw-r--r-- 1 trr users 3101804844 2013-07-25 19:41 human_g1k_v37.fasta.bwt -rw-r--r-- 1 trr users 2746 2013-07-25 11:32 human_g1k_v37.fasta.fai -rw-r--r-- 1 trr users 775451186 2013-07-25 19:41 human_g1k_v37.fasta.pac -rw-r--r-- 1 trr users 1550902424 2013-07-25 20:02 human_g1k_v37.fasta.sa 5 and I changed the bwa_index and bw_index_color.loc to the path /genedata/human_genome_GRCh37/hg19.fa This is the problem - the path must point to the full name of the fasta file. In your case human_g1kv_v37.fasta. Using the name without the fasta as the base name in other fields as the unique dbkey is a good practice. Hopefully this helps, Jen Galaxy team 6 This is my *$PATH* trr@portalmoritz:~ echo $PATH /home/trr/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/home/trr/bpipe-0.9.8/bin:/home/trr/bwa-0.7.5a:/home/trr/samtools-0.1.19 Is there anything I am missing? I would be glad about some help. Best Moritz ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ Jennifer Hillman-Jackson Galaxy Support Training http://galaxyproject.org ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] BWA missing index on reference genome
Hey, now I'm stuck with the calculate SA coordinate. Segmenation fault. Google tells me that it could be a problem with -space (I have 4GB left) -the version I use. (I use 0.5.9 and did a succesful alignment with 0.7.5a, but the tool shed says 0.5.9, hence could I just change to the current version, with which I did a succesful alignment?) Best Moritz On 26 July 2013 16:15, Jennifer Jackson j...@bx.psu.edu wrote: Hi Moritz, Pls see below On Jul 26, 2013, at 6:37 AM, Moritz Juchler juch...@stud.uni-heidelberg.de wrote: Hey, 1 I installed Galaxy locally. 2 I also have BWA installed and then used the 3 shed repository to get the bwa wrapper. 4 I have the reference genome: -rw-r--r-- 1 trr users 10509 2013-07-25 11:31 human_g1k_v37.dict -rw-r--r-- 1 trr users 3153506519 2013-07-25 11:28 human_g1k_v37.fasta -rw-r--r-- 1 trr users 6597 2013-07-25 19:41 human_g1k_v37.fasta.amb -rw-r--r-- 1 trr users 6844 2013-07-25 19:41 human_g1k_v37.fasta.ann -rw-r--r-- 1 trr users 3101804844 2013-07-25 19:41 human_g1k_v37.fasta.bwt -rw-r--r-- 1 trr users 2746 2013-07-25 11:32 human_g1k_v37.fasta.fai -rw-r--r-- 1 trr users 775451186 2013-07-25 19:41 human_g1k_v37.fasta.pac -rw-r--r-- 1 trr users 1550902424 2013-07-25 20:02 human_g1k_v37.fasta.sa 5 and I changed the bwa_index and bw_index_color.loc to the path /genedata/human_genome_GRCh37/hg19.fa This is the problem - the path must point to the full name of the fasta file. In your case human_g1kv_v37.fasta. Using the name without the fasta as the base name in other fields as the unique dbkey is a good practice. Hopefully this helps, Jen Galaxy team 6 This is my *$PATH* trr@portalmoritz:~ echo $PATH /home/trr/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/home/trr/bpipe-0.9.8/bin:/home/trr/bwa-0.7.5a:/home/trr/samtools-0.1.19 Is there anything I am missing? I would be glad about some help. Best Moritz ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ Jennifer Hillman-Jackson Galaxy Support Training http://galaxyproject.org ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Regarding FTP configuration
Hello all: I am setting up a local instance of galaxy server. I have followed the steps mentioned in the galaxy wiki regarding the setup. I have setup a proftpd server with postgresql authentication. But, I am getting 530 login incorrect error when trying to connect using a FTP client. I followed the solution of a similar problem given here: http://dev.list.galaxyproject.org/ProFTPD-integration-with-Galaxy-td4660295.html without any success. In the last post of the given link Mr. Tobias has posted some thing like this: I followed the advice you offered back-channel and backed off the galaxy changes to support pbkdf2. I am unable to follow this part. For reference- My proftpd configuration: http://pastebin.com/LUjAyHvh proftpd SQL log: http://pastebin.com/2psTtnZv proftpd -l Compiled-in modules: mod_core.c mod_xfer.c mod_rlimit.c mod_auth_unix.c mod_auth.c mod_ls.c mod_log.c mod_site.c mod_delay.c mod_facts.c mod_auth_pam.c mod_sql.c mod_sql_postgres.c mod_sql_passwd.c mod_cap.c A solution will be appreciated. Thank you, -- Jibesh Patra ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Fwd: Intersect Tool
Dear All, I am running my own galaxy instance and I have been using the tool intersect (under the tab operate on genomic intervals) a few times successfully. However, recently I get following error message: /home/galaxy/galaxy-dist/lib/galaxy/__init__.py:79: UserWarning: Module markupsafe was already imported from /usr/local/lib/python2.7/dist-packages/markupsafe/__init__.pyc, but /home/galaxy/galaxy-dist/eggs/MarkupSafe-0.12-py2.7-linux-x86_64-ucs4.egg is being added to sys.path self.check_version_conflict() I googled it and there was never a straight answer. Could anyon explain how to fix it or at least tell me what it means. Thank you for your help in advance. Keep up the great work! Best wishes, Alex -- Alexander Kurze, DPhil University of Oxford Department of Biochemistry South Parks Road Oxford, OX1 3QU United Kingdom Tel: +44 1865 613 230 Fax:+44 1865 613 341 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] CloudMan Error
Dannon- the problems that I saw were with the workaround (see last post). The instance launches, but the program stops when it can't find something (module/dependencies). There were also some different steps in setting up the instance. I had to create my own key pair, whereas in the past I never had to do that. I guess that simply using the share string that Greg at Janelia created made this step automatic. The cluster was working well at the end of June. The problem arose suddenly Saturday June 29, when the share string would not load. Thanks again, Deniz On Thu, Jul 25, 2013 at 5:19 PM, Dannon Baker dannon.ba...@gmail.comwrote: On Thu, Jul 25, 2013 at 4:47 PM, Deniz Erezyilmaz deniz...@gmail.comwrote: The workaround doesn't seem to work Can you tell me what happens when you launch with the workaround mentioned previously ( https://main.g2.bx.psu.edu/cloudlaunch?ami=ami-da58aab3bucket_default=gxy-workshop)? This should launch the pre-migration AMI and Cloudman and shouldn't have any issues at all unless your cluster was already wonky. -Dannon ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] CloudMan Error
Ahh, ok, I misunderstood and thought you were saying there was a different error for the workaround. Are you able to send me the share string (directly if you want, feel free to drop the list) so that I might take a look? There should be no change at all in your instances from before the new release when using the workaround. On Fri, Jul 26, 2013 at 8:17 AM, Deniz Erezyilmaz deniz...@gmail.comwrote: Dannon- the problems that I saw were with the workaround (see last post). The instance launches, but the program stops when it can't find something (module/dependencies). There were also some different steps in setting up the instance. I had to create my own key pair, whereas in the past I never had to do that. I guess that simply using the share string that Greg at Janelia created made this step automatic. The cluster was working well at the end of June. The problem arose suddenly Saturday June 29, when the share string would not load. Thanks again, Deniz On Thu, Jul 25, 2013 at 5:19 PM, Dannon Baker dannon.ba...@gmail.comwrote: On Thu, Jul 25, 2013 at 4:47 PM, Deniz Erezyilmaz deniz...@gmail.comwrote: The workaround doesn't seem to work Can you tell me what happens when you launch with the workaround mentioned previously ( https://main.g2.bx.psu.edu/cloudlaunch?ami=ami-da58aab3bucket_default=gxy-workshop)? This should launch the pre-migration AMI and Cloudman and shouldn't have any issues at all unless your cluster was already wonky. -Dannon ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] support pbkdf2 in proftpd 1.3.5rc3
Dear galaxy developers, We have tried today to upgrade our proftpd configuration to make uploading for our galaxy users possible again, both for users with old as well as new style hashed passwords. We upgraded proftpd on the server to 1.3.5rc3 and have the following SQL part in our configuration file based on the post of http://dev.list.galaxyproject.org/ProFTPD-integration-with-Galaxy-td4660295.html SQLEngine on SQLLogFile /var/log/proftpd-sql.log SQLBackend postgres SQLConnectInfo galaxy@localhost:5840 galaxyftp [ourpassword] SQLAuthTypesSHA1 SHA256 PBKDF2 SQLPasswordPBKDF2SHA256 1000 24 SQLPasswordUserSaltsql:/GetUserSalt SQLAuthenticate users SQLDefaultUID 108 SQLDefaultGID 116 SQLDefaultHomedir /opt/cloudman/pkg/proftpd/var SQLUserInfo custom:/LookupGalaxyUser SQLNamedQuery LookupGalaxyUser SELECT email, (CASE WHEN substring(password from 1 for 6) = 'PBKDF2' THEN substring(password from 38 for 32) ELSE password END) AS password2,'108','116','/mnt/galaxyData/tmp/ftp/%U','/bin/bash' FROM galaxy_user WHERE email='%U' SQLNamedQuery GetUserSalt SELECT (CASE WHEN SUBSTRING (password from 1 for 6) = 'PBKDF2' THEN SUBSTRING (password from 21 for 16) END) AS salt FROM galaxy_user WHERE email='%U' We have executed the LookupGalaxyUser and GetUserSalt commands manually, and the results look good. Now, old users can login via ftp, but for a new user, the authentication still fails: 2013-07-26 13:15:06,989 mod_sql/4.3[31761]: cmd_check 2013-07-26 13:15:06,989 mod_sql/4.3[31761]: checking password using SQLAuthType 'sha1' 2013-07-26 13:15:06,989 mod_sql/4.3[31761]: 'sha1' SQLAuthType handler reports failure 2013-07-26 13:15:06,989 mod_sql/4.3[31761]: checking password using SQLAuthType 'pbkdf2' 2013-07-26 13:15:06,993 mod_sql/4.3[31761]: 'pbkdf2' SQLAuthType handler reports failure What are we missing? Thanks! Rob and Leon -- Hailiang (Leon) Mei Netherlands Bioinformatics Center BioAssist NGS Taskforce - http://ngs.nbic.nlhttps://wiki.nbic.nl/index.php/Next_Generation_Sequencing Skype: leon_meiMobile: +31 6 41709231 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Installing Galaxy behind an Apache proxy using mod_auth_cas for user auth
I'm trying to install Galaxy behind an Apache proxy using mod_auth_cas
for user authentication. I've got a ways in but am now stuck with
galaxy not getting the REMOTE_USER. Has anyone deployed with this module?
I have use_remote_user = true on in my universe_wsgi.ini along with a
valid maildomain.
Here's my apache config:
NameVirtualHost galaxy.utah.edu:80
NameVirtualHost galaxy.utah.edu:443
CASLoginURL https://go.utah.edu/cas/login
CASValidateURL https://go.utah.edu/cas/serviceValidate
CASValidateServer Off
CASAllowWildcardCert On
CASCertificatePath /etc/pki/tls/certs/ca-bundle.trust.crt
CASCookiePath /var/run/mod_auth_cas/
CASTimeout 3600
CASIdleTimeout 1800
CASDebug On
VirtualHost 155.101.xx.40:80
RedirectPermanent / https://galaxy.utah.edu/
/VirtualHost
VirtualHost galaxy.utah.edu:443
SSLEngine on
SSLProtocol -ALL +SSLv3 +TLSv1
SSLCipherSuite HIGH:-aNULL:-eNULL
# Export the SSL environment variables to scripts
Files ~ \.(cgi|pl|shtml|phtml|php3?)$
SSLOptions +StdEnvVars
/Files
# Protocol adjustments for broken clients
SetEnvIf User-Agent .*MSIE.* \
nokeepalive ssl-unclean-shutdown \
downgrade-1.0 force-response-1.0
SSLCertificateFile /etc/pki/tls/certs/wildcard.utah.edu.crt
SSLCertificateKeyFile /etc/pki/tls/certs/wildcard.utah.edu.key
SSLCertificateChainFile /etc/pki/tls/certs/gd_bundle.crt
Proxy http://localhost:8080
Order deny,allow
Allow from all
/Proxy
RewriteEngine on
Location /
AuthType CAS
AuthName Galaxy
Require valid-user
#CASAuthNHeader REMOTE_SHMUSER
# Take the $REMOTE_USER environment variable and set it as a header
in the proxy request.
RewriteCond %{IS_SUBREQ} ^false$
RewriteCond %{LA-U:REMOTE_USER} (.+)
#RewriteCond %{LA-U:REMOTE_SHMUSER} (.+)
RewriteRule . - [E=RU:%1]
RequestHeader set REMOTE_USER %{RU}e
XSendFile on
XSendFilePath /
# Compress all uncompressed content.
SetOutputFilter DEFLATE
SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.(?:t?gz|zip|bz2)$ no-gzip dont-vary
SetEnvIfNoCase Request_URI /history/export_archive no-gzip dont-vary
RequestHeader set X-URL-SCHEME https
/Location
Location /static
# Allow browsers to cache everything from /static for 6 hours
ExpiresActive On
ExpiresDefault access plus 6 hours
/Location
ServerName galaxy.utah.edu
RewriteRule ^/static/style/(.*)
/uufs/utah.edu/sys/pkg/galaxy/std/static/june_2007_style/blue/$1 [L]
RewriteRule ^/static/scripts/(.*)
/uufs/utah.edu/sys/pkg/galaxy/std/static/scripts/packed/$1 [L]
RewriteRule ^/static/(.*) /uufs/utah.edu/sys/pkg/galaxy/std/static/$1 [L]
RewriteRule ^/favicon.ico
/uufs/utah.edu/sys/pkg/galaxy/std/static/favicon.ico [L]
RewriteRule ^/robots.txt
/uufs/utah.edu/sys/pkg/galaxy/std/static/robots.txt [L]
RewriteRule ^(.*) http://localhost:8080$1 [P]
/VirtualHost
Any help is appreciated.
Steve Harper
Systems Administrator
Center for High Performance Computing
University of Utah
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
