[galaxy-dev] "The status is being set to Never installed"
When I install a tool from the toolshed that has type="set_environment" as a requirement, I get this in the main.log: Creating a new record for version None of tool dependency SCRIPT_PATH for revision 5fed38c7e063 of repository mirdeep2. The status is being set to Never installed. Consequently, nothing in the tool_dependency directory is created. The tool is shown to have missing dependencies in the 'Manage installed tool shed repositories' panel. Is there something special that must be done to allow a tool to have only an environment setting as a dependency? Is there something that can be done to fix the problem, other than going into the database and changing the tool_dependency.status from 'Never installed' to 'Installed' and manually creating the subdirectories? David Hoover, PhD Helix Systems Staff ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] tool for STAR RNA-seq aligner
Hi, I am developing a tool for STAR (https://code.google.com/p/rna-star/), and I realize I may be reinventing another wheel. Has anyone else created a tool for STAR? There's nothing else in the toolsheds for it yet. David David Hoover, PhD Helix Systems Staff SCB/DCSS/CIT/NIH 301-435-2986 http://helix.nih.gov ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] tool for STAR RNA-seq aligner
Why didn't I see these before? Hmm, I thought I had searched both toolsheds... I was kind of hoping someone had tackled this in a different way. It would be nice if there was a composite datatype for the reference genome. It is important for users to generate their own personal genome references, rather than rely on shared, admin-installed indices. And you're correct, we'd need at least 5-10 separate genome references for each organism, depending on read length and annotation GTF. Back to wheel reinvention. BTW, can you tell me which standardly installed tools use composite datatypes? It's always easier to build thing from comparison, rather than from scratch. David -------- David Hoover, PhD Helix Systems Staff On Sep 24, 2014, at 4:14 PM, Björn Grüning wrote: > Hi David, > > yes there is inital code in the https://testtoolshed.g2.bx.psu.edu/. I think > Ross has done some work on it. > The main problem with Star is that is needs special indices (and a lot of it) > and it would be great to offer data managers for it. > > Cheers, > Bjoern > > Am 24.09.2014 um 22:05 schrieb David Hoover: >> Hi, >> >> I am developing a tool for STAR (https://code.google.com/p/rna-star/), and I >> realize I may be reinventing another wheel. Has anyone else created a tool >> for STAR? There's nothing else in the toolsheds for it yet. >> >> David >> >> >> David Hoover, PhD >> Helix Systems Staff >> SCB/DCSS/CIT/NIH >> 301-435-2986 >> http://helix.nih.gov >> >> >> >> >> >> ___ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ >> ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] tool for STAR RNA-seq aligner
Ross, About the index files: It is way easier to have pre-built index files. However, when running a 2-pass STAR run, a user will need to generate their own reference index files based on the output SJ.tab.out file created in the first pass. Is this incorporated into your tool? About shared memory: I am under the impression that the latest version of STAR has deprecated this feature. I am unclear how this would help unless a single large-memory machine was dedicated to running all STAR jobs. Is this the case? Also, does the tool merge the SAM/BAM file with the output chimeric SAM file? David Hoover On 9/24/2014 7:03 PM, Ross wrote: Hi All, That (fubar in testtoolshed) star wrapper was derived from one originally written by Jeremy Goecks. I modified it for multiple inputs and added a few tweaks and it has been in production use in our group for about 6 months so I'm pretty sure it works reasonably well in our hands at least. I would really appreciate any available help getting it to a proven useful state - suggestions and code welcomed. I have not moved it to the main toolshed because aside from some encouragement, I've had no feedback to suggest it's working - or not. It is extremely fast - we regularly see 200-300M reads per minute in the logs! We regularly run a whole experiment worth (eg 12 - 24) fastq files simultaneously with the shared memory option working on our cluster - see the readme. Star index files made with a gene model (requires valid gff3) are huge - 20-30GB for hg19 - hence the need for shared memory if you run multiple jobs. That will eventually become a serious problem if you really want to allow users to make their own - we definitely do not. You need to be very careful about matching the gene model gff3 file to the reference and I had enough trouble getting it right for the few major genomes we use to make me think that I do not want users trying to do that generating 25GB of rubbish every time they get it wrong. There are challenges to do with needing different indexes for different length reads but we are seeing fairly consistent 60bp single ended reads for most of the incoming RNA seq experiments. A data manager would be a boon if anyone cares to write one... On Thu, Sep 25, 2014 at 6:55 AM, Curtis Hendrickson (Campus) mailto:curt...@uab.edu>> wrote: Bjorn We'd be interested in this tool, as well. Any idea how close to functional it is? I see it's only on TEST toolshed, and not on production, at this point. I don't see any related Trello card when searching on "star" Regards, Curtis Galaxy Admin @ University of Alabama at Birmingham -Original Message- From: galaxy-dev-boun...@lists.bx.psu.edu <mailto:galaxy-dev-boun...@lists.bx.psu.edu> [mailto:galaxy-dev-boun...@lists.bx.psu.edu <mailto:galaxy-dev-boun...@lists.bx.psu.edu>] On Behalf Of Björn Grüning Sent: Wednesday, September 24, 2014 3:15 PM To: galaxy-dev@lists.bx.psu.edu <mailto:galaxy-dev@lists.bx.psu.edu>; hoove...@helix.nih.gov <mailto:hoove...@helix.nih.gov> >> David Hoover Subject: Re: [galaxy-dev] tool for STAR RNA-seq aligner Hi David, yes there is inital code in the https://testtoolshed.g2.bx.psu.edu/. I think Ross has done some work on it. The main problem with Star is that is needs special indices (and a lot of it) and it would be great to offer data managers for it. Cheers, Bjoern Am 24.09.2014 um 22:05 schrieb David Hoover: > Hi, > > I am developing a tool for STAR (https://code.google.com/p/rna-star/), and I realize I may be reinventing another wheel. Has anyone else created a tool for STAR? There's nothing else in the toolsheds for it yet. > > David > > > David Hoover, PhD > Helix Systems Staff > SCB/DCSS/CIT/NIH > 301-435-2986 > http://helix.nih.gov > > > > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this and other > Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] tool for STAR RNA-seq aligner
A colleague of mine mentioned it. I'll ask him where he got his info. Just to clarify: do you always run STAR jobs on the same host? We are running Galaxy in front of a batch system cluster, and so by default STAR jobs would run on different nodes. It's not clear to me how long the memory allocated would last after the batch job finished. How do you determine whether the memory remains allocated and whether the job has been accelerated due to pre-loaded data? For example, if you create a genome reference, using --genomeLoad=LoadAndKeep, then run an alignment, are subsequent alignments using the same genome reference much faster? If so, how much faster? I apologize, I am a jack of all trades, master of none. I could test this myself, but everything I touch related to genomics takes >50GB of memory and >18 hours clocktime, and it gets painful to try testing everything. David -------- David Hoover, PhD Helix Systems Staff SCB/DCSS/CIT/NIH 301-435-2986 http://helix.nih.gov On Sep 26, 2014, at 4:13 AM, Ross wrote: > Hi David. > > I've not needed that workflow so haven't a solution for you and no, it > doesn't do anything with chimeric output - won't be hard to add I suspect. > There's no python wrapper - just shell script in the command segment. > > It's not in an IUC main tool shed repository because it lacks a data manager > - manual star indexes are a bit of a pain but less pain than writing a data > manager :( so I haven't yet. Might be run best through the API. > > On shared memory: Pity. it works a treat for us. I didn't see anything on the > google group - do you recall where you learned about this deprecation ? > > > On Thu, Sep 25, 2014 at 10:41 PM, David Hoover wrote: > Ross, > > About the index files: It is way easier to have pre-built index files. > However, when running a 2-pass STAR run, a user will need to generate their > own reference index files based on the output SJ.tab.out file created in the > first pass. Is this incorporated into your tool? > > About shared memory: I am under the impression that the latest version of > STAR has deprecated this feature. I am unclear how this would help unless a > single large-memory machine was dedicated to running all STAR jobs. Is this > the case? > > Also, does the tool merge the SAM/BAM file with the output chimeric SAM file? > > David Hoover > > > On 9/24/2014 7:03 PM, Ross wrote: >> Hi All, >> >> That (fubar in testtoolshed) star wrapper was derived from one originally >> written by Jeremy Goecks. I modified it for multiple inputs and added a few >> tweaks and it has been in production use in our group for about 6 months so >> I'm pretty sure it works reasonably well in our hands at least. >> >> I would really appreciate any available help getting it to a proven useful >> state - suggestions and code welcomed. I have not moved it to the main >> toolshed because aside from some encouragement, I've had no feedback to >> suggest it's working - or not. It is extremely fast - we regularly see >> 200-300M reads per minute in the logs! >> >> We regularly run a whole experiment worth (eg 12 - 24) fastq files >> simultaneously with the shared memory option working on our cluster - see >> the readme. >> >> Star index files made with a gene model (requires valid gff3) are huge - >> 20-30GB for hg19 - hence the need for shared memory if you run multiple >> jobs. That will eventually become a serious problem if you really want to >> allow users to make their own - we definitely do not. You need to be very >> careful about matching the gene model gff3 file to the reference and I had >> enough trouble getting it right for the few major genomes we use to make me >> think that I do not want users trying to do that generating 25GB of rubbish >> every time they get it wrong. >> >> There are challenges to do with needing different indexes for different >> length reads but we are seeing fairly consistent 60bp single ended reads for >> most of the incoming RNA seq experiments. >> >> A data manager would be a boon if anyone cares to write one... >> >> >> On Thu, Sep 25, 2014 at 6:55 AM, Curtis Hendrickson (Campus) >> wrote: >> Bjorn >> >> We'd be interested in this tool, as well. Any idea how close to functional >> it is? >> I see it's only on TEST toolshed, and not on production, at this point. >> >> I don't see any related Trello card when searching on "star" >> >> Regards, >> Curtis >> Galaxy Admin @ U
Re: [galaxy-dev] tool for STAR RNA-seq aligner
OKhmm My colleague can't recall where he heard it. Maybe he made it up, or it was a misunderstanding about a recent patch that Alex (author of STAR) issued. Never mind... In any case, would we need to specify a single large memory node to utilize this feature for STAR? How long does the memory allocation last? David David Hoover, PhD Helix Systems Staff SCB/DCSS/CIT/NIH 301-435-2986 http://helix.nih.gov On Sep 26, 2014, at 9:21 AM, David Hoover wrote: > A colleague of mine mentioned it. I'll ask him where he got his info. Just > to clarify: do you always run STAR jobs on the same host? We are running > Galaxy in front of a batch system cluster, and so by default STAR jobs would > run on different nodes. It's not clear to me how long the memory allocated > would last after the batch job finished. How do you determine whether the > memory remains allocated and whether the job has been accelerated due to > pre-loaded data? For example, if you create a genome reference, using > --genomeLoad=LoadAndKeep, then run an alignment, are subsequent alignments > using the same genome reference much faster? If so, how much faster? > > I apologize, I am a jack of all trades, master of none. I could test this > myself, but everything I touch related to genomics takes >50GB of memory and > >18 hours clocktime, and it gets painful to try testing everything. > > David > > > David Hoover, PhD > Helix Systems Staff > SCB/DCSS/CIT/NIH > 301-435-2986 > http://helix.nih.gov > > > > > On Sep 26, 2014, at 4:13 AM, Ross wrote: > >> Hi David. >> >> I've not needed that workflow so haven't a solution for you and no, it >> doesn't do anything with chimeric output - won't be hard to add I suspect. >> There's no python wrapper - just shell script in the command segment. >> >> It's not in an IUC main tool shed repository because it lacks a data manager >> - manual star indexes are a bit of a pain but less pain than writing a data >> manager :( so I haven't yet. Might be run best through the API. >> >> On shared memory: Pity. it works a treat for us. I didn't see anything on >> the google group - do you recall where you learned about this deprecation ? >> >> >> On Thu, Sep 25, 2014 at 10:41 PM, David Hoover >> wrote: >> Ross, >> >> About the index files: It is way easier to have pre-built index files. >> However, when running a 2-pass STAR run, a user will need to generate their >> own reference index files based on the output SJ.tab.out file created in the >> first pass. Is this incorporated into your tool? >> >> About shared memory: I am under the impression that the latest version of >> STAR has deprecated this feature. I am unclear how this would help unless a >> single large-memory machine was dedicated to running all STAR jobs. Is this >> the case? >> >> Also, does the tool merge the SAM/BAM file with the output chimeric SAM file? >> >> David Hoover >> >> >> On 9/24/2014 7:03 PM, Ross wrote: >>> Hi All, >>> >>> That (fubar in testtoolshed) star wrapper was derived from one originally >>> written by Jeremy Goecks. I modified it for multiple inputs and added a few >>> tweaks and it has been in production use in our group for about 6 months so >>> I'm pretty sure it works reasonably well in our hands at least. >>> >>> I would really appreciate any available help getting it to a proven useful >>> state - suggestions and code welcomed. I have not moved it to the main >>> toolshed because aside from some encouragement, I've had no feedback to >>> suggest it's working - or not. It is extremely fast - we regularly see >>> 200-300M reads per minute in the logs! >>> >>> We regularly run a whole experiment worth (eg 12 - 24) fastq files >>> simultaneously with the shared memory option working on our cluster - see >>> the readme. >>> >>> Star index files made with a gene model (requires valid gff3) are huge - >>> 20-30GB for hg19 - hence the need for shared memory if you run multiple >>> jobs. That will eventually become a serious problem if you really want to >>> allow users to make their own - we definitely do not. You need to be very >>> careful about matching the gene model gff3 file to the reference and I had >>> enough trouble getting it right for the few major genomes we use to make me >>> think that I do not want users try
[galaxy-dev] datatype directory
Why isn't there a datatype for a directory of files? This seems like such a simple thing. If an executable generates or expects a directory as its input or output, why must a fancy complicated composite datatype be created to handle this? David Hoover David Hoover, PhD Helix Systems Staff SCB/DCSS/CIT/NIH 301-435-2986 http://helix.nih.gov ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] samtools mpileup update
Is there any good reason why Galaxy can't use a more recent version of samtools? The only difference I can see is the mpileup command. The algorithm by which mpileup finds SNPs is a little different. Does anyone know if the file format has changed? The only files that need to change are sam_pileup.py and sam_pileup.xml. David Hoover Helix Systems Staff http://helix.nih.gov ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] prevent downloaded files with .txt added
Does anyone know how to prevent the .txt suffix from being added to files downloaded from Galaxy? For example, a fasta file is downloaded as xxx.fasta.txt, instead of xxx.fasta. Wondering, David Hoover Helix Systems Staff http://helix.nih.gov ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] prevent downloaded files with .txt added
I apologize for responding to my own email, but I thought others might want to know. This turned out to be a Safari-dependent problem. Lots of other people have complained, but there doesn't seem to be any direct fix. My quick solution was to make sure the downloaded file is gzipped. This kills two birds with one stone -- the download is smaller, and Safari doesn't mess with the extension. Now, does anyone know if it is possible to gzip a dataset from a history within Galaxy prior to downloading? David On Sep 26, 2011, at 2:00 PM, David Hoover wrote: > Does anyone know how to prevent the .txt suffix from being added to files > downloaded from Galaxy? For example, a fasta file is downloaded as > xxx.fasta.txt, instead of xxx.fasta. > > Wondering, > David Hoover > Helix Systems Staff > http://helix.nih.gov > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] scatterplot broken
Hi, The scatterplot tool is broken in the distributed version. It works on the main Galaxy server, but on our local installation it gives this error: An error occurred running this job:Warning messages: 1: In min(x) : no non-missing arguments to min; returning Inf 2: In max(x) : no non-missing arguments to max; returning -Inf 3: In min(x) : no non-missing arguments to min; returning Inf 4: In max(x) : no non-missing arguments to max; re Any ideas? David Hoover Helix Systems Staff SCB/CIT/NIH ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] scatterplot broken
Again, solved my own problem. diff -r 949e4f5fa03a tools/plotting/scatterplot.py --- a/tools/plotting/scatterplot.py Mon Aug 29 14:42:04 2011 -0400 +++ b/tools/plotting/scatterplot.py Wed Oct 12 10:35:58 2011 -0400 @@ -20,7 +20,8 @@ xlab = sys.argv[6] ylab = sys.argv[7] -matrix = [] +xarray = [] +yarray = [] skipped_lines = 0 first_invalid_line = 0 invalid_value = '' @@ -57,12 +58,13 @@ first_invalid_line = i+1 if valid: -matrix.append( row ) +xarray.append ( row[0] ) +yarray.append ( row[1] ) if skipped_lines < i: try: r.pdf( out_fname, 8, 8 ) -r.plot( array( matrix ), type="p", main=title, xlab=xlab, ylab=ylab, col="blue", pch=19 ) +r.plot( xarray, yarray, type="p", main=title, xlab=xlab, ylab=ylab, col="blue", pch=19 ) r.dev_off() except Exception, exc: stop_err( "%s" %str( exc ) ) David On Oct 6, 2011, at 10:16 AM, David Hoover wrote: > Hi, > > The scatterplot tool is broken in the distributed version. It works on the > main Galaxy server, but on our local installation it gives this error: > > An error occurred running this job:Warning messages: > 1: In min(x) : no non-missing arguments to min; returning Inf > 2: In max(x) : no non-missing arguments to max; returning -Inf > 3: In min(x) : no non-missing arguments to min; returning Inf > 4: In max(x) : no non-missing arguments to max; re > > Any ideas? > > David Hoover > Helix Systems Staff > SCB/CIT/NIH > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] cufflinks version
What version of cufflinks do the current Galaxy wrappers support? The wiki (http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Dependencies) says 1.0.1, but our local instance only works with 1.1.0. Can the wiki be updated? David Hoover Helix Systems Staff http://helix.nih.gov ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] LDAP authentification
Why do you need to create a proxy? The way I have ours set up is within a
virtual host:
AuthName "Helix Systems"
AuthType Basic
AuthBasicProvider ldap
AuthzLDAPAuthoritative off
AuthLDAPUrl ldaps://helixdrive.nih.gov/ou=Users,ou=helix.nih.gov,o=scb?uid
AuthLDAPBindDN "cn=maccount,ou=Special,o=scb"
AuthLDAPBindPassword
Require valid-user
RequestHeader set REMOTE_USER %{AUTHENTICATE_uid}e
Options None
Order Allow,Deny
Allow from all
RewriteRule ^/static/style/(.*)
/data/galaxy/pro/static/june_2007_style/blue/$1 [L]
RewriteRule ^/static/scripts/(.*) /data/galaxy/pro/static/scripts/packed/$1
[L]
RewriteRule ^/static/(.*) /data/galaxy/pro/static/$1 [L]
RewriteRule ^/favicon.ico /data/galaxy/pro/static/favicon.ico [L]
RewriteRule ^/robots.txt /data/galaxy/pro/static/robots.txt [L]
RewriteRule ^(.*) http://helixweb4.cit.nih.gov:8080$1 [P]
David
On Jan 19, 2012, at 10:49 AM, Sarah Maman wrote:
> Hi Brad, Hi Christopher,
>
> Thanks a lot.
> I'm talking to Open LDAP. I have changed
>
> RequestHeader set REMOTE_USER %{AUTHENTICATE_uid}e
> to
> RequestHeader set REMOTE_USER %{AUTHENTICATE_UID}e
>
> But I always have the same message /(Access to Galaxy is denied
> Galaxy is configured to authenticate users via an external method (such as
> HTTP authentication in Apache), but a username was not provided by the
> upstream (proxy) server. This is generally due to a misconfiguration in the
> upstream server. )/
>
> This is my .conf file :
> http://localhost:8080>
> Order deny,allow
> Allow from all
>
>
> RewriteEngine on
>
>
> AuthType Basic
> AuthName Galaxy
> AuthBasicProvider ldap
> AuthLDAPURL
> "ldap://server/ou=People,ou=genopole,ou=toulouse,o=inra,c=fr?u
> id?sub?(objectClass=person)"
> AuthzLDAPAuthoritative off
> Require valid-user
> # Take the $REMOTE_USER environment variable and set it as a
> header in the proxy request.
> #RewriteCond %{IS_SUBREQ} ^false$
> #RewriteCond %{LA-U:REMOTE_USER} (.+)
> #RewriteRule . - [E=RU:%1]
> #RequestHeader set REMOTE_USER %{RU}e
> RequestHeader set REMOTE_USER %{AUTHENTICATE_UID}e
>
>
>
> Thanks in advance,
> Sarah
>
>
>
>
> Langhorst, Brad a écrit :
>> Hi Sarah:
>>
>> I don't know what kind of LDAP you're talking to... I'm talking to Active
>> Directory and this configuration works in that situation.
>>
>> Order allow,deny
>>allow from all
>>
>>AuthType Basic
>>AuthName "NEB Credentials"
>>AuthBasicProvider ldap
>>AuthzLDAPAuthoritative off
>>AuthLDAPBindDN ccaloo...@neb.com
>>AuthLDAPBindPassword
>>AuthLDAPURL
>> "ldap://:389/dc=domain,dc=com?sAMAccountName"
>>require valid-user
>>RewriteCond %{IS_SUBREQ} ^false$
>>RewriteCond %{LA-U:REMOTE_USER} (.+)
>>RewriteRule . - [E=RU:%1]
>>RequestHeader set REMOTE_USER
>> %{AUTHENTICATE_sAMAccountName}e
>>
>>
>>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client. To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>
> http://lists.bx.psu.edu/
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] Unix user account/connection and Galaxy connection
I'd love to know answers to this question as well. So far, all I've done is create two tools for copying files between Galaxy and a user's personal directories using a rather dangerous setuid executable. I have mulled over two possibilities for this problem. One is to run Galaxy as root. The other is for each user to kick off their own personal instance of Galaxy as themselves, but accessing the same main database and file repository. The latter would require all files be read-write accesible to all users. Neither is good. On Jan 30, 2012, at 4:29 AM, Sarah Maman wrote: > Dear all, > > There is currently no link between an user ssh connection on his own account > (own space on Unix) and the Galaxy connection. > How to run Galaxy on an Unix user account? How to link the data storage > system and the Unix user account (symlink?)? > > The goal is to not need to copy or move data. > > > PS : Thanks a lot to David, Gordon, Brad, Christophe and Nate for your help > on LDAP authentification. > I always try to find a solution (Apache configuration). I will inform Galaxy > list when I will find a solution. > > > Thanks in advance, > > Sarah ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] limited tool shed browse
When I click on Admin->'Search and browse tool sheds'->'Galaxy main tool shed', it shows the first page of Valid repositories. However, clicking '2', '3', '4', or 'Show All' has no effect. I can only display the first page of repositories. However, if I search for valid tools using the menu under 'Galaxy main tool shed', I can get to any of the repositories listed at the Galaxy main toolshed. Is this a bug or a feature? David Hoover Helix Systems Staff ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] limited tool shed browse
It's a local install, hg summary gives 6621:26920e20157f. I'm using MySQL as a backend database and have the following options set to non-default: port = 8081 host = 0.0.0.0 tool_config_file = tool_conf.xml,shed_tool_conf.xml install_tool_config_file = shed_tool_conf.xml brand = Galaxy1 use_interactive = False enable_whoosh_library_search = True use_remote_user = True allow_user_deletion = True allow_user_impersonation = True allow_user_dataset_purge = True new_user_dataset_access_role_default_private = True enable_quotas = True retry_job_output_collection = 10 output_size_limit = 10737418240 start_job_runners = drmaa On Feb 22, 2012, at 4:07 PM, Greg Von Kuster wrote: > Hi David, > > I cannot reproduce this behavior from my local Galaxy instance, but clicking > '2', '3', '4' or 'Show All" does take a second or more to display the > response. Can you provide more information about the environment you have or > what you are doing to reproduce this? > > Thanks > > Greg Von Kuster > > On Feb 22, 2012, at 4:02 PM, David Hoover wrote: > >> When I click on Admin->'Search and browse tool sheds'->'Galaxy main tool >> shed', it shows the first page of Valid repositories. However, clicking >> '2', '3', '4', or 'Show All' has no effect. I can only display the first >> page of repositories. >> >> However, if I search for valid tools using the menu under 'Galaxy main tool >> shed', I can get to any of the repositories listed at the Galaxy main >> toolshed. >> >> Is this a bug or a feature? >> >> David Hoover >> Helix Systems Staff >> ___ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> >> http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] limited tool shed browse
My apologies. It takes about 10 seconds or more to load the other pages. There is no indication in my browser (Safari) that anything is happening, so I guess the moral is that I am too impatient. David On Feb 22, 2012, at 4:07 PM, Greg Von Kuster wrote: > Hi David, > > I cannot reproduce this behavior from my local Galaxy instance, but clicking > '2', '3', '4' or 'Show All" does take a second or more to display the > response. Can you provide more information about the environment you have or > what you are doing to reproduce this? > > Thanks > > Greg Von Kuster > > On Feb 22, 2012, at 4:02 PM, David Hoover wrote: > >> When I click on Admin->'Search and browse tool sheds'->'Galaxy main tool >> shed', it shows the first page of Valid repositories. However, clicking >> '2', '3', '4', or 'Show All' has no effect. I can only display the first >> page of repositories. >> >> However, if I search for valid tools using the menu under 'Galaxy main tool >> shed', I can get to any of the repositories listed at the Galaxy main >> toolshed. >> >> Is this a bug or a feature? >> >> David Hoover >> Helix Systems Staff >> ___ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> >> http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Data source tool doesn't source data
Mark, Do you have 'use_interactive' enabled (set to 'True') in universe_wsgi.ini? Also, have you clicked on the the bug icon? This shows the STDOUT and STDERR of the tool, if it was not redirected. David Hoover Helix Systems, CIT, NIH ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Error for draw quality score boxplot
You first need to find where on your Galaxy server the fonts are for gnuplot. An example is /usr/local/share/fonts/msttcorefonts. Once you know the directory, you can set an environment variable for your Galaxy user, like this: export GDFONTPATH=/usr/local/share/fonts/msttcorefonts Place this in the .bashrc file for the Galaxy user then restart your server. David Hoover Helix Systems Staff, NIH On Mar 27, 2012, at 4:31 PM, Xu, Jianpeng wrote: > Hi, > > I have installed the local galaxy and tried to run draw quality score > boxplot. > > It showed the error message: > Could not find/open font when opening font "arial", using internal > non-scalable font > > Could you please tell me how to fix it ? > > Thanks, > > Jianpeng > > > This e-mail message (including any attachments) is for the sole use of > the intended recipient(s) and may contain confidential and privileged > information. If the reader of this message is not the intended > recipient, you are hereby notified that any dissemination, distribution > or copying of this message (including any attachments) is strictly > prohibited. > > If you have received this message in error, please contact > the sender by reply e-mail message and destroy all copies of the > original message (including attachments). > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Interested in speaking with other institutions deploying Galaxy locally?
Not a bad idea. Most of the conversations on this list are very limited and detailed in scope, dealing with the minutia of tools and variables. There isn't much on the bird's eye view. I'm in. On Apr 27, 2012, at 2:45 PM, Ann Black-Ziegelbein wrote: > Hi everyone - > > Here at the University of Iowa we are working on deploying Galaxy locally for > campus wide access. I am interested in forming a community of other > institutions trying to deploy Galaxy locally and mange/operate it on a broad > level. Is anyone else? If there is enough interest, possibly we could have > a community conference call every other month to have an open discussion on > how we are all deploying galaxy, customizations we are making, problems we > are encountering, bugs, and any add-on operations management for galaxy being > developed, etc. > > Would love to hear from others operating Galaxy or in process of standing up > a local deployment. > > Thanks! > > Ann Black-Ziegelbein > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Idea for user-based dataset subdirectories
How about if there were a completely separate daemon that monitored the galaxy database periodically to determine what datasets belong to which user(s). Then it would move the actual dataset to an area owned by the user and group accessible to galaxy, replacing the dataset with a symlink. This would require no changes to the galaxy build, but it would require a constant monitoring system. There is already a mechanism for users to move their files into a joint user/galaxy directory, but it is (as far as I know) only allowed for libraries, not histories. It would be better if there were a way for users to browse through their own directories as a tool, and be able to load files directly into their history. David Hoover On May 15, 2012, at 7:40 PM, Josh Nielsen wrote: > Hello, > > Please forgive the length of this proposition as I try to explain my > reasoning behind this. Let me say first of all that I understand that Galaxy > is not meant to be everything to everyone and that requests for features may > not suit everyone who uses Galaxy. That being said I have an idea or request > that I think would be convenient for dealing with user's datasets from a > file-system perspective. > > Galaxy has the obvious benefit and advantage (compared to manual > job-submission for tools on a cluster) of providing an interface for using > all the analysis tools, and the history of the operations done on your data, > all in one place. However I have found that putting all the output & datasets > in one directory (the files/000/ directory) on the file-system causes a > problem for the users if they specifically want to interact with it *on the > file-system*, and not just through the Web interface - for whatever > complicated or diverse reasons. > > Since Galaxy runs on a cluster of its own in our environment, and we do not > allow users to remote connect into it to submit manual jobs (and individually > output it to their separate home directories) like we do our main cluster, it > is essentially a black box beyond the GUI interface of Galaxy. That is > essentially what we want except for how they can interact with the output > files. > > The issue is that our users would like an easy means of copying their files > off of the Galaxy cluster to other servers from a command line (possibly even > automated by scripts). Even if we allow an FTP share of the output directory > for users to do that, the common [galaxy-dist]/database/files/000/ directory > clumps all of the files for all users together in one directory and uses a > sequential file-naming scheme (dataset_N++) that is not easy to discriminate > between as to who the owner is for each file. > > Is there a way that the dataset output directory locations could be designed > (or set optionally?) like the FTP upload feature's expected directory > structure: where the files are dropped into the corresponding subdirectory of > the user who produced it? For example having under database/files/ > subdirectories named according to the user's Galaxy account id (like > [galaxy-dist]/database/files/jsmith, [galaxy-dist]/database/files/sparker, > etc.). If they could be segregated by user it would be much easier to keep > track of what datasets belong to whom on the file-system. Then I could > possibly set up a read-only FTP share to the files/ directory on the cluster, > from which the users could directly copy the files in their personal > subdirectory to other systems, and perhaps batch download them, rather than > having to rely solely on the Web interface. > > I understand that the way Galaxy is currently designed is that the files are > just generically named (the "behind-the-scenes" handling of data is a black > box) and it is the database that keeps track of which files belong to whom, > and which has the metadata for more meaningful dataset/job names, etc. But a > file-system hierarchy alternative would also be welcome in a heavily > command-line oriented computational environment too. > > Would setting up a more user-representative output directory hierarchy on the > file-system like that be possible? > > Best Regards, > Josh Nielsen > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Idea for user-based dataset subdirectories
No, this was all an idea I've had for a while, but never did anything about it. I'm pretty sure the Galaxy developers are not interested in anything this locally-centric, and I don't blame them. It ought to be something outside the Galaxy build completely, because Galaxy is meant to be system-independent. What I meant by 'joint user/galaxy directory' is a directory that is owned by a user, but that the galaxy user has read (and possibly write) access to. This is entirely possible given either a well-informed user population, or an iron-clad suexec executable. The mechanism I alluded to is a feature by which a user can upload a directory of files all at once. There is a configuration directive in universe_wsgi.ini, user_library_import_dir, that allows non-administrative users to upload an entire directory of files into a library. The directive identifies the base directory, within which subdirectories named as the galaxy user login (email address) are searched. The user_library_import_dir directory is owned by the galaxy user, and the subdirectories are owned by the user, but group owned by the galaxy user. A user will copy files to the subdirectory, login to galaxy, switch to their library, and upload all the files in the directory into a single library folder. There isn't much documentation about it in the main Galaxy wiki, so forget that. I haven't enabled it in our local production site, and I haven't played with it in a long time. I'm pretty sure that the files are not removed after uploading, and a user is free to re-upload the files again and again, so it's kind of quirky. Also, if the files are not readable by the galaxy user, a bizarre and unhelpful error is thrown. If this functionality could be extended and elaborated, it could do what you want. The user_library_import_dir requires that the user's login in Galaxy must be identical to the the user's login on the cluster, and that the permissions be kept correct. Typically users have no idea what is going on with their permissions, so what are you going to do? David On May 16, 2012, at 1:33 PM, Josh Nielsen wrote: > Hi David, > > Actually that is an interesting idea to use a daemon to move the files into > associated user directories. Is that something that Galaxy Dev is working/can > work on, or was that just a suggestion? I'm not opposed to doing any dev work > of my own, but I don't really know Python that well and I know most of the > Galaxy code is Python. > > I'm not sure that I follow what you are talking about with the joint > user/galaxy directory though. I'm of course wanting it to not be unified (not > all in the same directory) and rather be segregated by user into user > subdirectories, but I think you already caught that so I guess I just didn't > understand what you were getting at. > > Josh Nielsen > > -- > >How about if there were a completely separate daemon that monitored the > >galaxy database periodically to determine what datasets belong to which > >user(s). Then > >it would move the actual dataset to an area owned by the user and group > >accessible to galaxy, replacing the dataset with a symlink. This would > >require no changes to the galaxy build, but it would require a constant > >monitoring system. > > >There is already a mechanism for users to move their files into a joint > >user/galaxy directory, but it is (as far as I know) only allowed for > >libraries, not > >histories. It would be better if there were a way for users to browse > >through their own directories as a tool, and be able to load files directly > >into their history. > > > >David Hoover > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] migrate ownership of galaxy user
I have a somewhat unusual problem. I have a local instance of Galaxy wherein users are created automatically upon authentication (use_remote_user = True). However, for a handful of users, their remote_user names have changed. After that change, a new Galaxy user and role were created, and they can't access their old data. Worse, they continued using the new account and so now have two sets of (possibly) identical data. Is there a simple way of migrating datasets, jobs, quotas, etc., from one user to another? I've figured out a way to make the necessary changes at the MySQL database level. However, there is one sticky problem. None of the tables (aside from role and galaxy_user) define the role_id or the user_id as unique, so I would assume I can simply UPDATE the table to set the old role_id or user_id to the new role_id or user_id. Unfortunately, it appears that some of the tables are in practice keeping the role_id or user_id as unique. For example, default_user_permissions or user_role_association. Could a single user have multiple default permissions or multiple roles? What would happen if this were the case? In short, can anyone say which tables have practical unique role_id/user_id to data associations? Of course, I could just delete the old user and say too bad... Thanks, David Hoover Helix Systems Staff, CIT/NIH ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] configuring job runner for toolshed tools
Is there a way to configure a tool downloaded from the toolshed depot to use a job runner other than the local runner? The tool_id for the toolshed tool isn't honored in universe_wsgi.ini the way the default tools are. Specifically, has anyone configured a toolshed tool to use drmaa instead of localhost? David Hoover Helix Systems Staff ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] configuring job runner for toolshed tools
OK, forget what I said earlier. I was watching for a different toolshed tool than what I reconfigured. Never mind, using the guid (the long tool_id shown as an example below) worked correctly. The job was pushed out to the cluster and finished normally. Sorry about the fuss. David On Jul 21, 2012, at 8:40 AM, Greg Von Kuster wrote: > Hi David, > > If using the actual tool id (guid) does not work, then it's likely that the > job runner code needs to be fixed to correctly handle tools installed from a > tool shed. I've created a ticket to get this resolved which you can follow > so that you'll receive status changes on the issue. > > https://bitbucket.org/galaxy/galaxy-central/issue/779/fix-job-runner-to-handle-tools-installed > > Thanks for reporting this, and sorry for the inconvenience. > > Greg Von Kuster > > On Jul 21, 2012, at 8:29 AM, wrote: > >> I found the guid for the tool and tried the usual method of configuring >> galaxy to use it via drmaa. With your example, it would be >> >> toolshed.g2.bx.psu.edu/repos/devteam/freebayes/freebayes/0.0.2 = >> drmaa://-l nodes=1/ >> >> But this didn't have any effect. It still ran on the localhost. I guess >> no one has tried this. >> >> I haven't tried using the old_id. Would that work? >> >> I'd rather not run everything via drmaa, but if the toolshed tools can't >> be configured, I might have to. >> >> David >> >>> Hi David, >>> >>> I'm not familiar enough with the job runners to know what the problem may >>> be, However, for tools installed from the tool shed, the tool id is the >>> tool-shed generated "guid" instead of the "id" attribute value of the tool >>> config's tag. So, for example, if you install the freebayes tool >>> shed repository from the main Galaxy tool shed, your tool panel will >>> display the single freebayes tool, which is the only tool contained in the >>> freebayes repository. >>> >>> When this installed tool is loaded into Galaxy, the tools "id" attribute >>> value is: >>> >>> toolshed.g2.bx.psu.edu/repos/devteam/freebayes/freebayes/0.0.2 >>> >>> and the tool's "old_id" attribute value is the value os the "id" attribute >>> in it's tool config tag: >>> >>> freebayes >>> >>> Have your tried configuring the installed tool using the new id (guid) >>> rather than the old id? I'm not sure if this works, but just wanted to >>> make sure you were using the correct id value. If it still doesn't work, >>> let us know and we'll get a fix asap. >>> >>> Thanks! >>> >>> Greg Von Kuster >>> >>> On Jul 20, 2012, at 5:16 PM, David Hoover wrote: >>> >>>> Is there a way to configure a tool downloaded from the toolshed depot to >>>> use a job runner other than the local runner? The tool_id for the >>>> toolshed tool isn't honored in universe_wsgi.ini the way the default >>>> tools are. Specifically, has anyone configured a toolshed tool to use >>>> drmaa instead of localhost? >>>> >>>> David Hoover >>>> Helix Systems Staff >>>> ___ >>>> Please keep all replies on the list by using "reply all" >>>> in your mail client. To manage your subscriptions to this >>>> and other Galaxy lists, please use the interface at: >>>> >>>> http://lists.bx.psu.edu/ >>> >>> >> >> ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] configuring job runner for toolshed tools
Back on the topic again, I found another real issue with toolshed tools. I can configure the toolshed tool to be used by drmaa in universe_wsgi.ini, then restart Galaxy and everything's good. However, if I modify the tool .xml file and reload it using the Admin link, it goes back to the default job runner. Can this be changed so that the configured job runner remains the same after reloading the toolshed tool? On Jul 23, 2012, at 5:02 PM, David Hoover wrote: > OK, forget what I said earlier. I was watching for a different toolshed tool > than what I reconfigured. Never mind, using the guid (the long tool_id shown > as an example below) worked correctly. The job was pushed out to the cluster > and finished normally. > > Sorry about the fuss. > > David > > On Jul 21, 2012, at 8:40 AM, Greg Von Kuster wrote: > >> Hi David, >> >> If using the actual tool id (guid) does not work, then it's likely that the >> job runner code needs to be fixed to correctly handle tools installed from a >> tool shed. I've created a ticket to get this resolved which you can follow >> so that you'll receive status changes on the issue. >> >> https://bitbucket.org/galaxy/galaxy-central/issue/779/fix-job-runner-to-handle-tools-installed >> >> Thanks for reporting this, and sorry for the inconvenience. >> >> Greg Von Kuster >> >> On Jul 21, 2012, at 8:29 AM, wrote: >> >>> I found the guid for the tool and tried the usual method of configuring >>> galaxy to use it via drmaa. With your example, it would be >>> >>> toolshed.g2.bx.psu.edu/repos/devteam/freebayes/freebayes/0.0.2 = >>> drmaa://-l nodes=1/ >>> >>> But this didn't have any effect. It still ran on the localhost. I guess >>> no one has tried this. >>> >>> I haven't tried using the old_id. Would that work? >>> >>> I'd rather not run everything via drmaa, but if the toolshed tools can't >>> be configured, I might have to. >>> >>> David >>> >>>> Hi David, >>>> >>>> I'm not familiar enough with the job runners to know what the problem may >>>> be, However, for tools installed from the tool shed, the tool id is the >>>> tool-shed generated "guid" instead of the "id" attribute value of the tool >>>> config's tag. So, for example, if you install the freebayes tool >>>> shed repository from the main Galaxy tool shed, your tool panel will >>>> display the single freebayes tool, which is the only tool contained in the >>>> freebayes repository. >>>> >>>> When this installed tool is loaded into Galaxy, the tools "id" attribute >>>> value is: >>>> >>>> toolshed.g2.bx.psu.edu/repos/devteam/freebayes/freebayes/0.0.2 >>>> >>>> and the tool's "old_id" attribute value is the value os the "id" attribute >>>> in it's tool config tag: >>>> >>>> freebayes >>>> >>>> Have your tried configuring the installed tool using the new id (guid) >>>> rather than the old id? I'm not sure if this works, but just wanted to >>>> make sure you were using the correct id value. If it still doesn't work, >>>> let us know and we'll get a fix asap. >>>> >>>> Thanks! >>>> >>>> Greg Von Kuster >>>> >>>> On Jul 20, 2012, at 5:16 PM, David Hoover wrote: >>>> >>>>> Is there a way to configure a tool downloaded from the toolshed depot to >>>>> use a job runner other than the local runner? The tool_id for the >>>>> toolshed tool isn't honored in universe_wsgi.ini the way the default >>>>> tools are. Specifically, has anyone configured a toolshed tool to use >>>>> drmaa instead of localhost? >>>>> >>>>> David Hoover >>>>> Helix Systems Staff >>>>> ___ >>>>> Please keep all replies on the list by using "reply all" >>>>> in your mail client. To manage your subscriptions to this >>>>> and other Galaxy lists, please use the interface at: >>>>> >>>>> http://lists.bx.psu.edu/ >>>> >>>> >>> >>> > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] annoying bug when deleting datasets
I recently switched from galaxy-dist to galaxy-central as the source of my local install (version 7427:cb9272041d95). When I delete a dataset from a history, all the delete icons disappear from the datasets. They come back if I refresh the page. Has anyone else noticed this? David Hoover Helix Systems Staff ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] qa_url = slyfox?
Strangely, I had never noticed this before. On some tools, there is a link at the top of the tool page that attempts to connect to http://slyfox.bx.psu.edu:8080/, which it can't. This URL is set via the qa_url directive in the universe_wsgi.ini file. What is this URL, and what should it be? David Hoover Helix Systems Staff ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] browse directories on galaxy server
Is there any desire to create a way in galaxy to browse local directories on the galaxy server itself? Not the client machine running the web browwser, but the machine on which galaxy is running. I've created a tool that allows a user to upload a single file from their own personal directory on the galaxy server into galaxy directly, without going through Apache. However, this is very limiting. It would be very helpful to be able to browse directories that are deemed accessible to the galaxy user, and before I begin trying to develop anything I'd like to know if there is any support for such a feature. David Hoover Helix Systems Staff National Institutes of Health ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] FTP bug with automatically decompressed files
When a file is uploaded via FTP, and the file is compressed using gzip, the file is uploaded and decompressed on the fly; however, the gzipped file is left on the FTP server, rather than being deleted as normal. Is this a bug? David Hoover Helix Systems Staff ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Library permission page broken
After the latest update, the "Manage library permissions" page is broken. The "Roles not associated" boxes are pushed to the left of the frame, and the double arrow buttons are misplaced down the page. This occurs under Safari (5.1.10) and Chrome (31.0.1650.57) on a Mac running OS X 10.6.8. Firefox (25.0.1), however, seems ok. ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] error with unicode output
I have installed the ngsplot galaxy tool from http://code.google.com/p/ngsplot. This tool creates a set of three pdf files. In older versions of Galaxy, the tool ran correctly with no problems. A recent update broke the tool. The job runs but is unable to finish. Here is the error reported: Traceback (most recent call last): File "/spin1/users/galaxy/galaxy/lib/galaxy/jobs/runners/local.py", line 116, in queue_job job_wrapper.finish( stdout, stderr, exit_code ) File "/spin1/users/galaxy/galaxy/lib/galaxy/jobs/__init__.py", line 1015, in finish self.sa_session.flush() File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/scoping.py", line 114, in do return getattr(self.registry(), name)(*args, **kwargs) File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line 1718, in flush self._flush(objects) File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line 1789, in _flush flush_context.execute() File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/unitofwork.py", line 331, in execute rec.execute(self) File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/unitofwork.py", line 475, in execute uow File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/persistence.py", line 59, in save_obj mapper, table, update) File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/persistence.py", line 485, in _emit_update_statements execute(statement, params) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1449, in execute params) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1584, in _execute_clauseelement compiled_sql, distilled_params File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1691, in _execute_context context) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/default.py", line 331, in do_execute cursor.execute(statement, parameters) File "build/bdist.linux-x86_64/egg/MySQLdb/cursors.py", line 158, in execute query = query % db.literal(args) File "build/bdist.linux-x86_64/egg/MySQLdb/connections.py", line 265, in literal return self.escape(o, self.encoders) File "build/bdist.linux-x86_64/egg/MySQLdb/connections.py", line 203, in unicode_literal return db.literal(u.encode(unicode_literal.charset)) UnicodeEncodeError: 'latin-1' codec can't encode character u'\ufffd' in position 11: ordinal not in range(256) There is a set of files created in the job_working_directory that start with 'metadata_', some of which contain the unicode. Is there anything I can do to fix this? David Hoover Helix Systems Staff ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] error with unicode output
My version is 11216:c458a0fe1ba8. Does this cover the commit? On Dec 5, 2013, at 12:32 PM, John Chilton wrote: > Actually, can you verify that this commit > https://bitbucket.org/galaxy/galaxy-central/commits/e786022dc67ed918050bd81b9ac679ac958e4f75 > is in your distribution and if it is try changing: > > DEFAULT_ENCODING = 'utf-8' > > in lib/galaxy/util.py to > > DEFAULT_ENCODING = 'latin-1' > > If that works then - I can create a database_encoding_default option > in universe_wsgi.ini and let you switch it to latin-1 instead of > needing to patch Galaxy. Otherwise, setting the MySQL tables to be > UTF-8 is probably the better approach - though again - backup and test > before applying that change in production. > > Hope this helps, > -John > > > On Thu, Dec 5, 2013 at 11:17 AM, John Chilton wrote: >> David, Christian, >> >> Very sorry about this - this is probably related to fixing some other >> errors - >> http://dev.list.galaxyproject.org/Unicode-in-tool-stderr-crashing-galaxy-tt4661749.html#a4661750. >> I will try to look into this. >> >> Christian - what database are targeting? Is it MySQL as well? >> >> David - do you have a test setup you can hack on? I wonder if this >> would go away if you converted your tables to UTF-8. >> >> http://stackoverflow.com/questions/6115612/how-to-convert-an-entire-mysql-database-characterset-and-collation-to-utf-8 >> >> That is not my official recommendation though - I need to do some more >> research first. >> >> -John >> >> On Thu, Dec 5, 2013 at 11:04 AM, Christian Hundsrucker >> wrote: >>> Hi David, hi all! >>> >>> I have a similar/the same issue in another setting... >>> >>> galaxy/galaxy_dist/lib/galaxy/jobs/runners/local.py", line 116, in queue_job >>>job_wrapper.finish( stdout, stderr, exit_code ) >>> [...] >>> >>> galaxy/galaxy_dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/persistence.py", >>> line 485, in _emit_update_statements >>> [...] >>> >>> UnicodeEncodeError: 'latin-1' codec can't encode character u'\u2018' in >>> position 134: ordinal not in range(256) >>> >>> >>> I am integrating a set of R/Bioconductor modules into our local Galaxy >>> instance. >>> To do so, I use the discard_stderr_wrapper.sh. >>> It worked fine until the recent update* >>> As the error appears upon any R-output (via print, cat or error channel), I >>> just set the option "-v" for the cat command in the >>> discard_stderr_wrapper.sh: >>> >>> cat $TMPFILE >&2 >>> => >>> cat -v $TMPFILE >&2 >>> >>> >>> as a temporary workaround. >>> No idea if this is applicable in your case?! >>> >>> Cheers, >>> Christian >>> >>> * >>> changeset: 11219:5c789ab4144a >>> branch: stable >>> tag: tip >>> >>> >>> >>> >>> On 05.12.2013 17:29, David Hoover wrote: >>> >>> I have installed the ngsplot galaxy tool from >>> http://code.google.com/p/ngsplot. This tool creates a set of three pdf >>> files. In older versions of Galaxy, the tool ran correctly with no >>> problems. A recent update broke the tool. The job runs but is unable to >>> finish. Here is the error reported: >>> >>> Traceback (most recent call last): >>> File "/spin1/users/galaxy/galaxy/lib/galaxy/jobs/runners/local.py", line >>> 116, in queue_job >>>job_wrapper.finish( stdout, stderr, exit_code ) >>> File "/spin1/users/galaxy/galaxy/lib/galaxy/jobs/__init__.py", line 1015, >>> in finish >>>self.sa_session.flush() >>> File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/scoping.py", line 114, >>> in do >>>return getattr(self.registry(), name)(*args, **kwargs) >>> File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line 1718, >>> in flush >>>self._flush(objects) >>> File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line 1789, >>> in _flush >>>flush_context.execute() >>> File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/unitofwork.py", line >>> 331, in execute >>>rec.execute(self) >>> File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/unitofwork.p
Re: [galaxy-dev] error with unicode output
Right, nevermind, 'hg log' listed that changeset 10953:e786022dc67e. Changing DEFAULT_ENCODING to 'latin-1' in lib/galaxy/util/__init__.py worked. Do I need to alter ALL the MySQL tables to UTF-8, or just a selection of tables? Will future updates explicitly create new tables with CHARSET=utf-8, or do I need to reconfigure MySQL to have a new default? -- David On Dec 5, 2013, at 12:32 PM, John Chilton wrote: > Actually, can you verify that this commit > https://bitbucket.org/galaxy/galaxy-central/commits/e786022dc67ed918050bd81b9ac679ac958e4f75 > is in your distribution and if it is try changing: > > DEFAULT_ENCODING = 'utf-8' > > in lib/galaxy/util.py to > > DEFAULT_ENCODING = 'latin-1' > > If that works then - I can create a database_encoding_default option > in universe_wsgi.ini and let you switch it to latin-1 instead of > needing to patch Galaxy. Otherwise, setting the MySQL tables to be > UTF-8 is probably the better approach - though again - backup and test > before applying that change in production. > > Hope this helps, > -John > > > On Thu, Dec 5, 2013 at 11:17 AM, John Chilton wrote: >> David, Christian, >> >> Very sorry about this - this is probably related to fixing some other >> errors - >> http://dev.list.galaxyproject.org/Unicode-in-tool-stderr-crashing-galaxy-tt4661749.html#a4661750. >> I will try to look into this. >> >> Christian - what database are targeting? Is it MySQL as well? >> >> David - do you have a test setup you can hack on? I wonder if this >> would go away if you converted your tables to UTF-8. >> >> http://stackoverflow.com/questions/6115612/how-to-convert-an-entire-mysql-database-characterset-and-collation-to-utf-8 >> >> That is not my official recommendation though - I need to do some more >> research first. >> >> -John >> >> On Thu, Dec 5, 2013 at 11:04 AM, Christian Hundsrucker >> wrote: >>> Hi David, hi all! >>> >>> I have a similar/the same issue in another setting... >>> >>> galaxy/galaxy_dist/lib/galaxy/jobs/runners/local.py", line 116, in queue_job >>>job_wrapper.finish( stdout, stderr, exit_code ) >>> [...] >>> >>> galaxy/galaxy_dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/persistence.py", >>> line 485, in _emit_update_statements >>> [...] >>> >>> UnicodeEncodeError: 'latin-1' codec can't encode character u'\u2018' in >>> position 134: ordinal not in range(256) >>> >>> >>> I am integrating a set of R/Bioconductor modules into our local Galaxy >>> instance. >>> To do so, I use the discard_stderr_wrapper.sh. >>> It worked fine until the recent update* >>> As the error appears upon any R-output (via print, cat or error channel), I >>> just set the option "-v" for the cat command in the >>> discard_stderr_wrapper.sh: >>> >>> cat $TMPFILE >&2 >>> => >>> cat -v $TMPFILE >&2 >>> >>> >>> as a temporary workaround. >>> No idea if this is applicable in your case?! >>> >>> Cheers, >>> Christian >>> >>> * >>> changeset: 11219:5c789ab4144a >>> branch: stable >>> tag: tip >>> >>> >>> >>> >>> On 05.12.2013 17:29, David Hoover wrote: >>> >>> I have installed the ngsplot galaxy tool from >>> http://code.google.com/p/ngsplot. This tool creates a set of three pdf >>> files. In older versions of Galaxy, the tool ran correctly with no >>> problems. A recent update broke the tool. The job runs but is unable to >>> finish. Here is the error reported: >>> >>> Traceback (most recent call last): >>> File "/spin1/users/galaxy/galaxy/lib/galaxy/jobs/runners/local.py", line >>> 116, in queue_job >>>job_wrapper.finish( stdout, stderr, exit_code ) >>> File "/spin1/users/galaxy/galaxy/lib/galaxy/jobs/__init__.py", line 1015, >>> in finish >>>self.sa_session.flush() >>> File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/scoping.py", line 114, >>> in do >>>return getattr(self.registry(), name)(*args, **kwargs) >>> File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line 1718, >>> in flush >>>self._flush(objects) >>> File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.
Re: [galaxy-dev] error with unicode output
John, I stopped galaxy, then ran ALTER DATABASE galaxydb DEFAULT CHARACTER SET = 'utf8', then ran ALTER TABLE `[table]` DEFAULT CHARACTER SET = 'utf8' on all the tables in galaxydb. After starting up galaxy and rerunning the jobs (using the unaltered version of lib/galaxy/util/__init__.py), the job failed with the same error. Can I configure the sqlalchemy connection to use utf8? Or must I reconfigure the entire server to use utf8? --David On Dec 5, 2013, at 1:32 PM, John Chilton wrote: > Fantastic! > > For this particular problem - I guess you don't strictly need to > modify more than just job and maybe task tables. I suspect at some > point there will be a non-latin-1 job parameter or history name or > username, etc... that will result in a similar problem though - so if > you could just make it all UTF-8 that would probably be ideal. > > -John > > > On Thu, Dec 5, 2013 at 12:01 PM, David Hoover wrote: >> Right, nevermind, 'hg log' listed that changeset 10953:e786022dc67e. >> >> Changing DEFAULT_ENCODING to 'latin-1' in lib/galaxy/util/__init__.py worked. >> >> Do I need to alter ALL the MySQL tables to UTF-8, or just a selection of >> tables? Will future updates explicitly create new tables with >> CHARSET=utf-8, or do I need to reconfigure MySQL to have a new default? >> >> -- David >> >> On Dec 5, 2013, at 12:32 PM, John Chilton wrote: >> >>> Actually, can you verify that this commit >>> https://bitbucket.org/galaxy/galaxy-central/commits/e786022dc67ed918050bd81b9ac679ac958e4f75 >>> is in your distribution and if it is try changing: >>> >>> DEFAULT_ENCODING = 'utf-8' >>> >>> in lib/galaxy/util.py to >>> >>> DEFAULT_ENCODING = 'latin-1' >>> >>> If that works then - I can create a database_encoding_default option >>> in universe_wsgi.ini and let you switch it to latin-1 instead of >>> needing to patch Galaxy. Otherwise, setting the MySQL tables to be >>> UTF-8 is probably the better approach - though again - backup and test >>> before applying that change in production. >>> >>> Hope this helps, >>> -John >>> >>> >>> On Thu, Dec 5, 2013 at 11:17 AM, John Chilton wrote: >>>> David, Christian, >>>> >>>> Very sorry about this - this is probably related to fixing some other >>>> errors - >>>> http://dev.list.galaxyproject.org/Unicode-in-tool-stderr-crashing-galaxy-tt4661749.html#a4661750. >>>> I will try to look into this. >>>> >>>> Christian - what database are targeting? Is it MySQL as well? >>>> >>>> David - do you have a test setup you can hack on? I wonder if this >>>> would go away if you converted your tables to UTF-8. >>>> >>>> http://stackoverflow.com/questions/6115612/how-to-convert-an-entire-mysql-database-characterset-and-collation-to-utf-8 >>>> >>>> That is not my official recommendation though - I need to do some more >>>> research first. >>>> >>>> -John >>>> >>>> On Thu, Dec 5, 2013 at 11:04 AM, Christian Hundsrucker >>>> wrote: >>>>> Hi David, hi all! >>>>> >>>>> I have a similar/the same issue in another setting... >>>>> >>>>> galaxy/galaxy_dist/lib/galaxy/jobs/runners/local.py", line 116, in >>>>> queue_job >>>>> job_wrapper.finish( stdout, stderr, exit_code ) >>>>> [...] >>>>> >>>>> galaxy/galaxy_dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/persistence.py", >>>>> line 485, in _emit_update_statements >>>>> [...] >>>>> >>>>> UnicodeEncodeError: 'latin-1' codec can't encode character u'\u2018' in >>>>> position 134: ordinal not in range(256) >>>>> >>>>> >>>>> I am integrating a set of R/Bioconductor modules into our local Galaxy >>>>> instance. >>>>> To do so, I use the discard_stderr_wrapper.sh. >>>>> It worked fine until the recent update* >>>>> As the error appears upon any R-output (via print, cat or error channel), >>>>> I >>>>> just set the option "-v" for the cat command in the >>>>> discard_stderr_wrapper.sh: >>>>> >>>>> cat $TMPFILE >&2 >>>>&g
Re: [galaxy-dev] error with unicode output
Adding "?charset=utf8" to the connection string worked! However, there is one really weird side effect that probably has nothing to do with utf8. The tool creates two pdf files and one zip file as output. For the two pdf files, the expansion of the dataset in the history bar shows 'Image in pdf format'. The zip file shows gobbledygook. How do I tell Galaxy to recognize zip format and not try to parse/head it? Thanks John! On Dec 5, 2013, at 5:03 PM, John Chilton wrote: > Hmmm... can you try adding "?charset=utf8" to your database connection > string - that may fix the problem? > > If not - is there a way to tell if the actual columns have changed. > Some comments on stackoverflow make it sound like the commands you > listed will only affect new columns. > > Can you try the CONVERT TO CHARACTER SET. > > ALTER TABLE tbl_name CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci; > > I don't think the problem is sqlalchemy right - this works for > postgres and sqlite I believe - it is either that MySQL cannot store > UTF-8 data in that column or there is a problem in the mysql > connector. It is not clear to me where the problem is based on your > stack trace and explanation. I would be happy to work around a > limitation in the mysql connector by adding a config option to Galaxy > if I were certain that there was a bug in the mysql connector. > > -John > > On Thu, Dec 5, 2013 at 12:55 PM, David Hoover wrote: >> John, >> >> I stopped galaxy, then ran ALTER DATABASE galaxydb DEFAULT CHARACTER SET = >> 'utf8', then ran ALTER TABLE `[table]` DEFAULT CHARACTER SET = 'utf8' on all >> the tables in galaxydb. After starting up galaxy and rerunning the jobs >> (using the unaltered version of lib/galaxy/util/__init__.py), the job failed >> with the same error. >> >> Can I configure the sqlalchemy connection to use utf8? Or must I >> reconfigure the entire server to use utf8? >> >> --David >> >> On Dec 5, 2013, at 1:32 PM, John Chilton wrote: >> >>> Fantastic! >>> >>> For this particular problem - I guess you don't strictly need to >>> modify more than just job and maybe task tables. I suspect at some >>> point there will be a non-latin-1 job parameter or history name or >>> username, etc... that will result in a similar problem though - so if >>> you could just make it all UTF-8 that would probably be ideal. >>> >>> -John >>> >>> >>> On Thu, Dec 5, 2013 at 12:01 PM, David Hoover >>> wrote: >>>> Right, nevermind, 'hg log' listed that changeset 10953:e786022dc67e. >>>> >>>> Changing DEFAULT_ENCODING to 'latin-1' in lib/galaxy/util/__init__.py >>>> worked. >>>> >>>> Do I need to alter ALL the MySQL tables to UTF-8, or just a selection of >>>> tables? Will future updates explicitly create new tables with >>>> CHARSET=utf-8, or do I need to reconfigure MySQL to have a new default? >>>> >>>> -- David >>>> >>>> On Dec 5, 2013, at 12:32 PM, John Chilton wrote: >>>> >>>>> Actually, can you verify that this commit >>>>> https://bitbucket.org/galaxy/galaxy-central/commits/e786022dc67ed918050bd81b9ac679ac958e4f75 >>>>> is in your distribution and if it is try changing: >>>>> >>>>> DEFAULT_ENCODING = 'utf-8' >>>>> >>>>> in lib/galaxy/util.py to >>>>> >>>>> DEFAULT_ENCODING = 'latin-1' >>>>> >>>>> If that works then - I can create a database_encoding_default option >>>>> in universe_wsgi.ini and let you switch it to latin-1 instead of >>>>> needing to patch Galaxy. Otherwise, setting the MySQL tables to be >>>>> UTF-8 is probably the better approach - though again - backup and test >>>>> before applying that change in production. >>>>> >>>>> Hope this helps, >>>>> -John >>>>> >>>>> >>>>> On Thu, Dec 5, 2013 at 11:17 AM, John Chilton wrote: >>>>>> David, Christian, >>>>>> >>>>>> Very sorry about this - this is probably related to fixing some other >>>>>> errors - >>>>>> http://dev.list.galaxyproject.org/Unicode-in-tool-stderr-crashing-galaxy-tt4661749.html#a4661750. >>>>>> I will try to look into this. >>>>>>
[galaxy-dev] 'Lost connection to MySQL server during query' error
My recent Galaxy update has broken the feature that solves the 'Lost connection to MySQL server during query' error. I've set the variable database_engine_option_pool_recycle = 14400 to fix this. It used to work, but now periodically loading the page gives a gray warning page, which after reloading the URL goes to Galaxy. Is there something different that we need to do now? David Hoover Helix Systems Staff ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] numpy and scipy
Are numpy and scipy included with the standard python modules/eggs supplied by galaxy? David Hoover Helix Systems Staff ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] restarting Galaxy without affecting jobs
What are the configuration steps required for allowing a local Galaxy installation to be restarted without affecting currently running jobs? I have Galaxy using DRMAA to submit jobs onto a backend cluster. I thought that enable_job_recovery = True should allow this, but in a few tests I have found that although the batch jobs completed, Galaxy lost track of the jobs and classified them as failed. Would track_jobs_in_database = True be required? This is currently set to the default 'None'. Our local Galaxy installation has become quite busy, and restarts are not possible without forcing users to restart their jobs. David Hoover Helix Systems Staff ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] jobs stuck in new state
I have many jobs stuck in the 'new' state on our local Galaxy instance. The jobs can't be stopped using the Admin->Manage jobs tool. First, does anyone know why a job would get stuck in the 'new' state for weeks? I have cleaned things up by manually setting their states to 'error' in the MySQL database. Is there a better way of dealing with 'new' jobs? BTW, our Galaxy instance was updated about two weeks ago. Wondering, David Hoover Helix Systems Staff ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] jobs stuck in new state
This turned out to be my own ignorance. After creating identical handlers in both universe_wsgi.ini and job_conf.xml, and restarting Galaxy in daemon mode, the jobs became resilient to Galaxy restarts. DOH! Thanks Nate for pointing this out, as well as the limits collection. David On Mar 31, 2014, at 12:20 PM, Ido Tamir wrote: > In my case it was uncompleted metadata in one of the input files. > (but maybe it was not "new" state but something else?) > > HTH, > ido > > On Mar 26, 2014, at 5:25 PM, David Hoover wrote: > >> I have many jobs stuck in the 'new' state on our local Galaxy instance. The >> jobs can't be stopped using the Admin->Manage jobs tool. First, does anyone >> know why a job would get stuck in the 'new' state for weeks? I have cleaned >> things up by manually setting their states to 'error' in the MySQL database. >> Is there a better way of dealing with 'new' jobs? >> >> BTW, our Galaxy instance was updated about two weeks ago. >> >> Wondering, >> David Hoover >> Helix Systems Staff >> ___ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] problem with bioc_qvalue, Version: 1.34.0
I saw the same phenomenon. I think it stems from the current version of compute_q_values pointing to unavailable revisions of both R and the bioc_qvalue R package. Whoever maintains the compute_q_values needs to straighten out the revision ids. David Hoover Helix Systems Staff National Institutes of Health On Apr 17, 2014, at 1:38 PM, Tony Kusalik wrote: > Hi, > > I am having a problem with updating our local Galaxy server, and I am hoping > someone can help me. > > Yesterday I performed a 'hg incoming' command. A bunch of updates were > applied, from changeset 12443:ec9d31a8bc04 > to changeset 13068:c05752549163. I restarted the server (in daemon mode), > but it halted. > The content of paster.log instructed me to run > sh ./scripts/migrate_tools/0010_tools.sh > I did that. That script encountered an error: > The following error occurred from the InstallManager while installing tool > dependency bioc_qvalue : > Error installing tool dependency package bioc_qvalue version 1.34.0: Unable > to locate required tool shed repository named package_bioc_qvalue_1_34_0 > owned by devteam with revision 11735242a19e. > > How do I get around this problem? > > I checked the mailing lists for anyone reporting a problem with "bioc_qvalue" > or "11735242a19e" > but nothing was turned up. > > Tony Kusalik > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Globus tools for galaxy
Are there any tools available for transferring files using Globus? There’s nothing in the toolsheds, but there was a lot of chatter a few years ago. David Hoover Helix Systems Staff ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] display_at_callback
What is the display_at_callback configuration setting, and how does it work? Can it be used as a workaround for remotely authenticated instances of galaxy? David Hoover Helix Systems Staff NIH ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] fastQC report bug
Hi, I have a very recent version of Galaxy installed locally, and when I run FastQC:Read QC, it creates an HTML file with missing links to the plots and images within the HTML. The HTML source looks identical to the HTML source from the main Galaxy server, so there must be something wrong with my local install. Is there some option or modification that I need to enable to allow access to the generated HTML/PNG files in the database/files/000/database_###_files directory? David Hoover Helix Systems Staff ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] GATK v2.x
Is anyone working on adapting the GATK tools for a more recent version of GATK? The distributed version is 1.4, and GATK is up to 2.1. David Hoover Helix Systems Staff ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] environment variables and paths for toolshed tools
Are there any environment variables that are honored by toolshed installed
tools? I tried creating a tool that uses ${GALAXY_DATA_INDEX_DIR} or
$GALAXY_HOME, then uploaded it the test toolshed, then installed it
automatically. Neither of these resolved to what I expected. I don't want to
hard-code the path in the xml tool file, but rather have a default location for
other executables and jar files. How should this best be done?
David Hoover
Helix Systems Staff
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
[galaxy-dev] toolshed upload problem
I've created a repository in the test toolshed and tried uploading gzipped tarballs with all files required. I have several tool config files (.xml) wrapping a single python script with some tool-data dependencies. If I upload a single tool config file with the dependencies, everything is ok. However, if I try uploading more than one config file at a time, it complains with the error message for all but one of the config files: *.xml - This file requires an entry for "'gatk2_picard_indexes'" in the tool_data_table_conf.xml file. Upload a file named tool_data_table_conf.xml.sample to the repository that includes the required entry to correct this error. The tool_data_table_conf.xml.sample file already exists within the tarball. Nothing changes if I upload the file separately. Any ideas? David Hoover Helix Systems Staff ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] where are the tools?
I just recently updated to the latest version of galaxy-central, and when I started galaxy, the tools were not listed on the left panel. I could rerun old jobs from my history, however. Why are the tools not listed? David Hoover Helix Systems Staff ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] submitting jobs as the real user
I'm playing with the "submitting jobs as the real user" method, and immediately ran into a problem. Uploaded files are created in mode 600, owned by the user. Galaxy then gives an error that the tmp file is not readable: IOError: [Errno 13] Permission denied: '/spin1/users/galaxy/galaxy2/database/tmp/tmp3KHIDt' Is there any way of changing the permissions on the temporary uploaded file to 666? David Hoover Helix Systems Staff ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] submitting jobs as the real user
No, that's not it. The galaxy user, as well as all other users, have umask 022 by default. I believe the problem is with Apache uploading. I can't figure out how to reconfigure Apache to set the permissions they way I want them. David On Oct 2, 2012, at 1:17 PM, Oleksandr Moskalenko wrote: > Hi David, > > I wonder if the user's umask is the problem. > > I set the galaxy user's umask to 022 and our default umask on the cluster is > 0022 for all users. > > On Oct 2, 2012, at 12:55 PM, David Hoover wrote: > >> I'm playing with the "submitting jobs as the real user" method, and >> immediately ran into a problem. Uploaded files are created in mode 600, >> owned by the user. Galaxy then gives an error that the tmp file is not >> readable: >> >> IOError: [Errno 13] Permission denied: >> '/spin1/users/galaxy/galaxy2/database/tmp/tmp3KHIDt' >> >> Is there any way of changing the permissions on the temporary uploaded file >> to 666? >> >> David Hoover >> Helix Systems Staff >> ___ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> >> http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Fwd: submitting jobs as the real user
Found a fix. The external_chown_script.py script forcibly changes the
permissions on the file to be world-readable.
diff external_chown_script.py external_chown_script.py.ORIG
33d32
< os.system('chmod -R a+r %s' %(path))
Please let me know if this is a bad change or if there is a better change.
David
Begin forwarded message:
> From: David Hoover
> Date: October 2, 2012 1:21:02 PM EDT
> To: Oleksandr Moskalenko
> Cc: galaxy-dev@lists.bx.psu.edu
> Subject: Re: [galaxy-dev] submitting jobs as the real user
>
> No, that's not it. The galaxy user, as well as all other users, have umask
> 022 by default.
>
> I believe the problem is with Apache uploading. I can't figure out how to
> reconfigure Apache to set the permissions they way I want them.
>
> David
>
> On Oct 2, 2012, at 1:17 PM, Oleksandr Moskalenko wrote:
>
>> Hi David,
>>
>> I wonder if the user's umask is the problem.
>>
>> I set the galaxy user's umask to 022 and our default umask on the cluster is
>> 0022 for all users.
>>
>> On Oct 2, 2012, at 12:55 PM, David Hoover wrote:
>>
>>> I'm playing with the "submitting jobs as the real user" method, and
>>> immediately ran into a problem. Uploaded files are created in mode 600,
>>> owned by the user. Galaxy then gives an error that the tmp file is not
>>> readable:
>>>
>>> IOError: [Errno 13] Permission denied:
>>> '/spin1/users/galaxy/galaxy2/database/tmp/tmp3KHIDt'
>>>
>>> Is there any way of changing the permissions on the temporary uploaded file
>>> to 666?
>>>
>>> David Hoover
>>> Helix Systems Staff
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client. To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>
>>> http://lists.bx.psu.edu/
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client. To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>
> http://lists.bx.psu.edu/
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
[galaxy-dev] jobName lost when drmaa used with real user use
When I use the real user method for running drmaa jobs as the real user, the jobName value gets lost. It's kind of annoying, but not a show stopper. The other values (outputPath, errorPath, remoteCommand, nativeSpecification, etc.) are all there, but not jobName. This doesn't happen when the drmaa jobs are run using the default Galaxy user. Weird. David Hoover Helix Systems Staff ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] submitting jobs as the real user
> This script is used multiple times, so you might want to make sure that this > change won't cause files to become readable that you would prefer remain > unreadable. >From the log output, it looks like files are chmod'd to the user, then later >back to the Galaxy user. Is there somewhere else that isn't logged? > Since you mention Apache uploading, what are you using for Apache to handle > the uploads? I don't know, whatever the default mechanism. I'm not using nginx. David ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] submitting jobs as the real user
While I've got you on the line, do you know why enabling the real user method blanks out the jobName for the PBS job? The jobName is set if the jobs are submitted as the Galaxy user. David On Oct 3, 2012, at 11:03 AM, Nate Coraor wrote: > On Oct 3, 2012, at 10:06 AM, David Hoover wrote: > >> >>> This script is used multiple times, so you might want to make sure that >>> this change won't cause files to become readable that you would prefer >>> remain unreadable. >> >> From the log output, it looks like files are chmod'd to the user, then later >> back to the Galaxy user. Is there somewhere else that isn't logged? > > Hi David, > > It chowns the entire working directory, I double checked and I don't see > anywhere else it runs. But after looking at the code again, I'm surprised > this fix works since it shouldn't touch anything in new_file_path. > >> >>> Since you mention Apache uploading, what are you using for Apache to handle >>> the uploads? >> >> I don't know, whatever the default mechanism. I'm not using nginx. > > This would just upload straight through to Galaxy without any special > handling by Apache. Tempfiles are created by Galaxy's internal webserver via > the FieldStorage monkeypatch in lib/galaxy/web/framework/base.py > > --nate > >> >> David ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] trying to set up local toolshed
I'm trying to host a local toolshed, and I've hit a brick wall. It is accessible as a subdirectory under our galaxy installation, uses SQLite as the database, and I've configured it as much as I understand. After I start it up, I can see the main page in the browser, but get this error message: Not Found The resource could not be found. No route for /repository/browse_categories Clicking on 'Repositories' gives this error: This link may not be followed from within Galaxy. Beyond this, can the toolshed use externally authenticated users? David Hoover Helix Systems Staff ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Fwd: fastQC report bug
How should I configure my local galaxy to allow access to sub-files in /datasets/[hashcode]/display/? FastQC is still broken on the latest build. This issue was reported by others as well. David Hoover Helix Systems Staff, CIT/NIH Begin forwarded message: > From: David Hoover > Date: September 10, 2012 4:48:30 PM EDT > To: > Subject: [galaxy-dev] fastQC report bug > > Hi, > > I have a very recent version of Galaxy installed locally, and when I run > FastQC:Read QC, it creates an HTML file with missing links to the plots and > images within the HTML. The HTML source looks identical to the HTML source > from the main Galaxy server, so there must be something wrong with my local > install. > > Is there some option or modification that I need to enable to allow access to > the generated HTML/PNG files in the database/files/000/database_###_files > directory? > > David Hoover > Helix Systems Staff > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] picard bug in Add or Replace Groups
In the Add or Replace Groups tool for picard, if the read group platform unit is left blank, the read group sample name is lost as well. Must all the read group tags be set with this tool? Can some of them remain blank? David Hoover Helix Systems Staff, CIT/NIH ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] determination of errors
How does Galaxy determine that a job has failed? Does it simply see if the STDERR is empty? What happens if an application normally outputs to STDERR? This is a problem for our local installation, as I have enabled it to run as the local user on the backend cluster. If a user has an error in the .bashrc file, it will automatically write to STDERR, and all jobs, no matter what, are labelled as failing. David Hoover Helix Systems Staff ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] no module named gmap
Hi,
I'm trying to load an instance of Galaxy from scratch, and I'm getting
this error message after configuration and typing "./run.sh":
Traceback (most recent call last):
File "./scripts/paster.py", line 33, in
serve.run()
File
"/spin1/users/galaxy/galaxy1/lib/galaxy/util/pastescript/serve.py", line
1056, in run
invoke(command, command_name, options, args[1:])
File
"/spin1/users/galaxy/galaxy1/lib/galaxy/util/pastescript/serve.py", line
1062, in invoke
exit_code = runner.run(args)
File
"/spin1/users/galaxy/galaxy1/lib/galaxy/util/pastescript/serve.py", line
227, in run
result = self.command()
File
"/spin1/users/galaxy/galaxy1/lib/galaxy/util/pastescript/serve.py", line
650, in command
app = loadapp( app_spec, name=app_name, relative_to=base,
global_conf=vars)
File
"/spin1/users/galaxy/galaxy1/lib/galaxy/util/pastescript/loadwsgi.py",
line 350, in loadapp
return loadobj(APP, uri, name=name, **kw)
File
"/spin1/users/galaxy/galaxy1/lib/galaxy/util/pastescript/loadwsgi.py",
line 374, in loadobj
global_conf=global_conf)
File
"/spin1/users/galaxy/galaxy1/lib/galaxy/util/pastescript/loadwsgi.py",
line 399, in loadcontext
global_conf=global_conf)
File
"/spin1/users/galaxy/galaxy1/lib/galaxy/util/pastescript/loadwsgi.py",
line 423, in _loadconfig
return loader.get_context(object_type, name, global_conf)
File
"/spin1/users/galaxy/galaxy1/lib/galaxy/util/pastescript/loadwsgi.py",
line 561, in get_context
section)
File
"/spin1/users/galaxy/galaxy1/lib/galaxy/util/pastescript/loadwsgi.py",
line 620, in _context_from_explicit
value = import_string(found_expr)
File
"/spin1/users/galaxy/galaxy1/lib/galaxy/util/pastescript/loadwsgi.py",
line 125, in import_string
return pkg_resources.EntryPoint.parse("x=" + s).load(False)
File "/spin1/users/galaxy/galaxy1/lib/pkg_resources.py", line 1954,
in load
entry = __import__(self.module_name, globals(),globals(), ['__name__'])
File "/spin1/users/galaxy/galaxy1/lib/galaxy/web/buildapp.py", line
3, in
from galaxy.webapps.galaxy.buildapp import app_factory
File
"/spin1/users/galaxy/galaxy1/lib/galaxy/webapps/galaxy/buildapp.py",
line 18, in
from galaxy import config, jobs, util, tools
File "/spin1/users/galaxy/galaxy1/lib/galaxy/jobs/__init__.py", line
15, in
from galaxy import util, model
File "/spin1/users/galaxy/galaxy1/lib/galaxy/model/__init__.py", line
13, in
import galaxy.datatypes.registry
File "/spin1/users/galaxy/galaxy1/lib/galaxy/datatypes/registry.py",
line 9, in
import gmap # added for gmap tools
ImportError: No module named gmap
What is missing?
David Hoover
Helix Systems Staff
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] no module named gmap
This is a fresh clone, from galaxy-dist, but I have a pile of patch
files to keep my local hacks straight. I forgot about the registry.py
file. Removed that, as well as the blast import. It's hard to keep
local hacks and regular updates in sync...
Thanks,
David
On 3/6/2013 12:16 PM, Dannon Baker wrote:
Is this a fresh clone from galaxy-dist or have you pulled it from
somewhere else? It looks like your galaxy clone's
lib/galaxy/datatypes/registry.py has been altered to have an extra
import, 'gmap', for gmap tools.
-Dannon
On Wed, Mar 6, 2013 at 12:08 PM, David Hoover <mailto:hoove...@helix.nih.gov>> wrote:
Hi,
I'm trying to load an instance of Galaxy from scratch, and I'm
getting this error message after configuration and typing "./run.sh":
Traceback (most recent call last):
File "./scripts/paster.py", line 33, in
serve.run()
File
"/spin1/users/galaxy/galaxy1/lib/galaxy/util/pastescript/serve.py", line
1056, in run
invoke(command, command_name, options, args[1:])
File
"/spin1/users/galaxy/galaxy1/lib/galaxy/util/pastescript/serve.py", line
1062, in invoke
exit_code = runner.run(args)
File
"/spin1/users/galaxy/galaxy1/lib/galaxy/util/pastescript/serve.py", line
227, in run
result = self.command()
File
"/spin1/users/galaxy/galaxy1/lib/galaxy/util/pastescript/serve.py", line
650, in command
app = loadapp( app_spec, name=app_name, relative_to=base,
global_conf=vars)
File
"/spin1/users/galaxy/galaxy1/lib/galaxy/util/pastescript/loadwsgi.py",
line 350, in loadapp
return loadobj(APP, uri, name=name, **kw)
File
"/spin1/users/galaxy/galaxy1/lib/galaxy/util/pastescript/loadwsgi.py",
line 374, in loadobj
global_conf=global_conf)
File
"/spin1/users/galaxy/galaxy1/lib/galaxy/util/pastescript/loadwsgi.py",
line 399, in loadcontext
global_conf=global_conf)
File
"/spin1/users/galaxy/galaxy1/lib/galaxy/util/pastescript/loadwsgi.py",
line 423, in _loadconfig
return loader.get_context(object_type, name, global_conf)
File
"/spin1/users/galaxy/galaxy1/lib/galaxy/util/pastescript/loadwsgi.py",
line 561, in get_context
section)
File
"/spin1/users/galaxy/galaxy1/lib/galaxy/util/pastescript/loadwsgi.py",
line 620, in _context_from_explicit
value = import_string(found_expr)
File
"/spin1/users/galaxy/galaxy1/lib/galaxy/util/pastescript/loadwsgi.py",
line 125, in import_string
return pkg_resources.EntryPoint.parse("x=" + s).load(False)
File "/spin1/users/galaxy/galaxy1/lib/pkg_resources.py", line
1954, in load
entry = __import__(self.module_name, globals(),globals(),
['__name__'])
File "/spin1/users/galaxy/galaxy1/lib/galaxy/web/buildapp.py",
line 3, in
from galaxy.webapps.galaxy.buildapp import app_factory
File
"/spin1/users/galaxy/galaxy1/lib/galaxy/webapps/galaxy/buildapp.py",
line 18, in
from galaxy import config, jobs, util, tools
File "/spin1/users/galaxy/galaxy1/lib/galaxy/jobs/__init__.py",
line 15, in
from galaxy import util, model
File "/spin1/users/galaxy/galaxy1/lib/galaxy/model/__init__.py",
line 13, in
import galaxy.datatypes.registry
File
"/spin1/users/galaxy/galaxy1/lib/galaxy/datatypes/registry.py",
line 9, in
import gmap # added for gmap tools
ImportError: No module named gmap
What is missing?
David Hoover
Helix Systems Staff
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
[galaxy-dev] admin menu column won't scroll
I've installed the latest version of galaxy, and the admin column won't scroll. I can't see any of the toolshed functions unless mybrowser is expanded beyond the size of my desktop display. Is there something wrong? David Hoover Helix Systems Staff ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] remote user environment variable?
I'm trying to set up a mechanism to copy a local file using a setuid executable, based on whether the registered user has read access to the local file in question. Is there any means for a tool to know the registered user who is running it? David Hoover Helix Systems Staff http://helix.nih.gov ___ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
[galaxy-dev] phastCons and phastOdds scores
Does anyone have a clear explanation of what files are required/available for Get Genomic Scores -> Aggregate datapoints, Compute phastOdds (binned_scores.loc and phastOdds.loc files)? I know there are .mod and .pp files and .wigFix files from UCSC, but I can't figure out exactly what Galaxy is looking for. Do I need to install PHAST and generate these score files? David Hoover Helix Systems Staff http://helix.nih.gov ___ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
[galaxy-dev] histogram tool not working
I think I have everything installed correctly for the histogram tool. However, when I try to generate a histogram with a simple tabular file of numbers, I get the following error message: An error occurred running this job: array() argument 1 must be char, not list Any ideas? David Hoover Helix Systems Staff http://helix.nih.gov___ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
[galaxy-dev] quality_scores.loc files
Where do the files in quality_scores.loc come from? Can they be downloaded from UCSC or somewhere else? David Hoover Helix Systems Staff http://helix.nih.gov ___ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
Re: [galaxy-dev] histogram tool not working
I just updated to the most recent version of Galaxy (hg pull -u), and now the error is different: An error occurred running this job: Error in hist.default(list(8, 6, 14, 8, 10, 3, 8, 6, 3, 12, 12, 8, 8, : 'x' must be numeric What gives? David On Feb 1, 2011, at 12:28 PM, David Hoover wrote: > I think I have everything installed correctly for the histogram tool. > However, when I try to generate a histogram with a simple tabular file of > numbers, I get the following error message: > > An error occurred running this job: array() argument 1 must be char, not list > > Any ideas? > > David Hoover > Helix Systems Staff > http://helix.nih.gov > ___ > galaxy-dev mailing list > galaxy-dev@lists.bx.psu.edu > http://lists.bx.psu.edu/listinfo/galaxy-dev ___ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
Re: [galaxy-dev] quality_scores.loc files
I've successfully installed bx-python, and there are mountain of utility scripts. Which ones do you use for converting the quals.fa file to binned array format? David On Feb 1, 2011, at 2:50 PM, Guru Ananda wrote: > Hi David, > > We download quality score files (qual.fa) from UCSC and convert them to > binned array format using scripts from bx-python > (https://bitbucket.org/james_taylor/bx-python/wiki/Home) to enable fast seek. > If you interested in the particulars (which script etc.), please let me know > and I can provide you with the details. > > Thanks, > Guru. > > On Tue, Feb 1, 2011 at 1:23 PM, David Hoover wrote: > Where do the files in quality_scores.loc come from? Can they be downloaded > from UCSC or somewhere else? > > David Hoover > Helix Systems Staff > http://helix.nih.gov > ___ > galaxy-dev mailing list > galaxy-dev@lists.bx.psu.edu > http://lists.bx.psu.edu/listinfo/galaxy-dev > > > > -- > Graduate student, Bioinformatics and Genomics > Makova lab/Galaxy team > 505 Wartik lab > University Park PA 16802 > g...@psu.edu > ___ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
Re: [galaxy-dev] histogram tool not working
> R gives the error message 'x' must be numeric, so relevant questions > are what version of R do you have, what version or rpy, and what > version of Python (since Galaxy's tools tends to invoke R from > Python via the ryp library - certainly the histogram tool does). python = 2.6 R = 2.10.0 rpy = 1.0.3 > P.S. Am I getting old, or is "What gives?" just regionally popular slang? What region? David ___ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
Re: [galaxy-dev] quality_scores.loc files
And, exactly what file(s) do you get from UCSC? I can find some for various organisms (goldenPath/[org]/bigZips/[org].quals.fa.gz), but hg18, hg19, mm8, and mm9 are not among them. David On Feb 1, 2011, at 3:57 PM, Guru Ananda wrote: > I've successfully installed bx-python, and there are mountain of utility > scripts. Which ones do you use for converting the quals.fa file to binned > array format? > > bx-python/scripts/qv_to_bqv.py > > > > On Feb 1, 2011, at 2:50 PM, Guru Ananda wrote: > >> Hi David, >> >> We download quality score files (qual.fa) from UCSC and convert them to >> binned array format using scripts from bx-python >> (https://bitbucket.org/james_taylor/bx-python/wiki/Home) to enable fast >> seek. >> If you interested in the particulars (which script etc.), please let me know >> and I can provide you with the details. >> >> Thanks, >> Guru. >> >> On Tue, Feb 1, 2011 at 1:23 PM, David Hoover wrote: >> Where do the files in quality_scores.loc come from? Can they be downloaded >> from UCSC or somewhere else? >> >> David Hoover >> Helix Systems Staff >> http://helix.nih.gov >> ___ >> galaxy-dev mailing list >> galaxy-dev@lists.bx.psu.edu >> http://lists.bx.psu.edu/listinfo/galaxy-dev >> >> >> >> -- >> Graduate student, Bioinformatics and Genomics >> Makova lab/Galaxy team >> 505 Wartik lab >> University Park PA 16802 >> g...@psu.edu >> > > > > > -- > Graduate student, Bioinformatics and Genomics > Makova lab/Galaxy team > 505 Wartik lab > University Park PA 16802 > g...@psu.edu > ___ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
[galaxy-dev] galaxy-dev on Helix mailing list
We've created a Galaxy development mailing list to go along with the Galaxy development site. This mailing list will be used for communicating changes and questions about developing for Galaxy on the Helix Systems. Not everyone is on it, only those who specifically requested for access. If you want to be on the galaxy-dev mailing list, just let me know. As you all are aware, Galaxy is up and running on Helix (http://galaxy.cit.nih.gov). Please log in and try it out. Most of all, please give us feedback about what works, what doesn't, and what could improve. Some of the tools are configured to run on the Biowulf cluster, most run locally. We also have a separate Galaxy development site (http://galaxy-dev.cit.nih.gov). This is mainly for developing new XML interfaces (so-called tools) for other command line applications already installed on Helix and Biowulf. We've added a few people to the galaxy-dev group, which is necessary to make any changes. The Galaxy development site is located in /data/galaxy/dev, and the XML files are in /data/galaxy/dev/tools. Because any changes to the XML files requires the Galaxy server be restarted, I've created a wrapper executable, init_galaxy. After making changes, on Biowulf type: rsh p1286 /data/galaxy/devlog/init_galaxy restart and this will restart the server on the Galaxy development host (p1286). Most importantly, the server logs are kept in /data/galaxy/devlog. These are key to debugging any changes made. The main configuration file (universe_wsgi.ini), the tool configuration file (tool_conf.xml), and the tool data configuration file (tool_data_table_conf.xml) are write accessible to those in the galaxy_dev group, as these will be necessary to change when new applications are added. The development server DOES NOT have access to the internet, so uploading files through HTTP and FTP won't work. Downloading data from UCSC or other external sites will also not work. However, you can copy local files from your /home and /data areas on Helix using the Import Helix file tool. There is a danger of stepping on each other's toes when making changes, so be sure to use this mailing list to let everyone know when you are making changes and restarting the server. Feel free to use this mailing list to ask questions and state what you did. Cheers, David Hoover Helix Systems Staff http://helix.nih.gov ___ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
[galaxy-dev] megablast and blastdb
When I try to load the megablast_wrapper tool, I get errors in the output: AssertionError: Data table named 'blastdb' is required by tool but not configured I am totally confused how blastdb.loc should be configured. Is it one tab, two, three? David Hoover Helix Systems Staff http://helix.nih.gov ___ To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] megablast and blastdb
Nope, figured it out. I forgot to add the appropriate table blocks to the tool_data_table_conf.xml file. So obvious! David On Feb 15, 2011, at 5:05 PM, Bossers, Alex wrote: > David, > this is what we have in blastdb.loc (nucleotide) or blastdb_p.loc (prot)a > snippettab delimited > caption can contain spaces so I increased the tab for display only > > #This is a sample file distributed with Galaxy that is used to define a > #list of nucleotide BLAST databases, using three columns tab separated > #(longer whitespace are TAB characters): > # > # > # > nt_24_10_2010 nt 2010 Nov 24 /share/BlastDB/nt > vector_24_10_2010 vector 2010 Nov 24 /share/BlastDB/vector > > > > Alex > > > Van: galaxy-dev-boun...@lists.bx.psu.edu > [galaxy-dev-boun...@lists.bx.psu.edu] namens David Hoover > [hoove...@helix.nih.gov] > Verzonden: dinsdag 15 februari 2011 22:33 > Aan: Galaxy Dev > Onderwerp: [galaxy-dev] megablast and blastdb > > When I try to load the megablast_wrapper tool, I get errors in the output: > > AssertionError: Data table named 'blastdb' is required by tool but not > configured > > I am totally confused how blastdb.loc should be configured. Is it one tab, > two, three? > > David Hoover > Helix Systems Staff > http://helix.nih.gov > ___ > To manage your subscriptions to this and other Galaxy lists, please use the > interface at: > > http://lists.bx.psu.edu/ ___ To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] upload directory of files error
This used to work. Now when I try to upload a directory of files, it gives this error: URL: https://galaxy.cit.nih.gov/library_common/ldda_info?library_id=a799d38679e985db&show_deleted=False&cntrller=library&folder_id=0a248a1f62a0cc04&use_panels=False&id=dff4190d282fb07a File '/gs1/users/galaxy/pro/eggs/WebError-0.8a-py2.6.egg/weberror/evalexception/middleware.py', line 364 in respond app_iter = self.application(environ, detect_start_response) File '/gs1/users/galaxy/pro/eggs/Paste-1.6-py2.6.egg/paste/debug/prints.py', line 98 in __call__ environ, self.app) File '/gs1/users/galaxy/pro/eggs/Paste-1.6-py2.6.egg/paste/wsgilib.py', line 539 in intercept_output app_iter = application(environ, replacement_start_response) File '/gs1/users/galaxy/pro/eggs/Paste-1.6-py2.6.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/gs1/users/galaxy/pro/lib/galaxy/web/framework/middleware/remoteuser.py', line 121 in __call__ return self.app( environ, start_response ) File '/gs1/users/galaxy/pro/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/gs1/users/galaxy/pro/lib/galaxy/web/framework/base.py', line 145 in __call__ body = method( trans, **kwargs ) File '/gs1/users/galaxy/pro/lib/galaxy/web/controllers/library_common.py', line 574 in ldda_info status=status ) File '/gs1/users/galaxy/pro/lib/galaxy/web/framework/__init__.py', line 645 in fill_template return self.fill_template_mako( filename, **kwargs ) File '/gs1/users/galaxy/pro/lib/galaxy/web/framework/__init__.py', line 656 in fill_template_mako return template.render( **data ) File '/gs1/users/galaxy/pro/eggs/Mako-0.2.5-py2.6.egg/mako/template.py', line 133 in render return runtime._render(self, self.callable_, args, data) File '/gs1/users/galaxy/pro/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py', line 364 in _render _render_context(template, callable_, context, *args, **_kwargs_for_callable(callable_, data)) File '/gs1/users/galaxy/pro/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py', line 381 in _render_context _exec_template(inherit, lclcontext, args=args, kwargs=kwargs) File '/gs1/users/galaxy/pro/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py', line 414 in _exec_template callable_(context, *args, **kwargs) File '/gs1/users/galaxy/pro/database/compiled_templates/base.mako.py', line 40 in render_body __M_writer(unicode(next.body())) File '/gs1/users/galaxy/pro/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py', line 255 in return lambda *args, **kwargs:callable_(self.context, *args, **kwargs) File '/gs1/users/galaxy/pro/database/compiled_templates/library/common/ldda_info.mako.py', line 224 in render_body if job.stdout.strip() != '': AttributeError: 'NoneType' object has no attribute 'strip' Huh? David Hoover Helix Systems Staff http://helix.nih.gov ___ To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] substitute or replace a value?
Is there a tool for doing a global or column-based substitution in a text or tabular file? For example, replace all instances of the number 2 in a column to a number 1? David Hoover Helix Systems Staff http://helix.nih.gov ___ To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Python bug?
We've run into an error with the SAM to BAM converter. An error occurred running this job: Samtools Version: 0.1.15 (r949:203) Error extracting alignments from (/gs1/users/galaxy/pro/database/files/000/dataset_285.dat), requested number of bytes is more than a Python string can hold We are running 32-bit Python. Would this be the problem, or do we need a newer version of samtools? David Hoover Helix Systems Staff http://helix.nih.gov ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
