Re: [galaxy-dev] Galaxy instance file upload problem
Hi Bongsoo we had once a similar situation with one of our development Galaxy installations. This was due to a change in the proxy settings of the server. Regards, Hans-Rudolf On 10/17/2014 01:06 AM, Bongsoo Park wrote: Aysam, Thanks for your reply. It works well on usegalaxy.org http://usegalaxy.org, but it doesn't work in the Galaxy instance I've developed. I downloaded the latest Galaxy version, and installed on the Redhat 6.5 system. I created a galaxy user, and just ran it as usual. I have to update any part of the configuration? I have to allow any specific port to use file upload function? Thanks! Best, Bongsoo On Thu, Oct 16, 2014 at 6:25 PM, Aysam Guerler aysam.guer...@gmail.com mailto:aysam.guer...@gmail.com wrote: Hey Bongsoo, Does this happen on usegalaxy.org http://usegalaxy.org? Also does this happen with other files too? Thanks, Sam On Thu, Oct 16, 2014 at 2:49 PM, Bongsoo Park bx...@psu.edu mailto:bx...@psu.edu wrote: Hi folks, I encountered the problem in file upload. The error message is like below 'Failed: Not found (404)'. Attached is the screen shot for this error message. Any idea? Thanks! Best, Bongsoo ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Set a new metadata attribute
Hi Nikos As a 'quick and dirty' work-around, I suggest generating in addition to the output file a 'log' file, which can be read by the subsequent tool. Hans-Rudolf On 09/26/2014 04:54 PM, Peter Cock wrote: On Fri, Sep 26, 2014 at 3:01 PM, Nikos Sidiropoulos nikos.sid...@gmail.com wrote: Hi all, In a tool that I am writting I want to pass an input parameter value (string) into the output file's metadata. Meaning that one of the tool parameters is a barcode signature, 'NNWTGXN' for example. I want that attribute to be stored somehow in the output file in order to be read by a subsequent tool without the user having to set that parameter again. The files I'll be working with are in FASTQ, BAM and tabular format. Is it possible? Bests, Nikos Your code can write the value directly into an output file (e.g. one of the SAM/BAM headers might work), but I don't think there is anything suitable within Galaxy for re-exporting the parameter value as an input parameter for a future tool. However, at the workflow level you can set variables - might that be a way forward? https://wiki.galaxyproject.org/Learn/AdvancedWorkflow/VariablesEdit Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Tool Errors
On 09/12/2014 05:11 PM, Calvin Morrison wrote:
The stderr and stdout is empty, according to galaxy.
here is paster.log output for quikr when i run it.
galaxy.jobs.runners DEBUG 2014-09-12 10:33:45,997 (86) command is: # if
user == user quikr -v -k 0 -s
/data/galaxy/galaxy-dist/database/files/000/dataset_103.dat -i
/data/galaxy/galaxy-dist/database/files/000/dataset_55.dat -o
/data/galaxy/galaxy-dist/database/files/000/dataset_104.dat # else
quikr -v -k 0 -s
/data/galaxy/galaxy-dist/database/files/000/dataset_103.dat.mat.gz -i
/data/galaxy/galaxy-dist/database/files/000/dataset_55.dat -o
/data/galaxy/galaxy-dist/database/files/000/dataset_104.dat# end if
that doesn't really seem all that helpful though.
Actually, it is very helpful, isn't-t? It shows that the command tag
from your xml file is not properly executed.
my xml file:
command
# if $qdb.dbtype == user
quikr -v -k 0 -s ${qdb.dbname} -i ${input} -o ${output}
# else
quikr -v -k 0 -s
${qdb.dbname}${qdb.dbsize}.__mat.gz -i ${input} -o
${output}
# end if
/command
try removing the spaces between # and if, # and else, and #
and end if. And re-load the tool configuration.
See also:
https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#A.3Cwhen.3E_tag_set
Hope this helps
Hans-Rudolf
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] error with multi dataset tool run
Hi Robert
Are you using the built in SQLite database ?
Hans-Rudolf
On 08/31/2014 01:27 AM, Robert Baertsch wrote:
I submitted 13 fastq files to tophat2 using DRMAA and got this error.
Is it fatal? BTW: This is a super cool feature.
I’m running the following version of galaxy-dist.
changeset: 14212:91547729ffde
branch: stable
tag: tip
user:Nate Coraor n...@bx.psu.edu mailto:n...@bx.psu.edu
date:Fri Aug 29 14:00:23 2014 -0400
summary: Update tag latest_2014.08.11 for changeset ea12550fbc34
-Robert
There were errors setting up 2 submitted job(s):
* *Error executing tool: (OperationalError) database is locked
u'UPDATE history_dataset_association SET update_time=?, name=?,
blurb=? WHERE history_dataset_association.id = ?' ('2014-08-30
23:14:44.683957', 'Tophat2 on data 7: insertions', 'queued', 137)*
* *Error executing tool: (OperationalError) database is locked
u'UPDATE dataset SET update_time=?, state=? WHERE dataset.id = ?'
('2014-08-30 23:15:57.204718', 'queued', 446)*
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Solved - Re: testing the visualization plugins
Hi all First of all, a big Thanks to Carl who helped me fixing this problem. So as a summary for all, the problem was caused by a datatype (an extension to tabular), I manually added to datatypes_conf.xml Removing the datatype fixed the problem. I couldn't identify a syntax problem, neither in datatypes_conf.xml nor in ~/lib/galaxy/datatypes/registry.py and ~/lib/galaxy/datatypes/tabular.py. However, renaming it (in all three files) fixed it as well. Regards, Hans-Rudolf ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] testing the visualization plugins
Hi Carl What was the datatype (listed as 'format') of the dataset you tested with here?: However, if I hoover over the bar-plot icon, Visualize in Trackster pops up (instead of just Visualize) any data type, eg wig, tabular, bed, etc I currently have: 'result_type=datatypedata.Data' in ~/config/plugins/visualizations/trackster/config/trackster.xml If I change that to 'result_type=datatypeinterval.Interval' (and do a restart), I lose the bar-plot icon for wig and tabular. It is still there for the bed file in the history. Do the 'charts' and 'scatterplot' visualizations show up in the visualization icon popup menu when you check on an interval or bed datatype? They never show up, I don't get the menu when I click on the bar-plot icon (for bed and/or interval data typ). It goes straight to the Trackster page, where, I am asked to define the Browser name and the Reference genome. Hans-Rudolf Thanks for the report and any info you can provide, Carl On Wed, Aug 27, 2014 at 4:03 AM, Hans-Rudolf Hotz h...@fmi.ch mailto:h...@fmi.ch wrote: Hi As part of my upgrade/testing work, I am looking at the visualization plugins. In a new galaxy installation (latest_2014.08.11), they (ie 'Charts', 'Scatterplot', and 'Trackster') seem to work fine. All I have done was: - I defined the visualization_plugins___directory in the universe_wsgi.ini file If I comment-out this line, I only get trackster, when I click on the icon which looks like a bar-plot Next, I have tried it on our development server (after I have upgraded from release_2014.04.14 to latest_2014.08.11). I changed the universe_wsgi.ini file. I the logs I get: galaxy.web.base.__pluginframework INFO 2014-08-27 09:43:45,639 VisualizationsRegistry, loaded plugin: charts galaxy.web.base.__pluginframework WARNING 2014-08-27 09:43:45,641 VisualizationsRegistry, plugin load failed or disabled: ./config/plugins/__visualizations/circster. Skipping... galaxy.web.base.__pluginframework INFO 2014-08-27 09:43:45,655 VisualizationsRegistry, loaded plugin: graphview galaxy.web.base.__pluginframework INFO 2014-08-27 09:43:45,657 VisualizationsRegistry, loaded plugin: phyloviz galaxy.web.base.__pluginframework INFO 2014-08-27 09:43:45,675 VisualizationsRegistry, loaded plugin: scatterplot galaxy.web.base.__pluginframework WARNING 2014-08-27 09:43:45,677 VisualizationsRegistry, plugin load failed or disabled: ./config/plugins/__visualizations/sweepster. Skipping... galaxy.web.base.__pluginframework INFO 2014-08-27 09:43:45,679 VisualizationsRegistry, loaded plugin: trackster wrt to 'Charts', 'Scatterplot', and 'Trackster', this looks fine to me. However, if I hoover over the bar-plot icon, Visualize in Trackster pops up (instead of just Visualize), and when I click on it I get straight to the Trackster page, where, I am asked to define the Browser name and the Reference genome. No links for 'Charts' and 'Scatterplot'. On the other hand, the plug in infrastructure seems to work: If I change ~/config/plugins/__visualizations/trackster/__config/trackster.xml I see the effects, ie if I change result_type=datatypedata.__Data to result_type=datatype__interval.Interval I can restrict the visualization to only the interval data. Somehow, I am running a mixture of the 'old' and the 'new' visualization set-up ? Has anyone seen similar problems when testing the new plug in infrastructure on an old/upgraded Galaxy installation Thank you very much for any hints and help Hans-Rudolf -- Hans-Rudolf Hotz, PhD Bioinformatics Support Friedrich Miescher Institute for Biomedical Research Maulbeerstrasse 66 4058 Basel/Switzerland _ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/__search/mailinglists/ http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] testing the visualization plugins
Hi Carl
Thanks for your help - unfortunately no success so far
Let's try two things:
*First*, let's clear the browser cache to rule out that annoyance:
This didn't change anything. I have also tried different computers and
different browsers
*Second*, let's see what the server ends up sending the client for
visualizations:
it just looks like this:
visible: true,
visualizations: [
{
embeddable: false,
href:
/visualization/trackster?dataset_id=b1c72d84f3501797dbkey=%3Fhda_ldda=hda,
html: Trackster,
target: _top
}
]
}
Hans-Rudolf
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
[galaxy-dev] testing the visualization plugins
Hi As part of my upgrade/testing work, I am looking at the visualization plugins. In a new galaxy installation (latest_2014.08.11), they (ie 'Charts', 'Scatterplot', and 'Trackster') seem to work fine. All I have done was: - I defined the visualization_plugins_directory in the universe_wsgi.ini file If I comment-out this line, I only get trackster, when I click on the icon which looks like a bar-plot Next, I have tried it on our development server (after I have upgraded from release_2014.04.14 to latest_2014.08.11). I changed the universe_wsgi.ini file. I the logs I get: galaxy.web.base.pluginframework INFO 2014-08-27 09:43:45,639 VisualizationsRegistry, loaded plugin: charts galaxy.web.base.pluginframework WARNING 2014-08-27 09:43:45,641 VisualizationsRegistry, plugin load failed or disabled: ./config/plugins/visualizations/circster. Skipping... galaxy.web.base.pluginframework INFO 2014-08-27 09:43:45,655 VisualizationsRegistry, loaded plugin: graphview galaxy.web.base.pluginframework INFO 2014-08-27 09:43:45,657 VisualizationsRegistry, loaded plugin: phyloviz galaxy.web.base.pluginframework INFO 2014-08-27 09:43:45,675 VisualizationsRegistry, loaded plugin: scatterplot galaxy.web.base.pluginframework WARNING 2014-08-27 09:43:45,677 VisualizationsRegistry, plugin load failed or disabled: ./config/plugins/visualizations/sweepster. Skipping... galaxy.web.base.pluginframework INFO 2014-08-27 09:43:45,679 VisualizationsRegistry, loaded plugin: trackster wrt to 'Charts', 'Scatterplot', and 'Trackster', this looks fine to me. However, if I hoover over the bar-plot icon, Visualize in Trackster pops up (instead of just Visualize), and when I click on it I get straight to the Trackster page, where, I am asked to define the Browser name and the Reference genome. No links for 'Charts' and 'Scatterplot'. On the other hand, the plug in infrastructure seems to work: If I change ~/config/plugins/visualizations/trackster/config/trackster.xml I see the effects, ie if I change result_type=datatypedata.Data to result_type=datatypeinterval.Interval I can restrict the visualization to only the interval data. Somehow, I am running a mixture of the 'old' and the 'new' visualization set-up ? Has anyone seen similar problems when testing the new plug in infrastructure on an old/upgraded Galaxy installation Thank you very much for any hints and help Hans-Rudolf -- Hans-Rudolf Hotz, PhD Bioinformatics Support Friedrich Miescher Institute for Biomedical Research Maulbeerstrasse 66 4058 Basel/Switzerland ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] problems with database migration 119 - 120
Hi all I am in the process of updating our galaxy servers (from release_2014.04.14 to latest_2014.08.11). when I execute ~/lib/galaxy/model/migrate/versions/0120_dataset_collections.py as part of the 'manage_db.sh upgrade' I run into a bizarre error: First, it produces 10 Mysql 1050 Error 'Table already exists' , I have encountered this before, and usually everything is fine. The table gets created and for whatever reason, the command get's executed a second time - no big deal. However, this time for two of those ten table the situation has been different. As usual, I have checked all the tables (where I got the errors) with the MySQL command describe table. For two tables: history_dataset_collection_association library_dataset_collection_association the order of the columns was wrong (ie did not correspond to the order in the create statement) - see below for example. I have dropped the tables and executed the create statements manually, everything seems fine, eg mysql describe library_dataset_collection_association; +---+--+--+-+-++ | Field | Type | Null | Key | Default | Extra | +---+--+--+-+-++ | id| int(11) | NO | PRI | NULL| auto_increment | | collection_id | int(11) | YES | MUL | NULL|| | folder_id | int(11) | YES | MUL | NULL|| | name | varchar(255) | YES | | NULL|| | deleted | tinyint(1) | YES | | NULL|| +---+--+--+-+-++ 5 rows in set (0.00 sec) mysql drop table library_dataset_collection_association; Query OK, 0 rows affected (0.02 sec) mysql CREATE TABLE library_dataset_collection_association ( - id INTEGER NOT NULL AUTO_INCREMENT, - collection_id INTEGER, - name VARCHAR(255), - deleted BOOL, - folder_id INTEGER, - PRIMARY KEY (id), - FOREIGN KEY(collection_id) REFERENCES dataset_collection (id), - CHECK (deleted IN (0, 1)), - FOREIGN KEY(folder_id) REFERENCES library_folder (id) - ); Query OK, 0 rows affected (0.01 sec) mysql describe library_dataset_collection_association; +---+--+--+-+-++ | Field | Type | Null | Key | Default | Extra | +---+--+--+-+-++ | id| int(11) | NO | PRI | NULL| auto_increment | | collection_id | int(11) | YES | MUL | NULL|| | name | varchar(255) | YES | | NULL|| | deleted | tinyint(1) | YES | | NULL|| | folder_id | int(11) | YES | MUL | NULL|| +---+--+--+-+-++ 5 rows in set (0.00 sec) mysql Has anyone else (among those few who are still using MySQL) seen something similar? Regards, Hans-Rudolf PS: Please don't make any comments about the fact, that I should change to PostgreSQL. I am more than aware of that, and eventually, I will do it. ;) -- Hans-Rudolf Hotz, PhD Bioinformatics Support Friedrich Miescher Institute for Biomedical Research Maulbeerstrasse 66 4058 Basel/Switzerland ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] problems with database migration 119 - 120
Hi John Thanks for the insights Hi Iyad Yes, the user account has the ability to drop and alter tables. Hans-Rudolf On 08/26/2014 03:08 PM, John Chilton wrote: Well it looks like the migration file has these columns listed in a different order than the mapping Galaxy uses - and the order yours appeared in were the ones from Galaxy's mapping file. So somehow Galaxy is automatically creating those tables prior to running the migration based on the code in Galaxy proper. That is odd. Ultimately though - I don't think it is harmful that the order is wrong since Galaxy always references the columns by name instead of order. I would be more concerned that you aren't getting the right indices / foreign keys - but it looks like you are still on MyISAM so you aren't going to get a ton of forced integrity anyway (and maybe this is why these errors have been okay in the past?). -John On Tue, Aug 26, 2014 at 8:52 AM, Kandalaft, Iyad iyad.kandal...@agr.gc.ca wrote: I've ran into other mysql problems but never anything like that. This is just a hunch and not based on anything concrete, but using an outdated version of sqlalchemy is probably not helping things. We're talking 2 migrations. Are they even implementing fixes for 0.7 anymore? It is odd you would ever get Table already exists since the ORM's job is to ensure the model consistency in the database. Why it would try to create the table before it checks for its existence is beyond me. You might want to make sure that the database user account has the ability to drop and alter tables. It may be that it tried to revert a failed upgrade and it wasn't able to. Iyad Kandalaft -Original Message- From: galaxy-dev-boun...@lists.bx.psu.edu [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Hans-Rudolf Hotz Sent: Tuesday, August 26, 2014 5:22 AM To: galaxy-...@bx.psu.edu Subject: [galaxy-dev] problems with database migration 119 - 120 Hi all I am in the process of updating our galaxy servers (from release_2014.04.14 to latest_2014.08.11). when I execute ~/lib/galaxy/model/migrate/versions/0120_dataset_collections.py as part of the 'manage_db.sh upgrade' I run into a bizarre error: First, it produces 10 Mysql 1050 Error 'Table already exists' , I have encountered this before, and usually everything is fine. The table gets created and for whatever reason, the command get's executed a second time - no big deal. However, this time for two of those ten table the situation has been different. As usual, I have checked all the tables (where I got the errors) with the MySQL command describe table. For two tables: history_dataset_collection_association library_dataset_collection_association the order of the columns was wrong (ie did not correspond to the order in the create statement) - see below for example. I have dropped the tables and executed the create statements manually, everything seems fine, eg mysql describe library_dataset_collection_association; +---+--+--+-+-++ | Field | Type | Null | Key | Default | Extra | +---+--+--+-+-++ | id| int(11) | NO | PRI | NULL| auto_increment | | collection_id | int(11) | YES | MUL | NULL|| | folder_id | int(11) | YES | MUL | NULL|| | name | varchar(255) | YES | | NULL|| | deleted | tinyint(1) | YES | | NULL|| +---+--+--+-+-++ 5 rows in set (0.00 sec) mysql drop table library_dataset_collection_association; Query OK, 0 rows affected (0.02 sec) mysql CREATE TABLE library_dataset_collection_association ( - id INTEGER NOT NULL AUTO_INCREMENT, - collection_id INTEGER, - name VARCHAR(255), - deleted BOOL, - folder_id INTEGER, - PRIMARY KEY (id), - FOREIGN KEY(collection_id) REFERENCES dataset_collection (id), - CHECK (deleted IN (0, 1)), - FOREIGN KEY(folder_id) REFERENCES library_folder (id) - ); Query OK, 0 rows affected (0.01 sec) mysql describe library_dataset_collection_association; +---+--+--+-+-++ | Field | Type | Null | Key | Default | Extra | +---+--+--+-+-++ | id| int(11) | NO | PRI | NULL| auto_increment | | collection_id | int(11) | YES | MUL | NULL|| | name | varchar(255) | YES | | NULL|| | deleted | tinyint(1) | YES | | NULL|| | folder_id | int(11) | YES | MUL | NULL|| +---+--+--+-+-++ 5 rows in set
Re: [galaxy-dev] Settings multiple Genome Indices
Hi Geert A simple solution might be: you hand over a string containing all three paths to your wrapper, and then the wrapper splits the string into the three individual paths. Hans-Rudolf On 07/27/2014 04:44 PM, Geert Vandeweyer wrote: Hi, I'm working on tool wrapper that needs three reference genome locations (2bit/lastz, all_fasta and novoalign). Is there a way to fetch the three paths from the loc files using a single select box? I've looked around in a couple of tool wrappers on the toolshed, but could find any leads. Something like the following would be ok: Select options from all_fasta.loc - upon change, set hidden variables to novoalign.loc + lastz_seqs.loc using a filter tag or something against the selected options from all_fasta. Even more simple : access in the command tag like: --novoidx novoalign-loc.options[dbkey=$input.dbkey,path] --fasta all_fasta-loc[dbkey=$input.dbkey,path] etc , using the dbkey from the input file (which is a bam file in this case) In this case, error handling should come from the tool, if one of the indices is missing. I'm currently using three selection boxes, but that looks a but redundant to the user, as they all have to select the same thing. Best, Geert ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Settings multiple Genome Indices
On 07/28/2014 10:02 AM, Geert Vandeweyer wrote: If I understand you correctly, would this mean generating a custom .loc file for this tool containing the concatenated paths? Or can I concatenate paths from different .loc files somehow, which would allow me to use the default loc files? This would require a custom .loc file Hans-Rudolf Geert On 07/28/2014 09:33 AM, Hans-Rudolf Hotz wrote: Hi Geert A simple solution might be: you hand over a string containing all three paths to your wrapper, and then the wrapper splits the string into the three individual paths. Hans-Rudolf On 07/27/2014 04:44 PM, Geert Vandeweyer wrote: Hi, I'm working on tool wrapper that needs three reference genome locations (2bit/lastz, all_fasta and novoalign). Is there a way to fetch the three paths from the loc files using a single select box? I've looked around in a couple of tool wrappers on the toolshed, but could find any leads. Something like the following would be ok: Select options from all_fasta.loc - upon change, set hidden variables to novoalign.loc + lastz_seqs.loc using a filter tag or something against the selected options from all_fasta. Even more simple : access in the command tag like: --novoidx novoalign-loc.options[dbkey=$input.dbkey,path] --fasta all_fasta-loc[dbkey=$input.dbkey,path] etc , using the dbkey from the input file (which is a bam file in this case) In this case, error handling should come from the tool, if one of the indices is missing. I'm currently using three selection boxes, but that looks a but redundant to the user, as they all have to select the same thing. Best, Geert ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Local deployments of galaxy
Hi David There is another number, which might help you: - there are over 1000 people on the dev-mailing list (https://wiki.galaxyproject.org/GalaxyProject/Statistics#Membership) assuming there are two galaxy admins per site - wishful thinking ;) - and each site has on average two Galaxy servers running...you end up with ~1000 Galaxy servers in the Universe. Hans-Rudolf On 07/24/2014 07:02 PM, Martin Čech wrote: There is a list of the ones that reached back to us and are publicly accessible (~60): https://wiki.galaxyproject.org/PublicGalaxyServers As of private ones there is no good clue as Galaxy does not call home. M. On Thu, Jul 24, 2014 at 12:55 PM, Lapointe, David david.lapoi...@tufts.edu mailto:david.lapoi...@tufts.edu wrote: Not exactly a dev question, but I was asked recently how many sites are running galaxy. The wiki page listing deployments seems not to reflect the actual number, since may are most likely private. Is there an estimate? -- David Lapointe Ph.D. Sr. Bioinformatics Specialist Research and Geospatial Technology Services Tufts Technology Services (TTS) 16 Dearborn Road Somerville MA 02144 Phone: 617-627-5319 Fax: 617-627-3667 http://it.tufts.edu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Jobs stay queued
Hi Julie Are there any jobs running at all? What do you get when you click on Manage jobs in the admin menu? Maybe you have some old jobs 'running' which are now blocking your queue? Hans-Rudolf On 07/23/2014 11:14 AM, julie dubois wrote: Hi all, I've seen several posts for jobs which stay queued and never start with an installation of galaxy on a cluster. But our installation is just a local instance of galaxy in 1 PC and I have more than 100 jobs which stay queued and never start. How can I debug this problem (my version of galaxy is the latest 2june2014). And is it possible to run those jobs without restart galaxy because it would be very long to rerun each job manually after reloading galaxy. Thanks. Julie ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Jobs stay queued
On 07/23/2014 12:05 PM, julie dubois wrote: Hi Hans-Rudolf, Thanks for your answer. I've launched lot of jobs since monday. And some jobs were finished and some jobs never started. In Manage jobs I can see jobs queued because they need inputs which are queued, so I think it's normal. But other jobs are described like this : 9554 user_login 0 hours agosamtools_flagstatnew23199 ok, 23243 okNoneNoneNone The two necessary inputs for this job (23199 and 2343) are ok and the job doesn't run. Another mysterious line is : 9561 user_login0 hours ago fastqc new Unable to determine inputs None NoneNone After check this job in my history, I'm sure that the input of this job is green and really available. Why can galaxy not determine inputs ? Finally, in Manage Jobs there are only jobs queued. No jobs running. Any idea ? I have one suspicion: which database are you using? If you are just using the built in SQlite you might have run in a database-lock? Because you have submitted too many jobs at once? In this case, I guess, the only way to recover is to re-start... Hans-Rudolf Thanks Julie 2014-07-23 11:35 GMT+02:00 Hans-Rudolf Hotz h...@fmi.ch: Hi Julie Are there any jobs running at all? What do you get when you click on Manage jobs in the admin menu? Maybe you have some old jobs 'running' which are now blocking your queue? Hans-Rudolf On 07/23/2014 11:14 AM, julie dubois wrote: Hi all, I've seen several posts for jobs which stay queued and never start with an installation of galaxy on a cluster. But our installation is just a local instance of galaxy in 1 PC and I have more than 100 jobs which stay queued and never start. How can I debug this problem (my version of galaxy is the latest 2june2014). And is it possible to run those jobs without restart galaxy because it would be very long to rerun each job manually after reloading galaxy. Thanks. Julie ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] cleanup_datasets.py not deleting files...
On 07/17/2014 12:38 PM, neil.burd...@csiro.au wrote: Thanks. Is there a script that will delete all files older than a certain date even if not marked as deleted by the user? I am not aware of such a script Hans-Rudolf Thanks Neil From: Hans-Rudolf Hotz [h...@fmi.ch] Sent: Thursday, July 17, 2014 5:42 PM To: Burdett, Neil (DPS, Herston - RBWH); galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] cleanup_datasets.py not deleting files... Hi Neil The cleanup_datasets.py script only removes the files if older than the time given (you did this, ie older than 20 days) AND if the dataset has been marked as 'deleted' by the user - have you done that? see also: https://wiki.galaxyproject.org/Admin/Config/Performance/Purge%20Histories%20and%20Datasets Hope this helps Hans-Rudolf On 07/17/2014 02:42 AM, neil.burd...@csiro.au wrote: Hi, I'm trying to use the cleanup_datasets.py file to remove all files on my system older than 20 days. My crontab looks like this: # m h dom mon dow command 34 10 * * * cd /export/barium-data3/galaxy-suvr python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -d 20 -1 /home/galaxy/crontab_purge_milxcloud.log 21 python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -d 20 -2 -r /home/galaxy/crontab_purge_milxcloud.log 21 python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -d 20 -3 -r /home/galaxy/crontab_purge_milxcloud.log 21 python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -d 20 -5 -r /home/galaxy/crontab_purge_milxcloud.log 21 python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -d 20 -4 -r /home/galaxy/crontab_purge_milxcloud.log 21 python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -d 20 -6 -r /home/galaxy/crontab_purge_milxcloud.log 21 The crontab executes and the contents of /home/galaxy/crontab_purge_milxcloud.log is: cat /home/galaxy/crontab_purge_milxcloud.log psycopg2 egg successfully loaded for postgres dialect ## # 2014-07-17 10:34:02 - Handling stuff older than 20 days Datasets will NOT be removed from disk. Deleted 0 histories Elapsed time: 0.170083045959 ## psycopg2 egg successfully loaded for postgres dialect ## # 2014-07-17 10:34:02 - Handling stuff older than 20 days Datasets will be removed from disk. Purged 0 histories. Elapsed time: 0.174137830734 ## psycopg2 egg successfully loaded for postgres dialect ## # 2014-07-17 10:34:03 - Handling stuff older than 20 days Datasets will be removed from disk. Purged 0 datasets Freed disk space: 0 Elapsed time: 0.168104887009 ## psycopg2 egg successfully loaded for postgres dialect ## # 2014-07-17 10:34:04 - Handling stuff older than 20 days Datasets will be removed from disk. # Purged 0 folders. Elapsed time: 0.168635129929 ## psycopg2 egg successfully loaded for postgres dialect ## # 2014-07-17 10:34:05 - Handling stuff older than 20 days Datasets will be removed from disk. # Purged 0 libraries . Elapsed time: 0.166506052017 ## psycopg2 egg successfully loaded for postgres dialect ## # 2014-07-17 10:34:06 - Handling stuff older than 20 days Datasets will be removed from disk. Examined 0 datasets, marked 0 datasets and 0 dataset instances (HDA) as deleted Total elapsed time: 0.00744795799255 ## However, today is the 17th July and when I look into the database directory i.e. ~/database/files/000 I still have files from the 18th June so older than 20 days i.e.: drwxr-xr-x 2 galaxy nogroup 4096 Jun 18 09:19 dataset_213_files -rw-r--r-- 1 galaxy nogroup 1061 Jun 18 09:19 dataset_213.dat drwxr-xr-x 2 galaxy nogroup 4096 Jun 18 09:19 dataset_215_files -rw-r--r-- 1 galaxy nogroup 270 Jun 18 09:19 dataset_215.dat drwxr-xr-x 2 galaxy nogroup 4096 Jun 18 09:19 dataset_221_files drwxr-xr-x 2 galaxy nogroup 4096 Jun 18 09:19 dataset_219_files drwxr-xr-x 2 galaxy nogroup 4096 Jun 18 09:19 dataset_220_files drwxr-xr-x 2 galaxy nogroup 4096 Jun 18 09:19 dataset_222_files drwxr-xr-x 2 galaxy nogroup 4096 Jun 18 09:19 dataset_218_files -rw-r--r-- 1 galaxy nogroup 994 Jun 18 09:19 dataset_216.dat -rw-r--r-- 1 galaxy nogroup 161 Jun 18 09:19 dataset_214.dat Am I doing something wrong? Using wrong arguments/file etc ? Thanks Neil ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other
Re: [galaxy-dev] Galaxy mobile site
Hi Mark There used to be a nice one: try adding '/mobile to your galaxy url. It is still kind of working (just without the graphics), and it is sufficient to monitor jobs. Hans-Rudolf There used to be one. On 06/26/2014 08:03 AM, MF Rogers wrote: I hope this isn't a FAQ; I have not yet found the question (or answer) online, though I can't imagine I'm the first to ask: Are there plans to make a mobile version of the Galaxy website? Many of the biologists with whom I work would like to be able to monitor and start Galaxy jobs from their iPhones (or other smart phone). While it is possible to do so using the regular website, a mobile-specific site could make it easier. We searched in vain for an app, but I gather that would be a headache to maintain with such a dynamic website. A mobile site seems the right approach, unless there are good reasons not to try it? We may experiment with creating one for our own local Galaxy installation, but would prefer to use existing code if it's available. Many thanks, -- Dr. Mark Rogers Intelligent Systems Laboratory Department of Engineering Mathematics University of Bristol ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Galaxy updated botched?
On 06/12/2014 12:06 AM, John Chilton wrote: If anyone out there has tired using Galaxy with strictly MyISAM tables and found it untenable - by all means please chime in. Well, due to several reasons we are (still) using MySQL (version: 5.1.69) with only MyISAM tables. We have been running into troubles but they were never related to running MyISAM. However, I haven't done the latest Galaxy code upgrade, hence we are still on database version '118'. So, I am curious to see what will happen when I do the next upgrade (after GCC). Also, on my list of things to do for this year is actually changing all our Galaxy servers from MySQL to PostgreSQL. With my current workload, I doubt I will manage to do this and have to postpone this 'adventure' to next year. Regards, Hans-Rudolf ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] ClustalW jobs aren't working
On 06/09/2014 07:33 PM, Malcolm Tobias wrote: All, I apologize for an off-topic question, by my update failed: [galaxy@login002 galaxy-dist]$ /export/mercurial-1.7.5/bin/hg pull -u | tee hg.pull.out ... added 3716 changesets with 9929 changes to 2124 files (+46 heads) ... merging lib/galaxy/app.py merging lib/galaxy/app.py failed! warning: conflicts during merge. merging lib/galaxy/model/__init__.py merging lib/galaxy/model/__init__.py failed! warning: conflicts during merge. merging lib/galaxy/model/mapping.py merging lib/galaxy/model/mapping.py failed! merging tool-data/shared/ucsc/ucsc_build_sites.txt and tool-data/shared/ucsc/ucsc_build_sites.txt.sample to tool-data/shared/ucsc/ucsc_build_sites.txt.sample 1528 files updated, 1 files merged, 1289 files removed, 3 files unresolved use 'hg resolve' to retry unresolved file merges resolve apparently can't fix this issue: [galaxy@login002 galaxy-dist]$ /export/mercurial-1.7.5/bin/hg resolve --all merging lib/galaxy/app.py warning: conflicts during merge. merging lib/galaxy/app.py failed! merging lib/galaxy/model/__init__.py warning: conflicts during merge. merging lib/galaxy/model/__init__.py failed! merging lib/galaxy/model/mapping.py warning: conflicts during merge. merging lib/galaxy/model/mapping.py failed! What's the recommended way to fix this? Depending on your local customization, it is not unusual to get unresolved files. Open the files in an editor and complete the merge manually. Regards, Hans-Rudolf Thanks! Malcolm On Monday 09 June 2014 10:47:04 Malcolm Tobias wrote: Ross, Thanks for the response. My instance of Galaxy is indeed ancient: [galaxy@login002 galaxy-dist]$ /export/mercurial-1.7.5/bin/hg tip changeset: 10003:b4a373d86c51 tag: tip parent: 10001:471484ff8be6 user: greg date: Wed Jun 12 11:48:09 2013 -0400 summary: Add targets to Repository Actions menu items. Previous attempts at updating have been pretty painful. I'll give it a shot and let you know whether or not this fixes the problem. Cheers, Malcolm On Friday 06 June 2014 18:30:01 Ross wrote: Hi Malcolm, That error makes me think you might be running an outdated version of galaxy code - the toolshed code has undergone extensive revision over the last few months? I just tested that repository on a freshly updated galaxy-central clone and it installed without drama, so I wonder what: hg tip shows? I just tested using : (vgalaxy)rlazarus@rlazarus-UX31A:~/galaxy$ hg tip changeset: 13756:84a00e4f7d06 tag: tip user:Dannon Baker dannonba...@me.com date:Fri Jun 06 17:12:30 2014 -0400 summary: Clarify language in DeleteIntermediateDataset PJA. If your clone is not up to date, I'd recommend completely removing the failed installation (through the admin menu - check the box for complete removal), shut down galaxy, backup your database, do the usual hg pull -u dance and any necessary database upgrade steps then try a clean install? Thanks for reporting this - if it persists on recent Galaxy code we'll need to do some deeper investigation. On Fri, Jun 6, 2014 at 11:32 PM, Malcolm Tobias mtob...@wustl.edu wrote: Ross, Thanks for the reply! Unfortunately I am the local Galaxy admin ;-) I had tried installing the clustalw tool from the toolshed, but that failed with an error (more on that later). I disabled the local tool: [galaxy@login002 galaxy-dist]$ diff tool_conf.xml tool_conf.xml.bkup 226a227,229 section name=Multiple Alignments id=clustal tool file=rgenetics/rgClustalw.xml / /section bounced galaxy in case that's necessary, then retried installing from the toolshed. Shortly after clicking install, I get this message: Internal Server Error Galaxy was unable to sucessfully complete your request An error occurred. This may be an intermittent problem due to load or other unpredictable factors, reloading the page may address the problem. The error has been logged to our team. The logs appear to be complaining about 'prior_installation_required' which I'm assuming means the package_clustalw_2_1 dependency. I was able to install that, and I can verify by looking at the local toolshed: [galaxy@login002 ~]$ ls galaxy-toolshed/toolshed.g2.bx.psu.edu/repos/devteam/ bowtie_wrappers package_clustalw_2_1 package_vcftools_0_1_11 bwa_wrappers package_fastx_toolkit_0_0_13 Again, I'll post the logs from when the install fails in case that helps. Any suggestions are much appreciated. Cheers, Malcolm 10.28.56.101 - - [06/Jun/2014:08:20:38 -0500] GET /admin_toolshed/prepare_for_install?tool_shed_url=http://toolshed.g2.bx.psu.edu/repository_ids=0e5d027cf47ecae0changeset_revisions=7cc64024fe92 HTTP/1.1 500 -
Re: [galaxy-dev] to put or not to put a tool suite into the toolshed
Hi Wolfgang short answer: Please, put your package into the Galaxy Tool Shed. This makes it visible to the whole Galaxy community and everybody can profit from it. long answer: I have been in the same boat. We never put the Galaxy wrappers and the perl scripts of our old NGS pipeline into the Tool Shed. I do feel bad about this. My excuse has always been that too much local customization would be necessary, eg making possible to store data outside the Galaxy directory tree. Hence, we only distributed the package as bunch of files on request. Obviously, there were hardly any requests. Now we are in a similar situation. We are in the process of making our new R/Bioconductor based NGS pipeline available on our internal Galaxy server. And the code is far from being ready to be put into the Tool Shed. But this time we will make the effort. So the Galaxy community can benefit from it (@Dave Clements: Unfortunately, it won't be ready for Baltimore). Regards, Hans-Rudolf On 06/05/2014 11:52 PM, Wolfgang Maier wrote: Dear all, I am maintaining a (still young) suite of command line tools (written in Python) for identification of mutations in model organism genomes through whole-genome sequencing (https://sourceforge.net/projects/mimodd/). MiModD aims at geneticists that do not have much background in bioinformatics, and it's supposed to make WGS analysis for small model organism genomes (anything from yeast to fish) possible on regular PCs, so it's not a cloud/cluster solution. We do support Galaxy though through a complete set of tool wrappers for three reasons: - to provide a graphical user interface - to offer labs the possibility to install the software on one dedicated machine, but run analyses from any machine (typically with Windows installed) - to keep analysis workflows documented and reproducible. Currently, we advise users to take advantage of these features and install a local instance of Galaxy even though it will be running then only on a single desktop PC (or even just a notebook). After installation of Galaxy our software simply copies its wrapper xmls over to the tools folder and modifies the tool_conf file to integrate itself. In addition, users have to install a bit of other third-party software (samtools, snap aligner, optionally snpeff) that our code relies on. [End of lengthy introduction] So my question is: in what way could the package profit from being uploaded to a Galaxy toolshed ? I guess it would mean quite some extra work from my side since I'm not familiar with the whole procedure, so are there benefits (visibility, ease of installation, etc.) that are worth the effort ? Thanks a real lot for any feedback, Wolfgang ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Fwd: Re: Move galaxy/database (SQLite) to a new location
oops - forget to keep it on the list Original Message Subject: Re: [galaxy-dev] Move galaxy/database (SQLite) to a new location Date: Wed, 04 Jun 2014 09:11:00 +0200 From: Hans-Rudolf Hotz h...@fmi.ch To: neil.burd...@csiro.au Hi Neil First a little comment: I strongly suggest to switch over to PostgreSQL. Keep in mind that the 'database' (in your case the SQLite database) does not contain any data. The 'database' contains just the meta data and the users data which is required to run Galaxy. #file_path = database/files is the location where the data sets are stored. You can copy the contents of ~/database/files/ to a new location and make the appropriate changes in 'universe_wsgi.ini' or set up a sym link. I hope this helps/clarifies Regards, Hans-Rudolf On 06/04/2014 04:49 AM, neil.burd...@csiro.au wrote: Hi, we have the default setup for galaxy hence using SQLite database and all data store in the same location as the galaxy install. In Universe_wsgi.ini file there is an entry # Dataset files are stored in this directory. #file_path = database/files How do we go about moving the database to a new location? We don't really want to delete the data already present in the database just move it to a new location. If we modify file_path and restart wouldn't that cause further problems as some data would be in one location and other in another location. Is it even possible to move the database once it has been created and has data in it? Thanks Neil ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Move galaxy/database (SQLite) to a new location
As far as I know, there is no reference to the path of the datafiles in the 'database' (no matter whether it is SQLite, MySQL, or PostgreSQL) - may be the Galaxy developers can confirm this? If you wanna be on the safe side, work with a sym link, and don't change the entry in the 'universe_wsgi.ini' file. Hans-Rudolf On 06/04/2014 09:14 AM, neil.burd...@csiro.au wrote: Thanks Hans-Rudolf, would this also be the same case for PostgreSQL database? i.e. we can move the database/files somewhere else , change universe_wsg.ini and all would be good? There is no link from the database to where the flat files are stored? cheers Neil From: Hans-Rudolf Hotz [h...@fmi.ch] Sent: Wednesday, June 04, 2014 5:11 PM To: Burdett, Neil (CCI, Herston - RBWH) Subject: Re: [galaxy-dev] Move galaxy/database (SQLite) to a new location Hi Neil First a little comment: I strongly suggest to switch over to PostgreSQL. Keep in mind that the 'database' (in your case the SQLite database) does not contain any data. The 'database' contains just the meta data and the users data which is required to run Galaxy. #file_path = database/files is the location where the data sets are stored. You can copy the contents of ~/database/files/ to a new location and make the appropriate changes in 'universe_wsgi.ini' or set up a sym link. I hope this helps/clarifies Regards, Hans-Rudolf On 06/04/2014 04:49 AM, neil.burd...@csiro.au wrote: Hi, we have the default setup for galaxy hence using SQLite database and all data store in the same location as the galaxy install. In Universe_wsgi.ini file there is an entry # Dataset files are stored in this directory. #file_path = database/files How do we go about moving the database to a new location? We don't really want to delete the data already present in the database just move it to a new location. If we modify file_path and restart wouldn't that cause further problems as some data would be in one location and other in another location. Is it even possible to move the database once it has been created and has data in it? Thanks Neil ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] How to unsubscribe - was: Re: RFC: Citations for tools
Hi Shari
Have a look at the last few lines of the e-mail:
... To manage your subscriptions to this
and other Galaxy lists, please use the interface
at: http://lists.bx.psu.edu/
Hans-Rudolf
On 05/28/2014 03:56 AM, Shari Javadiyan wrote:
Hi all,
Is there any way to unsubscribes from mail lists?
Cheers,
Shari
On Wednesday, 28 May 2014 2:52 AM, Jim McCusker
jmccus...@5amsolutions.com wrote:
I would suggest using as much as possible from PROV, especially since
other workflow engines (Taverna and Pegasus come to mind) already
support it. Rather than looking for bibtex mappings in XML, we should be
looking for vocabularies that represent the elements we need to
represent, and the relevant bibtex should be generated from that. PROV
and Dublin Core Terms can get us most of the way there, I think.
Jim
On Tue, May 27, 2014 at 1:09 PM, Peter Cock p.j.a.c...@googlemail.com
mailto:p.j.a.c...@googlemail.com wrote:
Hi Eric,
I was sure there was a Trello card for this, but I can't find it
right now...
I've pushed this idea on the mailing list before, and in person at
the Galaxy Community Conference too. See also these threads:
http://lists.bx.psu.edu/pipermail/galaxy-dev/2011-December/007873.html
http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-June/010178.html
Are you familiar enough with the area of semantic web/linked
data to know what would be the best XML based markup to
use for embedding the citations?
i.e. We should not reinvent the wheel here ;)
Peter
On Tue, May 27, 2014 at 5:54 PM, James Taylor ja...@jamestaylor.org
mailto:ja...@jamestaylor.org wrote:
Eric, I'm very much in favor of this feature, and particularly the
idea of generating a list of citations from a history or workflow. I
imagine the only thing to quibble about will be the syntax. There are
already some efforts to represent bibtex in xml (e.g.
https://github.com/Zearin/BibTeXML), however they have always struck
me as overly verbose.
-- jt
On Tue, May 27, 2014 at 12:45 PM, Eric Rasche
rasche.e...@yandex.ru mailto:rasche.e...@yandex.ru wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
I'd want to open up discussion on a feature I'd like to see.
I'll try
and implement it if I can find time this summer. I didn't see a
trello
card for anything like this yet, but please feel free to direct
me there
if I missed it.
I'd like to see citations as a part of every tool.
This would happen in the form of a citation block in the XML,
which
would contain sub-elements with text. These sub-elements could
be based
off of BibTeX, since they have existing specs for citing things:
https://en.wikipedia.org/wiki/BibTeX
These citations would then be accessible in the HTML generated tool
pages, or via a View/Download button somewhere on the tool page. By
storing as an XML tree, we could render these citations as BibTeX
entries for the LaTeX users, and I believe there are ways to convert
BibTeX to EndNote XML and so on.
This could be extended for use in workflows so that when you run
workflows, somehow a list of citations for all tools used could be
generated.
Anyone have thoughts or opinions on this?
Using the example bibtex entry from the wikipedia page:
@Book{abramowitz+stegun,
author= Milton {Abramowitz} and Irene A. {Stegun},
title = Handbook of Mathematical Functions with
Formulas, Graphs, and Mathematical Tables,
publisher = Dover,
year = 1964,
address = New York,
edition = ninth Dover printing, tenth GPO printing
}
I imagine it'd look like the following in a real-life tool:
tool
...
citation type=book
authorMilton Abramowitz and Irene A. Stegun/author
titleHandbook of Mathematical Functions with Formulas,
Graphs, and
Mathematical Tables/title
publisherDover/publisher
year1964/year
addressNew York/address
editionninth Dover printing, tenth GPO printing/edition
/citation
/tool
Cheers,
Eric
- --
Eric Rasche
Programmer II
Center for Phage Technology
Texas AM University
College Station, TX 77843
404-692-2048
e...@tamu.edu mailto:e...@tamu.edu
rasche.e...@yandex.ru mailto:rasche.e...@yandex.ru
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.17 (GNU/Linux)
iQIcBAEBAgAGBQJThMEUAAoJEMqDXdrsMcpVe9AP/1BjPbP5JdK6KDybOeV2ElvC
jvmUGetAjdQzkKO1Ikeuxb46yp0j4abAGGG92AccxlBYALsT3jsPv5dYjm505vcU
IwlfTBE7gc5y19x3zx1CuAd7PB11tryODz2LwXNKI75f39bNdX6Qe5aA74Vn/k8a
Re: [galaxy-dev] Galaxy queue
On 05/26/2014 10:00 AM, Peter Cock wrote: Hi Neil, Via the GUI, if you are logged in as an administrator (i.e. your email address is listed in universe_wsgi.ini) there is a an admin tab (top banner, starting analyze data etc). This includes an option to see the current jobs. Via the command line API? I'm not sure, never looked. connect to the PostgreSQL (or MySQL) database - prferably with a read-only user, and count the jobs with state new (ie grey) and running (ie yellow) in the job table. And based on the other information stored in the job table (eg 'tool_id'), you can make a guess on the expected wait time Regards, Hans-Rudolf Peter On Mon, May 26, 2014 at 6:59 AM, neil.burd...@csiro.au wrote: Hi, is there anyway to find out (from the command line or api) what is on the Galaxy queue? When items are queued they appear in grey or processing in yellow can I get hold of these details from the command line as I could then email the user who has just submitted a job with his place on queue and expected wait time. Thanks Neil ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] user specific access/options
Hi Emma Have a look at https://wiki.galaxyproject.org/Admin/Config/Access%20Control and https://wiki.galaxyproject.org/UserDefinedToolboxFilters as an alternative you could also do a bad hack using dynamic_options Regards, Hans-Rudolf On 05/19/2014 10:38 AM, Emma Prudent wrote: Hi, I am currently an intern and I work on galaxy, particularly on this acces problem. So I am very interested in the answers that could be given to this question. Thanks to you, Emma Prudent Hi Petr, I am going to forward your email to the galaxy-dev list so that the development community can offer comments/suggestions. Best, Jen Galaxy team On 8/30/11 2:27 AM, Petr Novak wrote: / Hi everybody, // I am developing the application on Galaxy server. One of the requirement // is to create user specific list of options. Is it possible to access // somehow $__user_email__ in options tag or in conditional ?. I did // not found documentation how to used cheetah in galaxy tool xml files but // from files provided with galaxy, cheetah is used only in command and // config tag. Is that rigth? If it can be used in any part of xml // definition file it would make much easier to generate xml dynamicaly // based on the $__user_email__ // Does anybody have an idea how to manage this problem? // Petr Novak/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Manual Tool and dependency installation
Hi Ravi To start, have a look at those wiki pages: https://wiki.galaxyproject.org/Admin/Config/ToolDependencies https://wiki.galaxyproject.org/Admin/Tools/ToolDependencies Regards, Hans-Rudolf On 05/14/2014 06:43 PM, Ravi Alla wrote: Hi All, Due to the nature of our cluster we are not allowed to have compilers on our webserver, which is where our galaxy instance runs. Because of this tool dependencies and tools that need to be compiled fail to install through the galaxy tool shed webpage. Is there a way for me to install dependencies and tools manually? For example I am trying to install the GATK2 wrapper. This depends on samtools-0.1.19 from iuc which in turn requires ncurses-5.9. Is there instructions somewhere as to where to install these tools such that they are tied to the GATK2 wrapper? Pardon me if this is a naive question, but I am really out of sorts here. Thank you Ravi ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] reports page download
Hi Neil I am not aware of any (hidden) download options. I usually go with 'copy-paste' and process in R or manually. Also, if I need specific information which is often not displayed on the reports page I get it directly out of the MySQL/PostgreSQL database. For such cases it is handy to have a read-only user (just to make sure you don't do something bad by accident) Hans-Rudolf On 05/15/2014 08:07 AM, neil.burd...@csiro.au wrote: Hi, is there a way to download the information found on the reports page so it can be used in excel for example? Thanks Neil ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Taking backups of my galaxy instance
Hi Nilaksha I don't really understand your question, but a simple answer is: Make a backup of the complete galaxy directory tree and make a backup of your PostgreSQL database. You should do this anyway on a regularly basis, if you use your Galaxy server in production. Since much more likely, you will run into a storage failure, than experiencing galaxy stop responding. Regards, Hans-Rudolf On 05/09/2014 06:52 AM, Nilaksha Neththikumara wrote: Hi all, I have installed and currently running a local galaxy instance successfully, customized with desired tools and datasets but suddenly got realized what would I do if anything goes wrong and galaxy stop responding :O. Is there a specific way to take backups of a locally installed galaxy instance with its own settings, so i can perform that task regularly. Thanks in advance :) Sincerely, Nilaksha. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] xml file display problem
Hi This is long shot, but maybe someone can help us We are in the process of upgrading our production galaxy server from (release_2013.11.04 to release_2014.04.14). Despite some hiccups it went very smooth (I might come back with those in a different mail thread next week). However, we are running into a display problem: A former co-worker has written a tool which generates a big xml file. When clicking on the 'eye' icon, we don't wan't display the complete file, but only part of it. For this he has written an xsl file (qProject.xsl). This file is placed in ~/galaxy-dist/static/ Correspondingly, the beginning of the xml file looks like: ?xml version=1.0? ?xml-stylesheet type=text/xsl href=../../../static/qProject.xsl? qProject version=0.1 etc. And (at least for me by magic), when you now click on the 'eye' icon a nicely formatted output was displayed, instead of the full xml file. This has been the case with the release_2013.11.04 distribution. Now with the release_2014.04.14 distribution (and also already with release_2014.02.10), this magic trick does not work anymore, and the complete, un-formatted xml file is displayed. I am aware, there has been major changes introduced for the displays with the release_2014.02.10 distribution. So, can anybody give me some hints how to get this working again. I am happy to provide more details, if required. Thank you very much Hans-Rudolf -- Hans-Rudolf Hotz, PhD Bioinformatics Support Friedrich Miescher Institute for Biomedical Research Maulbeerstrasse 66 4058 Basel/Switzerland ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] xml file display problem
Hi John Thank you very much for the explanation. I will discuss it with our sys-admin Hans-Rudolf On 05/08/2014 03:29 PM, John Chilton wrote: I broke your Galaxy and did so intentionally - I feel bad about that. The way Galaxy was serving out XML content allowed XSS attacks between users - http://dev.list.galaxyproject.org/using-svg-foreignObject-tags-can-circumvent-html-sanitization-td4663390.html. If you aren't running a public instance, one could just set serve_xss_vulnerable_mimetypes to True in your universe_wsgi.ini file to disable the changed behavior. There are some relevant Trello cards referenced in that e-mail for longer term fixes more appropriate for public servers or servers where you don't or cannot trust your users. -John On Thu, May 8, 2014 at 7:39 AM, Hans-Rudolf Hotz h...@fmi.ch wrote: Hi This is long shot, but maybe someone can help us We are in the process of upgrading our production galaxy server from (release_2013.11.04 to release_2014.04.14). Despite some hiccups it went very smooth (I might come back with those in a different mail thread next week). However, we are running into a display problem: A former co-worker has written a tool which generates a big xml file. When clicking on the 'eye' icon, we don't wan't display the complete file, but only part of it. For this he has written an xsl file (qProject.xsl). This file is placed in ~/galaxy-dist/static/ Correspondingly, the beginning of the xml file looks like: ?xml version=1.0? ?xml-stylesheet type=text/xsl href=../../../static/qProject.xsl? qProject version=0.1 etc. And (at least for me by magic), when you now click on the 'eye' icon a nicely formatted output was displayed, instead of the full xml file. This has been the case with the release_2013.11.04 distribution. Now with the release_2014.04.14 distribution (and also already with release_2014.02.10), this magic trick does not work anymore, and the complete, un-formatted xml file is displayed. I am aware, there has been major changes introduced for the displays with the release_2014.02.10 distribution. So, can anybody give me some hints how to get this working again. I am happy to provide more details, if required. Thank you very much Hans-Rudolf -- Hans-Rudolf Hotz, PhD Bioinformatics Support Friedrich Miescher Institute for Biomedical Research Maulbeerstrasse 66 4058 Basel/Switzerland ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Galaxy Error Message
Hi Brad
I don't you are using R to execute your R script
How do you call your R script, ie how does the command tag looks in the
tool definition file?
Also, why do you have a #!/bin/bash at the beginning of your R script?
Regards, Hans-Rudolf
On 04/27/2014 09:50 PM, Bradley Belfiore wrote:
I am new to galaxy and while trying to convert an R tool to a Galaxy
usable tool I keep getting:
/Users/bmbelfio/galaxy-dist/tools/pathview/pathview.R: line 1: syntax
error near unexpected token `('
/Users/bmbelfio/galaxy-dist/tools/pathview/pathview.R: line 1:
`args=(commandArgs(TRUE))'
any help as to how to remedy this would be greatly appreciated. The
first couple lines of my .R file are
#
#!/bin/bash
args-(commandArgs(TRUE))
Thanks,
Brad
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Galaxy Error Message
Please keep all replies on the list by using reply all
well, what happens if you run
Rscript_wrapper.sh
/Users/bbelfio1/galaxy-dist/tools/pathview/Pathview.R $genedata
$pathwayid $species $output
on the command line yourself (or rather as the user galaxy is running
as), replacing $genedata $pathwayid $species $output with meaningful
arguments
Hans-Rudolf
On 04/28/2014 03:32 PM, Bradley Belfiore wrote:
Thank you for your quick response, my command tag is:
command
interpreter=bashRscript_wrapper.sh
/Users/bbelfio1/galaxy-dist/tools/pathview/Pathview.R $genedata
$pathwayid $species $output
/command
So I believe Im calling R via the Rscript_wrapper.sh?
As far as the #!/bin/bash, not sure why I had that there, I took it out now.
thanks,
Bradley Belfiore
On Mon, Apr 28, 2014 at 9:25 AM, Hans-Rudolf Hotz h...@fmi.ch
mailto:h...@fmi.ch wrote:
Hi Brad
I don't you are using R to execute your R script
How do you call your R script, ie how does the command tag looks in
the tool definition file?
Also, why do you have a #!/bin/bash at the beginning of your R script?
Regards, Hans-Rudolf
On 04/27/2014 09:50 PM, Bradley Belfiore wrote:
I am new to galaxy and while trying to convert an R tool to a Galaxy
usable tool I keep getting:
/Users/bmbelfio/galaxy-dist/__tools/pathview/pathview.R: line 1:
syntax
error near unexpected token `('
/Users/bmbelfio/galaxy-dist/__tools/pathview/pathview.R: line 1:
`args=(commandArgs(TRUE))'
any help as to how to remedy this would be greatly appreciated. The
first couple lines of my .R file are
#
#!/bin/bash
args-(commandArgs(TRUE))
Thanks,
Brad
_
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/__search/mailinglists/
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] automatically adding fasta files to built-in databases and selecting files questions
Hi Nik Have a look at the Profile Annotations tool (~/tools/annotation_profiler/annotation_profiler.xml) and the file ~/tool-data/annotation_profiler_options.xml Regards, Hans-Rudolf On 04/19/2014 12:48 AM, Nikhil Joshi wrote: Hi Hans, Thanks for your response. I am trying to get just one part working right now, and that is the selecting of a database file. You suggest using a drill_down, but I can't seem to find good documentation on how to get it to work. I have been looking at the annotation_profiler tool to get ideas, but it still is unclear to me. Can you give me a simple example of how one would create the file that the drill_down would use and then how to use that file? - Nik. On Wed, Apr 9, 2014 at 1:13 AM, Hans-Rudolf Hotz h...@fmi.ch mailto:h...@fmi.ch wrote: Hi Nik I don't think there is a solution out of the box for all of this, but I will try to give you some pointers - others might have different or even better suggestions (my suggestion are very 'hacky'). Let me answer your questions in reverse order: - Instead of using Add another file, display all your files as checkboxes: type=select display=checkboxes multiple=true you then get a comma separated string with containing all the selected files one option to populate the checkboxes is using dynamic_options or from_file which both allow you to read the list of files stored in a extra table - A list with thousands of files to choose from is obviously not very handy. You can build a tree where you can open and collapse individual branches using type=drill_down together with from_file Or as an alternative to searching, you can use conditional and then only display part of the list. - you can write a little tool, which allows people to select a history item (or uploads a new file) which is then stored at a new location (outside the galaxy database directory). In a second step, this tool adds the item to the table which is read by dynamic_options or from_file. Using from_file would require a restart, but a tool reload. I hope you get a few ideas out of this Regards, Hans-Rudolf On 04/09/2014 04:34 AM, Nikhil Joshi wrote: Hi all, We have a local install of Galaxy that we use for testing for our bioinformatics courses. I am creating some new tool interfaces for the install. I want to create a tool that will allow a user to automatically add a fasta database to the built-in databases (without needing to reboot the server). Is this possible? Along with that issue is that the number of databases for this tool can number in the thousands. Is there a way to easily search the built-in databases for the database you are looking for, and then use that database in a tool? Right now, it is an enormous drop down list which is really annoying and unwieldy. Finally, these tools can take multiple input files, but I really don't like the way galaxy handles that where you have to continually click on Add another file. Is there a method or work-around in galaxy where you can have a multi-file select? - Nik. -- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.__edu/ http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 tel:530.752.2698 (w) _ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/__search/mailinglists/ http://galaxyproject.org/search/mailinglists/ -- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w) ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Migrate user data to a new server
Hi Simon I recommend the following (minimal) steps to migrate to a new server: - set up the new server using the SAME Galaxy version your old server is running. - copy the contents of the 'database' directory - make the site specific adjustment, e.g. universe_wsgi.ini - connect your new server to your existing PostgreSQL database - and start your new server, followed by an upgrade to the new Galaxy version Of course, there are many other points to take care of like the existing data libraries and other site specific changes only you know about. Hence I recommend to first work with a copy of your PostgreSQL database, test everything, and then make an rsync of the 'database' directory and switch to the real PostgreSQL database. This way, your user can continue working with the old server, while you test the new one. I hope this gives you a few ideas Regards, Hans-Rudolf On 04/15/2014 06:00 AM, Simon Gladman wrote: Hi all, I've tried looking in the mailing list for this to no avail. (Maybe my search terms were no good..) I administer a server with about 15 users. The server is failing and so I have set up a new server (with an updated OS, Galaxy version etc (galaxy versions: old - nov 2013, new - Feb 2014). Is there a method for migrating my users and their data across to the new server, without getting them to tell me their passwords Cheers, Simon. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] visualization button
Philip Using a the first column as label should work if you tick the box Include a third column as data point IDs? Regards, Hans-Rudolf On 04/09/2014 07:47 AM, Philippe Moncuquet wrote: Hi Hans-Rudolph, Ok so the syntax in the wrapper was not the problem. My file was tabular but the first column contained string and numbers whereas the second column contained only numbers. When I added a 3rd column with numbers to test I was able to see the button and generate some graphics. I wish It could have taken the first column as the label for the second but I understand that would produces unwanted behavior in other context. Anyway, all good, thanks for the help. Regards, Philip 2014-04-08 17:33 GMT+10:00 Hans-Rudolf Hotz h...@fmi.ch mailto:h...@fmi.ch: On 04/08/2014 02:05 AM, Philippe Moncuquet wrote: Hi Rudolph, Thanks for the answer. I understand that it would make no sense to allow generation of graphics on non relevant data. My data is in tabular format and that is the reason why I was surprise not to see that button. outputs data name=tabular_file label=report.tsv type=tabular format=tabular / /outputs I would re-write this like: outputs data name=tabular_file label=report.tsv format=tabular / /outputs Also, make sure your output is actually a table, ie it does consist of more than one (tab-separated) column. Hans-Rudolf I wouldn't be surprised if something was wrong in my syntax. Thanks for the help anyway. Cheers, Philip 2014-04-07 16:52 GMT+10:00 Hans-Rudolf Hotz h...@fmi.ch mailto:h...@fmi.ch mailto:h...@fmi.ch mailto:h...@fmi.ch: Hi Philip The appearance of the visualization button depends on the datatype of your output. E.g. when it is set to 'tabular', you should get the button allowing you to create a scatter plot. However, depending on your data, it might not make much sense to visualize it in 'Trackster', 'Circster' or as scatter plot. Regards, Hans-Rudolf On 04/07/2014 02:11 AM, Philippe Moncuquet wrote: Hi, I have developed a simple script that give some statistics on an assemble. It generates a tabular output. The visualization button is not available when I extend the output in the history pane. Is there any code that I need to put in my xml wrapper to get this button available ? Regards, Philip _ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ http://galaxyproject.org/__search/mailinglists/ http://galaxyproject.org/__search/mailinglists/ http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] automatically adding fasta files to built-in databases and selecting files questions
Hi Nik I don't think there is a solution out of the box for all of this, but I will try to give you some pointers - others might have different or even better suggestions (my suggestion are very 'hacky'). Let me answer your questions in reverse order: - Instead of using Add another file, display all your files as checkboxes: type=select display=checkboxes multiple=true you then get a comma separated string with containing all the selected files one option to populate the checkboxes is using dynamic_options or from_file which both allow you to read the list of files stored in a extra table - A list with thousands of files to choose from is obviously not very handy. You can build a tree where you can open and collapse individual branches using type=drill_down together with from_file Or as an alternative to searching, you can use conditional and then only display part of the list. - you can write a little tool, which allows people to select a history item (or uploads a new file) which is then stored at a new location (outside the galaxy database directory). In a second step, this tool adds the item to the table which is read by dynamic_options or from_file. Using from_file would require a restart, but a tool reload. I hope you get a few ideas out of this Regards, Hans-Rudolf On 04/09/2014 04:34 AM, Nikhil Joshi wrote: Hi all, We have a local install of Galaxy that we use for testing for our bioinformatics courses. I am creating some new tool interfaces for the install. I want to create a tool that will allow a user to automatically add a fasta database to the built-in databases (without needing to reboot the server). Is this possible? Along with that issue is that the number of databases for this tool can number in the thousands. Is there a way to easily search the built-in databases for the database you are looking for, and then use that database in a tool? Right now, it is an enormous drop down list which is really annoying and unwieldy. Finally, these tools can take multiple input files, but I really don't like the way galaxy handles that where you have to continually click on Add another file. Is there a method or work-around in galaxy where you can have a multi-file select? - Nik. -- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w) ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] visualization button
On 04/08/2014 02:05 AM, Philippe Moncuquet wrote: Hi Rudolph, Thanks for the answer. I understand that it would make no sense to allow generation of graphics on non relevant data. My data is in tabular format and that is the reason why I was surprise not to see that button. outputs data name=tabular_file label=report.tsv type=tabular format=tabular / /outputs I would re-write this like: outputs data name=tabular_file label=report.tsv format=tabular / /outputs Also, make sure your output is actually a table, ie it does consist of more than one (tab-separated) column. Hans-Rudolf I wouldn't be surprised if something was wrong in my syntax. Thanks for the help anyway. Cheers, Philip 2014-04-07 16:52 GMT+10:00 Hans-Rudolf Hotz h...@fmi.ch mailto:h...@fmi.ch: Hi Philip The appearance of the visualization button depends on the datatype of your output. E.g. when it is set to 'tabular', you should get the button allowing you to create a scatter plot. However, depending on your data, it might not make much sense to visualize it in 'Trackster', 'Circster' or as scatter plot. Regards, Hans-Rudolf On 04/07/2014 02:11 AM, Philippe Moncuquet wrote: Hi, I have developed a simple script that give some statistics on an assemble. It generates a tabular output. The visualization button is not available when I extend the output in the history pane. Is there any code that I need to put in my xml wrapper to get this button available ? Regards, Philip _ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/__search/mailinglists/ http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] visualization button
Hi Philip The appearance of the visualization button depends on the datatype of your output. E.g. when it is set to 'tabular', you should get the button allowing you to create a scatter plot. However, depending on your data, it might not make much sense to visualize it in 'Trackster', 'Circster' or as scatter plot. Regards, Hans-Rudolf On 04/07/2014 02:11 AM, Philippe Moncuquet wrote: Hi, I have developed a simple script that give some statistics on an assemble. It generates a tabular output. The visualization button is not available when I extend the output in the history pane. Is there any code that I need to put in my xml wrapper to get this button available ? Regards, Philip ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Problem downloading large processed datasets : galaxy local instance
This might be due to different version of Galaxy. Is the Job Command-Line listed? - Depending on the tool, you might find the path there. Hans-Rudolf :Do you have the will also find the path in the job command On 04/03/2014 09:03 AM, Nilaksha Neththikumara wrote: Hi, Thanks a lot for the answer. But I'm afraid that detail panel does not give me the path to the dataset. Have you encountered this problem before? Nilaksha On Thu, Apr 3, 2014 at 12:26 PM, Hans-Rudolf Hotz h...@fmi.ch mailto:h...@fmi.ch wrote: Hi Nilaksha Use the ! icon (View details) to get the full path to the data set, and the use 'cp' or 'scp' (if not on the same host) on your terminal. Hope this helps, Hans-Rudolf On 04/03/2014 07:27 AM, Nilaksha Neththikumara wrote: Hello all, I'm currently successfully running a pipeline for metagenomic analysis on my locally installed galaxy instance and everything seemed to good by far. But when I'm trying to download my processed blast xml output (I used blast+ wrappers for this) , by clicking the small disk button nothing happens. I tried several options. 1) clicked the 'disk' icon to initiate the downloading 2) right clicked the 'disk' icon and tried to use saved linked as, but a dialog box doesnot appear for saving 3) tried copying the link and used curl on bash terminal, but progress bar is stuck at 0%. I think this is happening because my blast output dataset is much larger (around ~10GB) , but if that's the case, how am I supposed to get the data out of galaxy any way?? Thank a lot in advance. Regards! Nilaksha _ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/__search/mailinglists/ http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Finding cruft in database/files/?
Hi Brian It is difficult to imagine where such files are coming from. As each file in ~/database/files/*/ has a corresponding entry in the 'dataset' table. If you have such files, then something went horribly wrong, eg by using two different databases on the same files system. To double check manually compare the list of files on the file system, with the contents of the 'dataset' table. Also, have a look at the ~/database/job_working_directory/ and ~/database/tmp/ directories. You might have some old, big files there using a lot of space. Regards, Hans-Rudolf On Mar 24, 2014, at 12:43 AM, Brian Claywell wrote: Daniel, I've been using the purge scripts in cron jobs for a long time now; unless I missed something on that page, what I'm looking for is something slightly different -- rather than purging datasets marked as old/deleted/whatever in the Galaxy db, I want to find files that *aren't* in the Galaxy db but still exist on the filesystem. On Sun, Mar 23, 2014 at 12:39 PM, Daniel Blankenberg d...@bx.psu.edu wrote: Hi Brian, Have a look at purging histories and datasets: https://wiki.galaxyproject.org/Admin/Config/Performance/Purge%20Histories%20and%20Datasets Thanks for using Galaxy, Dan On Mar 22, 2014, at 9:24 PM, Brian Claywell bclay...@fhcrc.org wrote: Is there an existing method for finding files in $GALAXY_DIR/database/files that have been orphaned due to errors, or leftover from testing, etc? I'm pretty sure ours is bigger than it ought to be, and I'd really like to look at a little audit of the files there vs the ones registered in Galaxy. Thanks! -- Brian Claywell, Systems Analyst/Programmer Fred Hutchinson Cancer Research Center bclay...@fhcrc.org ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Brian Claywell, Systems Analyst/Programmer Fred Hutchinson Cancer Research Center bclay...@fhcrc.org ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Question concerning the xml file for local tools with multiple output files.
Hi Lifeng The easiest way to execute your script will be to provide a wrapper script (written in your preferred language, eg Python, perl, etc). Call the wrapper script like this: commandwrapper $input $output1 $output2/command and define $output1 $output2 according to your needs: format=fasta, format=txt the wrapper will call your script and rename/move the output. Hope this helps, Regards, Hans-Rudolf On Mar 25, 2014, at 3:23 AM, galaxy-user-boun...@lists.bx.psu.edu galaxy-user-boun...@lists.bx.psu.edu wrote: From: Lifeng Lin linlif...@gmail.com Date: March 25, 2014 12:58:52 AM GMT+01:00 To: galaxy-u...@lists.bx.psu.edu Subject: Question concerning the xml file for local tools with multiple output files. Hi folks, I am trying to integrate some of my local Perl scripts into a downloaded instance of Galaxy. So far the script with a simple in_file to out_file format worked great, but I am having problems understanding how to treat scripts with multiple output files that share the same input argv. for example: the script run like this in command line: script name input_name output_name_base and two files are generated from this script: output_name_base.fasta and output_name_base.txt. I am at a loss of how these parameter should be represented in the xml format, especially how the outputs data name= tag should be filled, since in the command tag, there is only one $output. Any suggestions? thanks! #GalaxyNoobie ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Question concerning the xml file for local tools with multiple output files.
Hi Lifeng I am glad to hear it works WRT your thoughts about using a wrapper script for each tool: I agree it might help you to standardize your tools, however it also introduces an extra step, which needs to be taken care of if you want to change/upgrade your tool. Personally, I would only use a wrapper if it is necessary or it adds some benefits for the tool. Others on the list might have a different opinion? Hans-Rudolf On Mar 25, 2014, at 3:13 PM, Lifeng Lin wrote: works like a charm. Thank you! I start to wonder if i should use this wrapper approach on all scripts regardless of the original input-output format for standardized integration, and possible automated integrations in the future. On Tue, Mar 25, 2014 at 6:53 AM, Hans-Rudolf Hotz hansrudolf.h...@fmi.ch wrote: Hi Lifeng The easiest way to execute your script will be to provide a wrapper script (written in your preferred language, eg Python, perl, etc). Call the wrapper script like this: commandwrapper $input $output1 $output2/command and define $output1 $output2 according to your needs: format=fasta, format=txt the wrapper will call your script and rename/move the output. Hope this helps, Regards, Hans-Rudolf On Mar 25, 2014, at 3:23 AM, galaxy-user-boun...@lists.bx.psu.edu galaxy-user-boun...@lists.bx.psu.edu wrote: From: Lifeng Lin linlif...@gmail.com Date: March 25, 2014 12:58:52 AM GMT+01:00 To: galaxy-u...@lists.bx.psu.edu Subject: Question concerning the xml file for local tools with multiple output files. Hi folks, I am trying to integrate some of my local Perl scripts into a downloaded instance of Galaxy. So far the script with a simple in_file to out_file format worked great, but I am having problems understanding how to treat scripts with multiple output files that share the same input argv. for example: the script run like this in command line: script name input_name output_name_base and two files are generated from this script: output_name_base.fasta and output_name_base.txt. I am at a loss of how these parameter should be represented in the xml format, especially how the outputs data name= tag should be filled, since in the command tag, there is only one $output. Any suggestions? thanks! #GalaxyNoobie ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] error in workflow
Hi Milad The error message: database is locked might be a hint. Are you using a PostgreSQL database or just SQLite ? If the latter, you will often run in troubles when running more than one job at a time, due to file locking. Regards, Hans-Rudolf On 03/11/2014 07:41 PM, Milad Bastami wrote: Dear Galaxy Developers, I'm running galaxy local on ubuntu and trying to run a workflow on multiple datasets (separately). Occasionally when I try to run workflow on input dataset I get Unable to finish job error in some steps, in most time the problem will be solved when I run the workflow again, but this going to happen more frequently. I attached the error message here. I suspect it may be related to increase in number of datasets of the current history (1757 datasets with ~350GB size). I will appreciate your help Traceback (most recent call last): File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/lib/galaxy/jobs/runners/local.py, line 116, in queue_job job_wrapper.finish( stdout, stderr, exit_code ) File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/lib/galaxy/jobs/__init__.py, line 1068, in finish self.sa_session.flush() File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/scoping.py, line 114, in do return getattr(self.registry(), name)(*args, **kwargs) File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py, line 1718, in flush self._flush(objects) File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py, line 1804, in _flush transaction.commit() File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py, line 365, in commit t[1].commit() File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py, line 2045, in commit self._do_commit() File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py, line 2075, in _do_commit self.connection._commit_impl() File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py, line 1280, in _commit_impl self._handle_dbapi_exception(e, None, None, None, None) File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py, line 1277, in _commit_impl self.engine.dialect.do_commit(self.connection) File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/default.py, line 307, in do_commit connection.commit() OperationalError: (OperationalError) database is locked None None ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] History seems to load forever
Hi Carrie Was the history which is causing troubles created before you switched to external authentication? Or has the user a slightly different e-mail address now? eg different use of capital letters, like 'foo.bar@home' and 'Foo.Bar@home' - we had such problems, when we switched to external authentication. I had to dig into our MySQL database and fix the galaxy_user table. In your case, I recommend looking at the galaxy_user table and check whether the user has more than one entry. Take the id (ie the number) and search the history table using the id as 'user_id'. I would guess you can fix your problem by changing the 'user_id' in the history table for the history which is causing troubles. BUT, BE VERY CAREFUL! you can do a lot of damage to the database. Switch off the galaxy server first, and MAKE a back up the database before you change anything. Regards, Hans-Rudolf On 03/11/2014 11:58 PM, Ganote, Carrie L wrote: Hi List, I just pulled and merged from galaxy-dist (I think it's dc067a95261d), updated my database, and migrated my tools. The issues I am able to see are minor, but I have a user who can no longer access a particular history. It apparently just hangs. If they delete the history, Galaxy responds, but it will hang again when they try to add back any datasets. Unfortunately, I don't see any errors in the paster logs about this. I fear that their dataset entries in the Postgresql db have been corrupted during the update, or perhaps the files themselves were purged. I cannot reproduce their issues with my own histories - I'm able to bring up saved histories and upload etc. I don't know if they can share a history if it can't be loaded. How should I troubleshoot this? I'm set up to authenticate through apache, which makes it harder to get in as the user and see what's up. Is there an easy way to find out what histories belong to what users? I can search the database? Any advice would be greatly appreciated! -Carrie ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] can I change to PostgreSQL?
Have a look at https://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer#Switching_to_a_database_server on how to set up the PostgreSQL database for Galaxy. I am not aware of any tools to port the contents of the SQLite file to a PostgreSQL database, except for setting up an all new Galaxy server (running with PostgreSQL) and exporting the important histories from your old server followed by importing them into your new server - though looking at the size of your history, this might not work. Hans-Rudolf On 03/12/2014 09:57 AM, Milad Bastami wrote: Many thanks for your help. Your are right about database (I'm using SQLite). I've found a tutorial at GMOD (http://gmod.org/wiki/Galaxy_Tutorial_2012) on how to make galaxy work with PostgreSQL, but can I change the current database to PostgreSQL without missing my datasets? I am running a single-user galaxy instance. Cheers, Milad Date: Wed, 12 Mar 2014 09:07:00 +0100 From: h...@fmi.ch To: mi.bast...@live.com CC: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] error in workflow Hi Milad The error message: database is locked might be a hint. Are you using a PostgreSQL database or just SQLite ? If the latter, you will often run in troubles when running more than one job at a time, due to file locking. Regards, Hans-Rudolf PostgreSQL On 03/11/2014 07:41 PM, Milad Bastami wrote: Dear Galaxy Developers, I'm running galaxy local on ubuntu and trying to run a workflow on multiple datasets (separately). Occasionally when I try to run workflow on input dataset I get Unable to finish job error in some steps, in most time the problem will be solved when I run the workflow again, but this going to happen more frequently. I attached the error message here. I suspect it may be related to increase in number of datasets of the current history (1757 datasets with ~350GB size). I will appreciate your help Traceback (most recent call last): File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/lib/galaxy/jobs/runners/local.py, line 116, in queue_job job_wrapper.finish( stdout, stderr, exit_code ) File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/lib/galaxy/jobs/__init__.py, line 1068, in finish self.sa_session.flush() File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/scoping.py, line 114, in do return getattr(self.registry(), name)(*args, **kwargs) File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py, line 1718, in flush self._flush(objects) File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py, line 1804, in _flush transaction.commit() File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py, line 365, in commit t[1].commit() File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py, line 2045, in commit self._do_commit() File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py, line 2075, in _do_commit self.connection._commit_impl() File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py, line 1280, in _commit_impl self._handle_dbapi_exception(e, None, None, None, None) File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py, line 1277, in _commit_impl self.engine.dialect.do_commit(self.connection) File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/default.py, line 307, in do_commit connection.commit() OperationalError: (OperationalError) database is locked None None ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Galaxy database size and location
Hi Ravi I don't quite understand question. It looks like you are mixing up two different things? A few comments, which might clarify and help you: - the postgresql db does not store the data. It tracks the users, their jobs and their histories. Hence, it stays pretty small. - the actual data is stored in ~/galaxy_dist/database/files/ And this directory (or rather its numbered subdirectories) can grow pretty quickly - depending on the kind of jobs you run. - there are clean-up scripts which you can use to remove 'deleted' history items (ie the data), see: https://wiki.galaxyproject.org/Admin/Config/Performance/Purge%20Histories%20and%20Datasets Hope this helps, Hans-Rudolf On 03/06/2014 02:39 AM, Ravi Alla wrote: Hi fellow galaxy devs, I am trying to understand how to implement the galaxy database and get an idea of how big it could get. Currently we are running galaxy on a webserver, and want to have the postgresql db on locally mounted partition and not on an NFS partition. This limits us to around 100GB of storage for the db. We will create data libraries for users to load their data without copying to galaxy, so input files won't be duplicated. Is there anything we can do about the output files? Do these files need to end up in the database or can we put them on the NFS partition somewhere with the db holding information about their location? I noticed that on a routine small analysis I could easily have 20GB or more of output files and history and all this is in the database. If output files and history files are written to the database, are they cleaned up daily to avoid storage issues? Please advise. Thanks Ravi Alla ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] UCSC Display option
Hi Alper Have a look at ~/galaxy_dist/tool-data/shared/ucsc/ucsc_build_sites.txt check the line starting with main http://genome.ucsc.edu/cgi-bin/hgTracks?; and add mm10 do a restart and you should get the link to 'UCSC main' for data mapped to mm10 if that fixes the problem for UCSC, you can do the same for ensembl by fixing: galaxy_dist/tool-data/shared/ensembl/ensembl_sites.txt Regards, Hans-Rudolf On 02/26/2014 08:01 PM, Alper Kucukural wrote: Hi, When I map the reads with bowtie2 to hg19. I got the output like below and there are four display options. When I map the reads to mm10. I got only display in “IGB Local Web option. So, how can I fix this for mm10 or others and show all options? 101: Bowtie2 on data 4: aligned reads 91.6 KB format: bam, database: hg18 [bam_header_read] EOF marker is absent. The input is probably truncated. [samopen] SAM header is present: 45 sequences. 2500 reads; of these: 2500 (100.00%) were unpaired; of these: 1829 (73.16%) aligned 0 times 279 (11.16%) aligned exactly 1 display at UCSC main test display at Ensembl May 2009 display with IGV web current local display in IGB Local Web Best, Alper Kucukural, PhD Bioinformatics Core, University of Massachusetts Medical School 368 Plantation St.Room AS4.2067 Worcester, MA 01605-2324 Phone: 774-312-4493 E-mail: al...@kucukural.com ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Local Galaxy web layout
Hi Shenwiyn Have a look at: ~/galaxy_dist/database/compiled_templates/base_panels.mako.py Regards, Hans-Rudolf On 02/17/2014 08:04 AM, shenw...@gmail.com wrote: Galaxy developers: I want to change my local Galaxy web layout,for example,delete the top button of Visualization and Cloud or change their positions.Could anyone show me some details about this? Thank you very much. shenw...@gmail.com ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] moving galaxy instance to different location
Hi Ido Over the last few years, we have moved our production server to different hardware twice. And we also made copies of our production server in order to have the right development environment. It always work without much hassle, once we have taken care of the 'obvious', which usually was a combination of changing the absolute paths the configuration files and creating sym-links. As always, make sure you have a back-up of your PostgreSQL (or MySQL) database. Regards, Hans-Rudolf On 01/16/2014 02:43 PM, Ido Tamir wrote: Hi, I have to move my galaxy instance to a different file system location. Is there anything that I have to look out for besides the obvious like absolute paths in some data configs? thank you very much, ido ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Galaxy Reports link - registed users per month - does not work
Hi Liisa
This rings a bell
I assume you use a MYSQL database? and I guess other pages are broken as
well?
We are also using MySQL and I had to fix several lines in several files
in order to get all report pages to work.
To do the fixes, have a look at this old (but still valid) e-mail from
Assaf Gordon:
http://lists.bx.psu.edu/pipermail/galaxy-dev/2009-August/000641.html
I can send you all my fixed files next week
Regards, Hans-Rudolf
On 01/17/2014 05:38 PM, Liisa Koski wrote:
Hello,
First..I love the Galaxy Reports tool...thanks so much for this.
I did however come across an error when trying to few the Registered
Users per month link. It would be really nice to have this working :)
Thanks in advance,
Liisa
URL: http://domain:9001/users/registered_users_per_month
http://martin.dnalandmarks.ca:9001/users/registered_users_per_month
Module paste.exceptions.errormiddleware:*144* in __call___
__ _
http://martin.dnalandmarks.ca:9001/users/registered_users_per_month#app_iter*=*
self*.*application*(*environ*,* sr_checker*)*
Module paste.debug.prints:*106* in __call___
__ _
http://martin.dnalandmarks.ca:9001/users/registered_users_per_month#environ*,*
self*.*app*)*
Module paste.wsgilib:*543* in intercept_output_
__ _
http://martin.dnalandmarks.ca:9001/users/registered_users_per_month#app_iter*=*
application*(*environ*,* replacement_start_response*)*
Module paste.lint:*170* in lint_app_
__ _
http://martin.dnalandmarks.ca:9001/users/registered_users_per_month#iterator*=*
application*(*environ*,* start_response_wrapper*)*
Module paste.recursive:*84* in __call___
__ _
http://martin.dnalandmarks.ca:9001/users/registered_users_per_month#*return*self*.*application*(*environ*,*
start_response*)*
Module paste.httpexceptions:*633* in __call___
__ _
http://martin.dnalandmarks.ca:9001/users/registered_users_per_month#*return*self*.*application*(*environ*,*
start_response*)*
Module galaxy.web.framework.base:*132* in __call___
__ _
http://martin.dnalandmarks.ca:9001/users/registered_users_per_month#*return*self*.*handle_request*(*
environ*,* start_response*)*
Module galaxy.web.framework.base:*190* in handle_request_
__ _
http://martin.dnalandmarks.ca:9001/users/registered_users_per_month#body*=*
method*(* trans*,* kwargs*)*
Module galaxy.webapps.reports.controllers.users:*29* in
registered_users_per_month_
__ _
http://martin.dnalandmarks.ca:9001/users/registered_users_per_month#*for*row*in*q*.*execute*():*
Module sqlalchemy.sql.expression:*2841* in execute
Module sqlalchemy.engine.base:*2453* in _execute_clauseelement
Module sqlalchemy.engine.base:*1584* in _execute_clauseelement
Module sqlalchemy.engine.base:*1698* in _execute_context
Module sqlalchemy.engine.base:*1691* in _execute_context
Module sqlalchemy.engine.default:*331* in do_execute
Module MySQLdb.cursors:*173* in execute
Module MySQLdb.connections:*36* in defaulterrorhandler*
OperationalError: (OperationalError) (1305, 'FUNCTION
galaxy_production.date_trunc does not exist') 'SELECT date_trunc(%s,
date(galaxy_user.create_time)) AS date, count(galaxy_user.id) AS
num_users \nFROM galaxy_user GROUP BY date_trunc(%s,
date(galaxy_user.create_time)) ORDER BY date DESC' ('month', 'month')*
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Avoid integrated_tool_panel.xml update
Hi Christian Have a look at the file: .hgignore Regards, Hans-Rudolf On 01/16/2014 03:23 AM, Cristian Alejandro Rojas wrote: Hi all, There is a setting on somewhere to skip the integrated_tool_panel.xml file update? Best regards, -- *Cristian Alejandro Rojas Quintero* /*Estudiante Ingeniería de Sistemas */ /*Universidad Distrital Francisco José de Caldas*/ Bogotá - Colombia ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] allow_user_impersonation (was: creating a hierarchy of students teachers and courses)
Hi I am sorry for hijacking someone else's thread: 3. teachers having read access to students’ work This one is why I would give the teachers admin access - I think they could impersonate the students to access their work (set allow_user_impersonation = True in universe_wsgi.ini). It would mean that every teacher would have full access to every students work - if this is a problem you might want to consider standing up a Galaxy instance per class - there is some up front overhead with this - but it would make cleaning things up really easy. allow_user_impersonation does not work when using external authentication. Are there any fixes for this in progress? If not shall I create a Trello card? Regards, Hans-Rudolf ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Careful data migration
Hi Sebastian I am also not aware of any such documentation. It is probably impossible to write it, as each such situation is different. I don't quite understand what your 'script' is doing, but in your situation, I recommend the following: First of all, make sure you have a backup of your PostgreSQL (or MySQL) database. Don't make any changes to 'universe.wsgi', but move the bulky directories to new locations on the NFS share and replace them with symbolic links. This has also the advantage, that you can do it step by step (ie directory by directory). Regards, Hans-Rudolf On 01/13/2014 05:33 PM, Sebastian Schaaf wrote: Hi @ all, I would like to know if anyone could give me some guidance or hint on how to migrate data 'correctly' if Galaxy already wrote files, worked on the database etc. The situation is that we set up an instance, it was already in use and wrote files e.g. into the directory 'galaxy-dist/database/'. The local space is quite small, but more storage is available via a NFS share. I already wrote a bash script for setting up the connections (and editing the 'universe.wsgi' file accordingly) for the entries 'genome_data_path, 'ftp_upload_dir', 'file_path', 'new_file_path' and 'job_working_directory'. Those should address the most bulky features... The application of that script is fine as long as the Galaxy system has never started before. While applying that on a system that already 'did something', e.g. the files in the 'database/' subdirectory remain there and are not transferred. Anyhow, if Galaxy is started up after the application of the new settings, no error is reported. I wonder how Galaxy now deals with that situation: * Handling data from both sources? (read and/or write?) * Automatic movements of the 'historic' data when touched the next time? * Crashes if those older objects are intended to be read/edited? * Removes them silently from the database? = Is there a procedure/module to used in order to migrate the data? Is it sufficient (and appropriate) just to move all contents of the old folders to the new locations? Did I miss any existing documentation on that issue? The answer(s) may diverge, depending on to which of the five parameters announced above those questions are addressed... Any help appreciated before blowing up our production instance :). Thanks in advance, Best regards, Sebastian ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Quota problem
Hi Thomas Just double checking: what value are you using for the 'days' option? If you use the default (which is 60 days if I remember correctly), only data sets which have been 'deleted' more than 60 days ago will actually be removed. Regards, Hans-Rudolf On 01/14/2014 09:49 AM, Thomas Bellembois wrote: Hello, I am Thomas, IT engineer at the IGFL in France (http://igfl.ens-lyon.fr/). We are running our own Galaxy instance in front of an OGE cluster. Users quota never decrease when they delete items in their history. I have run the scripts in the cleanup_datasets directory but without success. I have no errors in the logs. Any idea ? Regards, Thomas ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] funny bug with latest galaxy release when using IE
Hi Ambrose simple answer: don't use Explorer and use Firefox or chrome instead. I have tested the Nov. 4th release before updating our production server and it worked on explorer 9 (on Windows 7). Nevertheless, I have been running into the same issue during our Galaxy training course this week. Again, the simple fix was to install Firefox Hans-Rudolf On 12/10/2013 06:28 PM, Ambrose Andongabo (RRes-Roth) wrote: Dear All, We get a very annoying behaviour using the latest release(04 November) of galaxy on Internet explorer. We tested it on I.E version 9 and version 10 and we get the same behaviour as you can see from the screenshot below I hope there is somebody who knows why this is happening Thanks Ambrose -- This message has been scanned for viruses and dangerous content by *MailScanner* http://www.mailscanner.info/, and we believe but do not warrant that this e-mail and any attachments thereto do not contain any viruses. However, you are fully responsible for performing any virus scanning. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] galaxy local installation
Hi Edgar please find my comments within your text below: On 12/11/2013 06:30 PM, fernandez Edgar wrote: Hello, My name is Edgar Fernandez. I’m a sys. admin. at University of Montreal. I’ve been ask by a teacher to check out Galaxy. So I’ve installed locally on my computer galaxy (I’m running openSUSE x86_64 but I’ll install it on SUSE Linux ia64). I’ve executed the run.sh script and I’ve somewhat successfully made the website work. I say somewhat because I have a couple of issues: 1.The website is not well displayed (refer to the attachment) just double checking: have you reloaded your page? 2.I’ve changed the value of host to 0.0.0.0 in the universe_wsgi.ini file and it still launches to 127.0.0.1 127.0.0.1 seems correct to me, as this is the loopback address (ie 'localhost'. I was also wondering if there is a step by step tutorial to install all the dependencies? I’m finding myself overwhelmed with all the different dependencies. well, I recommend, you first make a list of the tools you want to offer in Galaxy, and then figure out which dependencies you actually need. And then use these wiki pages: https://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies https://wiki.galaxyproject.org/Admin/Tools/Tool%20Dependencies Regards, Hans-Rudolf Cordialement / Regards, ** *Edgar Fernandez* System Administrator (Linux) Direction Générale des Technologies de l'Information et de la Communication ( Bur. : *1-514-343-6111 poste 16568**//* *//* */Université de Montréal/**//* PAVILLON ROGER-GAUDRY, bureau X-218 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] survey
On 12/02/2013 11:14 PM, Philippe Moncuquet wrote: Hi, I hope this is relevant to the dev list but if it is not please let me know. I was trying to estimate active users out of our registered users. I ended counting users that had a job during the last month. How would you guys do ? I assume, you are also using the 'report tools'? In general it is difficult to do the estimate, ie what is the difference between a user who does 5 jobs every month and a user who does 50 jobs at once and then never uses Galaxy again? Hence, I would count the total number of jobs. Although, again, the number could be misleading, as there are 'big' jobs and 'small' jobs which use different amounts of compute resources, and so on During this process I also wanted to see how we were doing in comparison to other instance. I set up a simple survey and would greatly appreciate if instance admin would take 2 min to complete it. I will publish results here (or maybe to some more relevant place, let me know) at the end of the month. https://fr.surveymonkey.com/s/CD6NN88 done. Regards, Hans-Rudolf Regards, Philip ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Lock Galaxy job submission
Hi Peter No, you are not imagining things. It has been removed. I guess, this was the commit: https://bitbucket.org/galaxy/galaxy-central/commits/445b212b1c6d/ Though, we used this features very often in the early days, I didn't notice it was gone (maybe because, I have turned harsh and just kill jobs when people don't listen to announcements about scheduled a system shutdown). Nevertheless, I have added a Trello card https://trello.com/c/BTDaHy9m/1269-re-institute-job-lock-feature Hans-Rudolf On 11/20/2013 03:31 PM, Peter Cock wrote: Hello all, It is my recollection that under the Admin panel, Manage jobs page, there used to be a toggle control to lock the submission of new jobs. This was very helpful in preparation for scheduling a system shutdown (e.g. for hardware or software updates). However, I no longer see this option on either galaxy-central or galaxy-dist. Was it removed, or am I imagining things - in which case should I file an enhancement request ;) (I've had a look in Trello and didn't find anything) Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Trouble setting up a local instance of Galaxy
Hi Starr I will try to answer some of your questions. Next time I recommend to ask just one question (with a corresponding subject line). This will make it much easier for the community to help you, and for others with similar problems to find the correct e-mail thread. see below for my answers... On 11/06/2013 11:43 PM, Hazard, E. Starr wrote: Hello, I am a new user of Galaxy. I have a Galaxy instance running (sort of) on a local research cluster. I issued the command hg update stable” today and it retrieved no files. SO presume I am up-to-date on the stable release. I start the instance as a user named “galaxy”. Right now I am still running in “local” mode. I hope to migrate to DRMAA LSF eventually. I have tried to set up ProFTP to upload files but have not succeeded so I use Galaxy Web-upload. The upload was working nicely and I had added a couple of new tools and they were working with the uploaded files. Getting LSF/DRMAA to work was giving me fits and ultimately I deleted all my history files in an effort to start over. Presently, files being uploaded appear in history as say job 1 ( in a new history) The job status in the history panel of the web GUI changes from purple to yellow then then to red indicating some sort of error. There is no viewable error text captured, but I can click on the “eye” icon and see the first megabyte of the data (for tiny files I can see the entire content and it’s intact). In the Galaxy file system, however, these files appear but have a different number , say, dataset_399.dat On my system the uploaded files appear in /PATH/galaxy-dist/database/files/000 My first question is why is the data going into the “000” subdirectory and not one “owned’ by the user who is uploading? all files are owned by galaxy (independent of who is uploading them and/or generate new files). The first 1000 files are stored in '000', the next 1000 files in '001', and so on. Access permission is handled by the SQLite database (unless you have already switched to a PostgreSQL database). My second question is why is the dataset being labeled as dataset_399.dat and not dataset_001.dat? The number is given by the SQLite database. When you want a clean start, you need to remove all files from ~/database/files/000 and start with an empty database. My third question is why do the uploaded files not appear as selectable options ( say I have paired-end fastq files and tool wants to have choices about filenames)? This problem is present for programs that seek one input file as well. When you uploaded the files, was the 'format' properly recognized? I presume that Galaxy is confused because the numbering in history is not the same as the numbering in the file upload archive (e.g. /PATH/galaxy-dist/database/files/000 in my case) so my last question is how do I “reset” my system to get the dataset and history numbers to be the same? The numbering in the history has nothing to do with the file numbering. See above wrt my comment about a 'clean start' Regards, Hans-Rudolf Here’s how I launch the Galaxy instance sh /shared/app/Galaxy/galaxy-dist/run.sh -v --daemon --pid-file=Nov6Localdaemon.pid.txt --log-file=Nov6Local1639daemon.log.txt Entering daemon mode Here are the last lines of the log Starting server in PID 26236. serving on 0.0.0.0:8089 view at http://127.0.0.1:8089 galaxy.tools.actions.upload_common DEBUG 2013-11-06 16:48:49,624 Changing ownership of /shared/app/Galaxy/galaxy-dist/database/tmp/upload_file_data_QZGHm4 with: /usr/bin/sudo -E scripts/external_chown_script.py /shared/app/Galaxy/galaxy-dist/database/tmp/upload_file_data_QZGHm4 hazards 502 galaxy.tools.actions.upload_common WARNING 2013-11-06 16:48:49,750 Changing ownership of uploaded file /shared/app/Galaxy/galaxy-dist/database/tmp/upload_file_data_QZGHm4 failed: sudo: no tty present and no askpass program specified galaxy.tools.actions.upload_common DEBUG 2013-11-06 16:48:49,751 Changing ownership of /shared/app/Galaxy/galaxy-dist/database/tmp/tmpEsyGfO with: /usr/bin/sudo -E scripts/external_chown_script.py /shared/app/Galaxy/galaxy-dist/database/tmp/tmpEsyGfO hazards 502 galaxy.tools.actions.upload_common WARNING 2013-11-06 16:48:49,775 Changing ownership of uploaded file /shared/app/Galaxy/galaxy-dist/database/tmp/tmpEsyGfO failed: sudo: no tty present and no askpass program specified galaxy.tools.actions.upload_common INFO 2013-11-06 16:48:49,805 tool upload1 created job id 170 galaxy.jobs DEBUG 2013-11-06 16:48:50,678 (170) Persisting job destination (destination id: local) galaxy.jobs.handler INFO 2013-11-06 16:48:50,698 (170) Job dispatched galaxy.jobs.runners.local DEBUG 2013-11-06 16:48:50,994 (170) executing: python /shared/app/Galaxy/galaxy-dist/tools/data_source/upload.py /depot/shared/app/Galaxy/galaxy-dist /shared/app/Galaxy/galaxy-dist/database/tmp/tmpTq22ot /shared/app/Galaxy/galaxy-dist/database/tmp/tmpEsyGfO
Re: [galaxy-dev] browse directories on galaxy server
Hi David Using Data Libraries, as suggested by Martin is definitely the way to go. Especially if you want to use the files with different tools. However, if you have a predefined set of tools, you might wanna consider adding an extra tools which reads and process the files, eg: you have your raw fastq files in a central location and you always do the same pre-processing steps. For such a case, I recommend using the dynamic_options attribute. This allows you to present a list of files, which you can then select from and do the processing on. May I refer to a similar question for the details on how to implement this: http://tinyurl.com/6uom4uo (I can give you more details if interested) Of course, you can go to the extreme case, and just trigger a copy of the selected file into your history. But this will end up in data duplication and you wanna avoid this. Regards, Hans-Rudolf On 11/01/2013 01:46 AM, Martin Čech wrote: Hello David, please see whether Data Libraries (http://wiki.galaxyproject.org/Admin/DataLibraries/Libraries) with the file import (http://wiki.galaxyproject.org/Admin/DataLibraries/UploadingLibraryFiles) and especially the /Upload files from filesystem paths/ feature do solve your problem. It enables you to work with the datasets while keeping them untouched on the file system. thank you for using Galaxy! regards Martin On Thu, Oct 31, 2013 at 3:05 PM, David Hoover hoove...@helix.nih.gov mailto:hoove...@helix.nih.gov wrote: Is there any desire to create a way in galaxy to browse local directories on the galaxy server itself? Not the client machine running the web browwser, but the machine on which galaxy is running. I've created a tool that allows a user to upload a single file from their own personal directory on the galaxy server into galaxy directly, without going through Apache. However, this is very limiting. It would be very helpful to be able to browse directories that are deemed accessible to the galaxy user, and before I begin trying to develop anything I'd like to know if there is any support for such a feature. David Hoover Helix Systems Staff National Institutes of Health ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] [galaxy-user] Inquiring
Hi Yan Assuming you have copied everything to the new server and you have changed the host = xxx.xxx.xxx.xxx line in the 'universe_wsgi.ini' file it should all work Regards, Hans-Rudolf On 10/31/2013 04:36 PM, Yan Luo wrote: Dear All, We have a question regarding the Galaxy server. We installed a local version on our Linux machine and need move to a new place. The IP address will be changed. Could you please let me know if the server can be still used? I mean, we can use the new IP to open the home page of Galaxy and use it. For example if the new IP is xxx.xxx.xxx.xxx, which is similar as before(I just replace the IP address). Do we need reset anything for Galaxy? Will the Galaxy use the fixed IP address in the system when it was installed? xxx.xxx.xxx.xxx:8080/root Looking forward to hearing from you. Best Wishes, Yan On Thu, Feb 10, 2011 at 10:01 AM, Nate Coraor n...@bx.psu.edu mailto:n...@bx.psu.edu wrote: Hi Yan, I've moved this discussion to the galaxy-dev list since it pertains to a local installation of Galaxy. Responses to your questions follow, in-line. Yan Luo wrote: Dear Sir, (1)We installed Galaxy, but recently the user can't registered and got the following error, how can we fix it? Sever error An error occurred. See the error logs for more information.(To turn debug on to display ...). Since debug = False in universe_wsgi.ini, you should be able to find a more detailed error message in the log file. If starting Galaxy with: % sh run.sh --daemon The default log file is 'paster.log' in Galaxy's root directory. (2) Could you please let me know if there is any command to stop galaxy? If starting with the --daemon flag (as above), you can use: % sh run.sh --stop-daemon If running in the foreground, you can use Ctrl-C to terminate the process. There is a recent bug whereby Ctrl-C is ineffective on some platforms under Python 2.6 - in this case you will have to kill/pkill the process manually. We are working on a fix for the latter. (3) If I reset universe_wsgi.ini file and want to set an administrator user(I can add a line in the above file), how can I get the password? Should I stop galaxy(See question 2) first? then run ./setup.sh and ./run.sh. setup.sh would have only been necessary prior to running Galaxy the first time, however, this step has recently been removed. If you are referencing documentation that still refers to setup.sh, please let us know so we can update it - I did notice this was still on the Production Server page, so I removed it from there. You no longer need to run setup.sh at all. (4) If I run setup.sh, will a new file universe_wsgi.ini be generated? if I want to change this file,should I edit it before run.sh and after setup.sh. Is it right? setup.sh and its replacements in run.sh and the Galaxy application itself never overwrite files, they only create files from sample files if they do not exist. (5) I read some of your docs, command sh setup.sh(sh run.sh) and ./setup.sh(./run.sh), which one is correct under Linux? Both syntaxes are effectively the same in most cases. --nate Looking forward to hearing from you. Best Wises, Yan Luo, Ph.D. NIH http://int.ask.com/web?siteid=1861webqsrc=999l=disq=By%20the%20way, ___ galaxy-user mailing list galaxy-u...@lists.bx.psu.edu mailto:galaxy-u...@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-user ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] How we found the filenames of all of the Galaxy datasets
Hi Yaron This is useful - Thank you very much. Coming up with such a sql statement to identify the files from people who have left our institute has been on my list of things to do for quite some time. Now I just need to figure how to distinguish the files which have been shared with people who are still here. Regards, Hans-Rudolf PS: for those of us who got stuck with a MySQL database, this is the corresponding select statement: SELECT CONCAT(substring(lpad(history_dataset_association.dataset_id, 6, 00) from 1 for 3), '/dataset_', history_dataset_association.dataset_id, '.dat') AS file, On 10/30/2013 10:00 AM, Yaron de Leeuw wrote: Hi. During IT housekeeping, we have decided that we must know the filenames of all of the galaxy datasets in our local deployment. Searching the docs and the web yielded no reasonable answer, so we figured we'll just write our own tool. At first, we thought about a Galaxy tool, but found no way to extract all the datasets of all histories. So we went deeper. To the SQL itself. Without further ado, I give you the query: SELECT substring(to_char(history_dataset_association.dataset_id, '00') from 2 for 3) || '/dataset_' || history_dataset_association.dataset_id || '.dat' , history_dataset_association.create_time, history_dataset_association.name, history_dataset_association.extension, history.name AS history_name,galaxy_user.username FROM history_dataset_association JOIN history ON history_dataset_association.history_id = history.id JOIN galaxy_user ON history.user_id = galaxy_user.id WHERE history_dataset_association.deleted = false AND history_dataset_association.purged = false ORDER BY username, history_name, create_time; We use PostgreSQL, so to run this you save the query as `get_filenames.sql`, and run: psql -U galaxy galaxy_prod -f get_filenames.sql -A galaxy_export_filenames.csv This prints out a line structured as: 003/dataset_3748.dat|2012-10-09 12:07:37.786751| index.fastq|fastq|nan|sean Which means: folder/filename | time of creation | name as appears in history | format | history name | history owner Only files that were not deleted in galaxy are printed (you can remove the WHERE line if you want them all). By the way, one of the main struggles was finding the folder/filename. We have searched the database repeatedly for a filename field to no avail, until someone recognised the fact that they can be derived from the dataset_id. If you zero pad the id to six chars, the first three chars would be your folder name. The whole id is the file name. e.g.: dataset_id = 12345 folder = 012 filename = dataset_12345.dat Yaron. Search engine candies: export migrate files datasets filenames on disk ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] about the sqlalchemy-migrate
Hi ngsf...@hygenomics.com You ask four different questions. I strongly recommend you ask them in four different mails each with an appropriate subject line, next time. This will make it much easier for the community to help you. Also, copy paste the logs and don't use screenshots, as they will get lost in the reply. On 10/14/2013 10:08 AM, ngsf...@hygenomics.com wrote: hi,all: 1. when I started the galaxy daemon. In the paster.log has a warning.who can help me. url= http://code.google.com/p/sqlalchemy-migrate/ When you start galaxy a second time, do you still get the error? Can you manually fetch the egg? 2. missing file. It is just a warning: You don't provide the index files. Do you really need all those tools? This page will probably help you: http://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup 3.What should I do about it? You don't provide an job_conf.xml file. It still works, as explained in the warning message. Altenatively, have a look at ~/galaxy_dist/job_conf.xml.sample_advanced ~/galaxy_dist/job_conf.xml.sample_basic 4.What should I do about it? Again, just warnings. You need to investigate the individual tool xml files Hope this helps Regards, Hans-Rudolf ngsf...@hygenomics.com ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Tabular data not displaying in main panel
Hi Peter and Bjoern
we had similar problems in August, when we upgraded to
'release_2013.06.03'. The problem has disappeared now.
At the time I had the suspicion, it only happened to browser sessions
which were already open before the upgrade, Hence, closing the browser,
and starting again helped. But this was just a guess.
Regards, Hans-Rudolf
On 10/02/2013 08:48 PM, Björn Grüning wrote:
Hi Peter,
we can confirm. Latest Galaxy stable version and its occurring randomly.
I was not able to reproduce it by myself.
Cheers,
Bjoern
Hi all,
Multiple users of our internal Galaxy server have reported with
multiple browsers that tabular data is not displayed in the main
central panel. The preview in the history works, but clicking on
the eye icon gives a blank page.
I can confirm this in Firefox 24.0 and Safari 6.0.5, the HTML
for the central frame looks blank but is actually a script when
viewed via the frame source. The tabular file data is there in
the script's first_data_chunk entry - but does not get shown.
An except is shown below.
Has anyone else noticed this problem? I believe it began
when I last updated this Galaxy instance, but was not
noticed immediately.
Thanks,
Peter
---
I believe we are running the current stable release, or at least
the latest commit on the default branch from galaxy-dist
$ hg pull
warning: bitbucket.org certificate with fingerprint
24:9c:45:8b:9c:aa:ba:55:4e:01:6d:58:ff:e4:28:7d:2a:14:ae:3b not
verified (check hostfingerprints or web.cacerts config setting)
real URL is https://bitbucket.org/galaxy/galaxy-dist
pulling from http://bitbucket.org/galaxy/galaxy-dist
searching for changes
no changes found
$ hg update
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg branch
default
[galaxy@ppserver galaxy-dist]$ hg log -b default | head
changeset: 10411:c42567f43aa7
user:greg
date:Mon Aug 19 13:19:56 2013 -0400
summary: Filter invalid objects when generating the list of
repository_dependencies objects that are associated with a tool shed
repository installed into Galaxy.
changeset: 10410:e9ee9c5d30ae
user:Dave Bouvier d...@bx.psu.edu
date:Mon Aug 19 13:08:45 2013 -0400
summary: Remove duplicate method from the install and test's
functional_tests.py.
...
---
!DOCTYPE HTML
html
!--base.mako--
head
titleDataset Display/title
meta http-equiv=Content-Type content=text/html; charset=utf-8 /
link href=/galaxy/static/style/base.css?v=1380731103
media=screen rel=Stylesheet type=text/css /
script src=/galaxy/static/scripts/libs/jquery/jquery.js?v=1380731103
type=text/javascript/script
script src=/galaxy/static/scripts/libs/jquery/jquery.migrate.js?v=1380731103
type=text/javascript/script
script src=/galaxy/static/scripts/libs/jquery/select2.js?v=1380731103
type=text/javascript/script
script src=/galaxy/static/scripts/libs/bootstrap.js?v=1380731103
type=text/javascript/script
script src=/galaxy/static/scripts/libs/underscore.js?v=1380731103
type=text/javascript/script
script src=/galaxy/static/scripts/libs/backbone/backbone.js?v=1380731103
type=text/javascript/script
script
src=/galaxy/static/scripts/libs/backbone/backbone-relational.js?v=1380731103
type=text/javascript/script
script src=/galaxy/static/scripts/libs/handlebars.runtime.js?v=1380731103
type=text/javascript/script
script src=/galaxy/static/scripts/galaxy.base.js?v=1380731103
type=text/javascript/script
script
src=/galaxy/static/scripts/templates/compiled/template-popupmenu-menu.js?v=1380731103
type=text/javascript/script
script src=/galaxy/static/scripts/mvc/ui.js?v=1380731103
type=text/javascript/script
script type=text/javascript
// console protection
window.console = window.console || {
log : function(){},
debug : function(){},
info: function(){},
warn: function(){},
error : function(){},
assert : function(){}
};
// Set up needed paths.
var galaxy_paths = new GalaxyPaths({
root_path: '/galaxy/',
image_path: '/galaxy/static/images'
});
/script
script src=/galaxy/static/scripts/libs/require.js?v=1380731103
type=text/javascript/script
script
src=/galaxy/static/scripts/templates/compiled/helpers-common-templates.js?v=1380731103
type=text/javascript/script
script type=text/javascript
require.config({
baseUrl: /galaxy/static/scripts,
shim: {
libs/backbone/backbone: { exports: Backbone },
libs/backbone/backbone-relational: [libs/backbone/backbone]
}
});
require(['mvc/data'], function(data) {
data.createTabularDatasetChunkedView(
_.extend( {misc_blurb: 7,891 lines, peek:
table cellspacing=\0\
Re: [galaxy-dev] Dynamic data library
On 10/01/2013 07:12 PM, Cole, Nathan (NIH/NCI) [C] wrote: One final question as I dive into looking at these two methods: can you expose whole hierarchies and directories using the from_file method or will this only work on an individual sample basis? yes, you can. as an example have a look at the affymetrix cel files we offer. See attachment for a screen shot and the coresponding fragment from the xml file: options option name=Affymetrix value=Affymetrix option name=HumanGeneST10_TissueData value=HumanGeneST10_TissueData option type=meta_key name=MouseTP_Brain_01_mGENE.CEL value=/***/***/external/Affymetrix/HumanGeneST10_TissueData/MouseTP_Brain_01_mGENE.CEL/ option type=meta_key name=MouseTP_Brain_02_mGENE.CEL value=/***/***/external/Affymetrix/HumanGeneST10_TissueData/MouseTP_Brain_02_mGENE.CEL/ option type=meta_key name=MouseTP_Brain_03_mGENE.CEL value=/***/***/external/Affymetrix/HumanGeneST10_TissueData/MouseTP_Brain_03_mGENE.CEL/ option type=meta_key name=MouseTP_Embryo_01_mGENE.CEL value=/***/***/external/Affymetrix/HumanGeneST10_TissueData/MouseTP_Embryo_01_mGENE.CEL/ option type=meta_key name=MouseTP_Embryo_02_mGENE.CEL value=/***/***/external/Affymetrix/HumanGeneST10_TissueData/MouseTP_Embryo_02_mGENE.CEL/ option type=meta_key name=MouseTP_Embryo_03_mGENE.CEL value=/***/***/external/Affymetrix/HumanGeneST10_TissueData/MouseTP_Embryo_03_mGENE.CEL/ option type=meta_key name=MouseTP_Heart_01_mGENE.CEL value=/***/***/external/Affymetrix/HumanGeneST10_TissueData/MouseTP_Heart_01_mGENE.CEL/ option type=meta_key name=MouseTP_Heart_02_mGENE.CEL value=/***/***/external/Affymetrix/HumanGeneST10_TissueData/MouseTP_Heart_02_mGENE.CEL/ option type=meta_key name=MouseTP_Heart_03_mGENE.CEL value=/***/***/external/Affymetrix/HumanGeneST10_TissueData/MouseTP_Heart_03_mGENE.CEL/ option type=meta_key name=MouseTP_Kidney_01_mGENE.CEL value=/***/***/external/Affymetrix/HumanGeneST10_TissueData/MouseTP_Kidney_01_mGENE.CEL/ option type=meta_key name=MouseTP_Kidney_02_mGENE.CEL value=/***/***/external/Affymetrix/HumanGeneST10_TissueData/MouseTP_Kidney_02_mGENE.CEL/ option type=meta_key name=MouseTP_Kidney_03_mGENE.CEL value=/***/***/external/Affymetrix/HumanGeneST10_TissueData/MouseTP_Kidney_03_mGENE.CEL/ // option type=meta_key name=MouseTP_Thymus_03_mGENE.CEL value=/***/***/external/Affymetrix/HumanGeneST10_TissueData/MouseTP_Thymus_03_mGENE.CEL/ /option option name=MouseGeneST10_TissueData value=MouseGeneST10_TissueData option type=meta_key name=MouseTP_Brain_01_mGENE.CEL value=/***/***/external/Affymetrix/MouseGeneST10_TissueData/MouseTP_Brain_01_mGENE.CEL/ option type=meta_key name=MouseTP_Brain_02_mGENE.CEL value=/***/***/external/Affymetrix/MouseGeneST10_TissueData/MouseTP_Brain_02_mGENE.CEL/ // option type=meta_key name=MouseTP_Thymus_03_mGENE.CEL value=/***/***/external/Affymetrix/MouseGeneST10_TissueData/MouseTP_Thymus_03_mGENE.CEL/ /option /option option name=GEO value=GEO // /option /options If not, is there any method for exposing a the whole of a directory on the file system? Thanks, Nathan -Original Message- From: Hans-Rudolf Hotz [mailto:h...@fmi.ch] Sent: Tuesday, October 01, 2013 10:11 AM To: Cole, Nathan (NIH/NCI) [C] Cc: 'galaxy-dev@lists.bx.psu.edu' Subject: Re: [galaxy-dev] Dynamic data library On 10/01/2013 03:53 PM, Cole, Nathan (NIH/NCI) [C] wrote: Thank you both for your responses. I will be looking into both of these. With regard to the from_file option to add the sample selection into the tool: I assume this means that the metadata and everything is loaded into galaxy at the time the tool is run. This depends on how you write your tool. Do you just wanna read the ie fastq file or do you also wanna read the meta data. Also, how is the meta data accessible? eg. is it stored in a txt file at the same location as the fastq file? Does this create a copy of the loaded file or simply read it in place? Also are there any efficiency issues created using this method, outside of the tool run time increase due to the load of the data taking place in-tool? It should just read it in place Hans-Rudolf Thanks, Nathan -Original Message- From: Hans-Rudolf Hotz [mailto:h...@fmi.ch] Sent: Tuesday, October 01, 2013 4:07 AM To: Cole, Nathan (NIH/NCI) [C] Cc: Martin Čech; galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Dynamic data library Hi Nathan Do you have many tools working with those samples or just a few? If you only have a limited, predefined set of tools you might wanna consider adding the sample selection into the tool. You can use the from_file, or from_data_table options to dynamically create sample selection list. You can even drill down a hierarchical list. Have a look at ~/tools
Re: [galaxy-dev] Dynamic data library
On 10/01/2013 03:53 PM, Cole, Nathan (NIH/NCI) [C] wrote: Thank you both for your responses. I will be looking into both of these. With regard to the from_file option to add the sample selection into the tool: I assume this means that the metadata and everything is loaded into galaxy at the time the tool is run. This depends on how you write your tool. Do you just wanna read the ie fastq file or do you also wanna read the meta data. Also, how is the meta data accessible? eg. is it stored in a txt file at the same location as the fastq file? Does this create a copy of the loaded file or simply read it in place? Also are there any efficiency issues created using this method, outside of the tool run time increase due to the load of the data taking place in-tool? It should just read it in place Hans-Rudolf Thanks, Nathan -Original Message- From: Hans-Rudolf Hotz [mailto:h...@fmi.ch] Sent: Tuesday, October 01, 2013 4:07 AM To: Cole, Nathan (NIH/NCI) [C] Cc: Martin Čech; galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Dynamic data library Hi Nathan Do you have many tools working with those samples or just a few? If you only have a limited, predefined set of tools you might wanna consider adding the sample selection into the tool. You can use the from_file, or from_data_table options to dynamically create sample selection list. You can even drill down a hierarchical list. Have a look at ~/tools/annotation_profiler/annotation_profiler.xml which uses the file ~/tool-data/annotation_profiler_options.xml All you need to do is keeping the file in sync with the directory structure of your samples directory Regards, Hans-Rudolf On 09/30/2013 09:48 PM, Martin Čech wrote: Hi Nathan, Dannon answered similar question few days ago: There's an import mechanism in libraries that'll allow you to simply link to the file on disk without copy/upload. I believe the example_watch_folder.py sample script (in the distribution) does just this via the API, if you want an example. This might be what you are looking for. Martin On Mon, Sep 30, 2013 at 2:43 PM, Cole, Nathan (NIH/NCI) [C] nathan.c...@nih.gov mailto:nathan.c...@nih.gov wrote: Hello, we’ve set up a local Galaxy instance in our genotyping and next-gen sequencing lab with local Apache LDAP (AD) integration, NFS mounts to a large NAS, and cluster integration coming. Do to the high volume of samples and staff that will be using the system, I want to set up data libraries (without copying to Galaxy). This is obviously no problem the first time, however I was wondering if there was a way to make a library, added from a system path, be dynamic so that it would stay synchronized with the underlying file structure? __ __ If a try dynamic library is not possible, is there a method for adding files to an existing library via that same system path that would not duplicate all of the original files in the data library? __ __ I did some scouring of the list and found some old unanswered questions and some things tangentially related topics, but I was unable to find a true answer or solution to my problem. Any information on how to do the tasks above or other solutions to provide the same functionality would be greatly appreciated. __ __ Thanks, Nathan __ __ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Limit number of Jobs to Trinity
Hi Carlos and John Please allow me to pick up this thread. I have been experimenting with this, but I cant get it to work. The default destination is always used. Does it work for you, Carlos? Regards, Hans-Rudolf On 09/23/2013 03:32 AM, John Chilton wrote: As your Galaxy instance becomes more production ready I would definitely recommend installing a job queue such as grid engine, Torque, or SLURM. For this simple use case though, you should be able t define multiple local job runner instances to ensure only one Trinity job runs a time. Your job_conf.xml should look something like this: ?xml version=1.0? job_conf plugins plugin id=local type=runner load=galaxy.jobs.runners.local:LocalJobRunner workers=4/ plugin id=local2 type=runner load=galaxy.jobs.runners.local:LocalJobRunner workers=1/ /plugins handlers handler id=main/ /handlers destinations default=local destination id=local runner=local/ destination id=local2 runner=local2 / /destinations tools tool id=trinity destination=local2/ /tools /job_conf Hope this helps. -John On Sat, Sep 21, 2013 at 10:40 AM, Carlos Canchaya ccanch...@gmail.com wrote: Hi adam, So far we have just a standalone server since we are still setting up our tools (we will scale up to our sge cluster, though). Any clue how to limit jobs per tool basis? Best, Carlos On Saturday, September 21, 2013, Adam Brenner wrote: Carlos, Are you using any sort of scheduler? SGE/Torque, etc? How are jobs being dispatched to your node(s)? More info here: http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Sat, Sep 21, 2013 at 4:47 AM, Carlos Canchaya ccanch...@gmail.com wrote: Hi guys, I have a running local galaxy installation on one of our servers. I've just installed Trinity for de novo assembly of transcripts and it will use up to 32 processors. However i would like to limit the number of concurrent jobs only for this tool. Do you know if it would be possible to limit the number of concurrent jobs per tool basis? Bests, Carlos -- Carlos Canchaya ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Carlos A. Canchaya Universidade de Vigo ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Limit number of Jobs to Trinity
well, I was just reading today's commits and I spotted: Nate's commit: Bugfix for tool-to-destination mapping, tool ids are lowercased but the mapping id was not lowercased. - Changeset: 0b955e54451c So I changed: tool id=addValue destination=local2 / to tool id=addvalue destination=local2 / and it seems to work. Regards, Hans-Rudolf On 09/26/2013 05:43 PM, Hans-Rudolf Hotz wrote: Hi Carlos and John Please allow me to pick up this thread. I have been experimenting with this, but I cant get it to work. The default destination is always used. Does it work for you, Carlos? Regards, Hans-Rudolf On 09/23/2013 03:32 AM, John Chilton wrote: As your Galaxy instance becomes more production ready I would definitely recommend installing a job queue such as grid engine, Torque, or SLURM. For this simple use case though, you should be able t define multiple local job runner instances to ensure only one Trinity job runs a time. Your job_conf.xml should look something like this: ?xml version=1.0? job_conf plugins plugin id=local type=runner load=galaxy.jobs.runners.local:LocalJobRunner workers=4/ plugin id=local2 type=runner load=galaxy.jobs.runners.local:LocalJobRunner workers=1/ /plugins handlers handler id=main/ /handlers destinations default=local destination id=local runner=local/ destination id=local2 runner=local2 / /destinations tools tool id=trinity destination=local2/ /tools /job_conf Hope this helps. -John On Sat, Sep 21, 2013 at 10:40 AM, Carlos Canchaya ccanch...@gmail.com wrote: Hi adam, So far we have just a standalone server since we are still setting up our tools (we will scale up to our sge cluster, though). Any clue how to limit jobs per tool basis? Best, Carlos On Saturday, September 21, 2013, Adam Brenner wrote: Carlos, Are you using any sort of scheduler? SGE/Torque, etc? How are jobs being dispatched to your node(s)? More info here: http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Sat, Sep 21, 2013 at 4:47 AM, Carlos Canchaya ccanch...@gmail.com wrote: Hi guys, I have a running local galaxy installation on one of our servers. I've just installed Trinity for de novo assembly of transcripts and it will use up to 32 processors. However i would like to limit the number of concurrent jobs only for this tool. Do you know if it would be possible to limit the number of concurrent jobs per tool basis? Bests, Carlos -- Carlos Canchaya ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Carlos A. Canchaya Universidade de Vigo ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Stable Galaxy for local install
Hi cngbgalaxy May be there is an issue with the version of grep you are using on the server Galaxy is running on? I vaguely remember discussions on the mailing list related to grep/sort/etc what happens when do it on the command line, ie: [hrh@foo]$ grep ^@ foo.sam @HD VN:1.0 SO:coordinate @SQ SN:chr1 LN:10001 @SQ SN:chr2 LN:11 @SQ SN:chr3 LN:10001 @SQ SN:chr4 LN:1001 @RG ID:rg1 SM:s1 @RG ID:rg2 SM:s3 [hrh@foo]$ grep -v ^@ foo.sam bar:record:477 chr11 0 * * 0 0 A 1 RG:Z:rg1 bar:record:677 chr11 0 * * 0 0 A 1 RG:Z:rg2 [haruhotz@xenon1 20130912]$ Alternatively, if everything fails, and all you wanna do is just removing the seven lines at the top of the sam file, you can also use the tool: Text Manipulation - Remove beginning of a file Regards, Hans-Rudolf On 09/12/2013 06:07 AM, cngbgal...@gmail.com wrote: Hi Adam Brenner, This is my running steps. * I got my sam file (as follow) upload: @HD VN:1.0 SO:coordinate @SQ SN:chr1 LN:10001 @SQ SN:chr2 LN:11 @SQ SN:chr3 LN:10001 @SQ SN:chr4 LN:1001 @RG ID:rg1 SM:s1 @RG ID:rg2 SM:s3 bar:record:4 77 chr1 1 0 * * 0 0 A 1 RG:Z:rg1 bar:record:6 77 chr1 1 0 * * 0 0 A 1 RG:Z:rg2 bar:record:1 77 chr1 10 0 * * 0 0 A 1 RG:Z:rg1 bar:record:3 77 chr1 10 0 * * 0 0 A 1 RG:Z:rg2 bar:record:5 141 chr3 40 0 * * 0 0 C 2 RG:Z:rg1 bar:record:9 77 chr4 10 0 * * 0 0 A 1 RG:Z:rg1 bar:record:7 141 chr4 20 0 * * 0 0 C 2 RG:Z:rg1 bar:record:5 77 chr1 40 0 * * 0 0 A 1 RG:Z:rg2 bar:record:6 141 chr1 50 0 * * 0 0 C 2 RG:Z:rg2 bar:record:2 77 chr2 10 0 * * 0 0 A 1 RG:Z:rg1 bar:record:2 141 chr2 30 0 * * 0 0 C 2 RG:Z:rg2 bar:record:3 141 chr3 20 0 * * 0 0 C 2 RG:Z:rg1 bar:record:9 141 chr4 60 0 * * 0 0 C
Re: [galaxy-dev] Creating Galaxy Admin / Local Install Mailing List - Updates?
Hi Adam I guess the latest is written down in the notes for the GalaxyAdmins BoF at GCC2013, see: http://wiki.galaxyproject.org/Events/GCC2013/BoF/GalaxyAdmins maybe we can discuss this again at the next 'meetup'? The next 'meetup' is scheduled for September 18, but I don't know whether it will happen? I am sure Dave will know more. Regards, Hans-Rudolf On 09/12/2013 04:52 AM, Adam Brenner wrote: Howdy, I know this has been discussed before, but I can not find the email threads on this...but what was the status on creating a galaxy admin / local install mailing list? The Devel list is very chatty and while my filters do a good job, I think it will be easier for others to find help on their own if it was on a separate mailing list It would be easier to search and look back at previous threads. However, the added benefit of having the devel list is that we get feedback from Galaxy Developers As the saying godes: Just food for thought, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Why does my trackster complain about not being able to display BED files?
Hi Thon double check your datatypes_conf.xml (which was not touched/changed during the update process, since it is listed in the .hgignore file, I guess) with the new datatypes_conf.xml.sample file for any changes regarding converters for bed files Regards, Hans-Rudolf On 08/22/2013 02:51 AM, Anthonius deBoer wrote: Yep... $ bedtools --version bedtools v2.17.0 Thon On Aug 21, 2013, at 01:40 PM, sam guerler aysam.guer...@gmail.com wrote: Hi Anthonius, Do you have the BEDTools installed and its directory added to your PATH environment variable? Trackster uses compressed file formats to reduce the amount of data transfers. I hope this helps, Sam On Wed, Aug 21, 2013 at 4:11 PM, Anthonius deBoer thondeb...@me.com mailto:thondeb...@me.com wrote: Hi, I am using the latest version of galaxy at my local installation and I try to use trackster to display a simple BED file, but it invariably complains about not having a converter for this datatype and it tells me to check my datatypes_conf.xml which I did (not sure what I should be looking for, but this is the section on BED files... Am I missing a converter? And why would a BED file even needed to be converted? Thanks Thon datatype extension=bed type=galaxy.datatypes.interval:Bed display_in_upload=true converter file=bed_to_gff_converter.xml target_datatype=gff/ converter file=interval_to_coverage.xml target_datatype=coverage/ converter file=bed_to_bgzip_converter.xml target_datatype=bgzip/ converter file=bed_to_tabix_converter.xml target_datatype=tabix depends_on=bgzip/ converter file=bed_to_summary_tree_converter.xml target_datatype=summary_tree/ converter file=bed_to_fli_converter.xml target_datatype=fli/ !-- display file=ucsc/interval_as_bed.xml / -- !-- display file=genetrack.xml / -- display file=igb/bed.xml / /datatype datatype extension=bedgraph type=galaxy.datatypes.interval:BedGraph display_in_upload=true converter file=bedgraph_to_bigwig_converter.xml target_datatype=bigwig/ /datatype datatype extension=bedstrict type=galaxy.datatypes.interval:BedStrict / !-- datatype extension=bed6 type=galaxy.datatypes.interval:Bed6 converter file=bed_to_genetrack_converter.xml target_datatype=genetrack/ /datatype -- datatype extension=bed12 type=galaxy.datatypes.interval:Bed12 / datatype extension=len type=galaxy.datatypes.chrominfo:ChromInfo display_in_upload=true converter file=len_to_linecount.xml target_datatype=linecount / /datatype datatype extension=bigbed type=galaxy.datatypes.binary:BigBed mimetype=application/octet-stream dis ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Passing user_email to a data source
Hi Anthony I doubt it will solve your problem, but just as a general hint: you can use the email address for filtering. In our case, the user gets a different list of groups he/she can select from, eg: param name=group type=select label=Select your group options from_file=access_to_miseq column name=email index=0/ column name=value index=1/ filter type=static_value name=external_source_filter value=$__user_email__ column=0/ /options /param and our file '~/tool-data/access_to_miseq' looks like: 1...@fmi.ch gbioinfo 1...@fmi.ch gfungen 1...@fmi.ch gmeier # # a...@fmi.ch gfungen # # 1...@fmi.ch gfungen 1...@fmi.ch gmeier Regards, Hans-Rudolf On 08/20/2013 05:51 PM, Anthony Bretaudeau wrote: Hello, I'm trying to add a new data source to my Galaxy instance, using synchronous data depositing. The data source needs to know the email address of the user who tries to import data to the Galaxy server. I would have like to write something like that: inputs action=https://example.com/my/data/source; check_values=false method=get displayBrowse the XX data source $GALAXY_URL/display param name=user_email type=hidden value=$__user_email__ / /inputs Then Galaxy would have generated this url: https://example.com/my/data/source?GALAXY_URL=httpuser_email=t...@example.com But this doesn't work, because the $__user_email__ seems to be available only in command tag. I end up with this url which is useless: https://example.com/my/data/source?GALAXY_URL=httpuser_email=$__user_email__ Is there any way to do what I need to do? It would be great if variables were available in param tags too. Otherwise, maybe I could write a custom tool type to do this. Is there any guideline for this? I have seen someone talking about that a few months ago, but with no answer: http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-September/011258.html Thanks for your help Regards Anthony Bretaudeau ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Galaxy Release Cycle Length
Hi Dave two months is a good time between releases. Much more important than the release cycle length is fixing identified bugs on the release branch as well. Regards, Hans-Rudolf On 08/20/2013 08:36 PM, Dave Clements wrote: Hello all, At one of the GCC2013 Birds of a Feather sessions http://wiki.galaxyproject.org/Events/GCC2013/BoF/PublicGalaxyServers the group was very clear that they would like to see less frequent releases of Galaxy. We're currently aiming to do a release every 2 months and have been pretty successful at making that target. In the past, we have tried doing releases more often and less often. Is there a sweet spot for the time between releases? Please reply to the group. We are interested in a discussion. Thanks, Dave C -- http://galaxyproject.org/GCC2013 http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://wiki.galaxyproject.org/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Troubleshooting file uploads (to Data Library)
Hi Clare a few points to start: - do you define the 'File Format'? (don't use 'Auto-detect' for big files) - and similar to a recent question on the list: check your proxy settings Regards, Hans-Rudof On 08/21/2013 09:09 AM, Clare Sloggett wrote: Sorry, missed some information: there are a handful of files, 35MB (gzipped) each. The issue occurs even if I only try to upload one of them though. The server is a 16-core machine. On 21 August 2013 17:08, Clare Sloggett s...@unimelb.edu.au mailto:s...@unimelb.edu.au wrote: Hi, I am having trouble uploading files to a Data Library, and I'm not sure where to begin troubleshooting. I'm uploading from a URL (but I had a similar issue from the desktop). The symptom is that the datasets in the library have the message This job is queued and never seem to progress. I am one of very few users of this instance (quite likely the only user right this moment). I don't think the server is busy, so I'm not sure why the files uploads don't seem to be proceeding. How can I investigate further? Thanks, Clare -- Clare Sloggett Research Fellow / Bioinformatician Life Sciences Computation Centre Victorian Life Sciences Computation Initiative University of Melbourne, Parkville Campus 187 Grattan Street, Carlton, Melbourne Victoria 3010, Australia Ph: 03 903 53357 M: 0414 854 759 -- Clare Sloggett Research Fellow / Bioinformatician Life Sciences Computation Centre Victorian Life Sciences Computation Initiative University of Melbourne, Parkville Campus 187 Grattan Street, Carlton, Melbourne Victoria 3010, Australia Ph: 03 903 53357 M: 0414 854 759 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] upload file not functioning in local installation galaxy
Hi One of our development boxes is (unfortunately) behind a different firewall, compared to the the other servers. We fixed the problem (ie Get Data - Upload File - choose file. It is not uploading at all) by setting a proxy server. This could be done eg. by adding the following lines to run.sh: http_proxy=http://**.fmi.ch:port number export http_proxy Obviously, this solved other problems as well, like Get Data - UCSC Regards, Hans-Rudolf On 08/19/2013 04:01 PM, Yinan Wan wrote: Hi Hans-Rudolf, Thanks for your reply. But do you have any idea what might be related with this proxy issue? What is required to open/set for galaxy to run? Or do you might share your experience of how you solve the problem? Thanks, Yinan On Mon, Aug 19, 2013 at 3:16 AM, Hans-Rudolf Hotz h...@fmi.ch mailto:h...@fmi.ch wrote: Hi Yinan It might be unrelated, but we had a similar issue. The problem was due to the proxy settings of our server. Regards, Hans-Rudolf On 08/16/2013 08:23 PM, Yinan Wan wrote: Hi, I have installed galaxy on our local server, when I was trying to upload data from: Get Data - Upload File - choose file. It is not uploading at all; but it works fine when using the URL option. I wonder do you have any suggestions about this problem? The output from the command line is pasted below: 128.118.200.158 - - [15/Aug/2013:14:15:41 -0400] GET /api/histories/__5969b1f7201f12ae/contents?ids=__f09437b8822035f7 HTTP/1.1 200 - http://128.118.201.68:8080/__history http://128.118.201.68:8080/history Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36 128.118.200.158 - - [15/Aug/2013:14:15:45 -0400] GET /api/histories/__5969b1f7201f12ae HTTP/1.1 200 - http://128.118.201.68:8080/__history http://128.118.201.68:8080/history Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36 128.118.200.158 - - [15/Aug/2013:14:15:45 -0400] GET /api/histories/__5969b1f7201f12ae/contents?ids=__f09437b8822035f7 HTTP/1.1 200 - http://128.118.201.68:8080/__history http://128.118.201.68:8080/history Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36 128.118.200.158 - - [15/Aug/2013:14:15:45 -0400] GET /api/users/current HTTP/1.1 200 - http://128.118.201.68:8080/; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36 128.118.200.158 - - [15/Aug/2013:14:16:49 -0400] GET /tool_runner?tool_id=upload1 HTTP/1.1 200 - http://128.118.201.68:8080/; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36 128.118.200.158 - - [15/Aug/2013:14:17:03 -0400] POST /tool_runner/upload_async___create HTTP/1.1 200 - http://128.118.201.68:8080/; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36 128.118.200.158 - - [15/Aug/2013:14:17:04 -0400] GET /tool_runner/upload_async___message HTTP/1.1 200 - http://128.118.201.68:8080/; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36 128.118.200.158 - - [15/Aug/2013:14:17:08 -0400] GET /history HTTP/1.1 200 - http://128.118.201.68:8080/; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36 == Thanks, Yinan -- Yinan Wan, graduate student Bioinformatics and Genomics program Huck Institutes of Life Sciences the Pennsylvania State University _ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/__search/mailinglists/ http://galaxyproject.org/search/mailinglists/ -- Yinan Wan, graduate student Bioinformatics and Genomics program Huck Institutes of Life Sciences the Pennsylvania State University ___ Please keep all replies on the list by using reply all in your mail client. To manage
Re: [galaxy-dev] Mail is not configured for this Galaxy instance.
On 08/06/2013 08:17 AM, shenwiyn wrote: Hi Hans-Rudolf, Thank you very much for you help.Actually I want some more information about how to configure 'smtp_server' in universe_wsgi.ini.Could you show me some samples to have a look please? you need the address of your smtp server, eg: smtp_server = abcd.fmi.ch Regards, shenwiyn *From:* Hans-Rudolf Hotz mailto:h...@fmi.ch *Date:* 2013-07-31 23:40 *To:* shenwiyn mailto:shenw...@gmail.com *CC:* galaxy-dev@lists.bx.psu.edu mailto:galaxy-dev@lists.bx.psu.edu *Subject:* Re: [galaxy-dev] Mail is not configured for this Galaxy instance. Hi Shenwiyn You need to specify a SMTP server. Have a look at the 'smtp_server' settings in your universe_wsgi.ini file. Regards, Hans-Rudolf On 07/31/2013 04:19 PM, shenwiyn wrote: Hi everyone, In my local Galaxy,when I want to reset my Login password,some error occurre :Mail is not configured for this Galaxy instance. Please contact an administrator. Could anyone tell me some imformation about how to configure the mail for my Galaxy instance? Thank you very much. shenwiyn ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] getting large files into a local instance of Galaxy
Hi Gerald Have you looked into 'Data Libraries'? http://wiki.galaxyproject.org/Admin/DataLibraries/Libraries http://wiki.galaxyproject.org/Admin/DataLibraries/UploadingLibraryFiles This will allow you to access the files without data duplication. Regards, Hans-Rudolf On 08/05/2013 08:48 PM, Gerald Bothe wrote: Trying out a local instance of Galaxy. Small files can be uploaded with browse.. but this works only for files under 2Gb. Is there a folder in a local copy of galaxy-dist where I can copy a large file and have it show up in the Get Data/Upload File/Files uploaded via FTP window so that I can properly get it into the local history? Copying it directly into the database seemed not to work, it didn't show up anywhere. Gerald ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] How to get the data-view on local instance just like the Galaxy Main ?
Hi Nikhil what is the data format for this data set in your local installation and on main? Regards, Hans-Rudolf On 08/02/2013 09:56 AM, Nikhil Mallela wrote: Hi, A few days ago, I have installed the stable release of of Galaxy local instanceon my Workstation I am running it behind Apache web server proxy . In regard to the data snapshots, I've observed that, it is much more organized on the Galaxy main rather than on my local instance. For example, Take a look at these pictures: On my local instance: Inline image 1 On the Galaxy main: (also, this Inline image 4 is missing on my local instance) Inline image 2 Also, when I click the view Inline image 3 on my local instance,the headers of the data is not corresponding to its respective columns. So, how can I fine tune these ? Thanks in advance! Nikhil. -- Nikhil Vinod Mallela, M.Sc. University of Muenster, Institute of Bioinformatics, Neil-Stensen Str. 12-14, 48149, Muenster,Germany. Tel.:+49 0251 83 50006 Fax: +49 0251 83 53005 Email: mall...@uni-muenster.de mailto:mall...@uni-muenster.de http://complex-systems.uni-muenster.de/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] How to get the data-view on local instance just like the Galaxy Main ?
and now, what happens, if you change it to 'tabular'? (click on the pencil icon - 'Edit Attributes', and then on Datatype On 08/02/2013 12:18 PM, Nikhil Mallela wrote: Hi Hans-Rudolf, I was wrong previously. It is output produced (.txt) while computing the quality statistics on the .qual file The output format seems to have been detected by Galaxy as a .txt file. Here is a snapshot. Inline image 1 Nikhil. On Fri, Aug 2, 2013 at 11:58 AM, Hans-Rudolf Hotz h...@fmi.ch mailto:h...@fmi.ch wrote: Hi I guess, you misunderstood my question. I was revering to the format within galaxy, ie what is written in the preview window (unfortunately, you did cut this off from your pictures in your original post) between format: and , database. Hans-Rudolf ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Mail is not configured for this Galaxy instance.
Hi Shenwiyn You need to specify a SMTP server. Have a look at the 'smtp_server' settings in your universe_wsgi.ini file. Regards, Hans-Rudolf On 07/31/2013 04:19 PM, shenwiyn wrote: Hi everyone, In my local Galaxy,when I want to reset my Login password,some error occurre :Mail is not configured for this Galaxy instance. Please contact an administrator. Could anyone tell me some imformation about how to configure the mail for my Galaxy instance? Thank you very much. shenwiyn ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] BWA Illumina Mapping / BWA Reference Genome
Hi Moritz I am not a galaxy tool shed expert, but I recommend you start with reading these wiki pages: http://wiki.galaxyproject.org/Tool%20Shed http://wiki.galaxyproject.org/InstallingRepositoriesToGalaxy and then try installing BWA via the tool shed and ask (if any) specific questions (preferably with a new subject line) on the mailing list. Regards Hans-Rudolf On 07/22/2013 11:20 AM, Moritz Juchler wrote: Hey, with the tool file line I got it working :) Thanks a lot. Could you answer the question regarding Tool Shed: If I would use Tool Shed, could I skip all these manual steps? Is the installation of BWA with Tool Shed also this complicated or is it more simle? Best Moritz On 17 July 2013 09:51, Moritz Juchler juch...@stud.uni-heidelberg.de mailto:juch...@stud.uni-heidelberg.de wrote: Hey, no thats correct I did not use Tool Shed. If I would use it, could I skip all these manual steps? Is the installation of BWA with Tool Shed also this complicated or is it more simle? How did you not follow my steps :) I wrote down everything clearly, at least thats what I was hoping. And no I didnt write this line tool file=sr_mapping/bwa_wrapper.__xml / Thats the first time, I am seeing this tutorial :( I will try this out right now Best Moritz On 17 July 2013 09:42, Hans-Rudolf Hotz h...@fmi.ch mailto:h...@fmi.ch wrote: Hi Moritz I am struggling to follow what exactly you have done. As far as I can see, you did not use the toolshed (http://wiki.galaxyproject.__org/Tool%20Shed http://wiki.galaxyproject.org/Tool%20Shed) to install the BWA alinger tool, but did all manually? If so, have you added the following line: tool file=sr_mapping/bwa_wrapper.__xml / to the tool_conf.xml file, and restarted Galaxy? see also: http://wiki.galaxyproject.org/__Admin/Tools/Add%20Tool%__20Tutorial http://wiki.galaxyproject.org/Admin/Tools/Add%20Tool%20Tutorial Hope this helps Hans-Rudolf On 07/16/2013 08:35 PM, Moritz Juchler wrote: Hello Ladies and Gentlemen, I am Moritz Juchler from University Heidelberg. For my Bachelor thesis I have to choose a bioinformatic pipeline management tool to find SNP's in genomes from hcc patients. My decision was made in favor of galaxy. I have a 64-bit openSuse 11.3 server. I have installed Galaxy locally, since we have a) very large files (30GB per patient) and b) the data is protection sensitive. I kept close to http://wiki.galaxyproject.org/__Admin/Get%20Galaxy http://wiki.galaxyproject.org/Admin/Get%20Galaxy Now I would like to run this bpipe pipeline: http://pastebin.com/sZd5vfdL And the first step is to align my genome to a _hg19 reference genome_ which I have locally under /genedata/human_genome_GRCh37/__. trr@portalmoritz:~ ls -l /genedata/human_genome_GRCh37/ total 8486312 -rw-r--r-- 1 trr root 3199905909 2013-06-25 16:44 hg19.fa -rw-r--r-- 1 trr root 8591 2013-07-01 16:06 hg19.fa.amb -rw-r--r-- 1 trr root 4040 2013-07-01 16:06 hg19.fa.ann -rw-r--r-- 1 trr root 3137161344 2013-07-01 16:05 hg19.fa.bwt -rw-r--r-- 1 trr root 784290318 2013-07-01 16:06 hg19.fa.pac -rw-r--r-- 1 trr root 1568580688 2013-07-01 16:31 hg19.fa.sa http://hg19.fa.sa http://hg19.fa.sa _bwa is installed and gives me:_ trr@portalmoritz:~ bwa Program: bwa (alignment via Burrows-Wheeler transformation) Version: 0.7.5a-r405 Contact: Heng Li l...@sanger.ac.uk mailto:l...@sanger.ac.uk mailto:l...@sanger.ac.uk mailto:l...@sanger.ac.uk Then I tried to follow this guide: http://wiki.galaxyproject.org/__Admin/NGS%20Local%20Setup http://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup to get the reference files and http://wiki.galaxyproject.org/__Admin/Config/Tool%__20Dependencies http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies. This is my _$PATH_ trr@portalmoritz:~ echo $PATH /home/trr/bin:/usr/local/bin:/__usr/bin:/bin:/usr/bin/X11:/__usr/X11R6/bin:/usr/games:/__home/trr/bpipe-0.9.8/bin:/__home/trr/bwa-0.7.5a:/home/trr/__samtools-0.1.19 _In the universe_wsgi.ini I changed:_ tool_dependency_dir = /home/trr/galaxy-dist/tool___dependency_dir debug
Re: [galaxy-dev] BWA Illumina Mapping / BWA Reference Genome
Hi Moritz I am struggling to follow what exactly you have done. As far as I can see, you did not use the toolshed (http://wiki.galaxyproject.org/Tool%20Shed) to install the BWA alinger tool, but did all manually? If so, have you added the following line: tool file=sr_mapping/bwa_wrapper.xml / to the tool_conf.xml file, and restarted Galaxy? see also: http://wiki.galaxyproject.org/Admin/Tools/Add%20Tool%20Tutorial Hope this helps Hans-Rudolf On 07/16/2013 08:35 PM, Moritz Juchler wrote: Hello Ladies and Gentlemen, I am Moritz Juchler from University Heidelberg. For my Bachelor thesis I have to choose a bioinformatic pipeline management tool to find SNP's in genomes from hcc patients. My decision was made in favor of galaxy. I have a 64-bit openSuse 11.3 server. I have installed Galaxy locally, since we have a) very large files (30GB per patient) and b) the data is protection sensitive. I kept close to http://wiki.galaxyproject.org/Admin/Get%20Galaxy Now I would like to run this bpipe pipeline: http://pastebin.com/sZd5vfdL And the first step is to align my genome to a _hg19 reference genome_ which I have locally under /genedata/human_genome_GRCh37/. trr@portalmoritz:~ ls -l /genedata/human_genome_GRCh37/ total 8486312 -rw-r--r-- 1 trr root 3199905909 2013-06-25 16:44 hg19.fa -rw-r--r-- 1 trr root 8591 2013-07-01 16:06 hg19.fa.amb -rw-r--r-- 1 trr root 4040 2013-07-01 16:06 hg19.fa.ann -rw-r--r-- 1 trr root 3137161344 2013-07-01 16:05 hg19.fa.bwt -rw-r--r-- 1 trr root 784290318 2013-07-01 16:06 hg19.fa.pac -rw-r--r-- 1 trr root 1568580688 2013-07-01 16:31 hg19.fa.sa http://hg19.fa.sa _bwa is installed and gives me:_ trr@portalmoritz:~ bwa Program: bwa (alignment via Burrows-Wheeler transformation) Version: 0.7.5a-r405 Contact: Heng Li l...@sanger.ac.uk mailto:l...@sanger.ac.uk Then I tried to follow this guide: http://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup to get the reference files and http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies. This is my _$PATH_ trr@portalmoritz:~ echo $PATH /home/trr/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/home/trr/bpipe-0.9.8/bin:/home/trr/bwa-0.7.5a:/home/trr/samtools-0.1.19 _In the universe_wsgi.ini I changed:_ tool_dependency_dir = /home/trr/galaxy-dist/tool_dependency_dir debug = False use_interactive = True library_import_dir = /genedata/ allow_library_path_paste = True admin_users = ... This is my _tool_dependency_dir:_ trr@portalmoritz:~/galaxy-dist/tool_dependency_dir/bwa ls -l total 4 drwxr-xr-x 3 trr users 4096 2013-07-16 14:28 0.7.4 lrwxrwxrwx 1 trr users6 2013-07-16 14:17 default - 0.7.4/ This is the_version folder of bwa:_ trr@portalmoritz:~/galaxy-dist/tool_dependency_dir/bwa/0.7.4 ls -l total 8 drwxr-xr-x 2 trr users 4096 2013-07-16 14:18 bin -rw-r--r-- 1 trr users 47 2013-07-16 14:22 env.sh This is the _content of env.sh:_ trr@portalmoritz:~/galaxy-dist/tool_dependency_dir/bwa/0.7.4 cat env.sh PATH=/home/trr/bwa-0.7.5a/:$PATH export PATH And this is the _content of the bin folder:_ trr@portalmoritz:~/galaxy-dist/tool_dependency_dir/bwa/0.7.4/bin ls -l total 3896 -rw-r--r-- 1 trr users 6098 2013-07-16 14:18 bamlite.c -rw-r--r-- 1 trr users 3124 2013-07-16 14:18 bamlite.h -rw-r--r-- 1 trr users 24816 2013-07-16 14:18 bamlite.o -rw-r--r-- 1 trr users 11508 2013-07-16 14:18 bntseq.c -rw-r--r-- 1 trr users 2557 2013-07-16 14:18 bntseq.h -rw-r--r-- 1 trr users 37440 2013-07-16 14:18 bntseq.o -rwxr-xr-x 1 trr users 998217 2013-07-16 14:18 bwa -rw-r--r-- 1 trr users 24225 2013-07-16 14:18 bwa.1 -rw-r--r-- 1 trr users 9416 2013-07-16 14:18 bwa.c -rw-r--r-- 1 trr users 1381 2013-07-16 14:18 bwa.h I got the xmls and .py from https://bitbucket.org/galaxy/galaxy-dist/src/da9d740fce31/tools/sr_mapping and i didnt change them at all and put them into ~/galaxy-dist/_tools/sr_mapping_ (since they were missing in this folder) bwa_color_wrapper.xml bwa_wrapper.py bwa_wrapper.xml I added _bwa_index_color.loc and bwa_index.loc_ to ~galaxy-dist/_tool-data_ (they were missing as well, there/*were no*/ bwa_index_color.loc.sample or bwa_index.lox.sample files!!!) I only have this single line in both bwa_index_color.loc and bwa_index.loc trr@portalmoritz:~/galaxy-dist/tool-data cat bwa_index_color.loc #This is a sample file distributed with Galaxy that enables tools # #unique_build_id dbkey display_name file_path hg19hg19hg19/genedata/human_genome_GRCh37/hg19.fa (Spaces are actually tabs!) After all that, I neither have the Map with BWA for Illumina https://main.g2.bx.psu.edu/tool_runner?tool_id=toolshed.g2.bx.psu.edu/repos/devteam/bwa_wrappers/bwa_wrapper/1.2.3 in my local Galaxy version, nor do I find
Re: [galaxy-dev] Galaxy Local Admin user
Please keep all replies on the list by using reply all ! for my reply see below... On 07/17/2013 06:58 AM, Chadi Abdul Kader El Farran wrote: Dear Hans-Rudolf, Thank you for your reply. The problem was solved by restarting. Now my problem is that I am unable to upload files into my Galaxy. I have put the following code in Universe_wsgi.ini: # Directories of files contained in the following directory can be uploaded to a library from the Admin view library_import_dir = /upload However when I upload files to a data library, all files will seem to have no information (0 bytes) That is very crypticand you need to provide more details - did you get an error? - how do you know the files are empty - what happens if you work with the files make sure to follow the wiki page: http://wiki.galaxyproject.org/Admin/DataLibraries/Libraries Hans-Rudolf Can you kindly help me? Thanks a lot for the support. With warmest regards Chadi EL Farran (Post Graduate Student) Dr. Jonathan Yuin-Han LOH's Laboratory #08-01, Institute of Molecular and Cell Biology (IMCB) 61 Biopolis Drive, Proteos Singapore 138673 Contact No: +65-91944793 E-mail: ch...@nus.edu.sg From: Hans-Rudolf Hotz [h...@fmi.ch] Sent: Tuesday, July 16, 2013 9:39 PM To: Chadi Abdul Kader El Farran Cc: galaxy-...@bx.psu.edu Subject: Re: [galaxy-dev] Galaxy Local Admin user Hi Just double checking: Have you restarted Galaxy? Regards, Hans-Rudolf On 07/16/2013 07:23 AM, Chadi Abdul Kader El Farran wrote: Dear Madam/Sir, I have installed Galaxy Local in our system. I registered as a user. However when I try to give admin status to some users, I am unable despite following the steps mentioned in the help. To give a user Galaxy admin privileges, add their Galaxy login ( email ) to the list in the following config setting in the Galaxy configuration file universe_wsgi.ini. # this should be a comma-separated list of valid Galaxy users admin_users = us...@bx.psu.edu,us...@bx.psu.edu I have done what is mentioned above and the universe_wsgi.ini is edited accordingly. Still when I log on to Galaxy Local I don't see the admin tab. Can you kindly help me. With warmest regards Chadi EL Farran (Post Graduate Student) Dr. Jonathan Yuin-Han Loh's Laboratory #08-01, Institute of Molecular and Cell Biology (IMCB) 61 Biopolis Drive, Proteos Singapore 138673 Contact No: +65-91944793 E-mail: ch...@nus.edu.sg ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Galaxy Local Admin user
Hi Chadi
I am not familiar with the this tool
I recommend to try running the tool manually on the command line first,
and then post the error on the mailing list (with a new subject) IF the
error has something to do with Galaxy.
Hans-Rudolf
On 07/17/2013 05:00 PM, Chadi Abdul Kader El Farran wrote:
Dear Hans-Rudolf,
Thank you for your reply once again.
It appears that the problem is from a tool I installed in Galaxy. It is called
ngsplot.
Whenever I try to do any analysis, I get the following error:
Traceback (most recent call last):
File /home/swarna/galaxy-dist/lib/galaxy/jobs/runners/local.py, line 94,
in queue_job
job_wrapper.finish( stdout, stderr, exit_code )
File /home/swarna/galaxy-dist/lib/galaxy/jobs/__init__.py, line 1009, in
finish
self.sa_session.flush()
File
/home/swarna/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/scoping.py,
line 114, in do
return getattr(self.registry(), name)(*args, **kwargs)
File
/home/swarna/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py,
line 1718, in flush
self._flush(objects)
File
/home/swarna/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py,
line 1789, in _flush
flush_context.execute()
File
/home/swarna/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/unitofwork.py,
line 331, in execute
rec.execute(self)
File
/home/swarna/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/unitofwork.py,
line 475, in execute
uow
File
/home/swarna/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/persistence.py,
line 59, in save_obj
mapper, table, update)
File
/home/swarna/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/persistence.py,
line 485, in _emit_update_statements
execute(statement, params)
File
/home/swarna/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py,
line 1449, in execute
params)
File
/home/swarna/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py,
line 1584, in _execute_clauseelement
compiled_sql, distilled_params
File
/home/swarna/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py,
line 1698, in _execute_context
context)
File
/home/swarna/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py,
line 1691, in _execute_context
context)
File
/home/swarna/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/default.py,
line 331, in do_execute
cursor.execute(statement, parameters)
ProgrammingError: (ProgrammingError) You must not use 8-bit bytestrings unless
you use a text_factory that can interpret 8-bit bytestrings (like text_factory
= str). It is highly recommended that you instead just switch your application
to Unicode strings. u'UPDATE job SET update_time=?, stdout=?, stderr=? WHERE
job.id = ?' ('2013-07-17 14:56:30.112896', '\nzip error: Nothing to do!
(data.zip)\n', 'mv: cannot stat \xe2\x80\x98data.zip\xe2\x80\x99: No such file
or directory\n', 41)
Tool execution generated the following error message:
Unable to finish job
Can you kindly help me?
With warmest regards,
Chadi EL Farran (Post Graduate Student)
Dr. Jonathan Yuin-Han LOH's Laboratory
#08-01, Institute of Molecular and Cell Biology (IMCB)
61 Biopolis Drive, Proteos
Singapore 138673
Contact No: +65-91944793
E-mail: ch...@nus.edu.sg
From: Hans-Rudolf Hotz [h...@fmi.ch]
Sent: Wednesday, July 17, 2013 3:54 PM
To: Chadi Abdul Kader El Farran
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] Galaxy Local Admin user
Please keep all replies on the list by using reply all !
for my reply see below...
On 07/17/2013 06:58 AM, Chadi Abdul Kader El Farran wrote:
Dear Hans-Rudolf,
Thank you for your reply.
The problem was solved by restarting.
Now my problem is that I am unable to upload files into my Galaxy.
I have put the following code in Universe_wsgi.ini:
# Directories of files contained in the following directory can be uploaded to
a library from the Admin view
library_import_dir = /upload
However when I upload files to a data library, all files will seem to have no
information (0 bytes)
That is very crypticand you need to provide more details
- did you get an error?
- how do you know the files are empty
- what happens if you work with the files
make sure to follow the wiki page:
http://wiki.galaxyproject.org/Admin/DataLibraries/Libraries
Hans-Rudolf
Can you kindly help me?
Thanks a lot for the support. With warmest regards
Chadi EL Farran (Post Graduate Student)
Dr. Jonathan Yuin-Han LOH's Laboratory
#08-01, Institute of Molecular and Cell Biology (IMCB)
61 Biopolis Drive
Re: [galaxy-dev] Galaxy Local Admin user
Hi Just double checking: Have you restarted Galaxy? Regards, Hans-Rudolf On 07/16/2013 07:23 AM, Chadi Abdul Kader El Farran wrote: Dear Madam/Sir, I have installed Galaxy Local in our system. I registered as a user. However when I try to give admin status to some users, I am unable despite following the steps mentioned in the help. To give a user Galaxy admin privileges, add their Galaxy login ( email ) to the list in the following config setting in the Galaxy configuration file universe_wsgi.ini. # this should be a comma-separated list of valid Galaxy users admin_users = us...@bx.psu.edu,us...@bx.psu.edu I have done what is mentioned above and the universe_wsgi.ini is edited accordingly. Still when I log on to Galaxy Local I don't see the admin tab. Can you kindly help me. With warmest regards Chadi EL Farran (Post Graduate Student) Dr. Jonathan Yuin-Han Loh's Laboratory #08-01, Institute of Molecular and Cell Biology (IMCB) 61 Biopolis Drive, Proteos Singapore 138673 Contact No: +65-91944793 E-mail: ch...@nus.edu.sg ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Default history
Hi Jordi Have you looked into using Data Libraries, see: http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Libraries http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Uploading%20Library%20Files Your anonymous users can then select any of the prepared data sets (via the 'Shared Data' tab), and easily import them into their histories without data duplication. Regards, Hans On 06/19/2013 09:32 PM, Jordi Vaquero wrote: Hello everyone, I am managing an instance of galaxy. We want to provide a set of example datasets for being used, we want this is by default datasets accessible for the anonymous user. there is any way to do that? I have tried to work the shared and publish option of history but they don't work as I wanted. Thank you in advanced Jordi ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] visulaization tool
Hi Shashi Have a look at: http://wiki.galaxyproject.org/Learn/Visualization http://wiki.galaxyproject.org/Visualization%20Setup Regards, Hans-Rudolf On 06/19/2013 08:07 AM, shashi shekhar wrote: Hi, i want to add new visualization tool in galaxy . can i get the required documents ? can u send me the details how can i add the visualization tool in galaxy . Regards shashi ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] trimest tool cuts the begining of my seq names in the output fasta file
Hi Inbar If you want to keep the NCBI stile fasta header, you need to change the setting for Output Sequence File Format: to NCBI style FASTA (m). By default, most EMBOSS tools change the fasta header format to the EMBOSS style format. Regards, Hans-Rudolf On 06/16/2013 12:01 PM, Inbar Plaschke wrote: Dear galaxy developers, When I use the EMBOSS tool: trimest https://main.g2.bx.psu.edu/tool_runner?tool_id=toolshed.g2.bx.psu.edu/repos/devteam/emboss_5/EMBOSS%3A%20trimest102/5.0.0,Trim poly-A tails off EST sequences , within galaxy The sequences names in the fasta output file comes out without their beginning, for instance: *gi|259479195|dbj|*AB491800.1|Brachionusplicatilisstt3mRNAforoligosaccharytransferaseSTT3subunithomolog,partialcds Will be shortened into: AB491800.1|Brachionusplicatilisstt3mRNAforoligosaccharytransferaseSTT3subunithomolog,partialcds How can I keep the beginning of the sequence names? Am I doing something wrong? Many thanks for your help Inbar Description: logo_50prcnt Inbar Plaschkes Bioinformatics Core Facility National Institute for Biotechnology Ben-Gurion University of the Negev Beer-Sheva 84105, Israel Building 51, room 323 Email: inba...@bgu.ac.il mailto:inba...@bgu.ac.il Tel: +972-8-6428676 Fax: +972-8-6479035 Web: http://bioinfo.bgu.ac.il http://bioinfo.bgu.ac.il/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Basic setup question
Hi Malcolm Displaying your data in browser outside of Galaxy is always a little bit tricky. And there are two issues: - first, getting the 'display at UCSC main' option to show up in the history item - second, getting the actual display (at UCSC) to work As far as I understand your e-mail, you currently struggle with the first? There are a few points to check a) (as you have written) make sure to have the option turned on in the ~/universe_wsgi.ini file, eg: ucsc_display_sites = main b) make sure two edit ~/tool-data/shared/ucsc/ucsc_build_sites.txt by listing all genomes you are interested to use for the display, eg: main http://genome.ucsc.edu/cgi-bin/hgTracks?mm7,mm8,mm9 The database attribute of your dataset (ie history item) has to be set to either mm7 or mm8 or mm9. You will not get the the display option if the database is set to mm10 or hg18 c) make sure two edit ~/datatypes_conf.xml, eg display file=ucsc/bam.xml / should not be commented-out. If all of this is correctly set up, you should get the 'display at UCSC main' for your mouse (mm9) bam file. Now comes the second issue, getting the display to work and here you might run into firewall-related issues. Check: Display Sites at http://wiki.galaxyproject.org/Admin/Config/Apache%20Proxy Unfortunately, at this point we struggle as well on our main production galaxy server which relies on external authentication. Regards, Hans-Rudolf On 06/06/2013 07:43 PM, Malcolm Tobias wrote: All, I manage a cluster and have been asked to set up a local instance of galaxy. I'm neither a galaxy user nor a biologist, so apologies in advance if my terminology is off. I have a basic setup running now, but have been asked to include the option to display at UCSC Main when a user uploads a bam and clicks on the dataset. Currently I have the options of display in IGB Local Web. I've tried this at a few of the other Galaxy sites, and the behavior seems unpredictable. Uploading the same data set to some sites gives me this option, while others don't. I'm assuming this is determined by the option in the universe_wsgi.ini file: # UCSC browsers: tool-data/shared/ucsc/ucsc_build_sites.txt ucsc_display_sites = main,test,archaea,ucla but uncommenting this option and restarting the server doesn't help. Increasing the log_level to DEBUG doesn't provide any insight. Any suggestions about how to proceed? Could there be any firewall-related issues? I ask because of this in the configuration file: # If use_remote_user = True, display application servers will be denied access # to Galaxy and so displaying datasets in these sites will fail. # display_servers contains a list of hostnames which should be allowed to # bypass security to display datasets. Please be aware that there are security # implications if this is allowed. More details (including required changes to # the proxy server config) are available in the Apache proxy documentation on # the wiki. Thanks in advance, Malcolm -- Malcolm Tobias 314.362.1594 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Error / import rpy module
Hi Sarah you need to install the rpy module: http://rpy.sourceforge.net/rpy.html As a warning: we have been struggling to get rpy to work with newer versions of R. In or experience in only works with R version 2.9 and older. Thanks to a recent effort of our sysadmin, we can now use it with 2.15 - which is good, but not great, as our default version of R is now 3.0 Although we haven't started to us it, I recommend using rpy2: http://rpy.sourceforge.net/rpy2.html Most tools have been refactored by John Chilton see: http://dev.list.galaxyproject.org/rpy2-Core-Tools-td4659832.html Regards, Hans-Rudolf On 06/05/2013 10:19 AM, Sarah Maman wrote: Hello, Some Galaxy tools failed to import rpy module and displays the following message: Traceback (most recent call last): File /usr/local/bioinfo/src/galaxy/galaxy-dist/tools/plotting/histogram.py, line 5, in module from rpy import * ImportError: No module named rpy For instance, the tool Regrouping (tools/stats/grouping.py) have been refactored in 2011 by Kanwei Li to use numpy instead of rpy. And this tool runs on my local instance of Galaxy. But I would like to be able use rpy module instead of refactor the others tools. Could you, please, help me to understand why the eggs ' import does not allow the use of rpy module? How to debug these tools? Thnaks in advance, Sarah ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Death to RPY - The BoF
On 06/05/2013 03:33 PM, Peter Cock wrote: On Wed, Jun 5, 2013 at 2:18 PM, John Chilton chil...@msi.umn.edu wrote: // - Finally, an idea - Death to RPY - The BoF. Everyone with a stake in this meets with laptops at the Galaxy conference together and we divvy up the remaining tools to convert (hopefully one tool per person). Every person is also assigned two of the existing converted tools to retest and clean up. As they are completed they are given to a core Galaxy developer (Nate, Dannon, Daniel, any volunteers?) to collect, re-retest, and okay. The changes are then committed all together at the end and the next Galaxy release contains no dependencies on rpy. These tools are not updated frequently, so the advantage of these all being upgraded in one commit before being shipped off to the tool shed is that an institution that is really stuck on rpy1 for whatever reason can just undo that single commit and keep going with the old tools indefinitely. If anybody with commit access signs on I will be happy create a BoF page and attend this. Thanks, -John If you all can pull that off in Oslo, great. Otherwise a piecewise conversion seems a sensible plan B, where both rpy (v1) and rpy2 are installed and things are updated on a tool by tool basis. The BoF group might also have some collective wisdom on how to deal with the question of multiple versions of R/BioConductor given this is important for reproducibility of many more complex and rapidly developed R tools. Peter I like the BoF suggestion, I am just afraid we will be too busy in Oslo to do it...we probably have to go for Peter's back-up plan. And, thank to JJ's efforts we are already well into the piecewise conversion. As part of our current work to set up a new box for our Galaxy development server at the FMI, we will use the already converted tools. I'll keep you posted. Regarding the question of multiple versions of R/BioConductor: we intend to keep them, and if necessary 'freeze' a tool...but we are looking for a better solution. Regards, Hans-Rudolf ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] FW: order of menu items
Hi Stephanie Try removing integrated_tool_panel.xml before you restart your galaxy server. There used to be a bug, where integrated_tool_panel.xml was not properly re-written after a restart. Regards, Hans-Rudolf On 06/03/2013 02:17 PM, Stephanie LE GRAS wrote: Dear all, We have 2 different instances of Galaxy within our institute (a test and a production instance). The thing is that the two tool menus are not in the same order and we don't understand why because we've got the same configuration files on both instance. Test instance Production instance: What could explain a difference in the order of the menu items? The thing is that I would like to get the NGS tools all together. Thank you in advance. Stephanie -- Stephanie Le Gras Bioinformatics engineer High throughput sequencing platform IGBMC 1, rue Laurent Fries 67404 ILLKIRCH Cedex France Tel. : +33 (0)3 88 65 32 73 Tel. Solexa : +33 (0)3 88 65 32 97 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Error in installing Galaxy.
On 05/21/2013 11:22 AM, sridhar srinivasan wrote:
Hi Rudolf,
Thanks for the reply..
Could you please tell how to check and set the PostgreSQL ..
I provide the correct username and password in universe.ini file..
when i give
#database_connection = postgres:///galaxy_prod
?user=galaxypassword=password
it gave error in history page
Content Encoding Error
The page you are trying to view cannot be shown because it uses an
invalid or unsupported form of compression.
* Please contact the website owners to inform them of this problem.
Now after changing it into
database_connection = postgres://galaxy:password@localhost:5432/galaxy_prod
it gives error which i mentioned in previous mail
Please suggest on this
Assuming you replace 'password' with the actual password, the line looks
correct to me. Try replacing 'localhost' with the IP address
Regards, Hans-Rudolf
Thanks
Sridhar
On Wed, May 8, 2013 at 7:16 PM, Hans-Rudolf Hotz h...@fmi.ch
mailto:h...@fmi.ch wrote:
Hi Sridhar
Have you set up your PostgreSQL database correctly? and provide the
right username and password in the 'universe_wsgi.ini' file ?
See:
http://wiki.galaxyproject.org/__Admin/Config/Performance/__ProductionServer?action=show__redirect=Admin%2FConfig%__2FPerformance#Switching_to_a___database_server
http://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer?action=showredirect=Admin%2FConfig%2FPerformance#Switching_to_a_database_server
Also, see this old thread, where the same error has been reported
before:
http://lists.bx.psu.edu/__pipermail/galaxy-dev/2010-May/__002624.html
http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-May/002624.html
Regards, Hans-Rudolf
On 05/08/2013 01:24 PM, sridhar srinivasan wrote:
Hi ,
I am getting error during Installing galaxy locally.
Traceback (most recent call last):
File
/illumina/apps/galaxy/galaxy-__dist/lib/galaxy/webapps/__galaxy/buildapp.py,
line 35, in app_factory
app = UniverseApplication( global_conf = global_conf,
**kwargs )
File
/illumina/apps/galaxy/galaxy-__dist/lib/galaxy/app.py, line 51,
in __init__
create_or_verify_database( db_url, kwargs.get(
'global_conf', {}
).get( '__file__', None ),
self.config.database_engine___options, app=self )
File
/illumina/apps/galaxy/galaxy-__dist/lib/galaxy/model/migrate/__check.py,
line 50, in create_or_verify_database
dataset_table = Table( dataset, meta, autoload=True )
File
/illumina/apps/galaxy/galaxy-__dist/eggs/SQLAlchemy-0.5.6___dev_r6498-py2.6.egg/__sqlalchemy/schema.py,
line 108, in __call__
return type.__call__(self, name, metadata, *args, **kwargs)
File
/illumina/apps/galaxy/galaxy-__dist/eggs/SQLAlchemy-0.5.6___dev_r6498-py2.6.egg/__sqlalchemy/schema.py,
line 236, in __init__
_bind_or_error(metadata).__reflecttable(self,
include_columns=include___columns)
File
/illumina/apps/galaxy/galaxy-__dist/eggs/SQLAlchemy-0.5.6___dev_r6498-py2.6.egg/__sqlalchemy/engine/base.py,
line 1261, in reflecttable
conn = self.contextual_connect()
File
/illumina/apps/galaxy/galaxy-__dist/eggs/SQLAlchemy-0.5.6___dev_r6498-py2.6.egg/__sqlalchemy/engine/base.py,
line 1229, in contextual_connect
return self.Connection(self, self.pool.connect(),
close_with_result=close_with___result, **kwargs)
File
/illumina/apps/galaxy/galaxy-__dist/eggs/SQLAlchemy-0.5.6___dev_r6498-py2.6.egg/__sqlalchemy/pool.py,
line 142, in connect
return _ConnectionFairy(self).__checkout()
File
/illumina/apps/galaxy/galaxy-__dist/eggs/SQLAlchemy-0.5.6___dev_r6498-py2.6.egg/__sqlalchemy/pool.py,
line 304, in __init__
rec = self._connection_record = pool.get()
File
/illumina/apps/galaxy/galaxy-__dist/eggs/SQLAlchemy-0.5.6___dev_r6498-py2.6.egg/__sqlalchemy/pool.py,
line 161, in get
return self.do_get()
File
/illumina/apps/galaxy/galaxy-__dist/eggs/SQLAlchemy-0.5.6___dev_r6498-py2.6.egg/__sqlalchemy/pool.py,
line 639, in do_get
con = self.create_connection()
File
/illumina/apps/galaxy/galaxy-__dist/eggs/SQLAlchemy-0.5.6___dev_r6498-py2.6.egg/__sqlalchemy/pool.py,
line 122, in create_connection
return _ConnectionRecord(self)
File
/illumina/apps/galaxy/galaxy-__dist/eggs/SQLAlchemy-0.5.6___dev_r6498-py2.6.egg/__sqlalchemy/pool.py,
line 198, in __init__
self.connection = self.__connect()
File
/illumina/apps/galaxy/galaxy-__dist/eggs/SQLAlchemy-0.5.6___dev_r6498
Re: [galaxy-dev] Error in installing Galaxy.
Hi Sridhar
Since we are using MySQL for our Galaxy server, I am kind of running out
of ideas...and hope someone else on the list can spot the error in your
universe.ini file.
Just one last remark to double check:
Can you log into the PostgreSQL database with the same credentials from
the command line?
psql -d galaxy_prod -p 5432 -h localhost -U galaxy
and this should prompt for the password
(again, I am not an PostgreSQL expert...)
Regards, Hans-Rudolf
On 05/21/2013 01:00 PM, sridhar srinivasan wrote:
Hello Rudolf,
I tried with the IP address it gave the error as
Is the server running on host 192.168.1.1 and accepting
TCP/IP connections on port 5432?
Then i try with 0.0.0.0 again i got same error
OperationalError: (OperationalError) FATAL: Ident authentication failed
for user galaxy
None None
Thanks
Sridhar
On Tue, May 21, 2013 at 3:08 PM, Hans-Rudolf Hotz h...@fmi.ch
mailto:h...@fmi.ch wrote:
On 05/21/2013 11:22 AM, sridhar srinivasan wrote:
Hi Rudolf,
Thanks for the reply..
Could you please tell how to check and set the PostgreSQL ..
I provide the correct username and password in universe.ini file..
when i give
#database_connection = postgres:///galaxy_prod
?user=galaxypassword=password
it gave error in history page
Content Encoding Error
The page you are trying to view cannot be shown because it uses an
invalid or unsupported form of compression.
* Please contact the website owners to inform them of
this problem.
Now after changing it into
database_connection =
postgres://galaxy:password@__localhost:5432/galaxy_prod
it gives error which i mentioned in previous mail
Please suggest on this
Assuming you replace 'password' with the actual password, the line
looks correct to me. Try replacing 'localhost' with the IP address
Regards, Hans-Rudolf
Thanks
Sridhar
On Wed, May 8, 2013 at 7:16 PM, Hans-Rudolf Hotz h...@fmi.ch
mailto:h...@fmi.ch
mailto:h...@fmi.ch mailto:h...@fmi.ch wrote:
Hi Sridhar
Have you set up your PostgreSQL database correctly? and
provide the
right username and password in the 'universe_wsgi.ini' file ?
See:
http://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer?action=showredirect=Admin%2FConfig%2FPerformance#Switching_to_a_database_server
http://wiki.galaxyproject.org/__Admin/Config/Performance/__ProductionServer?action=show__redirect=Admin%2FConfig%__2FPerformance#Switching_to_a___database_server
http://wiki.galaxyproject.__org/Admin/Config/Performance/__ProductionServer?action=show__redirect=Admin%2FConfig%__2FPerformance#Switching_to_a___database_server
http://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer?action=showredirect=Admin%2FConfig%2FPerformance#Switching_to_a_database_server
Also, see this old thread, where the same error has been
reported
before:
http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-May/002624.html
http://lists.bx.psu.edu/__pipermail/galaxy-dev/2010-May/__002624.html
http://lists.bx.psu.edu/__pipermail/galaxy-dev/2010-May/__002624.html
http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-May/002624.html
Regards, Hans-Rudolf
On 05/08/2013 01:24 PM, sridhar srinivasan wrote:
Hi ,
I am getting error during Installing galaxy locally.
Traceback (most recent call last):
File
/illumina/apps/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py,
line 35, in app_factory
app = UniverseApplication( global_conf = global_conf,
**kwargs )
File
/illumina/apps/galaxy/galaxy-dist/lib/galaxy/app.py, line 51,
in __init__
create_or_verify_database( db_url, kwargs.get(
'global_conf', {}
).get( '__file__', None ),
self.config.database_engine_options, app=self )
File
/illumina/apps/galaxy/galaxy-dist/lib/galaxy/model/__migrate/__check.py,
line 50, in create_or_verify_database
dataset_table = Table( dataset, meta,
autoload=True )
File
/illumina/apps/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/schema.py,
line 108, in __call__
return type.__call__(self, name, metadata, *args,
**kwargs)
File
/illumina/apps/galaxy/galaxy-dist/eggs
Re: [galaxy-dev] Bug in history when using Admin account ??
Hi Neil Just double checking: Have you looked at the list of 'Saved Histories'? For each user, you probably have several histories called 'Unnamed history' and one of them may contain the data you uploaded previously. Whenever you log out and log in as a different user, a new, empty history called 'Unnamed history' is created. Regards, Hans-Rudolf On 05/17/2013 08:04 AM, neil.burd...@csiro.au wrote: Hi, I believe I have found a bug? Or maybe I’m not using Galaxy correctly? These are the steps I used: ·Downloaded the latest version of Galaxy ·Created a user m...@me.com mailto:m...@me.com ·Uploaded several files ·Logged out ·Create a user y...@you.com mailto:y...@you.com ·Uploaded several files ·Logged out and then logged in as an Admin user (specified in universe_wsgi.ini file) ·Uploaded a file. Now when I log into m...@me.com mailto:m...@me.com or y...@you.com mailto:y...@you.com the history shows only the uploaded file that I uploaded as an Admin. Even logging in as Admin and using the “impersonate” feature doesn’t allow me to view the history of m...@me.com mailto:m...@me.com or y...@you.com mailto:y...@you.com. The files still exist as I can see them in database/files/000 Is this a bug? Or are Admins not supposed to upload files? Also I tried: ·Deleted the above galaxy-dist ·Downloaded the latest version of Galaxy ·Created a user m...@me.com mailto:m...@me.com ·Uploaded several files ·Logged out ·Create a user y...@you.com mailto:y...@you.com ·Uploaded several files ·Logged out and then logged in as an Admin user (specified in universe_wsgi.ini file) ·Didn’t upload a file ·Logged out of Admin Now when I log into m...@me.com mailto:m...@me.com or y...@you.com mailto:y...@you.com the history is empty. This can’t be correct is it? Neil ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] rpy2 Core Tools
This is great! We will use them Thank you very much Hans-Rudolf On 05/16/2013 05:01 PM, John Chilton wrote: I had promised these to someone on a bitbucket issue years ago and I said I would send them to Dannon and Nate on IRC today, so I am just going to spam these out far and wide. These are the MSI versions of many of the core tools that have been rewritten to target rpy2 instead of rpy . Caveat emptor, these were never meant for external consumption. It looks like the ones we never got to include: rgenetics/rgQC.py, regVariation/rcve.py, ngs_simulation/ngs_simulation.py, metag_tools/short_reads_figure_high_quality_length.py, and taxonomy/poisson2test.py. -John ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Error in installing Galaxy.
Hi Sridhar
Have you set up your PostgreSQL database correctly? and provide the
right username and password in the 'universe_wsgi.ini' file ?
See:
http://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer?action=showredirect=Admin%2FConfig%2FPerformance#Switching_to_a_database_server
Also, see this old thread, where the same error has been reported before:
http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-May/002624.html
Regards, Hans-Rudolf
On 05/08/2013 01:24 PM, sridhar srinivasan wrote:
Hi ,
I am getting error during Installing galaxy locally.
Traceback (most recent call last):
File
/illumina/apps/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py,
line 35, in app_factory
app = UniverseApplication( global_conf = global_conf, **kwargs )
File /illumina/apps/galaxy/galaxy-dist/lib/galaxy/app.py, line 51,
in __init__
create_or_verify_database( db_url, kwargs.get( 'global_conf', {}
).get( '__file__', None ), self.config.database_engine_options, app=self )
File
/illumina/apps/galaxy/galaxy-dist/lib/galaxy/model/migrate/check.py,
line 50, in create_or_verify_database
dataset_table = Table( dataset, meta, autoload=True )
File
/illumina/apps/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/schema.py,
line 108, in __call__
return type.__call__(self, name, metadata, *args, **kwargs)
File
/illumina/apps/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/schema.py,
line 236, in __init__
_bind_or_error(metadata).reflecttable(self,
include_columns=include_columns)
File
/illumina/apps/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py,
line 1261, in reflecttable
conn = self.contextual_connect()
File
/illumina/apps/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py,
line 1229, in contextual_connect
return self.Connection(self, self.pool.connect(),
close_with_result=close_with_result, **kwargs)
File
/illumina/apps/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/pool.py,
line 142, in connect
return _ConnectionFairy(self).checkout()
File
/illumina/apps/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/pool.py,
line 304, in __init__
rec = self._connection_record = pool.get()
File
/illumina/apps/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/pool.py,
line 161, in get
return self.do_get()
File
/illumina/apps/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/pool.py,
line 639, in do_get
con = self.create_connection()
File
/illumina/apps/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/pool.py,
line 122, in create_connection
return _ConnectionRecord(self)
File
/illumina/apps/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/pool.py,
line 198, in __init__
self.connection = self.__connect()
File
/illumina/apps/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/pool.py,
line 261, in __connect
connection = self.__pool._creator()
File
/illumina/apps/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/strategies.py,
line 80, in connect
raise exc.DBAPIError.instance(None, None, e)
OperationalError: (OperationalError) FATAL: Ident authentication failed
for user galaxy
None None
I created a user galaxy to install galaxy locally.
Thanks in Advance.
Sridhar
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Teaching using Galaxy
Hi David I have not used or tested it myself yet, but: have you looked into Bio-Linux ? The latest release (Bio-Linux 7) comes with Galaxy pre-installed, see: http://nebc.nerc.ac.uk/tools/bio-linux/bio-linux-7-info Regards, Hans-Rudolf On 04/25/2013 12:52 AM, David Joly wrote: Thanks Dave! The focus is clearly more like your second description. The course is not intended to show them how to use command lines, or how to navigate in a UNIX environment and how to program. In fact, I think some would call what I have in mind more of a computational biology course than a bioinformatics course, but I think it is not really important here. I'm thinking of another course to introduce them to the UNIX environment and using Python to script simple programs, but for this one, I'd prefer staying away from a terminal that can be horrifying at first (for most students, black screen = evil) So, the course is about knowing the tools and their basics, and how to use them (in this case using Galaxy as the platform). What is a sequence alignment, why should we do a sequence alignment, how a sequence alignment work, and how can I do one in Galaxy. This is only one example, other (and often inter-related) topics would include BLAST, phylogenetics, sequence polymorphisms, and so on, up to analyzing next-generation sequencing data... Regarding how I'll set my environment, I was thinking of setting a local instance (I have one on my computer right now and I'm already getting familiar with it). The hardware on which I'll set the instance for the course is another issue, but this is not the topic of this email... I'll have a better look at how I could use published histories and workflows... Thanks, DJ 2013/4/24 Dave Clements cleme...@galaxyproject.org mailto:cleme...@galaxyproject.org Hi David, Using Galaxy to teach undergraduates is a long term interest of mine. Which, unfortunately, does not mean I have yet put a lot of thought into it. However, lack of thought hasn't stopped me yet. First, this topic was discussed in a breakout at last year's GCC: http://wiki.galaxyproject.org/Events/GCC2012/Program/Breakouts/BioinformaticsTraining That's more of a discussion than a set of best practices. What do you want them to spend time learning? Do you want them to learn the (sometimes grinding) details of using the command line, and how to install software and their dependencies on a Linux box? Or do you want to focus mainly on the high level stuff like here is how and why BLAST works, and get some practice using it? I think this question of focus is a central one, and it's one that well-informed people disagree on. Galaxy is a great platform for focussing on the high-level stuff and avoiding the frustration that can come with installing a C compiler, for example. On the other hand, if you really want them to learn the command line, then you might want to start elsewhere. If you do use Galaxy for teaching, there are a couple of ways you could do it. First, I recommend setting up your own server(s) either locally or on the cloud. (The AWS in Education grant program is built just for this case.) If you wanted to teach them a mixture of high-level and low level, you could start them out as users on a shared server, and then later in the course have them setup their own Galaxy on a cloud instance. Also, for Galaxy training we find that published histories, workflows, and Galaxy Pages, are a superb way to create exercises. Hope this helps, Dave C. On Mon, Apr 22, 2013 at 12:30 PM, David Joly idj...@gmail.com mailto:idj...@gmail.com wrote: Hi everybody! I am currently creating a bioinformatics course for undergraduate (biology students with no knowledge of programming). I would like to use Galaxy as their everyday platform where they would learn the basics and use the appropriate tools (BLAST and databases, multiple alignment, phylogenetics, dealing with omics data, and so on). Is there any available resources about using Galaxy for teaching (undergraduates)? Any suggestions of good textbooks? Not a Galaxy textbook of course, but a bioinformatics textbook that would be a good companion to help the students understand the basics behind the tools. Thanks, DJ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- http://galaxyproject.org/GCC2013 http://galaxyproject.org/
Re: [galaxy-dev] where the log in information saved?
Hi Shenwiyn I am not sure why you are interested in passwordsbut to see the e-mail address' have a look at the Manage users link on the Admin interface. All the user information is stored in the PostgreSQL (or MySQL) database. Regards, Hans-Rudolf On 04/16/2013 11:21 AM, shenwiyn wrote: Hi everyone, I installed my local galaxy successfully.But I have a problem about how to find out the log in information of each user ,such as the register mail and the password .And which files do this informations saved? Thank you! shenwiyn ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Trouble with Toolshed on MySQL
Hi Rob Yes, this looks like an SQLAlchemy issue, producing an 'create index' statement which is not compatible with MySQL. I guess line 146 in ~/lib/galaxy/webapps/community/model/migrate/versions/0001_initial_tables.py is the culprit: Column( annotation, TEXT, index=True) ) I don't know whether this works, but I would suggest you change that line to: Column( annotation, TEXT) ) run it again. And then manually add the index by executing CREATE INDEX ix_tool_annotation_association_annotation ON tool_annotation_association (annotation(100)) The problem is: MySQL can only index the first x characters of a BLOB or TEXT. Using the above suggested statement will create an index using the first 100 characters of the name column. See also: http://dev.mysql.com/doc/refman/5.0/en/create-index.html Regards, Hans-Rudolf On 04/12/2013 08:24 AM, Rob Hooft wrote: To ease the maintenance of our Dutch National galaxy, we're trying to set up our own toolshed on a system that happens to have a nicely large MySQL database running. We'd like to make it use that database. The setup has been ok, until we started running the toolshed on the MySQL server for the first time, which resulted in the message in the community_webapp.log I copied below. Is this an incompatibility? Regards, Rob Hooft File lib/galaxy/webapps/community/model/migrate/versions/0001_initial_tables.py, line 150, in upgrade metadata.create_all() ... self._handle_dbapi_exception(e, statement, parameters, cursor, context) File /opt/toolshed/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py, line 931, in _handle_dbapi_exception raise exc.DBAPIError.instance(statement, parameters, e, connection_invalidated=is_disconnect) OperationalError: (OperationalError) (1170, BLOB/TEXT column 'annotation' used in key specification without a key length) u'CREATE INDEX ix_tool_annotation_association_annotation ON tool_annotation_association (annotation)' () Removing PID file community_webapp.pid -- Rob W.W. Hooft Chief Technology Officer BioAssist, Netherlands Bioinformatics Centre http://www.nbic.nl/Skype: robhooftGSM: +31 6 27034319 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
