Re: [galaxy-dev] Tool XML parameter options from locally stored files
Hi Greg, thanks for the suggestion. This line doesn't seem to work: options from_file=some_value Galaxy looks for a file at galaxy_dir/tool-data/some_value rather than the value given in the .loc file I would also like to be able to use the galaxy data tables rather then referring directly to a .loc file. Any ideas? Shaun Quoting Greg Von Kuster g...@bx.psu.edu on Wed, 13 Jul 2011 10:28:30 -0400: Hi Shaun, This should be possible. It's been quite a long time since I worked on tools, so maybe others can provide better assistance if I am incorrect here. You'll need to have a conditional something like the following ( not tested whatsoever ): conditional name=selected_file_select_list param name=selected_gtf_file type=select label=Gtf file by species options from_file=some_file column name=name index=2/ column name=some_value index=1/ /options /param when value=some_value param name=values_from_selected_gtf_file type=select label=Values from selected gtf file / options from_file=some_value column name=name index=2/ column name=some_value index=1/ /options /when /conditional On Jul 13, 2011, at 9:45 AM, SHAUN WEBB wrote: Hi Greg, I'm not sure if this answers my question. Using options_from_file is fine if I want to refer to the same file each time. In this case I want the first select to choose which gtf file to look at (by species) and the second select to pull options (gtf annotation types) from this file. So I would somehow have to reference the file using the first parameter rather than giving a hard-coded path. Is that possible? Shaun Quoting Greg Von Kuster g...@bx.psu.edu on Wed, 13 Jul 2011 09:23:19 -0400: Hello Shaun, Use the 'from_file' option. The referenced file should be in the ~/tool-data directory in the Galaxy root. There are several example tools in the distribution that use this option. Here is the example code from the annotation profiler tool: param name=table_names type=drill_down display=checkbox hierarchy=recurse multiple=true label=Choose Tables to Use help=Selecting no tables will result in using all tables. from_file=annotation_profiler_options.xml/ and here is an example from the microbial import tool: param name=kingdom type=select label=Select the Desired Kingdom options from_file=microbial_data.loc startswith=ORG column name=name index=3/ column name=value index=3/ filter type=unique_value name=unique column=3/ /options /param On Jul 13, 2011, at 6:17 AM, SHAUN WEBB wrote: Hi, I am trying to write a tool xml file and want to do the following: I have a number of GTF files stored locally and referenced in a data table. I give the user a drop down list of those available using the following code: param name=gtf type=select label=GTF File options from_data_table=gtf_files/ /param I then want them to choose an annotation type based on those present in the selected gtf file above. It seems simple enough to do this if the gtf file is in your history already: param name=feature type=select multiple=true label=Extract features options from_dataset=gtf column name=name index=2/ column name=value index=2/ filter type=unique_value name=unique column=2/ /options /param But is there a way to query locally stored files via datatables/.loc files to pull out options when the file is dynamically chosen by a different parameter? This code is also wrapped in a conditional, I'm not sure if that will affect anything. Thanks Shaun -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ Greg Von Kuster Galaxy Development Team g...@bx.psu.edu -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ Greg Von Kuster Galaxy Development Team g...@bx.psu.edu -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the
Re: [galaxy-dev] fyi: can't upload to toolshed
Server ErrorAn error occurred. See the error logs for more information. (Turn debug on to display exception reports here) My guess is that this will be a problem with your tool configuration file in XML format. -- Vipin ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Installing tools with dependencies using mi-deployment scripts
Each use of fabric's `run` method is independent so they cannot be chained;
you would either need to prepend the export command to the command that
depends on the fact PYTHONPATH is set (e.g., run(export PYTHONPATH=...;
command that requires given PP) or (better approach) create an env.sh
file in the Qiime's install dir so that it includes the required 'export
PYTHONPATH=...'. When Galaxy invokes Qiime, it will automatically source
that env.sh (assuming you have config variable `tool_dependency_dir =
/mnt/galaxyTools/tools` set in Galaxy's universe_wsgi.ini).
Enis
On Fri, Jul 15, 2011 at 12:19 PM, Mattias de Hollander
m.dehollan...@nioo.knaw.nl wrote:
Hello Enis,
I am trying to add PyCogent to the PYTHONPATH, but it seems the Fabric
run command does not recognize it.
I tried: run(export
PYTHONPATH=/mnt/galaxyTools/tools/PyCogent/1.5.1/lib/python2.6/site-packages/:$PYTHONPATH)
and then use cmd_install() to install Galaxy, but I keep getting the
message that PyCogent is not installed.
The normal strategy to use sys.path will not work because that will
adjust the PYTHONPATH on my local machine, not in the image if I am
correct.
Do you know a way to set the PYTHONPATH for Fabric' run() command?
Thanks,
Mattias
On Thu, 2011-07-14 at 09:05 +, Enis Afgan wrote:
Hi Mattias,
PyCogent is not currently being installed as part of mi-deployment, so
the method for installing it should be added (if you do so, please
consider issuing a pull request on bitbucket so it can be added to the
project).
When it comes to composing $PYTHONPATH to include the reference to
PyCogent via Qiime's env.sh - this should be done as part of the Qiime
installation method in mi-deployment (something like export
PYHTONPATH=path where PyCogent was installed; export PATH=...).
Sourcing of the tool's env.sh script is prepended to the execution
command when running the tool so the appropriate environment settings
will be loaded before the tool is run.
Enis
On Wed, Jul 13, 2011 at 12:09 PM, Mattias de Hollander
m.dehollan...@nioo.knaw.nl wrote:
Hello,
I am trying to add tools (PyCogent, Qiime) to a Cloudman image
using the
mi-deployment scripts. Qiime depends on PyCogent and it needs
to be
accessible on the python path, otherwise I get an error:
PyCogent not installed but required. (Is it installed? Is it
in the
current user's $PYTHONPATH or site-packages?)
Is there a way to add the paths in env.sh to the sys.path
during
installation using tools_fabfile.py?
All the tools are in /mnt/galaxyTools/tools.
I tried the DependencyManager from galaxy but then I only
receive the
path to the env.sh file:
In [43]: dependency_manager.find_dep('blast')
Out[43]:
('/mnt/galaxyTools/tools/blast/2.2.25/env.sh',
'/mnt/galaxyTools/tools/blast/2.2.25',
'2.2.25')
Is there a smarter way to add files to the PYTHONPATH during
the
execution of tools_fabfile.py?
Thanks,
Mattias
--
Bioinformatician
Netherlands Institute of Ecology (NIOO-KNAW)
Wageningen, the Netherlands
--
Bioinformatician
Netherlands Institute of Ecology (NIOO-KNAW)
Wageningen, the Netherlands
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] fyi: can't upload to toolshed
Hello Ed, Looking at the logs, I see that you uploaded to your Roche454 tool suite repo. Is that correct? I don't see any exceptions in the logs, so not sure what happened. Are you still having this issue? Thanks, Greg Von Kuster On Jul 14, 2011, at 9:32 PM, Edward Kirton wrote: Server Error An error occurred. See the error logs for more information. (Turn debug on to display exception reports here) ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ Greg Von Kuster Galaxy Development Team g...@bx.psu.edu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Database scheme migration.
Good afternoon
I am having some trouble updating our local galaxy, in summary it seems to
fail. I have included the errors below and would really appreciate any
assistance that could be offered.
galaxy@jic55119:~/software/galaxy-ceneral$ sh manage_db.sh upgrade
57 - 58...
Migration script to create table for exporting histories to archives.
(ProgrammingError) there is no unique constraint matching given keys for
referenced table job
'\nCREATE TABLE job_export_history_archive (\n\tid SERIAL NOT NULL, \n\tjob_id
INTEGER, \n\thistory_id INTEGER, \n\tdataset_id INTEGER, \n\tcompressed
BOOLEAN, \n\thistory_attrs_filename TEXT, \n\tdatasets_attrs_filename TEXT,
\n\tjobs_attrs_filename TEXT, \n\tPRIMARY KEY (id), \n\t FOREIGN KEY(job_id)
REFERENCES job (id), \n\t FOREIGN KEY(history_id) REFERENCES history (id), \n\t
FOREIGN KEY(dataset_id) REFERENCES dataset (id)\n)\n\n' {}
done
58 - 59...
….INTERMEDIATE STEPS REMOVED NO ERRORS REPORTED……
78 - 79...
Migration script to add the job_to_input_library_dataset table.
Creating job_to_input_library_dataset table failed: (ProgrammingError) there is
no unique constraint matching given keys for referenced table job
'\nCREATE TABLE job_to_input_library_dataset (\n\tid SERIAL NOT NULL,
\n\tjob_id INTEGER, \n\tldda_id INTEGER, \n\tname VARCHAR(255), \n\tPRIMARY KEY
(id), \n\t FOREIGN KEY(job_id) REFERENCES job (id), \n\t FOREIGN KEY(ldda_id)
REFERENCES library_dataset_dataset_association (id)\n)\n\n' {}
Done
Further information
galaxy@jic55119:~/software/galaxy-ceneral$ hg tip
changeset: 5793:f2638528e904
tag: tip
user:jeremy goecks jeremy.goe...@emory.edu
date:Thu Jul 14 22:54:43 2011 +0200
summary: Fix bug in trackster filtering of GFF datasets.
galaxy@jic55119:~/software/galaxy-ceneral$ python -V
Python 2.6.6
galaxy@jic55119:~/software/galaxy-ceneral$ psql --version
psql (PostgreSQL) 8.4.8
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] Installing tools with dependencies using mi-deployment scripts
I just found out the former approach you mentioned. Its now installing
perfectly. The latter approach won't work for a Qiime installation
because its needs PyCogent on its path during the setup.py phase.
Thanks for clarification.
If I finished adding Qiime' dependencies I will put it on bitbucket.
Greetz,
Mattias
On Fri, 2011-07-15 at 13:44 +, Enis Afgan wrote:
Each use of fabric's `run` method is independent so they cannot be
chained; you would either need to prepend the export command to the
command that depends on the fact PYTHONPATH is set (e.g., run(export
PYTHONPATH=...; command that requires given PP) or (better
approach) create an env.sh file in the Qiime's install dir so that it
includes the required 'export PYTHONPATH=...'. When Galaxy invokes
Qiime, it will automatically source that env.sh (assuming you have
config variable `tool_dependency_dir = /mnt/galaxyTools/tools` set in
Galaxy's universe_wsgi.ini).
Enis
On Fri, Jul 15, 2011 at 12:19 PM, Mattias de Hollander
m.dehollan...@nioo.knaw.nl wrote:
Hello Enis,
I am trying to add PyCogent to the PYTHONPATH, but it seems
the Fabric
run command does not recognize it.
I tried: run(export
PYTHONPATH=/mnt/galaxyTools/tools/PyCogent/1.5.1/lib/python2.6/site-packages/:$PYTHONPATH)
and then use cmd_install() to install Galaxy, but I keep
getting the
message that PyCogent is not installed.
The normal strategy to use sys.path will not work because that
will
adjust the PYTHONPATH on my local machine, not in the image if
I am
correct.
Do you know a way to set the PYTHONPATH for Fabric' run()
command?
Thanks,
Mattias
On Thu, 2011-07-14 at 09:05 +, Enis Afgan wrote:
Hi Mattias,
PyCogent is not currently being installed as part of
mi-deployment, so
the method for installing it should be added (if you do so,
please
consider issuing a pull request on bitbucket so it can be
added to the
project).
When it comes to composing $PYTHONPATH to include the
reference to
PyCogent via Qiime's env.sh - this should be done as part of
the Qiime
installation method in mi-deployment (something like export
PYHTONPATH=path where PyCogent was installed; export
PATH=...).
Sourcing of the tool's env.sh script is prepended to the
execution
command when running the tool so the appropriate environment
settings
will be loaded before the tool is run.
Enis
On Wed, Jul 13, 2011 at 12:09 PM, Mattias de Hollander
m.dehollan...@nioo.knaw.nl wrote:
Hello,
I am trying to add tools (PyCogent, Qiime) to a
Cloudman image
using the
mi-deployment scripts. Qiime depends on PyCogent and
it needs
to be
accessible on the python path, otherwise I get an
error:
PyCogent not installed but required. (Is it
installed? Is it
in the
current user's $PYTHONPATH or site-packages?)
Is there a way to add the paths in env.sh to the
sys.path
during
installation using tools_fabfile.py?
All the tools are in /mnt/galaxyTools/tools.
I tried the DependencyManager from galaxy but then I
only
receive the
path to the env.sh file:
In [43]: dependency_manager.find_dep('blast')
Out[43]:
('/mnt/galaxyTools/tools/blast/2.2.25/env.sh',
'/mnt/galaxyTools/tools/blast/2.2.25',
'2.2.25')
Is there a smarter way to add files to the
PYTHONPATH during
the
execution of tools_fabfile.py?
Thanks,
Mattias
--
Bioinformatician
Netherlands Institute of Ecology (NIOO-KNAW)
Wageningen, the Netherlands
--
Bioinformatician
Netherlands Institute of Ecology (NIOO-KNAW)
Wageningen, the Netherlands
--
Bioinformatician
Netherlands Institute of Ecology (NIOO-KNAW)
Wageningen, the Netherlands
___
Please keep all replies on the list by using reply all
in your mail client. To manage your
Re: [galaxy-dev] Multiple fastq files per forward/reverse mates (tophat and related tools)
Gus, Good to hear about your progress. Please send me a patch of your changes and I'll take a look. You can create a patch using Mercurial from the command line as follows. % hg diff tophat.patch Thanks, J. On Jul 13, 2011, at 7:37 PM, W. Augustine Dunn III wrote: First off thank you heartily for pointing me in the right direction here. It SEEMS like i have managed to accomplish this. Meaning: i finally got the run to get past the parsing of the config file and now have a process running on the cpu and the server log shows a command line string that matches what I was trying to get. However I am VERY new at the whole Cheetah craziness, so I DEF want you folks to double check my work. I would LOVE to contribute a patch but have these questions and concerns: I cloned the 'dist' branch not the 'central' branch originally and am not sure if my tophat_wrapper.xml version will clash with any that have been included in the central branch, I would assume that the system's diff capabilities SHOULD be able to sort this out but I am WAY more familiar with git than hg. I have created a bitbucket account and supplied my pubkey, but I dont see anything on the wiki that describes how a person gets push or similar permissions on the galaxy project what is the procedure for correctly submitting these types of piecemeal single file changes? Thanks again for your help and I am very excited to have a chance to give back! Gus On Mon, Jul 11, 2011 at 8:51 AM, Jeremy Goecks jeremy.goe...@emory.edu wrote: I recommend to use the repeat tag (see http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax) around the existing parameter to make it possible to select multiple files. Sarah's suggestion is spot on. See the Cuffcompare wrapper for an example of how to use this tag. Gus, if you end up modifying the Tophat wrapper to support multiple input datasets, please submit a patch or fork galaxy-central via bitbucket and I'll integrate these enhancements into the Galaxy code base as this would be nice functionality to have. Best, J. -- In science, fact can only mean confirmed to such a degree that it would be perverse to withhold provisional assent. I suppose that apples might start to rise tomorrow, but the possibility does not merit equal time in physics classrooms. -Stephen Jay Gould ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Galaxy: self-cleaning on file_path
Hello, We have recently put a local installation of Galaxy on our network and connected it to a MYSQL database. In order to have more space available for uploaded and created files, we also changed the values of file_path and new_file_path in the config file. The file_path field was changed to a larger, communal location. The new_file_path was changed to a directory called tmp in that same communal location. This location, however, undergoes self-cleaning; deleting any files that reach 7 days old (and any directories that have no files less than 7 days old). This would include any datasets uploaded or created by Galaxy. Can the Galaxy framework and schema handle this occurrence? Or will it cause errors to occur? --- Ravi Sanka ICS - Bioinformatics Engineer J. Craig Venter Institute 301-795-7743 --- ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
