[galaxy-dev] Data uploaded with new upload tool doesn't get added to history
Hey there Using the new upload tool my data uploads but never appears in the history. All I can see in paster.log when I do an upload is: 10.8.0.2 - - [09/Oct/2014:20:43:58 +0200] POST /tool_runner/index HTTP/1.0 200 - https://phillipl.sanbi.ac.za/root; Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:32.0) Gecko/20100101 Firefox/32.0 10.8.0.2 - - [09/Oct/2014:20:43:58 +0200] GET /api/histories/f597429621d6eb2b/contents HTTP/1.0 200 - https://phillipl.sanbi.ac.za/root; Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:32.0) Gecko/20100101 Firefox/32.0 10.8.0.2 - - [09/Oct/2014:20:43:58 +0200] GET /api/histories/f597429621d6eb2b HTTP/1.0 200 - https://phillipl.sanbi.ac.za/root; Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:32.0) Gecko/20100101 Firefox/32.0 No errors in the javascript console, and this happens with both Chrome and Firefox (on Linux), code is latest galaxy-central. Any ideas where to debug? Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Installed tool invisible in Galaxy
Hi there I updated a tool of mine recently, committed to the local toolshed, did the update from within the Admin panel and then got this error: Error - type 'exceptions.AttributeError': 'Tool' object has no attribute 'citations' A restart of Galaxy seemed to fix the problem, but since then the tool could not be found - it is not in any of the Tool categories. Deleting and re-installing the tool does not fix this problem - and on installation time I never get asked what group to add it to. It seems as if it is somehow stuck in an incorrect state. Any ideas where to go poking? Presumably there is somewhere in the database to go poking to reset things. Thanks, Peter P.S. this is with the latest Galaxy from galaxy-central. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Installed tool invisible in Galaxy
Aha, that's true! With the XML fixed, the installation dialogue changes, and displays the Add new tool panel section and subsequent parts of the page. This would *appear* to come from the check for includes_tools_for_display_in_tool_panel on line 1620 of lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py which in turn is (eventually) pulled from the repository model I think. Peter On 12/08/2014 20:03, John Chilton wrote: I think your tool has an XML error in it (the in_graph_filename param is not being closed) - I imagine this would prevent Galaxy from displaying the tool - cannot say if it is the only error though: param format=graphml name=in_graph_filename type=data label=Protein interaction graph /inputs -John On Tue, Aug 12, 2014 at 2:00 PM, Peter van Heusden p...@sanbi.ac.za wrote: 1. Yes, its definitely a delete (checked the box). 2. The tool, once added, creates: a. the correct directory structure in shed_tools/ b. an entry in the tool_shed_repository table that *appears* to be correct c. an entry in shed_tool_conf.xml with the annotation of the Graph section (where it was originally located before being deleted d. *no entry* in integrated_tool_panel.xml 3. When the tool is deleted: a. the shed_tools/ directory is cleaned out except for the top level directory (graph_report) of the tool b. the entry in the tool_shed_repository table remains c. the entry in shed_tool_conf.xml remains d. there is still no integrated_tool_panel.xml In terms of the database, tool_version_association references tool_version which references tool_shed_repository, so you need to delete entries all along that chain to remove an entry from tool_shed_repository. The tool and its type dependency are attached. On 12/08/2014 16:43, John Chilton wrote: Hey Peter, When you are deleting and reinstalling the tool - can you confirm for me that you are definitely deleting the tool and not just deactivating it? I don't know where to hack around from there - I think repositories are tracked in the database (tool_shed_repository) table, in shed_tool_conf.xml, integrated_tool_panel.xml, shed_tools directory, tool_dependencies directory. It would be interesting to know which of these still have references to the tool after you delete it. I am also interested in the underlying bug - are you running multiple Galaxy processes? Can you send me the tool that produced this error? -John On Tue, Aug 12, 2014 at 10:08 AM, Peter van Heusden p...@sanbi.ac.za wrote: Hi there I updated a tool of mine recently, committed to the local toolshed, did the update from within the Admin panel and then got this error: Error - type 'exceptions.AttributeError': 'Tool' object has no attribute 'citations' A restart of Galaxy seemed to fix the problem, but since then the tool could not be found - it is not in any of the Tool categories. Deleting and re-installing the tool does not fix this problem - and on installation time I never get asked what group to add it to. It seems as if it is somehow stuck in an incorrect state. Any ideas where to go poking? Presumably there is somewhere in the database to go poking to reset things. Thanks, Peter P.S. this is with the latest Galaxy from galaxy-central. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] No action for /admin_toolshed/prepare_for_install
I am trying to install a tool (tmap_wrapper) from the Galaxy toolshed using the Admin interface to Galaxy. Unfortunately, each time I do this I get an error page with the message: No action for /admin_toolshed/prepare_for_install My Galaxy installation is galaxy-dist, revision 7148:17d57db9a7c0. Does anyone know why this might be happening? Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Giardine et al Galaxy paper (2005)
Hi everyone The Galaxy system as described in Belinda Giardine et al's 2005 Genome Research paper (Galaxy: A platform for interactive large-scale genome analysis) appears to be radically different from the current Galaxy, at least in its technical specification. The paper mentions a C core spoken to by a Perl-based web user interface (referred to as the History User Interface). The co-authors on the paper are, however, familiar names from the current Galaxy team. Was this paper describing something in the pre-history of current Galaxy (it sounds rather like the current History but without workflows and of course implemented on a different platform)? Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Using workflows as components in workflows?
Thanks Jeremy I've given up on that approach to nested workflows - too inefficient, too fragile. If I have lots of time (and a grad student to help out) I've got some ideas about having a deep look at the tool runner side of Galaxy with an eye to efficiently implementing the mapping of input data to (sub)workflows. Right now the process of spawning a new Galaxy task for e.g. each line in a tabular file is just too heavyweight, thus the sub-workflow got implemented in Python. Peter On 24/05/2012 04:30, Jeremy Goecks wrote: Peter, I'm not ignoring you. However, there are others on the Galaxy team that are more familiar with the API and can provider better answers. I expect they'll chime in soon to address your questions. Best, J. On May 20, 2012, at 5:39 PM, Peter van Heusden wrote: Hi Jeremy I'm need this for something I'm implementing at the moment, and how I'm thinking about it is making a tool that uses the API to call a workflow. There are a few problems though, correct me if I'm wrong: 1) In order to make an input history item available to the called workflow, the tool needs to somehow know about history items, but the tool xml passes in parameters as data files. This could probably be remedied by providing a type=history_item parameter to param that would provide the id associated with the history item. In the interem, just to test things, I'm passing in parameters as a history:history_item string (yeah I know, ugly!). 2) My particular tool needs to take a history item, splits it into partitions, and call a workflow with each of those partitions. For this to work, the partition needs to be uploaded as a new history item, but that is currently not possible. The other possibility is to create a tool that does the split, have it in a single-tool workflow (because workflows can be called from the API in such a way that their output goes to a new history, whereas I don't see that in the tool interface) and then iterate through the history that contains the split data, calling the analysis workflow on each item. Peter P.S. for my particular problem - call a bunch of tools, once for each row in a file of tabular data - it would be WAY easier to just write everything in a Python script, but I'm trying to see what is do-able within Galaxy. On 20/05/2012 16:55, Jeremy Goecks wrote: Is there any way we can speed up the implementation of this issue? Community contributions and always encouraged and welcomed. Partial solutions are fine, and self-contained contributions are likely to be included more quickly because they are easier to review. Thanks, J. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] LateValidationError in Extract features
Hi there I've got a workflow that uses BioPerl's bp_genbank2gff3 to convert Genbank to GFF3, then hands the GFF3 to Extract features to filter to only genes, before moving on. The workflow JSON is at http://pastebin.com/zHWsC6YT. Step 4 - the Genbank2GFF runs fine, and if I view the output in the history I can see the resulting GFF, but step 8 - the Extract features, fails with a LateValidationError, as follows: Traceback (most recent call last): File /net/datasrv3.sanbi.ac.za/datastore/cip0/software/galaxy-dev/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py, line 144, in queue_job job_wrapper.prepare() File /net/datasrv3.sanbi.ac.za/datastore/cip0/software/galaxy-dev/galaxy-dist/lib/galaxy/jobs/__init__.py, line 130, in prepare self.tool.handle_unvalidated_param_values( incoming, self.app ) File /net/datasrv3.sanbi.ac.za/datastore/cip0/software/galaxy-dev/galaxy-dist/lib/galaxy/tools/__init__.py, line 1872, in handle_unvalidated_param_values self.handle_unvalidated_param_values_helper( self.inputs, input_values, app ) File /net/datasrv3.sanbi.ac.za/datastore/cip0/software/galaxy-dev/galaxy-dist/lib/galaxy/tools/__init__.py, line 1890, in handle_unvalidated_param_values_helper self.handle_unvalidated_param_values_helper( input.cases[current].inputs, values, app, context, prefix ) File /net/datasrv3.sanbi.ac.za/datastore/cip0/software/galaxy-dev/galaxy-dist/lib/galaxy/tools/__init__.py, line 1912, in handle_unvalidated_param_values_helper raise LateValidationError( message ) LateValidationError This Galaxy is Postgres and SGE, with galaxy-dist as of last night. Another strange thing is that while I select that the output of step 4 should be visible, it is, in fact, hidden in the new history. How to debug this? How can I see which input is not validating? Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Visualise Galaxy workflows using graphviz
Hi there Because my Galaxy workflows tend to sprawl outside the viewable space in my browser, I've written a script that, when given a workflow JSON file as input, writes out a graphviz dot format graph of the workflow. By default the graph treats datasets as nodes and analyses as edges (unlike the Galaxy workflow editor, but quite like the way the history is presented), but you can choose to have analyses as nodes (and thus datasets as edges) too. In case anyone else finds this useful, the code can be downloaded from my bitbucket: https://bitbucket.org/pvanheus/galaxy/src/f29453f3d9d8/contrib/workflow_to_dot.py Usage is something like: ./workflow_to_dot.py Galaxy-Workflow.ga |dot -Tsvg workflow.svg and then you can view your workflow.svg with eog or another viewer. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Using workflows as components in workflows?
Hi Jeremy I'm need this for something I'm implementing at the moment, and how I'm thinking about it is making a tool that uses the API to call a workflow. There are a few problems though, correct me if I'm wrong: 1) In order to make an input history item available to the called workflow, the tool needs to somehow know about history items, but the tool xml passes in parameters as data files. This could probably be remedied by providing a type=history_item parameter to param that would provide the id associated with the history item. In the interem, just to test things, I'm passing in parameters as a history:history_item string (yeah I know, ugly!). 2) My particular tool needs to take a history item, splits it into partitions, and call a workflow with each of those partitions. For this to work, the partition needs to be uploaded as a new history item, but that is currently not possible. The other possibility is to create a tool that does the split, have it in a single-tool workflow (because workflows can be called from the API in such a way that their output goes to a new history, whereas I don't see that in the tool interface) and then iterate through the history that contains the split data, calling the analysis workflow on each item. Peter P.S. for my particular problem - call a bunch of tools, once for each row in a file of tabular data - it would be WAY easier to just write everything in a Python script, but I'm trying to see what is do-able within Galaxy. On 20/05/2012 16:55, Jeremy Goecks wrote: Is there any way we can speed up the implementation of this issue? Community contributions and always encouraged and welcomed. Partial solutions are fine, and self-contained contributions are likely to be included more quickly because they are easier to review. Thanks, J. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Pushing changes upstream to Galaxy
Hi there As I adapt Galaxy to the needs of the particular workflows I'm implementing, I invariably end up tweaking the existing code, especially the tool definitions and the datatypes config file. These tweaks are often minor - e.g. adding an extra parameter to Ross Lazarus' ClustalW's code, changing the way the binary is called (e.g. clustalw2 on ubuntu is installed as clustalw, BLAST+ is installed single threaded so the -num_threads option in the ncbi_blastn_wrapper.xml doesn't work), but a) Sometimes they make sense to share with the rest of the Galaxy community b) local changes make it more tricky to keep in sync with upstream changes to Galaxy What's the best procedure for proposing changes up upstream galaxy? Posting patches to galaxy-dev? Is there some mercurial magic that can help? Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Using a variable in data from_work_dir=
Hi there
I'm trying to wrap Bioperl's bp_genbank2gff script. This script, by
default, produces an output file named the same as the basename of the
input file, with .gff added. So e.g.
bp_genbank2gff data.gbk
produces a file named data.gbk.gff.
I'm trying to use the from_work_dir attribute of the data tag to pick
up this file. However, if I create a tool XML like:
tool id=Genbank2GFF3 name=Genbank to GFF3 version=1.0
descriptionConvert Genbank format to GFF3/description
command interpreter=python
bp_genbank2gff $genbank_filename
/command
inputs
param name=genbank_filename type=data format=txt
labelGenbank sequence record(s)/label
/param
/inputs
outputs
data format=gff3 name=gff3_filename
from_work_dir=${genbank_filename}.gff
/data
/outputs
help
/help
/tool
I get an error from Galaxy indicating that it is trying to find a file
name $genbank_filename.gff, I.e. the ${genbank_filename} is not expanded.
Is there any way to deal with this situation? Of course the other option
is to create a tiny wrapper around bp_genbank2gff that takes an output
filename argument and does the renaming...
Peter
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
[galaxy-dev] Using workflows as components in workflows?
Hi there Is there a way in Galaxy to use workflows as components in workflows? I've got two datasets that start as Genbank files and are processed (features extracted, translated to AA) by a small workflow, and then BLASTed against one another. Ideally I'd like to modularise the initial steps as a workflow so that I can say input data goes through this workflow and then the result is fed into either the query or the subject slot of blastp. So is this possible? Thanks! Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Interested in speaking with other, institutions deploying Galaxy locally?
We have a small collaboration between institutions in the greater Cape Town region (UCT, UWC, Stellenbosch) on this topic (the so-called Pipelines group). If anyone in South Africa is interested in talking about these topics, please get in touch because we could share expertise. Peter SANBI - UWC On 02/05/2012 09:49, Sarah Maman wrote: Hello, This is a great idea. I'm interested. Thanks ! Sarah Maman -- Message: 1 Date: Fri, 27 Apr 2012 13:45:16 -0500 From: Ann Black-Ziegelbein annbl...@eng.uiowa.edu To: galaxy-dev@lists.bx.psu.edu Subject: [galaxy-dev] Interested in speaking with other institutions deploying Galaxy locally? Message-ID: 4f9ae93c.20...@eng.uiowa.edu Content-Type: text/plain; charset=ISO-8859-1; format=flowed Hi everyone - Here at the University of Iowa we are working on deploying Galaxy locally for campus wide access. I am interested in forming a community of other institutions trying to deploy Galaxy locally and mange/operate it on a broad level. Is anyone else? If there is enough interest, possibly we could have a community conference call every other month to have an open discussion on how we are all deploying galaxy, customizations we are making, problems we are encountering, bugs, and any add-on operations management for galaxy being developed, etc. Would love to hear from others operating Galaxy or in process of standing up a local deployment. Thanks! Ann Black-Ziegelbein ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Debugging upload via nginx
Hi there I've got nginx 1.0.5 in front of my Galaxy instance, and I'm trying to get uploads to work as per 'Receiving files using nginx' instructions on the Galaxy wiki. I've added the relevant config to my nginx setup (pastebin http://paste.ubuntu.com/961983/) and universe_wsgi.ini (pastebin http://paste.ubuntu.com/961985/) but it isn't working. How do I debug this? What should I be looking for in the log files? What seems to be happening is that the files are being uploaded - entries are created in database/tmp/upload_store with the contents of the relevant files - but Galaxy is not being informed that the upload is complete. Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Using REMOTE_USER with nginx
Hey there
The instructions on using REMOTE_USER with nginx are still a bit vague
in the wiki, so let me share how I got this working with nginx's
http_auth_pam module and our local Kerberos setup. Really simple actually:
First, I created a pam.d entry for nginx, as follows:
auth[success=1 default=ignore]pam_krb5.so minimum_uid=1000
ignore_k5login
authrequisitepam_deny.so
authrequiredpam_permit.so
That can of course be adapted for your authentication scheme of choice.
The, after recompiling nginx to add the module (I actually used the
source from the Ubuntu .deb and installed from this customised .deb), I
added:
auth_pam SANBI Galaxy (dev);
auth_pam_service_name nginx;
proxy_set_header REMOTE_USER $remote_user;
That auth_pam_service_name must be the name of the file you add in
/etc/pam.d. So the complete location clause is now:
location / {
auth_pam SANBI Galaxy (dev);
auth_pam_service_name nginx;
proxy_set_header REMOTE_USER $remote_user;
proxy_pass http://galaxy_app;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-URL-SCHEME https;
}
Finally, set:
use_remote_user = True
remote_user_maildomain = YOUR DOMAIN NAME
And restart nginx and galaxy, and you're done. Of course, since you're
using Basic authentication, you should make sure that you are using ssl too.
If this all looks ok, maybe someone can update the wiki?
Peter
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
[galaxy-dev] Run as user code
Hi there I recently wrote some code to run Galaxy jobs as a different user, and was just merging my codebase (https://bitbucket.org/pvanheus/galaxy) with galaxy-central when I noticed that code doing the same thing is now part of galaxy-central. The code in galaxy-central is more straightforward than mine, in that it uses external scripts and sudo, whereas mine spawns daemons (each on owned by the relevant system user) and provides a proxy object (implementing the bits of the Python DRMAA API that Galaxy uses) to direct work to the correct daemon. In practice the differences are, however, minor - the effect is the same: processes running as the system user of choice control jobs using DRMAA calls. One issue though is the question of Galaxy user to system user mapping. The current approach (in galaxy-central) is to split the Galaxy username at the @ sign and use everything before the @ as the system user (so p...@sanbi.ac.za becomes pvh). If the system user does not in fact exist, an exception (in the DRMAA job runner) will result. I don't know if anyone will find it useful to have a different approach to Galaxy to system user mapping? For now the current approach is fine for the setup we have at SANBI. Is anyone using this run as user code in production? Thanks! Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Proposed patch to blastxml_to_tabular.py
On 30/11/2011 11:58, Peter Cock wrote: On Wed, Nov 30, 2011 at 9:49 AM, Peter van Heusden wrote: Yes, this went away with newer versions of BLAST (and BLAST+). The 2.2.15 version is from 2006. OK, that would explain why I didn't find the issue myself during the development. I've added test data and tests to the tool's XML file. Is this on a bitbucket fork? Yes, its now at https://bitbucket.org/pvanheus/galaxy I'm still trying to get my head around how to use mercurial, so things are no doubt messy. Look for changes from p...@sanbi.ac.za Do you know how to run a single tool's test in Galaxy? Peter Yes, run_functional_tests.sh supports this. You'll need the test ID, which you can work out from the wrapper XML, or $ ./run_functional_tests.sh -list ... Then, in this case, $ ./run_functional_tests.sh -id blastxml_to_tabular ... Ok doing that generates all sorts of stuff but ends with: ok -- Ran 9 tests in 100.146s OK which sounds encouraging. Peter Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Patch to add twobit type
Hi everyone
I've been using the twobit datatype as generated by faToTwoBit (part of
Jim Kent's BLAT package) and read by bx-python (bx.seq.twobit). So
here's a patch to add the twobit datatype to Galaxy.
Peter
# HG changeset patch
# User Peter van Heusden p...@sanbi.ac.za
# Date 1307966741 -7200
# Node ID 3b68bc0d67b43af2ce69fb1eeb9160ca053c4c72
# Parent 8bcc0877b39bf10c2330f0651d2409a2b2e9c469
Added TwoBit datatype for twobit binary nucleotide datatype. Sniffer code
based on bx-python's bx.seq.twobit.
diff -r 8bcc0877b39b -r 3b68bc0d67b4 datatypes_conf.xml.sample
--- a/datatypes_conf.xml.sample Fri Jun 10 20:10:09 2011 -0400
+++ b/datatypes_conf.xml.sample Mon Jun 13 14:05:41 2011 +0200
@@ -116,6 +116,7 @@
datatype extension=svg type=galaxy.datatypes.images:Image mimetype=image/svg+xml/
datatype extension=taxonomy type=galaxy.datatypes.tabular:Taxonomy display_in_upload=true/
datatype extension=tabular type=galaxy.datatypes.tabular:Tabular display_in_upload=true/
+ datatype extension=twobit type=galaxy.datatypes.binary:TwoBit mimetype=application/octet-stream display_in_upload=true/
datatype extension=txt type=galaxy.datatypes.data:Text display_in_upload=true/
datatype extension=memexml type=galaxy.datatypes.xml:MEMEXml mimetype=application/xml display_in_upload=true/
datatype extension=blastxml type=galaxy.datatypes.xml:BlastXml mimetype=application/xml display_in_upload=true/
@@ -279,6 +280,7 @@
defined format first, followed by next-most rigidly defined,
and so on.
--
+sniffer type=galaxy.datatypes.binary:TwoBit/
sniffer type=galaxy.datatypes.binary:Bam/
sniffer type=galaxy.datatypes.binary:Sff/
sniffer type=galaxy.datatypes.xml:BlastXml/
diff -r 8bcc0877b39b -r 3b68bc0d67b4 lib/galaxy/datatypes/binary.py
--- a/lib/galaxy/datatypes/binary.py Fri Jun 10 20:10:09 2011 -0400
+++ b/lib/galaxy/datatypes/binary.py Mon Jun 13 14:05:41 2011 +0200
@@ -6,6 +6,10 @@
from galaxy.datatypes.metadata import MetadataElement
from galaxy.datatypes import metadata
from galaxy.datatypes.sniff import *
+from galaxy import eggs
+import pkg_resources
+pkg_resources.require( bx-python )
+from bx.seq.twobit import TWOBIT_MAGIC_NUMBER, TWOBIT_MAGIC_NUMBER_SWAP, TWOBIT_MAGIC_SIZE
from urllib import urlencode, quote_plus
import zipfile, gzip
import os, subprocess, tempfile
@@ -292,3 +296,29 @@
def get_track_type( self ):
return LineTrack, {data_standalone: bigbed}
+class TwoBit (Binary):
+Class describing a TwoBit format nucleotide file
+
+file_ext = twobit
+
+def sniff(self, filename):
+try:
+input = file(filename)
+magic = struct.unpack(L, input.read(TWOBIT_MAGIC_SIZE))[0]
+if magic == TWOBIT_MAGIC_NUMBER or magic == TWOBIT_MAGIC_NUMBER_SWAP:
+return True
+except IOError:
+return False
+
+def set_peek(self, dataset, is_multi_byte=False):
+if not dataset.dataset.purged:
+dataset.peek = Binary TwoBit format nucleotide file
+dataset.blurb = data.nice_size(dataset.get_size())
+else:
+return super(TwoBit, self).set_peek(dataset, is_multi_byte)
+
+def display_peek(self, dataset):
+try:
+return dataset.peek
+except:
+return Binary TwoBit format nucleotide file (%s) % (data.nice_size(dataset.get_size()))
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
[galaxy-dev] Paper on workflow patterns
As discussed in the Workflows and APIs breakaway session, here is the paper on Workflow patterns: van der Aalst, ter Hofsted, Kiepuszewski and Barros, Distributed and Parallel Databases, Volume 14, Number 1, 5-51: Workflow Patterns PDF available at: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.90.1538rep=rep1type=pdf ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
