Re: [galaxy-dev] Help writing a tool
On 03/19/2012 10:19 PM, Peter Cock wrote: On Mon, Mar 19, 2012 at 8:41 PM, Mark Johnsonmjohn...@ncbi.nlm.nih.gov wrote: I'm writing some tools to integrate NCBI data resources with Galaxy. I have two questions. The first is simple. I want to write a tool for a long-running process that is handled by some other scheduler, and that produces its own job ids. Some web services, like BLAST, for example, receive a request, and take a while to complete processing. The job id can be used to fetch either job status or results from the server, depending on whether it has completed. How do you make a Galaxy tool that polls the server, and produces an output set only when the process is complete? Why do you need to do anything special at all for Galaxy here? I'd just write it as a single command line call which blocks. As far as Galaxy will know it is just a slow tool. Yes, Galaxy is pretty good with handling 'slow' tools (ie: you can close the browser, and come back next morning) However, we have one tool where we just use Galaxy to execute a job which manipulates data outside of the Galaxy data directory. We do something similar to what you have asked for initially: We have a perl wrapper, which first submits the job using IPC::Open3, followed by a 'while' loop checking the status of the job and finally produces a 'log' file which ends up as the new galaxy history item. Regards, Hans The second question is, besides this mailing list, and the Galaxy wiki, is there are good online video or text resource that explains the Galaxy architecture and how to use it? The docs are good as far as they go, but most of what's in thecommand scripts in the tool files isn't documented. There are quite a few Galaxy videos... not sure if there are any aimed at potential developers. Are you asking about the Cheetah template language used inside the XML for thecommand which is almost a scripting language in itself, or the actual wrapper scripts used in some tools (which can be written in Python, Perl, etc)? Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] refresh_on_change is broken?
Hi everyone, Sorry to ping again, having the refresh_on_change functionality not working in Galaxy has us making tool forms in ways we really don't want to, no one needs to use refresh_on_change or am I the only one that finds it's broken? We've started to look at what Galaxy is trying to do, seems like you are using jQuery to bind a custom Javascript refresh_on_change function but when you look at that function it doesn't do any form reloading, maybe just the feature is unfinished? sincerely, Leandro On Wed, Feb 29, 2012 at 10:34 AM, Leandro Hermida soft...@leandrohermida.com wrote: Hello, Seems like the refresh_on_change functionality is broken in the latest galaxy-dist. If you apply it to an input parameter it doesn't seem to refresh the page when you change the selection of that parameter. Does anyone else also have the problem? regards, Leandro ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Help writing a tool
On 3/19/12 5:19 PM, Peter Cock wrote: Why do you need to do anything special at all for Galaxy here? I'd just write it as a single command line call which blocks. As far as Galaxy will know it is just a slow tool. I suppose the tool could just poll the server, and only produce results when the process is complete. I could swear I read something somewhere in the Galaxy documentation that described two kinds of tools: one kind that finish quickly, and another that run for awhile (or a long while), and produce results when they're done. Maybe it was among the cloud-related documentation, about using the scheduler. Thanks for the suggestion. I'll try polling. The second question is, besides this mailing list, and the Galaxy wiki, is there are good online video or text resource that explains the Galaxy architecture and how to use it? The docs are good as far as they go, but most of what's in thecommand scripts in the tool files isn't documented. There are quite a few Galaxy videos... not sure if there are any aimed at potential developers. Are you asking about the Cheetah template language used inside the XML for thecommand which is almost a scripting language in itself, or the actual wrapper scripts used in some tools (which can be written in Python, Perl, etc)? Cheetah documentation is findable. I'm asking more about understanding how the inputs relate to what's available in Python in the Cheetah template in the command section. I guess there's not much for developers to get an overview: how inputs, parameters, outputs, the command, the template, and Python all work together. Seems the only way is trial, error, experiment, and trying to understand the existing tools. Thanks --Mark ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Possible bug: Tags in a workflow gets duplicated hundreds of times
Hi Thon, Thanks for reporting this. I see what the problem is here at least for the clone duplication, and I've committed a fix in 6833:e8e361707865 that will affect all workflows going forward. Unfortunately, there isn't a complete solution for fixing the extra tags. The problem was that any time you cloned a workflow, the mapping that retrieved the set of tags that should have been copied was actually returning *all* of the user's tags. That means that even if the source workflow had no tags, all workflow tags owned by the user would get copied to the newly created workflow. Because of this, just removing duplicate tags (while an obvious step in the right direction) won't be enough and you'll still have tags where they shouldn't be. I'll add the duplicate removal code as a migration, but in the short term if there are a few individual workflows that you need to fix (by simply removing all tags) the easiest thing to do on your end would be to manually remove the stored_workflow_tag_associations for those particular workflows, using something like this sql statement: delete from stored_workflow_tag_associations where stored_workflow_id = 'enter stored_workflow id to fix' -Dannon On Mar 19, 2012, at 1:47 PM, Anthonius deBoer wrote: Hi, I have run into an issue with a tag being duplicated hundreds of times each time I open, edit or clone a workflow. I have tried to manually remove them, but each time the tag gets inserted somehow and duplicated each time I edit the workflow. By now there are hundreds of copies of the tags at it makes loading and running the workflow very slow. Is there a way to remove a tag from the system somehow? I am willing to dig into the database with some SQL statement but not sure where to start... Thanks Thon ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] run cluster jobs as real user
Hi, I (re-)installed galaxy on our server and managed to set it up to run jobs through our cluster (GridEngine). It's also running behind an Apache proxy now. But now I'm wondering how to setup to have the cluster jobs run as the real cluster users. I found the documentation on this topic a little thin. We use NIS for managing users and passwords across the cluster. How do I connect that to Galaxy? regards, Andreas -- Andreas Kuntzagk SystemAdministrator Berlin Institute for Medical Systems Biology at the Max-Delbrueck-Center for Molecular Medicine Robert-Roessle-Str. 10, 13125 Berlin, Germany http://www.mdc-berlin.de/en/bimsb/BIMSB_groups/Dieterich ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] refresh_on_change is broken?
Leandro, refresh_on_change is used quite a bit throughout the tool UI and appears to work fine both on a local instance and on the public instance. E.g. look at Tophat and change settings from 'Use Defaults' to 'Full Parameter List' -- the refresh_on_change works on reload the tool with more parameters. Some questions that should shed light on your issues: *Does the Tophat tool refresh on the main server? If not, this is probably a browser issue. *Are you seeing any errors in the Javascript console? *Can you be more specific about what problems you're seeing and whether you can reproduce with known tools? Thanks, J. On Mar 20, 2012, at 9:28 AM, Leandro Hermida wrote: Hi everyone, Sorry to ping again, having the refresh_on_change functionality not working in Galaxy has us making tool forms in ways we really don't want to, no one needs to use refresh_on_change or am I the only one that finds it's broken? We've started to look at what Galaxy is trying to do, seems like you are using jQuery to bind a custom Javascript refresh_on_change function but when you look at that function it doesn't do any form reloading, maybe just the feature is unfinished? sincerely, Leandro On Wed, Feb 29, 2012 at 10:34 AM, Leandro Hermida soft...@leandrohermida.com wrote: Hello, Seems like the refresh_on_change functionality is broken in the latest galaxy-dist. If you apply it to an input parameter it doesn't seem to refresh the page when you change the selection of that parameter. Does anyone else also have the problem? regards, Leandro ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] [galaxy-user] Tophat error
Hi David, I don't know if you are still having this problem or not, but I did a web search and found this thread on seqanswers from 2/16 that seems like a good match to the problem you were having: http://seqanswers.com/forums/showthread.php?p=65085 These scientists resolved the problem by removing the --closure-search option from the command string. On the Galaxy tool form, this is the option Use Closure Search:. which is No by default. Perhaps you set this to be Yes? I would try switching it to No to see if that solves the problem. If not, then contacting the tool authors would probably be the best next step, either at seqanswers or directly at tophat.cuffli...@gmail.com. The original guess about genome indexes was way off base, this is a python error statement. I don't believe this to be related to the Galaxy wrapper but will cc Jeremy for a second opinion. Hopefully the first option will resolve the issue! Thanks, Jen Galaxy team On 3/14/12 11:10 AM, Jennifer Jackson wrote: Hi David, You question has posted to the list now and we will be getting back to you. It didn't post immediately due to some mail mailman server issues here. This looks like a problem that came up on a local instance. Because of that, I am going to send this over to the galaxy-...@bx.psu.edu mailing list. At first glance, this appears to be a problem with the NGS genome indexes used for the target genome. These are the instructions you followed? http://wiki.g2.bx.psu.edu/Admin/NGS%20Local%20Setup (Bowtie indexes are used for TopHat) We will be looking at this more later today, but I wanted to get back to you, so you that you know that this doesn't need to be posted again. Thanks! Jen Galaxy team On 3/14/12 6:48 AM, David Matthews wrote: Hi, JUst running a TopHat job which returned the following error: Executing: /gpfs/cluster/isys/galaxy/Software/bin/bowtie-inspect /local/tmp5Ywx45/dataset_942 ./tophat_out/tmp/dataset_942.fa [Tue Mar 13 12:45:08 2012] Checking for Bowtie Bowtie version: 0.12.7.0 [Tue Mar 13 12:45:08 2012] Checking for Samtools Samtools Version: 0.1.18 [Tue Mar 13 12:45:08 2012] Generating SAM header for /local/tmp5Ywx45/dataset_942 format: fastq quality scale: phred33 (default) [Tue Mar 13 12:45:21 2012] Preparing reads left reads: min. length=56, count=29523921 right reads: min. length=56, count=29543412 [Tue Mar 13 13:07:54 2012] Mapping left_kept_reads against dataset_942 with Bowtie [Tue Mar 13 13:45:26 2012] Processing bowtie hits [Tue Mar 13 14:11:28 2012] Mapping left_kept_reads_seg1 against dataset_942 with Bowtie (1/2) [Tue Mar 13 14:43:27 2012] Mapping left_kept_reads_seg2 against dataset_942 with Bowtie (2/2) [Tue Mar 13 14:57:50 2012] Mapping right_kept_reads against dataset_942 with Bowtie [Tue Mar 13 15:37:46 2012] Processing bowtie hits [Tue Mar 13 16:04:28 2012] Mapping right_kept_reads_seg1 against dataset_942 with Bowtie (1/2) [Tue Mar 13 16:37:18 2012] Mapping right_kept_reads_seg2 against dataset_942 with Bowtie (2/2) [Tue Mar 13 16:50:40 2012] Searching for junctions via segment mapping Traceback (most recent call last): File /gpfs/cluster/isys/galaxy/Software/bin/tophat, line 3063, inmodule sys.exit(main()) File /gpfs/cluster/isys/galaxy/Software/bin/tophat, line 3029, in main user_supplied_deletions) File /gpfs/cluster/isys/galaxy/Software/bin/tophat, line 2681, in spliced_alignment [maps[initial_reads[left_reads]].unspliced_bwt, maps[initial_reads[left_reads]].seg_maps[-1]], TypeError: list indices must be integers, not str Does anyone know what this kind of error is? Best Wishes, David. __ Dr David A. Matthews Senior Lecturer in Virology Room E49 Department of Cellular and Molecular Medicine, School of Medical Sciences University Walk, University of Bristol Bristol. BS8 1TD U.K. Tel. +44 117 3312058 Fax. +44 117 3312091 d.a.matth...@bristol.ac.uk mailto:d.a.matth...@bristol.ac.uk ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] GCC2012 Early Registration is Now Open
Hello all, We are pleased to announce that early registrationhttp://wiki.g2.bx.psu.edu/Events/GCC2012/Registerfor the 2012 Galaxy Community Conference (GCC2012)http://wiki.g2.bx.psu.edu/Events/GCC2012is now open. GCC2012 will be held July 25-27, at the UIC Forum http://www.uic.edu/depts/uicforum/, in Chicago, Illinois. The conference will feature two full dayshttp://wiki.g2.bx.psu.edu/Events/GCC2012/Programof presentations, discussions, lightning talks, and breakouts. We have also added a new full day of traininghttp://wiki.g2.bx.psu.edu/Events/GCC2012/TrainingDaythis year, featuring 3 parallel tracks with four workshops each, covering seven different topics. The Galaxy Community Conference is for: - Sequencing core facility staff - Bioinformatics core staff - Bioinformatics tool and workflow developers - Bioinformatics focused principal investigators and researchers - Data producers - Power bioinformatics users This event will help you with integrating, analyzing, and sharing the diverse and very large datasets that are now typical in biomedical research. GCC2012 is an opportunity to share best practices with, and learn from, a large community of researchers and support staff who are facing the challenges of data-intensive biology. Registration http://wiki.g2.bx.psu.edu/Events/GCC2012/Register is very affordable this year, especially for post-docs and students. You can can save an additional 40% by registering on or before June 11. Conference lodginghttp://wiki.g2.bx.psu.edu/Events/GCC2012/Logistics#Lodgingcan also be booked. Low-cost rooms have been reserved on the UIC campushttp://wiki.g2.bx.psu.edu/Events/GCC2012/Logistics#James_Stuckel_Towers. You can also stay at the official conference hotelhttp://wiki.g2.bx.psu.edu/Events/GCC2012/Logistics#Crowne_Plaza_Chicago_Metro_Downtown, at a substantial discount arranged for conference attendees. There are a limited rooms available in both, and you are encouraged to register earlyhttp://wiki.g2.bx.psu.edu/Events/GCC2012/Register, if you would like to stay in either. Finally, abstract submissionhttp://wiki.g2.bx.psu.edu/Events/GCC2012/Abstractsis also open, from now through April 16. Abstracts http://wiki.g2.bx.psu.edu/Events/GCC2012/Abstracts on any topic of interest to the Galaxy community are strongly encouraged. See the Abstracts http://wiki.g2.bx.psu.edu/Events/GCC2012/Abstracts page for details. Looking forward to seeing you in Chicago! Dave Clements http://wiki.g2.bx.psu.edu/DaveClements, on behalf of the GCC2012 Organizing Committeehttp://wiki.g2.bx.psu.edu/Events/GCC2012/Organizing%20Committee PS: Please help get the word out! A flyer and graphics are at http://wiki.g2.bx.psu.edu/Events/GCC2012/Promotion -- http://galaxyproject.org/GCC2012 http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://galaxyproject.org/wiki/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Reading user id before the execution of the tool
Dear Galaxy Developers,
This post went unanswered could I re-submit in a hope if there is any guy out
there who could stop me pulling my few left hairs.
My tool meant to load up (in a dropdown) some custom data from a non-galaxy
database for the user to select and submit a job. I am using dynamic_options
for loading such dynamic values, however to restrict data for a logged-in user
I need to know the user id.
So far I have tried many things including setting
dynamic_options=load_dynamic_values($__user_id__) in the xml wrapper but it
did not work. I know $__user_id__ can be passed to the tool as a command line
arguments, however, my issue is to read user id even before the execution of
main tool.
Thanks for your help guys.
I have also tried following code in a hope to grab the webapp session.
ini_file = '/galaxy-dist/universe_wsgi.ini'
conf_parser = ConfigParser.ConfigParser( {'here':os.getcwd()} )
conf_parser.read( ini_file )
configuration = {}
for key, value in conf_parser.items( app:main ):
configuration[key] = value
global_conf = { '__file__' : ini_file }
app = galaxy.app.UniverseApplication(**configuration )
But this generates following error.
UnboundExecutionError: Parent instance History at 0x581cdd0 is not bound to a
Session; lazy load operation of attribute 'active_datasets' cannot proceed
Regards
Matloob Khushi
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] Looking where to edit web view
Hello Galaxy team, I've recently begun building a resource usage monitor / predictor for Galaxy as part of my Computer Science Honours degree and I was wanting to at some later stage in my project, once the actual resource monitor is built, to incorporate it to the actual Galaxy web interface. have been looking through your code and have found it somewhat difficult to track down where exactly i would go about doing this. I would like to ahve a separate tab up in the top nav bar and then be able to display resource information about Galaxy when its running (processes, I/O, RAM, CPU etc.) Are you able to point me in the right direction for this? Thankyou, Ed ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Looking where to edit web view
Ed, The top bar is defined in templates/webapps/galaxy/base_panels.mako. To create a new masthead tab, you would have to edit that file in addition to creating the relevant controller methods for the functionality you wish to add. -Dannon On Mar 20, 2012, at 8:32 PM, Edward Hills wrote: Hello Galaxy team, I've recently begun building a resource usage monitor / predictor for Galaxy as part of my Computer Science Honours degree and I was wanting to at some later stage in my project, once the actual resource monitor is built, to incorporate it to the actual Galaxy web interface. have been looking through your code and have found it somewhat difficult to track down where exactly i would go about doing this. I would like to ahve a separate tab up in the top nav bar and then be able to display resource information about Galaxy when its running (processes, I/O, RAM, CPU etc.) Are you able to point me in the right direction for this? Thankyou, Ed ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Looking where to edit web view
I like this idea very much but i think it might be better placed as a link in http://galaxy.neb.com/admin (maybe as a replacement for profile memory usage?) brad On Mar 20, 2012, at 8:53 PM, Dannon Baker wrote: Ed, The top bar is defined in templates/webapps/galaxy/base_panels.mako. To create a new masthead tab, you would have to edit that file in addition to creating the relevant controller methods for the functionality you wish to add. -Dannon On Mar 20, 2012, at 8:32 PM, Edward Hills wrote: Hello Galaxy team, I've recently begun building a resource usage monitor / predictor for Galaxy as part of my Computer Science Honours degree and I was wanting to at some later stage in my project, once the actual resource monitor is built, to incorporate it to the actual Galaxy web interface. have been looking through your code and have found it somewhat difficult to track down where exactly i would go about doing this. I would like to ahve a separate tab up in the top nav bar and then be able to display resource information about Galaxy when its running (processes, I/O, RAM, CPU etc.) Are you able to point me in the right direction for this? Thankyou, Ed ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Brad Langhorst langho...@neb.com 978-380-7564 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] potential bug in DefaultToolAction collecting chrom info
Hello, In the file lib/galaxy/tools/actions/__init__.py for the execute function of the DefaultToolAction class line 198 has the following code: if trans.user and ( 'dbkeys' in trans.user.preferences ) and ( input_dbkey in trans.user.preferences[ 'dbkeys' ] ): In this case trans.user.preferences[ 'dbkeys' ] is just returning a json string version of the dictionary. The result is that if input_dbkey is a substring of any part of this then it will be returned as True. Something like this would fix (though with the next lines of code it would be converting from json twice): if trans.user and ( 'dbkeys' in trans.user.preferences ) and ( input_dbkey in from_json_string(trans.user.preferences[ 'dbkeys' ] )): Cheers, Andrew ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
