[galaxy-dev] Best practice: set tool version on wrapper or on tool level?
Hi all, I am writing a wrapper around Razers mapper. Currentty, Razers has three versions, the latest being razers3. Since it's under development, what is the best way to deal with new versions? - write a different tool config for every version, giving them different ids like 'razers3', 'razers4'. These tools will appear all simultaneously in the tool box. Every version needs to be installed independently from the toolshed. - or write a tool config with id 'razers' and set a version in the xml version_command. Then when a new version comes out, I update the wrapper completely to match the new version. In this way a user can select the different versions can be selected at the top of the tool parameters page in Galaxy, right? How should I deal with naming the version: follow the config file I wrote (version 1.0.0 for razer3, 2.0.0 for razers4 - can cause confusion though.) - or write a wrapper for the tool 'razers' in general, parsing the selection of which version to use to the wrapper, and deal with it over there? I hope I explained my issue clearly. Thanks, Joachim -- Joachim Jacob, PhD Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Best practice: set tool version on wrapper or on tool level?
On Thu, Sep 27, 2012 at 11:32 AM, Joachim Jacob joachim.ja...@vib.be wrote: Hi all, I am writing a wrapper around Razers mapper. Currentty, Razers has three versions, the latest being razers3. Since it's under development, what is the best way to deal with new versions? - write a different tool config for every version, giving them different ids like 'razers3', 'razers4'. These tools will appear all simultaneously in the tool box. Every version needs to be installed independently from the toolshed. - or write a tool config with id 'razers' and set a version in the xml version_command. Then when a new version comes out, I update the wrapper completely to match the new version. In this way a user can select the different versions can be selected at the top of the tool parameters page in Galaxy, right? How should I deal with naming the version: follow the config file I wrote (version 1.0.0 for razer3, 2.0.0 for razers4 - can cause confusion though.) - or write a wrapper for the tool 'razers' in general, parsing the selection of which version to use to the wrapper, and deal with it over there? I hope I explained my issue clearly. Thanks, Joachim I think that using a single wrapper only really makes sense when the underlying tool's command line API doesn't change. Even then, for major version jumps I personally would think a separate tool in Galaxy makes more sense. After all, you would expect potentially different output between versions. i.e. I would lean to separate wrappers for razers3, razers4 etc. I'm looking at a similar issue with MIRA, but there is an additional complication here that the newest version also changes the MIRA command line interface extensively. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Mercurial to push changes to toolshed repo
Hi all, As a test, I have uploaded some template files to a newly created repository on our test toolshed. I pulled that repository on my development machine using mercurial: from this folder I start now developing the tool wrapper. I have noticed that once the tool name has been set in the tool config (tool name=x ) during the initial upload, pushing a subsequent change to this name, this does not change the name on the repository anymore (and the other settings, for this matter). Is this correct? Thanks, Joachim -- Joachim Jacob, PhD Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Mercurial to push changes to toolshed repo
Hello Joachim, On Sep 27, 2012, at 8:05 AM, Joachim Jacob wrote: Hi all, As a test, I have uploaded some template files to a newly created repository on our test toolshed. This is great! It's good to hear when the Galaxy community is using local tool sheds within their environments. I pulled that repository on my development machine using mercurial: from this folder I start now developing the tool wrapper. This is a great approach because the tool shed is not a source code repository for developing tools, so development generally takes place within an environment running a development Galaxy instance. When the tool development is stabilized and the tool's functionality is proven to be correct within the Galaxy development instance, the tool and all of it's associated files can be uploaded to the tool shed for sharing with others. Treating the tool shed as a source code repository for tools while they are in development will undoubtedly result in undesired behavior by the tool shed. I have noticed that once the tool name has been set in the tool config (tool name=x ) during the initial upload, pushing a subsequent change to this name, this does not change the name on the repository anymore (and the other settings, for this matter). Is this correct? I'm not quite clear on your question here. If by tool config you mean the Galaxy tool config wrapper (e.g., my_tool.xml), the tool shed never makes any changes to them when they are uploaded to a repository in the shed. The repository in the tool shed is just an enabler for sharing what is there. Changes to the content must be made in your development environment and pushed to the repository in the tool shed. The name of the tool shed repository itself can be changed only until the first clone occurs. Once a clone occurs, the cloned repository path includes the name of the repository, so if it was subsequently changed in the tool shed, those that cloned it previously could not get updates to their local clones. Greg Von Kuster Thanks, Joachim -- Joachim Jacob, PhD Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Mercurial to push changes to toolshed repo
Hi Greg, Indeed, I mean the tool config wrapper, named razers3_wrapper.xml, when I mention 'tool config'. This xml is the alpha version, and is loaded correctly in my development Galaxy. So I pushed it to the repository = Changeset 997118f8f033 added: razers3_wrapper.xml razers3_wrapper.xml --- /dev/null Thu Jan 01 00:00:00 1970 + +++ b/razers3_wrapper.xml Thu Sep 27 14:39:54 2012 +0200 @@ -0,0 +1,372 @@ +tool id=rasers3 name=Map reads with rasers3 version=1.0.0 + !-- Additional info: wrapper compatible with versions . -- + description + a read mapping program for all kinds of reads with adjustable sensitivity. + /description + + version_command + razers3 --version + /version_command + (etc.) = But it still displayed the wrong name in the interface. I just found out the reason: the original file wrapper_template.xml was not removed from the repo (although not anymore present in my local repo). After removal, the tool is displayed correctly! One more question though about mercurial: how come that when I changed the name of the wrapper_template.xml to razers3_wrapper.xml, and that after pushing the commits wrapper_template.xml was still on the toolshed? Is this default behaviour of Mercurial? Many thanks for the assistance, Joachim Joachim Jacob, PhD Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On Thu 27 Sep 2012 03:08:27 PM CEST, Greg Von Kuster wrote: Hello Joachim, On Sep 27, 2012, at 8:05 AM, Joachim Jacob wrote: Hi all, As a test, I have uploaded some template files to a newly created repository on our test toolshed. This is great! It's good to hear when the Galaxy community is using local tool sheds within their environments. I pulled that repository on my development machine using mercurial: from this folder I start now developing the tool wrapper. This is a great approach because the tool shed is not a source code repository for developing tools, so development generally takes place within an environment running a development Galaxy instance. When the tool development is stabilized and the tool's functionality is proven to be correct within the Galaxy development instance, the tool and all of it's associated files can be uploaded to the tool shed for sharing with others. Treating the tool shed as a source code repository for tools while they are in development will undoubtedly result in undesired behavior by the tool shed. I have noticed that once the tool name has been set in the tool config (tool name=x ) during the initial upload, pushing a subsequent change to this name, this does not change the name on the repository anymore (and the other settings, for this matter). Is this correct? I'm not quite clear on your question here. If by tool config you mean the Galaxy tool config wrapper (e.g., my_tool.xml), the tool shed never makes any changes to them when they are uploaded to a repository in the shed. The repository in the tool shed is just an enabler for sharing what is there. Changes to the content must be made in your development environment and pushed to the repository in the tool shed. The name of the tool shed repository itself can be changed only until the first clone occurs. Once a clone occurs, the cloned repository path includes the name of the repository, so if it was subsequently changed in the tool shed, those that cloned it previously could not get updates to their local clones. Greg Von Kuster Thanks, Joachim -- Joachim Jacob, PhD Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Mercurial to push changes to toolshed repo
Hello Joachim, It sounds like you may be pushing changes from your development environment to your tool shed repository using hg from the command line - is this correct? if so, you must manually make all changes you want to your local repository before pushing the changes to the tool shed repository. So in this scenario, you have to 'hg remove' the original file name from your development repository as part of the change set you push to the tool shed repository. You should also be running at least hg version 2.2.3 in your local development environment so that when the push is received by the tool shed repository, repository metadata can be set on the new changeset. These sections of the tool shed wiki provide details: http://wiki.g2.bx.psu.edu/ToolShedRepositoryFeatures#Pushing_changes_to_a_repository_using_hg_from_the_command_line http://wiki.g2.bx.psu.edu/RepositoryRevisions Greg Von kuster On Sep 27, 2012, at 9:42 AM, Joachim Jacob wrote: Hi Greg, Indeed, I mean the tool config wrapper, named razers3_wrapper.xml, when I mention 'tool config'. This xml is the alpha version, and is loaded correctly in my development Galaxy. So I pushed it to the repository = Changeset 997118f8f033 added: razers3_wrapper.xml razers3_wrapper.xml --- /dev/null Thu Jan 01 00:00:00 1970 + +++ b/razers3_wrapper.xml Thu Sep 27 14:39:54 2012 +0200 @@ -0,0 +1,372 @@ +tool id=rasers3 name=Map reads with rasers3 version=1.0.0 + !-- Additional info: wrapper compatible with versions . -- + description + a read mapping program for all kinds of reads with adjustable sensitivity. + /description + + version_command + razers3 --version + /version_command + (etc.) = But it still displayed the wrong name in the interface. I just found out the reason: the original file wrapper_template.xml was not removed from the repo (although not anymore present in my local repo). After removal, the tool is displayed correctly! One more question though about mercurial: how come that when I changed the name of the wrapper_template.xml to razers3_wrapper.xml, and that after pushing the commits wrapper_template.xml was still on the toolshed? Is this default behaviour of Mercurial? Many thanks for the assistance, Joachim Joachim Jacob, PhD Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On Thu 27 Sep 2012 03:08:27 PM CEST, Greg Von Kuster wrote: Hello Joachim, On Sep 27, 2012, at 8:05 AM, Joachim Jacob wrote: Hi all, As a test, I have uploaded some template files to a newly created repository on our test toolshed. This is great! It's good to hear when the Galaxy community is using local tool sheds within their environments. I pulled that repository on my development machine using mercurial: from this folder I start now developing the tool wrapper. This is a great approach because the tool shed is not a source code repository for developing tools, so development generally takes place within an environment running a development Galaxy instance. When the tool development is stabilized and the tool's functionality is proven to be correct within the Galaxy development instance, the tool and all of it's associated files can be uploaded to the tool shed for sharing with others. Treating the tool shed as a source code repository for tools while they are in development will undoubtedly result in undesired behavior by the tool shed. I have noticed that once the tool name has been set in the tool config (tool name=x ) during the initial upload, pushing a subsequent change to this name, this does not change the name on the repository anymore (and the other settings, for this matter). Is this correct? I'm not quite clear on your question here. If by tool config you mean the Galaxy tool config wrapper (e.g., my_tool.xml), the tool shed never makes any changes to them when they are uploaded to a repository in the shed. The repository in the tool shed is just an enabler for sharing what is there. Changes to the content must be made in your development environment and pushed to the repository in the tool shed. The name of the tool shed repository itself can be changed only until the first clone occurs. Once a clone occurs, the cloned repository path includes the name of the repository, so if it was subsequently changed in the tool shed, those that cloned it previously could not get updates to their local clones. Greg Von Kuster Thanks, Joachim -- Joachim Jacob, PhD Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib ___ Please keep all replies on the list by
[galaxy-dev] Can't create custom visualization
Hello,
I'm trying to create a custom visualization in Trackster. I am doing the
following steps. Visualization - New Visualization - Add a Custom Build
I get directed to the url below:
http://galaxy.ca:8080/user/dbkeys?use_panels=True
But the web page says:
Server Error
An error occurred. See the error logs for more information. (Turn debug on
to display exception reports here)
My paster.log file shows the following:
WSGI Variables
--
application: paste.recursive.RecursiveMiddleware object at 0xf070d90
paste.cookies: (SimpleCookie:
__utma='99541067.1404064366.1320847925.1320847925.1320857512.2'
galaxycommunitysession='eb142648ac45b770e95464ae1d51cc6457899dd48ca7f3e61a826ece0a2b5d2a65f4ac4aabb6e5e9'
galaxyreportssession='c6ca0ddb55be603a922aa045b4afae662fca8487217f2ea25e5b488dcf5d52aefac5347681c8bf98'
galaxysession='c6ca0ddb55be603a67ec94afb2c9a07cafdf91af0226f4689d096411ea1bc0c1004977df81ecca90'
toolshedgalaxyurl='http://martin.dnalandmarks.ca:8080/',
'__utma=99541067.1404064366.1320847925.1320847925.1320857512.2;
galaxysession=c6ca0ddb55be603a67ec94afb2c9a07cafdf91af0226f4689d096411ea1bc0c1004977df81ecca90;
galaxycommunitysession=eb142648ac45b770e95464ae1d51cc6457899dd48ca7f3e61a826ece0a2b5d2a65f4ac4aabb6e5e9;
toolshedgalaxyurl=http://martin.dnalandmarks.ca:8080/;;
galaxyreportssession=c6ca0ddb55be603a922aa045b4afae662fca8487217f2ea25e5b488dcf5d52aefac5347681c8bf98')
paste.expected_exceptions: [class
'paste.httpexceptions.HTTPException']
paste.httpexceptions: paste.httpexceptions.HTTPExceptionHandler object
at 0xf0702d0
paste.httpserver.thread_pool: paste.httpserver.ThreadPool object at
0x2ae44714c0d0
paste.parsed_querystring: ([('use_panels', 'True')], 'use_panels=True')
paste.recursive.forward: paste.recursive.Forwarder from /
paste.recursive.include: paste.recursive.Includer from /
paste.recursive.include_app_iter: paste.recursive.IncluderAppIter from
/
paste.recursive.script_name: ''
paste.throw_errors: True
webob._parsed_query_vars: (MultiDict([('use_panels', 'True')]),
'use_panels=True')
wsgi process: 'Multithreaded'
Any help would be much appreciated.
Thanks,
Liisa
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] ToolShed README files, was: Blast2GO local instance
Hello Peter, My recent change set 7753:c28c5f906c6b added the ability to view a tool shed repository's README file from the pop-up menu on pages in the tool shed repository as well as in the Galaxy Admin UUI for installed repositories. Since many of these README files are rather large, I needed to display them on a separate page, hence the need for the new Repository Actions popup menu View README item. On Aug 28, 2012, at 9:48 AM, Peter Cock wrote: This works only if using the tool shed installation process, so for example, if installing the venn_list repository, clicking Install to local Galaxy in this screenshot: Ah. I see now it just isn't doing what I expected - apologies for any misdirection. I had assumed if a README file was present, it would be shown prominent while browsing on the ToolShed. On closer examination of an example repository where there is a file named exactly 'README' (e.g. fubar's clustalw tool) that isn't shown either. Is there any reason not to show the contents of the README file (using the discussed name variations) on the main ToolShed repository view, e.g. under a new yellow section header? ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] ToolShed README files, was: Blast2GO local instance
Hi Peter, I've handled this request in change set revision 7761:9972ac6ee91d, which will be included in the next Galaxy release scheduled for about 10 days from now. Thanks! Greg Von Kuster On Aug 28, 2012, at 9:48 AM, Peter Cock wrote: P.S. It would be nice to automatically line wrap the Detailed description, as in many cases the raw text doesn't do this. For example, fubar's clustalw tool. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] ToolShed README files, was: Blast2GO local instance
On Thursday, September 27, 2012, Greg Von Kuster wrote: Hello Peter, My recent change set 7753:c28c5f906c6b added the ability to view a tool shed repository's README file from the pop-up menu on pages in the tool shed repository as well as in the Galaxy Admin UUI for installed repositories. Since many of these README files are rather large, I needed to display them on a separate page, hence the need for the new Repository Actions popup menu View README item. Great - and thank you for the line wrapping change too - I look forward to seeing these refinements in action :) Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Can't create custom visualization
What version of Galaxy are you using? I cannot reproduce the issue using the
most recent distribution or galaxy-central.
Also, there should be a full stacktrace in your paster.log. Can you share that
please?
Thanks,
J.
On Sep 27, 2012, at 11:05 AM, Liisa Koski wrote:
Hello,
I'm trying to create a custom visualization in Trackster. I am doing the
following steps. Visualization - New Visualization - Add a Custom Build
I get directed to the url below:
http://galaxy.ca:8080/user/dbkeys?use_panels=True
But the web page says:
Server Error
An error occurred. See the error logs for more information. (Turn debug on to
display exception reports here)
My paster.log file shows the following:
WSGI Variables
--
application: paste.recursive.RecursiveMiddleware object at 0xf070d90
paste.cookies: (SimpleCookie:
__utma='99541067.1404064366.1320847925.1320847925.1320857512.2'
galaxycommunitysession='eb142648ac45b770e95464ae1d51cc6457899dd48ca7f3e61a826ece0a2b5d2a65f4ac4aabb6e5e9'
galaxyreportssession='c6ca0ddb55be603a922aa045b4afae662fca8487217f2ea25e5b488dcf5d52aefac5347681c8bf98'
galaxysession='c6ca0ddb55be603a67ec94afb2c9a07cafdf91af0226f4689d096411ea1bc0c1004977df81ecca90'
toolshedgalaxyurl='http://martin.dnalandmarks.ca:8080/',
'__utma=99541067.1404064366.1320847925.1320847925.1320857512.2;
galaxysession=c6ca0ddb55be603a67ec94afb2c9a07cafdf91af0226f4689d096411ea1bc0c1004977df81ecca90;
galaxycommunitysession=eb142648ac45b770e95464ae1d51cc6457899dd48ca7f3e61a826ece0a2b5d2a65f4ac4aabb6e5e9;
toolshedgalaxyurl=http://martin.dnalandmarks.ca:8080/;;
galaxyreportssession=c6ca0ddb55be603a922aa045b4afae662fca8487217f2ea25e5b488dcf5d52aefac5347681c8bf98')
paste.expected_exceptions: [class 'paste.httpexceptions.HTTPException']
paste.httpexceptions: paste.httpexceptions.HTTPExceptionHandler object at
0xf0702d0
paste.httpserver.thread_pool: paste.httpserver.ThreadPool object at
0x2ae44714c0d0
paste.parsed_querystring: ([('use_panels', 'True')], 'use_panels=True')
paste.recursive.forward: paste.recursive.Forwarder from /
paste.recursive.include: paste.recursive.Includer from /
paste.recursive.include_app_iter: paste.recursive.IncluderAppIter from /
paste.recursive.script_name: ''
paste.throw_errors: True
webob._parsed_query_vars: (MultiDict([('use_panels', 'True')]),
'use_panels=True')
wsgi process: 'Multithreaded'
Any help would be much appreciated.
Thanks,
Liisa
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
[galaxy-dev] collapse FASTA
Hi Galaxy team, I tried to do collapse FASTA sequences for removing the duplicate reads. but error occured due to the lowercase letters at both the ends of the sequences. Is there any option available to clean my reads? some of my reads contains hyphen too. Thanks, D. -- Deepthi Theresa Thomas Kannanayakal 1919 University Drive NW, Apt. No. E104 T2N 4K7 Calgary, Alberta Canada Ph: (403) 483 7409, (403) 618 5956 Email: deepthither...@gmail.com ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Automatic installation of third party dependancies
I'm actually still a bit confused as to what the expected workflow is. Should I be able to clone the tool shed repo to my local development workstation (TOOL_DEV_LOCAL_DIR), commit a change, and then push that change back to the Tool Shed? I've added some comments inline to clarify. Greg Von Kuster wrote: Hi Lance, I need to figure out precisely what steps you are taking to produce this behavior, as I have not been able to do so. Please see my inline comments, and let me know more information about each step you are taking to produce this behavior. On Sep 25, 2012, at 10:24 AM, Lance Parsons wrote: I'm sorry I wasn't more clear. I do believe that those links explain the behavior I am seeing. However, let me try to describe it a different way. It seems that there will, at most, be one installable revision of a given version of a tool. Here I use revision to denote a mercurial revision and version to denote the version described in the tool's config xml. However, while it seems this is the desired behavior, it seems to lead to an undesirable situation in some circumstances. Consider this scenario: 1) Upload version 0.1 of MYTOOL at revision 0: to the tool shed. I assume you are uploading the initial tarball or separate files from a certain local directory. Let's call it TOOL_DEV_LOCAL_DIR. Not quite. I would typically create the repository using the Tool Shed web interface. Then, on my local workstation, I would create the TOOL_DEV_LOCAL_DIR by using the 'hg clone http://toolshedurl/repo' command. 2) Install version 0.1 of MYTOOL at revision 0:XXX to my local Galaxy instance. This now creates a new mercurial repository at a specified location on your Galaxy server, let's call it REPO_INSTALL_DIR. Yes, I use the Galaxy Admin interface to install the tool from a tool shed to my Galaxy instance. 3) Modify MYTOOL, leaving the version at 0.1, You are doing the above step in TOOL_DEV_LOCAL_DIR, correct? Yes, I modify the files in TOOL_DEV_LOCAL_DIR, leaving the version the same. but changing the revision to 1:. The tool shed is creating the new change set hash for you when you push the changes from TOOL_DEV_LOCAL_DIR to the repository in the tool shed, correct? I don't think so. I would typically commit the changes by issuing an 'hg commit' while in my TOOL_DEV_LOCAL_DIR and then push them to the tool shed using the 'hg push' command. These updates would be for things that don't change the input or output of a tool (such as updates to documentation, the addition of tool dependencies, etc.) OK, this make sense. Right, I think we're on the same page here, though this should probably be spelled out somewhere in a Best Practices type document for tool development (which could also help to clarify the steps to setup a sane development environment, test tools, and push them to the public tool shed. I've not even quite gotten a solid workflow for myself, so publishing one might be useful. 4) Attempt to update MYTOOL from my local Galaxy instance. You are using the Galaxy UI to Get updates to your cloned repository, which is located in REPO_INSTALL_DIR, correct? Yes, then I use the Galaxy web admin interface to Get updates. This now results in an exception: RepoLookupError: unknown revision '1' . If answers to all of my above questions are Yes, then this is where I cannot reproduce the behavior you are seeing. If you are actually adding changsets to your cloned repository located in REPO_INSTALL_DIR, then pushing them to the repository in the tool shed, then attempting to get updates from the tool shed to your cloned repository in REPO_INSTALL_DIR, then this process may be causing the exception you are seeing. You should not be adding changesets to the cloned repository, but to your local development environment in TOOL_DEV_LOCAL_DIR. I don't add changesets to the REPO_INSTALL_DIR, but instead make changes in TOOL_DEV_LOCAL_DIR, commit them using mercurial, and then push them to the tool shed using hg push. Deactivating and/or uninstalling MYTOOL works, but any attempt to reinstall the latest version results in an error stating that an older revision of the tool was installed. I now have to way to (easily) update my local Galaxy instance to this updated revision of the tool. My only workaround to this was to increment the version of MYTOOL and push another revision. This allowed me to install the new version and deactivate the old version in my local Galaxy instance. As for the mercurial issue, I'm running OS X, using the Homebrew version of Mercurial: hg --version Mercurial Distributed SCM (version 2.3.1) (see http://mercurial.selenic.com for more information) Copyright (C) 2005-2012 Matt Mackall and others This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. I'm
Re: [galaxy-dev] Automatic installation of third party dependancies
Hi Lance, On Sep 27, 2012, at 4:37 PM, Lance Parsons wrote: I'm actually still a bit confused as to what the expected workflow is. Should I be able to clone the tool shed repo to my local development workstation (TOOL_DEV_LOCAL_DIR), commit a change, and then push that change back to the Tool Shed? Assuming that you eliminate everything in your TOOL_DEV_LOCAL_DIR (you have a clean directory, eliminating any .hg directories that may exist), pull a fresh clone from the tool shed, make changes to the clone you've just pull into TOOL_DEV_LOCAL_DIR, push the changes to the tool shed, this should be fine. A better approach would probably be to simply keep your TOOL_DEV_LOCAL_DIR as the master repository, never pulling to it from the tool shed (because no one except you is able to change the repository in the tool shed) and just pushing changes from TOOL_DEV_LOCAL_DIR to the tool shed repository. So, the path is something like: TOOL_DEV_LOCAL_DIR pushes new changes to your tool shed repository, and others in the Galaxy community clone from your tool shed repository, including updates you committed over time Greg Von Kuster I've added some comments inline to clarify. Greg Von Kuster wrote: Hi Lance, I need to figure out precisely what steps you are taking to produce this behavior, as I have not been able to do so. Please see my inline comments, and let me know more information about each step you are taking to produce this behavior. On Sep 25, 2012, at 10:24 AM, Lance Parsons wrote: I'm sorry I wasn't more clear. I do believe that those links explain the behavior I am seeing. However, let me try to describe it a different way. It seems that there will, at most, be one installable revision of a given version of a tool. Here I use revision to denote a mercurial revision and version to denote the version described in the tool's config xml. However, while it seems this is the desired behavior, it seems to lead to an undesirable situation in some circumstances. Consider this scenario: 1) Upload version 0.1 of MYTOOL at revision 0: to the tool shed. I assume you are uploading the initial tarball or separate files from a certain local directory. Let's call it TOOL_DEV_LOCAL_DIR. Not quite. I would typically create the repository using the Tool Shed web interface. Then, on my local workstation, I would create the TOOL_DEV_LOCAL_DIR by using the 'hg clone http://toolshedurl/repo' command. 2) Install version 0.1 of MYTOOL at revision 0:XXX to my local Galaxy instance. This now creates a new mercurial repository at a specified location on your Galaxy server, let's call it REPO_INSTALL_DIR. Yes, I use the Galaxy Admin interface to install the tool from a tool shed to my Galaxy instance. 3) Modify MYTOOL, leaving the version at 0.1, You are doing the above step in TOOL_DEV_LOCAL_DIR, correct? Yes, I modify the files in TOOL_DEV_LOCAL_DIR, leaving the version the same. but changing the revision to 1:. The tool shed is creating the new change set hash for you when you push the changes from TOOL_DEV_LOCAL_DIR to the repository in the tool shed, correct? I don't think so. I would typically commit the changes by issuing an 'hg commit' while in my TOOL_DEV_LOCAL_DIR and then push them to the tool shed using the 'hg push' command. These updates would be for things that don't change the input or output of a tool (such as updates to documentation, the addition of tool dependencies, etc.) OK, this make sense. Right, I think we're on the same page here, though this should probably be spelled out somewhere in a Best Practices type document for tool development (which could also help to clarify the steps to setup a sane development environment, test tools, and push them to the public tool shed. I've not even quite gotten a solid workflow for myself, so publishing one might be useful. 4) Attempt to update MYTOOL from my local Galaxy instance. You are using the Galaxy UI to Get updates to your cloned repository, which is located in REPO_INSTALL_DIR, correct? Yes, then I use the Galaxy web admin interface to Get updates. This now results in an exception: RepoLookupError: unknown revision '1' . If answers to all of my above questions are Yes, then this is where I cannot reproduce the behavior you are seeing. If you are actually adding changsets to your cloned repository located in REPO_INSTALL_DIR, then pushing them to the repository in the tool shed, then attempting to get updates from the tool shed to your cloned repository in REPO_INSTALL_DIR, then this process may be causing the exception you are seeing. You should not be adding changesets to the cloned repository, but to your local development environment in TOOL_DEV_LOCAL_DIR. I don't add changesets to the REPO_INSTALL_DIR, but instead make changes in
Re: [galaxy-dev] problem installing blast_datatypes manually
Hi Liisa and Peter, There was an issue if your workflows were created using the ncbi blast tools from the distribution (e.g. versions 0.0.11) and you only had installed the latest tools from the toolshed (e.g. versions 0.0.13). This has been corrected in changesets 7754:371f6770d25c through 7756:c76e2882d2d2 and will be available in the next distribution. Please let us know if you are still experiencing issues after the update us released. The preferred method for handling tools that have been migrated is to use the provided migration scripts (in this case ' sh ./scripts/migrate_tools/0004_tools.sh' or 'sh ./scripts/migrate_tools/0004_tools.sh install_dependencies') and then starting the Galaxy server; after running the migrations to get the original tool versions installed, you can then install the the newest versions of the tools via the admin interface. However, using the migration scripts is optional, and you can always install the original tools via the admin interface, or manually, if that is desired. Thanks for using Galaxy, Dan On Sep 26, 2012, at 9:29 AM, Peter Cock wrote: On Wed, Sep 26, 2012 at 2:20 PM, Liisa Koski liisa.ko...@basf.com wrote: Thanks Peter, It was in fact the snffier type. I changed it to sniffer type=galaxy.datatypes.blast:BlastXml/ And its all working now (as far as you can tell)? I am using the July07 distribution. The reason I installed the datatypes manually is because when I did it via the directions with the toolshed all my workflows that used blast tools were broken. They could no longer find the blast tools. So I immediately removed the toolshed installation and tried to set up as before. With the blast wrappers back in the tools/ncbi_blast_plus directory. Now my workflows are functional again. That is strange - and one for the Galaxy team to puzzle over. I've not tried this myself (since I'm actively developing the BLAST+ wrappers, and often trying things before pushing them to the ToolShed for wider user, it is easier for me to install them locally via the manual route). Thanks for your feedback, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
