Hi lance,
Another best practice I should have mentioned: if you are using your
TOOL_DEV_LOCAL_DIR as a source code repository for developing your tools
(where you are committing tool development change sets into your hg repository
in TOOL_DEV_LOCAL_DIR), you should not push the change log history to the tool
shed when you have your developed tools functional in TOOL_DEV_LOCAL_DIR
The tool shed generates metadata for the contents of each change set pushed to
its repository, so you want a single, functionally correct change set for each
push to the tool shed, not the history of small development changes you made in
your development repository in TOOL_DEV_LOCAL_DIR.
A good process to use for this is to create a new tarball of your
TOOL_DEV_LOCAL_DIR repository when you are ready to push it to the tool shed
and upload the tarball. The tool shed will strip out unwanted items from your
tarball like .hg subdirectories.
You could also choose to not use a source code revision system in your
TOOL_DEV_LOCAL_DIR if that is more appropriate.
Greg Von Kuster
On Sep 27, 2012, at 5:02 PM, Greg Von Kuster wrote:
Hi Lance,
On Sep 27, 2012, at 4:37 PM, Lance Parsons wrote:
I'm actually still a bit confused as to what the expected workflow is.
Should I be able to clone the tool shed repo to my local development
workstation (TOOL_DEV_LOCAL_DIR), commit a change, and then push that change
back to the Tool Shed?
Assuming that you eliminate everything in your TOOL_DEV_LOCAL_DIR (you have a
clean directory, eliminating any .hg directories that may exist), pull a
fresh clone from the tool shed, make changes to the clone you've just pull
into TOOL_DEV_LOCAL_DIR, push the changes to the tool shed, this should be
fine.
A better approach would probably be to simply keep your TOOL_DEV_LOCAL_DIR as
the master repository, never pulling to it from the tool shed (because no
one except you is able to change the repository in the tool shed) and just
pushing changes from TOOL_DEV_LOCAL_DIR to the tool shed repository.
So, the path is something like:
TOOL_DEV_LOCAL_DIR pushes new changes to your tool shed repository, and
others in the Galaxy community clone from your tool shed repository,
including updates you committed over time
Greg Von Kuster
I've added some comments inline to clarify.
Greg Von Kuster wrote:
Hi Lance,
I need to figure out precisely what steps you are taking to produce this
behavior, as I have not been able to do so. Please see my inline comments,
and let me know more information about each step you are taking to produce
this behavior.
On Sep 25, 2012, at 10:24 AM, Lance Parsons wrote:
I'm sorry I wasn't more clear. I do believe that those links explain the
behavior I am seeing. However, let me try to describe it a different way.
It seems that there will, at most, be one installable revision of a given
version of a tool. Here I use revision to denote a mercurial revision and
version to denote the version described in the tool's config xml.
However, while it seems this is the desired behavior, it seems to lead to
an undesirable situation in some circumstances. Consider this scenario:
1) Upload version 0.1 of MYTOOL at revision 0: to the tool shed.
I assume you are uploading the initial tarball or separate files from a
certain local directory. Let's call it TOOL_DEV_LOCAL_DIR.
Not quite. I would typically create the repository using the Tool Shed web
interface. Then, on my local workstation, I would create the
TOOL_DEV_LOCAL_DIR by using the 'hg clone http://toolshedurl/repo' command.
2) Install version 0.1 of MYTOOL at revision 0:XXX to my local Galaxy
instance.
This now creates a new mercurial repository at a specified location on your
Galaxy server, let's call it REPO_INSTALL_DIR.
Yes, I use the Galaxy Admin interface to install the tool from a tool shed
to my Galaxy instance.
3) Modify MYTOOL, leaving the version at 0.1,
You are doing the above step in TOOL_DEV_LOCAL_DIR, correct?
Yes, I modify the files in TOOL_DEV_LOCAL_DIR, leaving the version the same.
but changing the revision to 1:.
The tool shed is creating the new change set hash for you when you push the
changes from TOOL_DEV_LOCAL_DIR to the repository in the tool shed, correct?
I don't think so. I would typically commit the changes by issuing an 'hg
commit' while in my TOOL_DEV_LOCAL_DIR and then push them to the tool shed
using the 'hg push' command.
These updates would be for things that don't change the input or output of
a tool (such as updates to documentation, the addition of tool
dependencies, etc.)
OK, this make sense.
Right, I think we're on the same page here, though this should probably be
spelled out somewhere in a Best Practices type document for tool
development (which could also help to clarify the steps to setup a sane
