Hi John,

On Jun 8, 2012, at 1:22 PM, John Chilton wrote:

> Hello Greg,
> Thanks for the prompt and detailed response (though it did make me
> sad). I think deploying tested, static components and configurations
> to production environments and having production environments not
> depending on outside services (like the tool shed) should be
> considered best practices.

I'm not sure I understand this issue.  What processes are you using to upgrade 
your test and production servers with new Galaxy distributions?  If you are 
new Galaxy distributions from our Galaxy dist repository in bitbucket, then 
pulling tools from the Galaxy tool shed is not much different - both are 
outside services.  Updating your test environment, determining it is 
functionally correct, and then updating your production environment using the 
same approach would generally follow a best practice approach.  This is the 
approach we are currently using for our public test and main Galaxy instances 
at Penn State.

> Oh well, I guess. Would there be a way to at least automate the
> pulling of tools in.

The process is completely automated - all you need to do execute a script, 
something like this:

sh ./scripts/migrate_tools/0002_tools.sh

This is the same process used when the Galaxy database schema migrates as part 
of a new Galaxy release, except in that case you would run a script like this:

sh manage_db.sh upgrade

> For instance, would it make sense to tweak
> InstallManager to parse a new kind of migration file that is a lot
> like the "official" migration files, but with the sections defined in
> the file. For this new kind of migration,  the InstallManager would
> then import everything in the file and not just the tools that are
> also in a tool_conf? Does that make sense? If yes, I imagine it could
> be modified to handle updates the same way?

If I understand this correctly, this is how the InstallManage works.  The 
entire tool shed repository is installed into your local Galaxy environment, 
but only the tools that are currently defined in your tool_conf.xml file are 
loaded into your tool panel.  

> Rephrased, I guess the idea would be to have the sequence of official
> galaxy migrations that check tool_conf, and then have a sequence of
> migration defined by the deployer that could be used to install new
> tools or update existing ones.
> My concern isn't just with the dev to production transition, it is
> also the ability to sort of programmatically define Galaxy
> installations the way I am doing with the galaxy-vm-launcher
> (https://bitbucket.org/jmchilton/galaxy-vm-launcher) or the way
> mi-deployment works.

You have complete control with these migrations.  You can chose to not install 
any tools shed repositories, and just start your Galaxy server.  If you choose 
to install the defined repositories, you have control over what specific tools 
included in the repository are loaded into your tool panel by having them 
defined in your tool_conf.xml prior to the installation.  This whole process is 
associated only with tools that have moved from the Galaxy distribution to the 
tool shed.

> Thanks again for your time and patience in explaining these things to me,
> -John
> On Fri, Jun 8, 2012 at 4:19 AM, Greg Von Kuster <g...@bx.psu.edu> wrote:
>> Hi John,
>> On Jun 7, 2012, at 11:55 PM, John Chilton wrote:
>>> I have read through the documentation a couple times, but I still have
>>> a few questions about the recent tool shed enhancements.
>>> At MSI we have a testing environment and a production environment and
>>> I want to make sure the tool versions and configurations don't get out
>>> of sync, I would also like to test everything in our testing
>>> environment before it reaches production.
>>> Is there a recommended way to accomplish this rather than just
>>> manually repeating the same set of UI interactions twice?
>>> Can I just import tools through the testing UI and run the
>>> ./scripts/migrate_tools/XXXX scripts on our testing repository and
>>> then move the resulting migrated_tools_conf.xml and
>>> integrated_tool_panel.xml files into production? I have follow up
>>> questions, but I will wait for a response on this point.
>> Tools that used to be in the Galaxy distribution but have been moved to the 
>> main Galaxy tool shed are automatically installed when you start up your 
>> Galaxy server and presented with the option of running the migration script 
>> to automatically install the tools that were migrated in the specific Galaxy 
>> distribution release.  If you choose to install the tools, they are 
>> installed only in that specific Galaxy instance.  Installation produces 
>> mercurial repositories that include the tools on disk in your Galaxy server 
>> environment.  Several other things are produced as well, including database 
>> records for the installation.  Each Galaxy instance consists of it's own 
>> separate set of components, this installation process must be done for each 
>> instance.  The installation is fully automatic, requiring little interaction 
>> on the part of the Galaxy admin, and doesn't require much time, so 
>> performing the process for each Galaxy instance should not be too intensive. 
>>  Also, the tools that !
 are installed into each Galaxy instance's tool panel are only those tools that 
were originally defined in the tool panel configuration file (tool_conf.xml).  
This approach provides for the case where each Galaxy instance having different 
tools defined will not be altered by the migration process.
>>> Also as you are removing tools from Galaxy and placing them into our
>>> tool shed, what is the recommended course of actions for deployers
>>> that have made local minor tweaks to those tool configs and scripts
>>> and adapt them to our local environments? Along the same lines, what
>>> is the recommended course of action if we need to make minor tweaks to
>>> tools pulled into through the UI to adapt them to our institution.
>> In both cases you should upload your proprietary tools to either a local 
>> Galaxy tool shed that you administer, or the main Galaxy tool shed if you 
>> want.  You can choose to not execute any of the tool migration scripts, so 
>> the Galaxy tools that were migrated from the distribution will not be 
>> installed into your Galaxy environment.  You can use the Galaxy admin UI to 
>> install your proprietary versions of the migrated tools from the tool shed 
>> in which you chose to upload and store them.  New versions of the tools can 
>> be uploaded to respective tool shed repositories over time.
>>> Thanks for your time,
>>> -John
>>> ------------------------------------------------
>>> John Chilton
>>> Senior Software Developer
>>> University of Minnesota Supercomputing Institute
>>> Office: 612-625-0917
>>> Cell: 612-226-9223
>>> ___________________________________________________________
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>> http://lists.bx.psu.edu/

Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:


Reply via email to