Hello Björn, I've analyzed this and discovered some very important points to make sure everyone understands, so I hope my information below is very clear. Please don't hesitate to ask any questions for clarification. The items fall into 2 categories.
First item: ======== The first item for discussion is a known weakness of the tool shed upload process (or, to be more correct, the process of adding a new changeset revision to a repository). When you commit a changeset to a repository that has previous revisions, the tool shed repository metadata process is executed and the contents of the new changeset are analyzed to determine if a new metadata record should be created for the changeset or if the previous metadata record should be moved up in the repository changelog chain (this is a very complex analysis). Until about 1 year ago, I reset all metadata on the entire repository chjangelog when a new changeset was committed. This process worked very well, and metadata was cleanly set for the entire changelog. However, this process began to take significant periods of time as soon as repositories became larger and the changelogs became longer (a good example is http://testtoolshed.g2.bx.psu.edu/view/mzytnicki/s_mart). To improve performance when adding new changesets to repositories, the metadata analysis process was enhanced to inspect only those changesets that go back to the previous revision with associated metadata. This process works very well in most cases. However, there are unusual cases where the process breaks, and your confab repository fell into this category. The issue arises when the contents of a specific change set do not properly move the previous metadata revision forward in the changelog, but instead create a new, additional metadata revision. When I initially inspected your confab repository had metadata associated with the following revisions: 9:dd3ee8e742dc 8:113e876c2ec6 7:9e38b8bd4cdb 6:7593411dcd5a 5:aac0c82ac354 4:6c8f72ee4a51 3:09acaeb233d1 2:e7bb18ef7f54 1:49274c60f392 0:ea7816847e5e I inspected the main contents (tools and dependencies) of these revisions and noticed that some did not seem to have any differences (contents of the containers had the same labels between revisions - I did not inspect the changelog itself though). Because of this I decided to reset all metadata on the repository (available in the Repository Actions menu), and now metadata is associated with only the following revisions: 9:dd3ee8e742dc 6:7593411dcd5a 5:aac0c82ac354 3:09acaeb233d1 2:e7bb18ef7f54 1:49274c60f392 0:ea7816847e5e This is a very complex issue to handle, as there are many things to consider. Obviously automatically inspecting the entire changelog when a new changeset is added will not work. I have added the ability for a repository owner to reset all metadata on the repository, but there is nothing to alert them to initiate this process (and I'm not sure how / when to do so). Also, resetting all metadata on a repository eliminates the tool test results associated with each of the original metadata revisions. Second item: ========== It seems there is some confusion about the changeset revision that is defined in dependency definitions. This is a fairly complex subject, and it seems I may have muddied the waters a bit yesterday. If so, I apologize, but I think Peter's last response to my example clarifies the issue. Looking at the changelog of your confab repository, I see the following. You have unnecessarily changed the changeset_revision setting in your tool dependency definitions, and this has resulted in a new additional metadata record being associated with changeset 9:dd3ee8e742dc of your confab repository. This changeset revision setting should not have been changed - see my discussion below. Repository 'confab' =============== Changeset 9e38b8bd4cdb modified: tool_dependencies.xml tool_dependencies.xml --- a/tool_dependencies.xmlTue May 14 04:12:06 2013 -0400 +++ b/tool_dependencies.xmlTue May 14 04:17:08 2013 -0400 @@ -1,13 +1,13 @@ <tool_dependency> <package name="eigen" version="2.0.17"> - <repository toolshed="http://testtoolshed.g2.bx.psu.edu/"; name="package_eigen_2_0" owner="bgruening" changeset_revision="294a30630e0b" prior_installation_required="True" /> + <repository toolshed="http://testtoolshed.g2.bx.psu.edu/"; name="package_eigen_2_0" owner="bgruening" changeset_revision="09eb05087cd0" prior_installation_required="True" /> </package> <package name="confab" version="1.0.1"> <install version="1.0"> <actions> <!-- populate the environment variables from the dependend repos --> <action type="set_environment_for_install"> - <repository toolshed="http://testtoolshed.g2.bx.psu.edu/"; name="package_eigen_2_0" owner="bgruening" changeset_revision="294a30630e0b"> + <repository toolshed="http://testtoolshed.g2.bx.psu.edu/"; name="package_eigen_2_0" owner="bgruening" changeset_revision="09eb05087cd0"> <package name="eigen" version="2.0.17" /> </repository> </action> Looking at the changelog of your package_eigen_2_0 repository, I see the following: Repository 'package_eigen_2_0' ========================== Changesets Repository metadata is associated with this change set. Description: Uploaded Commit: 2:09eb05087cd0 Parent: 1:294a30630e0b Commited by: bgruening Pushed: ~ 11 hours ago Description: Uploaded Commit: 1:294a30630e0b Parent: 0:5c44ce7e3f7b Commited by: bgruening Pushed: ~ 11 hours ago Description: Uploaded Commit: 0:5c44ce7e3f7b Parent: None Commited by: bgruening Pushed: ~ 11 hours ago Notice that metadata is associated with only changeset revision 2:09eb05087cd0. What this implies is that any existing dependency definition that defines the changeset_revision value as one of 5c44ce7e3f7b, 294a30630e0b or 09eb05087cd0 will all get the same changeset revision installed, and that installed revision will be 09eb05087cd0. This is a very important point, so please let me know if the reasons for this are not clear. Yesterday we had a discussion about the "minimum" setting for a changeset revision. Here is Peter's feedback at the end of that discussion. ==================== Perhaps this is clearer?: changset revision Installable revision ----------------------------------------------------------------------------- 0: sjekvub not any more, superseded by r1 1: jjtofvp not any more, superseded by r2 2: htocegy yes, final revision with this metadata ----------------------------------------------------------------------------- 3: jswofpt not any more, superseded by r4 4: jaqvkrc yes, final revision with this metadata ==================== Do you understand why you should not have changed the value of the changeset_revision setting in the tool dependency definition your confab repository? It is because this concept of "minimum" setting comes into play here. When the value of the changeset_revision setting is changed, it will always generate a new metadata revision, which is not what you wanted in this case. - <repository toolshed="http://testtoolshed.g2.bx.psu.edu/"; name="package_eigen_2_0" owner="bgruening" changeset_revision="294a30630e0b" prior_installation_required="True" /> + <repository toolshed="http://testtoolshed.g2.bx.psu.edu/"; name="package_eigen_2_0" owner="bgruening" changeset_revision="09eb05087cd0" prior_installation_required="True" /> - <repository toolshed="http://testtoolshed.g2.bx.psu.edu/"; name="package_eigen_2_0" owner="bgruening" changeset_revision="294a30630e0b"> + <repository toolshed="http://testtoolshed.g2.bx.psu.edu/"; name="package_eigen_2_0" owner="bgruening" changeset_revision="09eb05087cd0"> Greg Von Kuster On May 14, 2013, at 3:59 AM, Björn Grüning wrote: > Hi Greg, > > during the addition of the next dependency (libeigen) I found an other > bug related to repository updates. Please see revision 6c8f72ee4a51 of > confab in the testtoolshed. I installed that and want to update to > revision aac0c82ac354 with new dependency. I saw that a new metadata is > associated, but I'm not able to update: > > 'The installed repository named 'confab' is current, there are no > updates available.' > > Cheers, > Bjoern > > > > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
