Jorrit,
I've not been able to reproduce the first scenario using a local Galaxy
instance against the test tool shed. Could you tell me which revision of
Galaxy you're running when you encounter this behavior?
I do see that the revision bb20766df19c that you were initially trying
to update is immediately followed by another downloadable revision, in
which case Galaxy will (correctly) not find any updates for it, due to
the reproducibility concerns I mentioned in my previous email.
--Dave B.
On 3/28/13 17:58:10.000, jorrit poelen wrote:
Hey Dave,
I took some time to try to reproduce the behavior you described using a local
test tool shed.
As I understand your previous email, the first scenario (below) outcome of
using my local test toolshed is expected, but the second scenario is not.
Also, the first scenario outcome using the psu test toolshed seems different
compared to running it against my local test tool shed. I was unable to test
the second scenario, because of a apparent (recently introduced) bug in the psu
test tool shed login.
What am I missing?
Thanks much for you patience,
-jorrit
-------
First scenario - Change README / update
--------
Steps:
1. install repository A from local test tool shed B into local galaxy instance C
2. clone respository A from local test tool into local dev environment
**3. apply change to README**
4. hg commit / push changes to local test tool shed B
5. verify that test tool shed received changes by checking content of README
and change set hash.
6. in galaxy instance C > admin > manage installed tool shed repositories >
repository A > get updates
7. galaxy instance C web page says that updates have been detected and applied
**8. verify that README of repo A in galaxy instance C has been changed.**
Outcome:
local tool shed instance test - update detected, changes were applied to local
galaxy instance C
psu tool shed test - update *not* detected, changes were *not* applied to local
galaxy instance C (see initial post)
------
Second scenario - Bump tool version / update
------
Steps: same as first scenario, except for
**3. apply change to tool version 0.1 --> 0.11 **
**8. verify that tool version 0.1 has changed to 0.11 of repo A in galaxy
instance C has been changed.**
Outcome:
local tool shed instance test - update detected, changes were applied: a new
version of tool is installed in Galaxy instance.
psu tool shed test - unable to test because I could not login to psu test
toolshed
On Mar 28, 2013, at 2:19 PM, Dave Bouvier wrote:
Jorrit,
I may have been a bit unclear in my previous response. Allow me to illustrate
two scenarios:
First scenario:
1. A repository is created containing a BWA tool with version 0.5.9
2. The repository is installed to Galaxy.
3. New README file is uploaded, tool version is *not* changed.
Second scenario:
1. A repository is created containing a BWA tool with version 0.5.9
2. The repository is installed to Galaxy.
3. New README file is uploaded, and updated BWA tool is uploaded with version
0.6
In the first scenario, Galaxy will find that the repository has been updated,
and apply the update.
In the second scenario, Galaxy will not find that the repository has been
updated, but will allow you to install both repositories, the one containing
version 0.5.9 and the one containing version 0.6 of the BWA tool.
The reason for this behavior is that if a tool has been updated to produce
different output, the tool version should be updated, which leads to a separate
installable changeset revision being generated. This way, any analyses run on
the old tool version can be re-run and produce the same output, but any new
analyses can run on the new version of the tool, with whatever new features
have been added. This ensures reproducibility throughout the version lineage of
the tool, while still allowing Galaxy users to use the most recent version of
an available tool for future analyses.
--Dave B.
On 3/28/13 17:03:10.000, jorrit poelen wrote:
Hey Dave,
Thanks for pointing me to the wiki that helped demystify the hg changeset <-->
repository metadata connection for me.
I'll make sure to change the version numbers in the tool xml definitions
whenever a change needs to be make available to galaxy instances through the
toolshed. So, even for an update in the README file, I'll make sure to change
some version to make the change available. This feels a little clunky for me
now, but I am sure there's a good reason to make it work as it does now.
I very much appreciate your prompt response and I'll make sure to dig around
the wiki a little longer before posting a question next time ; )
-jorrit
On Mar 26, 2013, at 10:41 AM, Dave Bouvier wrote:
Jorrit,
First, allow me to clarify a point of naming:
It's not quite right to say "tools" in this situation, as tools are only one of
several utilities that a tool shed repository can contain, along with datatypes,
workflows, repository dependencies, and so on. This is an important distinction, because
some repositories can (and do) contain no tools, and only datatypes and/or other Galaxy
utilities.
In your process description for the current behavior, the tool shed will not
report updates to a repository if the tool version has changed. For more
details on how and why this happens, see the following wiki page:
http://wiki.galaxyproject.org/RepositoryRevisions#Adding_additional_change_sets_to_the_initial_change_set_in_a_repository
Where the behavior is detailed in these sections:
Adding additional change sets to the initial change set in a repository
• What is repository metadata?
• What does it mean when repository metadata is associated with a
change set in the repository change log?
• Adding a change set that does not generate a new repository metadata
record
• Adding a change set that generates a new repository metadata record
• More examples of installable repository change set revisions
--Dave B.
On 3/26/13 12:45:41.000, jorrit poelen wrote:
Hey Dave,
Thanks against for your quick response!
I checked my universe_wsgi.ini file and can confirm that the
enabled_tool_shed_check was *commented*.
I've uncommented the settings and changed them to:
enable_tool_shed_check = True
hours_between_check = 1
Then I followed the steps outlined in my initial email and the observed the
same (unexpected) behavior.
Perhaps I am describing a feature request rather than a bug: My expectation is
that whenever I explicitly request to update a tool, a immediate check is
performed against the relevant tool shed.
To make sure that I correctly understand the existing tool update feature,
please confirm that the following describes the currently expected tool update
behavior:
0. admin of galaxy instance Z enables the feature to check for tool updates in
toolsheds every hour
1. tool X revision 1.0 is installed from tool shed Y into galaxy instance Z by
tool developer using hg push
2. tool X is current revision is updated to revision 1.1 in tool shed Y
3. immediately after, admin of galaxy instance Z attempts to (manually) upgrade
tool X to version 1.1 using the admin web interface
4. galaxy instance Z reports that no new version of tool X are available, even
though a new version exists in the toolshed
5. galaxy instance Z detects version 1.1 of tool X at most one hour after the
update was made
6. admin of galaxy instance Z tries again
Thanks for your patience and efforts,
-jorrit
On Mar 26, 2013, at 8:19 AM, Dave Bouvier wrote:
Jorrit,
Just to eliminate one possible cause of this behavior, could you confirm that
your local Galaxy instance's universe_wsgi.ini file's enable_tool_shed_check
setting is uncommented and set to True, and the hours_between_check is also
uncommented and set to a value between 1 and 12?
--Dave B.
On 3/25/13 21:26:07.000, jorrit poelen wrote:
Hey y'all,
I noticed that my local galaxy instance doens't pickup latest updates from test
toolshed.
Steps to reproduce:
1. install obotools from test toolshed using hg clone
http://jor...@testtoolshed.g2.bx.psu.edu/repos/jorrit/obotools , revision
45:bb20766df19c
2. hg push new change of obotools of galaxy tool to testtoolshed , creating
revision 46:532808477800
3. update obotools in galaxy admin
expected result: galaxy admin detects new version and give an option to update.
actual result: galaxy admin does not detect new version.
I've attached some galaxy logging below. One thing I noticed that the reported commit
time in testtoolshed seems 7 hours off - just minutes after I pushed the change,
testtoolshed says the change was commit "~ 7 hours ago".
Workaround I am using to do update is to deactivate and remove the tool all
together, and re-install straight from test-toolshed. What should I do to make
the tool version update work? I am probably missing something.
Thanks!
-jorrit
Galaxy logging on looking for updates in test toolshed:
---
24.130.122.217 - - [26/Mar/2013:01:12:46 +0000] "GET /admin_toolshed/browse_repositories HTTP/1.1"
200 - "http://galaxy:7474/admin" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5)
AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10"
24.130.122.217 - - [26/Mar/2013:01:12:51 +0000] "GET
/admin_toolshed/browse_repositories?operation=get+updates&id=a799d38679e985db HTTP/1.1" 302 -
"http://galaxy:7474/admin_toolshed/browse_repositories" "Mozilla/5.0 (Macintosh; Intel Mac OS X
10_7_5) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10"
24.130.122.217 - - [26/Mar/2013:01:12:51 +0000] "GET
/admin_toolshed/check_for_updates?id=a799d38679e985db HTTP/1.1" 302 -
"http://galaxy:7474/admin_toolshed/browse_repositories" "Mozilla/5.0 (Macintosh; Intel Mac OS
X 10_7_5) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10"
24.130.122.217 - - [26/Mar/2013:01:12:52 +0000] "GET
/admin_toolshed/update_to_changeset_revision?tool_shed_url=http://testtoolshed.g2.bx.psu.edu/&name=obotools&owner=jorrit&changeset_revision=bb20766df19c&latest_changeset_revision=bb20766df19c&latest_ctx_rev=45
HTTP/1.1" 302 - "http://galaxy:7474/admin_toolshed/browse_repositories" "Mozilla/5.0 (Macintosh; Intel Mac
OS X 10_7_5) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10"
24.130.122.217 - - [26/Mar/2013:01:12:52 +0000] "GET
/admin_toolshed/manage_repository?status=done&message=The+installed+repository+named+%27obotools%27+is+current%2C+there+are+no+updates+available.++&id=a799d38679e985db
HTTP/1.1" 200 - "http://galaxy:7474/admin_toolshed/browse_repositories" "Mozilla/5.0 (Macintosh;
Intel Mac OS X 10_7_5) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10"
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/