[galaxy-dev] Updating tools on new hg based Galaxy Tool Shed
Hi all, I just tried updating one of my tools on the new hg based Galaxy Tool Shed and ran into some issues. I guess I'm the first person to try this... First of all, there is no clear update action like there used to me. Could that be restored please? https://bitbucket.org/galaxy/galaxy-central/issue/585/hg-tool-shed-lacks-update-tool-action I tried using the upload files to repository option, and picked my tarball. I did not pick a location since I wanted the tar ball unpacked at the repository root, but this isn't what happened: https://bitbucket.org/galaxy/galaxy-central/issue/586/hg-tool-shed-upload-files-to-repository Finally, it seems there is currently no delete files action, so I can't fix this (delete the misplaced files) via the web interface. https://bitbucket.org/galaxy/galaxy-central/issue/584/cant-delete-files-in-new-hg-tool-shed Are you expecting tool authors to work primarily at the hg level? Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Updating tools on new hg based Galaxy Tool Shed
Hi Peter, On Jun 16, 2011, at 10:10 AM, Peter Cock wrote: On Thu, Jun 16, 2011 at 2:45 PM, Greg Von Kuster g...@bx.psu.edu wrote: Peter, I've added comments to each of these issues in bitbucket, and have pated them inline below as well. On Jun 16, 2011, at 5:03 AM, Peter Cock wrote: Hi all, I just tried updating one of my tools on the new hg based Galaxy Tool Shed and ran into some issues. I guess I'm the first person to try this... First of all, there is no clear update action like there used to me. Could that be restored please? https://bitbucket.org/galaxy/galaxy-central/issue/585/hg-tool-shed-lacks-update-tool-action The requested functionality is currently possible using normal mercurial features ( clone - make changes to the cloned repo - commit changes to the cloned repo - push changes to the tool shed repo ). Right, but not everyone wants to use hg for this. I understand, and that is why I am working feverishly to add features that do not force the use of hg form the command line (I've got several features functional, but not everything yet). The current tool shed requires some use of hg if you want to delete files from the repo, but other than that, hg is not really required. The current tool shed also provides some very useful features (e.g., browsing tool code files) that were not available in the old tool shed. In addition, tool developers are no longer forced to to the extra work that was necessary in the old tool shed to add a tool (i.e., there is no longer a requirement for tool suite config definitions ). My hope is that the current features make everyone happy enough that they will be able to wait for those messing featuers that are wanted, using the current features in the meantime. It is also currently possible to upload a new tarball to any upload point (position) in the tool shed repository by selecting the upload point in the built-in repository file browser. - The hierarchy that the uploaded tarball contains will be added to that upload point in the tool shed repository. - Any files in the uploaded tarball that do not currently exist in the tool shed repository will be added. - Any files that exist in the hierarchy of the uploaded tarball that also exist in the same location of the hierarchy of the repository relative to the upload point will be replaced in the repository. Yes, I tried that and its broken (bug 586 below). This is because you uploaded a gzipped tarball, which mercurial treats like a single file. The current tool shed is based on mercurial, so mercurial's behavior is basically the tool shed's behavior. It is on my list to determine if a file is compressed and if so uncompress it upon upload. A potential problem with this is that some developers may not want their uploaded file uncompressed. It is on my list to: Delete any files that do not exist in the uploaded tarball's hierarchy but that do existin the same position of the tool shed repository's hierarchy relative to the upload point. This feature will require clear documented communication to the user, which I've not yet had a chance to do, but will soon. That would be a change from upload files to replace all the files, in line with the missing update tool functionality (#585). To me mixing these two is quite strange. This should become more clear when the feature to delete files in the hierarchy is added. I tried using the upload files to repository option, and picked my tarball. I did not pick a location since I wanted the tar ball unpacked at the repository root, but this isn't what happened: https://bitbucket.org/galaxy/galaxy-central/issue/586/hg-tool-shed-upload-files-to-repository Peter, I believe this is a duplicate of # 584, correct? Not really, 584 is about deleting files via the web interface. Perhaps you regard 585 (missing tool update) and 586 (broken upload at root) as the same basic issue? See above explanation. Finally, it seems there is currently no delete files action, so I can't fix this (delete the misplaced files) via the web interface. https://bitbucket.org/galaxy/galaxy-central/issue/584/cant-delete-files-in-new-hg-tool-shed You can currently delete specific files by cloning the repository, making any changes you want (including deleting), and committing and pushing the changes using mercurial's command line. Again, I'd prefer not to have to do this. Understood... I've added it to my list to provide a feature allowing a user to select a specific file in the repository (hopefully using the built-in file browser) and deleting the file in some way that doesn't require mercurial's command line, but in the meantime, use the above existing feature. Are you expecting tool authors to work primarily at the hg level? You didn't answer that one ;) Not necessarily, but hg is the basis for uploading and downloading tools. I'm not sure if it will be possible
Re: [galaxy-dev] Updating tools on new hg based Galaxy Tool Shed
On Thu, Jun 16, 2011 at 3:26 PM, Greg Von Kuster g...@bx.psu.edu wrote: Hi Peter, I understand, and that is why I am working feverishly to add features that do not force the use of hg form the command line (I've got several features functional, but not everything yet). The current tool shed requires some use of hg if you want to delete files from the repo, but other than that, hg is not really required. That's great, but I do feel the new hg went live a bit prematurely. The current tool shed also provides some very useful features (e.g., browsing tool code files) that were not available in the old tool shed. Really? I find the browsing the tool code files has regressed. In old tool shed had a simple non-interactive tree of the files visible on the main page for a tool or suite, where the individual files could be clicked on to view/download. It was simple and effective (at least when there weren't many files which seems typical). The new tool shed makes you click on tool actions, browse repo, and then gives an interactive tree which is fully collapsed by default, forcing lots more clicking to drill down to the files. All that clicking makes it slow and fiddly to use. In addition, tool developers are no longer forced to to the extra work that was necessary in the old tool shed to add a tool (i.e., there is no longer a requirement for tool suite config definitions ). For tool suites you have a point, that was a bit of a hassle. My hope is that the current features make everyone happy enough that they will be able to wait for those messing featuers that are wanted, using the current features in the meantime. It'll get there :) Another big regression I would prioritise is the complete lack of any user provided text on a tool's main page. The old tool shed had the minimal text box where you could summarise the tool, its requirements, recent changes etc. The new tool shed has nothing to replace this as far as I can see. A simple wiki like, ReST, or just plain text details box would do. Automatic detection of a readme text file in the repo, to be displayed on the tools' main page would be an elegant solution (are you familiar with how github.com does this?). It also keeps the description in the hg repo which is a plus point. The mock up idea would be another (more complex) way to tackle this, https://bitbucket.org/galaxy/galaxy-central/issue/565 Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Updating tools on new hg based Galaxy Tool Shed
On Jun 16, 2011, at 10:23 AM, Peter Cock wrote: On Thu, Jun 16, 2011 at 3:26 PM, Greg Von Kuster g...@bx.psu.edu wrote: Hi Peter, On Jun 16, 2011, at 10:10 AM, Peter Cock wrote: Are you expecting tool authors to work primarily at the hg level? You didn't answer that one ;) Not necessarily, but hg is the basis for uploading and downloading tools. I'm not sure if it will be possible to completely eliminate the requirement of a tool developer using hg, but we're making every attempt to do so. Good. Why are you so averse to using hg? Because git suits me better? ;) Seriously, I have no strong aversion to hg - what puts me off a little is the need to have one repo per tool or tool-suite, compared to my current setup where all by tools are in a branch from the main Galaxy repo. There would be a significant time and effort cost in switching, made worse by having multiple tools on the Tool Shed. I'm actually thinking of this (requiring hg knowledge) as a more general issue, namely a potential impediment to new Tool Shed contributors. Peter I'm of the opposite mind; I'm not sure there is an absolute need to have one's tools be a branch or fork of the main tool shed, though I agree there is also utility in allowing that (having a customized 'main' repo, or optimizing tools already present). Having completely separate repos for tools seems cleaner, focusing development on those tools alone (not any of the others present in a branch) and pushes maintenance of the tool back to the tool developer themselves (e.g. there is no need for the galaxy devs to 'pull' in changes at any point into a main repo). This might also allow the galaxy devs to designate a set of 'blessed' or supported tools in various repositories. Re: git: as Peter knows I'm also primarily a git/github user. It is feasible at some future point to allow git/github repos. For instance, one could possibly integrate github usage via this: http://hg-git.github.com/ Of course, I think it's much more important that any additional vcs integration wait until the new tool shed interface stabilizes somewhat, but (at least from the github perspective) seems like it shouldn't be terribly hard to do. chris ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Updating tools on new hg based Galaxy Tool Shed
On Thu, Jun 16, 2011 at 5:40 PM, Chris Fields cjfie...@illinois.edu wrote: I'm of the opposite mind; I'm not sure there is an absolute need to have one's tools be a branch or fork of the main tool shed, though I agree there is also utility in allowing that (having a customized 'main' repo, or optimizing tools already present). To clarify, I have been using branches on an hg fork of the main Galaxy repository up until now, then preparing tarballs for upload to the Galaxy Tool Shed. I could equally have tracked my local changes under git, svn or cvs. The point is under this model, the Tool Shed has no connection to whatever repository (or repositories) the tool author uses on their machine. Barring teething issues like Bug 584/585/586 tool authors could continue to use this development model, where the fact that the Galaxy Tool Shed uses an hg repository internally is an irrelevant internal implementation detail. As far as I know, there are no plans to make the Tool Shed work directly with a full Galaxy hg repo - only mini-repositories, one for each tool or tool suite. Having completely separate repos for tools seems cleaner, focusing development on those tools alone (not any of the others present in a branch) and pushes maintenance of the tool back to the tool developer themselves Yes, and that is the model the new Galaxy Tool Shed is following. Although I'm a little hazy on the details of tool suites in the new model (and there are lots of situations where it makes sense to bundle several tools). (e.g. there is no need for the galaxy devs to 'pull' in changes at any point into a main repo). Well, there still is for general bug fixes, adding new file formats, and merging any tools which they decide to include in the core set. This might also allow the galaxy devs to designate a set of 'blessed' or supported tools in various repositories. Indeed. Re: git: as Peter knows I'm also primarily a git/github user. It is feasible at some future point to allow git/github repos. For instance, one could possibly integrate github usage via this: http://hg-git.github.com/ Of course, I think it's much more important that any additional vcs integration wait until the new tool shed interface stabilizes somewhat, but (at least from the github perspective) seems like it shouldn't be terribly hard to do. True - if there were sufficient demand from Tool Shed contributors. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
