Re: [galaxy-dev] Difficulty dispatching toolshed tool jobs via job_conf.xml
Hi Bjoern, Nate, That's good to hear. Sorry, I have been otherwise engaged for the last few months, and was unable to test this. Hoping to get back onto Galaxy stuff again in a month or so! cheers, Simon -Original Message- From: Nate Coraor [mailto:n...@bx.psu.edu] Sent: Saturday, 15 February 2014 4:37 a.m. To: Björn Grüning Cc: Guest, Simon; galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Difficulty dispatching toolshed tool jobs via job_conf.xml Hi Björn, It should be fixed, the problem was with tools with uppercase characters in the guid: https://bitbucket.org/galaxy/galaxy-central/commits/0b955e54451c/ --nate On Thu, Feb 13, 2014 at 2:38 AM, Björn Grüning bjoern.gruen...@gmail.com wrote: Hi Nate Simon, was that solved? I think it's still not working? Or is that a regression. In particular shed_host/repos/owner/repo/tool_id is not working, and that is what is 90% needed, or? Thanks, Bjoern From: Nate Coraor [mailto:n...@bx.psu.edu] This should work, it looks like there may be a bug with handling tool shed IDs when determining job destinations. You are actually supposed to be able to use any of: shed_host/repos/owner/repo/tool_id/version shed_host/repos/owner/repo/tool_id tool_id I'll take a look at this as soon as possible. One thing you might try in the short term is using the percent encoded tool id in the tool tag in job_conf.xml: Hi Nate, Using the percent encoded form was the first thing I tried (before emailing the list). It didn't make any difference. I think there's another reason it's not picking it up. Thanks for looking into this. cheers, Simon === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Toolshed reset all repository metadata
At Thu, 24 Oct 2013 10:03:14 -0400, Greg Von Kuster wrote: Hello Simon, On Oct 23, 2013, at 5:01 PM, Simon Guest simon.gu...@agresearch.co.nz wrote: We've noticed a funny about resetting repository metadata from the toolshed web interface. My understanding is that this is used to get the toolshed back in sync with the underlying mercurial repo, for example after pushing updates from hg on the command line. This is not quite correct. When committing a new changeset to a repository (either via hg push from the command line or by using any feature in the Tool Shed's upload component), metadata is reset on the repository automatically, but only back to the previous installable changeset revision if one exists. So for those repositories that have multiple changeset revisions, not all of the repository changelog is inspected with a new commit. The Reset all repository metadata feature, however, does inspect the complete repository changelog and resets metadata from revision 0 through the repository tip. In addition to being useful for the current repository itself, it can also be useful for all other repositories that have a dependency relationship to it. See http://wiki.galaxyproject.org/RepositoryRevisions#Resetting_metadata_on_your_repositories. In general, it is not necessary to use this feature. Hi Greg, Thanks for the explanation, and the link to that helpful Wiki page. And also that code update 4:71c35dbde130 which will help our tool test/development cycle. I have a remaining question, though, about the recommended working practices for developing a toolshed tool. I didn't find this described on the Wiki. There is obviously a cycle of development to go through, making updates, installing to a test galaxy instance, etc. It seems to us that the only sensible way to handle this is within a local test toolshed, essentially using the repo in the test toolshed as a development repo (despite the warnings not use the toolshed as a source code repo on the Wiki!). During such an iterative process, we don't want to keep bumping the tool version. Therefore it seems necessary to reset the repository meta data after every push, in order to install the just-updated-for-testing tool. Once the tool is working, the files can be uploaded into a clean repo on a production toolshed, so we don't keep all the intermediate development versions. Or is there a better way to do this? cheers, Simon ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Configurable toolshed package installation to support tool-dependency-resolver-plugins
At Mon, 14 Oct 2013 20:22:06 -0500, John Chilton wrote: Simon, Very cool! I have two concerns. Rather than adding a new configuration option I think I would prefer to just check the configured dependency resolvers and then infer from them if the tool shed will be used. The configuration option strikes me as having to configure the same thing twice, and this change would make your setup slightly easier. Do you have any objection to me reworking your patch to do this? On the other hand, perhaps it is made more clear to the deployer that they are definitely disabling tool dependency installations if they have to add the explicit option this way. Hi John, I have no problem with you reworking it in that way. There are two reasons I didn't do that myself: 1. I would have had to change the interface to the dependency resolvers somehow to support this query, and I wasn't sure that was a good thing. 2. I wanted to make it explicit that toolshed package installation was disabled in this case, as I thought that would make it more likely this change gets accepted into the mainline. Whichever way you Greg and Dave are happy with is OK by me. Actually, I like your implicit approach better, so hope that's the one that gets agreed. cheers, Simon ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Configurable toolshed package installation to support tool-dependency-resolver-plugins
At Fri, 4 Oct 2013 23:12:01 -0500, John Chilton wrote: I understand the desire to not want to try to execute the tool shed actions if tool_shed_packages are not specified in the dependency resolvers list. I have created a Trello card for it (https://trello.com/c/CPeU3VlR). It sounds like the new status quo will be functional it will just be kind of annoying to have those actions try and fail and then marked as errors, right? If that is right then for this reason I don't think implementing this behavior will be a high priority for the team, but if you send another brilliant patch or pull request I will be happy to test it and merge it. Hi John, I have now implemented this. Mercurial changeset attached. This adds another configuration item for universe_wsgi.ini as documented in the updated universe_wsgi.ini.sample: # This option may be used to disable installation of package # dependencies from tool sheds, which usually means downloading a # source tarball and compiling it locally. You would disable this if # you want to make use of system installed packages for example. In # that case, alternative tool dependency resolvers should be # configured in dependency_resolvers_conf.xml #enable_tool_shed_package_dependency_installation = True The default behaviour is per standard toolshed. If configured to False, then the package components of tool dependencies will never be installed from the toolshed. Rather, they will rely on the tool dependency manager to resolve the requirement. This is integrated with the missing_tool_dependencies status, so that if dependency resolution fails, the repository gets flagged with missing tool dependencies on the installed tool shed repositories page, and the paster.log file contains a warning of exactly what wasn't found. Of course, if the dependency can be satisfied, then it's green lights all the way, and everything should work fine. I've tested this interactively by installing the standard emboss_5 repository, with various combinations of environment module available and not. It looks to work fine. I'm afraid I haven't yet read up on the Galaxy unit testing framework, so no unit tests for now. What do you think? cheers, Simon configurable-toolshed-package-installation.patch Description: Binary data ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Unicode in tool stderr crashing galaxy? - solved, patch attached
At Thu, 10 Oct 2013 00:31:56 -0500, John Chilton wrote: Simon, using your test tools I was able to reproduce your errors. Pull request 233 is my attempt to fix them with a lighter touch than just immediately converting outputs to ascii. Do these patches work on your end? Bjoern, do these patches fix your problem as well? If I don't hear anything from anyone, I will merge these changes in a few days. Hi John, That works nicely, and is better than what I did, thanks! cheers, Simon ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Deploying LOC files for tool built-in data during a tool installation
I look forward to some more details from Dan on *.loc file setup. Hi Peter, Dan and all, What a timely discussion! I am just in the process of setting up loc files for some new indexes I have created (bowtie2, etc), and would really like to see this automated. I see there is a Galaxy script scripts/loc_files/create_all_fasta_loc.py, which is quite sophisticated, and does this job nicely for all_fasta.loc. I'm feeling an urge to somehow extend this script to cope with other datatypes besides fasta, but am wondering if this will be wasted effort if there will soon be a better way to handle this. Can Dan or anyone else comment on this? cheers, Simon ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Migrating database from development back to stable?
Dear Galaxy developers, Currently our production Galaxy runs the stable branch. However, I see one or two reasons to want to run the development (default) branch for a while. This will involved migrating the database to a later version, which is fine, as migration scripts exists. What I am wondering is whether it is guaranteed that I will be able to get back onto stable at a later date, i.e. there will be a database migration path from whatever development database version I end up on, onto a future stable database version. Can anyone comment? cheers, Simon ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Test post, please ignore
(Just testing that my Exchange Server admin guy has managed to disable the stupid legal disclaimers on my postings to this list.) ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Test Toolshed Biopython package dependency Atlas fails to install (Was: Re: UnboundLocalError: local variable 'prior_installation_required' referenced before assignment)
At Fri, 27 Sep 2013 00:23:37 -0500, John Chilton wrote: Simon, What is the advantage of putting that XML definition in the tool shed? It is not 100% true because of prior_install_required dependencies, but for the most part sourcing/load the environment for tools is a Galaxy problem, not so much a tool shed one. What if we did this instead? Add an option to Galaxy's universe_wsgi.ini with the following default: tool_dependency_resolution_order = gx_package_manual, gx_package_toolshed Which essentially implements my idea above, with James' additional configuration. But which can be overridden as: tool_dependency_resolution_order = plugin_module, gx_package_manual, gx_package_toolshed If set this way then placing requirement package=0.5.9bwa/requirement in a tool will result in the module bwa/0.5.9 being loaded if it is 'avail'able, else it will check for a manually installed env.sh (which is where MSI is currently putting its module loads), and else it will fallback to source the tool shed installed dependency. I feel like this will give you everything you want without any extra XML or configuration. Let me know if I am wrong. Hi John, I think you're right. Your scheme is neater than what I was proposing. The extra flexibility I was aiming at via some toolshed XML stuff appears not to be necessary upon further reflection. (I wanted to ensure a Galaxy admin could just install some RPMs, install a toolshed tool, and have everything resolve nicely. You seem to have achieved that with your scheme.) I haven't had a chance to try your code yet, but as soon as I can I will do so, and get back to you. If I leave out gx_package_toolshed altogether from tool_dependency_resolution_order, will the tool installation in Galaxy simply fail with a nice error message if the environment module and/or env.sh files are not found? (This is what I would like, as it would serve as a prompt to the Galaxy admin to install some extra RPMs or whatever.) Will this also work for those toolshed packages which bundle their package definitions (to download, make and install the tool dependency) along with their wrappers? Thanks for working on this. cheers, Simon === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Test Toolshed Biopython package dependency Atlas fails to install (Was: Re: UnboundLocalError: local variable 'prior_installation_required' referenced before assignment)
At Thu, 26 Sep 2013 22:03:09 -0400, Greg Von Kuster wrote: Hi John, On Sep 26, 2013, at 9:15 PM, John Chilton chil...@msi.umn.edu wrote: I was not even thinking we needed to modify the tool shed to implement this. I was hoping (?) you could just modify: Nothing in the Tool Shed itself would be affected or require modification for this new feature as this is completely on the Galaxy side. lib/galaxy/tools/deps/__init__.py to implement this. If some tool contains the tag requirement type=package version=1.7.1numpy/requirement then if there is a manually installed tool_dependency in `tool_dependency_dir/numpy/1.7.1/env.sh` that would take precedence over the tool shed installed version (would that be something like `tool_dependency_dir/numpy/1.7.1/owner/name/changeset/env.sh`)? Let me know if this is way off base. This is a possibility perhaps, but there seems to be a potential weakness in that it doesn't require the Tool Dependency object to exist since the tool will function without a installed dependency from the Tool Shed. Or, if the installed depndency s required, then it is meaningless because it won't be used. If the former case, then the tool dependency cannot be shared via the Tool Shed's dependency mechanism because none of the relationships will be defined since nothing is installed. Wouldn't it be better to allow the Galaxy admin to point the ToolDependency object to a specified binary on disk? In this way, all relationships defined by Tool Shed installations will work as expected, with all contained tools with that dependency point to that same shared location on disk. Hi Greg, John, What you are discussing is pretty close to what I am after, and what I am prepared to spend some time working on, if that can help. This is what I have posted about a couple of times previously. The option to use a pre-installed package rather than a Galaxy installed one I think would be very useful in general. I think it can be done in a way that doesn't break the tool dependency model. I envisage providing access to existing programs on disk by loading the relevant environment module. Platforms will differ greatly on where they have programs installed (usr/local/bwa-0.5.9, /usr/libexec/bwa-0.5.9, etc.). However, we could arrange to have a conventionally named environment module available, so Galaxy just has to be told somehow to do a module load bwa/0.5.9, say, prior to trying to run that tool, which will then be found on the PATH. Hooking this in without killing the dependency on named and versioned toolshed repos could work like this. In parallel with the repo sets named, e.g. package_bwa_0_5_9, which download and build the package, we have a set named like native_package_bwa_0_5_9. This could have a tool_dependencies.xml file like this: ?xml version=1.0? tool_dependency native_package name=bwa version=0.5.9 actions action type=load_module module name=bwa version=0.5.9/module /action /actions readme Uses native BWA via environment module bwa/0.5.9 /readme /native_package /tool_dependency What this does is ensure that any repo which requires version 0.5.9 of bwa can have this dependency resolved by either package_bwa_0_5_9, or native_package_bwa_0_5_9. I envisage a Galaxy configuration setting that enables native package and/or Galaxy package dependency resolution, and which one should be preferred. Of course, if one of these has been manually installed, it will be used as is. I would really like to be able to install tools from the toolshed, and have them resolve their dependencies automatically using my preinstalled application suite. I see that would also meet the needs being discussed here. How does that sound? cheers, Simon === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Unicode in tool stderr crashing galaxy? - solved, patch attached
At Thu, 19 Sep 2013 11:44:35 +0200,
Bjoern Gruening wrote:
Hi Simon,
it's just a guess but can you try the attached patch if you have a
reproducible case. It is not tested, but at least it should point you in
the right direction. If its working, I will create a pull request.
Hi Bjoern,
Thanks for your reply and your patch. (Sorry for the delay in responding, I
have been busy with other stuff.)
While your patch didn't solve my problem, it certainly pointed me in the right
direction.
I made a new patch based on your idea, which I attach. The idea is, stdout and
stderr of each tool run is converted from possible UTF-8 to ASCII, using a new
function asciify which I added to Galaxy util.
I made a repository on the Galaxy test toolshed called unicode_tester, to test
this. This has 2 tools which generates some Unicode output, either on stdout
or stderr accordingly. Log output shows that unicode characters are being
removed.
What else should I do to contribute this patch into Galaxy core?
cheers,
Simon
My optimism was short lived. I think I've hit a galaxy bug in Unicode
processing.
Having managed to get the deseq tool not to crash galaxy in the sqlalchemy
layer,
I now get a crash when clicking on the info-stderr for the tool output
dataset, probably because it contains Unicode. Here's the traceback from
paster.log:
Exception happened during processing of request from ('127.0.0.1', 47124)
Traceback (most recent call last):
File
/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py,
line 1068, in process_request_in_thread
self.finish_request(request, client_address)
File /usr/lib64/python2.6/SocketServer.py, line 322, in finish_request
self.RequestHandlerClass(request, client_address, self)
File /usr/lib64/python2.6/SocketServer.py, line 617, in __init__
self.handle()
File
/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py,
line 442, in handle
BaseHTTPRequestHandler.handle(self)
File /usr/lib64/python2.6/BaseHTTPServer.py, line 329, in handle
self.handle_one_request()
File
/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py,
line 437, in handle_one_request
self.wsgi_execute()
File
/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py,
line 290, in wsgi_execute
self.wsgi_write_chunk(chunk)
File
/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py,
line 150, in wsgi_write_chunk
self.wfile.write(chunk)
File /usr/lib64/python2.6/socket.py, line 316, in write
data = str(data) # XXX Should really reject non-string non-buffers
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2018' in
position 169: ordinal not in range(128)
I don't know where to look to find the actual stderr output. Any pointers?
Is this actually a galaxy bug now?
cheers,
Simon
===
Attention: The information contained in this message and/or attachments
from AgResearch Limited is intended only for the persons or entities
to which it is addressed and may contain confidential and/or privileged
material. Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by persons or
entities other than the intended recipients is prohibited by AgResearch
Limited. If you have received this message in error, please notify the
sender immediately.
===
unicode-fix.patch
Description: Binary data
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Patch for deseq_and_sam2counts Galaxy wrapper
Hi Nikhil, I've been using your Galaxy wrapper for deseq_and_sam2counts. I fixed it up to run the R script from the tool install directory directly, so no manual copying of the script is required after installation in the Galaxy GUI. I attach my patch (as a Mercurial export), and hope you will incorporate this into the official repository, to avoid me having to maintain my own version. If it's more convenient for you to receive the update in a different form, please say. Thanks for making the tool wrapper available. cheers, Simon === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === deseq.patch Description: Binary data ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Mercurial updates not working - caused by http_proxy?
Well, having just hit this again myself, I found what was *actually* causing the problem, rather than my vague recollection of what caused it, for which, apologies. In my case just now, it was caused by having an HTTP proxy set via the environment variable http_proxy. Unsetting that fixed my problem. Pesky web proxy, bah! You should never trust random strangers on the net, by the way. ;-) cheers, Simon -Original Message- From: Guest, Simon Sent: Thursday, 5 September 2013 1:38 p.m. To: Alistair Chilcott Cc: galaxy-dev@lists.bx.psu.edu Subject: RE: Mercurial updates not working Hi Alistair, I have seen error output like this from mercurial when the client is running a newer version than the server. Maybe you recently upgraded your machine, and your mercurial version is now too recent. Hope that helps. Of course, it may be a complete red herring, YMMV. ;-) cheers, Simon -Original Message- From: galaxy-dev-boun...@lists.bx.psu.edu [mailto:galaxy-dev- boun...@lists.bx.psu.edu] On Behalf Of Alistair Chilcott Sent: Wednesday, 28 August 2013 12:33 p.m. To: galaxy-dev@lists.bx.psu.edu Subject: [galaxy-dev] Mercurial updates not working Hello all, Mercurial has recently started throwing errors: galaxy@hitmsbhpc1:~/galaxy-dist hg incoming abort: error: _ssl.c:517: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol It has previously worked with issues has anyone seen this before? I don't think anything has changed that would have caused this problem but googling error has proved to be unhelpful so far. Regards, Alistair ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] UnicodeDecodeError - suggested note regarding UTF8 encoding in postgres databases
I've finally joined the happy throng suffering from Unicode errors. In my particular case, running the deseq tool from the main toolshed brought down my Galaxy server, and it wouldn't restart. The traceback in paster.log contained this Traceback (most recent call last): [snip] File /home/galaxy-dev/galaxy/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py, line 2903, in __init__ self._init_metadata() File /home/galaxy-dev/galaxy/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py, line 3306, in _init_metadata self.__buffer_rows() File /home/galaxy-dev/galaxy/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py, line 3326, in __buffer_rows self.__rowbuffer = collections.deque(self.cursor.fetchmany(size)) UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 162: ordinal not in range(128) The same error occurred each time I tried to restart the galaxy server until I restored the galaxy database from last night's postgres backup. The problem was my galaxy database in postgres was using SQL_ASCII encoding, which turns out to be a bad idea if you're ever going to write Unicode characters there. (I'm quite disappointed that this is the default encoding for postgres databases on CentOS 6.) The fix was to dump the galaxy database, recreate it with UTF8 encoding, and reload it. Now everything works, and that one tool doesn't crash my galaxy server. I thought it would be worth noting that postgres databases should be created with UTF8 encoding on this page: http://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer#Switching_to_a_database_server I think the right thing to do is to ensure you set the default database encoding for postgres when you run initdb. You only get one chance to get this right. After that, a way to change the default database encoding to UTF8 is given here: https://gist.github.com/ffmike/877447 Scary stuff, but it worked well enough for me. Alternatively, if you didn't think about encodings when you ran initdb, you can think about it every time you run createdb. Hope this helps someone. cheers, Simon === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Towards Galaxy Linux (not) ? [was [RFC] Storing of tarballs and patches for tool_dependencies to enable reproducibility]
Hi Bjoern, I can see man years of effort being spent on solving this problem within Galaxy. I was going to title this email Danger, Will Robinson, but I didn't want to be disrespectful. I think the path being embarked upon, tool dependency packaging, tool versioning, reproducibility, and long term archive of source tarballs is going to lead inevitably to creation of a new Linux distribution, which I guess will be called Galaxy Linux. The packaging and archival you are talking about is exactly the service provided by a Linux distribution. There's well established infrastructure to handle this, and years of experience have gone into solving the problems well. Surely the number of Linux distributions in the world now exceeds 100, but I don't see that the world will become a better place if we increase that number by one more. We at AgResearch can't be alone in having to pick a Linux distribution to run from the short list supported by our hardware vendor. I can't see Galaxy Linux being on that list anytime soon. So we have to make Galaxy run on the particular distribution we have here. For us that's CentOS 6. Now, I see scary mention of platform independence as a goal for Galaxy packaging, which I interpret as will run on any Linux distribution. I think that's essentially infeasible. All you can do is write install scripts which you hope are portable (by following as many best practices as you know about), and then work patiently with users on strange platforms, to adapt each install script to work on that platform also. I think this is not a good use of anyone's time. How many Linux distributions do the Galaxy community actually care about today? The RHEL family is surely important, as is Ubuntu LTS. Anything else? I'd be quite interested to understand this, as it provides a context for the discussion, and ensures we're not just solving a hypothetical problem. I'm just starting work on a native packaging infrastructure for Galaxy, that will enable tool dependencies to use defined versions of natively installed packages. That frees me up to make my packages work nicely on the RHEL family. It looks like the RPMs themselves (including SRPMs obviously) will be hosted by the CentOS project before too long. Once they're there, they can easily be archived forever. Anyone else on that platform is welcome to use the same infrastructure. Then, all we really need is someone to handle the packaging effort for the other major Linux distributions (a small number, I hope), and the problem is essentially solved. Getting the Bio-Linux team interested in multi-version packaging would be a great next step. I'll be posting here when I have progress to report on my native packaging effort. cheers, Simon === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Resolved toolshed mercurial authentication issue, public name change issue remains
Hi Greg,
I finally got to the bottom of the reasons for my abort: authorization failed
message when trying to push tool updates into my local toolshed using hg
command line (rather than web interface), having given up for a while, and just
come back to it.
Since my authentication agent (apache LDAP module using corporate Active
Directory) was providing bare usernames to the Toolshed, I had been relying on
the remote_user_maildomain feature to turn this into an email address.
However, it seems this function is not triggered when using hg command line
access, so I had to change my rewrite rule which sets up the apache REMOTE_USER
in my apache conf file for the toolshed, from this:
RequestHeader set REMOTE_USER %{RU}e
to this:
RequestHeader set REMOTE_USER %{RU}e...@agresearch.co.nz
Now I can push updates directly from hg, and it's much more convenient. I
didn't have to change anything in that toolshed middleware (although some
temporary log messages I added there were the key for me solving the problem).
However, that bug remains where I can continue to change my public name in the
web user interface, even after I have created repos. I know not to do this.
If there's more information you want from me to try to track down what is going
wrong here, just say what you need, and I will endeavour to provide it. But
for me, this is now a minor issue, so don't do anything just on my account.
Thanks for your help.
cheers,
Simon
-Original Message-
From: Greg Von Kuster [mailto:g...@bx.psu.edu]
Sent: Friday, 6 September 2013 1:00 p.m.
To: Guest, Simon
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] Bug in toolshed: changing public name breaks
repo path
Hello Simon,
The tool shed middleware basically consists of 2 files in
~/lib/galaxy/webapps/tool_shed/framework/middleware: hg.py which handles
authentication when pushing from the command line. The remoteuser.py file
handles remote user authentication, but I cannot guarantee it properly
handles hg push from the command line since I don't have an environment
set up that way. That would be the place to take a look though, and if
you discover problems let me know and we'll apply a patch.
Thanks very much,
Greg Von Kuster
On Sep 5, 2013, at 4:53 PM, Guest, Simon simon.gu...@agresearch.co.nz
wrote:
Hi Greg,
I just changed the public name on the web interface, where you click on
the User dropdown, and choose Public Name. Something about my environment
let it happen, when it apparently should have not been possible.
I suspect the reason lies in some mismatch between my apache user
authentication and toolshed userids. See my previous email for the
details of what I've been doing with that. I don't understand the
connection between userids (email addresses) used for authentication, and
public names, which appear to be used by the mercurial server inside the
toolshed (by inference from the public name appearing in the allow_push in
the toolshed repo). Is there a layer of toolshed middleware that maps
between the two?
I am going to investigate other ways of doing the authentication, as I
don't think passing in REMOTE_USER from apache is working (for me). I
would like to know how others are doing that.
cheers,
Simon
-Original Message-
From: Greg Von Kuster [mailto:g...@bx.psu.edu]
Sent: Friday, 6 September 2013 1:17 a.m.
To: Guest, Simon
Cc: Dave Bouvier; galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] Bug in toolshed: changing public name
breaks repo path
Hello Simon,
On Sep 4, 2013, at 5:13 PM, Guest, Simon
simon.gu...@agresearch.co.nz
wrote:
Hi Dave, Greg,
Thanks for your reply.
I'm running a recently checked out stable branch. hg log shows this
tip:
changeset: 10473:c42567f43aa7
tag: tip
user:greg
date:Mon Aug 19 13:19:56 2013 -0400
summary: Filter invalid objects when generating the list of
repository_dependencies objects that are associated with a tool shed
repository installed into Galaxy.
I wonder if this problem is related to my more fundamental problem
of
being unable to get mercurial push authentication working properly.
I care much more about that, as that's blocking me (whereas the
problem I reported has an easy work-around).
Even though you've found a work-around to the problem that results
when you change your public user name in the tool shed after you have
created a repository that uses it, it is extremely important that we
get a fix ffor the issue committed to the tool shed code. Can you
let us know how you went about changing your public user name? The
only way we've been able to do it is by manually updaing the database
table record using a sql statement. If you have discovered another
way to do it, please let us know the steps we can take to reproduce
it so that we can get a fix as soon
[galaxy-dev] A proposed modules extension for toolshed wrappers
Another vote for the excellent modules system from us at AgResearch Galaxy's dependency injection works in a similar way, without requiring all the external dependencies. We don't need unload since we compose a unique environment for every tool execution. This is much cleaner that loading everything into the environment before running Galaxy, we want tools to be executed with as clean an environment as possible. Setting the environment for each tool just prior to running the tool is clearly better. What I'm doing currently is loading an environment module just before running a tool in Galaxy. So far, I have done this in the tool XML file by prefixing the command with a module load command, e.g. command. /etc/profile; module load EMBOSS/5.0.0; infoseq -sequence [snipped] /command The ugly . /etc/profile is needed for technical reasons. I don't like this, as it's a bit hacky, but what it achieves is useful and flexible. I envisage an extension to the tool XML syntax which would let me write it like this, say: requirementsrequirement type=module version=5.0.0EMBOSS/requirement/requirements commandinfoseq -sequence [snipped] /command This will enable my native EMBOSS wrapper to run the defined version of the (system-installed) EMBOSS suite. However, it would only be worth me writing this extension if there's a likelihood of getting it into Galaxy. Is there? Here's where I'm coming from. I already have most of the applications I want installed on my system. I am working on updating the RPMs to support side-by-side installs of multiple versions, with version selection by the environment modules facility. This is a small change (and improvement) on what I have today. It's a work in progress, but it's a manageable amount of work. It's looking likely that this multi-version packaging approach may be adopted by an official CentOS Scientific Repo, and this could change the landscape in terms of packaging of scientific applications for major Linux distributions. Who knows, that's all in the future. In the meantime, there's work going on in Galaxy land to package every application wanted in Galaxy, by writing install scripts, encapsulating environment variables in tool XML files, etc. (There is difficulty here in writing install scripts which work on every platform, which I have commented on previously.) The big question is, will the Galaxy team accept contributions to Galaxy to support different ways of solving this packaging problem? In particular, allowing for environment modules to be loaded as part of running a tool, and having this functionality in the toolshed? cheers, Simon === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] A proposed modules extension for toolshed wrappers
Just been reading a bit more about the Galaxy packaging system. Here's a slight modification to what I was suggesting that might fit in a bit better. Apologies for not being more familiar with the existing system before proposing extensions. Recall that my goal is to support using a system-installed (native) package, at a defined version, which I aim to achieve by loading the appropriate environment module before running a tool. We still have tool_dependencies.xml defining a package at a particular version, but rather than download and build the source code, there's just a directive that says how to pick up the correct program version at runtime, e.g. which environment module to load. So instead of the tool_dependencies.xml fragment: tool_dependency package name=bwa version=0.6.2 install version=1.0 actions action type=download_by_urlhttp://downloads.sourceforge.net/project/bio-bwa/bwa-0.6.2.tar.bz2/action action type=shell_commandmake/action action type=move_file sourcebwa/source destination$INSTALL_DIR/bin/destination /action action type=set_environment environment_variable name=PATH action=prepend_to$INSTALL_DIR/bin/environment_variable /action /actions /install /package /tool_dependency We have something like this (NB: element and attribute names are for illustrative purposes only): tool_dependency package name=bwa version=0.6.2 use_native actions action type=module_loadbwa/0.6.2/action /actions /use_native /package /tool_dependency This causes the right thing (module load bwa/0.6.2) to be stuck into the dependencies env.sh file when this package is installed from the toolshed. We could call this toolshed tool native_package_bwa_0_6_2, to avoid confusion with the existing download-and-make one. We might want a bit of flexibility on what actions are supported (in case we want to support Software Collections, for example). What do you think? cheers, Simon PS: In case it wasn't already clear, solving this problem well is quite important to us here at AgResearch. ;-) === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] ZFS storage recommendations
Hi Joachim, At AgResearch we are using ZFS for our HPC storage, which is used by our internal Galaxy instance. Currently we are running on FreeNAS (FreeBSD derivative), but we are in transition to ZFS on Linux. We export the filesystem over NFS (10Gb Ethernet), but not the database (PostgreSQL). In general, you want block storage for a database, so I suggest you look for a solution other than NFS to host that. Our experience with ZFS has been very positive. However, FreeNAS is not really suited to our needs - it's more of a storage appliance, probably great for a home NAS. Hence the planned transition. I strongly recommend you follow the discussion on the ZFS discuss mailing list. There's a lot to learn about ZFS configuration, much more than you will glean from few posts here. http://zfsonlinux.org/lists.html cheers, Simon -Original Message- From: galaxy-dev-boun...@lists.bx.psu.edu [mailto:galaxy-dev- boun...@lists.bx.psu.edu] On Behalf Of Joachim Jacob | VIB | Sent: Tuesday, 10 September 2013 10:29 p.m. To: galaxy-dev@lists.bx.psu.edu Subject: [galaxy-dev] ZFS storage recommendations Hi all, I am performing some tests to move my galaxy database to ZFS. Does anybody have experience with ZFS on linux, and some recommendations/experiences to optimize performance? The purpose is to share the database over NFS to the Galaxy VM. Thanks, Joachim. -- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Difficulty dispatching toolshed tool jobs via job_conf.xml
From: Nate Coraor [mailto:n...@bx.psu.edu] This should work, it looks like there may be a bug with handling tool shed IDs when determining job destinations. You are actually supposed to be able to use any of: shed_host/repos/owner/repo/tool_id/version shed_host/repos/owner/repo/tool_id tool_id I'll take a look at this as soon as possible. One thing you might try in the short term is using the percent encoded tool id in the tool tag in job_conf.xml: Hi Nate, Using the percent encoded form was the first thing I tried (before emailing the list). It didn't make any difference. I think there's another reason it's not picking it up. Thanks for looking into this. cheers, Simon === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Difficulty dispatching toolshed tool jobs via job_conf.xml
I am having trouble getting a toolshed tool to be dispatched to the destination
I list in the job_conf.xml file.
My job_conf.xml file looks like this:
?xml version=1.0?
job_conf
plugins
plugin id=local type=runner
load=galaxy.jobs.runners.local:LocalJobRunner workers=20/
plugin id=condor type=runner
load=galaxy.jobs.runners.condor:CondorJobRunner /
/plugins
handlers
handler id=main/
/handlers
destinations default=condor
destination id=local runner=local/
destination id=condor runner=condor
!-- With no params, jobs are submitted to the 'vanilla' universe
with:
notification = NEVER
getenv = true
Additional/override query ClassAd params can be specified with
param tags, e.g
param id=request_cpus8/param
--
/destination
/destinations
tools
tool id=upload1 destination=local/
!-- Upload File --
tool id=ucsc_table_direct1 destination=local/
!-- UCSC Main --
... stuff omitted ...
tool
id=toolshed-dev.agresearch.co.nz/toolshed/repos/guestsi/emboss_5_native/EMBOSS:
infoseq46/5.0.0destination=local/
/tools
/job_conf
So, you can see I have a default destination of condor, but I'm trying to run
my toolshed EMBOSS infoseq tool on local. However, it is stubbornly running on
condor.
In lib/galaxy/tools/__init__.py:1132, I see this comment which got me wondering:
# In the toolshed context, there is no job config.
Is it possible to define tool destinations for toolshed tools? Are there some
gotchas that I should know about? Any other ideas why my job is ignoring the
config in job_conf.xml? (By the way, I can change say the upload1 tool to run
on Condor by setting its destination in that file, so it is doing something.)
The other thing I saw in the source code is stuff about old_id and toolshed
guids. Do I need to understand this stuff?
The paster.log contains the following when I submit the infoseq job:
147.158.130.216 - - [11/Sep/2013:13:53:34 +1300] GET
/tool_runner?tool_id=toolshed-dev.agresearch.co.nz/toolshed/repos/guestsi/emboss_5_native/EMBOSS%3A%20infoseq46/5.0.0
HTTP/1.1 200 - http://galaxy-dev.agresearch.co.nz/; Mozilla/5.0 (X11; Linux
x86_64; rv:23.0) Gecko/20100101 Firefox/23.0
147.158.130.216 - - [11/Sep/2013:13:53:40 +1300] POST /tool_runner/index
HTTP/1.1 200 -
http://galaxy-dev.agresearch.co.nz/tool_runner?tool_id=toolshed-dev.agresearch.co.nz/toolshed/repos/guestsi/emboss_5_native/EMBOSS%3A%20infoseq46/5.0.0;
Mozilla/5.0 (X11; Linux x86_64; rv:23.0) Gecko/20100101 Firefox/23.0
galaxy.jobs DEBUG 2013-09-11 13:53:40,886 (92) Working directory for job is:
/home/galaxy-dev/galaxy/database/job_working_directory/000/92
galaxy.tools DEBUG 2013-09-11 13:53:40,886 Tool::get_job_destination:
{'runner': 'condor', 'legacy': False, 'params': {}, 'tags': None, 'url': None,
'converted': False, 'id': 'condor'}.
galaxy.jobs.handler DEBUG 2013-09-11 13:53:40,894 (92) Dispatching to condor
runner
(I added the debug output for Tool::get_job_destination to see what was going
on.)
Any ideas?
cheers,
Simon
===
Attention: The information contained in this message and/or attachments
from AgResearch Limited is intended only for the persons or entities
to which it is addressed and may contain confidential and/or privileged
material. Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by persons or
entities other than the intended recipients is prohibited by AgResearch
Limited. If you have received this message in error, please notify the
sender immediately.
===
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Bug in toolshed: changing public name breaks repo path
Hi Greg, I just changed the public name on the web interface, where you click on the User dropdown, and choose Public Name. Something about my environment let it happen, when it apparently should have not been possible. I suspect the reason lies in some mismatch between my apache user authentication and toolshed userids. See my previous email for the details of what I've been doing with that. I don't understand the connection between userids (email addresses) used for authentication, and public names, which appear to be used by the mercurial server inside the toolshed (by inference from the public name appearing in the allow_push in the toolshed repo). Is there a layer of toolshed middleware that maps between the two? I am going to investigate other ways of doing the authentication, as I don't think passing in REMOTE_USER from apache is working (for me). I would like to know how others are doing that. cheers, Simon -Original Message- From: Greg Von Kuster [mailto:g...@bx.psu.edu] Sent: Friday, 6 September 2013 1:17 a.m. To: Guest, Simon Cc: Dave Bouvier; galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Bug in toolshed: changing public name breaks repo path Hello Simon, On Sep 4, 2013, at 5:13 PM, Guest, Simon simon.gu...@agresearch.co.nz wrote: Hi Dave, Greg, Thanks for your reply. I'm running a recently checked out stable branch. hg log shows this tip: changeset: 10473:c42567f43aa7 tag: tip user:greg date:Mon Aug 19 13:19:56 2013 -0400 summary: Filter invalid objects when generating the list of repository_dependencies objects that are associated with a tool shed repository installed into Galaxy. I wonder if this problem is related to my more fundamental problem of being unable to get mercurial push authentication working properly. I care much more about that, as that's blocking me (whereas the problem I reported has an easy work-around). Even though you've found a work-around to the problem that results when you change your public user name in the tool shed after you have created a repository that uses it, it is extremely important that we get a fix ffor the issue committed to the tool shed code. Can you let us know how you went about changing your public user name? The only way we've been able to do it is by manually updaing the database table record using a sql statement. If you have discovered another way to do it, please let us know the steps we can take to reproduce it so that we can get a fix as soon as possible. If this is, in fact, a bug, then it is a very critical one. Thanks, Greg Von Kuster Here is more information about my environment. I'm using apache for digest authentication, with the htdigest file containing email addresses for usernames (so it matches what the toolshed wants). It passes it through to the toolshed using REMOTE_USER. (We have something like this working well for our main Galaxy instance.) I can access the toolshed just fine via the web interface, and I can do e.g. hg outgoing on the command line against a toolshed repo to see what needs pushing. But when I try hg push, I get abort: authorization failed. Looking inside the hgrc file inside my toolshed repo, I see it has allow_push = simon-guest (i.e. my public name, not my email address). This isn't going to match the REMOTE_USER passed in from apache, which is an email address. So I think to get this working with my authentication framework, public name has to match the email address. (I tried hacking the allow_push in hgrc to be the email address, but that didn't work.) Is anyone else using a toolshed behind apache with authentication handled by apache, like this? Should I expect hg push to work in this configuration, or am I outside the bounds of what has been tested so far? Thanks for your help, much appreciated. cheers, Simon -Original Message- From: Dave Bouvier [mailto:d...@bx.psu.edu] Sent: Thursday, 5 September 2013 1:07 a.m. To: Greg Von Kuster Cc: Guest, Simon; galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Bug in toolshed: changing public name breaks repo path Simon, I am unable to reproduce this behavior on my local tool shed installation, which revision are you running? --Dave B. On 09/04/2013 06:57 AM, Greg Von Kuster wrote: Hello Simin, Bjoern is correct - if this is possible (which it looks like it is from your traceback), then a bug has been introduced somewhat recently. We'll take a look at this today and get a fix committed. Thanks for reporting this. Greg Von Kuster On Sep 4, 2013, at 3:17 AM, Bjoern Gruening bjoern.gruen...@gmail.com wrote: Hi Simon, you are right. Public usernames, or usernames associated with a repository can't be changed. If you can that is a bug, please fill a trello card with it. The reason
Re: [galaxy-dev] Bug in toolshed: changing public name breaks repo path
Hi Dave, Greg, Thanks for your reply. I'm running a recently checked out stable branch. hg log shows this tip: changeset: 10473:c42567f43aa7 tag: tip user:greg date:Mon Aug 19 13:19:56 2013 -0400 summary: Filter invalid objects when generating the list of repository_dependencies objects that are associated with a tool shed repository installed into Galaxy. I wonder if this problem is related to my more fundamental problem of being unable to get mercurial push authentication working properly. I care much more about that, as that's blocking me (whereas the problem I reported has an easy work-around). Here is more information about my environment. I'm using apache for digest authentication, with the htdigest file containing email addresses for usernames (so it matches what the toolshed wants). It passes it through to the toolshed using REMOTE_USER. (We have something like this working well for our main Galaxy instance.) I can access the toolshed just fine via the web interface, and I can do e.g. hg outgoing on the command line against a toolshed repo to see what needs pushing. But when I try hg push, I get abort: authorization failed. Looking inside the hgrc file inside my toolshed repo, I see it has allow_push = simon-guest (i.e. my public name, not my email address). This isn't going to match the REMOTE_USER passed in from apache, which is an email address. So I think to get this working with my authentication framework, public name has to match the email address. (I tried hacking the allow_push in hgrc to be the email address, but that didn't work.) Is anyone else using a toolshed behind apache with authentication handled by apache, like this? Should I expect hg push to work in this configuration, or am I outside the bounds of what has been tested so far? Thanks for your help, much appreciated. cheers, Simon -Original Message- From: Dave Bouvier [mailto:d...@bx.psu.edu] Sent: Thursday, 5 September 2013 1:07 a.m. To: Greg Von Kuster Cc: Guest, Simon; galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Bug in toolshed: changing public name breaks repo path Simon, I am unable to reproduce this behavior on my local tool shed installation, which revision are you running? --Dave B. On 09/04/2013 06:57 AM, Greg Von Kuster wrote: Hello Simin, Bjoern is correct - if this is possible (which it looks like it is from your traceback), then a bug has been introduced somewhat recently. We'll take a look at this today and get a fix committed. Thanks for reporting this. Greg Von Kuster On Sep 4, 2013, at 3:17 AM, Bjoern Gruening bjoern.gruen...@gmail.com wrote: Hi Simon, you are right. Public usernames, or usernames associated with a repository can't be changed. If you can that is a bug, please fill a trello card with it. The reason is to guarantee reproducibility. If anyone has installed your repository the triple (username,revision,toolshed) is saved to reinstall that repository at any time, if needed. Cheers, Bjoern It looks like it's trying to use the public name in the repo path, which isn't going to work very well if public names can be changed. Changing my public name back to what it was before made it start working again. To reply to my own post, I see now that when you create a user, a comment on that page says: You cannot change your public name after you have created a repository in this tool shed. Does this mean: If you change your public name after you have created a repository in this tool shed, those repositories won't work any more or some such ;-). Or is there actually some code to stop the user changing their public name in this case, which would imply my toolshed is somehow broken in a way unique to me? (I'm currently struggling to push tools into my toolshed using mercurial command line, through my apache front end, with the various layers of authentication, without it saying abort: authorization failed.) cheers, Simon === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please
Re: [galaxy-dev] Mercurial updates not working
Hi Alistair, I have seen error output like this from mercurial when the client is running a newer version than the server. Maybe you recently upgraded your machine, and your mercurial version is now too recent. Hope that helps. Of course, it may be a complete red herring, YMMV. ;-) cheers, Simon -Original Message- From: galaxy-dev-boun...@lists.bx.psu.edu [mailto:galaxy-dev- boun...@lists.bx.psu.edu] On Behalf Of Alistair Chilcott Sent: Wednesday, 28 August 2013 12:33 p.m. To: galaxy-dev@lists.bx.psu.edu Subject: [galaxy-dev] Mercurial updates not working Hello all, Mercurial has recently started throwing errors: galaxy@hitmsbhpc1:~/galaxy-dist hg incoming abort: error: _ssl.c:517: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol It has previously worked with issues has anyone seen this before? I don't think anything has changed that would have caused this problem but googling error has proved to be unhelpful so far. Regards, Alistair ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Bug in toolshed: changing public name breaks repo path
I'm just setting up a local toolshed. After creating a repo, I changed my public name (from the default guestsi to simon-guest) on the web interface, and got this traceback when I tried to look at my repo: Internal Server Error Galaxy was unable to sucessfully complete your request URL: http://galaxy-dev.agresearch.co.nz/toolshed/repository/manage_repository?sort=namepage=1async=falseshow_item_checkboxes=falseoperation=view_or_manage_repositoryid=5250d6d52a525eba Module galaxy.web.framework.middleware.error:149 in __call__ app_iter = self.application(environ, sr_checker) Module paste.debug.prints:106 in __call__ environ, self.app) Module paste.wsgilib:543 in intercept_output app_iter = application(environ, replacement_start_response) Module paste.recursive:84 in __call__ return self.application(environ, start_response) Module galaxy.webapps.tool_shed.framework.middleware.remoteuser:74 in __call__ return self.app( environ, start_response ) Module paste.httpexceptions:633 in __call__ return self.application(environ, start_response) Module galaxy.web.framework.base:132 in __call__ return self.handle_request( environ, start_response ) Module galaxy.web.framework.base:190 in handle_request body = method( trans, **kwargs ) Module galaxy.web.framework:98 in decorator return func( self, trans, *args, **kwargs ) Module galaxy.webapps.tool_shed.controllers.repository:1883 in manage_repository repo_dir = repository.repo_path( trans.app ) Module galaxy.webapps.tool_shed.model:205 in repo_path return app.hgweb_config_manager.get_entry( os.path.join( repos, self.user.username, self.name ) ) Module galaxy.webapps.tool_shed.util.hgweb_config:62 in get_entry raise Exception( Entry for repository %s missing in file %s. % ( lhs, self.hgweb_config ) ) Exception: Entry for repository repos/simon-guest/emboss_5_native missing in file /home/galaxy-dev/hgweb/hgweb.config. It looks like it's trying to use the public name in the repo path, which isn't going to work very well if public names can be changed. Changing my public name back to what it was before made it start working again. cheers, Simon === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Bug in toolshed: changing public name breaks repo path
It looks like it's trying to use the public name in the repo path, which isn't going to work very well if public names can be changed. Changing my public name back to what it was before made it start working again. To reply to my own post, I see now that when you create a user, a comment on that page says: You cannot change your public name after you have created a repository in this tool shed. Does this mean: If you change your public name after you have created a repository in this tool shed, those repositories won't work any more or some such ;-). Or is there actually some code to stop the user changing their public name in this case, which would imply my toolshed is somehow broken in a way unique to me? (I'm currently struggling to push tools into my toolshed using mercurial command line, through my apache front end, with the various layers of authentication, without it saying abort: authorization failed.) cheers, Simon === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Tool Shed packages for BLAST+ binaries
Simon wrote: An example is the NCBI BLAST+ suite, which failed to build on my (EL6) system, because it couldn't run /usr/bin/touch. That's pretty dumb, and pretty simple to solve in isolation - it needs to be running /bin/touch instead. This was an oddity in the NCBI compile scripts for BLAST 2.2.26+ which can be avoided via a symlink, editing their code, or more simply just downloading their provided binaries. I prefer the later because it is simpler, faster, and should better ensure reproducibility by eliminating things like different compilers and their settings. Unfortunately the new install process to do this from within the Tool Shed is glitching on the Tool Shed's nightly tests: Nevertheless, the simple binary install is working for me and Bjoern. Could you try the Test Tool Shed package and let us know if that works for you? Hi Peter, It seems to me that there's quite a bit still to do on the Tool Shed dependency packaging machinery. I don't think I'm inclined to be part of that testing process, sorry. (Rather short of time, having to choose carefully where I get most bang for buck.) As an alternative, I note your requirement for having multiple versions of a package installed side-by-side, which isn't traditionally offered by package repositories, although there is nothing to preclude that. In fact, I am currently in discussion with the CentOS project leader about collaborating on a scientific repo for Enterprise Linux (RHEL, CentOS, Scientific Linux, Oracle Linux). I will be feeding in my ideas for multiple version installs, and will see where that takes us. Who knows, you may have better choices soon on how to install packages for the Tool Shed. cheers, Simon === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Better packaging for toolshed binaries
Dear Galaxians, This email is about difficulties with the current approach for installing tool dependency binaries from the Galaxy Toolshed, and what might be done to improve the situation. It comes down to this: packaging software to run on different systems is tricky. It is a problem that has been solved by various Linux distributions with their packaging systems (RPM, deb, etc.), and package archives. The Galaxy Toolshed is trying to solve this problem again, but so far it doesn't work very well. There must be something better we can do. Since gaining a better understanding from the Galaxy Community Conference of what the Toolshed is trying to do (versioned tools, reproducibility), I have been working on switching over from locally installed tools to Toolshed versions. However, it has not gone well, and I think I am about to revert to my previous approach. Here's the problem: building software from source on any system requires certain tweaks to the build process which are dependent on the target platform. An example is the NCBI BLAST+ suite, which failed to build on my (EL6) system, because it couldn't run /usr/bin/touch. That's pretty dumb, and pretty simple to solve in isolation - it needs to be running /bin/touch instead. But the general point is this: it's not feasible (i.e. too much work, too hard) to produce build scripts to build software from source that work on any platform, even the common ones. Packaging source code for a given platform is a non-trivial task. The RPM and deb packagers are doing a good job here. It's a significant amount of work. I know that, as I've been packaging bioinformatics software as binary RPMs for EL6 for 18 months or so now, and have done nearly 300 packages. What do we want? Simply to be able to install a given version of some software, and all its dependencies, with a single click, or a single command, and have it Just Work (tm). It's the dependencies that make this hard. Things get installed in different ways on different systems. Does your platform need #include bam.h, or #include bam/bam.h? If the former, then you'll have to patch tophat, say, (in a trivial way) before building it. I think this is simply too hard to do by embedding some commands and conditionals in Toolshed XML build files. It seems to me that a number of people out there are currently having some issues installing tool dependencies from the Toolshed, because things are not building as expected. I think it's much easier for just one person to troubleshoot why things go wrong when they are packaging the software for a given platform, rather than for each end user (Galaxy admin) to wonder why a tool failed to install. So, what to do? My starting point is that I have packaged a large amount of bioinformatics software for EL6, which is freely available at http://rpm.agresearch.co.nz/. I'm after some Galaxy tool wrappers for the tools that we use here at AgResearch, which can simply make use of packages installed from this repo. Is there any interest in exploring the merits or otherwise of this approach in the Galaxy community? cheers, Simon Simon Guest Senior UNIX Technical Consultant AgResearch, New Zealand === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Patch for RPy for R 3.0.1
Just catching up with the galaxy-dev mailing list, and found some remarks about RPy not working with R 3.0. If that's bugging you, you may be interested in my patches for RPy-1.0.3a, which enable it to work with R 3.0.1 (and probably 3.0.0, but we've moved on from that here). Attached, in case it helps someone. cheers, Simon PS: Anyone on EL6 could just install the python-rpy package from the AgResearch repo, http://rpm.agresearch.co.nz/ ;-) === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === rpy-1.0.3a-PrintWarnings.patch Description: rpy-1.0.3a-PrintWarnings.patch rpy-1.0.3a-build.patch Description: rpy-1.0.3a-build.patch ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Problem referencing EMBOSS tools in universe_wsgi.ini
Hi Simon, Significant changes to the way that job run parameters are defined will be included in a future release. In specific, there'll be no more job runner URLs, and parameters will be specified with a more robust XML language. These changes aren't stable enough to go out in the next release (scheduled for the end of this week), but they should be in the one after that. --nate Hi Nate, Thanks for that. We look forward to that release! cheers, Simon -- === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Patch for RepeatMasker.xml for RepeatMasker 3.3.0
Dear Galaxians, We are using Galaxy with RepeatMasker 3.3.0. I don't know whether that is the expected version, as I can't find any statement about what is required in the Galaxy tool dependencies for RepeatMasker. Anyway, when a job is submitted with no repeats, RepeatMasker 3.3.0 doesn't create the various output and summary files (.tbl, .out.gff, .out.html, .masked). This causes the Galaxy job to fail with an error message that scares the end user. I attach a patch to handle this case, by touch'ing these files before trying to copy them. Now such jobs succeed with empty output. I think this is clearer, and certainly less scary. Is there a better mechanism for me to contribute such fixes other than posting patches on this list? cheers, Simon === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === galaxy-repeatmasker-agr.patch Description: galaxy-repeatmasker-agr.patch ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Problem referencing EMBOSS tools in universe_wsgi.ini
We have set up our local Galaxy installation to submit jobs by default through HTCondor, using the drmaa tool runner. We override this on a tool-by-tool basis in universe_wsgi.ini, to specify certain tools we always want to run locally. The problem is that EMBOSS tools have a tool id like 'emboss: infoseq46', and it's not possible to reference a tool with an id containing a colon in the universe_wsgi.ini file (because colon is the character which separates a configuration item name from its value). It seems to me there are two possible solutions: 1. Implement some form of escape character processing in the config file parser (on top of Python's ConfigParser) so we can write tool ids in that file like 'emboss%3a infoseq46' 2. Change all the EMBOSS tools to have ids like 'emboss-infoseq46' rather than using a colon and space. For now, we are modifying our XML files locally (with a script) to do 2. Are there any comments from Galaxy developers on which of these solutions they prefer, or any reason why (2) shouldn't be incorporated in the Galaxy mainline? cheers, Simon === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Galaxy with EMBOSS 5 or EMBOSS 6 ?
Dear Galaxians, I am troubleshooting a problem with running EMBOSS infoseq on our local galaxy installation. We are running EMBOSS 6.5. There is a mismatch between the options Galaxy is passing to infoseq (-version), and what infoseq wants (-seqversion). This breaks the tool. While this would be easy to fix in this instance, I wonder if I should be using EMBOSS 5 (which is *really* old, mind you) instead of EMBOSS 6. I suspect with EMBOSS 6 there may be a whole host of similar changes to be made. Can anyone advise? If it turns out that EMBOSS 5 is actually required, perhaps someone could update the relevant info on the Galaxy dependencies page, or at least point me to the place where it says this. cheers, Simon Simon Guest Senior UNIX Technical Consultant AgResearch === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Galaxy with EMBOSS 5 or EMBOSS 6 ?
You're right -- the EMBOSS version supported by the Galaxy wrappers is currently EMBOSS 5. We do have a Tool Dependency page here: http://wiki.galaxyproject.org/Admin/Tools/Tool%20Dependencies, but unfortunately while it lists versions for many tools it doesn't cover them all. I'll update the EMBOSS entry. Hi Dannon, Thanks. After a bit more looking around, I also found the version in the tool_conf.xml file, which confirmed I need to install EMBOSS 5.0.0. As far as updating the galaxy EMBOSS wrappers to use a newer version, I'm not aware of any plans to do this currently (though someone could certainly do this if so inclined). We may do this ourselves, and contribute the updates. First we'll get it all rolled out with EMBOSS 5. Later on I may be back in touch about how we can contribute updates into the Galaxy mainline. cheers, Simon === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] RPy testsuite some tests passing - enough for Galaxy?
Rpy is only used by a handful of tools (should be listed on the tool dependencies page). Everything else in Galaxy should work fine without it. Yes, I realise that. It's that handful of tools I'm interested in. I expect they will do their thing just fine with a semi-broken RPy, but I was hoping someone might be able to confirm that. Will have to test locally then. cheers, Simon === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] RPy testsuite some tests passing - enough for Galaxy?
Dear Galaxy devs,
We are in the process of deploying Galaxy internally here at AgResearch, and I
am currently installing required dependencies. My question relates to RPy. It
seems that RPy2 is not yet supported by Galaxy, so we're stuck with RPy 1.0.3a,
and this seems to have rusted quite badly. (I note however that the remarks
previously made on the Galaxy lists about changes required to RPy appear to
have been rolled into 1.0.3a.)
After installing RPy 1.0.3a, I ran the bundled test suite. Many tests succeed,
but there are several errors, in the following categories:
1. Errors about numpy, e.g. object of type numpy.int64 has no len() (Maybe
numpy has changed since RPy 1.0.3a was written. I am using numpy 1.4.1 on
CentOS 6.)
2. Errors due to missing files, e.g. logit.r and table.txt test data files do
not appear to ship with RPy-1.0.3a (!)
3. An error in testLapackLoad, which may be because of (2), I'm not sure yet.
4. An R object index out of range error in testGetItemSlice, which looks quite
serious.
So here's my question. Just how much of RPy has to work, for its use in Galaxy
to be OK?
I attach my RPy test suite output in case anyone is (a) interested, (b) has any
useful comments.
cheers,
Simon
Simon Guest
Senior UNIX Technical Consultant
AgResearch, NZ
===
Attention: The information contained in this message and/or attachments
from AgResearch Limited is intended only for the persons or entities
to which it is addressed and may contain confidential and/or privileged
material. Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by persons or
entities other than the intended recipients is prohibited by AgResearch
Limited. If you have received this message in error, please notify the
sender immediately.
===
impulsive$ python testall.py --random
Modules to be tested: ['test_io.py', 'test_boolean.py', 'test_util.py',
'test_lapack_load.py', 'test_array.py', 'test_cleanup.py', 'test_topy.py',
'test_tor.py', 'test_vars.py', 'test_modes.py', 'test_numeric.py',
'test_init.py', 'test_noninteractive.py', 'test_robj.py', 'test_sigint.py']
Testing: test_topy
..Warning message:
In file(file, rt) :
cannot open file 'table.txt': No such file or directory
E..
==
ERROR: testDataFrameToList (test_topy.TypeConversionToPyTestCase)
--
Traceback (most recent call last):
File /home/guestsi/rpmbuild/BUILD/rpy-1.0.3/tests/test_topy.py, line 65, in
testDataFrameToList
assert(r.read_table('table.txt', header=1) ==
RPy_RException: Error in file(file, rt) : cannot open the connection
--
Ran 21 tests in 0.027s
FAILED (errors=1)
Testing: test_boolean
..
--
Ran 2 tests in 0.000s
OK
Testing: test_numeric
.E.E.E...E.E.E..
==
ERROR: test16BitIntScalar (test_numeric.ArrayTestCase)
--
Traceback (most recent call last):
File /home/guestsi/rpmbuild/BUILD/rpy-1.0.3/tests/test_numeric.py, line
229, in test16BitIntScalar
b = r.c(a[0])
TypeError: object of type 'numpy.int16' has no len()
==
ERROR: test32BitFloatScalar (test_numeric.ArrayTestCase)
--
Traceback (most recent call last):
File /home/guestsi/rpmbuild/BUILD/rpy-1.0.3/tests/test_numeric.py, line
268, in test32BitFloatScalar
b = r.c(a[0])
TypeError: object of type 'numpy.float32' has no len()
==
ERROR: test32BitIntScalar (test_numeric.ArrayTestCase)
--
Traceback (most recent call last):
File /home/guestsi/rpmbuild/BUILD/rpy-1.0.3/tests/test_numeric.py, line
219, in test32BitIntScalar
b = r.c(a[0])
TypeError: object of type 'numpy.int32' has no len()
==
ERROR: test64BitIntScalar (test_numeric.ArrayTestCase)
--
Traceback (most recent call last):
File /home/guestsi/rpmbuild/BUILD/rpy-1.0.3/tests/test_numeric.py, line
209, in test64BitIntScalar
b = r.c(a[0])
TypeError: object of type 'numpy.int64' has no len()
==
ERROR: test8BitIntScalar (test_numeric.ArrayTestCase)
