Re: [galaxy-dev] Proper Galaxy Installation
Dear Marija, I would recommend to google for the "ngs-fablab" - there are two posters and a talk online from the GCCs 2014 and 2015, respectively. I admit, it may look like self-advertising :). But in fact we set up a local instance at the medical faculty in Munich (LMU) which is quite similar to yours in terms of pre-conditions: we (have to) deal with SLES 11, do virtualization (although via vSphere, SLES just for the guests) and intended to go modular, step by step, including cluster usage. As we had to rely on our own hardware exclusively and clean system setups where necessary repetively, we created a setup framework that configures servers remotely for being Galaxy host, fileserver, cluster head/node or arbitrary combinations. You just need a blank SLES (we tested also other UNIXes like Ubuntu and CentOS) and SSH enabled. As SLES is hopelessly outdated regarding software packages, everything is compiled from source. The framework's technical base is Ansible, which has become quite well known also in other parts of the Galaxy community (initially introduced by the core team). If you like to contact us feel free - we are curious for others in the medical field :). Cheers, Sebastian -- Sebastian Schaaf, M.Sc. Bioinformatics Faculty Coordinator NGS Infrastructure Chair of Biometry and Bioinformatics Department of Medical Informatics, Biometry and Epidemiology (IBE) University of Munich DKTK Munich Marchioninistr. 15, K U1 808 D-81377 Munich (Germany) Tel: +49 89 4400 77499 > Dear all, > > I am currently working on Galaxy installation at our Center for Medical > Research. As a login server, we are using VM based on SLES-12. We are > planning to extend it on a Cluster later and we would like to use SLURM as > DRM. Is there any recommendation or "step-by-step" recipe how to setup > Galaxy in a proper way to avoid future issues? Are they any recommendation > regarding the filesystem of directory management system that I should care > about in advance? > Any advice from experienced developers will be more than helpful. > Thank you in advance. > Regards, > > Mag. Marija Ðurðeviæ > Core Facility Computational Bioanalytics > > Medical University of Graz > Center for Medical Research > Stiftingtalstraße 24, A-8010 Graz > Austria > > Phone: +43 316/385-73024 > Fax:+43 316/385-73009 > > Email: marija.djurdje...@medunigraz.at > Email: marija.djurdje...@klinikum-graz.at > > Web: https://zmf.medunigraz.at/ > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > https://lists.galaxyproject.org/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Python 2.7 vs 2.6 for galaxy after 16.01
For us, it was more the fact that we are locked down to a certain OS version due to our local IT policy and the software repositories don't provide a newer Python than v2.6.8. It was not the first occasion in terms of desperately outdated software (recently it's gcc itself), so we are used to compile from source. Using pre-compiled packages from OpenSUSE repos is not guaranteed to work properly. For us, it is just another (larger) paragraph for our setup environment written in Ansible. Overall, it is quite generic and nearly everything needed by Galaxy is installed in home scheme. On top, although we are in a quite tight internal network, we realized that our OpenSSL is of 'pre-heartbleed' times, so it was a good deal to re-integrate that. Python, wget and others take advantage from this update if properly linked. Certificate exchange and SSL version usage have moved on in the last years, and from time to time we got errors/warnings when accessing external servers. > On Wed, Mar 23, 2016 at 10:54 PM, D K wrote: >> Great! thanks for the suggestions. I just tried using centos software >> collections and that seems to work. I'll do some more testing but >> hopefully >> it's as simple as that! Is there any reason that most of you who >> responded >> have decided to compile your own pythons? >> >> D > > It sounds like both options are effective. > > In my case it was perhaps a choice out of ignorance about just how the > CentOS software collections works - and fear about the CentOS 6 to 7 > migration. > > In our case both the Galaxy server and the cluster is still on CentOS 6, > so we need to use a custom Python 2.7 on both the Galaxy server and > also on the cluster nodes - ideally the exact same version so that all > the wheel dependencies match up for running the set metadata scripts. > > I'm hoping that using a custom Python 2.7 installation (from source) > ought to work in our favour if we have to deal with a mixture of CentOS > 6 and 7 (although ideally we'd update the cluster and Galaxy server > together). > > Regards, > > Peter > -- Sebastian Schaaf, M.Sc. Bioinformatics Faculty Coordinator NGS Infrastructure Chair of Biometry and Bioinformatics Department of Medical Informatics, Biometry and Epidemiology (IBE) University of Munich DKTK Munich Marchioninistr. 15, K U1 808 D-81377 Munich (Germany) Tel: +49 89 4400 77499 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Python 2.7 vs 2.6 for galaxy after 16.01
Same here for v2.7.11 on SLES 11. If you want to integrate as many modules as possible at compilation time (we had to skip four unimportant ones in the end) it may be painful with an older OS, but I assume CentOS is a bit less recalcitrant than SLES. We installed everything in home scheme, which requires proper setup of environment variables like $PATH, $LD_LIBRARY_PATH, $PYTHONPATH, etc... For us the trigger for switching the Python version was an issue with an outdated OpenSSL (just days before the Galaxy announcement regarding the coming deprecation of 2.6), so we included the reworked/conmpiled OpenSSL libraries first. I would recommend the same for you, if your OpenSSL package is We Iocally compile 2.7.8 and keep it in a distinct location that our paths > references. > > This is very easy and has worked well for us. > > Please excuse any typos -- Sent from my iPhone > >> On Mar 21, 2016, at 5:32 PM, Peter Cock >> wrote: >> >> Hi D. K., >> >> We're currently trying a locally compiled Python 2.7 (from source) >> under the shared Galaxy mount for use with a CentOS 6 cluster >> and Galaxy server. >> >> This seems to be working once we got the $PATH working for >> cluster jobs (initially they would try to set the Galaxy metadata >> using the system Python 2.6, which would fail). >> >> I don't know if this is any easier than using the CenOS Software >> Collections as suggested by Nicola, especially when it comes to >> updating the OS. >> >> Peter >> >>> On Mon, Mar 21, 2016 at 5:59 PM, Nicola Soranzo >>> wrote: >>> Hi Daniel, >>> I've been using python27 from CentOS Software Collections for quite >>> some >>> time. It's a bit annoying because you have to remember to load it, but >>> it is >>> surely doable. >>> >>> Cheers, >>> Nicola >>> >>> >>> On 21/03/16 17:46, D K wrote: >>> >>> Hi Galaxy Devs, >>> >>> This may be premature but I read that for Galaxy after 16.01 that the >>> plan >>> is for python 2.6 to no longer be supported. I'm currently running >>> CentOS >>> 6.6 which uses python 2.6. In order to be able to use newer versions of >>> Galaxy is my only option to upgrade to a newer version of CentOS? Is it >>> possible to use Conda, RedHat Software Collections or some other >>> similar >>> method instead of doing this upgrade? >>> >>> Thanks! >>> >>> >>> ___ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> https://lists.galaxyproject.org/ >>> >>> To search Galaxy mailing lists use the unified search at: >>> http://galaxyproject.org/search/mailinglists/ >>> >>> >>> >>> ___ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> https://lists.galaxyproject.org/ >>> >>> To search Galaxy mailing lists use the unified search at: >>> http://galaxyproject.org/search/mailinglists/ >> ___ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> https://lists.galaxyproject.org/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > https://lists.galaxyproject.org/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ -- Sebastian Schaaf, M.Sc. Bioinformatics Faculty Coordinator NGS Infrastructure Chair of Biometry and Bioinformatics Department of Medical Informatics, Biometry and Epidemiology (IBE) University of Munich DKTK Munich Marchioninistr. 15, K U1 808 D-81377 Munich (Germany) Tel: +49 89 4400 77499 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Gemini installation error
Hi Nate,
Great to read from you :).
In the time between my request and your answer I tried out some things
starting from the codes and googled a lot. As I left the desk soon after I
could add it up to now. I'll try to keep it short and as comprehensible as
possible:
* The issue is related to Python itself, not necessarely Unix. 'wget' on
the URL you also used works fine, so proxying is ok.
* There are bugreports on the 'urllib' libraries (there are three of
them), for the two older ones there is no indication for a fix. Here
https://github.com/kennethreitz/requests/issues/1204
a bug report on the 'requests' module appears, the guys where waiting for
a fix in 'urllib3' (which was finally introduced). But: the tool shed
package comes around with libraries 'urllib' ad 'urllib2', only.
* Generally, it looks like the issue is due to a default behaviour of
Python 2.x's 'ssl' module that may have been no problem in the past: the
handshake is by default performed with SSLv2.3. As on a rising number of
servers SSLv2 gets deactivated for security reasons, connections fail
(SSLv3 is expected and maybe sent - client and server don't unterstand
each other anymore). Check out the second and third highest ranked answer
on this issue:
http://stackoverflow.com/questions/14102416/python-requests-requests-exceptions-sslerror-errno-8-ssl-c504-eof-occurred.
The second also includes a link to relevant line of code. For me it sounds
reasonable, but I am a relative noob on web protocols...
Consequently, the answer to your question is the following:
---
>>> import urllib
>>> urllib.urlretrieve("https://raw.githubusercontent.com/arq5x/gemini/master/versioning/0.18.0/requirements_conda.txt";)
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib64/python2.6/urllib.py", line 93, in urlretrieve
return _urlopener.retrieve(url, filename, reporthook, data)
File "/usr/lib64/python2.6/urllib.py", line 239, in retrieve
fp = self.open(url, data)
File "/usr/lib64/python2.6/urllib.py", line 207, in open
return getattr(self, name)(url)
File "/usr/lib64/python2.6/urllib.py", line 441, in open_https
h.endheaders()
File "/usr/lib64/python2.6/httplib.py", line 914, in endheaders
self._send_output()
File "/usr/lib64/python2.6/httplib.py", line 786, in _send_output
self.send(msg)
File "/usr/lib64/python2.6/httplib.py", line 745, in send
self.connect()
File "/usr/lib64/python2.6/httplib.py", line 1122, in connect
self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file)
File "/usr/lib64/python2.6/ssl.py", line 338, in wrap_socket
suppress_ragged_eofs=suppress_ragged_eofs)
File "/usr/lib64/python2.6/ssl.py", line 120, in __init__
self.do_handshake()
File "/usr/lib64/python2.6/ssl.py", line 279, in do_handshake
self._sslobj.do_handshake()
IOError: [Errno socket error] [Errno 1] _ssl.c:497: error:140770FC:SSL
routines:SSL23_GET_SERVER_HELLO:unknown protocol
---
I also encountered the same error message when I try to install or update
Python packages via 'easy_install' or 'pip' - both cannot connect to
repositories. Also I don't know if the failed handshake is at the target
server or our proxy - any idea for a command to check this?
Finally, I am running out of ideas what to do, for me it looks like it is
too deep in the code for establishing a general workaround. On top, if I
interpreted correctly the findings from the above bullet point list, this
may have an impact on many pieces of software. For every case, where
Python 2.x-driven instances try to access repositories on servers not
offering SSLv2 anymore (maybe via proxy server).
Ideas? :)
Cheers,
Sebastian
> On Thu, Feb 11, 2016 at 12:43 PM, Sebastian Schaaf <
> sch...@ibe.med.uni-muenchen.de> wrote:
>
>> Hey all,
>>
>> I am currently trying to install Gemini (v0.18.1) via the test toolshed.
>> It finally fails in dependencies as 'package_gemini_0_18_1' finds
>> dependency 'gemini' to end in error state. There the message looks like
>> the following:
>>
>> ---
>> 2016-02-11 16:34:47 (72.4 MB/s) - `Miniconda-latest-Linux-x86_64.sh'
>> saved
>> [25416725/25416725]
>>
>> Python 2.7.11 :: Continuum Analytics, Inc.
>> Traceback (most recent call last):
>> File "gemini_install.py", line 190, in
>> main(parser.parse_args())
>> File "gemini_install.py", line 80, in main
>> gemini = install_conda_pkgs(anaconda, remotes, args)
>> File "gemini_install.py", line 97, in install_conda_pkgs
>> urllib.urlretrieve(remotes["requirements_conda"], fi
[galaxy-dev] Gemini installation error
Hey all, I am currently trying to install Gemini (v0.18.1) via the test toolshed. It finally fails in dependencies as 'package_gemini_0_18_1' finds dependency 'gemini' to end in error state. There the message looks like the following: --- 2016-02-11 16:34:47 (72.4 MB/s) - `Miniconda-latest-Linux-x86_64.sh' saved [25416725/25416725] Python 2.7.11 :: Continuum Analytics, Inc. Traceback (most recent call last): File "gemini_install.py", line 190, in main(parser.parse_args()) File "gemini_install.py", line 80, in main gemini = install_conda_pkgs(anaconda, remotes, args) File "gemini_install.py", line 97, in install_conda_pkgs urllib.urlretrieve(remotes["requirements_conda"], filename=req_file) File "/home/galaxy/tool_libs/python2.7/lib/python2.7/urllib.py", line 98, in urlretrieve return opener.retrieve(url, filename, reporthook, data) File "/home/galaxy/tool_libs/python2.7/lib/python2.7/urllib.py", line 245, in retrieve fp = self.open(url, data) File "/home/galaxy/tool_libs/python2.7/lib/python2.7/urllib.py", line 213, in open return getattr(self, name)(url) File "/home/galaxy/tool_libs/python2.7/lib/python2.7/urllib.py", line 443, in open_https h.endheaders(data) File "/home/galaxy/tool_libs/python2.7/lib/python2.7/httplib.py", line 1049, in endheaders self._send_output(message_body) File "/home/galaxy/tool_libs/python2.7/lib/python2.7/httplib.py", line 893, in _send_output self.send(msg) File "/home/galaxy/tool_libs/python2.7/lib/python2.7/httplib.py", line 855, in send self.connect() File "/home/galaxy/tool_libs/python2.7/lib/python2.7/httplib.py", line 1274, in connect server_hostname=server_hostname) File "/home/galaxy/tool_libs/python2.7/lib/python2.7/ssl.py", line 352, in wrap_socket _context=self) File "/home/galaxy/tool_libs/python2.7/lib/python2.7/ssl.py", line 579, in __init__ self.do_handshake() File "/home/galaxy/tool_libs/python2.7/lib/python2.7/ssl.py", line 808, in do_handshake self._sslobj.do_handshake() IOError: [Errno socket error] [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:590) --- Some browsing at Google brought hints on three common cases where this could appear *plus* a fourth one which looks by error code pretty much what I got: http://blog.techstacks.com/2010/03/3-common-causes-of-unknown-ssl-protocol-errors-with-curl.html So, may I be right that there is a typo in an URL needed for downloading during the setup process? Or is it, according to http://stackoverflow.com/questions/10667960/python-requests-throwing-up-sslerror due to some failed SSL certificate check? Some help (or ideas for targeted checking) would be great :). Cheers, Sebastian -- Sebastian Schaaf, M.Sc. Bioinformatics Faculty Coordinator NGS Infrastructure Chair of Biometry and Bioinformatics Department of Medical Informatics, Biometry and Epidemiology (IBE) University of Munich DKTK Munich Marchioninistr. 15, K U1 808 D-81377 Munich (Germany) Tel: +49 89 4400 77499 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Happy new year!
Hey Björn, Same for you, and a successful year to everyone! Curious on your promise ;). Cheers, Sebastian > Hi Galaxy Developers, > > last year was a blast for Galaxy and it seems that the Galaxy core team > is preparing a new release soon with some big big features :) > > I really enjoy working with the Galaxy community and wish you all a > happy new year. Let's keep rocking the Galaxy and make Bioinformatics > even more accessible to everyone in 2016. > > Happy new year! > Bjoern > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > https://lists.galaxyproject.org/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ -- Sebastian Schaaf, M.Sc. Bioinformatics Faculty Coordinator NGS Infrastructure Chair of Biometry and Bioinformatics Department of Medical Informatics, Biometry and Epidemiology (IBE) University of Munich DKTK Munich Marchioninistr. 15, K U1 808 D-81377 Munich (Germany) Tel: +49 89 4400 77499 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Galaxy Interactive Environment devs
Björn Grüning wrote: Am 13.10.2015 um 13:41 schrieb Sebastian Schaaf: Hey Eric, We are not yet on IEs, No, no, no. How can this happen? ;) I interpret ";)" as "Indeed I remember all the issues you poor guys are faced with" ;) but we plan to upgrade our local production instance soon (a recent version is already 'waiting' to take over), apart from other goodies we intended to allow one pro user and ourselves to use iPython's power this way. So, do I read your message correctly as 'In terms of iPython it would be best for us to wait for v15.10'? Or could we just start and do an incremental upgrade as soon as v15.10 is released? 15.10 is not needed for IPython. It's running since 15.05 as I remember correctly. But we improved it a bit and you should take care that you use the correct Docker Image for IPython. But it should work out of the box. If not, let us know :) Cheers, Bjoern We'll have an eye on this, thanks for the hint. And yes: We know how to precisely annoy you and/or Eric ;). As it somehow picks up Eric's initial question: is your impression the same as mine (because of no further replies to the post) that still very few people use in fact the great opportunities of IEs in "non-main" Galaxy instances? Cheers, Sebastian Cheers to Texas, Sebastian Eric Rasche schrieb: Howdy y'all, I'm building some changes to the IE infrastructure that will fix some initial hacks we did, however they will be backwards incompatible. Specifically these changes will fix some of the enterprise deployment pains (running with an external upstream proxy like apache) and allow for safely running IEs on a separate host. I'm emailing the dev list to see if anyone else has started building IEs, or if it just Björn and myself. Feel free to make yourselves known to us so we know who to contact when we're planning new features :) If you're just building on top of our existing containers, this won't be an issue for you, you'll just need to rebuild your containers once we release 15.10 versions. Cheers, Eric -- Sebastian Schaaf, M.Sc. Bioinformatics Faculty Coordinator NGS Infrastructure Chair of Biometry and Bioinformatics Department of Medical Informatics, Biometry and Epidemiology (IBE) University of Munich DKTK Munich Marchioninistr. 15, K U1 808 D-81377 Munich (Germany) Tel: +49 89 4400 77499 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] QIIME tools for Galaxy (WIP, call for collaborators)
Hi Björn, The discussion looks good, although (or right because?) it gained a somewhat more generic scope, where QIIME is one of several construction sites in the metagenomics field. Do I get it right that the date for the hackathon is tba? Will there be a notification? I think we would like to join :). Anyhow we discussed yesterday in the office and came up the following: 1. Ashok collected all dependencies (names/versions of tools/libraries), he also formalized them in recipes. The community has to check whether it fits the needs/expectations, as we are relatively new to tool sheds and started to load a private one recently with tools. But the plain list should already save some work. As we run SLES as OS (which is close to blank in terms of up to date software...) the dependency list should go quite far. 2. The codes which were written by him for an internship will anyhow get some improvements: currently the "qiime_connector" is simply a kind of archive extractor which makes an arbitrary file from a QIIME tool output archive available in the Galaxy history, so that there is neither a dead end nor the need for a "download/upload" step. For user who know QIIME from the command line (and therefore know how the modules work) this may be a suitable solution, but not for beginners who start with metagenomics. Somehow it does not feel "galaxy'sh"... So far, looking forward to joining forces :). Cheers, Sebastian Björn Grüning schrieb: Hi Sebastian, I think we try to work on this during our upcoming hackathon. https://github.com/galaxyproject/tools-iuc/issues/299 It would be nice to join and contribute your work to it. Thanks, Bjoern Am 13.10.2015 um 15:03 schrieb Sebastian Schaaf: Hey there, Any news on this? My colleague Aarif made me aware of the discussion, because one of our students, Ashok, worked on this months ago. QIIME was requested by a project which never started and therefore the developed codes did not gain priority. Basically we faced the same issues, and at least the 'dead end' problem with the resulting tgz archives was solved. We are currently assembling the codes into a toolshed entry, but I need to ask Ashok tomorrow about the current progress. Is there still interest on it or is everything already on the move..? Cheers, Sebastian Björn Grüning schrieb: Hi Yvan, Hi Lance, Björn, Dan We are several scientists working on metagenomics in France and at a previous GUGGO (western France User Group) meeting, we mention the fact to "benchmark" our Galaxy pipelines. One is using new Qiime tools integration, one Mothur and the last one, the recently developped FROGS (Find Rapidly OTU with Galaxy Solution) pipeline using Swarm. We already have had some exchanges with Björn concerning metagenomics flavours on Docker, and I think it will be particularly interesting to exchange one with each other. The Qiime integration was started in march by colleagues from Brest Ifremer centre. I was just trying to install and test FROGS from Toulouse and Jouy-en-Josas INRA labs. If they are not on the list, I can contact them. Please do so. This is great to know. Please also consider to join our hackathon: https://github.com/galaxyproject/tools-iuc/issues/299 We are always open for new ideas. Would be great to get a nice suite and a few workflows for metagenomic in the near future! :) Thanks, Bjoern All the best, Yvan - Mail original - De: "Björn Grüning" À: "Lance Parsons" , "Daniel Blankenberg" Cc: "Galaxy Dev List" Envoyé: Mercredi 7 Octobre 2015 14:25:38 Objet: Re: [galaxy-dev] QIIME tools for Galaxy (WIP,call for collaborators) Hi Lance, I will help if I can. I also think we should begin too write 2-3 nice tools in the beginning to test the handshake between them and lay the ground work. This will also help to get familiar with the tool. I still hope some one else from the community shows interest! :) Thanks, Bjoern Am 06.10.2015 um 20:33 schrieb Lance Parsons: I agree that it would be very nice to get the data flowing between each of the tools and to be able to mix/match with other tools. That is an area of Galaxy tool-dev that I'm less familiar with, so any help would be greatly appreciated. As for the manual massaging, I agree, however, at this point I'm really hoping that the changes between Qiime versions are manageable, esp. once we have the data types and basic framework of the tools down. Along that front, I would very much like to get something usable (at least for some specific workflows) done sooner rather than later, then be able to iterate, adding new tools, datatypes, etc. as we move forward. Does that sound reasonable? I love the idea of a Metagenomics toolfest (https://github.com/galaxyproject/tools-iuc/issues/299). One thing I would like to do soon then, would be to define some functionality for the first roun
Re: [galaxy-dev] QIIME tools for Galaxy (WIP, call for collaborators)
n addition to the output tar ball —> local, extract —> upload not being great, as you mention, the input datatypes, etc, could use some work — in the very least, we should definitely create a nice biom datatype and have some converters available (import and export). Definitely worth spending some extra time to make sure that we have the data flowing well between each of the different parts/tools, and even better to make sure that its done in a way that allows mixing and matching with other non-qiime tools. One thing that we want to avoid is large amounts of manual massaging of the automatically generated xml; fixing things up once might not be too bad, but having to do it with each new tool version can be “frustrating". Although perhaps having a good starting point and only needing to manually modify for any updates could be good enough (I’m not very familiar with the extent of typical changes between qiime versions to make a call on how much changes). Dan Lance Parsons <mailto:lpars...@princeton.edu> October 5, 2015 at 5:26 PM I was recently asked if I could provide a QIIME analysis pipeline for 16S data in Galaxy using tools in the QIIME pipeline (http://qiime.org/). I did a bit of looking around for existing Galaxy wrappers and found an application that generates the wrappers for QIIME scripts for Galaxy (https://github.com/qiime/qiime-galaxy). This is a very well written application that does a great job of wrapping the QIIME scripts for Galaxy. However, there are a few things about it that don't quite fit my needs. 1. The tools output tgz files of all of the output files. This means that to execute a pipeline, the user would have to download the tgz files, untar, and then upload whichever file(s) are needed for the next step. 2. There is no toolshed repository to install the dependencies for these tools, making it a tricky for administrators to automate and also maintain various versions of QIIME going forward. 3. There are no toolshed versions of the tools themselves, which also makes installation and integration a bit tricky and makes it hard to me to create and manage updates, fixes, tweaks, etc. There are also no tests, etc. For these reasons I decided to investigate the feasibility of using the generated wrappers as a basis for a "toolshed" version of QIIME. If anyone is interested in helping, or has suggestions, or is working on something related, I'd be very happy to collaborate. The repository for the WIP is at https://github.com/lparsons/galaxy_tools/tree/qiime/tools/qiime1.9.0. There is also a package on the testtoolshed as well as a first pass at package_qiime_1_9_1 (https://github.com/lparsons/galaxy_tools/tree/qiime/packages/package_qiime_1_9_1). ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Sebastian Schaaf, M.Sc. Bioinformatics Faculty Coordinator NGS Infrastructure Chair of Biometry and Bioinformatics Department of Medical Informatics, Biometry and Epidemiology (IBE) University of Munich DKTK Munich Marchioninistr. 15, K U1 808 D-81377 Munich (Germany) Tel: +49 89 4400 77499 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Galaxy Interactive Environment devs
Hey Eric, We are not yet on IEs, but we plan to upgrade our local production instance soon (a recent version is already 'waiting' to take over), apart from other goodies we intended to allow one pro user and ourselves to use iPython's power this way. So, do I read your message correctly as 'In terms of iPython it would be best for us to wait for v15.10'? Or could we just start and do an incremental upgrade as soon as v15.10 is released? Cheers to Texas, Sebastian Eric Rasche schrieb: Howdy y'all, I'm building some changes to the IE infrastructure that will fix some initial hacks we did, however they will be backwards incompatible. Specifically these changes will fix some of the enterprise deployment pains (running with an external upstream proxy like apache) and allow for safely running IEs on a separate host. I'm emailing the dev list to see if anyone else has started building IEs, or if it just Björn and myself. Feel free to make yourselves known to us so we know who to contact when we're planning new features :) If you're just building on top of our existing containers, this won't be an issue for you, you'll just need to rebuild your containers once we release 15.10 versions. Cheers, Eric -- Sebastian Schaaf, M.Sc. Bioinformatics Faculty Coordinator NGS Infrastructure Chair of Biometry and Bioinformatics Department of Medical Informatics, Biometry and Epidemiology (IBE) University of Munich DKTK Munich Marchioninistr. 15, K U1 808 D-81377 Munich (Germany) Tel: +49 89 4400 77499 ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
