Re: [galaxy-dev] 'galaxy-api-tools' project
On Mon, Aug 26, 2013 at 12:27 PM, Carlos Borroto
carlos.borr...@gmail.com wrote:
Hi,
I'm interested in developing a tool that would allow easy interaction
with the Galaxy API. I started coding and this[1] is what I have so
far.
[1]https://github.com/cjav/galaxy-api-tools
Second effort session on this project. The first time around I somehow
missed Bioblend python library, I made sure this is not the case this
time around. Bioblend is making development quite easy and is opening
a bunch of possibilities I didn't even know were possible.
The project is now also on pypi, all you need to install it is:
$ pip install galaxy-api-tools
This is a use case fully supported currently.
- List all workflows available to you:
$ galaxy-api-tools get-workflows
[ { u'id': u'f597429621d6eb2b',
u'model_class': u'StoredWorkflow',
u'name': u'Map and Variant Calling',
u'published': False,
u'tags': [],
u'url': u'/api/workflows/f597429621d6eb2b'},
{ u'id': u'f2db41e1fa331b3e',
u'model_class': u'StoredWorkflow',
u'name': u'Combine and Split',
u'published': False,
u'tags': [],
u'url': u'/api/workflows/f2db41e1fa331b3e'}]
- Show workflow details so you can clearly see what to expect and what
you need to provide:
$ galaxy-api-tools show-workflow name=Combine and Split
{ u'id': u'f2db41e1fa331b3e',
u'inputs': { u'5': { u'label': u'FASTA', u'value': u''},
u'6': { u'label': u'QUAL', u'value': u''},
u'7': { u'label': u'BARCODE_INDEX', u'value': u''}},
u'model_class': u'StoredWorkflow',
u'name': u'Combine and Split',
u'published': False,
u'steps': { u'5': { u'id': 5,
u'input_steps': { },
u'tool_id': None,
u'type': u'data_input'},
u'6': { u'id': 6,
u'input_steps': { },
u'tool_id': None,
u'type': u'data_input'},
u'7': { u'id': 7,
u'input_steps': { },
u'tool_id': None,
u'type': u'data_input'},
u'8': { u'id': 8,
u'input_steps': { u'fasta_file': {
u'source_step': 5,
u'step_output': u'output'},
u'qual_file': {
u'source_step': 6,
u'step_output': u'output'}},
u'tool_id': u'fastq_combiner',
u'type': u'tool'},
u'9': { u'id': 9,
u'input_steps': { u'barcode_index': {
u'source_step': 7,
u'step_output': u'output'},
u'input_format|inputs_0|input': { u'source_step': 8,
u'step_output': u'output_file'}},
u'tool_id': u'ngs-tools_split_by_barcode',
u'type': u'tool'}},
u'tags': [],
u'url': u'/api/workflows/f2db41e1fa331b3e'}
- Upload two pairs of FNA/QUAL files from a 454 run and run the
previous workflow for combining them into a FASTQ file and split them
by barcode:
$ galaxy-api-tools upload --library name=API Imports --upload-method
galaxy_filesystem \
--genome hg19 --folder Run 20130918 --workflow name=Combine and Split \
--add-file .qual=qual454=QUAL --workflow-input
BARCODE_INDEX=1cd8e2f6b131e891=ldda \
--workflow-param ngs-tools_split_by_barcode=barcodes=RL001,RL002 \
/local/seq_data/R_2013_09_18/1.fna /local/seq_data/R_2013_09_18/2.fna
{ u'history': u'0c5ffef6d88a1e97',
u'outputs': [u'1a5b83933dc4bf08', u'aebaa141e7243ebf']}
{ u'history': u'911dde3ddb677bcd',
u'outputs': [u'07a38ebd55a6989d', u'56fc5a09f8ae2546']}
There are a few issues I need to workout. Also, a lot of code
refactoring and commenting needs to happen if a seriously want other
people to join this effort. Ideally something like this could be built
inside Bioblend.
I hope someone else finds it useful,
Carlos
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
[galaxy-dev] datacache bowtie2 for mm9 ?
Folks, First, I wanted to thank you for making the datacache available (http://wiki.galaxyproject.org/Admin/Data%20Integration; rsync://datacache.g2.bx.psu.edu). It's a great resource. However, what is the best way to stay abreast of changes to what's in datacache, and understand how these indexes are computed? We are currently upgrading to bowtie2, but I notice that the bowtie2 indices for mm9, which used to be in rsync://datacache.g2.bx.psu.edu/indexes/mm9/mm9*/bowtie2_index have been removed, and only the hg19 genome has bowtie2 indices. Why only that one, and not the others? Where are the scripts you use to make these indices, in case I want to create bowtie2 indices for other So, how do I find out *why* they were removed? (Can I safely use the copy I have, or was there a problem with them?) More generally, how do I understand the policies and logic behind the datacache indices, and be notified of changes, short of running my own periodic rsync/diff? Finally, since I'm doing reproducible research is anything planned for systematically versioning genome indices, so I can easily tell what version of a system (ie, what BWA version) was used to create the index, and be sure that an index will not suddenly disappear. Thanks, Curtis Research Associate/CTSA-Informatics Team University of Alabama at Birmingham ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Unicode in tool stderr crashing galaxy?
Hi Simon,
it's just a guess but can you try the attached patch if you have a
reproducible case. It is not tested, but at least it should point you in
the right direction. If its working, I will create a pull request.
Cheers,
Bjoern
My optimism was short lived. I think I've hit a galaxy bug in Unicode
processing.
Having managed to get the deseq tool not to crash galaxy in the sqlalchemy
layer,
I now get a crash when clicking on the info-stderr for the tool output
dataset, probably because it contains Unicode. Here's the traceback from
paster.log:
Exception happened during processing of request from ('127.0.0.1', 47124)
Traceback (most recent call last):
File
/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py,
line 1068, in process_request_in_thread
self.finish_request(request, client_address)
File /usr/lib64/python2.6/SocketServer.py, line 322, in finish_request
self.RequestHandlerClass(request, client_address, self)
File /usr/lib64/python2.6/SocketServer.py, line 617, in __init__
self.handle()
File
/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py,
line 442, in handle
BaseHTTPRequestHandler.handle(self)
File /usr/lib64/python2.6/BaseHTTPServer.py, line 329, in handle
self.handle_one_request()
File
/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py,
line 437, in handle_one_request
self.wsgi_execute()
File
/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py,
line 290, in wsgi_execute
self.wsgi_write_chunk(chunk)
File
/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py,
line 150, in wsgi_write_chunk
self.wfile.write(chunk)
File /usr/lib64/python2.6/socket.py, line 316, in write
data = str(data) # XXX Should really reject non-string non-buffers
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2018' in
position 169: ordinal not in range(128)
I don't know where to look to find the actual stderr output. Any pointers?
Is this actually a galaxy bug now?
cheers,
Simon
===
Attention: The information contained in this message and/or attachments
from AgResearch Limited is intended only for the persons or entities
to which it is addressed and may contain confidential and/or privileged
material. Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by persons or
entities other than the intended recipients is prohibited by AgResearch
Limited. If you have received this message in error, please notify the
sender immediately.
===
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
diff -r e19c7819ea3c lib/galaxy/jobs/__init__.py
--- a/lib/galaxy/jobs/__init__.py Tue Sep 17 17:21:10 2013 +0200
+++ b/lib/galaxy/jobs/__init__.py Thu Sep 19 11:41:32 2013 +0200
@@ -21,6 +21,7 @@
from galaxy.jobs.actions.post import ActionBox
from galaxy.jobs.mapper import JobRunnerMapper
from galaxy.jobs.runners import BaseJobRunner
+from galaxy.util import unicodify
from galaxy.util.bunch import Bunch
from galaxy.util.expressions import ExpressionContext
from galaxy.util.json import from_json_string
@@ -1081,6 +1082,8 @@
# By default, the tool succeeded. This covers the case where the code
# has a bug but the tool was ok, and it lets a workflow continue.
success = True
+stdout = unicodify( stdout )
+stderr = unicodify( stderr )
try:
# Check exit codes and match regular expressions against stdout and
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] configuring auth_ldap for galaxy
Hey Srinivas,
I have no insights here, I am sorry. My knowledge of LDAP is rather limited.
-John
On Wed, Sep 18, 2013 at 9:10 AM, Maddhi, Srinivas
srinivas-mad...@uiowa.edu wrote:
Another galaxy + nginx + LDAP user here.
Iry,
I have it working as well with help from Galaxy Wiki docs and mailing list
archives. The configure options that were used to compile nginx:
--add-module=../nginx-auth-ldap.20130823
--add-module=../nginx-upload-module-2.2 --with-pcre=../pcre-8.33
--with-http_ssl_module --conf-path=/usr/local/etc/nginx/nginx.conf
--pid-path=/var/run/nginx.pid --error-log-path=/var/log/nginx/error.log
--http-log-path=/var/log/nginx/access.log --user=galaxy-qa
--group=galaxy-qa --with-debug
I also supplied the proxy_set_header X-URL-SCHEME https; directive in
the / and /api location blocks.
Good luck!
John,
Thanks for the detailed response to Iry's question, and docs for this
setup.
Related question: Are you aware of a mechanism that would transmit LDAP
attributes, say mail for example, to downstream apps ? I have that
question in to the author of that module and haven't heard back.
The need for that: in my use case, the email address associated with
remote_user is not remote_u...@example.com but rather
first_name-last_n...@example.com. As such, I am stuck, less than
ideally, with requiring users to use first_name-last_n...@example.com as
their username instead of just short_name. Our previous solution to this
was to hack galaxy-dist/lib/galaxy/web/framework/__init__.py which I would
like to avoid for ease of maintainability purposes.
Thank you.
-Original Message-
From: John Chilton chil...@msi.umn.edu
Date: Wednesday, September 18, 2013 7:49 AM
To: Iry Witham iry.wit...@jax.org
Cc: galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] configuring auth_ldap for galaxy
You have to compile nginx with the auth-ldap module. Have you done
this? I am doing using CloudBioLinux and here is how it is being done
in that context, but it would probably be easier just doing it by
hand:
https://github.com/chapmanb/cloudbiolinux/blob/master/cloudbio/galaxy/__in
it__.py
I have posted my information on configuring CloudMan's nginx for LDAP
auth, but you can ignore the CloudMan stuff:
https://production-galaxy-instances-with-cloudman-and-cloudbiolinux.readth
edocs.org/en/latest/#configuring-nginx-conf
http {
auth_ldap_url
ldap://ldap.example.com/dc=example,dc=com?uid?sub?(objectClass=person);
#auth_ldap_binddn cn=nginx,ou=service,dc=example,dc=com;
#auth_ldap_binddn_passwd mYsUperPas55W0Rd
#auth_ldap_group_attribute uniquemember; # default 'member'
#auth_ldap_group_attribute_is_dn on; # default on
...
}
location / {
auth_ldap_require valid_user;
auth_ldap LDAP Auth Source Description;
proxy_set_header REMOTE_USER $remote_user;
proxy_pass http://galaxy_app;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-URL-SCHEME https;
...
}
# For API access, set REMOTE_USER if available so Galaxy
# session based requests are let through, if REMOTE_USER is not
# available pass the request through and let Galaxy determine
# if a key is present and valid.
location /api {
proxy_set_header REMOTE_USER $remote_user;
proxy_pass http://galaxy_app;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
-John
On Wed, Sep 18, 2013 at 7:37 AM, Iry Witham iry.wit...@jax.org wrote:
Hi Team,
Has anyone had any experience with setting up auth_ldap for nginx? I am
attempting to make this work, but nginx fails to start with the
following
message:
Starting nginx nginx: [emerg] unknown directive auth_ldap_url in
/usr/local/nginx/conf/nginx.conf:48
startproc: exit status of parent of /usr/local/nginx/sbin/nginx: 1
failed
Any help would be appreciated.
Thanks,
Iry Witham
The information in this email, including attachments, may be
confidential
and is intended solely for the addressee(s). If you believe you received
this email by mistake, please notify the sender by return email as soon
as
possible.
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Towards Galaxy Linux (not) ? [was [RFC] Storing of tarballs and patches for tool_dependencies to enable reproducibility]
Hi Simon, thank you very much for your comments! I can see man years of effort being spent on solving this problem within Galaxy. I was going to title this email Danger, Will Robinson, but I didn't want to be disrespectful. I think the path being embarked upon, tool dependency packaging, tool versioning, reproducibility, and long term archive of source tarballs is going to lead inevitably to creation of a new Linux distribution, which I guess will be called Galaxy Linux. I'm not sure it is comparable to a entire Linux distribution, its more like an Appstore, like pypi, bioconductor or gems, and yes that is reinvented somehow. I want to point out, that the pool of bioinformatic applications are not so huge compared to an entire linux distribution and that many of them exists as pre-compiled binaries, which makes everything easier. The packaging and archival you are talking about is exactly the service provided by a Linux distribution. Sorry maybe I was misleading. I only want a central storage for binaries/tarballs where the source can not be trusted for long term. 'long term' and 'trusted' needs to be defined in such a discussion here. I do not think we should copy python packages that are stored in pypi. We should make it easy as possible to install them in our repository. If you do not trust pypi, we can offer a mirror. Some goes for gems. But what about packages that do not store different versions of packages? We should have a central place to store them. UCSC tools for example. Easy to install, but we need to store them somewhere. There's well established infrastructure to handle this, and years of experience have gone into solving the problems well. Sure, we can learn from them, or use them. Surely the number of Linux distributions in the world now exceeds 100, but I don't see that the world will become a better place if we increase that number by one more. I'm not talking about a new linux distribution. Galaxy is running everywhere, RHEL, OS-X, SUSE, what ever is used in the Amazon Cloud and we need to run Galaxy on top of that. We at AgResearch can't be alone in having to pick a Linux distribution to run from the short list supported by our hardware vendor. I can't see Galaxy Linux being on that list anytime soon. So we have to make Galaxy run on the particular distribution we have here. For us that's CentOS 6. Sure, agree. Now, I see scary mention of platform independence as a goal for Galaxy packaging, which I interpret as will run on any Linux distribution. I think that's essentially infeasible. I hope not :) We should define a minimal subset of dependencies a Galaxy system needs. Python, libz, gccX.Y, libfortran and so on, that's it. That can be understood as some kind of abstraction layer. If your distribution can offer it, Galaxy is supported, otherwise take care of such an abstraction layer for your system. All you can do is write install scripts which you hope are portable (by following as many best practices as you know about), and then work patiently with users on strange platforms, to adapt each install script to work on that platform also. I think this is not a good use of anyone's time. I see your point, but as we support a minimal subset of requirements that argument does not hold. Moreover, I do not expect that issue in the Galaxyland. We are dealing with professional administrators/bioinformaticians running on large clusters, not with desktop users. I hope the set of different distributions that are really in use are minimalistic. How many Linux distributions do the Galaxy community actually care about today? The RHEL family is surely important, as is Ubuntu LTS. Anything else? Maybe a few Solaris Systems and do not forget OS-X. I'd be quite interested to understand this, as it provides a context for the discussion, and ensures we're not just solving a hypothetical problem. I'm just starting work on a native packaging infrastructure for Galaxy, that will enable tool dependencies to use defined versions of natively installed packages. That frees me up to make my packages work nicely on the RHEL family. It looks like the RPMs themselves (including SRPMs obviously) will be hosted by the CentOS project before too long. Once they're there, they can easily be archived forever. Anyone else on that platform is welcome to use the same infrastructure. Then, all we really need is someone to handle the packaging effort for the other major Linux distributions (a small number, I hope), and the problem is essentially solved. Sure, but the problem is not solved, or? It's just transferred from your Linux metaphor to 'packaging formats'. How many different packaging formats we have ... do we need a new one ... Getting the Bio-Linux team interested in multi-version packaging would be a great next step. I really think that is the important part! We need to convince and cooperate
Re: [galaxy-dev] pass user groups to dynamic job runner
Feature request received. The following changeset demonstrates how one could implement this https://github.com/jmchilton/galaxy-central/commit/b500a24bc1aa94bef48db20a7cc1f3d68855b7fd First step is to create a new dynamic job destination, this is demonstrated in job_conf.xml in the above changeset. This demonstrates the new style job_conf section - this will need to be adapted for the old style runners but is still doable. Then you will want to add a dynamic job runner rule that implements this logic. This is the file lib/galaxy/jobs/rules/license_checker.py. This will do what you asked, you can adjust the tools it targets and the message that gets displayed to the user pretty easily (right now it is just No license, no tool). There is no longer any need to display such tools to the user thanks to dynamic toolbox filters. This is that last file: lib/galaxy/tools/filters/license_filter.py. This will prevent any tool in the list 'LICENSED_TOOLS' from even being seen by unlicensed users. You do need to specify this filter is being used by adding the line: tool_filters = license_filter:has_license in the [app:main] section of universe_wsgi.ini. Hope this helps! -John On Thu, Sep 19, 2013 at 2:45 AM, Vandeweyer Geert geert.vandewe...@uantwerpen.be wrote: Hi, Could somebody point me to the place where I can create a ticket for the following feature: - We want to have tools available only to users who provided a licence for this tool. - To prevent very long email lists (see example on dev-only tools), I'd like to have a group 'have_licence' - in dynamic job runner function : check if user is in usergroup : ok = run ; fail = give message. Or can I hide tools from the menu based on usergroup? Best, Geert ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] testtoolshed.g2.bx.psu.edu not compatible with galaxy-central stable
my galaxy instance: parent: 10587:1f34ec186fcf Pack script. branch: stable commit: 3 modified, 91 unknown update: (current) testtoolshed.g2.bx.psu.edu: parent: 10592:ea0c57f1cdf0 tip Fix for unicode values in rst_to_html. branch: default commit: 34 unknown (clean) update: (current) Try to install this repository: http://testtoolshed.g2.bx.psu.edu/view/rico/genome_diversity URL: https://oocyte.bx.psu.edu/admin_toolshed/prepare_for_install? tool_shed_url=http://testtoolshed.g2.bx.psu.edu/ repository_ids=d43c67507430e220changeset_revisions=b63d27fdf040 File '/scratch/galaxy/home/oocyte/galaxy_oocyte/eggs/WebError-0.8a- py2.7.egg/weberror/evalexception/middleware.py', line 364 in respond app_iter = self.application(environ, detect_start_response) File '/scratch/galaxy/home/oocyte/galaxy_oocyte/eggs/Paste-1.7.5.1- py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/scratch/galaxy/home/oocyte/galaxy_oocyte/lib/galaxy/web/ framework/middleware/remoteuser.py', line 91 in __call__ return self.app( environ, start_response ) File '/scratch/galaxy/home/oocyte/galaxy_oocyte/eggs/Paste-1.7.5.1- py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/scratch/galaxy/home/oocyte/galaxy_oocyte/lib/galaxy/web/ framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/scratch/galaxy/home/oocyte/galaxy_oocyte/lib/galaxy/web/ framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/scratch/galaxy/home/oocyte/galaxy_oocyte/lib/galaxy/web/ framework/__init__.py', line 221 in decorator return func( self, trans, *args, **kwargs ) File '/scratch/galaxy/home/oocyte/galaxy_oocyte/lib/galaxy/webapps/ galaxy/controllers/admin_toolshed.py', line 911 in prepare_for_install includes_tool_dependencies ) File '/scratch/galaxy/home/oocyte/galaxy_oocyte/lib/tool_shed/util/ common_install_util.py', line 85 in get_dependencies_for_repository installed_rd, missing_rd = get_installed_and_missing_repository_dependencies_for_new_install ( trans, repo_info_tuple ) File '/scratch/galaxy/home/oocyte/galaxy_oocyte/lib/tool_shed/util/ common_install_util.py', line 214 in get_installed_and_missing_repository_dependencies_for_new_install tool_shed, name, owner, changeset_revision, prior_installation_required = suc.parse_repository_dependency_tuple ( rd_tup ) File '/scratch/galaxy/home/oocyte/galaxy_oocyte/lib/tool_shed/util/ shed_util_common.py', line 1197 in parse_repository_dependency_tuple prior_installation_required = str( prior_installation_required ) UnboundLocalError: local variable 'prior_installation_required' referenced before assignment contains_error = False repository_dependency_tuple = ['http://testtoolshed.g2.bx.psu.edu/', 'package_clapack_3_2_1', 'rico', '56a949e5f998', 'True', 'False'] -rico ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Test Toolshed Biopython package dependency Atlas fails to install (Was: Re: UnboundLocalError: local variable 'prior_installation_required' referenced before assignment)
On Thu, Sep 19, 2013 at 5:15 PM, Björn Grüning bjoern.gruen...@pharmazie.uni-freiburg.de wrote: Hi Peter and Carlos, On Mon, Sep 16, 2013 at 8:57 PM, Carlos Borroto carlos.borr...@gmail.com wrote: I did an extra test. Started with a clean 'galaxy-dist'. This time both repositories fail with the same error. I guess before something was cached for the repository with version 0.1.4. I used biopython repository as a guide to write my tool dependency definition: http://testtoolshed.g2.bx.psu.edu/view/biopython/package_biopython_1_61 I can confirm biopython repository is failing to install for me with exactly the same error. I wonder if a recently addition in the test toolshed broke the treatment of prior_installation_required. Thanks, Carlos Could be - note that currently Biopython isn't currently installing properly on the Test Tool Shed due to ALTAS failing (a requirement of NumPy which is a requirement of Biopython). Dave and Bjoern are I think looking at this already... I can't do much I tested it again and for me its working fine on my computers I have at hand ... sorry. In case it helps, this is how the INSTALLATION.log file ends on OS X 10.8.4: $ tail -n 3 ~/src/shed_tools_dependencies.central/atlas/3.10.1/iuc/package_atlas_3_10/3508de0ebae1/INSTALLATION.log x ATLAS/tune/lapack/lanbsrch.c tar: Error exit delayed from previous errors. # This is the relevant part I can find in Galaxy's log: [localhost] local: tar xfvj atlas3.10.1.tar.bz2 Warning: local() encountered an error (return code 1) while executing 'tar xfvj atlas3.10.1.tar.bz2' After noticing this I got what I'm guessing is the original file from sourceforge: http://downloads.sourceforge.net/project/math-atlas/Stable/3.10.1/atlas3.10.1.tar.bz2 I can confirm that trying to untar this file fails with the exact same error. However, on Ubuntu 13.04 untaring this file works just fine. On Ubuntu 13.04 the error I see is: STDERR It appears you have cpu throttling enabled, which makes timings unreliable and an ATLAS install nonsensical. Aborting. See ATLAS/INSTALL.txt for further information # Björn, you said cpu throttling can be easily disable on Ubuntu. Can you comment how? Do I need to disable it completely, or should the install script do it when installing? Thanks, Carlos ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Access to history_id from tool config XML
It would be useful to have access $__history_id__ in the same way as one accesses $__user_id__. I am running a local instance of galaxy and have modified the following file to achieve this: lib/galaxy/jobs/__init__.py At line ~694: incoming['__user_name__'] = user_name + if job.history and job.history.id: + incoming['__history_id__'] = job.history.id + else: + incoming['__history_id__'] = 'unknown' I have tested this change and it appears to give me exactly what I want. My question: does this change appear correct and can it be incorporated into the main galaxy code-base? Thanks, -tim ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] pass user groups to dynamic job runner
Hi, Could somebody point me to the place where I can create a ticket for the following feature: - We want to have tools available only to users who provided a licence for this tool. - To prevent very long email lists (see example on dev-only tools), I'd like to have a group 'have_licence' - in dynamic job runner function : check if user is in usergroup : ok = run ; fail = give message. Or can I hide tools from the menu based on usergroup? Best, Geert ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] August 2013 Galaxy Update Newsletter is out
Hi Samuel, everyone, There is now a Vimeo channel containing all 25 taped talks from GCC2013: https://vimeo.com/channels/581875 We're still figuring out how to best use Vimeo. Please do let us know if you have any suggestions for how to improve the site. Look for more content here going forward. And it's on a Vimeo Plus account, so you won't see any ads for phones, football teams, telescopes, or faster than light travel. :-) Dave C On Mon, Aug 12, 2013 at 12:28 PM, Dave Clements cleme...@galaxyproject.orgwrote: Hi Samuel, everyone, Our main reason for avoiding these ad supported services is that we don't want people to have to deal with ads to see Galaxy Video content. Dave C On Sat, Aug 10, 2013 at 2:54 AM, Samuel Lampa samuel.la...@gmail.comwrote: On 08/08/2013 07:38 PM, Dave Clements wrote: I wanted to ask though, are there specific reasons for not using a service like YouTube / Vimeo etc? Yes! Um, but I can't remember what they are (sadly, this is true). We'll talk about this on the next outreach call, and I'll post here what we decide. Ok, nice, many thanks! Best Regards // Samuel -- http://galaxyproject.org/GCC2013 http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://wiki.galaxyproject.org/ -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://wiki.galaxyproject.org/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Towards Galaxy Linux (not) ? [was [RFC] Storing of tarballs and patches for tool_dependencies to enable reproducibility]
If I might chime in, I am a bit worried about all the automatic installation going on in galaxy, and it seems that the trend is to enhance this. A small R or python script calling into well known libraries that come from well known repositories (bioconductor etc… ) I can check. (Of course I install too much stuff from github, bioconductor etc… without checking). I'm not sure it is comparable to a entire Linux distribution, its more like an Appstore, like pypi, bioconductor or gems, and yes that is The app stores are checked by Apple or google for malicious code, the apps are sandboxed. There are many eyes for python, bioconductor packages and gems because much more people interact with them directly compared to galaxy-tools. Sorry maybe I was misleading. I only want a central storage for binaries/tarballs where the source can not be trusted for long term. 'long term' and 'trusted' needs to be defined in such a discussion here. I do not think we should copy python packages that are stored in pypi. We should make it easy as possible to install them in our repository. If you do not trust pypi, we can offer a mirror. Some goes for gems. Trusted for me means I trust the source not having dangerous code. I trust pypi more than some mirror, bioconductor base packages from more than some freshly published package that few people have used, tools from galaxy core developers more than from tool-shed etc… I know this is not the type of trust you were talking about. best, ido ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] cuffdiff wrapper not synchronized with cuffdiff version
Jeremy First of all, thanks for all your efforts, and for updating the cuffdiff, etc, wrappers. However, it's a little frustrating to us that a non-backward compatible change in the supported version of the cuffdiff executable was not reflected in a new version number for the cuffdiff (and other cuff) wrappers. This leaves us no way to support both versions of cuffdiff, and makes all histories that were run with the older version non-reproducible. I know the general versioning problem is address through the tool-shed mechanism, but for the remaining core wrappers, can maintainers please update wrapper version numbers when making changes? For this wrapper set, how would you advise us to support both? It seems to me that I HAVE to leave the cuffdiff v1.3-compatible as wrapper_version=0.0.5, or old histories/workflows break. However, if I change the cuffdiff_v2.1-compatible wrapper to something else, I'll be out of sync with galaxy-central, and will create migration/reproducibility issues in the future. Would you accept a pull request to update the version number of the cuff wrappers that were changed in the migration to cuff*2.1? Thanks again for updating the wrappers, and help me figure out how to support them cleanly. Regards, Curtis From: galaxy-dev-boun...@lists.bx.psu.edu [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Jeremy Goecks Sent: Wednesday, June 12, 2013 3:09 PM To: Karen Chait Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] cuffdiff wrapper not synchronized with cuffdiff version Hello, After the latest update of Galaxy I saw that the cuffdiff wrapper is for version 2.0. I updated the $PATH to cuffdiff v. 2.0.2 but when I run cuffdiff in Galaxy I get the following error: Error running cuffdiff. cuffdiff: /lib64/libz.so.1: no version information available (required by cuffdiff) cuffdiff: unrecognized option '--library-norm-method' cuffdiff v1.3.0 (3022) What else do I need to define inorder to run the latest version of cuffdiff. This depends on your setup. If you're running Galaxy under a particular user, you may have to update that user's $PATH. If you're running on a cluster, you'll need to ensure that the submission script uses the correct $PATH. If you provide more details on your set up, it may be possible to provide additional insight/suggestions for your problem. Best, J. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] cuffdiff wrapper not synchronized with cuffdiff version
Jeremy Thanks for the super-quick response. That should resolve things nicely. We can hack the old wrapper back in, using a hard-coded path to the old cuffdiff. We can pull your new wrapper, and install the new cuffdiff. Then we should have backward and forward compatibility. Shantanu suggests it might be a better world if we create a tool-shed repository for the cuffdiff 1.3-2.1 (wrapper 0.0.5) version, then install that. Would that be a better path, or just add confusion? Regards, Curtis From: Jeremy Goecks [mailto:jeremy.goe...@emory.edu] Sent: Thursday, September 19, 2013 2:18 PM To: Curtis Hendrickson (Campus) Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] cuffdiff wrapper not synchronized with cuffdiff version I know the general versioning problem is address through the tool-shed mechanism, but for the remaining core wrappers, can maintainersplease update wrapper version numbers when making changes? Not bumping wrapper versions is an oversight, not deliberate. For this wrapper set, how would you advise us to support both? Fortunately, I refactored the Cuffdiff wrapper recently and correctly bumped the tool version: https://bitbucket.org/galaxy/galaxy-central/commits/6a32d26aefc758cf576e0b753c2626ac56005c18 You can either pull this changeset manually or wait until the next distribution in early October to get the changeset. Best, J. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] cuffdiff wrapper not synchronized with cuffdiff version
I know the general versioning problem is address through the tool-shed mechanism, but for the remaining core wrappers, can maintainersplease update wrapper version numbers when making changes? Not bumping wrapper versions is an oversight, not deliberate. For this wrapper set, how would you advise us to support both? Fortunately, I refactored the Cuffdiff wrapper recently and correctly bumped the tool version: https://bitbucket.org/galaxy/galaxy-central/commits/6a32d26aefc758cf576e0b753c2626ac56005c18 You can either pull this changeset manually or wait until the next distribution in early October to get the changeset. Best, J.___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Part Time Job opening at GenePeeks
Hi everyone, I am posting this position description for a Part Time - Galaxy (BioLinux) Server Administrator / Python Programmer. We are a really great company in an interesting space. Please check out this video for a good overview of what we do. (We also have some full time openings that range from computational geneticist to senior scientist, if you are interested, send me an e-mail and I will send you more information.) GenePeeks: Protecting our Children http://vimeo.com/68860139 Company GenePeeks is a genetic information company with a mission to help families protect the next generation. Our patent-pending technology reduces the risk of more than 600 severe, pediatric diseases by digitally weaving together the DNA of prospective parents and analyzing thousands of digital babies pre-pregnancy. Our initial focus is the donor sperm industry, where we help sperm bank clients avoid donor matches that carry a high risk of passing on life-threatening and life-altering diseases. GenePeeks was founded by Princeton genetics professor Lee Silver and Anne Morriss, an entrepreneur and mother whose first child was born with a rare disease that could have been identified with GenePeeks technology. For more information, please visit genepeeks.com. GenePeeks is a unique genetic information company with a mission to help families protect the health of their future children. We are searching for a Part Time Galaxy (BioLinux) Server Administrator to join our growing team and help develop leading-edge technologies in a fast-paced, early-stage environment. Description: Manage and maintain several cloud based BioLinux instances running Galaxy on Amazon Web Services Manage and maintain a source code control baseline on BitBucket Manage and maintain internal wiki site using mediawiki Help install and maintain new cloud based technologies as required Perform code reviews of Python scripts Assist in organizing the code base and proper source code Manage backup and disaster recovery planning Requirements Must be fast learning and responsive Must have experience using BioLinux Must have Python coding experience Must have experience using BitBucket Background in Genetics or Molecular Biology is a plus Sense of humor required Details Position is part time and will bill hourly Hours will fluctuate (estimate about 10 hours a week) Location in the New York area is a plus, but will consider remote working arrangements for the right candidate Hourly Rate Commensurate with experience and efficiency If you are interested please contact me directly at adamc...@genepeeks.com -- Adam S. Cohn Chief Technology Officer / Chief Operations Officer GenePeeks 175 Varick Street, New York, NY (646) 664-4268 (703) 473-8811 (mobile) adamc...@genepeeks.com www.genepeeks.com ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] cuffdiff wrapper not synchronized with cuffdiff version
We can hack the old wrapper back in, using a hard-coded path to the old cuffdiff. You could also use Managed Tool Dependencies for multiple versions of cuffdiff: http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies Shantanu suggests it might be a better world if we create a tool-shed repository for the cuffdiff 1.3-2.1 (wrapper 0.0.5) version, then install that. Would that be a better path, or just add confusion? This is on the right path. The right thing to do is migrate the Cuff* tool wrappers out of the galaxy-central/dist repositories; this is a bit more effort than simply creating the tool-shed repository for the wrappers. There are two reasons I haven't done this: (a) time; (b) because the tools—and consequently the wrappers—are still in active development, there needs to be a place for managing development of the wrappers. Hence, the wrappers need a home on bitbucket or github, but we (the Galaxy team) haven't determined where these wrappers should live. (a) is a much more significant barrier than (b), so if a pull request migrating Cuff* tools out of the distribution were made, I'm sure (b) could be worked out. Best, J. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
