Re: [galaxy-dev] Tool shed environment dependency - HOW TO do this?

2014-11-06 Thread Nicola Soranzo

Hi Pieter,
you should probably add:

action type=set_environment_for_install
  repository name=prims_metabolomics_r_dependencies 
owner=pieterlukasse

package name=R_bioc_metams version=3.1.1 /
  /repository
/action

before using the R_ROOT_DIR environment variable.

Best,
Nicola

Il 2014-11-06 11:28 Lukasse, Pieter ha scritto:

Hi Bjoern,

I tried your suggestion, but it didn't work. So now I have split up
the R part from the Bioconductor part (blue and yellow below,
respectively). But the problem is that the install part executes
before the package part is finished and the $R_ROOT_DIR reference
also doesn't seem to work as I get the following error :  /bin/sh: 
1:

/bin/Rscript: not found  . This R_ROOT_DIR is set by the package
script in my other tool_dependency.xml but is not visible in the
tool_dependency.xml below. But maybe I misunderstood what you meant 
by

You can now access R_ROOT_DIR from any other tool_dependency file
...?

tool_dependency

!-- see also http://wiki.galaxyproject.org/ToolShedToolFeatures for
syntax help

 --

 package name=R_bioc_metams version=3.1.1

 repository changeset_revision=0af753942e7b
name=prims_metabolomics_dependencies owner=pieterlukasse
prior_installation_required=True
toolshed=http://testtoolshed.g2.bx.psu.edu; /

 install version=1.0

 actions_group

 !-- the Bioconductor and metaMS part --

 action type=shell_commandwget -P $INSTALL_DIR

http://testtoolshed.g2.bx.psu.edu/repos/pieterlukasse/prims_metabolomics/raw-file/tip/INSTALL.r/action


 action type=shell_command$R_ROOT_DIR/bin/Rscript
$INSTALL_DIR/INSTALL.r/action

 /actions_group

 /install

 /package

 readme

 This dependency:

 Ensures R 3.1.1 installation is triggered (via dependency).

 Ensures Bioconductor 3.0 and package metaMS, multtest and snow are
installed.

 /readme

/tool_dependency

Thanks,

Pieter

-Original Message-
 From: galaxy-dev-boun...@lists.bx.psu.edu
[mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Lukasse,
Pieter
 Sent: donderdag 6 november 2014 10:02
 To: 'Björn Grüning'; galaxy-dev@lists.bx.psu.edu; Dave Bouvier
 Subject: Re: [galaxy-dev] Tool shed environment dependency - HOW TO
do this?

Hi Bjoern,

Thanks for this reply. I will have to try referring to the R_ROOT_DIR
from another tool_dependency. If it doesn't work I'll come back ;)

Regards,

Pieter

-Original Message-

From: Björn Grüning [mailto:bjoern.gruen...@gmail.com [1]]

Sent: donderdag 30 oktober 2014 11:25

To: galaxy-dev@lists.bx.psu.edu [2]; Lukasse, Pieter; Dave Bouvier

Subject: Re: [galaxy-dev] Tool shed environment dependency - HOW TO 
do

this?

Hi Pieter,

Am 30.10.2014 um 11:15 schrieb Lukasse, Pieter:


Hi,







I have a Bioconductor package which depends on R 3.1.1 , so I think

I cannot use the setup_r_environment trick.

Why? Do you need a new release? The latest current release in the 
Tool
Shed is R 3.1.0. If this is not enough we need to poke Dave to build 
a

new package. But this way would be the preferred way.


I already got a tool_dependency.xml working that installs R 3.1.1

and the necessary Bioconductor packages using bioclite method (see
attachment). Now my question is:










* I would like to split up this into two steps as I don't want to

trigger the compilation of new R environment every time I when I need
to just update the


Bioconductor packagethe question is: how to do such things in

general? How can I access the INSTALL_DIR of another tool from within
another tool_dependency.xml? If I can do this, then my problem is
solved.

If you really want to build your R packages by your own. Have a look
at

https://github.com/bgruening/galaxytools/blob/master/packages/package_r_3_0_3/tool_dependencies.xml
[3]

You will find an exhaustive installation instruction how to build R
properly. Also note that we export a variable called R_ROOT_DIR that
is set to $INSTALL_DIR. You can now access R_ROOT_DIR from any other
tool_dependency file ...

Hope this helps,

Bjoern

P.S. Do you mind asking this question on biostar again, I think this
is a very nice question that can be of interest to a lot more people.


Thanks!







Pieter Lukasse



Wageningen UR, Plant Research International Department of



Bioinformatics (Bioscience) Wageningen Campus, Building 107,



Droevendaalsesteeg 1, 6708 PB, Wageningen, the Netherlands



T: +31-317481122;



M: +31-628189540;



skype: pieter.lukasse.wur



http://www.pri.wur.nlhttp://www.pri.wur.nl/ [4]



















___



Please keep all replies on the list by using reply all



in your mail client. To manage your subscriptions to this and other



Galaxy lists, please use the interface at:



http://lists.bx.psu.edu/ [5]







To search Galaxy mailing lists use the unified search at:



http://galaxyproject.org/search/mailinglists/ [6]






___

Please keep all replies on the list by using reply all


Re: [galaxy-dev] Data uploaded with new upload tool doesn't get added to history

2014-10-10 Thread Nicola Soranzo
Hi Dannon,
so the very last line (set $dst /galaxy/tool_runner/index;) on the
wiki page https://wiki.galaxyproject.org/Admin/Config/nginxProxy is
still wrong?

Ciao,
Nicola

Il giorno ven, 10/10/2014 alle 08.50 -0400, Dannon Baker ha scritto: 
 Just to follow up on this thread -- we resolved this over IRC.  It was a
 proxy configuration issue with nginx's upload_config settings.
 
 Correct settings are now on the wiki, but the previous rule for:  location
 _upload_done : set $dst /tool_runner/index; is was causing a redirect and
 preventing communication with the tools API, which the new upload form
 works.
 
 -Dannon
 
 
 On Thu, Oct 9, 2014 at 3:00 PM, Peter van Heusden p...@sanbi.ac.za wrote:
 
  Hey there
 
  Using the new upload tool my data uploads but never appears in the
  history. All I can see in paster.log when I do an upload is:
 
  10.8.0.2 - - [09/Oct/2014:20:43:58 +0200] POST /tool_runner/index
  HTTP/1.0 200 - https://phillipl.sanbi.ac.za/root; Mozilla/5.0 (X11;
  Ubuntu; Linux x86_64; rv:32.0) Gecko/20100101 Firefox/32.0
  10.8.0.2 - - [09/Oct/2014:20:43:58 +0200] GET
  /api/histories/f597429621d6eb2b/contents HTTP/1.0 200 -
  https://phillipl.sanbi.ac.za/root; Mozilla/5.0 (X11; Ubuntu; Linux
  x86_64; rv:32.0) Gecko/20100101 Firefox/32.0
  10.8.0.2 - - [09/Oct/2014:20:43:58 +0200] GET
  /api/histories/f597429621d6eb2b HTTP/1.0 200 -
  https://phillipl.sanbi.ac.za/root; Mozilla/5.0 (X11; Ubuntu; Linux
  x86_64; rv:32.0) Gecko/20100101 Firefox/32.0
 
  No errors in the javascript console, and this happens with both Chrome
  and Firefox (on Linux), code is latest galaxy-central. Any ideas where
  to debug?
 
  Thanks,
  Peter
 
  ___
  Please keep all replies on the list by using reply all
  in your mail client.  To manage your subscriptions to this
  and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
 
  To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
 
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
   http://lists.bx.psu.edu/
 
 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


Re: [galaxy-dev] Galaxy Bioblend option for importing dataset into a library?

2014-10-06 Thread Nicola Soranzo
Il giorno gio, 02/10/2014 alle 16.50 -0700, Dooley, Damion ha scritto:
 I see Galaxy API has a feature to import a history dataset into the
 library (in copy_hda_to_ldda() fn from GCC2013 training day course).
 Is this available as well via Bioblend? Latest docs don't seem to
 include this feature. It would be the opposite of Bioblend's
 upload_dataset_from_library(history_id, lib_dataset_id) )

Hi Damion,
indeed this feature is not in BioBlend yet, but I have it in my TODO
list because we also need it. Hopefully I'll soon have some time to
implement the method, test it and open a PR.

Best,
Nicola

 Objective is to get customized blast indexes into library that way for
 shared use. Or have them actually exist outside galaxy, and linked
 in. 
  
 Regards,
 
 Damion
 
 Hsiao lab, BC Public Health Microbiology  Reference Laboratory, BC
 Centre for Disease Control
 655 West 12th Avenue, Vancouver, British Columbia, V5Z 4R4 Canada
 
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
   http://lists.bx.psu.edu/
 
 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


Re: [galaxy-dev] Tool Shed Categories

2014-10-06 Thread Nicola Soranzo
Hi Khalid,
I'd suggest you to open a Trello card for this:

http://galaxyproject.org/trello/

Best,
Nicola

Il giorno ven, 03/10/2014 alle 17.30 +, Alam, Khalid K. (MU-Student)
ha scritto: 
 Hi;
 
 I’m getting ready to submit several (5) tools for processing data
 related to combinatorial selection technologies (aptamer and
 (deoxy)ribozyme selections, phage display, etc.) and would like to
 propose a new category for the Tool Shed entitled “Combinatorial
 Selections” where these repositories can be published. 
 
 Khalid Kamal Alam
 University of Missouri
 Department of Biochemistry
 415 Life Sciences Center
 1201 Rollins St.
 Columbia, MO 65211
 mailto:khalidka...@mail.missouri.eduKhalidKAlammailto:khalidka...@mail.missouri.edu@mail.missouri.edumailto:kka...@mail.missouri.edu
  
 @BiochemPhD
 
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
   http://lists.bx.psu.edu/
 
 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Updated Freebayes Wrapper - Bugs

2014-07-16 Thread Nicola Soranzo

Il 2014-06-30 18:40 Anton Nekrutenko ha scritto:

Lance:

Here is github URL:

https://github.com/nekrut/freebayes [3]

 On Wed, Jun 25, 2014 at 2:40 PM, Lance Parsons
lpars...@princeton.edu [4] wrote:


Thanks for the major update of the Freebayes wrapper, excellent!

I've run into two issues, however.

1) When using set allelic scope I get the following error:

Fatal error: Exit code 1 ()
freebayes: unrecognized option `--min-repeat-length'
did you mean --min-repeat-size ?


This bug (and other 2) would be fixed by this pull request:

https://github.com/nekrut/freebayes/pull/1

Best,
Nicola


2) When using a vcf file as input I get the following error:

Fatal error: Exit code 1 ()
open: No such file or directory
[bgzf_check_bgzf] failed to open the file: input_variant_vcf.vcf.gz
[tabix++] was bgzip used to compress this file?
input_variant_vcf.vcf.gz

I'd be happy to help resolve these.  Is there a bitbucket repo
somewhere to submit pull requests?

--
Lance Parsons - Scientific Programmer
134 Carl C. Icahn Laboratory
Lewis-Sigler Institute for Integrative Genomics
Princeton University

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/ [1]

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/ [2]


--

Anton Nekrutenko
Associate Professor
Dept. of Biochemistry and Molecular Biology
http://nekrut.bx.psu.edu [5]
 (814) 826-3051

Links:
--
[1] http://lists.bx.psu.edu/
[2] http://galaxyproject.org/search/mailinglists/
[3] https://github.com/nekrut/freebayes
[4] mailto:lpars...@princeton.edu
[5] http://nekrut.bx.psu.edu


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] fix for GATK2 wrapper from IUC

2014-06-27 Thread Nicola Soranzo

Hi Geert,
thanks for the bug report, I've fixed the bug upstream:

https://github.com/bgruening/galaxytools/commit/cab6d1e84d440b7c73e99adc2bcb2c3c64c5b2c5

Björn, can you release an update on the Tool Shed? I think we have 
accumulated a lot of fixes for GATK2 wrappers already.


Nicola

Il 2014-06-27 10:15 Geert Vandeweyer ha scritto:

Hi,

There is a missing parameter in the unified genotyer config from the
iuc gatk2 wrapper:
The following line should be added around line 204 (just before
/expand/inputs):

param name=sample_ploidy type=integer value=2
label=Sample Ploidy (number of allles). For tumour, set to 2x the
number of expected subclones help=-ploidy,--sample_ploidy /

Without this line, activating the advanced analysis options causes
the job preparation to fail because there is a reference to it in the
command section


gatk2 wrapper used:
version : 8bcc13094767
hg tip:
changeset:   3:2553f84b8174
tag: tip
user:iuc
date:Wed Feb 19 04:39:38 2014 -0500
summary: Uploaded


Best,

Geert

--

Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail: geert.vandewe...@ua.ac.be
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/in/geertvandeweyer

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] monitoring jobs start time

2014-05-19 Thread Nicola Soranzo

Il 2014-05-19 04:47 neil.burd...@csiro.au ha scritto:

Hi,
 I am currently using the reports tool to see what jobs have been
executed etc ...

 When you look at the information for each job it states the time the
job was put on the queue (creation time) not the actual start time of
the job. Do you know if the start time of the job is stored 
somewhere?


Hi Neil,
take a look at this pull request, which was merged recently in 
galaxy-central and will be available in the next Galaxy release:


https://bitbucket.org/galaxy/galaxy-central/pull-request/352/implement-plugin-framework-and-plugins-for/diff

Best,
Nicola
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/


Re: [galaxy-dev] Jobs deleted staying in 'dr' status

2014-04-10 Thread Nicola Soranzo

Il 2013-10-11 17:21 Sytchev, Ilya ha scritto:

On 9/12/13 10:35 AM, Peter Cock p.j.a.c...@googlemail.com wrote:

On Thu, Sep 12, 2013 at 2:01 PM, Mathieu Bahin 
mathieu.ba...@irisa.fr

wrote:

Hi all,

We have been developing our own Galaxy instance for a while now. We
have a
cluster on which the job are sent to be executed, it is managed 
through

SGE.
Usually, communication between SGE and DRMAA is ok and we don't 
have any

problem with that.

When a job is deleted by the user, most of the times, the job
disappears but
sometimes, we don't know why, the job stays and has the status 'dr'
within
SGE. If we don't kill it 'manually', it stays forever. It is not 
always

the
same tools which produces this error.
Have you any idea why how manage it ?


I have noticed problem with our DRMMA/SGE setup where a
user can cancel a large job (using the job splitter in at least some
cases), but Galaxy does not seem to cancel the jobs on the cluster.
I've not tried to diagnose this yet - it could be a similar issue 
though.


Also, in our DRMAA/LSF setup (using a fork of the latest galaxy-dist) 
jobs
generated by the current workflow step continue running on the 
cluster

after history is deleted.

Ilya


Hi Ilya,
I also see this behaviour with DRMAA/GridEngine.
I think this has been already reported:

https://trello.com/c/1whC9did/245-currently-running-jobs-in-deleted-histories-should-be-killed

Please upvote it!

Best,
Nicola
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/


Re: [galaxy-dev] Launching a workflow via API, multiple params for one tool

2014-04-10 Thread Nicola Soranzo

Il 2014-04-09 22:18 John Marmaduke Eppley ha scritto:

As far as I can figure out, the parameter map for running a workflow
via the API is supposed to look like this:

{'blastn': {'param': 'evalue', 'value': '1e-06’}}

This doesn’t seem to allow for multiple parameters to a single tool.
Is there a way to submit multiple parameters that I’m missing? For
example:

{'blastn': {'param': 'evalue', 'value': '1e-06’, 'param':
‘identity_cutoff’, ‘value’: ’70’}}

…will not work. The dict() container doesn’t allow duplicate keys,
it just overwrites the values.


Hi John,
you are using the old and deprecated syntax, you can use instead:

{'blastn': {'evalue': '1e-06', 'identity_cutoff': 70}}

For more details look at _update_step_parameters() function 
documentation:


https://bitbucket.org/galaxy/galaxy-dist/src/tip/lib/galaxy/webapps/galaxy/api/workflows.py

Or look at this complete example:

https://github.com/crs4/bioblend/blob/objects/docs/examples/w5_galaxy_api.py

Moreover, you may consider using the more high-level bioblend API:

http://bioblend.readthedocs.org/en/latest/

Best,
Nicola
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] enforcing minimum length on text params

2014-04-10 Thread Nicola Soranzo
Hi Ravi,
for that you need a regex validator:

https://bitbucket.org/galaxy/galaxy-central/src/tip/lib/galaxy/tools/parameters/validation.py#cl-27

e.g.

validator type=regex message=This field should contain some
non-whitespace character.*\S/validator

Best,
Nicola

Il giorno gio, 10/04/2014 alle 10.20 -0400, Sanka, Ravi ha scritto: 
 Hi Nicola,
 
 Perchance, do you know what type of validator I can put to ensure that the
 string input is does not consist of only whitespace? Such a string is
 still invalid for my tool, but counts as a string.
 
 --
 Ravi Sanka
 ICS ­ Sr. Bioinformatics Engineer
 J. Craig Venter Institute
 301-795-7743
 --
 
 
 
 
 On 4/9/14 2:35 PM, Sanka, Ravi rsa...@jcvi.org wrote:
 
 Hi Nicola,
 
 Thank you. I tried the validator as you illustrated, and it worked
 perfectly.
 
 --
 Ravi Sanka
 ICS ­ Sr. Bioinformatics Engineer
 J. Craig Venter Institute
 301-795-7743
 --
 
 
 
 
 On 4/9/14 11:06 AM, Nicola Soranzo sora...@crs4.it wrote:
 
 Il 2014-04-09 17:00 Sanka, Ravi ha scritto:
  Greetings,
 
  I am adding a new tool to our in-house Galaxy, which has a few string
  inputs. I have made each one a text param in the XML, but want to
  ensure that the user does not leave any empty when he executes the
  tool.
 
  How can I do this? I tried setting the optional field of each text
  param to false, but that doesn't work since, technically, an empty
  string still counts as a string.
 
  Is there a minimum length setting I can enforce?
 
 Hi Ravi,
 you should use a validator, as in the following example:
 
 param name=rgid type=text label=Read group identifier (ID)
  validator type=empty_field /
 /param
 
 Best,
 Nicola
 
 


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Running a different command based on selection in the tool

2014-03-20 Thread Nicola Soranzo

Hi Saba and Jens,
: are not needed, they are optional, see

http://www.cheetahtemplate.org/docs/users_guide_html/users_guide.html#SECTION00068

Maybe it's a problem with the left quotes around Use_Snapshot in the 
#if line?


Nicola

Il 2014-03-20 06:26 Keilwagen, Jens ha scritto:

Hi Saba,

there is a syntax error in the line of the #if. Please add : at
the end and report whether the error still occurs.

All the best, Jens

VON: galaxy-dev-boun...@lists.bx.psu.edu
[mailto:galaxy-dev-boun...@lists.bx.psu.edu] IM AUFTRAG VON Saba
Sehrish
GESENDET: Mittwoch, 19. März 2014 23:07
AN: galaxy-dev@lists.bx.psu.edu
BETREFF: [galaxy-dev] Running a different command based on selection
in the tool

Hi all,

I am trying to write a tool that based on user criteria will run a
different script.

In the code below, If I select Use_Snapshot, I see the right 
interface

but somehow galaxy tries to execute the #else part and look for m
input parameter. The example I have seen in documentation is using 
the

script with same name in both if and else and the number of input
arguments is the same as well. Any suggestions/ideas why always #else
part gets executed although right interface appears based on user
selection.

Thanks,

Saba

 command

 #if $source.source_select==Use_Snapshot

 cM-snapshot.sh $input $output

 #else

 cM.sh $m $b $ns $w $s $z $output

 #end if

/command

inputs

 conditional name=source

 param name=source_select type=select label=Specify the input

 option value=Use_SnapshotUse Snapshot/option

 option value=Set_ValuesSet Values/option

 /param

 when value=Use_Snapshot

 param name=input type=data format=dbm size=100 label=Input
Snapshot/

 sanitizer sanitize=False/

 /when

 when value=Set_Values

 param name=m label=Omega_m h^2 type=float value=0.1279
min=0.120 max=0.155/

 param name=b label=Omega_b h^2 type=float value=0.0232
min=0.0215 max=0.0235/

 param name=ns label=n_s type=float value=0.8629 min=0.85
max=1.05/

 param name=w label=w type=float value=-1.184 min=-1.30
max=-0.70/

 param name=s label=sigma_8 type=float value=0.6159
min=0.61 max=0.9/

 param name=z label=z type=float value=1.0 min=0.0
max=1.0/

 sanitizer sanitize=False/

 /when

 /conditional

 /inputs


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] GATK 3.1

2014-03-18 Thread Nicola Soranzo

Il 2014-03-18 14:41 Berner, Thomas ha scritto:

Hi,

I've seen there is a new version (3.1-1) of GATK available at
http://www.broadinstitute.org/gatk/download [1] .

Are there any plans of getting this version into Galaxy in the nearer
future?


Hi Thomas,
as you probably know there is a wrapper for GATK v.2 on the Tool Shed:

http://toolshed.g2.bx.psu.edu/view/iuc/gatk2

which is developed and maintained by Jim Johnson, Björn Grüning and me. 
GATK v.3 has some new tools and also some changes to logging which 
prevent the use of gatk2 wrapper with v.3 , so we are planning to create 
a new gatk3 Tool Shed repository sooner or later, no time frame decided 
yet.
You can follow development, and contribute if you want, at this git 
repository:


https://github.com/bgruening/galaxytools

Best,
Nicola

--
Nicola Soranzo, Ph.D.
Bioinformatics Program, CRS4
Loc. Piscina Manna, 09010 Pula (CA), Italy
http://www.bioinformatica.crs4.it/
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] filter tag in output, multiple criteria

2014-01-31 Thread Nicola Soranzo

Il 2014-01-31 11:52 Geert Vandeweyer ha scritto:

Hi,

I'm looking for the correct syntax to achieve the following:

data type=data format=txt name=loh label=${tool.name} result
on ${on_string} (loh)  
filterouttype == 0 || outttype == 2 /filter
/data
data type=data format=txt name=cnv label=${tool.name} result
on ${on_string} (loh)  
filterouttype == 1 || outttype == 2 /filter
/data

I tried the above, with and without parentheses, but it creates the
second dataset also when outtype == 0. Is this type of dual filtering
possible?


I don't know if this is the reason, but you seem to have a typo:

outttype == 2

should be

outtype == 2

Best,
Nicola
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/


Re: [galaxy-dev] Import workflows via API

2014-01-22 Thread Nicola Soranzo
Hi Neil,
can you retry after applying this commit from my new pull request?

https://bitbucket.org/nsoranzo/galaxy-central/commits/11597a078e6505087064315567ad3357c2d40c56

--
Nicola Soranzo
Bioinformatics Program, CRS4
Loc. Piscina Manna, 09010 Pula (CA), Italy
http://www.bioinformatica.crs4.it/

neil.burd...@csiro.au ha scritto:

Hi Nicola,
 I've merged your changes into my version of the galaxy code 
 (slightly older than galaxy-central) I believe, however, I get the following 
 error when I try and execute your script:

Traceback (most recent call last):
  File get_wfs.py, line 13, in module
common.post(api_key, 'http://barium-rbh:9100/extras/api/workflows/import', 
 data)
  File /home/galaxy/milxcloud/scripts/api/common.py, line 48, in post
return simplejson.loads( urllib2.urlopen( req ).read() )
  File /usr/lib/python2.7/urllib2.py, line 126, in urlopen
return _opener.open(url, data, timeout)
  File /usr/lib/python2.7/urllib2.py, line 406, in open
response = meth(req, response)
  File /usr/lib/python2.7/urllib2.py, line 519, in http_response
'http', request, response, code, msg, hdrs)
  File /usr/lib/python2.7/urllib2.py, line 444, in error
return self._call_chain(*args)
  File /usr/lib/python2.7/urllib2.py, line 378, in _call_chain
result = func(*args)
  File /usr/lib/python2.7/urllib2.py, line 527, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 500: Internal Server Error

so I printed out the some lines in common.py 

def post( api_key, url, data ):
# Do the actual POST.
url = make_url( api_key, url )
print URL is %s % url

 URL is 
 http://barium-rbh:9100/extras/api/workflows/import?key=34ee80757f0d03de36e33a1676d245e4

the script is:
import sys
#sys.path.insert(1, 'milxcloud/scripts/api')
import common
api_key = '34ee80757f0d03de36e33a1676d245e4'
workflows = common.get(api_key, 
'http://barium-rbh:9100/api/workflows?show_published=True')
print workflows
published_workflow_ids = [str(workflow[u'id']) for workflow in workflows if 
bool(workflow[u'published'])]
print published_workflow_ids

for pw_id in published_workflow_ids:
data = {}
data['workflow_id'] = pw_id
common.post(api_key, 'http://barium-rbh:9100/extras/api/workflows/import', 
 data)


and I run it from ~/scripts/api

The output from the script is :
python get_wfs.py 
[{'name': 'SUVR', 'tags': [], 'url': '/extras/api/workflows/f2db41e1fa331b3e', 
'published': True, 'model_class': 'StoredWorkflow', 'id': 'f2db41e1fa331b3e'}, 
{'name': 'processSUVRData', 'tags': [], 'url': 
'/extras/api/workflows/f597429621d6eb2b', 'published': True, 'model_class': 
'StoredWorkflow', 'id': 'f597429621d6eb2b'}]
['f2db41e1fa331b3e', 'f597429621d6eb2b']
URL is 
http://barium-rbh:9100/extras/api/workflows/import?key=34ee80757f0d03de36e33a1676d245e4

Any ideas why the import maybe failing?

Thanks
Neil




From: Nicola Soranzo [sora...@crs4.it]
Sent: Monday, January 20, 2014 9:03 PM
To: Burdett, Neil (CCI, Herston - RBWH)
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: Import workflows via API

Hi Neil,
to import all published workflows you can use a script like this:

import sys
sys.path.insert(1, 'galaxy-central/scripts/api')
import common
api_key = 'YOUR_USER_API_KEY'
workflows = common.get(api_key,
'http://YOUR_SERVER/api/workflows?show_published=True')
published_workflow_ids = [str(workflow[u'id']) for workflow in
workflows if bool(workflow[u'published'])]
for pw_id in published_workflow_ids:
 data['workflow_id'] = pw_id
 common.post(api_key, 'http://YOUR_SERVER/api/workflows/import',
data)

Nicola

Il 2014-01-19 03:12 neil.burd...@csiro.au ha scritto:
 Hi Nicola,
 that is exactly what I'm looking for, however, how do
 I execute the script/tool? I would like to import all published
 workflows. What is the name of the script to run and the arguments ?
 Can you give an example please?

 Thanks again
 Neil


 Date: Fri, 17 Jan 2014 11:36:03 +0100
 From: Nicola Soranzo sora...@crs4.it
 To: galaxy-dev@lists.bx.psu.edu
 Subject: Re: [galaxy-dev] Import workflows via API
 Message-ID: d834d247437269704a26e8bbf8f67...@crs4.it
 Content-Type: text/plain; charset=UTF-8; format=flowed

 Il 2014-01-17 06:45 neil.burd...@csiro.au ha scritto:
 Hi,

  I execute workflows via the API. However, if I want another user to
 use my workflows, I can publish my workflows, but the new user then
 has to go on to the web browser and import this workflow.

 Is there a method/script which I can call via the API which can
 import
 all available (published) workflows so the user doesn't have to
 click
 a button on the web browser import workflow ?

 Thanks for any help

 Hi Neil,
 you are very lucky, just yesterday my pull request implementing
 exactly
 this has been merged in galaxy-central:


 https://bitbucket.org/galaxy/galaxy-central/pull-request/298/api-display-and-import-workflows-shared

Re: [galaxy-dev] Import workflows via API

2014-01-20 Thread Nicola Soranzo

Hi Neil,
to import all published workflows you can use a script like this:

import sys
sys.path.insert(1, 'galaxy-central/scripts/api')
import common
api_key = 'YOUR_USER_API_KEY'
workflows = common.get(api_key, 
'http://YOUR_SERVER/api/workflows?show_published=True')
published_workflow_ids = [str(workflow[u'id']) for workflow in 
workflows if bool(workflow[u'published'])]

for pw_id in published_workflow_ids:
data['workflow_id'] = pw_id
common.post(api_key, 'http://YOUR_SERVER/api/workflows/import', 
data)


Nicola

Il 2014-01-19 03:12 neil.burd...@csiro.au ha scritto:

Hi Nicola,
that is exactly what I'm looking for, however, how do
I execute the script/tool? I would like to import all published
workflows. What is the name of the script to run and the arguments ?
Can you give an example please?

Thanks again
Neil


Date: Fri, 17 Jan 2014 11:36:03 +0100
From: Nicola Soranzo sora...@crs4.it
To: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] Import workflows via API
Message-ID: d834d247437269704a26e8bbf8f67...@crs4.it
Content-Type: text/plain; charset=UTF-8; format=flowed

Il 2014-01-17 06:45 neil.burd...@csiro.au ha scritto:

Hi,

 I execute workflows via the API. However, if I want another user to
use my workflows, I can publish my workflows, but the new user then
has to go on to the web browser and import this workflow.

Is there a method/script which I can call via the API which can
import
all available (published) workflows so the user doesn't have to 
click

a button on the web browser import workflow ?

Thanks for any help


Hi Neil,
you are very lucky, just yesterday my pull request implementing 
exactly

this has been merged in galaxy-central:


https://bitbucket.org/galaxy/galaxy-central/pull-request/298/api-display-and-import-workflows-shared-by/diff

and is also available in BioBlend thanks to my colleague Simone Leo:

https://github.com/afgane/bioblend/pull/51

Best,
Nicola

--
Nicola Soranzo, Ph.D.
Bioinformatics Program, CRS4
Loc. Piscina Manna, 09010 Pula (CA), Italy
http://www.bioinformatica.crs4.it/


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/


Re: [galaxy-dev] Import workflows via API

2014-01-17 Thread Nicola Soranzo

Il 2014-01-17 06:45 neil.burd...@csiro.au ha scritto:

Hi,

 I execute workflows via the API. However, if I want another user to
use my workflows, I can publish my workflows, but the new user then
has to go on to the web browser and import this workflow.

Is there a method/script which I can call via the API which can 
import

all available (published) workflows so the user doesn't have to click
a button on the web browser import workflow ?

Thanks for any help


Hi Neil,
you are very lucky, just yesterday my pull request implementing exactly 
this has been merged in galaxy-central:


https://bitbucket.org/galaxy/galaxy-central/pull-request/298/api-display-and-import-workflows-shared-by/diff

and is also available in BioBlend thanks to my colleague Simone Leo:

https://github.com/afgane/bioblend/pull/51

Best,
Nicola

--
Nicola Soranzo, Ph.D.
Bioinformatics Program, CRS4
Loc. Piscina Manna, 09010 Pula (CA), Italy
http://www.bioinformatica.crs4.it/
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/


Re: [galaxy-dev] A new tool_data_table_conf.xml created under tool-data directory

2014-01-17 Thread Nicola Soranzo
Greg Von Kuster greg@... writes:
 On Nov 9, 2012, at 12:56 AM, Derrick Lin wrote:
  Also how does this change affect the existing shed tools like BWA that
is using tool_data_table_conf.xml?
 
 The existing shed tools will contiue to function as usual, but entries for
them will be in the tool_data_table_conf.xml file.  ANy new tools installed
from the tool shed that use entries like this will be added to the new
shed_tool_data_table_conf.xml file.  Entries from both of these files are
loaded into the ToolDataTableManager.  The intent is for the Galaxy admin to
be able to manuall make changes to the tool_data_table_conf.xml file and not
have them munged by the tool shed's installation process.  This is the same
approach used for the tool_conf.xml file, where the Galaxy tool shed
repository installation process uses the shed_tool_conf.xml file.

Reviving an old thread...

I noticed that when I install a repository from the Tool Shed which contains
a tool_data_table_conf.xml.sample , the sample files mentioned therein are
copied to:
- tool-data/ (with AND without .sample extension)
- tool-data/toolshed.g2.bx.psu.edu/repos/OWNER/REPO/REVISION/ (without
.sample extension)
The tool_data_table_conf.xml file in
tool-data/toolshed.g2.bx.psu.edu/repos/OWNER/REPO/REVISION/ references the
files in the same directory, so I find the duplication confusing.

Moreover, there is an increasing number of repositories which install the
same .loc files, e.g.:
- the 4 cufflinks tools plus sam_pileup, sam_to_bam and samtools_mpileup
install fasta_indexes.loc ;
- blat, lastz and lastz_paired_reads install lastz_seqs.loc .
I suppose it was made to prevent repositories from getting in each other's
way, but I see 2 problems:
- system administrators have to manually hard-link these files;
- duplications in shed_tool_data_table_conf.xml generate errors like:
  galaxy.tools.data ERROR 2014-01-17 15:29:41,409 Attempted to add fields
(['hg19full', 'hg19', 'Human Feb. 2009 (GRCh37/hg19) (hg19) Full',
'/mnt/genome/hg19/sam_index/hg19full/hg19full.fa']) to data table
'fasta_indexes', but this entry already exists and allow_duplicates is False.

Suggestions?

Thanks,
Nicola

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Tool Access Control

2013-12-20 Thread Nicola Soranzo

Hi Eric,
I've updated again the Python code in the wiki page since the previous 
version was not correctly retrieving 'required_role' and 
'final_destination' attributes for tools installed from a Tool Shed if 
you was using the short version of the tool ID in job_conf.xml .


Best,
Nicola

Il 2013-12-09 12:38 Nicola Soranzo ha scritto:

Thanks Eric,
I'm sure this will be useful to many Galaxy admins!

I've edited a bit the wiki page, in particular I've updated the code
to use roles instead of groups, which is more appropriate in the
context of access control.

Best,
Nicola

Il 2013-12-07 03:07 Rasche Eric ha scritto:

Hi Nicola,
 
Just thought I'd write back since I added a Wiki page in 
Admin/Config

[11] area for implementing access control.
 
Thank you for sharing the snippets you posted in the email thread 
you

linked me!
 
http://wiki.galaxyproject.org/Admin/Config/Access%20Control [12]
 
Cheers,
Eric Rasche
 
 
06.11.2013, 18:57, Eric Rasche rasche.e...@yandex.ru:


Hi Nicola,

Oh, excellent. I must've skipped over that, given the strange title
of
the thread.

Your solution at the end of that thread is very promising, and
certainly
handles failure MUCH better than mine does (i.e. raising exceptions
and
not breaking a workflow if the user isn't permitted access.)

(Did you put it on the galaxy wiki anywhere? If it weren't for you
linking that, I never would've known about it and that's very 
useful

info!)

In my organisation's case; if a user isn't allowed access to a 
given

tool, we believe that

- - galaxy has no reason to admit it exists
- - galaxy should not share default information about a tool

Which is a bit different from the case of having a license to use a
tool. For licensing issues, naturally it would be fine to say yes
this
exists and if you can't run it, obtain a license.

For my org's case, we might want to store administrative tools (for
other services) in galaxy. It's a very convenient platform for more
than
just bioinformatics and we have some non-technical people on staff
who
occasionally need to pull various data sets from various
services/make
database backups/etc. Students and clients who use our galaxy
instance
don't need to know that these tools are available.

Thoughts?

On 11/06/2013 12:12 PM, Nicola Soranzo wrote:


 Hi Eric,
 please also take a look at this mailing list thread:






 
http://dev.list.galaxyproject.org/pass-user-groups-to-dynamic-job-runner-td4661753.html

[7]

 If you are interested in the is_user_in_group solution, I have a
 slightly updated version which also uses roles instead of
groups.

 Nicola

 Il giorno mer, 06/11/2013 alle 11.38 -0600, Eric Rasche ha
scritto:


 Howdy devs,

 I've implemented some rather basic tool access control and am
looking
 for feedback on my implementation.

 # Why

 Our organisation wanted the ability to restrict tools to
different
 users/roles. As such I've implemented as an execute tag
which can be
 applied to either section or tools in the tool
configuration file.

 # Example galaxy-admin changes

 For example:

   section execute=a...@b.co [1],b...@b.co [2] id=EncodeTools
name=ENCODE Tools
 tool file=encode/gencode_partition.xml /
 tool execute=b...@b.co [3]
file=encode/random_intervals.xml /
   /section

 which would allow A and B to access gencode_parition, but only
B would
 be able to access random_intervals. To put it explicity

 - by default, everyone can access all tools
 - if section level permissions are set, then those are set as
defaults
 for all tools in that section
 - if tool permissions are set, they will override the
defaults.

 # Pros and Cons

 There are some good features

 - non-accessible tools won't show up in the left hand panel,
based on user
 - non-accessible tools cannot be run or accessed.

 There are some caveats however.

 - existence of tools is not completely hidden.
 - Labels are not hidden at all.
 - workflows break completely if a tool is unavailable to a
shared user
 and the user copies+edits. They can be copied, and viewed
(says tool not
 found), but cannot be edited.

 Tool names/id/version info can be found in the javascript
object due to
 the call to app.toolbox.tool_panel.items() in
 templates/webapps/galaxy/workflow/editor.mako, as that returns
the raw
 tool list, rather than one that's filtered on whether or not
the user
 has access. I'm yet to figure out a clean fix for this.
Additionally,
 empty sections are still shown even if there aren't tools
listed in them.

 For a brief overview of my changes, please see the attached
diff. (It's
 missing one change because I wasn't being careful and started
work on
 multiple different features)

 # Changeset overview

 In brief, most of the changes consist of
 - new method in model.User to check if an array of roles
overlaps at all
 with a user's roles
 - modifications to appropriate files for reading in the new
 tool_config.xml's options
 - modification to get_tool to pass user information, as
whether or not a
 tool exists

Re: [galaxy-dev] Skip automated testing of tools in this revision setting?

2013-12-18 Thread Nicola Soranzo

Thanks Greg for the heads-up!
These are great news, I can surely wait until mid-January (the wait 
will be mostly vacation time :).
Just let us know when we can start checking again the test results on 
the main Tool Shed.


Nicola

Il 2013-12-17 21:15 Greg Von Kuster ha scritto:

Hi Nicola,

There's some bad news and good new regarding this issue.

The bad news is that the old version of the tool shed's install and
test framework required complete reengineering, and this resulted in
it not being functional in the main tool shed's environment which 
runs

the Galaxy stable branch. I've tried several approaches to committing
and grafting changes to the stable branch in an attempt to piece
together a functional environment for the main tool shed until the
next Galaxy release, but the reengineering effort was so extensive
that it made this extremely difficult.

So the main tool shed cannot be tested until the next-stable branch
is created in preparation for the next Galaxy release.  The
next-stable branch is currently scheduled for mid January, 2014, with
the Galaxy release currently scheduled for the first week of 
February,

2014.

The good news is that the reengineerig effort has resulted in a much
more stable and functional install and test framework that will
provide many more benefits than its previous incarnation.  I am now
working on enhancing the framework to include installing and testing
repositories of type tool_dependency_definition.  This enhancement
will be included in the next-stable branch when it is created in
mid-January.

Sorry for the inconvenience this may cause, but after mid-January
things should be in good shape on the main tool shed.

Greg Von Kuster


On Dec 12, 2013, at 5:00 PM, Nicola Soranzo sora...@crs4.it wrote:


Hi Greg,
thanks for looking at this!
Unfortunately still no test results displayed for
http://toolshed.g2.bx.psu.edu/view/crs4/blat

Best,
Nicola

Il giorno mar, 10/12/2013 alle 10.43 -0500, Greg Von Kuster ha 
scritto:

Hello Nicola,

The main tool shed is running a bit behind the test tool shed since 
it
is on the stable branch. I've commited a fix for the main tool shed 
in

11664:3bf805dfa14e that should correct the problem resulting in no
test results being displayed. Results should begin being displayed
with the test run tomorrow morning. I'll make sure to inspect the 
test

runs for the main tool shed more closely from here on (I've been
focusing more on the test tool shed) to make sure the different 
issues

that arise on the stable branch are handled more quickly.

Thanks for letting me know about this.

Greg Von Kuster

On Dec 10, 2013, at 10:09 AM, Nicola Soranzo sora...@crs4.it 
wrote:



Hi Greg!
I don't know if it's the same problem, but I've not been able to 
see

test results for this repository for a while:

http://toolshed.g2.bx.psu.edu/view/crs4/blat

Best,
Nicola

Il giorno mar, 10/12/2013 alle 09.46 -0500, Greg Von Kuster ha 
scritto:
There are several flags for each repository revision that 
determine if and how the revision is tested.  If tested, flags also 
detemine which category the latest revision falls into when 
clicking on the links in the Tool Shed's menu.  One of these flags 
is tools_functionally_correct, which was incorrectly set on 
http://testtoolshed.g2.bx.psu.edu/view/peterjc/tmhmm_and_signalp/ee10017fcd80 
due to a bug introduced during the reengineering of the Topll 
Shed's install and test framework that I recently did.  This bug 
was corrected in the following change set.



https://bitbucket.org/galaxy/galaxy-central/commits/ca25579a90e97157e32b527e7471cde90e9beed0

I've corrected the setting for the tools_functionally_correct 
flag for your repository so it should no longer be displayed in the 
Latest revision: failing tool tests category.


Thanks for letting us know about this, and please let us know if 
you discover other reposiory revisions that require the same 
correction.


Greg Von Kuster


On Dec 9, 2013, at 5:06 AM, Peter Cock 
p.j.a.c...@googlemail.com wrote:



Hi Dave  Greg,

Could you double check how the Skip automated testing of tools 
in

this revision setting is being used on the Test Tool Shed?

e.g. 
http://testtoolshed.g2.bx.psu.edu/view/peterjc/tmhmm_and_signalp/ee10017fcd80


Test runs
Automated test environment
Not tested
Licensing issues restrict automated tool dependency 
installation.

2013-11-29 17:19:34
Automated test environment
Not tested
Licensing issues restrict automated tool dependency 
installation.


(Ticked) Skip automated testing of tools in this revision

That all looks fine, and it is listed under Latest revision: 
skip tool tests.


However, this tool is also listed under Latest revision: 
failing tool tests.


Regards,

Peter
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu

Re: [galaxy-dev] Skip automated testing of tools in this revision setting?

2013-12-12 Thread Nicola Soranzo
Hi Greg,
thanks for looking at this!
Unfortunately still no test results displayed for
http://toolshed.g2.bx.psu.edu/view/crs4/blat

Best,
Nicola

Il giorno mar, 10/12/2013 alle 10.43 -0500, Greg Von Kuster ha scritto: 
 Hello Nicola,
 
 The main tool shed is running a bit behind the test tool shed since it
 is on the stable branch. I've commited a fix for the main tool shed in
 11664:3bf805dfa14e that should correct the problem resulting in no
 test results being displayed. Results should begin being displayed
 with the test run tomorrow morning. I'll make sure to inspect the test
 runs for the main tool shed more closely from here on (I've been
 focusing more on the test tool shed) to make sure the different issues
 that arise on the stable branch are handled more quickly. 
 
 Thanks for letting me know about this.
 
 Greg Von Kuster
 
 On Dec 10, 2013, at 10:09 AM, Nicola Soranzo sora...@crs4.it wrote:
 
  Hi Greg!
  I don't know if it's the same problem, but I've not been able to see
  test results for this repository for a while:
  
  http://toolshed.g2.bx.psu.edu/view/crs4/blat
  
  Best,
  Nicola
  
  Il giorno mar, 10/12/2013 alle 09.46 -0500, Greg Von Kuster ha scritto: 
  There are several flags for each repository revision that determine if 
  and how the revision is tested.  If tested, flags also detemine which 
  category the latest revision falls into when clicking on the links in the 
  Tool Shed's menu.  One of these flags is tools_functionally_correct, 
  which was incorrectly set on 
  http://testtoolshed.g2.bx.psu.edu/view/peterjc/tmhmm_and_signalp/ee10017fcd80
   due to a bug introduced during the reengineering of the Topll Shed's 
  install and test framework that I recently did.  This bug was corrected in 
  the following change set.
  
  https://bitbucket.org/galaxy/galaxy-central/commits/ca25579a90e97157e32b527e7471cde90e9beed0
  
  I've corrected the setting for the tools_functionally_correct flag for 
  your repository so it should no longer be displayed in the Latest 
  revision: failing tool tests category.
  
  Thanks for letting us know about this, and please let us know if you 
  discover other reposiory revisions that require the same correction.
  
  Greg Von Kuster
  
  
  On Dec 9, 2013, at 5:06 AM, Peter Cock p.j.a.c...@googlemail.com wrote:
  
  Hi Dave  Greg,
  
  Could you double check how the Skip automated testing of tools in
  this revision setting is being used on the Test Tool Shed?
  
  e.g. 
  http://testtoolshed.g2.bx.psu.edu/view/peterjc/tmhmm_and_signalp/ee10017fcd80
  
  Test runs
  Automated test environment
  Not tested
  Licensing issues restrict automated tool dependency installation.
  2013-11-29 17:19:34
  Automated test environment
  Not tested
  Licensing issues restrict automated tool dependency installation.
  
  (Ticked) Skip automated testing of tools in this revision
  
  That all looks fine, and it is listed under Latest revision: skip tool 
  tests.
  
  However, this tool is also listed under Latest revision: failing tool 
  tests.
  
  Regards,
  
  Peter
  ___
  Please keep all replies on the list by using reply all
  in your mail client.  To manage your subscriptions to this
  and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/
  
  To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
  
  
  ___
  Please keep all replies on the list by using reply all
  in your mail client.  To manage your subscriptions to this
  and other Galaxy lists, please use the interface at:
   http://lists.bx.psu.edu/
  
  To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/
  
  
  ___
  Please keep all replies on the list by using reply all
  in your mail client.  To manage your subscriptions to this
  and other Galaxy lists, please use the interface at:
   http://lists.bx.psu.edu/
  
  To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/
 


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


Re: [galaxy-dev] Skip automated testing of tools in this revision setting?

2013-12-10 Thread Nicola Soranzo
Hi Greg!
I don't know if it's the same problem, but I've not been able to see
test results for this repository for a while:

http://toolshed.g2.bx.psu.edu/view/crs4/blat

Best,
Nicola

Il giorno mar, 10/12/2013 alle 09.46 -0500, Greg Von Kuster ha scritto: 
 There are several flags for each repository revision that determine if and 
 how the revision is tested.  If tested, flags also detemine which category 
 the latest revision falls into when clicking on the links in the Tool Shed's 
 menu.  One of these flags is tools_functionally_correct, which was 
 incorrectly set on 
 http://testtoolshed.g2.bx.psu.edu/view/peterjc/tmhmm_and_signalp/ee10017fcd80 
 due to a bug introduced during the reengineering of the Topll Shed's install 
 and test framework that I recently did.  This bug was corrected in the 
 following change set.
 
 https://bitbucket.org/galaxy/galaxy-central/commits/ca25579a90e97157e32b527e7471cde90e9beed0
 
 I've corrected the setting for the tools_functionally_correct flag for your 
 repository so it should no longer be displayed in the Latest revision: 
 failing tool tests category.
 
 Thanks for letting us know about this, and please let us know if you discover 
 other reposiory revisions that require the same correction.
 
 Greg Von Kuster
 
 
 On Dec 9, 2013, at 5:06 AM, Peter Cock p.j.a.c...@googlemail.com wrote:
 
  Hi Dave  Greg,
  
  Could you double check how the Skip automated testing of tools in
  this revision setting is being used on the Test Tool Shed?
  
  e.g. 
  http://testtoolshed.g2.bx.psu.edu/view/peterjc/tmhmm_and_signalp/ee10017fcd80
  
  Test runs
  Automated test environment
  Not tested
  Licensing issues restrict automated tool dependency installation.
  2013-11-29 17:19:34
  Automated test environment
  Not tested
  Licensing issues restrict automated tool dependency installation.
  
  (Ticked) Skip automated testing of tools in this revision
  
  That all looks fine, and it is listed under Latest revision: skip tool 
  tests.
  
  However, this tool is also listed under Latest revision: failing tool 
  tests.
  
  Regards,
  
  Peter
  ___
  Please keep all replies on the list by using reply all
  in your mail client.  To manage your subscriptions to this
  and other Galaxy lists, please use the interface at:
   http://lists.bx.psu.edu/
  
  To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/
 
 
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
   http://lists.bx.psu.edu/
 
 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


Re: [galaxy-dev] Tool Access Control

2013-12-09 Thread Nicola Soranzo

Thanks Eric,
I'm sure this will be useful to many Galaxy admins!

I've edited a bit the wiki page, in particular I've updated the code to 
use roles instead of groups, which is more appropriate in the context of 
access control.


Best,
Nicola

Il 2013-12-07 03:07 Rasche Eric ha scritto:

Hi Nicola,
 
Just thought I'd write back since I added a Wiki page in Admin/Config
[11] area for implementing access control.
 
Thank you for sharing the snippets you posted in the email thread you
linked me!
 
http://wiki.galaxyproject.org/Admin/Config/Access%20Control [12]
 
Cheers,
Eric Rasche
 
 
06.11.2013, 18:57, Eric Rasche rasche.e...@yandex.ru:


Hi Nicola,

Oh, excellent. I must've skipped over that, given the strange title
of
the thread.

Your solution at the end of that thread is very promising, and
certainly
handles failure MUCH better than mine does (i.e. raising exceptions
and
not breaking a workflow if the user isn't permitted access.)

(Did you put it on the galaxy wiki anywhere? If it weren't for you
linking that, I never would've known about it and that's very useful
info!)

In my organisation's case; if a user isn't allowed access to a given
tool, we believe that

- - galaxy has no reason to admit it exists
- - galaxy should not share default information about a tool

Which is a bit different from the case of having a license to use a
tool. For licensing issues, naturally it would be fine to say yes
this
exists and if you can't run it, obtain a license.

For my org's case, we might want to store administrative tools (for
other services) in galaxy. It's a very convenient platform for more
than
just bioinformatics and we have some non-technical people on staff
who
occasionally need to pull various data sets from various
services/make
database backups/etc. Students and clients who use our galaxy
instance
don't need to know that these tools are available.

Thoughts?

On 11/06/2013 12:12 PM, Nicola Soranzo wrote:


 Hi Eric,
 please also take a look at this mailing list thread:






 
http://dev.list.galaxyproject.org/pass-user-groups-to-dynamic-job-runner-td4661753.html

[7]

 If you are interested in the is_user_in_group solution, I have a
 slightly updated version which also uses roles instead of
groups.

 Nicola

 Il giorno mer, 06/11/2013 alle 11.38 -0600, Eric Rasche ha
scritto:


 Howdy devs,

 I've implemented some rather basic tool access control and am
looking
 for feedback on my implementation.

 # Why

 Our organisation wanted the ability to restrict tools to
different
 users/roles. As such I've implemented as an execute tag
which can be
 applied to either section or tools in the tool
configuration file.

 # Example galaxy-admin changes

 For example:

   section execute=a...@b.co [1],b...@b.co [2] id=EncodeTools
name=ENCODE Tools
 tool file=encode/gencode_partition.xml /
 tool execute=b...@b.co [3]
file=encode/random_intervals.xml /
   /section

 which would allow A and B to access gencode_parition, but only
B would
 be able to access random_intervals. To put it explicity

 - by default, everyone can access all tools
 - if section level permissions are set, then those are set as
defaults
 for all tools in that section
 - if tool permissions are set, they will override the
defaults.

 # Pros and Cons

 There are some good features

 - non-accessible tools won't show up in the left hand panel,
based on user
 - non-accessible tools cannot be run or accessed.

 There are some caveats however.

 - existence of tools is not completely hidden.
 - Labels are not hidden at all.
 - workflows break completely if a tool is unavailable to a
shared user
 and the user copies+edits. They can be copied, and viewed
(says tool not
 found), but cannot be edited.

 Tool names/id/version info can be found in the javascript
object due to
 the call to app.toolbox.tool_panel.items() in
 templates/webapps/galaxy/workflow/editor.mako, as that returns
the raw
 tool list, rather than one that's filtered on whether or not
the user
 has access. I'm yet to figure out a clean fix for this.
Additionally,
 empty sections are still shown even if there aren't tools
listed in them.

 For a brief overview of my changes, please see the attached
diff. (It's
 missing one change because I wasn't being careful and started
work on
 multiple different features)

 # Changeset overview

 In brief, most of the changes consist of
 - new method in model.User to check if an array of roles
overlaps at all
 with a user's roles
 - modifications to appropriate files for reading in the new
 tool_config.xml's options
 - modification to get_tool to pass user information, as
whether or not a
 tool exists is now dependent on who is asking.

 Please let me know if you have input on this before I create a
pull
 request on this feature.

 # Fixes

 I believe this will fix a number of previously brought up
issues (at
 least to my understanding of the issues listed)

 +







https://trello.com/c/Zo7FAXlM/286-24-add-ability-to-password-secure

Re: [galaxy-dev] Including descriptions etc in extended BLAST+ tabular output

2013-12-05 Thread Nicola Soranzo

Il 2013-12-04 20:48 Jim Johnson ha scritto:

On 12/4/13, 10:21 AM, Peter Cock wrote:


On Mon, Dec 2, 2013 at 3:37 PM, Peter Cock
p.j.a.c...@googlemail.com [1] wrote:
 On Sun, Dec 1, 2013 at 7:10 PM, Björn Grüning
 bjoern.gruen...@pharmazie.uni-freiburg.de [2] wrote:

 Great. I'll probably merge the c25 branch and push that
 to the Test Tool Shed next week (adding the descriptions
 as a new 25th column to the default extended tabular
 output).

 +1

 Done, see this and the preceding commits:




https://github.com/peterjc/galaxy_blast/commit/eb1e522a864e5274a2d274a49fcb15c313e93d69

[3]

 And the Test Tool Shed repository has been updated:




http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus/8f9023b30384

[4]

 We'll also be running this update locally for a little more
 testing, but I'd appreciate some of you also testing this,
 e.g. via the Test Tool Shed on your own local (development)
 Galaxy instances.

 If there are no problems, that can probably go out to the
 main Tool Shed by the end of the week.

I pushed another update to the Test Tool Shed addressing
a slight change in the autogenerated IDs used in the BLAST
XML output, and an overlooked mention of 24 columns:




http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus/72170c3f515a

[5]

The functional tests are passing on the Test Tool Shed,
so before I push this to the main Tool Shed, any other
comments?

 I'll look at the pick-your-own column output for the next
 revision of the NCBI BLAST+ wrappers.

I've been working on this using JJ's work on column picking
in the BLAST XML to tabular tool - viewable here:



https://github.com/jmchilton/galaxy_blast/commit/d79afc03522768323494818a40aac10513a6fa09

[6]

See this branch updating and extending that work:
https://github.com/peterjc/galaxy_blast/tree/blastxml_jj [7]

I'm considering splitting the column picker into three, the
standard 12 columns, the further 13 used in our extended
25 column output, and a third category for any other columns
offered by BLAST+ (like the taxonomy columns added in
BLAST+ 2.2.28).

Here's a screenshot showing the column picker offering
the 25 columns, split in two:

Note that (following JJ's earlier work) these all include the column
numbers as used in the standard 12 column BLAST tabular output,

or our extended 25 column output.

Note I would not intend to number the final set of extra columns.

Do people think this is helpful, or potentially confusing (since
the
column numbering will change in a custom selection)?

Peter

 Splitting into 3 sections seems like a good idea.
 That makes it easier to select groups of columns.

 I also think labeling the column options with the column number in
the standard tabular outputs will be useful to users.


I also like the splitting into 3 sections, but I think that the column 
numbering may be confusing and I would prefer it's not included.


Nicola
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Including descriptions etc in extended BLAST+ tabular output

2013-12-02 Thread Nicola Soranzo

Il 2013-12-02 13:08 Peter Cock ha scritto:

On Sun, Dec 1, 2013 at 7:10 PM, Björn Grüning wrote:

Peter wrote:
 But, I would think we should offer the option to get all 
available result
 information.   The multi-select checkboxes could serve that 
purpose.


OK, so adding a pick-you-own columns tabular output
would be useful :)


+1


Any thoughts on if this should be the standard 12 columns
plus a user selection, or a free selection of any columns
(e.g. could omit most of the standard 12).


I'd go with free selection.

Best,
Nicola
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Tool Access Control

2013-11-06 Thread Nicola Soranzo
Hi Eric,
please also take a look at this mailing list thread:

http://dev.list.galaxyproject.org/pass-user-groups-to-dynamic-job-runner-td4661753.html

If you are interested in the is_user_in_group solution, I have a
slightly updated version which also uses roles instead of groups.

Nicola

Il giorno mer, 06/11/2013 alle 11.38 -0600, Eric Rasche ha scritto: 
 Howdy devs,
 
 I've implemented some rather basic tool access control and am looking
 for feedback on my implementation.
 
 # Why
 
 Our organisation wanted the ability to restrict tools to different
 users/roles. As such I've implemented as an execute tag which can be
 applied to either section or tools in the tool configuration file.
 
 # Example galaxy-admin changes
 
 For example:
 
   section execute=a...@b.co,b...@b.co id=EncodeTools name=ENCODE Tools
 tool file=encode/gencode_partition.xml /
 tool execute=b...@b.co file=encode/random_intervals.xml /
   /section
 
 which would allow A and B to access gencode_parition, but only B would
 be able to access random_intervals. To put it explicity
 
 - by default, everyone can access all tools
 - if section level permissions are set, then those are set as defaults
 for all tools in that section
 - if tool permissions are set, they will override the defaults.
 
 # Pros and Cons
 
 There are some good features
 
 - non-accessible tools won't show up in the left hand panel, based on user
 - non-accessible tools cannot be run or accessed.
 
 There are some caveats however.
 
 - existence of tools is not completely hidden.
 - Labels are not hidden at all.
 - workflows break completely if a tool is unavailable to a shared user
 and the user copies+edits. They can be copied, and viewed (says tool not
 found), but cannot be edited.
 
 Tool names/id/version info can be found in the javascript object due to
 the call to app.toolbox.tool_panel.items() in
 templates/webapps/galaxy/workflow/editor.mako, as that returns the raw
 tool list, rather than one that's filtered on whether or not the user
 has access. I'm yet to figure out a clean fix for this. Additionally,
 empty sections are still shown even if there aren't tools listed in them.
 
 For a brief overview of my changes, please see the attached diff. (It's
 missing one change because I wasn't being careful and started work on
 multiple different features)
 
 # Changeset overview
 
 In brief, most of the changes consist of
 - new method in model.User to check if an array of roles overlaps at all
 with a user's roles
 - modifications to appropriate files for reading in the new
 tool_config.xml's options
 - modification to get_tool to pass user information, as whether or not a
 tool exists is now dependent on who is asking.
 
 Please let me know if you have input on this before I create a pull
 request on this feature.
 
 # Fixes
 
 I believe this will fix a number of previously brought up issues (at
 least to my understanding of the issues listed)
 
 + https://trello.com/c/Zo7FAXlM/286-24-add-ability-to-password-secure-tools
 + (I saw some solution where they were adding _beta to tool names
 which gave permissions to developers somewhere, but cannot find that now)
 
 
 
 Cheers,
 Eric Rasche
 
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
   http://lists.bx.psu.edu/
 
 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


Re: [galaxy-dev] RFC: remove trailing semicolons from command line - Broken bowtie2_wrapper on some SGE systems

2013-10-11 Thread Nicola Soranzo
Il giorno ven, 11/10/2013 alle 01.10 +0200, Björn Grüning ha scritto: 
 Dear all,
 
 some SGE instances will add automatically an semicolon add the end of
 each command, resulting in a disrupted job, because ';;' are not
 allowed.
 
 The latest changes to the Bowtie2 wrapper resulting in such a case and a
 crash in our instance. An easy solution would be to fix the wrapper and
 omitting the trailing ';' but maybe its better to fix it once for all in
 the corresponding tools code, patched attached.

The Bowtie2 bug may be fixed if my pull request 234 will be merged:

https://bitbucket.org/galaxy/galaxy-central/pull-request/234/bug-fixes-for-3-tools-rebased-222

 I'm not familiar with other Job schedulers so I'm seeking for
 comments ... it there any disadvantage of removing trailing ';' from the
 command line?

That would be sensible too.

Ciao,
Nicola

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] RFC: remove trailing semicolons from command line - Broken bowtie2_wrapper on some SGE systems

2013-10-11 Thread Nicola Soranzo

I already fixed it one week ago:

https://bitbucket.org/nsoranzo/galaxy-central-tools/commits/74b5fd2318d33b76382b052eae6a28b035dc075f

but then Ross committed a similar fix to galaxy-central, I merged it in 
my pull request and then Ross reverted the fix!

This is the reason for rebasing pull request #222 to #234.

Ciao,
Nicola

Il 2013-10-11 16:05 Björn Grüning ha scritto:

Hi Nicola,

interesting you fixed bowtie in the same way than we did. But somehow 
I

would like to strip a trailing ';' by default in Galaxy, when it does
not break anything else.

Thanks for sharing!
Bjoern

Il giorno ven, 11/10/2013 alle 01.10 +0200, Björn Grüning ha 
scritto:

 Dear all,

 some SGE instances will add automatically an semicolon add the end 
of

 each command, resulting in a disrupted job, because ';;' are not
 allowed.

 The latest changes to the Bowtie2 wrapper resulting in such a case 
and a
 crash in our instance. An easy solution would be to fix the 
wrapper and
 omitting the trailing ';' but maybe its better to fix it once for 
all in

 the corresponding tools code, patched attached.

The Bowtie2 bug may be fixed if my pull request 234 will be merged:


https://bitbucket.org/galaxy/galaxy-central/pull-request/234/bug-fixes-for-3-tools-rebased-222

 I'm not familiar with other Job schedulers so I'm seeking for
 comments ... it there any disadvantage of removing trailing ';' 
from the

 command line?

That would be sensible too.

Ciao,
Nicola



___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Tool Shed packages for BLAST+ binaries

2013-10-07 Thread Nicola Soranzo
actions_group is neither in stable nor in default branch of galaxy-dist
repository, it's present only in galaxy-central.

Nicola

Il giorno lun, 07/10/2013 alle 11.24 -0400, Dave Bouvier ha scritto: 
 Peter,
 
 The platform detection code is not yet in the stable branch, but is 
 intended to be included in the upcoming release. The automated testing 
 framework does run the default branch, though, so the nightly tests will 
 make use of the new definitions.
 
 --Dave B.
 
 On 10/07/2013 11:22 AM, Peter Cock wrote:
  On Mon, Oct 7, 2013 at 3:44 PM, Peter Cock p.j.a.c...@googlemail.com 
  wrote:
  On Mon, Oct 7, 2013 at 3:37 PM, Dave Bouvier d...@bx.psu.edu wrote:
  Peter,
 
  I did some investigating, and it turns out that the if [[ condition ]]
  syntax used in the ncbi tool dependency is only compatible with bash, not
  sh.
 
  Does that mean the test system has switched between sh and bash
  and back to sh again?
 
  Is there anything written down about the expected shell(s)
  available for a standard Galaxy instance?
 
  For sh, the corresponding syntax would look like:
 
  if [ $string = value ]
  then
   echo string is equal to value
  fi
 
  However, it strikes me that the platform detection and download url
  determination could also be done using the recently introduced
  actions_group feature, which would bypass shell-dependent
  conditional syntax entirely.
 
  Yes, but is that in stable releases for galaxy-dist yet?
 
  In the meantime I will try that on the Test Tool Shed, based
  on the tool_dependencies.xml example you shared before:
 
  http://testtoolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_2_26/c83440a9bdfb
 
  Note there seems to me to be a lot of duplication - I would like
  a way to unambiguously combine multiple CPU architectures
  into a single action.
 
  For example, combine these:
 
  actions os=darwin architecture=x86_64
  actions os=darwin architecture=i386
 
  into:
 
  actions os=darwin architecture=x86_64,i386
 
  or just:
 
  actions os=darwin
 
  and similarly, combine these:
 
  actions os=linux architecture=i386
  actions os=linux architecture=i686
 
  into something like:
 
  actions os=linux architecture=i386,i686
 
  Assuming that works on the Test Tool Shed, and the code for
  this is already in the stable releases, I will update the main
  Tool Shed definition as well.
 
  Thanks,
 
  Peter
 
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
   http://lists.bx.psu.edu/
 
 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


Re: [galaxy-dev] Download URL not accepted in tool_dependencies.xml

2013-09-30 Thread Nicola Soranzo
This problem is probably due to the 2 ampersands '', which are not
allowed in XML. Try substituting them with 'amp;'.

Nicola

Il giorno lun, 30/09/2013 alle 13.51 +0200, Joachim Jacob | VIB | ha
scritto: 
 Hi all,
 
 
 The download URL seems not to be accepted in tool_dependencies.xml. It 
 is an URL from sourceforge, the direct link to a package.
 
 The error upon uploading the tool to my toolshed:
 **
 Metadata may have been defined for some items in revision 
 '5665a799775d'. Correct the following problems if necessary and reset 
 metadata.
 tool_dependencies.xml - Exception attempting to parse 
 /mnt/toolsheddb/database/000/repo_26/tool_dependencies.xml: not 
 well-formed (invalid token): line 6, column 149
 **
 
 The tool_dependencies.xml file:
 **
 1 ?xml version=1.0?
 2 tool_dependency
 3 package name=transpose version=2.0.0
 4install version=1.0
 5 actions
 6 action 
 type=download_by_urlhttp://downloads.sourceforge.net/project/transpose/transpose/transpose-2.0/2.0/transpose-2.0.zip?r=ts=1380535239use_mirror=surfnet/action
 7action type=shell_commandmkdir bin/action
 8  action type=shell_commandunzip 
 transpose-2.0.zip/action
  action type=shell_commandcd transpose-2.0/src/action
  action type=shell_commandgcc transpose.c -o 
 transpose/action
  action type=move_file
 source$INSTALL_DIR/transpose-2.0/src/transpose/source
 destination$INSTALL_DIR/bin/destination
  /action
  action type=shell_commandchmod +x 
 $INSTALL_DIR/bin/transpose/action
  action type=set_environment
  environment_variable name=PATH 
 action=prepend_to$INSTALL_DIR/bin/environment_variable
  /action
  /actions
  /install
  readme
  Compiling transpose and putting in the path.
  /readme
 /package
 /tool_dependency
 **
 
 
 Cheers,
 Joachim
 


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


Re: [galaxy-dev] pass user groups to dynamic job runner

2013-09-27 Thread Nicola Soranzo
Il giorno ven, 20/09/2013 alle 10.25 -0500, John Chilton ha scritto: 
 On Fri, Sep 20, 2013 at 9:08 AM, Nicola Soranzo sora...@crs4.it wrote:
  First step is to create a new dynamic job destination, this is
  demonstrated in job_conf.xml in the above changeset. This demonstrates
  the new style job_conf section - this will need to be adapted for the
  old style runners but is still doable.
 
  Then you will want to add a dynamic job runner rule that implements
  this logic. This is the file lib/galaxy/jobs/rules/license_checker.py.
 
  Is there a way to specify in job_conf.xml a per-tool final destination
  after the has_license filtering ? E.g.:
 
  tool id=cat1 destination=has_license final_destination=local /
  tool id=cat2 destination=has_license
  final_destination=remote_cluster /
 
  The 'final_destination' attribute would then be used in has_license
  function instead of returning DEFAULT_JOB_DESTINATION_ID.
 
 
 There is no way currently to pass information from the tool tag to the
 dynamic runner or to have the dynamic runner pass on making a
 decision in some way. It could potentially make sense to add these -
 but I would put forward a workaround that in my opinion is actually a
 little better:
 ...

I have actually found a way to implement my idea of specifying a final
destination! I also generalized your approach giving the possibility to
also specify a per-tool required group in jobs_conf.xml .

job_conf.xml :

?xml version=1.0?
job_conf
plugins workers=4
plugin id=local type=runner
load=galaxy.jobs.runners.local:LocalJobRunner /
plugin id=drmaa type=runner
load=galaxy.jobs.runners.drmaa:DRMAAJobRunner /
/plugins
handlers
handler id=main/
/handlers
destinations default=remote_cluster
destination id=local runner=local /
destination id=is_user_in_group runner=dynamic
param id=functionis_user_in_group/param
/destination 
destination id=remote_cluster runner=drmaa /
/destinations
tools
tool id=tool1 destination=is_user_in_group
required_group=other_group /
tool id=tool2 destination=is_user_in_group
final_destination=local /
/tools
/job_conf



lib/galaxy/jobs/rules/is_user_in_group.py :

from galaxy.jobs.mapper import JobMappingException
DEFAULT_GROUP = 'have_license'

def is_user_in_group(user, app, tool_id):
# app is a galaxy.app.UniverseApplication object
if user is None:
raise JobMappingException('You must login to use this tool!')
user_group_assocs = user.groups or []
default_destination_id = app.job_config.get_destination(None)
for jtc in app.job_config.get_job_tool_configurations(tool_id): #
app.job_config.get_job_tool_configurations(tool_id) returns a list of
galaxy.jobs.JobToolConfiguration objects
if not jtc.params:
required_group = jtc.get('required_group', DEFAULT_GROUP)
final_destination = jtc.get('final_destination',
default_destination_id)
break
else:
required_group = DEFAULT_GROUP
final_destination = default_destination_id
user_in_group = required_group in [user_group_assoc.group.name for
user_group_assoc in user_group_assocs]
if not user_in_group:
raise JobMappingException('This tool is restricted to users in
the %s group, please contact a site administrator to be authorized!' %
required_group)
else:
return final_destination

Hope that helps,
Nicola


-- 
Nicola Soranzo, Ph.D.
CRS4
Bioinformatics Program
Loc. Piscina Manna
09010 Pula (CA), Italy
http://www.bioinformatica.crs4.it/

  There is no longer any need to display such tools to the user thanks
  to dynamic toolbox filters. This is that last file:
  lib/galaxy/tools/filters/license_filter.py. This will prevent any tool
  in the list 'LICENSED_TOOLS' from even being seen by unlicensed users.
  You do need to specify this filter is being used by adding the line:
 
  tool_filters = license_filter:has_license
 
  in the [app:main] section of universe_wsgi.ini.
 
  I think that something like this:
 
  # Tool filtering. Multiple such filters can be specified by giving
  # module (relative to galaxy.tools.filters) and function names for each
  # as a comma separated list.
  # See lib/galaxy/tools/filters/examples.py.sample for some examples.
  #tool_filters =
  #tool_label_filters =
  #tool_section_filters =
 
  should be added to universe_wsgi.ini.sample , otherwise this wonderful
  feature is very difficult to discover.
 
 Do you mean to suggest that merely documenting my contributions to
 Galaxy on my Twitter feed is insufficient? Hmm I don't know about
 that...
 
 Seriously though, Björn has implemented some extensions to dynamic
 toolbox filters that include this:
 
 https://bitbucket.org/galaxy/galaxy-central/pull-request/179/implement-the-ability-to-change-the-tool/diff#chg-universe_wsgi.ini.sample
 
 I have added more options to Galaxy then I have documented in that
 file and I am always

Re: [galaxy-dev] error in 2013_08_12 DevNewsBriefs

2013-09-25 Thread Nicola Soranzo

Il 2013-09-23 20:43 Curtis Hendrickson (Campus) ha scritto:

Dear Galaxy Team

I was upgrading our servers with the 8/12 release, and going over the
DevNotes [1].

I noted item Pull Requets Merged #5: SAMtools indexes #188 [2].

But the file created by the pull request didn't seem to exist.

After looking in bitbucket at the history, it looks like the pull
request was backed out [3] before the release.

Have I correctly understood the situation?


Dear Curtis,
unfortunately you're right, my pull request has been backed out before 
the latest release.
It seems that the Galaxy Team intention is to first move SAMtools and 
cufflinks wrapper to the Tool Shed and then re-apply my changes.


There is a Trello card if you would like to monitor the progresses:

https://trello.com/c/GvypU9T7/1021-tools-migrate-sam-fa-indices-tools-to-the-tool-shed-and-reimplement-nicola-soranzo-s-enhancements-to-this-data-table

Nicola


Regards,

Curtis


Links:
--
[1] http://wiki.galaxyproject.org/DevNewsBriefs/2013_08_12
[2]

http://wiki.galaxyproject.org/DevNewsBriefs/2013_08_12#Pull_Requests_Merged
[3]

https://bitbucket.org/galaxy/galaxy-central/commits/73d7a93e2f8d0bafbcb0b4dd9bf562d1fa6a88e0#general-comments


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/


Re: [galaxy-dev] pass user groups to dynamic job runner

2013-09-20 Thread Nicola Soranzo
Il giorno gio, 19/09/2013 alle 12.42 -0500, John Chilton ha scritto: 
 Feature request received.
 
 The following changeset demonstrates how one could implement this
 
 https://github.com/jmchilton/galaxy-central/commit/b500a24bc1aa94bef48db20a7cc1f3d68855b7fd

Hi John,
that's pretty interesting and we are probably going to use something
similar for some problematic tools, thanks for sharing!

 First step is to create a new dynamic job destination, this is
 demonstrated in job_conf.xml in the above changeset. This demonstrates
 the new style job_conf section - this will need to be adapted for the
 old style runners but is still doable.
 
 Then you will want to add a dynamic job runner rule that implements
 this logic. This is the file lib/galaxy/jobs/rules/license_checker.py.

Is there a way to specify in job_conf.xml a per-tool final destination
after the has_license filtering ? E.g.:

tool id=cat1 destination=has_license final_destination=local /
tool id=cat2 destination=has_license
final_destination=remote_cluster /

The 'final_destination' attribute would then be used in has_license
function instead of returning DEFAULT_JOB_DESTINATION_ID.

 There is no longer any need to display such tools to the user thanks
 to dynamic toolbox filters. This is that last file:
 lib/galaxy/tools/filters/license_filter.py. This will prevent any tool
 in the list 'LICENSED_TOOLS' from even being seen by unlicensed users.
 You do need to specify this filter is being used by adding the line:
 
 tool_filters = license_filter:has_license
 
 in the [app:main] section of universe_wsgi.ini.

I think that something like this:

# Tool filtering. Multiple such filters can be specified by giving
# module (relative to galaxy.tools.filters) and function names for each
# as a comma separated list.
# See lib/galaxy/tools/filters/examples.py.sample for some examples.
#tool_filters =
#tool_label_filters =
#tool_section_filters =

should be added to universe_wsgi.ini.sample , otherwise this wonderful
feature is very difficult to discover.

 Hope this helps!
 
 -John

Thanks again!

Nicola

 On Thu, Sep 19, 2013 at 2:45 AM, Vandeweyer Geert
 geert.vandewe...@uantwerpen.be wrote:
  Hi,
 
  Could somebody point me to the place where I can create a ticket for the 
  following feature:
 
  - We want to have tools available only to users who provided a licence for 
  this tool.
  - To prevent very long email lists (see example on dev-only tools), I'd 
  like to have a group 'have_licence'
  - in dynamic job runner function : check if user is in usergroup : ok = 
  run ; fail = give message.
 
  Or can I hide tools from the menu based on usergroup?
 
  Best,
 
  Geert
  ___
  Please keep all replies on the list by using reply all
  in your mail client.  To manage your subscriptions to this
  and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
 
  To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
   http://lists.bx.psu.edu/
 
 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


Re: [galaxy-dev] Defining $GALAXY_SLOTS for use in tool wrappers

2013-08-01 Thread Nicola Soranzo

Il 2013-07-30 17:18 Peter Cock ha scritto:

Hello all,

Re:
http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-June/010153.html
http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-October/011557.html

Something I raised during the GCC2013, and we talked about
via Twitter as well was a Galaxy environment variable for use
within Tool Wrappers setting the number of threads/CPUs to
use.

The idea is that you can configure a default value, and then
override this per runner or per tool etc.


Thanks Peter for pushing this idea, I totally support this proposal.
In the mean time, I've been using for my tools the solution by Jim 
Johnson for its CD-HIT wrapper:


http://toolshed.g2.bx.psu.edu/view/jjohnson/cdhit

But this requires the system administrator to modify both the tool 
env.sh and job_conf.xml to be in sync.



Is there an open Trello card for this?


A Trello card would be useful indeed.


Nicola
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/


Re: [galaxy-dev] blastn: Error: NCBI C++ Exception

2013-06-13 Thread Nicola Soranzo
Il giorno gio, 13/06/2013 alle 18.06 +0100, Peter Cock ha scritto: 
 On Thu, Jun 13, 2013 at 6:02 PM, Nicola Soranzo sora...@crs4.it wrote:
 
  There are two issues here, one that we would like to be able to
  handle multi-threading better via Galaxy configuration on a
  per-tool level, but for now it must often be hard coded or
  done via tool-specific environment variables.
 
  Regarding the configuration of multi-threading for BLAST+ tools, instead
  of hardcoding -num_threads 8 in the XML files, I'd like to do
  implement something like this:
 
  http://toolshed.g2.bx.psu.edu/repos/jjohnson/cdhit/rev/cca0838c1597
 
  What do you think? Should I prepare a pull request to discuss at
  GCC2013?
 
 That's a sensible idea, something like $BLASTTHREADS as
 an environment variable?

I was thinking something like BLAST_SITE_OPTIONS='-num_threads 8', which
could also include other parameters (e.g. memory limits) in the future.

 I'd still prefer something built into Galaxy like $THREADS or
 $GALAXYTHREADS or whatever which can be sets with a
 default value and adjusted in the per-tool job runner setup
 (e.g. send BLAST jobs to this cluster queue with 16 threads).
 
 This is definitely a good general topic for the tool authors
 and/or BLAST wrapping BoF sessions at the conference:
 http://wiki.galaxyproject.org/Events/GCC2013/BoF/ToolDevelopers
 http://wiki.galaxyproject.org/Events/GCC2013/BoF/GalaxyBlast
 
 Peter


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


Re: [galaxy-dev] ASN.1 (text and binary) formats in Galaxy Tool Shed

2013-04-03 Thread Nicola Soranzo
Il giorno mer, 03/04/2013 alle 14.27 +0100, Peter Cock ha scritto: 
 On Tue, Feb 19, 2013 at 2:00 PM, James Taylor ja...@jamestaylor.org wrote:
  On Tue, Feb 19, 2013 at 6:32 AM, Peter Cock p.j.a.c...@googlemail.com 
  wrote:
  Would a pull request implementing this be acceptable?
 
  Yes. My understanding is that ASN is a completely flexible metaformat,
  like XML, and so should be under either Text or Data, with appropriate
  subtypes defined for blast, et cetera.
 
 Nicola made the pull request we discussed a month ago:
 
 https://bitbucket.org/galaxy/galaxy-central/pull-request/135/add-generic-asn1-text-and-binary-datatypes/diff
 
 Could someone please review and commit this?

Please note that this pull request contains also this commit:

https://bitbucket.org/nsoranzo/galaxy-central/commits/62d0924f163b8731adaf582056587c0c4f415a7b

which fixes a bug where uploading an unsniffable binary file whose
filename has more than one dot (e.g. foo.bar.scf ) results in the error
'The uploaded binary file contains inappropriate content'.

Nicola

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


Re: [galaxy-dev] Problem running ncbi_makeblastdb as real user in cluster

2013-03-20 Thread Nicola Soranzo
Il giorno mar, 19/03/2013 alle 15.20 -0400, Raj Ayyampalayam ha
scritto: 
 One of the first few tools I tested was the ncbi makebkastdb tool. When 
 I try to run it the history for that particular job is immediately shown 
 as follows:
 
 0: (unnamed dataset)
 Failed to retrieve dataset information.
 An error occurred with this dataset:/hasattr(): attribute name must be 
 string/

Hi Raj,
this error in the history should be fixed by this patch:

https://bitbucket.org/nsoranzo/peterjc-galaxy-central-tools2/commits/35163cbc6c694f598ed1a1875353c58b275e2423

I have sent a pull request to Peter (the maintainer), hopefully he will
release soon a new version of blast_datatypes with this fix.

Best,
Nicola


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


Re: [galaxy-dev] Problem running ncbi_makeblastdb as real user in cluster

2013-03-20 Thread Nicola Soranzo
Il giorno mer, 20/03/2013 alle 12.41 +, Peter Cock ha scritto: 
 On Wed, Mar 20, 2013 at 10:08 AM, Nicola Soranzo sora...@crs4.it wrote:
  Il giorno mar, 19/03/2013 alle 15.20 -0400, Raj Ayyampalayam ha
  scritto:
  One of the first few tools I tested was the ncbi makebkastdb tool. When
  I try to run it the history for that particular job is immediately shown
  as follows:
 
  0: (unnamed dataset)
  Failed to retrieve dataset information.
  An error occurred with this dataset:/hasattr(): attribute name must be
  string/
 
  Hi Raj,
  this error in the history should be fixed by this patch:
 
  https://bitbucket.org/nsoranzo/peterjc-galaxy-central-tools2/commits/35163cbc6c694f598ed1a1875353c58b275e2423
 
  I have sent a pull request to Peter (the maintainer), hopefully he will
  release soon a new version of blast_datatypes with this fix.
 
  Best,
  Nicola
 
 The patch just removes the MetadataElement call - is that wise?

Hi Peter,
my question instead is, what are they for? With 'name' they have no
meaning.

 Looking at the code, I don't see a recent API change regarding
 the name:
 
 https://bitbucket.org/galaxy/galaxy-central/history-node/default/lib/galaxy/datatypes/metadata.py?at=default
 
 Is there perhaps a universe_wsgi.ini setting which might be involved,
 since I've not seen this error locally?

$ cd lib/galaxy/datatypes/
$ grep 'MetadataElement(' *.py|grep -v name

returns nothing, so the 'name' parameter is really mandatory.

Best,
Nicola

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


Re: [galaxy-dev] ASN.1 (text and binary) formats in Galaxy Tool Shed

2013-02-22 Thread Nicola Soranzo
Il giorno mar, 19/02/2013 alle 20.38 +, Peter Cock ha scritto: 
 On Tue, Feb 19, 2013 at 6:45 PM, Nicola Soranzo sora...@crs4.it wrote:
  Il giorno mar, 19/02/2013 alle 14.15 +, Peter Cock ha scritto:
  On Tue, Feb 19, 2013 at 2:00 PM, James Taylor ja...@jamestaylor.org 
  wrote:
   On Tue, Feb 19, 2013 at 6:32 AM, Peter Cock wrote:
  I think it could make sense to define generic 'asn1' and
  'asn1-binary' formats in the Galaxy core (name suggestions
  welcome)
 
  What about
 
  extension=asn type=galaxy.datatypes.data:GenericAsn1
 
  and
 
  extension=asnb type=galaxy.datatypes.binary:GenericAsn1Binary
 
  like GenericXml class?
 
 Those seem sensible to me as the class names, although I'm
 not so sure about the format names (aka 'extensions' in Galaxy
 terms). I'd prefer to see the '1' in the name for clarity. My
 suggestions of 'asn1' and 'asn1-binary' were based on NCBI usage.

Ok, no problem!

  Nicola - do you want to make a pull request to galaxy-central defining
  ASN.1 text and binary formats (which we can then subclass for the
  NCBI BLAST+ wrappers)? Or should I?
 
  If you mean a minimal implementation, I can surely do that. If something
  more elaborated is needed, then probably you are more qualified than me!
 
 The current minimal implementation you sent me for BLAST+
 would be an excellent start. Things like data type sniffers etc
 would be a nice to have feature, but can be added later I think.
 And the sooner this gets into the Galaxy core, the sooner we
 can use it in the BLAST+ wrappers :)

Before proceeding on this route, one question: we are going to introduce
a dependency for blast_datatypes/ncbi_blast_plus on a future galaxy
release, is it ok?
I don't think it is possible to express such dependency in a
repository_dependencies.xml file yet.

Best,
Nicola
-- 
Nicola Soranzo, Ph.D.
CRS4
Bioinformatics Program
Loc. Piscina Manna
09010 Pula (CA), Italy
http://www.bioinformatica.crs4.it/

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


Re: [galaxy-dev] ASN.1 (text and binary) formats in Galaxy Tool Shed

2013-02-19 Thread Nicola Soranzo
Il giorno mar, 19/02/2013 alle 14.15 +, Peter Cock ha scritto: 
 On Tue, Feb 19, 2013 at 2:00 PM, James Taylor ja...@jamestaylor.org wrote:
  On Tue, Feb 19, 2013 at 6:32 AM, Peter Cock p.j.a.c...@googlemail.com 
  wrote:
 I think it could make sense to define generic 'asn1' and
 'asn1-binary' formats in the Galaxy core (name suggestions
 welcome)

What about 

extension=asn type=galaxy.datatypes.data:GenericAsn1

and

extension=asnb type=galaxy.datatypes.binary:GenericAsn1Binary 

like GenericXml class?

  Would a pull request implementing this be acceptable?
 
  Yes. My understanding is that ASN is a completely flexible metaformat,
  like XML, and so should be under either Text or Data, with appropriate
  subtypes defined for blast, et cetera.
 
 Thank James,
 
 Yes - very like XML, but with the subtlety that ASN.1 comes in text and
 binary favours (which I presume applies to all the variants, although
 the binary versions may not be as commonly used for the smaller files).
 
 Nicola - do you want to make a pull request to galaxy-central defining
 ASN.1 text and binary formats (which we can then subclass for the
 NCBI BLAST+ wrappers)? Or should I?

If you mean a minimal implementation, I can surely do that. If something
more elaborated is needed, then probably you are more qualified than me!

 I think the mime-type for the base ASN.1 text format should probably
 be text/plain based on the NCBI usage patterns.

Ok.

 I'm not sure what the mime-type for the base ASN.1 binary format
 should be (but I don't think it should be chemical/ncbi-asn1-binary).

application/octet-stream ?

Best,
Nicola

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


Re: [galaxy-dev] DustMasker tool for ncbi_blast_plus

2013-02-15 Thread Nicola Soranzo
Il giorno lun, 11/02/2013 alle 13.19 +, Peter Cock ha scritto:
 On Fri, Feb 8, 2013 at 4:30 PM, Nicola Soranzo sora...@crs4.it wrote:
  Il giorno mer, 06/02/2013 alle 20.01 +0100, Nicola Soranzo ha scritto:
  Hi Peter,
  I added these file formats mostly as placeholders for a future
  implementation. Now I have changed a bit the tool by removing acclist
  and seqloc_xml formats since they are not recognized by the last
  versions of dustmasker (I also sent an email to
  blast-h...@ncbi.nlm.nih.gov to inform them of this bug).
  As before, you can find the new version at:
 
  https://bitbucket.org/nsoranzo/ncbi_blast_plus
 
  I stripped the old commit and did a new one, not a very good practice,
  sorry about that.
 
 It seems to have confused the bitbucket page a little, but I have
 checked in your initial wrapper to my development repository (I
 use the tools branch):
 https://bitbucket.org/peterjc/galaxy-central/commits/2284d485e36f74f19b0dbe78709b098d9eba4ef6
 
 Note I'm not going to include this in the Tool Shed release yet,
 we need to sort out the file format definitions first.


Hi Peter,
I implemented minimal datatypes for maskinfo ASN.1 binary and text, plus
some other improvements to ncbi_blast_plus, and I sent you a pull
request through Bitbucket for your development repository. I think that
would be easier for you, let me know if it is not.

Nicola

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


[galaxy-dev] Datatype display_in_upload not respected in upload1

2013-02-14 Thread Nicola Soranzo
Hi all,
I noticed that when I go in Get Data - Upload File from your
computer (i.e. upload1 tool) the File Format list contains also
datatypes which have display_in_upload=false in datatypes_conf.xml
(e.g. afg, fli), while the list does not contain datatypes which
have no display_in_upload attribute at all (e.g. bedstrict, bed6).
This sounds like a bug to me, but I have not found any reference around.

Thanks,
Nicola
-- 
Nicola Soranzo, Ph.D.
CRS4
Bioinformatics Program
Loc. Piscina Manna
09010 Pula (CA), Italy
http://www.bioinformatica.crs4.it/

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


Re: [galaxy-dev] DustMasker tool for ncbi_blast_plus

2013-02-08 Thread Nicola Soranzo
Il giorno mer, 06/02/2013 alle 20.01 +0100, Nicola Soranzo ha scritto: 
 Hi Peter,
 I added these file formats mostly as placeholders for a future
 implementation. Now I have changed a bit the tool by removing acclist
 and seqloc_xml formats since they are not recognized by the last
 versions of dustmasker (I also sent an email to
 blast-h...@ncbi.nlm.nih.gov to inform them of this bug).
 As before, you can find the new version at:
 
 https://bitbucket.org/nsoranzo/ncbi_blast_plus
 
 I stripped the old commit and did a new one, not a very good practice,
 sorry about that.

Hi Peter,
I've added a new commit to this repo which updates the test output files
to (recommended) BLAST 2.2.26+, since functional tests were returning
errors.

Hope you find it useful.

Nicola
-- 
Nicola Soranzo, Ph.D.
CRS4
Bioinformatics Program
Loc. Piscina Manna
09010 Pula (CA), Italy
http://www.bioinformatica.crs4.it/

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


Re: [galaxy-dev] DustMasker tool for ncbi_blast_plus

2013-02-06 Thread Nicola Soranzo
Adding galaxy-dev list in CC as suggested by Peter.

Il giorno mer, 06/02/2013 alle 16.57 +, Peter Cock ha scritto: 
 On Tue, Feb 5, 2013 at 11:45 AM, Nicola Soranzo sora...@crs4.it wrote:
  Dear Peter,
  I have created a simple Galaxy tool for DustMasker of the NCBI BLAST+
  suite, which I think would be a useful addition to the ncbi_blast_plus
  repository you're maintaining in the Galaxy Tool Shed.
 
  You can find it and hopefully pull it from:
 
  https://bitbucket.org/nsoranzo/ncbi_blast_plus
 
  Kind regards,
  Nicola
 
 Hi Nicola,
 
 Thanks for getting involved - we can discuss this on the galaxy-dev
 mailing list if you prefer? For now I have CC'd Edward Kirton as he
 is/was working on masking in BLAST databases for Galaxy.
 
 I can see the new file
 tools/ncbi_blast_plus/ncbi_dustmasker_wrapper.xml however it refers to
 multiple new file formats - where are they defined?
 
 * acclist
 * maskinfo_asn1_bin
 * maskinfo_asn1_text
 * seqloc_asn1_bin
 * seqloc_asn1_text

Hi Peter,
I added these file formats mostly as placeholders for a future
implementation. Now I have changed a bit the tool by removing acclist
and seqloc_xml formats since they are not recognized by the last
versions of dustmasker (I also sent an email to
blast-h...@ncbi.nlm.nih.gov to inform them of this bug).
As before, you can find the new version at:

https://bitbucket.org/nsoranzo/ncbi_blast_plus

I stripped the old commit and did a new one, not a very good practice,
sorry about that.

 Have you looked at the (commented out) bits in the makeblastdb wrapper
 which would perhaps be relevant? This is something Edward Kirton wrote
 which I haven't integrated yet:
 
 !-- SEQUENCE MASKING OPTIONS --
 !-- TODO
 repeat name=mask_data title=Provide one or more files
 containing masking data
 param name=file type=data format=asnb label=File
 containing masking data help=As produced by NCBI masking
 applications (e.g. dustmasker, segmasker, windowmasker) /
 /repeat
 repeat name=gi_mask title=Create GI indexed masking data
 param name=file type=data format=asnb label=Masking
 data output file /
 /repeat
 --
 
 Perhaps all you need to offer in ncbi_dustmasker_wrapper.xml is
 'fasta' and 'asnb' (binary ASN) formats? Edward - did you have an
 'asnb' definition?

'fasta' and 'interval' are the ones I'm interested for my use case.

'maskinfo_asn1_bin' is probably the one referenced as 'asnb' in the
cited code (ASN1 is a general data serialization format like XML). A
file in this format can be given as input to makeblastdb -mask_data.

Nicola
-- 
Nicola Soranzo, Ph.D.
CRS4
Bioinformatics Program
Loc. Piscina Manna
09010 Pula (CA), Italy
http://www.bioinformatica.crs4.it/

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


Re: [galaxy-dev] File Upload doesn't generate any activity

2013-02-04 Thread Nicola Soranzo
Il giorno lun, 04/02/2013 alle 16.33 +, Hakeem Almabrazi ha
scritto: 
 Thank you Dannon.  Hopefully things should be moving forward from now on.
 
 This issue with the LWR has been fixed in changeset 8731:3299529e0fe8 which 
 will be in the next distribution release.
 
 That will be awesome.

It's already in galaxy-dist since Saturday 2013/02/02, changeset
8531:3299529e0fe8 .

Kind Regards,
Nicola

-- 
Nicola Soranzo, Ph.D.
CRS4
Bioinformatics Program
Loc. Piscina Manna
09010 Pula (CA), Italy
http://www.bioinformatica.crs4.it/

 -Original Message-
 From: Dannon Baker [mailto:dannonba...@me.com] 
 Sent: Monday, February 04, 2013 9:40 AM
 To: Hakeem Almabrazi
 Cc: Galaxy Dev
 Subject: Re: [galaxy-dev] File Upload doesn't generate any activity
 
 On Feb 4, 2013, at 10:36 AM, Hakeem Almabrazi halmabr...@idtdna.com wrote:
  No there was nothing special about this installation.  I installed it on 
  CentOS and I tried to follow instructions in installing and configuring 
  tools and data.  The only thing that I did not was to test this feature 
  before I integrate it with postgres, nginx, tools such as bwa, samtools, 
  bowtie etc.  I might have done something not right while doing so.  But 
  once I was done with all the configuration steps, I tried to load files via 
  Get Data or Data Library.  It kept spinning and nothing is actually 
  being written to the system.  I tailed the logs and it was writing some 
  logs every seconds but no errors or warnings.  
  
  The issue was resolved when I used URL/Text to go to this link as another 
  trail test.  After that everything works just fine.  Now we can load files 
  via either way.
 
 Very strange, but I'm glad it's working for you now.  If this does crop up 
 again, let me know and we can try to reproduce what's happening and fix it.
 
  The only error I keep seeing in the log is when we restart the server and 
  from what I read, this is a bug...
  
  Traceback (most recent call last):
   File /Users/dtenenba/dev/galaxy-dist/lib/galaxy/jobs/handler.py,
  line 447, in _load_plugin
 module = __import__( module_name )
   File /Users/dtenenba/dev/galaxy-dist/lib/galaxy/jobs/runners/lwr.py,
  line 232
 worker = threading.Thread( ( name=LwrJobRunner.thread-%d % i ), 
  target=self.run_next )
  ^
  SyntaxError: invalid syntax
  
  
  
  Please let me know if you want me to do anything else that might help 
  debugging this issue.


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


Re: [galaxy-dev] File Upload doesn't generate any activity

2013-02-04 Thread Nicola Soranzo
Il giorno lun, 04/02/2013 alle 17.41 +, Hakeem Almabrazi ha
scritto: 
 Thank you Nicola,
 
 Should I replace my current galaxy-dist with this version or is there 
 different way to upgrade?

Take a look at this wiki page:

http://wiki.galaxyproject.org/Admin/Get%
20Galaxy#Keep_your_code_up_to_date

Nicola

 -Original Message-
 From: Nicola Soranzo [mailto:sora...@crs4.it] 
 Sent: Monday, February 04, 2013 10:50 AM
 To: Hakeem Almabrazi
 Cc: Dannon Baker; Galaxy Dev
 Subject: Re: [galaxy-dev] File Upload doesn't generate any activity
 
 Il giorno lun, 04/02/2013 alle 16.33 +, Hakeem Almabrazi ha
 scritto: 
  Thank you Dannon.  Hopefully things should be moving forward from now on.
  
  This issue with the LWR has been fixed in changeset 8731:3299529e0fe8 
  which will be in the next distribution release.
  
  That will be awesome.
 
 It's already in galaxy-dist since Saturday 2013/02/02, changeset
 8531:3299529e0fe8 .
 
 Kind Regards,
 Nicola
 


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


[galaxy-dev] Pull request for bowtie_wrappers tool

2013-01-28 Thread Nicola Soranzo
Dear Galaxy DevTeam,
I have made a few updates to the bowtie_wrappers tool from the Tool
Shed, may someone please pull from:

https://bitbucket.org/nsoranzo/bowtie_wrappers/

for the following changes:
- Add fields for --nofw and --norc options also for single-end reads
- Remove field for --cutoff option of bowtie-build
- Do not show fields for --best and --strata options for paired-end
reads
- Add control to choose between -a and -k options
- Show fields for --pairtries and --maxbts options only when noTryHard
- Add control to choose between Maq- and SOAP-like alignment
- Add range validators to all integer parameters

I am not sure this mailing list is the correct place for such pull
requests, if not please advise me!

Thanks,
Nicola
-- 
Nicola Soranzo, Ph.D.
CRS4
Bioinformatics Program
Loc. Piscina Manna
09010 Pula (CA), Italy
http://www.bioinformatica.crs4.it/

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/