[galaxy-dev] Re: pbs_python + python 3 issue

[Peter Cock via galaxy-dev]

From the syntax I can see pbs-python still uses a Python 2 style
print, it will need updating for Python 3.

I suggest opening an issue on the repository, and seeing what the
author says. I would guess the Python 2 to 3 changes will be fairly
easy, but worry that there have been other changes to Galaxy in the
meantime which would be more trouble for this PBS extension as it does
not appear to have been updated for 6 years.

Peter

On Mon, Nov 9, 2020 at 5:34 AM Sandra Maksimovic
 wrote:
>
> Hi,
>
> Has anyone out there had any success installing the pbs_python module into a 
> Python 3+ Galaxy installation? I run into the following issue when installing 
> https://github.com/ehiggs/pbs-python as per the documentation.
>
> # python setup.py install
>   File "setup.py", line 68
> print "Version: %s is not supported" %(PBS_VERSION)
>^
> SyntaxError: invalid syntax
>
> I provide the PBS/Torque version via the PBS_PYTHON_INCLUDE_DIR environment 
> variable but it doesn't seem to detect the version properly.
>
> Thanks,
> Sandra
>
> Disclaimer
>
> This e-mail and any attachments to it (the "Communication") are, unless 
> otherwise stated, confidential, may contain copyright material and is for the 
> use only of the intended recipient. If you receive the Communication in 
> error, please notify the sender immediately by return e-mail, delete the 
> Communication and the return e-mail, and do not read, copy, retransmit or 
> otherwise deal with it. Any views expressed in the Communication are those of 
> the individual sender only, unless expressly stated to be those of Murdoch 
> Children’s Research Institute (MCRI) ABN 21 006 566 972 or any of its related 
> entities. MCRI does not accept liability in connection with the integrity of 
> or errors in the Communication, computer virus, data corruption, interference 
> or delay arising from or in respect of the Communication.
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   %(web_page_url)s
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  %(web_page_url)s

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

[galaxy-dev] Re: How to include a large datafile in a bioconda package?

[Peter Cock via galaxy-dev]

That seems a good compromise within conda, since BioConda
wouldn't want the binary package itself to be too big.

(I'm doing something similar with some real sample data for
a tool, putting it up on Zenodo. Of course, this is optional for
my tool - your use case is different.)

The Galaxy Data Manager route seems more appropriate if
there is a choice of large data files which could be used with
the tool (not just one).

Peter

On Wed, Jul 24, 2019 at 10:26 PM Björn Grüning
 wrote:
>
> Hi Jin,
>
> you can use a post-link script in conda.
>
> Like here:
> https://github.com/bioconda/bioconda-recipes/blob/master/recipes/picrust2/post-link.sh
>
> This way the data can be fetch during tool installation.
>
> See more information here:
> https://docs.conda.io/projects/conda-build/en/latest/resources/link-scripts.html
>
> Ciao,
> Bjoern
>
> Am 24.07.19 um 18:43 schrieb Jin Li:
> > Hi Brad,
> >
> > Thank you for your quick reply. I can put the data file to Zenodo so
> > that I will have a permanent location for it.
> >
> > As for re-computing the data file locally, it may need several days to
> > run, so it may be quite inefficient to do the computing. I am
> > expecting an automatic download of the data file when installing the
> > package. Do we have a convention to do that? Thank you.
> >
> > Best regards,
> > Jin
> >
> > On Wed, Jul 24, 2019 at 11:31 AM Langhorst, Brad  wrote:
> >>
> >> Hi:
> >>
> >> I’d be concerned about that file changing or disappearing and causing 
> >> irreproducibility.
> >> If the URL were to a permanent location (e.g. NCBI or zenodo) maybe it’s 
> >> ok.
> >>
> >> Could it be re-computed locally if necessary (like a genome index)?
> >>
> >> Maybe others know of examples where this is done.
> >>
> >>
> >> Brad
> >>
> >> On Jul 24, 2019, at 12:24 PM, Jin Li  wrote:
> >>
> >> Hi all,
> >>
> >> I am not sure if this mailing list is a good place to ask a bioconda
> >> question. Sorry to bother if not. I want to ask how to include a large
> >> data file when publishing a bioconda package. Our program depends on a
> >> pre-computed data file, which is too large to be included in the
> >> source code package. The data file can be accessed via a public URL.
> >> Can I put the downloading command in `build.sh` when publishing a
> >> bioconda package? If not, do we have a convention to deal with
> >> dependent large datafiles? Thank you.
> >>
> >> Best regards,
> >> Jin
> >> ___
> >> Please keep all replies on the list by using "reply all"
> >> in your mail client.  To manage your subscriptions to this
> >> and other Galaxy lists, please use the interface at:
> >>   %(web_page_url)s
> >>
> >> To search Galaxy mailing lists use the unified search at:
> >>   http://galaxyproject.org/search/
> >>
> >>
> >> Bradley W. Langhorst, Ph.D.
> >> Development Group Leader
> >> New England Biolabs
> >>
> >>
> >>
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client.  To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> >%(web_page_url)s
> >
> > To search Galaxy mailing lists use the unified search at:
> >http://galaxyproject.org/search/
> >
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   %(web_page_url)s
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  %(web_page_url)s

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

[galaxy-dev] Re: Share Docker Volumes when integrating Containers in Galaxy

[Peter Cock via galaxy-dev]

I don't have any advise from the BLAST DB side, but I would
expect the same solution to work with any *.loc file and the
file system it points at.

Peter

On Mon, May 20, 2019 at 10:50 AM Schäfer, Richard
 wrote:
>
> Hello,
>
> I want to integrate a Docker container as a tool in Galaxy. This works great. 
> However,
> I need the forward a Volume to the container (at least that how it works 
> outside of Galaxy).
> In particular, this would be the BLAST database.
>
> This means, I would want to share the location specified in blastdb.loc and 
> forward it to the
> container. Is there a way to do this?
>
> I see that there is an option to expose volumes to containers by specifying 
> it in the job.conf.xml
>
>  id="docker_volumes">$defaults,/mnt/galaxyData/libraries:ro,/mnt/galaxyData/indices:ro
>
> However, I guess this will then be done for all containers? And how do I 
> integrate blastdb.loc in that command?
> For that, my tool config looks like this:
>
> 
>
>
>
>
> 
>
> I simply want to access the path inside of the container and therefore shared 
> it with the container.
>
> Thanks for any help
>
> Cheers
> Rich
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   %(web_page_url)s
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  %(web_page_url)s

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Fastq.gz datatype not recognized on import to data library

[Peter Cock]

Christopher filed this as https://github.com/galaxyproject/galaxy/issues/7068

On Mon, Dec 3, 2018 at 4:01 PM Previti
 wrote:
>
> Ok, I submitted it, hope somebody can help with this.
>
> Cheers,
> Christoper
>
> On 12/3/18 3:20 PM, Nicola Soranzo wrote:
>
> Hi Christopher,
> can you please submit an issue for this at 
> https://github.com/galaxyproject/galaxy/issues/ with the relevant logs and/or 
> tracebacks?
>
> Cheers,
> Nicola
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Get Data and 'xlsx'

[Peter Cock]

Thanks Martin, and D K,

I've commented on https://github.com/galaxyproject/galaxy/issues/6849
but am puzzled as to what might be going on here. The BLAST XML
sniffer code (which I wrote) is deliberately very selective with the intent
of avoid any false positives.

I would try it locally (thank you for sharing the test file), but right now
it looks like a recent cluster change has disrupted our local Galaxy
instance.

Peter
On Wed, Oct 10, 2018 at 7:20 PM Martin Čech  wrote:
>
> Hi all,
>
> I can confirm this file is sniffed as `format: blastxml` on usegalaxy.org, 
> which is surely wrong.
>
> I created an issue for it here: 
> https://github.com/galaxyproject/galaxy/issues/6849
>
> M.
>
>
>
> On Wed, Oct 10, 2018 at 2:11 PM D K  wrote:
>>
>> Sure, I've attached a sample xlsx file and my datatypes_conf.xml file
>>
>> Thanks!
>>
>> On Tue, Oct 9, 2018 at 11:52 PM Peter Cock  wrote:
>>>
>>> That sounds very strange - could you share the test file (ideally publicly, 
>>> but at least by private email)? Modern MS files do use XML but I don't 
>>> understand how it would be recognised as BLAST XML...
>>>
>>> It may also be useful to see your Galaxy data types configuration file.
>>>
>>> Peter
>>>
>>>
>>> On Wed, 10 Oct 2018 at 01:39, D K  wrote:
>>>>
>>>> It looks like uploading 'xlsx' data through 'Get Data' is not working. 
>>>> When I try this on my local Galaxy (v18.05) and on usegalaxy.org it gets 
>>>> automatically converted to 'BLAST xml' when using Automatic Detection. If 
>>>> I select 'xlsx' during uploading I get the error message ( Warning: The 
>>>> file 'Type' was set to 'xlsx' but the file does not appear to be of that 
>>>> type  )
>>>> Does anyone have any suggestions?
>>>> ___
>>>> Please keep all replies on the list by using "reply all"
>>>> in your mail client.  To manage your subscriptions to this
>>>> and other Galaxy lists, please use the interface at:
>>>>   https://lists.galaxyproject.org/
>>>>
>>>> To search Galaxy mailing lists use the unified search at:
>>>>   http://galaxyproject.org/search/
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Get Data and 'xlsx'

[Peter Cock]

That sounds very strange - could you share the test file (ideally publicly,
but at least by private email)? Modern MS files do use XML but I don't
understand how it would be recognised as BLAST XML...

It may also be useful to see your Galaxy data types configuration file.

Peter

On Wed, 10 Oct 2018 at 01:39, D K  wrote:

> It looks like uploading 'xlsx' data through 'Get Data' is not working.
> When I try this on my local Galaxy (v18.05) and on usegalaxy.org it gets
> automatically converted to 'BLAST xml' when using Automatic Detection. If I
> select 'xlsx' during uploading I get the error message ( Warning: The
> file 'Type' was set to 'xlsx' but the file does not appear to be of that
> type  )
> Does anyone have any suggestions?
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] tool data confusion

[Peter Cock]

Hi Matthias,

I'm getting out of my depth here - but if tool-data/xxx.loc is being ignored,
you may need to enable this by adding the XML data table entries from
the relevant tool_data_table_conf.xml.sample file for tool xxx to
$GALAXY/config/tool_data_table_conf.xml.

At least, that's what I had to do and documented recently here:
https://github.com/abaizan/kodoja_galaxy/commit/2cd7579a15887ae4ffdc5ab3a346681ebb53b0a2

Again, this may be me sticking to old pre-tool shed habits - so I'd like
to hear how other people manage their *.loc files, especially when doing
hand editing to add entries.

Peter
On Tue, Oct 2, 2018 at 10:29 AM Matthias Bernt  wrote:
>
> Hi Peter,
>
> then the I need to update all the paths in:
>
> `config/shed_tool_data_table_conf.xml`?
>
> Because currently the $GALAXY/tool-data/*.loc files are ignored in my
> instance.
>
> For me it looks like a bug (caused by a misconfiguration?) that there
> are all xml files refer to the same loc file.
>
> Wouldn't this be more useful:
>
> - config/shed_tool_data_table_conf.xml -> tool-data/diamond_database.loc
>
> -
> tool-data/toolshed.g2.bx.psu.edu/repos/bgruening/diamond/64be1ac21109/tool_data_table_conf.xml
> ->
> tool-data/toolshed.g2.bx.psu.edu/repos/bgruening/diamond/64be1ac21109/diamond_database.loc
>
> Best,
> Matthias
>
>
> On 02.10.2018 11:22, Peter Cock wrote:
> > Personally I tend to work with $GALAXY/tool-data/*.loc (and ignore
> > the tool shed installed copies in their cryptically named folders) but
> > this reflects in part the fact that this used to be the only copy of the 
> > file,
> > and that's just what I always did.
> >
> > If you use a data manager, I don't know which copy it updates -
> > but the merging design is meant to hide these details I suppose.
> >
> > Peter
> >
> > On Mon, Oct 1, 2018 at 5:56 PM Matthias Bernt  wrote:
> >>
> >> Dear list,
> >>
> >> I still have problems to get my head around tool data. Lets consider
> >> diamond for example (lets ignore data managers for the moment). After
> >> installation it seems that there are two relevant xml files:
> >>
> >> - `config/shed_tool_data_table_conf.xml`
> >> -
> >> `tool-data/toolshed.g2.bx.psu.edu/repos/bgruening/diamond/64be1ac21109/tool_data_table_conf.xml`
> >>
> >> and two loc files:
> >>
> >> - `tool-data/diamond_database.loc`
> >> -
> >> `tool-data/toolshed.g2.bx.psu.edu/repos/bgruening/diamond/64be1ac21109/diamond_database.loc`
> >>
> >> In both xml files the latter loc file is referenced. And therefor the
> >> tool-data/diamond_database.loc file is seemingly ignored.
> >>
> >> What is the rational to have a loc and xml file for each tool version?
> >> It seems that they are merged upon startup anyway.
> >>
> >> What would be the best way to administrate a single tool loc file?
> >>
> >> Cheers,
> >> Matthias
> >>
> >> --
> >>
> >> ---
> >> Matthias Bernt
> >> Bioinformatics Service
> >> Molekulare Systembiologie (MOLSYB)
> >> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
> >> Helmholtz Centre for Environmental Research GmbH - UFZ
> >> Permoserstraße 15, 04318 Leipzig, Germany
> >> Phone +49 341 235 482296,
> >> m.be...@ufz.de, www.ufz.de
> >>
> >> Sitz der Gesellschaft/Registered Office: Leipzig
> >> Registergericht/Registration Office: Amtsgericht Leipzig
> >> Handelsregister Nr./Trade Register Nr.: B 4703
> >> Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board:
> >> MinDirig Wilfried Kraus
> >> Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
> >> Prof. Dr. Dr. h.c. Georg Teutsch
> >> Administrative Geschäftsführerin/ Administrative Managing Director:
> >> Prof. Dr. Heike Graßmann
> >> ---
> >> ___
> >> Please keep all replies on the list by using "reply all"
> >> in your mail client.  To manage your subscriptions to this
> >> and other Galaxy lists, please use the interface at:
> >>https://lists.galaxyproject.org/
> >>
> >> To search Galaxy mailing lists use the unified search at:
> >>http://galaxyproject.org/search/
>
> --
>
> ---
> Matthias Bernt
> Bioinformatics

Re: [galaxy-dev] tool data confusion

[Peter Cock]

Personally I tend to work with $GALAXY/tool-data/*.loc (and ignore
the tool shed installed copies in their cryptically named folders) but
this reflects in part the fact that this used to be the only copy of the file,
and that's just what I always did.

If you use a data manager, I don't know which copy it updates -
but the merging design is meant to hide these details I suppose.

Peter

On Mon, Oct 1, 2018 at 5:56 PM Matthias Bernt  wrote:
>
> Dear list,
>
> I still have problems to get my head around tool data. Lets consider
> diamond for example (lets ignore data managers for the moment). After
> installation it seems that there are two relevant xml files:
>
> - `config/shed_tool_data_table_conf.xml`
> -
> `tool-data/toolshed.g2.bx.psu.edu/repos/bgruening/diamond/64be1ac21109/tool_data_table_conf.xml`
>
> and two loc files:
>
> - `tool-data/diamond_database.loc`
> -
> `tool-data/toolshed.g2.bx.psu.edu/repos/bgruening/diamond/64be1ac21109/diamond_database.loc`
>
> In both xml files the latter loc file is referenced. And therefor the
> tool-data/diamond_database.loc file is seemingly ignored.
>
> What is the rational to have a loc and xml file for each tool version?
> It seems that they are merged upon startup anyway.
>
> What would be the best way to administrate a single tool loc file?
>
> Cheers,
> Matthias
>
> --
>
> ---
> Matthias Bernt
> Bioinformatics Service
> Molekulare Systembiologie (MOLSYB)
> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
> Helmholtz Centre for Environmental Research GmbH - UFZ
> Permoserstraße 15, 04318 Leipzig, Germany
> Phone +49 341 235 482296,
> m.be...@ufz.de, www.ufz.de
>
> Sitz der Gesellschaft/Registered Office: Leipzig
> Registergericht/Registration Office: Amtsgericht Leipzig
> Handelsregister Nr./Trade Register Nr.: B 4703
> Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board:
> MinDirig Wilfried Kraus
> Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
> Prof. Dr. Dr. h.c. Georg Teutsch
> Administrative Geschäftsführerin/ Administrative Managing Director:
> Prof. Dr. Heike Graßmann
> ---
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] "Merging tabular data tables with non-matching columns is not allowed"

[Peter Cock]

Hi Peter vH,

I wonder if adding columns is not supported - because some tools
would define the table without the extra column? You might have
to define a new data table instead (with largely the same content).
But to be clear, I am not an expert in this area of Galaxy.

However, it does seem the IUC could lay out some *.loc file best
practises for tool authors. There is currently a TODO entry here:

https://galaxy-iuc-standards.readthedocs.io/en/latest/best_practices/tool_xml.html#data-managers

Peter C.

On Mon, Sep 17, 2018 at 10:56 PM Peter van Heusden  wrote:

> A bit more tracing leads me to suspect that what is happening is that a
> merge is being called here:
>
>
> https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/tools/data/__init__.py#L120
>
> from:
>
>
> https://github.com/galaxyproject/galaxy/blob/dev/lib/tool_shed/tools/data_table_manager.py#L88
>
> i.e. the toolshed maintains an in-memory copy of tool-data tables and if
> the list of columns in an incoming tool-data table is not the same as the
> in-memory version the error I saw happens.
>
> Which leaves me with the question: how is one meant to update the column
> list of a tool data table?
>
> Peter
>
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] collections with more than 25,000 items

[Peter Cock]

There is a sweet spot for splitting your BLAST query fasta file
by sequence - one big file with 25000 sequences is not great,
but one sequence per file is the worst possible option.

This is due to all the extra overheads, you would have 25000
jobs submitted to the cluster, each of which would load the
BLAST binary and database off disk etc. And there are also
going to be Galaxy overheads with a large collection as well.

I would suggest somewhere around 500 to 1000 gene sequences
per FASTQ query file is likely a safe choice. If you have very
long sequences (e.g. chromosomes or contigs), then use less.

As to the number of threads for each BLAST job, more is better,
but what to pick will depend on your cluster and how often there
are threads free on nodes. I would suggest trying 4, 8 or 16 threads.

I hope that helps.

Peter


On Thu, Aug 30, 2018 at 3:50 PM Jochen Bick  wrote:
>
> Thanks Peter,
>
> so my idea was to split my problem into single blast jobs and run them
> only on one core...
> So my file has 25000 sequences and I'm blasting them against all NCBI
> proteins (nr). This just take to long time. I guess because the database
> is also very big? I tested this on the first 10 sequences and it took
> about 10mins. But maybe this is still not faster than running all at once?
> How many cores would you give such a job?
>
> Cheers Jochen
>
> On 30.08.2018 16:44, Peter Cock wrote:
> > If there are any limits, it would be down to the Galaxy Admin's job
> > settings - something generic with collections.
> >
> > Personally I've not done this - I tend to concatenate FASTA files
> > to make large files with multiple sequences instead.
> >
> > (And then we have the optional task splitting enabled so that Galaxy
> > breaks up the multiple-sequence FASTA file into chunks which
> > get shared out on our cluster for better throughput before
> > concatenating the output back into a single file.)
> >
> > Peter
> > On Thu, Aug 30, 2018 at 3:37 PM Jochen Bick  
> > wrote:
> >>
> >> Hi,
> >>
> >> is there any limit to run BLAST jobs from a collection of single FASTA
> >> files? I started a job but is does not get executed... its just sending
> >> for about an hour.
> >>
> >> Cheers Jochen
> >>
> >> --
> >> ETH Zurich
> >> *Jochen Bick*
> >> Animal Physiology
> >> Institute of Agricultural Sciences
> >> Postal address: Universitätstrasse 2 / LFW B 58.1
> >> Office: Tannenstrasse 1 / TAN D 6.2
> >> 8092 Zurich, Switzerland
> >>
> >> Phone +41 44 632 28 25
> >> jochen.b...@usys.ethz.ch <mailto:jochen.b...@usys.ethz.ch>
> >> www.ap.ethz.ch
> >> ___
> >> Please keep all replies on the list by using "reply all"
> >> in your mail client.  To manage your subscriptions to this
> >> and other Galaxy lists, please use the interface at:
> >>   https://lists.galaxyproject.org/
> >>
> >> To search Galaxy mailing lists use the unified search at:
> >>   http://galaxyproject.org/search/
>
> --
> ETH Zurich
> *Jochen Bick*
> Animal Physiology
> Institute of Agricultural Sciences
> Postal address: Universitätstrasse 2 / LFW B 58.1
> Office: Tannenstrasse 1 / TAN D 6.2
> 8092 Zurich, Switzerland
>
> Phone +41 44 632 28 25
> jochen.b...@usys.ethz.ch <mailto:jochen.b...@usys.ethz.ch>
> www.ap.ethz.ch
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] collections with more than 25,000 items

[Peter Cock]

If there are any limits, it would be down to the Galaxy Admin's job
settings - something generic with collections.

Personally I've not done this - I tend to concatenate FASTA files
to make large files with multiple sequences instead.

(And then we have the optional task splitting enabled so that Galaxy
breaks up the multiple-sequence FASTA file into chunks which
get shared out on our cluster for better throughput before
concatenating the output back into a single file.)

Peter
On Thu, Aug 30, 2018 at 3:37 PM Jochen Bick  wrote:
>
> Hi,
>
> is there any limit to run BLAST jobs from a collection of single FASTA
> files? I started a job but is does not get executed... its just sending
> for about an hour.
>
> Cheers Jochen
>
> --
> ETH Zurich
> *Jochen Bick*
> Animal Physiology
> Institute of Agricultural Sciences
> Postal address: Universitätstrasse 2 / LFW B 58.1
> Office: Tannenstrasse 1 / TAN D 6.2
> 8092 Zurich, Switzerland
>
> Phone +41 44 632 28 25
> jochen.b...@usys.ethz.ch 
> www.ap.ethz.ch
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] tool datatypes

[Peter Cock]

Hi Matthias,

If you look at the HTML composite datatype, there is a master file (HTML),
and an arbitrary number of arbitrarily named child files like images.

The BLAST database datatype follows this, but we have used a simple
text file as the master file (just the stdout from makeblastdb) in order to
have something for Galaxy to show in the main pane when clicking on
these datasets.

I can't think of any other composite examples which would be more
relevant, hopefully someone else on the list can?

Regards,

Peter

On Fri, Aug 17, 2018 at 2:55 PM, Matthias Bernt  wrote:
> Hi,
>
> thanks for your support. This helps.
>
> I have thought a bit about composite data types. And have additional
> questions.
>
> In my case the additional data is essentially a folder (w subfolders). The
> contents of the folder vary (it depends on the input of the programs that
> generate them). So the question is if I can/should
>
> - add a single folder (with all the hirarchy as a single composite data
> element) .. I have only seen a add_composite_file function.
>
> - add a dynamic number of composite files
>
> - or if it is better to use the extra_files_path mechanism (which seems to
> be the simplest for me)
>
> In any of these cases my next question would be how to create a test using
> such a data as input and output (for output I have seen an example
> somewhere).
>
> Cheers,
> Matthias
>
>
>
>
> On 17.08.2018 11:53, Peter Cock wrote:
>>
>> This is probably a John Chilton question, as the Planemo lead.
>>
>> The way I do it is to "manually" install the datatype into a Galaxy
>> test instance (adding entries to the datatypes_conf.xml and Python
>> files to Galaxy's internal library), and then call ``planemo test``
>> pointing at this test instance. You can see that approach in action
>> here:
>>
>>
>> https://github.com/peterjc/galaxy_blast/blob/579c348ced72d4e6f1ef7fb0cded98e52f454b92/.travis.yml#L106
>>
>> https://github.com/peterjc/galaxy_blast/blob/579c348ced72d4e6f1ef7fb0cded98e52f454b92/.travis.datatypes_conf.xml#L20
>>
>> and:
>>
>>
>> https://github.com/peterjc/galaxy_mira/blob/206259620376b322fc8ed99a6efdd3712f38764b/.travis.yml#L113
>>
>> https://github.com/peterjc/galaxy_mira/blob/206259620376b322fc8ed99a6efdd3712f38764b/.travis.datatypes_conf.xml#L42
>>
>> There may be a more elegant solution nowadays using planemo to do some
>> of the work.
>>
>> Peter
>>
>> On Thu, Aug 16, 2018 at 9:30 PM, Matthias Bernt  wrote:
>>>
>>> Dear Peter,
>>>
>>> you are right. I hope that this will be much less in the end (I'm still
>>> learning about the package).
>>>
>>> The main question still remains, how do I get `planemo test` to include
>>> the
>>> data types defined in the xml file?
>>>
>>> Best,
>>> Matthias
>>>
>>>
>>>
>>> Am 16/08/18 18:32 schrieb Peter Cock :
>>>
>>> Defining 20 different text-based formats does not look ideal (if that
>>> is what you are doing).
>>>
>>> Do you have sample output in the repository? Perhaps at least some of
>>> these can be better defined as tabular instead?
>>>
>>> Or, perhaps you can define one composite datatype for the folder of
>>> output instead?
>>>
>>> Peter
>>>
>>> On Thu, Aug 16, 2018 at 4:18 PM, Matthias Bernt  wrote:
>>>>
>>>> Hi Peter,
>>>>
>>>> I hope that subclassing simple data types will be sufficient.
>>>>
>>>> More details:
>>>>
>>>> I'm currently trying to (auto)wrap the checkm suite
>>>> https://github.com/Ecogenomics/CheckM. Current state here:
>>>>
>>>>
>>>> https://github.com/bernt-matthias/mb-galaxy-tools/tree/master/tools/checkm.
>>>>
>>>> These tools often create an output folder which is then the input to
>>>> other
>>>> tools. So I thought to create a (output) data type for each of the tools
>>>> and
>>>> save the folder using extra_files_path. Then I can pass the folder to
>>>> downstream tools (and can check for proper input). I hope that
>>>> subclassing
>>>> txt or tabular will be sufficient.
>>>>
>>>> Cheers,
>>>> Matthias
>>>>
>>>>
>>>>
>>>> On 16.08.2018 17:02, Peter Cock wrote:
>>>>>
>>>>>
>>>>> More details might help - are you just defining the new datatype as

Re: [galaxy-dev] tool datatypes

[Peter Cock]

This is probably a John Chilton question, as the Planemo lead.

The way I do it is to "manually" install the datatype into a Galaxy
test instance (adding entries to the datatypes_conf.xml and Python
files to Galaxy's internal library), and then call ``planemo test``
pointing at this test instance. You can see that approach in action
here:

https://github.com/peterjc/galaxy_blast/blob/579c348ced72d4e6f1ef7fb0cded98e52f454b92/.travis.yml#L106
https://github.com/peterjc/galaxy_blast/blob/579c348ced72d4e6f1ef7fb0cded98e52f454b92/.travis.datatypes_conf.xml#L20

and:

https://github.com/peterjc/galaxy_mira/blob/206259620376b322fc8ed99a6efdd3712f38764b/.travis.yml#L113
https://github.com/peterjc/galaxy_mira/blob/206259620376b322fc8ed99a6efdd3712f38764b/.travis.datatypes_conf.xml#L42

There may be a more elegant solution nowadays using planemo to do some
of the work.

Peter

On Thu, Aug 16, 2018 at 9:30 PM, Matthias Bernt  wrote:
> Dear Peter,
>
> you are right. I hope that this will be much less in the end (I'm still
> learning about the package).
>
> The main question still remains, how do I get `planemo test` to include the
> data types defined in the xml file?
>
> Best,
> Matthias
>
>
>
> Am 16/08/18 18:32 schrieb Peter Cock :
>
> Defining 20 different text-based formats does not look ideal (if that
> is what you are doing).
>
> Do you have sample output in the repository? Perhaps at least some of
> these can be better defined as tabular instead?
>
> Or, perhaps you can define one composite datatype for the folder of
> output instead?
>
> Peter
>
> On Thu, Aug 16, 2018 at 4:18 PM, Matthias Bernt  wrote:
>> Hi Peter,
>>
>> I hope that subclassing simple data types will be sufficient.
>>
>> More details:
>>
>> I'm currently trying to (auto)wrap the checkm suite
>> https://github.com/Ecogenomics/CheckM. Current state here:
>>
>> https://github.com/bernt-matthias/mb-galaxy-tools/tree/master/tools/checkm.
>>
>> These tools often create an output folder which is then the input to other
>> tools. So I thought to create a (output) data type for each of the tools
>> and
>> save the folder using extra_files_path. Then I can pass the folder to
>> downstream tools (and can check for proper input). I hope that subclassing
>> txt or tabular will be sufficient.
>>
>> Cheers,
>> Matthias
>>
>>
>>
>> On 16.08.2018 17:02, Peter Cock wrote:
>>>
>>> More details might help - are you just defining the new datatype as a
>>> subclass in the XML, or do you need to include Python code (e.g. for a
>>> sniffer)?
>>>
>>> If you want to see some examples of datatypes using Python code which
>>> are available via the Tool Shed, here are two:
>>>
>>>
>>>
>>> https://github.com/peterjc/galaxy_blast/tree/master/data_managers/ncbi_blastdb
>>> (now also in the Galaxy core)
>>>
>>>
>>> https://github.com/peterjc/galaxy_mira/tree/master/datatypes/mira_datatypes
>>>
>>> Peter
>>>
>>> On Thu, Aug 16, 2018 at 3:48 PM, Matthias Bernt  wrote:
>>>>
>>>> Dear list,
>>>>
>>>> just a request for links to documentation: How can I realize tool
>>>> specific
>>>> data types. I'm just developing a set of tools that need their own data
>>>> types, but I don't want to add them to Galaxy's core data types (yet).
>>>>
>>>> I've seen examples of tools that had a datatypes_conf.xml. So I created
>>>> one,
>>>> but it seems that it is ignored.
>>>>
>>>> Cheers,
>>>> Matthias
>>>>
>>>>
>>>> --
>>>>
>>>> ---
>>>> Matthias Bernt
>>>> Bioinformatics Service
>>>> Molekulare Systembiologie (MOLSYB)
>>>> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
>>>> Helmholtz Centre for Environmental Research GmbH - UFZ
>>>> Permoserstraße 15, 04318 Leipzig, Germany
>>>> Phone +49 341 235 482296,
>>>> m.be...@ufz.de, www.ufz.de
>>>>
>>>> Sitz der Gesellschaft/Registered Office: Leipzig
>>>> Registergericht/Registration Office: Amtsgericht Leipzig
>>>> Handelsregister Nr./Trade Register Nr.: B 4703
>>>> Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board:
>>>> MinDirig
>>>> Wilfried Kraus
>>>> Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
>>>> Prof. Dr. Dr. h.c. Georg Teutsch
>>&g

Re: [galaxy-dev] tool datatypes

[Peter Cock]

More details might help - are you just defining the new datatype as a
subclass in the XML, or do you need to include Python code (e.g. for a
sniffer)?

If you want to see some examples of datatypes using Python code which
are available via the Tool Shed, here are two:

https://github.com/peterjc/galaxy_blast/tree/master/data_managers/ncbi_blastdb
(now also in the Galaxy core)
https://github.com/peterjc/galaxy_mira/tree/master/datatypes/mira_datatypes

Peter

On Thu, Aug 16, 2018 at 3:48 PM, Matthias Bernt  wrote:
> Dear list,
>
> just a request for links to documentation: How can I realize tool specific
> data types. I'm just developing a set of tools that need their own data
> types, but I don't want to add them to Galaxy's core data types (yet).
>
> I've seen examples of tools that had a datatypes_conf.xml. So I created one,
> but it seems that it is ignored.
>
> Cheers,
> Matthias
>
>
> --
>
> ---
> Matthias Bernt
> Bioinformatics Service
> Molekulare Systembiologie (MOLSYB)
> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
> Helmholtz Centre for Environmental Research GmbH - UFZ
> Permoserstraße 15, 04318 Leipzig, Germany
> Phone +49 341 235 482296,
> m.be...@ufz.de, www.ufz.de
>
> Sitz der Gesellschaft/Registered Office: Leipzig
> Registergericht/Registration Office: Amtsgericht Leipzig
> Handelsregister Nr./Trade Register Nr.: B 4703
> Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig
> Wilfried Kraus
> Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
> Prof. Dr. Dr. h.c. Georg Teutsch
> Administrative Geschäftsführerin/ Administrative Managing Director:
> Prof. Dr. Heike Graßmann
> ---
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Expose Command Line in Galaxy

[Peter Cock]

You may have found a new Galaxy bug then :(

Could you log an issue here:

https://github.com/galaxyproject/galaxy/issues

(From what you've said, I don't think that this is a problem in the
BLAST+ wrappers themselves)

Peter

On Wed, Jul 11, 2018 at 3:20 PM, Matthias Enders <
m.end...@german-seed-alliance.de> wrote:

> Dear Peter,
>
>
>
> the reloading of the blast *.loc files did not solve the Problem. Actually
> only a complete restart of the container solved the problem for us.
>
>
>
> Mit freundlichen Grüßen
>
>
>
> Matthias Enders
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Expose Command Line in Galaxy

[Peter Cock]

Good. So is everything resolved now?

Peter

On Wed, Jul 11, 2018 at 3:08 PM, Matthias Enders <
m.end...@german-seed-alliance.de> wrote:

> Dear Peter,
>
>
>
> yes we did reload the Blast *.loc files using the admin interface. Our
> Version is 18.05 (based on the Galaxy – Docker image).
>
>
>
> Mit freundlichen Grüßen
>
>
>
> Matthias Enders
>
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Expose Command Line in Galaxy

[Peter Cock]

Is there anything in the Admin's interface to let you reload the BLAST
*.loc files?

If yes, did you try it but it didn't work?

What version of Galaxy do you have?

Peter

On Tue, Jul 10, 2018 at 3:11 PM, Matthias Enders <
m.end...@german-seed-alliance.de> wrote:

> Dear Peter,
>
>
>
> up to our galaxy admin, we make use of the  Data Manager, so this seems
> not to be the issue here?:
>
>
>
>
>
>
>
> Mit freundlichen Grüßen
>
>
>
> Matthias Enders
>
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Expose Command Line in Galaxy

[Peter Cock]

Thanks for getting back to us - this was actually my guess from the
original email - that it was not a blastn error message, but a Galaxy
message before ever calling BLAST.

My guess is you are seeing this issue, or something like it, with
having to restart Galaxy to reload the *.loc files?

https://github.com/galaxyproject/galaxy/issues/3171

Peter

On Tue, Jul 10, 2018 at 12:22 PM, Matthias Enders
 wrote:
> Dear all,
>
> finally a restart of the container solved the issue with the database/blast.
>
> Regarding the problem with the command line:
> For other tools, the command line showed up, but not for blastn, as the 
> wrapper encountered the error before creating the command line (this is my 
> interpretation).
>
> So the error: "An invalid option was selected for database, 
> 'myCoolDatabaseName', please verify." Was produced by the wrapper, which did 
> not find the new database before restarting the galaxy container.
>
> So the real issue here is perhaps more on "How could blast+ be made aware of 
> new integrated databases, without restarting"?
>
> Mit freundlichen Grüßen
>
> Matthias Enders
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Expose Command Line in Galaxy

[Peter Cock]

There is a setting to enable showing the command line string
and full paths on disk via the "i" icon from a history entry, see

https://docs.galaxyproject.org/en/master/admin/options.html#expose-dataset-path

# This option allows users to see the full path of datasets via the
# "View Details" option in the history. This option also exposes the
# command line to non-administrative users. Administrators can always
# see dataset paths.
#expose_dataset_path: false

https://github.com/galaxyproject/galaxy/blob/release_18.05/config/galaxy.yml.sample#L1495

It existed in galaxy.ini too, which has since be replaced by
galaxy.yml

Peter

On Tue, Jul 10, 2018 at 8:19 AM, Matthias Enders
 wrote:
> Dear Björn,
>
> thanks for the very fast reply. Unfortunately, this part is not visible in 
> our instance. (see screenshot). As you can see in the topmost bar, this is 
> done, while being loged in as admin.
>
> Mit freundlichen Grüßen
>
> Matthias Enders
> ---
>
>
> GERMAN SEED ALLIANCE GmbH
> c/o Norddeutsche Pflanzenzucht
> Hans-Georg Lembke KG
> Hohenlieth, 24363 Holtsee
> Tel.: +49 (0)4351/ 736-189
> Fax: + 49 (0)4351/ 736-271
> Mobil: +49 (0)151/ 14247360
>
> Email: m.end...@german-seed-alliance.de
>
> Firmensitz Köln
> Amtsgericht Köln, HRB 73844
>
> -Ursprüngliche Nachricht-
> Von: Björn Grüning 
> Gesendet: Dienstag, 10. Juli 2018 09:15
> An: Matthias Enders ; 
> galaxy-dev@lists.galaxyproject.org
> Betreff: Re: [galaxy-dev] Expose Command Line in Galaxy
> Priorität: Hoch
>
> Hi Matthias,
>
> please have a look at the attached screenshot. Given that you are an admin 
> and click on the info-symbol of the dataset you should see the commandline.
>
> This is with a 18.05 Docker.
> Cheers,
> Bjoern
>
> Am 10.07.2018 um 08:58 schrieb Matthias Enders:
>> Dear Galaxy Community,
>>
>> is there any possibility to expose the command line call, generated
>> within galaxy?
>>
>> We searched the net and found this discussions:
>> https://github.com/galaxyproject/galaxy/issues/3252
>> https://github.com/galaxyproject/galaxy/issues/2954
>>
>> But both are on the exposure to non-admin users. Within our instance
>> we see the command line call nor for admin, neither for normal users!
>>
>> ...
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Conda problem

[Peter Cock]

Good thinking, and that's useful advice if something like this comes up again.

I wonder if there is a bug here somewhere (remnants of the environment
remaining)?

Peter

On Thu, May 24, 2018 at 4:28 PM, David Lee <david@bristol.ac.uk> wrote:
> Hi Peter,
>
>
> Thanks for your reply which nudged me in the right direction and the problem
> seems to be solved. Although I uninstalled samtools and samba from Galaxy
> there were remnants of the packages remaining in
> galaxy/database/dependencies/_conda/envs and
> galaxy/database/dependencies/_conda/pkgs/. After I deleted these directories
> the error messages disappeared and tools are running again.
>
>
> Cheers, David
>
> ____
> From: Peter Cock <p.j.a.c...@googlemail.com>
> Sent: 24 May 2018 14:22:40
> To: David Lee
> Cc: galaxy-dev@lists.galaxyproject.org
> Subject: Re: [galaxy-dev] Conda problem
>
> On Thu, May 24, 2018 at 2:14 PM, David Lee <david@bristol.ac.uk> wrote:
>> Hi,
>>
>>
>> I hope somebody can help me with this. I've been maintaining a Galaxy
>> installation for the past few months that has suddenly developed a problem
>> that I can't undo.
>
> Which version of Galaxy? Can you tell what version of conda it is using?
>
>> I think it occurred after installing samtools and sambamba packages
>> which I have subsequently uninstalled but which has made no
>> difference. Every tool I try to run fails with this error:
>>
>> "Conda dependency failed to build job environment. This is most likely
>> a limitation in conda. You can try to shorten the path to the
>> job_working_directory."
>>
>
> There are some very long path names used in Conda, perhaps
> something has pushed you over an OS limit (like the longest
> allowed command line)?
>
>>
>> Also, in the paster.log file I'm seeing this error that I've never seen
>> before:
>>
>>
>> "CondaError: RuntimeError('EnforceUnusedAdapter called with url
>>
>> https://conda.anaconda.org/conda-forge/linux-64/ca-certificates-2018.4.16-0.tar.bz2\nThis
>> command is using a remote connection in offline mode.\n',)"
>>
>>
>> Any suggestions?
>
> This could be a different issue with similar symptoms, but there
> is a known issue in conda 4.3.17, fixed in conda 4.4.0, which looks
> related: https://github.com/conda/conda/issues/5443
>
> Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Conda problem

[Peter Cock]

On Thu, May 24, 2018 at 2:14 PM, David Lee  wrote:
> Hi,
>
>
> I hope somebody can help me with this. I've been maintaining a Galaxy
> installation for the past few months that has suddenly developed a problem
> that I can't undo.

Which version of Galaxy? Can you tell what version of conda it is using?

> I think it occurred after installing samtools and sambamba packages
> which I have subsequently uninstalled but which has made no
> difference. Every tool I try to run fails with this error:
>
> "Conda dependency failed to build job environment. This is most likely
> a limitation in conda. You can try to shorten the path to the
> job_working_directory."
>

There are some very long path names used in Conda, perhaps
something has pushed you over an OS limit (like the longest
allowed command line)?

>
> Also, in the paster.log file I'm seeing this error that I've never seen
> before:
>
>
> "CondaError: RuntimeError('EnforceUnusedAdapter called with url
> https://conda.anaconda.org/conda-forge/linux-64/ca-certificates-2018.4.16-0.tar.bz2\nThis
> command is using a remote connection in offline mode.\n',)"
>
>
> Any suggestions?

This could be a different issue with similar symptoms, but there
is a known issue in conda 4.3.17, fixed in conda 4.4.0, which looks
related: https://github.com/conda/conda/issues/5443

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Ballgown does not work

[Peter Cock]

On Tue, Apr 10, 2018 at 3:28 PM, Jochen Bick  wrote:
> Hi Peter,
>
>
>>
>> https://github.com/CollardT/Ballgown-Wrapper
>
> I can contact report the issue on github.

I think that would be best.

> It looks like its looking for
> a variable called dataset with is not defined in the input part.
>
> this looks like python code?:
>
> ##
> ## This function returns the name of the sample associated to a given file
> ##
> #def get_sample_name($dataset, $sample_mapping):
> ##If the file with samples mapping was provided
> #if $sample_mapping != None:
> #return $sample_mapping.get($dataset.name, None)
> ##Otherwise with extract the sample name from the filename
> #else:
> #return str($dataset.element_identifier)
> #end if
> #end def
>
> Cheers Jochen

Sort of. It is a templating language called Cheetah which
supports of lot of Python syntax:

http://cheetahtemplate.org/

Most Galaxy wrappers at most use this for if-statements and
for-loops, this might be the first time I've seen complicated
functions being defined in Cheetah code like this.

It would help greatly if the tool author provided a test dataset
(and even better used this to define a tool self-test).

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Ballgown does not work

[Peter Cock]

Hi Jochen,

If you are using release_17.05, that should be using (bio)conda
for dependencies:

https://docs.galaxyproject.org/en/latest/admin/conda_faq.html

"The short answer is that as of 17.01, Galaxy should install Conda
the first time it starts up and be configured to use it by default."

The Galaxy administration screen should let you review if you
have got ballgown itself installed like this, or not.

You didn't confirm which Ballgown you were trying to use in
Galaxy. Is it this version from Theo Collard:

https://toolshed.g2.bx.psu.edu/view/theo.collard/ballgown_wrapper/05977e96375b

If so, this appears to be where he develops it on GitHub, you
could file an issue there:

https://github.com/CollardT/Ballgown-Wrapper

Peter

On Tue, Apr 10, 2018 at 2:37 PM, Jochen Bick <jochen.b...@usys.ethz.ch> wrote:
> Hej Peter,
>
> thanks for the quick reply. My galaxy is a release_17.05
>
> I think the xml is also broken.
>
> Cheers Jochen
>
> On 10.04.2018 15:28, Peter Cock wrote:
>> Are you trying to install this?
>>
>> https://toolshed.g2.bx.psu.edu/view/theo.collard/ballgown_wrapper/05977e96375b
>>
>> If so, it does set out its requirements:
>>
>> 
>> > version="2.2.0">bioconductor-ballgown
>> r-dplyr
>> r-optparse
>> 
>>
>> However, your Galaxy would have to be using BioConda to do this.
>> This wrapper does not include a legacy Galaxy XML file to declare
>> its package dependencies.
>>
>> What version of Galaxy do you have?
>>
>> Peter
>>
>> On Tue, Apr 10, 2018 at 2:16 PM, Jochen Bick <jochen.b...@usys.ethz.ch> 
>> wrote:
>>> Hi,
>>>
>>> I'm trying to install Ballgown but it throws some error message after
>>> running it:
>>>
>>> cannot find 'dataset' while searching for 'param_name.dataset.dataset'
>>>
>>> the script does not have test-data file and it also does not install the
>>> requirements.
>>>
>>> Cheers Jochen
>>>
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>   https://lists.galaxyproject.org/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>   http://galaxyproject.org/search/
>
> --
> ETH Zurich
> *Jochen Bick*
> Animal Physiology
> Institute of Agricultural Sciences
> Postal address: Universitätstrasse 2 / LFW B 58.1
> Office: Tannenstrasse 1 / TAN D 6.2
> 8092 Zurich, Switzerland
>
> Phone +41 44 632 28 25
> jochen.b...@usys.ethz.ch <mailto:jochen.b...@usys.ethz.ch>
> www.ap.ethz.ch
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Ballgown does not work

[Peter Cock]

Are you trying to install this?

https://toolshed.g2.bx.psu.edu/view/theo.collard/ballgown_wrapper/05977e96375b

If so, it does set out its requirements:


bioconductor-ballgown
r-dplyr
r-optparse


However, your Galaxy would have to be using BioConda to do this.
This wrapper does not include a legacy Galaxy XML file to declare
its package dependencies.

What version of Galaxy do you have?

Peter

On Tue, Apr 10, 2018 at 2:16 PM, Jochen Bick  wrote:
> Hi,
>
> I'm trying to install Ballgown but it throws some error message after
> running it:
>
> cannot find 'dataset' while searching for 'param_name.dataset.dataset'
>
> the script does not have test-data file and it also does not install the
> requirements.
>
> Cheers Jochen
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Error during FASTQ preview

[Peter Cock]

This could be a special character unicode issue, but my guess
is something has gone wrong with compression.

Can you identify the file on disk (easy via the history "i" for
information icon if this option is enabled; by default Galaxy
does not tell you the file path)?

Then try the Unix command "head" on the file to inspect it
manually - is it plain text FASTQ?

Peter

On Tue, Mar 13, 2018 at 4:02 PM, Previti
 wrote:
> Dear all,
>
> I've recently had following error occur on our local Galaxy instance.
>
> Every time I want to preview a fastq file it gives this:
>
> Content Encoding Error
>
> The page you are trying to view cannot be shown because it uses an invalid
> or unsupported form of compression.
>
> Please contact the website owners to inform them of this problem.
>
> Any idea how to solve this...? This is the first time I'm seeing itI
> tried changing the data type multiple times (fastq, fastq.gz,
> fastqsanger...etc) but this doesn't get rid of it.
>
> Best regards,
>
> Christopher Previti
>
>
>
> --
> Dr. Christopher Previti
> Genomics and Proteomics Core Facility
> High Throughput Sequencing (W190)
> Bioinformatician
>
> German Cancer Research Center (DKFZ)
> Foundation under Public Law
> Im Neuenheimer Feld 580
> 69120 Heidelberg
> Germany
> Room: B2.102 (INF580/TP3)
> Phone: +49 6221 42-4661
>
> christopher.prev...@dkfz.de
> www.dkfz.de
>
> Management Board: Prof. Dr. Michael Baumann, Prof. Dr. Josef Puchta
> VAT-ID No.: DE143293537
>
> Vertraulichkeitshinweis: Diese Nachricht ist ausschließlich für die Personen
> bestimmt, an die sie adressiert ist.
> Sie kann vertrauliche und/oder nur für den/die Empfänger bestimmte
> Informationen enthalten. Sollten Sie nicht
> der bestimmungsgemäße Empfänger sein, kontaktieren Sie bitte den Absender
> und löschen Sie die Mitteilung.
> Jegliche unbefugte Verwendung der Informationen in dieser Nachricht ist
> untersagt.
>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
On Tue, Mar 13,
2018 at 4:02 PM, Previti mailto:christopher.prev...@dkfz-heidelberg.de;
target="_blank">christopher.prev...@dkfz-heidelberg.de
wrote:




  
Dear all,
I've recently had following error occur on our local Galaxy
  instance.
Every time I want to preview a fastq file it gives this:
Content Encoding Error
  
  The page you are trying to view cannot be shown because it uses an
  invalid or unsupported form of compression.
  
   Please contact the website owners to inform
them of this
  problem.
  
  Any idea how to solve this...? This is the first time I'm seeing
  itI tried changing the data type multiple times (fastq,
  fastq.gz, fastqsanger...etc) but this doesn't get rid of it.

Best regards,
Christopher Previti





-- 
  Dr.
  Christopher Previti
Genomics and Proteomics
Core Facility
High Throughput Sequencing (W190)
Bioinformatician
  
  German Cancer Research Center (DKFZ)
  Foundation under Public Law
  Im Neuenheimer Feld 580
  69120 Heidelberg
  Germany
  Room: B2.102 (INF580/TP3)
  Phone: +49 6221 42-4661

  http://www.dkfz.de/;
target="_blank"
data-saferedirecturl="https://www.google.com/url?hl=enq=http://www.dkfz.de/source=gmailust=1521043877592000usg=AFQjCNEvBdKF2YTYWA7XNiCe7e4pzzkuwg;>christopher.prev...@dkfz.de
  http://www.dkfz.de/; target="_blank"
data-saferedirecturl="https://www.google.com/url?hl=enq=http://www.dkfz.de/source=gmailust=1521043877592000usg=AFQjCNEvBdKF2YTYWA7XNiCe7e4pzzkuwg;>www.dkfz.de

  
  Management
  Board: Prof. Dr. Michael Baumann, Prof. Dr. Josef Puchta
  VAT-ID No.: DE143293537
  Vertraulichkeitshinweis:
  Diese Nachricht ist ausschließlich für die Personen bestimmt,
  an die sie adressiert ist. 
  Sie kann vertrauliche und/oder nur für den/die Empfänger
  bestimmte Informationen enthalten. Sollten Sie nicht 
  der bestimmungsgemäße
  Empfänger sein, kontaktieren Sie bitte den Absender und
  löschen Sie die Mitteilung. 
  Jegliche unbefugte Verwendung der Informationen in dieser
  Nachricht ist untersagt.
  
  

  
  

  

___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 https://lists.galaxyproject.org/;

Re: [galaxy-dev] Ensuring Python 2.7 when resolving conda dependencies for tool

[Peter Cock]

Oh yeah - thanks, I can see them in the default channel which does have
far older versions of Biopython packaged - currently 1.65 is on page 5:

https://anaconda.org/anaconda/biopython/files?sort=basename_order=desc=5

Peter


On Mon, Mar 5, 2018 at 5:00 PM, Marius van den Beek
<m.vandenb...@gmail.com> wrote:
> It's coming from the defaults channel, I didn't check if this is actually
> working,
> it may very well not be compatible with the remaining packages form
> conda-forge.
> AFAIK you can't specify packages to install via the regular conda commands
> (though you can use pip in a conda environemnt to do this manually).
>
> Hope that helps,
> Marius
>
> On 5 March 2018 at 17:51, Peter Cock <p.j.a.c...@googlemail.com> wrote:
>>
>> I stand corrected. Looking closer, there are conda packages
>> for both Python 2 and 3, e.g.
>>
>> https://anaconda.org/conda-forge/biopython/files
>>
>> However, in this case you're asking for an older Biopython
>> which has to date not been packaged in conda-forge or
>> bioconda, so I presume in Marius example it comes from
>> PyPI via pip?
>>
>>
>> Peter
>>
>> On Mon, Mar 5, 2018 at 4:46 PM, Marius van den Beek
>> <m.vandenb...@gmail.com> wrote:
>> > This should actually work properly if you install the dependencies via
>> > the Manage dependencies page in the admin menu or if you install the
>> > tool
>> > via the tool shed.
>> > This translates more or less to the following conda command:
>> > ```
>> > $ conda create -n mulled-v1 python=2.7 biopython=1.65 -c iuc -c
>> > bioconda -c conda-forge -c defaults
>> > Fetching package metadata ...
>> > Solving package specifications: .
>> >
>> > Package plan for installation in environment
>> > /Users/mvandenb/miniconda3/envs/test_resolution:
>> >
>> > The following NEW packages will be INSTALLED:
>> >
>> > biopython:   1.65-np19py27_0
>> > ca-certificates: 2018.1.18-0  conda-forge
>> > certifi: 2018.1.18-py27_0 conda-forge
>> > intel-openmp:2018.0.0-h8158457_8
>> > libgfortran: 3.0.1-h93005f0_2
>> > mkl: 2018.0.1-hfbd8650_4
>> > ncurses: 5.9-10   conda-forge
>> > numpy:   1.9.3-py27hb3dd696_3
>> > openssl: 1.0.2n-0 conda-forge
>> > pip: 9.0.1-py27_1 conda-forge
>> > python:  2.7.14-4 conda-forge
>> > readline:7.0-0conda-forge
>> > setuptools:  38.5.1-py27_0conda-forge
>> > sqlite:  3.20.1-2 conda-forge
>> > tk:  8.6.7-0  conda-forge
>> > wheel:   0.30.0-py27_2conda-forge
>> > zlib:1.2.11-0 conda-forge
>> > ```
>> >
>> > So given a recent conda (latest version should work) and the correct
>> > channel
>> > order
>> > it should work.
>> >
>> > Best,
>> > Marius
>> >
>> > On 5 March 2018 at 17:39, Peter Cock <p.j.a.c...@googlemail.com> wrote:
>> >>
>> >> Tricky, as I understand it the conda python packages are specific to
>> >> the conda version of Python - in this case Python 3 not 2.
>> >>
>> >> It might actually be simpler to fix pal_finder/pal_filter.py to cope
>> >> with Python 3 - is the code online somewhere, I could probably cast an
>> >> eye over it.
>> >>
>> >> Peter
>> >>
>> >> On Mon, Mar 5, 2018 at 4:28 PM, Peter Briggs
>> >> <peter.bri...@manchester.ac.uk> wrote:
>> >> > Hello
>> >> >
>> >> > I'm in the process of updating our local Galaxy tools to use conda
>> >> > dependency resolution, and I've hit a snag with a tool that requires
>> >> > Python
>> >> > 2.7 along with the Python 2.7-compatible version of Biopython 1.65.
>> >> >
>> >> > I'd assumed that if I explicitly used the following in the
>> >> > "requirements" section of the tool XML:
>> >> >
>> >> > python
>> >> > > >> > version="1.65">biopython
>> >> >
>> >> > that the biopython install would respect the specified Python
>> >> > version,
>> >>

Re: [galaxy-dev] Ensuring Python 2.7 when resolving conda dependencies for tool

[Peter Cock]

I stand corrected. Looking closer, there are conda packages
for both Python 2 and 3, e.g.

https://anaconda.org/conda-forge/biopython/files

However, in this case you're asking for an older Biopython
which has to date not been packaged in conda-forge or
bioconda, so I presume in Marius example it comes from
PyPI via pip?


Peter

On Mon, Mar 5, 2018 at 4:46 PM, Marius van den Beek
<m.vandenb...@gmail.com> wrote:
> This should actually work properly if you install the dependencies via
> the Manage dependencies page in the admin menu or if you install the tool
> via the tool shed.
> This translates more or less to the following conda command:
> ```
> $ conda create -n mulled-v1 python=2.7 biopython=1.65 -c iuc -c
> bioconda -c conda-forge -c defaults
> Fetching package metadata ...
> Solving package specifications: .
>
> Package plan for installation in environment
> /Users/mvandenb/miniconda3/envs/test_resolution:
>
> The following NEW packages will be INSTALLED:
>
> biopython:   1.65-np19py27_0
> ca-certificates: 2018.1.18-0  conda-forge
> certifi: 2018.1.18-py27_0 conda-forge
> intel-openmp:2018.0.0-h8158457_8
> libgfortran: 3.0.1-h93005f0_2
> mkl: 2018.0.1-hfbd8650_4
> ncurses: 5.9-10   conda-forge
> numpy:   1.9.3-py27hb3dd696_3
> openssl: 1.0.2n-0 conda-forge
> pip: 9.0.1-py27_1 conda-forge
> python:  2.7.14-4 conda-forge
> readline:7.0-0conda-forge
> setuptools:  38.5.1-py27_0conda-forge
> sqlite:  3.20.1-2 conda-forge
> tk:  8.6.7-0  conda-forge
> wheel:   0.30.0-py27_2conda-forge
> zlib:1.2.11-0 conda-forge
> ```
>
> So given a recent conda (latest version should work) and the correct channel
> order
> it should work.
>
> Best,
> Marius
>
> On 5 March 2018 at 17:39, Peter Cock <p.j.a.c...@googlemail.com> wrote:
>>
>> Tricky, as I understand it the conda python packages are specific to
>> the conda version of Python - in this case Python 3 not 2.
>>
>> It might actually be simpler to fix pal_finder/pal_filter.py to cope
>> with Python 3 - is the code online somewhere, I could probably cast an
>> eye over it.
>>
>> Peter
>>
>> On Mon, Mar 5, 2018 at 4:28 PM, Peter Briggs
>> <peter.bri...@manchester.ac.uk> wrote:
>> > Hello
>> >
>> > I'm in the process of updating our local Galaxy tools to use conda
>> > dependency resolution, and I've hit a snag with a tool that requires Python
>> > 2.7 along with the Python 2.7-compatible version of Biopython 1.65.
>> >
>> > I'd assumed that if I explicitly used the following in the
>> > "requirements" section of the tool XML:
>> >
>> > python
>> > biopython
>> >
>> > that the biopython install would respect the specified Python version,
>> > and that the command execution environment would be based on Python 2.7.
>> >
>> > But in practice it appears I'm getting Python3 as I'm seeing errors:
>> >
>> >   File "/galaxy-tools/tools/pal_finder/pal_filter.py", line
>> > 129
>> > print "\n~~"
>> >^
>> > SyntaxError: Missing parentheses in call to 'print'
>> >
>> > This is happening both when running the tool tests via the planemo
>> > tests, and also when I install the tool from a local toolshed instance.
>> >
>> > Can anyone advise how to coerce the Python dependencies to be
>> > 2.7-compatible?
>> >
>> > Thanks in advance for any help,
>> >
>> > Best wishes
>> >
>> > Peter
>> >
>> > --
>> > Peter Briggs peter.bri...@manchester.ac.uk
>> > Bioinformatics Core Facility University of Manchester
>> > B.1083 Michael Smith Bldg Tel: (0161) 2751482
>> >
>> > ___
>> > Please keep all replies on the list by using "reply all"
>> > in your mail client.  To manage your subscriptions to this
>> > and other Galaxy lists, please use the interface at:
>> >   https://lists.galaxyproject.org/
>> >
>> > To search Galaxy mailing lists use the unified search at:
>> >   http://galaxyproject.org/search/
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Ensuring Python 2.7 when resolving conda dependencies for tool

[Peter Cock]

Tricky, as I understand it the conda python packages are specific to
the conda version of Python - in this case Python 3 not 2.

It might actually be simpler to fix pal_finder/pal_filter.py to cope
with Python 3 - is the code online somewhere, I could probably cast an
eye over it.

Peter

On Mon, Mar 5, 2018 at 4:28 PM, Peter Briggs
 wrote:
> Hello
>
> I'm in the process of updating our local Galaxy tools to use conda dependency 
> resolution, and I've hit a snag with a tool that requires Python 2.7 along 
> with the Python 2.7-compatible version of Biopython 1.65.
>
> I'd assumed that if I explicitly used the following in the "requirements" 
> section of the tool XML:
>
> python
> biopython
>
> that the biopython install would respect the specified Python version, and 
> that the command execution environment would be based on Python 2.7.
>
> But in practice it appears I'm getting Python3 as I'm seeing errors:
>
>   File "/galaxy-tools/tools/pal_finder/pal_filter.py", line 129
> print "\n~~"
>^
> SyntaxError: Missing parentheses in call to 'print'
>
> This is happening both when running the tool tests via the planemo tests, and 
> also when I install the tool from a local toolshed instance.
>
> Can anyone advise how to coerce the Python dependencies to be 2.7-compatible?
>
> Thanks in advance for any help,
>
> Best wishes
>
> Peter
>
> --
> Peter Briggs peter.bri...@manchester.ac.uk
> Bioinformatics Core Facility University of Manchester
> B.1083 Michael Smith Bldg Tel: (0161) 2751482
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] use or convert tool.xml file for database webpage

[Peter Cock]

Hi Jochen,

I think you will find this very difficult without including a large
chunk of the Galaxy code in your project for turning a tool
XML file into an object representation.

Although the main features of the tool XML provide a quite
neutral tool description (list of arguments, output filenames,
and a template to build the command line), there are a lot
of complications and rarely used bits which make doing
this in general very hard.

Galaxy has at least two sections of the code which interpret
the tool file XML to present a user-interface. First, there is
the main interface for running a tool. Second, there is a tool
preview in the Tool Shed - but that does not fully understand
the tool XML and is much more limited.

Could you not provide a limited Galaxy instance instead?
Just your tool and any other key functionality needed to
use with it (importing files perhaps, maybe some plotting
or filtering commands)?

Peter


On Tue, Feb 6, 2018 at 9:34 AM, Jochen Bick  wrote:
> Hi,
>
> I have a question regarding the layout of each tool. I would like to use
> the same layout (same xml file) like inside galaxy to build a standalone
> webpage for a database. Is it possible to convert or use the xml file
> outside galaxy? My problem is that I have no idea how the xml file get
> interpreted inside galaxy to generate the interface of a galaxy tool.
>
> thanks in advance Jochen
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] No legal value defined in a planemo test for param type="select" and options="from_file"

[Peter Cock]

You can still use the workaround on TravisCI by creating a Galaxy
instance and telling planemo to use it (rather than the default of
letting planemo create the test Galaxy instance).

I do this here for example:

https://github.com/peterjc/pico_galaxy/blob/e2fa1c599b6670b418479447fe5a181a97a6c834/.travis.yml#L126

Peter

On Thu, Jan 18, 2018 at 11:21 AM, Olivier Inizan <olivier.ini...@inra.fr> wrote:
> Thank you Peter !
>
> If I have well understood the workaround implies having a local galaxy
> instance with the .loc file.
> But for a planemo test with Travis CI I still will have the error.
> Is it correct ?
>
> Thanks again Peter.
>
> Olivier
>
>
>
> Olivier INIZAN
> olivier.ini...@inra.fr
> Unité Mathématiques & Informatique Appliquées du Génome à l'Environnement
> Plateforme bioinformatique Migale
> Bâtiment 233
> Domaine de Vilvert
> 78352 Jouy-en-Josas
>
> Le 17 janv. 2018 à 17:39, Peter Cock <p.j.a.c...@googlemail.com> a écrit :
>
> It sounds like it could be this issue?:
>
> https://github.com/galaxyproject/planemo/issues/530
>
> If so, the good news is there is a workaround (see issue discussion).
>
> Peter
>
> On Wed, Jan 17, 2018 at 4:24 PM, Olivier Inizan <olivier.ini...@inra.fr>
> wrote:
>
> Dear list,
>
> I am trying to write a test with a param type "select" and options
> "from_file".
> I have the following param section in the wrapper:
>
> 
>  help="Select reference from the list">
>
>
>
>
> 
>
>
> And I  wrote the following test section:
> 
>...
>
> 
>
> The planemo test command failed with the following message:
> RunToolException: Error creating a job for these tool inputs - Parameter
> ref_file requires a value, but has no legal values defined.
>
> Looking at the planemo documentation I saw the entry "test index (.loc)
> data":
> http://planemo.readthedocs.io/en/latest/writing_how_do_i.html#test-index-loc-data
> but it seems to work with .
>
> Have you any test example working with  ?
>
> Thanks for your help.
>
> Olivier
>
>
> Olivier INIZAN
> olivier.ini...@inra.fr
> Unité Mathématiques & Informatique Appliquées du Génome à l'Environnement
> Plateforme bioinformatique Migale
> Bâtiment 233
> Domaine de Vilvert
> 78352 Jouy-en-Josas
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Cluster Install Path Errors

[Peter Cock]

I *think* this is a problem with your copy of Python 2.7 and a
standard library (hashlib) normally present. Do you know how this
Python was installed? If it was compiled from source, then it may have
been missing a few dependencies, and thus you have ended up with
missing a few normally present Python modules.

Peter

On Wed, Jan 17, 2018 at 11:15 PM, John Letaw  wrote:
> Hi all,
>
>
>
> I have been trying to finish up a production cluster Galaxy installation,
> and am having trouble with the below error.  In the past, when seeing
> something along these lines, I usually can adjust environmental variables
> either in startup scripts, or by including a script for Galaxy to source
> before it sends out a job.  I have tried all of these different methods, but
> I can’t seem to get rid of this error message in any tool invocation.  I
> currently have “embed_metadata_in_job” set to False in my job_conf.xml file.
> This removes a “No module named galaxy_ext.metadata.set_metadata” error, but
> this hashlib error remains.  If I could understand a little more about the
> steps that are taken when sending out a job, perhaps I could better diagnose
> this?
>
>
>
> “””
>
> Could not find platform dependent libraries
>
> Consider setting $PYTHONHOME to [:]
>
> Traceback (most recent call last):
>
>   File "~/galaxydev/galaxy/tools/data_source/upload.py", line 14, in
>
> import tempfile
>
>   File "/usr/lib64/python2.7/tempfile.py", line 35, in
>
> from random import Random as _Random
>
>  File "/usr/lib64/python2.7/random.py", line 49, in
>
> import hashlib as _hashlib
>
>   File "/usr/lib64/python2.7/hashlib.py", line 116, in
>
> import _hashlib
>
> ImportError: No module named _hashlib
>
> “””
>
>
>
> Thanks,
>
> John
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] contribute to tools without github

[Peter Cock]

I infer you are talking about this Tool Shed repository:

https://toolshed.g2.bx.psu.edu/view/fabad/sparql_uniprot/

Sadly there is no README file and no copyright statements.
The only clue to the author is one comment is written appears
to be written in Spanish.

Based on https://toolshed.g2.bx.psu.edu/view/fabad/ this is
their first and only Tool Shed release.

I have sent "fabad" a message via the Tool Shed (you need
to be logged into a Tool Shed account to do this) pointing
them at this thread.

Regards,

Peter

On Fri, Oct 20, 2017 at 2:34 PM, Matthias Bernt  wrote:
> I noticed that the sparql_uniprot tool does not run (python3
> incompatibility). Since there is no github page I do not know how to
> contribute to such a tool. I have a similar problem with the lefse tool.
>
> What would be the best way to fix sich tools?
>
> Best,
> Matthias
>
>
> --
>
> ---
> Matthias Bernt
> Bioinformatics Service
> Molekulare Systembiologie (MOLSYB)
> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
> Helmholtz Centre for Environmental Research GmbH - UFZ
> Permoserstraße 15, 04318 Leipzig, Germany
> Phone +49 341 235 482296,
> m.be...@ufz.de, www.ufz.de
>
> Sitz der Gesellschaft/Registered Office: Leipzig
> Registergericht/Registration Office: Amtsgericht Leipzig
> Handelsregister Nr./Trade Register Nr.: B 4703
> Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig
> Wilfried Kraus
> Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
> Prof. Dr. Dr. h.c. Georg Teutsch
> Administrative Geschäftsführerin/ Administrative Managing Director:
> Prof. Dr. Heike Graßmann
> ---
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Problems with BlastDB.loc

[Peter Cock]

What does your config/tool_data_table_conf.xml[.sample] file have?

Can you turn on the debugging as this will will give more information
about the *.loc files. e.g.

https://github.com/galaxyproject/galaxy/pull/3095
https://github.com/galaxyproject/galaxy/pull/3099

Peter

On Wed, Oct 18, 2017 at 8:02 AM, Enders Matthias
 wrote:
> Dear Galaxy Dev List,
>
>
>
> we currently facing a problem with integrating new Blast Databases in our
> Galaxy instance.
>
>
>
> We added a the new DB to the file:
>
> /galaxy-central/tool-data/blastdb.loc
>
>
>
> But the new Database didn’t show up in the respective tools, this situation
> remains also after restarting the galaxy instance and reloading the tool
> config (blasn for example).
>
>
>
> As we included several databases before (which worked fine) we tried to find
> the error an excluded (out-commented) an existing Db, but this database
> stayed in the tools. So the blastdb.loc file seems to be ignored by galaxy.
>
> (This is consistent for multiple users and browser, so we are sure this is
> not a client-sided Browser-Cache issue)
>
>
>
> So my questions are: Is there any kind of internal Cache in Galaxy which
> could explain this behavior? How can we track down / solve the issue?
>
>
>
> Our Galaxy Version is 17.05
>
>
>
> Thanks a lot in advance!
>
>
>
> Mit freundlichen Grüßen
>
>
>
> i.A. Matthias Enders
>
> **
>
>
>
> NPZ Innovation GmbH
>
> Hohenlieth-Hof, D-24363 Holtsee
>
> Tel.+49-4351-736 189
>
> Fax +49-4351-736 271
>
> Mobil:   +49-151-14247360
>
> Mail:  m.end...@npz-innovation.de
>
> Web: www.npz-innovation.de
>
>
>
> Firmensitz Hohenlieth, Holtsee
>
> Amtsgericht Kiel, HRB 15342 Kl
>
> Geschäftsführer: Dr. Gunhild Leckband, Thorsten Pommer
>
>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] upload file format is data - type not recognized in galaxy

[Peter Cock]

My first thought would be to double check the Galaxy startup logs
(which normally tell you about the datatypes as they are setup).
That ought to give some clues.

Also check the XML files where the datatypes are defined are
all still fine - do you have any of these and do they look fine?

datatypes_conf.xml
config/datatypes_conf.xml
config/datatypes_conf.xml.sample

(I know you said the settings and code were not changed,
but it seems worth double checking)

Peter

On Fri, Sep 22, 2017 at 12:31 PM, Hans-Rudolf Hotz  wrote:
> Hi Pavel
>
>
> what happens if you connect your galaxy instance to an empty PostgreSQL
> database, let galaxy build all the tables and then try to upload files.
>
>
> Regards, Hans-Rudolf
>
>
>
>
> On 09/22/2017 10:51 AM, Pavel Fibich wrote:
>>
>> Hello,
>>
>> do you have any idea why my galaxy 16.07 stop recognizing uploaded files
>> and format of uploaded files is set to the format 'data'?
>>
>> After using older version of database backup, galaxy stop recognizing
>> format of uploaded files (e.g. fasta, fastqc) and everything is set to
>> format 'data'. Uploaded files are OK, I have tested them on other public
>> galaxy servers, where they were recognized well. Before changing
>> database (after the technical problems with the server), recognizing of
>> the files were OK. Even in debugging level I do not see any errors or
>> problems in the log (I have compared logs before changing database with
>> new ones, and there are no differences). From the investigating of
>> upload.py, I guess something happen with galaxy.datatypes.registry
>> (settings and code of galaxy were not changed). Do you have any ideas? I
>> cannot switch the database back and changing file format in Edit
>> attributes is annoying.
>>
>> Best, Pavel
>>
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>http://galaxyproject.org/search/
>>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Displaying tool select list according to user

[Peter Cock]

On Thu, Sep 21, 2017 at 12:50 PM, SAPET, Frederic <
frederic.sa...@biogemma.com> wrote:

> Hi Peter,
>
> Good idea, I will keep that in mind for other needs.
> I'm not sure that it will be suitable for what I want to do. Users will
> have to first import database to their history (yet another step as they
> ask me for less, even the upload process is sometimes one step they would
> like to avoid !)


>
But it could be a way of achieving what I want, if Galaxy could get files
> (or .loc files) from a data library, within the tool form itself !


>
Maybe I will create an issue on the Galaxy git in order to discuss about
> that. I will be glad to help if something can be done.
>

Good idea! I wonder if the Galaxy UI team can think of a good way to
integrate this, which would make Shared Data library files available to any
tool without the import to current history step? Can you reply with a link
to the new GitHub issue for this?


> Maybe some explanations about the fact that my users ask me for "less
> steps".
> I really want Galaxy to be the main tool of our analysis platform.
> Right now I still maintain a plain old perl/php/CGI web interface. With a
> blast tool (among a lot of others tools). This proprietary Blast interface
> is really like you will find on NCBI or EBI sites. And you can quickly
> paste your sequences, pick up a database, a Blast algorythm and then
> launch. Few seconds later I have a 'htmlized' Blast report, with a nice
> picture.
> This use case (is this sequence a part of this database ?) is still faster
> on this plain old interface. And I afraid that I will have to keep it until
> they found Galaxy better than it...
>

We used to run wwwBLAST locally for similar reasons. If I was trying to
setup up something similar now, I'd look at http://www.sequenceserver.com/
for this - but making the Galaxy alternative easier to use would be neater.


> Maybe it could be another (or two) idea(s)  : the user can drag and drop
> his file within the tool form, Galaxy will handle the creation of this file
> within the history and then launch the tool. Or user can copy/paste a
> sequence within a text-area, and Galaxy will handle the creation of a file
> from these data within the history and then launch the tool.
> This skips upload process.
>

The Galaxy upload does now support drag and drop, so again in principle the
Galaxy team might be able to extend the tool interface as well. The tool
input's expected file formats could be used to guide the data-type sniffer
and/or short list the choices shown to the use.

As well or instead, I could imagine having an upload icon associated with
each tool input (again, creating a history item and selecting it for you)?
This could take advantage of the fact that each tool's input parameter
specifies the expected input file formats, so that might be mapped to known
file extensions for a file picker?


>
> thank you again for sharing your ideas, it helps!
>
> Fred
>

These are great usability ideas you are putting forward here - as a Galaxy
Tool Author they sound positive, but I cannot comment on how complicated it
would be for the Galaxy UI team. Or indeed if the UI would actually be
practical - or just overly complicate things which is always a risk.

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Displaying tool select list according to user

[Peter Cock]

Have you tried creating the BLAST databases as files in your history
(by running makeblastdb within Galaxy), and then importing them into
a shared data library (which can then be configured with role based
permissions)?

Peter


On Wed, Sep 20, 2017 at 9:39 AM, SAPET, Frederic
<frederic.sa...@biogemma.com> wrote:
> Hello Laure
>
>
>
> this idea cannot be applied in my use case.
>
> indeed I don’t want that users can choose which data to display. I want to
> hide some data based on their name/group/address (anything that can be
> useful to clasify my users).
>
>
>
> Thank you for sharing your ideas :)
>
>
>
> Fred
>
> De : Laure QUINTRIC [mailto:laure.quint...@ifremer.fr]
> Envoyé : mercredi 13 septembre 2017 13:54
> À : Peter Cock <p.j.a.c...@googlemail.com>; SAPET, Frederic
> <frederic.sa...@biogemma.com>
>
>
> Cc : Galaxy-dev <galaxy-dev@lists.galaxyproject.org>
> Objet : Re: [galaxy-dev] Displaying tool select list according to user
>
>
>
> A workaround would be to do this :
>
> - first, create a tool which will output a tabular loc file with all the
> custom banks for the connected user.
> - then add in the ncbi_blast wrapper a select list with the attribute
> from_dataset to load the loc file generated by the previous tool  : see the
> example here :
> https://docs.galaxyproject.org/en/latest/dev/schema.html#from-dataset
>
> I have something like this in the ncbi_blast tool:
> 
>  multiple="true" display="checkboxes" label="Select our custom db">
> 
> 
> 
> 
> 
>
> and I set in param userdb_loc_file the loc file which is in my history and
> looks like this :
> NCBI_ntNCBI_nt
> 2017-05-19/path/to/biobanks/n/NCBI_nt/current/NCBI_nt/NCBI_nt
>
> Laure
>
> Le 12/09/2017 à 17:25, Peter Cock a écrit :
>
> Thanks Fred,
>
>
>
> That's the previous discussion I was trying to find -
>
> sadly no clear solution but some ideas to explore.
>
>
>
> It does seem filtering example.loc by user or role
>
> is not a niche request. I wonder if this could be
>
> linked to the Galaxy data library permissions
>
> structure as a way to manage the access rights?
>
>
>
> Peter
>
>
>
> On Tue, Sep 12, 2017 at 4:20 PM, SAPET, Frederic
> <frederic.sa...@biogemma.com> wrote:
>
> Hi Laure,
>
>
>
> There are others users (including me! ) that need this kind of function :
>
> http://dev.list.galaxyproject.org/Blast-db-permission-td4670440.html
>
>
>
> @Galaxy team:
>
> Do we need to add an issue in GitHub ?
>
> Do you think that it could be enabled one day ?
>
>
>
> I think this something that could be nice to develop !
>
> I would be glad to help.
>
>
>
> Thank a lot
>
>
>
> Fred
>
>
>
> De : galaxy-dev [mailto:galaxy-dev-boun...@lists.galaxyproject.org] De la
> part de Peter Cock
> Envoyé : mardi 12 septembre 2017 14:56
> À : Laure QUINTRIC <laure.quint...@ifremer.fr>
> Cc : Galaxy-dev <galaxy-dev@lists.galaxyproject.org>
> Objet : Re: [galaxy-dev] Displaying tool select list according to user
>
>
>
> Hi Laure,
>
>
>
> I don't think there is any easy to use mechanism for this in Galaxy.
>
> Unfortunately Galaxy loads all the *.loc files at startup and treats
>
> them as global values available to all users equally.
>
>
>
> I can't find the email thread, but I recall someone previously trying
>
> to do something with a single large example.loc file for all users
>
> but with an extra column for filtering by account name. I forget if
>
> they got these changes to work or not.
>
>
>
> Assuming you only have a few separate versions of the example.loc
>
> file, you could create copies of the BLAST wrapper XML file which
>
> use the specific *.loc file - and then restrict the tool by user?
>
>
>
> In either case at a minimum you would have to maintain local
>
> changes to the BLAST wrappers (and potentially changes to
>
> Galaxy itself), which is not a good situation.
>
>
>
> Peter
>
>
>
> On Tue, Sep 12, 2017 at 1:41 PM, Laure QUINTRIC <laure.quint...@ifremer.fr>
> wrote:
>
> Hello Galaxy users,
>
> I try to figure out how I can display a select list into a galaxy tool
> according to the connected user.
>
> I have several .loc files containing the paths for custom blast databases I
> have created for several users (user1.loc, user2.loc, and so on). I would
> like the blast tool to display the specific databases for the connected user
> (user1 or

Re: [galaxy-dev] Displaying tool select list according to user

[Peter Cock]

Thanks Fred,

That's the previous discussion I was trying to find -
sadly no clear solution but some ideas to explore.

It does seem filtering example.loc by user or role
is not a niche request. I wonder if this could be
linked to the Galaxy data library permissions
structure as a way to manage the access rights?

Peter

On Tue, Sep 12, 2017 at 4:20 PM, SAPET, Frederic <
frederic.sa...@biogemma.com> wrote:

> Hi Laure,
>
>
>
> There are others users (including me! ) that need this kind of function :
>
> http://dev.list.galaxyproject.org/Blast-db-permission-td4670440.html
>
>
>
> @Galaxy team:
>
> Do we need to add an issue in GitHub ?
>
> Do you think that it could be enabled one day ?
>
>
>
> I think this something that could be nice to develop !
>
> I would be glad to help.
>
>
>
> Thank a lot
>
>
>
> Fred
>
>
>
> *De :* galaxy-dev [mailto:galaxy-dev-boun...@lists.galaxyproject.org] *De
> la part de* Peter Cock
> *Envoyé :* mardi 12 septembre 2017 14:56
> *À :* Laure QUINTRIC <laure.quint...@ifremer.fr>
> *Cc :* Galaxy-dev <galaxy-dev@lists.galaxyproject.org>
> *Objet :* Re: [galaxy-dev] Displaying tool select list according to user
>
>
>
> Hi Laure,
>
>
>
> I don't think there is any easy to use mechanism for this in Galaxy.
>
> Unfortunately Galaxy loads all the *.loc files at startup and treats
>
> them as global values available to all users equally.
>
>
>
> I can't find the email thread, but I recall someone previously trying
>
> to do something with a single large example.loc file for all users
>
> but with an extra column for filtering by account name. I forget if
>
> they got these changes to work or not.
>
>
>
> Assuming you only have a few separate versions of the example.loc
>
> file, you could create copies of the BLAST wrapper XML file which
>
> use the specific *.loc file - and then restrict the tool by user?
>
>
>
> In either case at a minimum you would have to maintain local
>
> changes to the BLAST wrappers (and potentially changes to
>
> Galaxy itself), which is not a good situation.
>
>
>
> Peter
>
>
>
> On Tue, Sep 12, 2017 at 1:41 PM, Laure QUINTRIC <laure.quint...@ifremer.fr>
> wrote:
>
> Hello Galaxy users,
>
> I try to figure out how I can display a select list into a galaxy tool
> according to the connected user.
>
> I have several .loc files containing the paths for custom blast databases
> I have created for several users (user1.loc, user2.loc, and so on). I would
> like the blast tool to display the specific databases for the connected
> user (user1 or user2).
>
> Any idea on how to proceed ?
> Regards,
> Laure
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/
>
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Displaying tool select list according to user

[Peter Cock]

Hi Laure,

I don't think there is any easy to use mechanism for this in Galaxy.
Unfortunately Galaxy loads all the *.loc files at startup and treats
them as global values available to all users equally.

I can't find the email thread, but I recall someone previously trying
to do something with a single large example.loc file for all users
but with an extra column for filtering by account name. I forget if
they got these changes to work or not.

Assuming you only have a few separate versions of the example.loc
file, you could create copies of the BLAST wrapper XML file which
use the specific *.loc file - and then restrict the tool by user?

In either case at a minimum you would have to maintain local
changes to the BLAST wrappers (and potentially changes to
Galaxy itself), which is not a good situation.

Peter

On Tue, Sep 12, 2017 at 1:41 PM, Laure QUINTRIC 
wrote:

> Hello Galaxy users,
>
> I try to figure out how I can display a select list into a galaxy tool
> according to the connected user.
>
> I have several .loc files containing the paths for custom blast databases
> I have created for several users (user1.loc, user2.loc, and so on). I would
> like the blast tool to display the specific databases for the connected
> user (user1 or user2).
>
> Any idea on how to proceed ?
> Regards,
> Laure
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Samtools problem in Docker

[Peter Cock]

Galaxy itself uses samtools for many operations, which is different
from when samtools is called as a dependency of a tool run by the
Galaxy user.

However, the Docker image ought to take care of that - can you
give more details of which Docker image you are using (URL,
version, etc)?

Thanks,

Peter


On Mon, Aug 14, 2017 at 5:08 PM, Deepak Tanwar  wrote:
> Hello everyone,
>
> I am facing problem in uploading bam files on Galaxy. I am using Docker
> image of Galaxy.
>
> I have installed samtools version 0.1.19 and added it to the path
> /tool_deps/samtools/0.1.19/iuc/package_samtools_0_1_19/c9bd782f5342/bin/samtools
>
> Error: Exception: Attempting to use functionality requiring samtools, but it
> cannot be located on Galaxy's PATH.
>
> Thank you,
> Deepak
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Data manager for ncbi_blast_plus in the main tool shed?

[Peter Cock]

Hi David,

Daniel Blankenberg wrote one (CC'd):

https://github.com/peterjc/galaxy_blast/tree/master/data_managers/ncbi_blastdb

http://testtoolshed.g2.bx.psu.edu/view/blankenberg/data_manager_example_blastdb_ncbi_update_blastdb

I've never used it in production and don't have enough understanding
of it and the Data Manager framework to understand what if anything
needs working on.

Locally we have mirrors of the NCBI blast databases (updated with cron),
local databases for organisms of interest - these are all listed in the
*.loc
files by hand.

Separately, our users can and do make their own databases within
Galaxy using makeblastdb.

Regards,

Peter

On Thu, Aug 10, 2017 at 10:32 PM, David Kovalic  wrote:

> Hi,
>
> Does anyone know if a data manager exists in the tool shed for creating
> and integrating NCBI BLAST+ indices within galaxy?
>
> I can't seem to find it in the tools shed or searching the internet  Surly
> such a tool must exist.
>
> Any info would be greatly appreciated.
>
> Thanks,
>
> David
>
>
>
> David Kovalic Ph.D.   President, Analome
> phone: +314.496.8808
> web: www.analome.com
> email: kova...@analome.com
> 
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Image in tool XML "help" section only renders when installed from toolshed?

[Peter Cock]

I'm puzzled now too, cross reference

https://github.com/galaxy-iuc/standards/pull/46

and the original:

https://github.com/galaxy-iuc/standards/issues/12

Peter


On Wed, Aug 9, 2017 at 3:01 PM, Peter Briggs
<peter.bri...@manchester.ac.uk> wrote:
> Hello Peter
>
> Thanks for the fast response. From reading around elsewhere I'd understood
> that using $PATH_TO_IMAGES was now deprecated? I think I tried it anyway and
> it didn't seem to help. But I might have missed something, so I'd be
> interested to know if it works for you.
>
> Best wishes
>
> Peter
>
>
> On 09/08/17 14:57, Peter Cock wrote:
>>
>> It looks like something important is not showing in the best practise
>> website, special variable $PATH_TO_IMAGES as per the linked example:
>>
>> .. image:: $PATH_TO_IMAGES/slop-glyph.png
>>
>> Let me look into this,
>>
>> Peter
>>
>> On Wed, Aug 9, 2017 at 2:52 PM, Peter Briggs
>> <peter.bri...@manchester.ac.uk> wrote:
>>>
>>> Dear devs
>>>
>>> I've been updating the  section for a locally developed tool and
>>> want
>>> to include some images, but I've had some issues trying to see them when
>>> testing locally.
>>>
>>> I've followed the instructions outlined here:
>>>
>>>
>>> https://github.com/galaxy-iuc/standards/blob/master/docs/best_practices/tool_xml.rst#including-images
>>>
>>> (i.e. put the images files into a 'static/images/' subdir of the tool
>>> directory and reference them using ".. image:: picture.png")
>>>
>>> However when I load the tool into a Galaxy using "planemo serve ..." the
>>> images fail to render: I just get a line of text with the image filename.
>>> The same thing happens when I add the tool manually on our local server
>>> (i.e. by adding a  reference to the tool XML in the
>>> appropriate conf.xml file).
>>>
>>> If I upload the same tool files to a toolshed and then install from
>>> there, I
>>> do see the images rendered correctly.
>>>
>>> I'm a bit baffled as to why there should be a difference. Is there an
>>> option
>>> or setting I've missed to turn on the image rendering for
>>> non-shed-installed
>>> tools? Or have I made some other mistake?
>>>
>>> Thanks for your help,
>>>
>>> Best wishes
>>>
>>> Peter
>>>
>>> --
>>> Peter Briggs peter.bri...@manchester.ac.uk
>>> Bioinformatics Core Facility University of Manchester
>>> B.1083 Michael Smith Bldg Tel: (0161) 2751482
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>   https://lists.galaxyproject.org/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>   http://galaxyproject.org/search/
>
>
> --
> Peter Briggs peter.bri...@manchester.ac.uk
> Bioinformatics Core Facility University of Manchester
> B.1083 Michael Smith Bldg Tel: (0161) 2751482
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Image in tool XML "help" section only renders when installed from toolshed?

[Peter Cock]

It looks like something important is not showing in the best practise
website, special variable $PATH_TO_IMAGES as per the linked example:

.. image:: $PATH_TO_IMAGES/slop-glyph.png

Let me look into this,

Peter

On Wed, Aug 9, 2017 at 2:52 PM, Peter Briggs
 wrote:
> Dear devs
>
> I've been updating the  section for a locally developed tool and want
> to include some images, but I've had some issues trying to see them when
> testing locally.
>
> I've followed the instructions outlined here:
>
> https://github.com/galaxy-iuc/standards/blob/master/docs/best_practices/tool_xml.rst#including-images
>
> (i.e. put the images files into a 'static/images/' subdir of the tool
> directory and reference them using ".. image:: picture.png")
>
> However when I load the tool into a Galaxy using "planemo serve ..." the
> images fail to render: I just get a line of text with the image filename.
> The same thing happens when I add the tool manually on our local server
> (i.e. by adding a  reference to the tool XML in the
> appropriate conf.xml file).
>
> If I upload the same tool files to a toolshed and then install from there, I
> do see the images rendered correctly.
>
> I'm a bit baffled as to why there should be a difference. Is there an option
> or setting I've missed to turn on the image rendering for non-shed-installed
> tools? Or have I made some other mistake?
>
> Thanks for your help,
>
> Best wishes
>
> Peter
>
> --
> Peter Briggs peter.bri...@manchester.ac.uk
> Bioinformatics Core Facility University of Manchester
> B.1083 Michael Smith Bldg Tel: (0161) 2751482
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Venn diagram example data

[Peter Cock]

How does the plugin expect cases where the sets have a different
number of entries to be presented? Simply blank cells in the tab
separated table?

Peter

On Thu, Jul 27, 2017 at 4:43 PM, Aysam Guerler <aysam.guer...@gmail.com> wrote:
> Sorry for the late response. So the venn diagram visualizes the overlap
> between sets. Each set is represented by entries in a column e.g.
>
> #SET1 SET2 SET3
> A GA
> B HB
> C AG
> D BH
> E CX
> F DZ
>
> I hope this helps.
>
> Thanks
> Sam
>
> On Thu, Jul 27, 2017 at 11:33 AM, Peter Cock <p.j.a.c...@googlemail.com>
> wrote:
>>
>> Hi Steven,
>>
>> Just in case, you're not talking about my older simpler Galaxy Tool
>> for drawing simple Venn Diagrams, are you?
>>
>> https://toolshed.g2.bx.psu.edu/view/peterjc/venn_list
>> https://github.com/peterjc/pico_galaxy/tree/master/tools/venn_list
>>
>> This reminds me that I ought to update that tool to work with
>> dependencies via conda...
>>
>> Peter
>>
>> On Mon, Jul 24, 2017 at 2:16 PM, Steven Shen <ishenwei...@gmail.com>
>> wrote:
>> > Dear Galaxy list,
>> >
>> > I am interested in Galaxy Charts visualization.But venn diagram confuses
>> > me
>> > when I upload my tabular data, it seems venn diagram need  one column
>> > observations at least, actually I don't clear what should be contained
>> > in
>> > the one column observations.
>> >
>> > So is there any venn diagram example data or usage for this charts
>> > plugin ?
>> > Sad to say I am not good at JavaScript :)
>> >
>> > Thank you very much.
>> >
>> > Regards,
>> > Steven
>> >
>> > ___
>> > Please keep all replies on the list by using "reply all"
>> > in your mail client.  To manage your subscriptions to this
>> > and other Galaxy lists, please use the interface at:
>> >   https://lists.galaxyproject.org/
>> >
>> > To search Galaxy mailing lists use the unified search at:
>> >   http://galaxyproject.org/search/
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Venn diagram example data

[Peter Cock]

Hi Steven,

Just in case, you're not talking about my older simpler Galaxy Tool
for drawing simple Venn Diagrams, are you?

https://toolshed.g2.bx.psu.edu/view/peterjc/venn_list
https://github.com/peterjc/pico_galaxy/tree/master/tools/venn_list

This reminds me that I ought to update that tool to work with
dependencies via conda...

Peter

On Mon, Jul 24, 2017 at 2:16 PM, Steven Shen  wrote:
> Dear Galaxy list,
>
> I am interested in Galaxy Charts visualization.But venn diagram confuses me
> when I upload my tabular data, it seems venn diagram need  one column
> observations at least, actually I don't clear what should be contained in
> the one column observations.
>
> So is there any venn diagram example data or usage for this charts plugin ?
> Sad to say I am not good at JavaScript :)
>
> Thank you very much.
>
> Regards,
> Steven
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] tool and planemo problems

[Peter Cock]

And mere minutes later that change has been accepted,
and will be part of the next release of Planemo.

Thanks for the constructive feedback Matthias!

Peter

On Wed, Jul 12, 2017 at 4:24 PM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
> Good idea, pull request submitted:
>
> https://github.com/galaxyproject/planemo/pull/709
>
> Peter
>
> On Wed, Jul 12, 2017 at 4:03 PM, Matthias Bernt <m.be...@ufz.de> wrote:
>> Perfect. Thanks a lot. Maybe one could add a note to planemo --help that
>> command specific help can be obtained with planemo command --help.
>>
>> Best,
>> Matthias
>>
>>
>> On 12.07.2017 16:57, Peter Cock wrote:
>>>
>>> Try:
>>>
>>> planemo lint --fail_level error sleepwaste.xml
>>>
>>> The default is to fail if there are warnings.
>>>
>>> Peter
>>>
>>>
>>> On Wed, Jul 12, 2017 at 3:55 PM, Matthias Bernt <m.be...@ufz.de> wrote:
>>>>
>>>> Hi Peter,
>>>>
>>>> I just call planemo l sleepwaste.xml
>>>>
>>>> Best,
>>>> Matthias
>>>>
>>>>
>>>> On 12.07.2017 16:54, Peter Cock wrote:
>>>>>
>>>>>
>>>>> How are you running planemo lint - in particular is it set
>>>>> to fail if there are any warnings present (like your missing
>>>>> citation warning)?
>>>>>
>>>>> Peter
>>>>>
>>>>> On Wed, Jul 12, 2017 at 3:52 PM, Matthias Bernt <m.be...@ufz.de> wrote:
>>>>>>
>>>>>>
>>>>>> Dear list,
>>>>>>
>>>>>> In order to test the job resubmission feature I wrote a little tool for
>>>>>> galaxy that just wastes a specified (minimum) time and memory (using
>>>>>> different programming languages) for testing the new DRMAA job runner
>>>>>> that I
>>>>>> wrote.
>>>>>>
>>>>>> When linting it with planemo seemingly all tests are successful, but in
>>>>>> the
>>>>>> end I still get a fail -- verbose mode is also of no help here.
>>>>>>
>>>>>> Applying linter tests... CHECK
>>>>>> .. CHECK: 7 test(s) found.
>>>>>> Applying linter output... CHECK
>>>>>> .. INFO: 1 outputs found.
>>>>>> Applying linter inputs... CHECK
>>>>>> .. INFO: Found 3 input parameters.
>>>>>> Applying linter help... CHECK
>>>>>> .. CHECK: Tool contains help section.
>>>>>> .. CHECK: Help contains valid reStructuredText.
>>>>>> Applying linter general... CHECK
>>>>>> .. CHECK: Tool defines a version [0.1.0].
>>>>>> .. CHECK: Tool defines a name [Waste time and memory].
>>>>>> .. CHECK: Tool defines an id [sleepwaste].
>>>>>> .. CHECK: Tool targets 16.01 Galaxy profile.
>>>>>> Applying linter command... CHECK
>>>>>> .. INFO: Tool contains a command.
>>>>>> Applying linter citations... WARNING
>>>>>> .. WARNING: No citations found, consider adding citations to your tool.
>>>>>> Applying linter tool_xsd... CHECK
>>>>>> .. INFO: File validates against XML schema.
>>>>>> Failed linting
>>>>>>
>>>>>> The test of the tool are all successful. Any ideas what is wrong?
>>>>>>
>>>>>> Once the problems are solved, would such a tool also be useful in the
>>>>>> toolshed?
>>>>>>
>>>>>> Cheers,
>>>>>> Matthias
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> ---
>>>>>> Matthias Bernt
>>>>>> Bioinformatics Service
>>>>>> Molekulare Systembiologie (MOLSYB)
>>>>>> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
>>>>>> Helmholtz Centre for Environmental Research GmbH - UFZ
>>>>>> Permoserstraße 15, 04318 Leipzig, Germany
>>>>>> Phone +49 341 235 482296,
>>>>>> m.be...@ufz.de, www.ufz.de
>>>>>>
>>>>>> Sitz der Gesellschaft/Registered Office: Leipzig
>>>>>> Registergericht/Registration Office: Amtsgericht Leipzig
>>>>>>

Re: [galaxy-dev] tool and planemo problems

[Peter Cock]

Good idea, pull request submitted:

https://github.com/galaxyproject/planemo/pull/709

Peter

On Wed, Jul 12, 2017 at 4:03 PM, Matthias Bernt <m.be...@ufz.de> wrote:
> Perfect. Thanks a lot. Maybe one could add a note to planemo --help that
> command specific help can be obtained with planemo command --help.
>
> Best,
> Matthias
>
>
> On 12.07.2017 16:57, Peter Cock wrote:
>>
>> Try:
>>
>> planemo lint --fail_level error sleepwaste.xml
>>
>> The default is to fail if there are warnings.
>>
>> Peter
>>
>>
>> On Wed, Jul 12, 2017 at 3:55 PM, Matthias Bernt <m.be...@ufz.de> wrote:
>>>
>>> Hi Peter,
>>>
>>> I just call planemo l sleepwaste.xml
>>>
>>> Best,
>>> Matthias
>>>
>>>
>>> On 12.07.2017 16:54, Peter Cock wrote:
>>>>
>>>>
>>>> How are you running planemo lint - in particular is it set
>>>> to fail if there are any warnings present (like your missing
>>>> citation warning)?
>>>>
>>>> Peter
>>>>
>>>> On Wed, Jul 12, 2017 at 3:52 PM, Matthias Bernt <m.be...@ufz.de> wrote:
>>>>>
>>>>>
>>>>> Dear list,
>>>>>
>>>>> In order to test the job resubmission feature I wrote a little tool for
>>>>> galaxy that just wastes a specified (minimum) time and memory (using
>>>>> different programming languages) for testing the new DRMAA job runner
>>>>> that I
>>>>> wrote.
>>>>>
>>>>> When linting it with planemo seemingly all tests are successful, but in
>>>>> the
>>>>> end I still get a fail -- verbose mode is also of no help here.
>>>>>
>>>>> Applying linter tests... CHECK
>>>>> .. CHECK: 7 test(s) found.
>>>>> Applying linter output... CHECK
>>>>> .. INFO: 1 outputs found.
>>>>> Applying linter inputs... CHECK
>>>>> .. INFO: Found 3 input parameters.
>>>>> Applying linter help... CHECK
>>>>> .. CHECK: Tool contains help section.
>>>>> .. CHECK: Help contains valid reStructuredText.
>>>>> Applying linter general... CHECK
>>>>> .. CHECK: Tool defines a version [0.1.0].
>>>>> .. CHECK: Tool defines a name [Waste time and memory].
>>>>> .. CHECK: Tool defines an id [sleepwaste].
>>>>> .. CHECK: Tool targets 16.01 Galaxy profile.
>>>>> Applying linter command... CHECK
>>>>> .. INFO: Tool contains a command.
>>>>> Applying linter citations... WARNING
>>>>> .. WARNING: No citations found, consider adding citations to your tool.
>>>>> Applying linter tool_xsd... CHECK
>>>>> .. INFO: File validates against XML schema.
>>>>> Failed linting
>>>>>
>>>>> The test of the tool are all successful. Any ideas what is wrong?
>>>>>
>>>>> Once the problems are solved, would such a tool also be useful in the
>>>>> toolshed?
>>>>>
>>>>> Cheers,
>>>>> Matthias
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> ---
>>>>> Matthias Bernt
>>>>> Bioinformatics Service
>>>>> Molekulare Systembiologie (MOLSYB)
>>>>> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
>>>>> Helmholtz Centre for Environmental Research GmbH - UFZ
>>>>> Permoserstraße 15, 04318 Leipzig, Germany
>>>>> Phone +49 341 235 482296,
>>>>> m.be...@ufz.de, www.ufz.de
>>>>>
>>>>> Sitz der Gesellschaft/Registered Office: Leipzig
>>>>> Registergericht/Registration Office: Amtsgericht Leipzig
>>>>> Handelsregister Nr./Trade Register Nr.: B 4703
>>>>> Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board:
>>>>> MinDirig
>>>>> Wilfried Kraus
>>>>> Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
>>>>> Prof. Dr. Dr. h.c. Georg Teutsch
>>>>> Administrative Geschäftsführerin/ Administrative Managing Director:
>>>>> Prof. Dr. Heike Graßmann
>>>>> ---
>>>>>
>>>>> ___
>>>>> Please k

Re: [galaxy-dev] tool and planemo problems

[Peter Cock]

Try:

planemo lint --fail_level error sleepwaste.xml

The default is to fail if there are warnings.

Peter


On Wed, Jul 12, 2017 at 3:55 PM, Matthias Bernt <m.be...@ufz.de> wrote:
> Hi Peter,
>
> I just call planemo l sleepwaste.xml
>
> Best,
> Matthias
>
>
> On 12.07.2017 16:54, Peter Cock wrote:
>>
>> How are you running planemo lint - in particular is it set
>> to fail if there are any warnings present (like your missing
>> citation warning)?
>>
>> Peter
>>
>> On Wed, Jul 12, 2017 at 3:52 PM, Matthias Bernt <m.be...@ufz.de> wrote:
>>>
>>> Dear list,
>>>
>>> In order to test the job resubmission feature I wrote a little tool for
>>> galaxy that just wastes a specified (minimum) time and memory (using
>>> different programming languages) for testing the new DRMAA job runner
>>> that I
>>> wrote.
>>>
>>> When linting it with planemo seemingly all tests are successful, but in
>>> the
>>> end I still get a fail -- verbose mode is also of no help here.
>>>
>>> Applying linter tests... CHECK
>>> .. CHECK: 7 test(s) found.
>>> Applying linter output... CHECK
>>> .. INFO: 1 outputs found.
>>> Applying linter inputs... CHECK
>>> .. INFO: Found 3 input parameters.
>>> Applying linter help... CHECK
>>> .. CHECK: Tool contains help section.
>>> .. CHECK: Help contains valid reStructuredText.
>>> Applying linter general... CHECK
>>> .. CHECK: Tool defines a version [0.1.0].
>>> .. CHECK: Tool defines a name [Waste time and memory].
>>> .. CHECK: Tool defines an id [sleepwaste].
>>> .. CHECK: Tool targets 16.01 Galaxy profile.
>>> Applying linter command... CHECK
>>> .. INFO: Tool contains a command.
>>> Applying linter citations... WARNING
>>> .. WARNING: No citations found, consider adding citations to your tool.
>>> Applying linter tool_xsd... CHECK
>>> .. INFO: File validates against XML schema.
>>> Failed linting
>>>
>>> The test of the tool are all successful. Any ideas what is wrong?
>>>
>>> Once the problems are solved, would such a tool also be useful in the
>>> toolshed?
>>>
>>> Cheers,
>>> Matthias
>>>
>>>
>>>
>>> --
>>>
>>> ---
>>> Matthias Bernt
>>> Bioinformatics Service
>>> Molekulare Systembiologie (MOLSYB)
>>> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
>>> Helmholtz Centre for Environmental Research GmbH - UFZ
>>> Permoserstraße 15, 04318 Leipzig, Germany
>>> Phone +49 341 235 482296,
>>> m.be...@ufz.de, www.ufz.de
>>>
>>> Sitz der Gesellschaft/Registered Office: Leipzig
>>> Registergericht/Registration Office: Amtsgericht Leipzig
>>> Handelsregister Nr./Trade Register Nr.: B 4703
>>> Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board:
>>> MinDirig
>>> Wilfried Kraus
>>> Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
>>> Prof. Dr. Dr. h.c. Georg Teutsch
>>> Administrative Geschäftsführerin/ Administrative Managing Director:
>>> Prof. Dr. Heike Graßmann
>>> ---
>>>
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>https://lists.galaxyproject.org/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>http://galaxyproject.org/search/
>
>
> --
>
> ---
> Matthias Bernt
> Bioinformatics Service
> Molekulare Systembiologie (MOLSYB)
> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
> Helmholtz Centre for Environmental Research GmbH - UFZ
> Permoserstraße 15, 04318 Leipzig, Germany
> Phone +49 341 235 482296,
> m.be...@ufz.de, www.ufz.de
>
> Sitz der Gesellschaft/Registered Office: Leipzig
> Registergericht/Registration Office: Amtsgericht Leipzig
> Handelsregister Nr./Trade Register Nr.: B 4703
> Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig
> Wilfried Kraus
> Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
> Prof. Dr. Dr. h.c. Georg Teutsch
> Administrative Geschäftsführerin/ Administrative Managing Director:
> Prof. Dr. Heike Graßmann
> ---
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] tool and planemo problems

[Peter Cock]

How are you running planemo lint - in particular is it set
to fail if there are any warnings present (like your missing
citation warning)?

Peter

On Wed, Jul 12, 2017 at 3:52 PM, Matthias Bernt  wrote:
> Dear list,
>
> In order to test the job resubmission feature I wrote a little tool for
> galaxy that just wastes a specified (minimum) time and memory (using
> different programming languages) for testing the new DRMAA job runner that I
> wrote.
>
> When linting it with planemo seemingly all tests are successful, but in the
> end I still get a fail -- verbose mode is also of no help here.
>
> Applying linter tests... CHECK
> .. CHECK: 7 test(s) found.
> Applying linter output... CHECK
> .. INFO: 1 outputs found.
> Applying linter inputs... CHECK
> .. INFO: Found 3 input parameters.
> Applying linter help... CHECK
> .. CHECK: Tool contains help section.
> .. CHECK: Help contains valid reStructuredText.
> Applying linter general... CHECK
> .. CHECK: Tool defines a version [0.1.0].
> .. CHECK: Tool defines a name [Waste time and memory].
> .. CHECK: Tool defines an id [sleepwaste].
> .. CHECK: Tool targets 16.01 Galaxy profile.
> Applying linter command... CHECK
> .. INFO: Tool contains a command.
> Applying linter citations... WARNING
> .. WARNING: No citations found, consider adding citations to your tool.
> Applying linter tool_xsd... CHECK
> .. INFO: File validates against XML schema.
> Failed linting
>
> The test of the tool are all successful. Any ideas what is wrong?
>
> Once the problems are solved, would such a tool also be useful in the
> toolshed?
>
> Cheers,
> Matthias
>
>
>
> --
>
> ---
> Matthias Bernt
> Bioinformatics Service
> Molekulare Systembiologie (MOLSYB)
> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
> Helmholtz Centre for Environmental Research GmbH - UFZ
> Permoserstraße 15, 04318 Leipzig, Germany
> Phone +49 341 235 482296,
> m.be...@ufz.de, www.ufz.de
>
> Sitz der Gesellschaft/Registered Office: Leipzig
> Registergericht/Registration Office: Amtsgericht Leipzig
> Handelsregister Nr./Trade Register Nr.: B 4703
> Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig
> Wilfried Kraus
> Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
> Prof. Dr. Dr. h.c. Georg Teutsch
> Administrative Geschäftsführerin/ Administrative Managing Director:
> Prof. Dr. Heike Graßmann
> ---
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Issue with job_name, UnicodeDecodeError in drmaa

[Peter Cock]

Thanks, pull request submitted:

https://github.com/galaxyproject/galaxy/pull/4121

Peter


On Mon, May 29, 2017 at 5:23 PM, Saravanaraj Ayyampalayam <ra...@uga.edu> wrote:
> Peter,
>
> Here is the git diff output that shows the changes I made:
>
> diff --git a/lib/galaxy/jobs/runners/__init__.py 
> b/lib/galaxy/jobs/runners/__init__.py
> index ebf4859..6ee51c8 100644
> --- a/lib/galaxy/jobs/runners/__init__.py
> +++ b/lib/galaxy/jobs/runners/__init__.py
> @@ -419,7 +419,7 @@ class JobState( object ):
>  job_name += '_%s' % self.job_wrapper.tool.old_id
>  if self.job_wrapper.user:
>  job_name += '_%s' % self.job_wrapper.user
> -self.job_name = ''.join( map( lambda x: x if x in ( 
> string.letters + string.digits + '_' ) else '_', job_name ) )
> +self.job_name = ''.join( map( lambda x: x if x in ( 
> string.ascii_letters + string.digits + '_' ) else '_', job_name ) )
>
>  @staticmethod
>  def default_job_file( files_dir, id_tag ):
> diff --git a/lib/galaxy/jobs/runners/drmaa.py 
> b/lib/galaxy/jobs/runners/drmaa.py
> index 7c08984..5c5dc62 100644
> --- a/lib/galaxy/jobs/runners/drmaa.py
> +++ b/lib/galaxy/jobs/runners/drmaa.py
> @@ -394,7 +394,7 @@ class DRMAAJobRunner( AsynchronousJobRunner ):
>  job_name += '_%s' % job_wrapper.tool.old_id
>  if external_runjob_script is None:
>  job_name += '_%s' % job_wrapper.user
> -job_name = ''.join( x if x in ( string.letters + string.digits + '_' 
> ) else '_' for x in job_name )
> +job_name = ''.join( x if x in ( string.ascii_letters + string.digits 
> + '_' ) else '_' for x in job_name )
>  if self.restrict_job_name_length:
>  job_name = job_name[:self.restrict_job_name_length]
>  return job_name
>
> -Raj
>
>> On May 29, 2017, at 11:33 AM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
>>
>> Thanks Saravanaraj,
>>
>> You should not need to change the locale, after all many Galaxy servers and
>> their users will be labelling their files in non-English with
>> "special" characters -
>> but perhaps en_US.UTF-8 would be safer in your case as it seems that few
>> people use Galaxy with en_US.iso885915 otherwise this would have been
>> reported before?
>>
>> Can you tell us both places you changed string.letters to 
>> string.ascii_letters
>> please? I could then submit this as a pull request to get Galaxy itself 
>> updated
>> for future release.
>>
>> Peter
>>
>> On Mon, May 29, 2017 at 4:28 PM, Saravanaraj Ayyampalayam <ra...@uga.edu> 
>> wrote:
>>> Peter,
>>>
>>> You are correct. Changing string.letters to string.ascii_letters fixed it. 
>>> I had to change it in two places.
>>>
>>> My locale is :
>>>
>>> LANG=en_US.iso885915
>>> LC_CTYPE="en_US.iso885915"
>>> LC_NUMERIC="en_US.iso885915"
>>> LC_TIME="en_US.iso885915"
>>> LC_COLLATE="en_US.iso885915"
>>> LC_MONETARY="en_US.iso885915"
>>> LC_MESSAGES="en_US.iso885915"
>>> LC_PAPER="en_US.iso885915"
>>> LC_NAME="en_US.iso885915"
>>> LC_ADDRESS="en_US.iso885915"
>>> LC_TELEPHONE="en_US.iso885915"
>>> LC_MEASUREMENT="en_US.iso885915"
>>> LC_IDENTIFICATION="en_US.iso885915"
>>> LC_ALL=
>>>
>>> I will talk to our sysadmin and see if I can change the locale to 
>>> en_US.UTF-8.
>>>
>>> Thank you very much.
>>> -Raj
>>>
>>>
>>>> On May 29, 2017, at 10:58 AM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
>>>>
>>>> Curious, my new guess is this is position 52 a temporary string from
>>>> this expression:
>>>>
>>>> string.letters + string.digits + '_'
>>>>
>>>> This would fit with it being something unusual about the locale of
>>>> your Galaxy server, and my suggested hack may work - replace
>>>> string.letters with string.ascii_letters
>>>>
>>>> Separately at the Linux command line of the server running Galaxy, and
>>>> on a cluster node, what does the command "locale" give? e.g. I get:
>>>>
>>>> $ locale
>>>> LANG=en_US.UTF-8
>>>> LC_CTYPE="en_US.UTF-8"
>>>> LC_NUMERIC="en_US.UTF-8"
>>>> LC_TIME="en_US.UTF-8"
>>>> LC_COLLATE="en_US.UTF-8&qu

Re: [galaxy-dev] Issue with job_name, UnicodeDecodeError in drmaa

[Peter Cock]

Thanks Saravanaraj,

You should not need to change the locale, after all many Galaxy servers and
their users will be labelling their files in non-English with
"special" characters -
but perhaps en_US.UTF-8 would be safer in your case as it seems that few
people use Galaxy with en_US.iso885915 otherwise this would have been
reported before?

Can you tell us both places you changed string.letters to string.ascii_letters
please? I could then submit this as a pull request to get Galaxy itself updated
for future release.

Peter

On Mon, May 29, 2017 at 4:28 PM, Saravanaraj Ayyampalayam <ra...@uga.edu> wrote:
> Peter,
>
> You are correct. Changing string.letters to string.ascii_letters fixed it. I 
> had to change it in two places.
>
> My locale is :
>
> LANG=en_US.iso885915
> LC_CTYPE="en_US.iso885915"
> LC_NUMERIC="en_US.iso885915"
> LC_TIME="en_US.iso885915"
> LC_COLLATE="en_US.iso885915"
> LC_MONETARY="en_US.iso885915"
> LC_MESSAGES="en_US.iso885915"
> LC_PAPER="en_US.iso885915"
> LC_NAME="en_US.iso885915"
> LC_ADDRESS="en_US.iso885915"
> LC_TELEPHONE="en_US.iso885915"
> LC_MEASUREMENT="en_US.iso885915"
> LC_IDENTIFICATION="en_US.iso885915"
> LC_ALL=
>
> I will talk to our sysadmin and see if I can change the locale to en_US.UTF-8.
>
> Thank you very much.
> -Raj
>
>
>> On May 29, 2017, at 10:58 AM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
>>
>> Curious, my new guess is this is position 52 a temporary string from
>> this expression:
>>
>> string.letters + string.digits + '_'
>>
>> This would fit with it being something unusual about the locale of
>> your Galaxy server, and my suggested hack may work - replace
>> string.letters with string.ascii_letters
>>
>> Separately at the Linux command line of the server running Galaxy, and
>> on a cluster node, what does the command "locale" give? e.g. I get:
>>
>> $ locale
>> LANG=en_US.UTF-8
>> LC_CTYPE="en_US.UTF-8"
>> LC_NUMERIC="en_US.UTF-8"
>> LC_TIME="en_US.UTF-8"
>> LC_COLLATE="en_US.UTF-8"
>> LC_MONETARY="en_US.UTF-8"
>> LC_MESSAGES="en_US.UTF-8"
>> LC_PAPER="en_US.UTF-8"
>> LC_NAME="en_US.UTF-8"
>> LC_ADDRESS="en_US.UTF-8"
>> LC_TELEPHONE="en_US.UTF-8"
>> LC_MEASUREMENT="en_US.UTF-8"
>> LC_IDENTIFICATION="en_US.UTF-8"
>> LC_ALL=
>>
>>
>> Peter
>>
>> On Mon, May 29, 2017 at 3:43 PM, Saravanaraj Ayyampalayam <ra...@uga.edu> 
>> wrote:
>>> Peter,
>>>
>>> The job name looks OK. Here is the output of the debug statements:
>>>
>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,423 Making job_name, 
>>> tag was '15132', plan to use u'g15132_fastq_to_fasta_python_ra...@uga.edu' 
>>> with underscores replacements
>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,423 Making job_name, 
>>> character 0 in proposed job_name u'g' is 103
>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,423 Making job_name, 
>>> character 1 in proposed job_name u'1' is 49
>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,423 Making job_name, 
>>> character 2 in proposed job_name u'5' is 53
>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,424 Making job_name, 
>>> character 3 in proposed job_name u'1' is 49
>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,424 Making job_name, 
>>> character 4 in proposed job_name u'3' is 51
>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,424 Making job_name, 
>>> character 5 in proposed job_name u'2' is 50
>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,424 Making job_name, 
>>> character 6 in proposed job_name u'_' is 95
>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,424 Making job_name, 
>>> character 7 in proposed job_name u'f' is 102
>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,425 Making job_name, 
>>> character 8 in proposed job_name u'a' is 97
>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,425 Making job_name, 
>>> character 9 in proposed job_name u's' is 115
>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,425 Making job_name, 
>>> character 10 in proposed job_name u't' is 116
>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,425 Making job_name, 
>>> character 11 in proposed job_name u'q' is 113
>>> galaxy.

Re: [galaxy-dev] Issue with job_name, UnicodeDecodeError in drmaa

[Peter Cock]

cter 28 in proposed job_name u'_' is 95
> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,428 Making job_name, 
> character 29 in proposed job_name u'r' is 114
> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,429 Making job_name, 
> character 30 in proposed job_name u'a' is 97
> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,429 Making job_name, 
> character 31 in proposed job_name u'j' is 106
> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,429 Making job_name, 
> character 32 in proposed job_name u'7' is 55
> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,429 Making job_name, 
> character 33 in proposed job_name u'6' is 54
> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,429 Making job_name, 
> character 34 in proposed job_name u'@' is 64
> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,430 Making job_name, 
> character 35 in proposed job_name u'u' is 117
> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,430 Making job_name, 
> character 36 in proposed job_name u'g' is 103
> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,430 Making job_name, 
> character 37 in proposed job_name u'a' is 97
> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,430 Making job_name, 
> character 38 in proposed job_name u'.' is 46
> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,430 Making job_name, 
> character 39 in proposed job_name u'e' is 101
> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,431 Making job_name, 
> character 40 in proposed job_name u'd' is 100
> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,431 Making job_name, 
> character 41 in proposed job_name u'u' is 117
> galaxy.jobs.runners ERROR 2017-05-29 10:37:21,431 (15132) Unhandled exception 
> calling queue_job
> Traceback (most recent call last):
>   File 
> "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/lib/galaxy/jobs/runners/__init__.py",
>  line 104, in run_next
> method(arg)
>   File 
> "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py",
>  line 132, in queue_job
> job_name = self._job_name(job_wrapper)
>   File 
> "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py",
>  line 400, in _job_name
> job_name = ''.join( x if x in ( string.letters + string.digits + '_' ) 
> else '_' for x in job_name )
>   File 
> "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py",
>  line 400, in 
> job_name = ''.join( x if x in ( string.letters + string.digits + '_' ) 
> else '_' for x in job_name )
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xa6 in position 52: 
> ordinal not in range(128)
>
>
> There is no sign of the 0xa6 at position 52 (the job name is only 42 chars 
> long) in the job name.
>
> Thanks!
> -Raj
>
>
>> On May 29, 2017, at 5:50 AM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
>>
>> Time for more debugging, try inserting something like this inserted
>> the line above into
>> /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py
>> (having backed up the file):
>>
>> log.debug( 'Making job_name, tag was %r, plan to use %r with
>> underscores replacements' % (galaxy_id_tag, job_name))
>>
>> And if that doesn't help, check all the characters:
>>
>> for i, letter in enumerate(job_name): log.debug( 'Making job_name,
>> character %i in proposed job_name %r is %i' % (i, letter,
>> ord(letter)))
>>
>> My hunch is that the locale of the Galaxy server differs from that on
>> the cluster, meaning DRMAA is stricter about allowed characters
>> because Python's string.letters would be whatever is allowed on the
>> Galaxy server's locale.
>>
>> If I am right, a crude fix would be switching to using
>> string.ascii_letters, i.e.
>>
>> job_name = ''.join( x if x in ( string.ascii_letters + string.digits +
>> '_' ) else '_' for x in job_name )
>>
>> Peter
>>
>> On Mon, May 29, 2017 at 6:54 AM, Saravanaraj Ayyampalayam <ra...@uga.edu> 
>> wrote:
>>> Hello,
>>>
>>> I updated to the latest version (17.05) galaxy code. I was updating from a 
>>> very old version.
>>> I fixed all the issues with except one. I am getting the following error 
>>> when I submit a job.
>>>
>>> galaxy.jobs.runners ERROR 2017-05-29 01:42:44,008 (15131) Unhandled 
>>> exception calling queue_job
>>> Traceback (most recent call last):
>>>  File 
>>> "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/lib/galaxy/jobs/runners/__init__.

Re: [galaxy-dev] Issue with job_name, UnicodeDecodeError in drmaa

[Peter Cock]

Time for more debugging, try inserting something like this inserted
the line above into
/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py
(having backed up the file):

log.debug( 'Making job_name, tag was %r, plan to use %r with
underscores replacements' % (galaxy_id_tag, job_name))

And if that doesn't help, check all the characters:

for i, letter in enumerate(job_name): log.debug( 'Making job_name,
character %i in proposed job_name %r is %i' % (i, letter,
ord(letter)))

My hunch is that the locale of the Galaxy server differs from that on
the cluster, meaning DRMAA is stricter about allowed characters
because Python's string.letters would be whatever is allowed on the
Galaxy server's locale.

If I am right, a crude fix would be switching to using
string.ascii_letters, i.e.

job_name = ''.join( x if x in ( string.ascii_letters + string.digits +
'_' ) else '_' for x in job_name )

Peter

On Mon, May 29, 2017 at 6:54 AM, Saravanaraj Ayyampalayam  wrote:
> Hello,
>
> I updated to the latest version (17.05) galaxy code. I was updating from a 
> very old version.
> I fixed all the issues with except one. I am getting the following error when 
> I submit a job.
>
> galaxy.jobs.runners ERROR 2017-05-29 01:42:44,008 (15131) Unhandled exception 
> calling queue_job
> Traceback (most recent call last):
>   File 
> "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/lib/galaxy/jobs/runners/__init__.py",
>  line 104, in run_next
> method(arg)
>   File 
> "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py",
>  line 132, in queue_job
> job_name = self._job_name(job_wrapper)
>   File 
> "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py",
>  line 397, in _job_name
> job_name = ''.join( x if x in ( string.letters + string.digits + '_' ) 
> else '_' for x in job_name )
>   File 
> "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py",
>  line 397, in 
> job_name = ''.join( x if x in ( string.letters + string.digits + '_' ) 
> else '_' for x in job_name )
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xa6 in position 52: 
> ordinal not in range(128)
>
> I spent a day trying to debug this and didn’t get anywhere. The job_name 
> looks OK.
>
> I would really appreciate some help from the community.
>
> Thanks!
> -Raj
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] atlas version 3.10.2 installation error

[Peter Cock]

Good news: With Marius' help Biopython 1.67 should be in bioconda
shortly, https://github.com/bioconda/bioconda-recipes/pull/4462

That means any Galaxy package using Biopython needing/wanting
to work under both the legacy XML packaging system and bioconda
will be able to point at Biopython 1.67.

The newer Biopython releases are only in bioconda (and right now,
I have no compelling reason to write new legacy XML packages for
them).

Likewise any Galaxy package using NCBI BLAST+ needing/wanting
to work under both the legacy XML packaging system and bioconda
will be able to point at BLAST+ 2.5.0.

Peter

On Wed, Apr 19, 2017 at 1:17 PM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
> I've emailed the rest of the IUC for their input - adding the old versions
> in bioconda should work, or adding the recent Biopython releases as packages
> on the Tool Shed.
>
> For blast_rbh just need a semi-recent Biopython available in both systems
> (until everyone moves over to bioconda - we've not yet done this on our
> local Galaxy for example).
>
> Peter
>
> Peter
>
> On Wed, 19 Apr 2017 at 13:04, Matthias Bernt <m.be...@ufz.de> wrote:
>>
>> Dear Peter,
>>
>> thanks. Just for understanding: Why not just change the biopython
>> dependency from 1.67 to 1.69, if this is the one available in bioconda.
>> blast_rbh seems to use only the SeqIO fasta parser and writer. This
>> should be no problem.
>>
>> But I guess that for older versions of blast_rbh one might still need
>> legacy packages of biopython. Or could one change the dependencies in
>> older versions? Or would this be no good idea because of backward
>> compatibility?
>>
>> Cheers,
>> Matthias
>>
>>
>>
>>
>>
>> On 19.04.2017 13:50, Peter Cock wrote:
>> > Just lucky timing, could you try blast_rbh v0.1.11,
>> >
>> > https://toolshed.g2.bx.psu.edu/view/peterjc/blast_rbh/d8d9a9069586
>> >
>> > For BLAST+ this ought to work with either bioconda, or the legacy XML
>> > based packages, as both have BLAST+ 2.5.0.
>> >
>> > However, it looks like bioconda only has Biopython 1.68 and 1.69, while
>> > the legacy XML based packages only go up to Biopython 1.67 at the time
>> > of writing... it looks like we'll need to add a couple more legacy
>> > packages.
>> >
>> > Peter
>> >
>> > On Wed, Apr 19, 2017 at 12:01 PM, Matthias Bernt <m.be...@ufz.de> wrote:
>> >> Dear Peter,
>> >>
>> >> thanks for the info. Would be great to get the update, but since I have
>> >> a
>> >> method that is working for the moment there is no need to hurry.
>> >>
>> >> Best,
>> >> Matthias
>> >>
>> >>
>> >>
>> >> On 19.04.2017 12:57, Peter Cock wrote:
>> >>>
>> >>> Oh right - I've just been updating ncbi_blast_plus this morning
>> >>> to transition to BLAST+ 2.5.0 via either bioconda or the older
>> >>> legacy Tool Shed.
>> >>>
>> >>> I can try to update blast_rbh next, which may solve this by
>> >>> letting you use bioconda.
>> >>>
>> >>> Peter
>> >>>
>> >>> On Wed, Apr 19, 2017 at 11:52 AM, Matthias Bernt <m.be...@ufz.de>
>> >>> wrote:
>> >>>>
>> >>>> Dear Marius,
>> >>>>
>> >>>> thanks again for the help. I'm trying to install blast_rbh (owner is
>> >>>> peterjc). The dependencies look as follows:
>> >>>>
>> >>>> blast_rbh
>> >>>> - biopython
>> >>>>   - package_biopython_1_64
>> >>>> - package_numpy_1_8
>> >>>>   - package_atlas_3_10
>> >>>> - blast
>> >>>> - package_blast_plus_2_2_31
>> >>>>   - blast+
>> >>>>
>> >>>> With unchecked "When available, install Tool Shed managed tool
>> >>>> dependencies?" I get all as "Installed, missing tool dependencies".
>> >>>> Blast_rbh is not working since blast+ is not installed. Even if I
>> >>>> install
>> >>>> blast+ manually (in galaxy) its not working (makeblastdb is missing).
>> >>>>
>> >>>> When I leave "When available, install Tool Shed managed tool
>> >>>> dependencies?"
>> >>>> I see blast_rbh, package_blast_plus package_biopython as insta

Re: [galaxy-dev] atlas version 3.10.2 installation error

[Peter Cock]

I've emailed the rest of the IUC for their input - adding the old versions
in bioconda should work, or adding the recent Biopython releases as
packages on the Tool Shed.

For blast_rbh just need a semi-recent Biopython available in both systems
(until everyone moves over to bioconda - we've not yet done this on our
local Galaxy for example).

Peter

Peter

On Wed, 19 Apr 2017 at 13:04, Matthias Bernt <m.be...@ufz.de> wrote:

> Dear Peter,
>
> thanks. Just for understanding: Why not just change the biopython
> dependency from 1.67 to 1.69, if this is the one available in bioconda.
> blast_rbh seems to use only the SeqIO fasta parser and writer. This
> should be no problem.
>
> But I guess that for older versions of blast_rbh one might still need
> legacy packages of biopython. Or could one change the dependencies in
> older versions? Or would this be no good idea because of backward
> compatibility?
>
> Cheers,
> Matthias
>
>
>
>
>
> On 19.04.2017 13:50, Peter Cock wrote:
> > Just lucky timing, could you try blast_rbh v0.1.11,
> >
> > https://toolshed.g2.bx.psu.edu/view/peterjc/blast_rbh/d8d9a9069586
> >
> > For BLAST+ this ought to work with either bioconda, or the legacy XML
> > based packages, as both have BLAST+ 2.5.0.
> >
> > However, it looks like bioconda only has Biopython 1.68 and 1.69, while
> > the legacy XML based packages only go up to Biopython 1.67 at the time
> > of writing... it looks like we'll need to add a couple more legacy
> packages.
> >
> > Peter
> >
> > On Wed, Apr 19, 2017 at 12:01 PM, Matthias Bernt <m.be...@ufz.de> wrote:
> >> Dear Peter,
> >>
> >> thanks for the info. Would be great to get the update, but since I have
> a
> >> method that is working for the moment there is no need to hurry.
> >>
> >> Best,
> >> Matthias
> >>
> >>
> >>
> >> On 19.04.2017 12:57, Peter Cock wrote:
> >>>
> >>> Oh right - I've just been updating ncbi_blast_plus this morning
> >>> to transition to BLAST+ 2.5.0 via either bioconda or the older
> >>> legacy Tool Shed.
> >>>
> >>> I can try to update blast_rbh next, which may solve this by
> >>> letting you use bioconda.
> >>>
> >>> Peter
> >>>
> >>> On Wed, Apr 19, 2017 at 11:52 AM, Matthias Bernt <m.be...@ufz.de>
> wrote:
> >>>>
> >>>> Dear Marius,
> >>>>
> >>>> thanks again for the help. I'm trying to install blast_rbh (owner is
> >>>> peterjc). The dependencies look as follows:
> >>>>
> >>>> blast_rbh
> >>>> - biopython
> >>>>   - package_biopython_1_64
> >>>> - package_numpy_1_8
> >>>>   - package_atlas_3_10
> >>>> - blast
> >>>> - package_blast_plus_2_2_31
> >>>>   - blast+
> >>>>
> >>>> With unchecked "When available, install Tool Shed managed tool
> >>>> dependencies?" I get all as "Installed, missing tool dependencies".
> >>>> Blast_rbh is not working since blast+ is not installed. Even if I
> install
> >>>> blast+ manually (in galaxy) its not working (makeblastdb is missing).
> >>>>
> >>>> When I leave "When available, install Tool Shed managed tool
> >>>> dependencies?"
> >>>> I see blast_rbh, package_blast_plus package_biopython as installed and
> >>>> the
> >>>> others as Installed, missing tool dependencies.
> >>>> Here blast_rbh is working.
> >>>>
> >>>> What would be your suggestion? Live with the latter and file an issue
> at
> >>>> the
> >>>> tools repository (difficult to say which causes the problem(s))?
> >>>>
> >>>> I get similar behavior if I try to install from the test toolshed.
> >>>> (Only a different error from atlas: /bin/sh: line 3:
> >>>> /host/static_full_blas_lapack.diff: No such file or directory).
> >>>>
> >>>> But, actually I'm trying to automatize tool installation using the
> >>>> ansible
> >>>> role ansible-galaxy-tools
> >>>> (https://github.com/galaxyproject/ansible-galaxy-tools). With this
> method
> >>>> I
> >>>> get the latter behavior. The role uses uses ephemeris==0.4.0 to
> install
> >>>> tools. There I have seen three global variables t

Re: [galaxy-dev] atlas version 3.10.2 installation error

[Peter Cock]

Just lucky timing, could you try blast_rbh v0.1.11,

https://toolshed.g2.bx.psu.edu/view/peterjc/blast_rbh/d8d9a9069586

For BLAST+ this ought to work with either bioconda, or the legacy XML
based packages, as both have BLAST+ 2.5.0.

However, it looks like bioconda only has Biopython 1.68 and 1.69, while
the legacy XML based packages only go up to Biopython 1.67 at the time
of writing... it looks like we'll need to add a couple more legacy packages.

Peter

On Wed, Apr 19, 2017 at 12:01 PM, Matthias Bernt <m.be...@ufz.de> wrote:
> Dear Peter,
>
> thanks for the info. Would be great to get the update, but since I have a
> method that is working for the moment there is no need to hurry.
>
> Best,
> Matthias
>
>
>
> On 19.04.2017 12:57, Peter Cock wrote:
>>
>> Oh right - I've just been updating ncbi_blast_plus this morning
>> to transition to BLAST+ 2.5.0 via either bioconda or the older
>> legacy Tool Shed.
>>
>> I can try to update blast_rbh next, which may solve this by
>> letting you use bioconda.
>>
>> Peter
>>
>> On Wed, Apr 19, 2017 at 11:52 AM, Matthias Bernt <m.be...@ufz.de> wrote:
>>>
>>> Dear Marius,
>>>
>>> thanks again for the help. I'm trying to install blast_rbh (owner is
>>> peterjc). The dependencies look as follows:
>>>
>>> blast_rbh
>>> - biopython
>>>   - package_biopython_1_64
>>> - package_numpy_1_8
>>>   - package_atlas_3_10
>>> - blast
>>> - package_blast_plus_2_2_31
>>>   - blast+
>>>
>>> With unchecked "When available, install Tool Shed managed tool
>>> dependencies?" I get all as "Installed, missing tool dependencies".
>>> Blast_rbh is not working since blast+ is not installed. Even if I install
>>> blast+ manually (in galaxy) its not working (makeblastdb is missing).
>>>
>>> When I leave "When available, install Tool Shed managed tool
>>> dependencies?"
>>> I see blast_rbh, package_blast_plus package_biopython as installed and
>>> the
>>> others as Installed, missing tool dependencies.
>>> Here blast_rbh is working.
>>>
>>> What would be your suggestion? Live with the latter and file an issue at
>>> the
>>> tools repository (difficult to say which causes the problem(s))?
>>>
>>> I get similar behavior if I try to install from the test toolshed.
>>> (Only a different error from atlas: /bin/sh: line 3:
>>> /host/static_full_blas_lapack.diff: No such file or directory).
>>>
>>> But, actually I'm trying to automatize tool installation using the
>>> ansible
>>> role ansible-galaxy-tools
>>> (https://github.com/galaxyproject/ansible-galaxy-tools). With this method
>>> I
>>> get the latter behavior. The role uses uses ephemeris==0.4.0 to install
>>> tools. There I have seen three global variables that (I guess) control
>>> the
>>> installation of dependencies.
>>>
>>> INSTALL_TOOL_DEPENDENCIES = False
>>> INSTALL_REPOSITORY_DEPENDENCIES = True
>>> INSTALL_RESOLVER_DEPENDENCIES = True
>>>
>>> I guess I could control the behavior with the latter two variables.
>>>
>>> Best,
>>> Matthias
>>>
>>>
>>>
>>> On 18.04.2017 19:24, Marius van den Beek wrote:
>>>>
>>>>
>>>> Hi Matthias,
>>>>
>>>> it's me again!
>>>> What tool are you trying to install?
>>>>
>>>> My -perhaps- unhelpful suggestion is to not install anything that starts
>>>> with
>>>> package_* (also called tool shed packages), those should be exclusively
>>>> installed when you absolutely need
>>>> to install an old tool that can only function with packages.
>>>> We've switched to conda nowadays, which is much easier to install and to
>>>> maintain.
>>>> In fact the IUC has ported all their latest tools to conda, and
>>>> similarly most devteam
>>>> tools that are still in active use have also been ported to use Conda
>>>> dependencies.
>>>>
>>>> If you look at this
>>>> document
>>>> https://galaxyproject.org/admin/tools/add-tool-from-toolshed-tutorial/,
>>>> there is a part where you confirm dependencies. If you click on show
>>>> detail, and you unselect
>>>> When available, install Tool Shed managed tool dependencies?
>>>> Galaxy will not try to install these old

Re: [galaxy-dev] atlas version 3.10.2 installation error

[Peter Cock]

Oh right - I've just been updating ncbi_blast_plus this morning
to transition to BLAST+ 2.5.0 via either bioconda or the older
legacy Tool Shed.

I can try to update blast_rbh next, which may solve this by
letting you use bioconda.

Peter

On Wed, Apr 19, 2017 at 11:52 AM, Matthias Bernt  wrote:
> Dear Marius,
>
> thanks again for the help. I'm trying to install blast_rbh (owner is
> peterjc). The dependencies look as follows:
>
> blast_rbh
> - biopython
>   - package_biopython_1_64
> - package_numpy_1_8
>   - package_atlas_3_10
> - blast
> - package_blast_plus_2_2_31
>   - blast+
>
> With unchecked "When available, install Tool Shed managed tool
> dependencies?" I get all as "Installed, missing tool dependencies".
> Blast_rbh is not working since blast+ is not installed. Even if I install
> blast+ manually (in galaxy) its not working (makeblastdb is missing).
>
> When I leave "When available, install Tool Shed managed tool dependencies?"
> I see blast_rbh, package_blast_plus package_biopython as installed and the
> others as Installed, missing tool dependencies.
> Here blast_rbh is working.
>
> What would be your suggestion? Live with the latter and file an issue at the
> tools repository (difficult to say which causes the problem(s))?
>
> I get similar behavior if I try to install from the test toolshed.
> (Only a different error from atlas: /bin/sh: line 3:
> /host/static_full_blas_lapack.diff: No such file or directory).
>
> But, actually I'm trying to automatize tool installation using the ansible
> role ansible-galaxy-tools
> (https://github.com/galaxyproject/ansible-galaxy-tools). With this method I
> get the latter behavior. The role uses uses ephemeris==0.4.0 to install
> tools. There I have seen three global variables that (I guess) control the
> installation of dependencies.
>
> INSTALL_TOOL_DEPENDENCIES = False
> INSTALL_REPOSITORY_DEPENDENCIES = True
> INSTALL_RESOLVER_DEPENDENCIES = True
>
> I guess I could control the behavior with the latter two variables.
>
> Best,
> Matthias
>
>
>
> On 18.04.2017 19:24, Marius van den Beek wrote:
>>
>> Hi Matthias,
>>
>> it's me again!
>> What tool are you trying to install?
>>
>> My -perhaps- unhelpful suggestion is to not install anything that starts
>> with
>> package_* (also called tool shed packages), those should be exclusively
>> installed when you absolutely need
>> to install an old tool that can only function with packages.
>> We've switched to conda nowadays, which is much easier to install and to
>> maintain.
>> In fact the IUC has ported all their latest tools to conda, and
>> similarly most devteam
>> tools that are still in active use have also been ported to use Conda
>> dependencies.
>>
>> If you look at this
>> document
>> https://galaxyproject.org/admin/tools/add-tool-from-toolshed-tutorial/,
>> there is a part where you confirm dependencies. If you click on show
>> detail, and you unselect
>> When available, install Tool Shed managed tool dependencies?
>> Galaxy will not try to install these old packages. There are some tools
>> which come both with
>> old toolshed dependencies and that have Conda available. You may be
>> lucky and find that
>> the tool you are trying to install has Conda dependencies available.
>> There is a lot more background on the Toolshed and Conda in this
>> document here:
>> https://docs.galaxyproject.org/en/master/admin/conda_faq.html
>>
>> Best,
>> Marius
>>
>> On 18 April 2017 at 19:03, Matthias Bernt > > wrote:
>>
>> Dear list,
>>
>> again me :( stumbling from problem to problem ...
>>
>> I got compilation errors when installing
>>
>> package_atlas_3_10 Revision: 98c017ec230d
>>
>> I tried with gcc 4x and 6x without success. Do I need the intel C
>> compiler for this tool?
>>
>> If I got earlier posts
>> (http://dev.list.galaxyproject.org/atlas-3-10-IUC-error-tt4668236.html
>>
>> )
>>
>> correct a precompiled version should be used anyway?
>>
>> I'm running galaxy on CentOS 6.8 (64bit) (running in a VirtualBox).
>>
>> Best,
>> Matthias
>>
>>
>> Here is the output (which I guess is verbose output from configure):
>>
>>
>> /tmp/cctyMGAH.o: In function `ATL_tmpnam':
>>
>> /gpfs1/data/galaxy_server/galaxy-dev/database/tmp/tmp-toolshed-mtdAAKCLe/ATLAS/build/..//CONFIG/include/atlas_sys.h:224:
>> warning: the use of `tmpnam' is dangerous, better use `mkstemp'
>> probe_OS.o: In function `ATL_tmpnam':
>>
>> /gpfs1/data/galaxy_server/galaxy-dev/database/tmp/tmp-toolshed-mtdAAKCLe/ATLAS/build/..//CONFIG/include/atlas_sys.h:224:
>> warning: the use of `tmpnam' is dangerous, better use `mkstemp'
>> probe_asm.o: In function `ATL_tmpnam':
>>
>> /gpfs1/data/galaxy_server/galaxy-dev/database/tmp/tmp-toolshed-mtdAAKCLe/ATLAS/build/..//CONFIG/include/atlas_sys.h:224:
>> warning: the use of `tmpnam' is 

Re: [galaxy-dev] Testing updated NCBI BLAST+ wrappers for version 2.5.0

[Peter Cock]

This just went live on the main tool shed:

https://toolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus/

You can report issues here or on GitHub at:

https://github.com/peterjc/galaxy_blast/issues

Thanks,

Peter

On Thu, Dec 8, 2016 at 11:35 AM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
> Hello all,
>
> I have updated the NCBI BLAST+ wrappers on the Test Tool Shed,
> the wrapper is now at v0.2.00:
>
> https://testtoolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus/
>
> The main changes is this now depends on BLAST+ 2.5.0, and that is
> available via either BioConda or the Tool Shed:
>
> https://toolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_5_0/
> https://testtoolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_5_0/
>
> In order for the dependency to work smoothly on both BioConda
> and the Tool Shed system, we have changed the package name
> from "blast+" to just "blast". Given the NCBI stopped updated the
> original "legacy" BLAST some time ago, when combined with the
> version number this is no longer ambiguous.
>
> Jumping from using BLAST+ 2.2.31 to using BLAST+ 2.5.0
> required updating lots of the test files for NCBI changes, including
> dropping the GI numbers in many outputs, expanding the percentage
> identity field from 2dp to 3dp, and also changing how -parse_deflines
> works with tabular output.
>
> The wrappers (deliberately) do not yet offer any new functionality
> added in the recent NCBI BLAST+ updates, in particular BLAST
> XML v2 is not yet available as an output with a datatype in Galaxy.
>
> At this point I would welcome feedback from those of you using the
> BLAST+ wrappers - including if you were able to install this with the
> dependencies from BioConda or the traditional Tool Shed packages.
>
> Once I'm confident that this is all OK, I will update the main Tool Shed
> (and think about adding new functionality in 2017).
>
> Thank you all,
>
> Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] GCC2107 Update: Oral presentation abstracts due 15 April

[Peter Cock]

Hello all,

In addition to 15 April being the GCC2017 Oral Presentation
deadline, it is also the next Open Bioinformatics Foundation
(OBF) Travel Fellowship application deadline.

The program offers up to USD $1000 and is aimed at increasing
diverse participation at events promoting open source bioinformatics
software development and open science in the biological research
community (like the Galaxy Community Conference).

http://news.open-bio.org/2016/03/01/obf-travel-fellowship-program/
https://github.com/OBF/obf-docs/blob/master/Travel_fellowships.md

Regards,

Peter

(Dr Peter Cock, OBF Treasurer)


On Thu, Apr 13, 2017 at 10:02 PM, Dave Clements
<cleme...@galaxyproject.org> wrote:
> Hello all,
>
>
> This is just a reminder of some GCC2017 deadlines:
>
>
>   15 April:  Oral presentation abstracts are due this Saturday at 23:59
> Paris local time.
>
>   15 May:  Early registration ends
>
>   27 May:  Poster presentation abstracts are due.
>
>   27 May:  Visualization and Computer Demo presentation abstracts are due.
>
>   31 May:  Regular registration ends
>
>   23 June: Lightning Talk presentation abstracts are due.
>
>
> About GCC2017:
>
> GCC2017 will be in Montpellier, France, 26-30 June and will feature two days
> of presentations, discussions, poster sessions, lightning talks, computer
> demos, keynotes, and birds-of-a-feather meetups, all about data-intensive
> biology and the tools that support it. GCC2017 also includes data and coding
> hackathons, and two days of training covering 16 different topics.
>
> GCC2017 will be held at Le Corum Conference Centre in the heart of
> Montpellier, just 10km from the Mediterranean. This event will gather
> several hundred researchers addressing diverse questions and facing common
> challenges in data intensive life science research. GCC participants work
> across the tree of life, come from around the world, and work at
> universities, research organizations, industry, medical schools and research
> hospitals.  If you work in or support data intensive life science research
> then GCC2017 is an ideal opportunity to present your work.
>
>
> Early registration is starts at less than 55€ / day for post-docs and
> students.  You can also book low cost conference housing when you register.
>
>
> GCC2017 has sold out all premiere sponsorship slots (a first). If you are
> interested in sponsoring (or know someone who is) there are still Silver and
> Bronze and Hackathon sponsorships available. Contact the organisers if you
> are interested.
>
>
> About Galaxy
>
> Galaxy is an open, web-based platform for data-intensive biomedical analysis
> used by tens of thousands of researchers around the world.  It supports ad
> hoc exploration and analysis through scalable and repeatable data analysis
> pipelines for large research studies. Galaxy is available in over 90 free
> and publicly accessible web servers, on commercial and national cloud
> infrastructures, and is locally installed at hundreds, if not thousands, of
> research organisations around the world.
>
>
> We hope to see you this summer in Montpellier!
>
>
> Au revoir,
>
>
> The GCC2017 Organising Committee
>
>
>
> --
> https://galaxyproject.org/
> https://getgalaxy.org/
> https://usegalaxy.org/
> https://gcc2017.sciencesconf.org/
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] [Bosc] BOSC 2017: Call for Abstracts

[Peter Cock]

Hello all,

This is a reminder that the main ISMB/ECCB 2017 abstract submission
deadline is this Thursday 13 April - this is one of the biggest and
broadest commutational biology conferences and there will I'm sure be a
strong Galaxy presence.

https://www.iscb.org/ismbeccb2017-keydates

This deadline also applies to talks at all the former satellite or special
interest group (SIG) meetings, which are now called Communities of Special
Intererst (COSIs), and will this year run though the main meeting.

In particular, please hurry up and get your open source/open science
abstracts finished and submitted to BOSC, the annual Bioinformatics Open
Source Conference [*].

https://www.open-bio.org/wiki/BOSC_2017

Also, BOSC's patent organisation the Open Bioinformatics Foundation (OBF)
[**] has a travel fellowship available to help improve diversity at
Bioinformatics meetings including but not limited to BOSC. The next call
for this closes 15 April 2017.

https://github.com/OBF/obf-docs/blob/master/Travel_fellowships.md

Thanks for reading to the end.
Peter

[*] I'm on the BOSC committee and used to co-chair it, so clearly biased.

[**] I'm on the OBF board as treasurer.

On Wed, 8 Mar 2017 at 18:05, Peter Cock <p.j.a.c...@googlemail.com> wrote:

> Dear all,
>
> Recent BOSC meetings have had a strong Galaxy presence, so please do
> consider submitting an abstract and/or attending BOSC 2017 in Prague.
>
> Thank you,
>
> Peter
>
> Dr. Peter Cock,
> Bioinformatician at The James Hutton Institute;
> Open Bioinformatics Foundation, board of directors, treasurer;
> BOSC 2017 organising committee member.
>
> -- Forwarded message --
> From: *Nomi Harris* <nlhar...@gmail.com>
> Date: Mon, Mar 6, 2017 at 6:50 PM
> Subject: [Bosc] BOSC 2017: Call for Abstracts
> To: bosc-annou...@mailman.open-bio.org, bosc-review...@open-bio.org
> Cc: Nomi Harris <nlhar...@gmail.com>, BOSC 2017 Organizing Committee <
> b...@open-bio.org>
>
>
> *Call for Abstracts for the 18th Annual Bioinformatics Open Source
> Conference (BOSC 2017)*
> An ISMB/ECCB Community of Special Interest (COSI)
>
> Dates: July 22-23, 2017
> Location: Prague, Czech Republic
> Web site: http://www.open-bio.org/wiki/BOSC_2017
> Email: b...@open-bio.org
> BOSC announcements mailing list:
> http://lists.open-bio.org/mailman/listinfo/bosc-announce
> Twitter: @OBF_BOSC
>
> *Important Dates*
>
>- Call for one-page abstracts
><https://www.open-bio.org/wiki/BOSC_Abstract_Submission> opens: March
>6, 2017
>- Abstract submission
><https://www.open-bio.org/wiki/BOSC_Abstract_Submission> deadline:
>April 13, 2017
>- Travel fellowship
><https://github.com/OBF/obf-docs/blob/master/Travel_fellowships.md> 
> application
>deadline: April 15, 2017
>- Authors notified: May 10, 2017
>- Codefest 2017 <https://www.open-bio.org/wiki/Codefest_2017>: July
>21-22, Prague
>- BOSC 2017 <https://www.open-bio.org/wiki/BOSC_2017>: July 22-23,
>Prague  (days 1 and 2 of ISMB/ECCB)
>- ISMB/ECCB 2017 <https://www.iscb.org/ismbeccb2017>: July 21-25,
>Prague
>
> *About BOSC*
> Since 2000, the yearly Bioinformatics Open Source Conference (BOSC) has
> provided a forum for developers and users to interact and share research
> results and ideas in open source bioinformatics. BOSC’s broad spectrum of
> topics includes practical techniques for solving bioinformatics problems;
> software development practices; standards and ontologies; approaches that
> promote open science and sharing of data, results and software; and ways to
> grow open source communities while promoting diversity within them.
>
> In the past, BOSC has taken place the two days before ISMB as a Special
> Interest Group (SIG). This year, ISMB is trying a new structure: the SIGs
> (now called COSIs) are integrated into the main ISMB meeting. BOSC will
> take place the first two full days of ISMB (July 22-23). Attendees will
> have the option to register for the full ISMB/ECCB meeting (July 21-25) or
> for just two days (there is no single-day registration option this year). A
> limited number of partial travel fellowships will be granted to some
> accepted speakers who would not otherwise be able to attend BOSC--please
> see https://github.com/OBF/obf-docs/blob/master/Travel_fellowships.md for
> more information.
>
> We encourage you to submit one-page abstracts on any topic of relevance to
> open source bioinformatics and open science. After review, some abstracts
> will be selected for lightning talks, longer talks, and/or posters.
> Abstract submission instructions and a link to the EasyChair portal can be
> found on https://www.open-bio.org/wiki/BOSC_Abstra

Re: [galaxy-dev] cluster access

[Peter Cock]

That's interesting. I didn't set this up on our system, but if the LDAP can
tell Galaxy the real username, maybe we can avoid having to setup
usern...@example.org aliases and insisting our users do that for
logging into Galaxy.

Hopefully some other SGE users can comment on how they got the
run-as-user system working for them?

Peter

On Mon, Apr 10, 2017 at 3:43 PM, Matthias Bernt <m.be...@ufz.de> wrote:
> Hi,
>
> sorry, I forgot this central piece of information: We also use Univa Grid
> Engine.
>
> What would be easier adapting galaxy's mapping to users or setting up email
> aliases? I would need to ask our cluster admins if they could realize this.
>
> Best,
> Matthias
>
>
>
> On 10.04.2017 16:39, Peter Cock wrote:
>>
>> This may depend on the cluster setup, and how Galaxy accesses it.
>> What are you using?
>>
>> We use SGE (actually Univa Grid Engine) via DRMAA, and had to setup
>> usern...@example.org email aliases just for this.
>>
>> Peter
>>
>> On Mon, Apr 10, 2017 at 3:36 PM, Matthias Bernt <m.be...@ufz.de> wrote:
>>>
>>> Dear list,
>>>
>>> I'm just starting to get jobs submitted to our cluster as real system
>>> user.
>>> I read in the documentation that the name of the system user is
>>> determined
>>> by "the Galaxy user's email address (with the @domain stripped off)". But
>>> this does not work on our system, rather the username stored in the
>>> galaxy_user table is the name of the user (this is correctly generated by
>>> the LDAP login).
>>>
>>> The job runner script seems to get the user id, so I guess I need to dig
>>> deeper. So my question is: Where could I change this behavior?
>>>
>>> Thanks a lot.
>>>
>>> Cheers,
>>> Matthias
>>>
>>>
>>> --
>>>
>>> ---
>>> Matthias Bernt
>>> Bioinformatics Service
>>> Molekulare Systembiologie (MOLSYB)
>>> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
>>> Helmholtz Centre for Environmental Research GmbH - UFZ
>>> Permoserstraße 15, 04318 Leipzig, Germany
>>> Phone +49 341 235 482296,
>>> m.be...@ufz.de, www.ufz.de
>>>
>>> Sitz der Gesellschaft/Registered Office: Leipzig
>>> Registergericht/Registration Office: Amtsgericht Leipzig
>>> Handelsregister Nr./Trade Register Nr.: B 4703
>>> Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board:
>>> MinDirig
>>> Wilfried Kraus
>>> Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
>>> Prof. Dr. Dr. h.c. Georg Teutsch
>>> Administrative Geschäftsführerin/ Administrative Managing Director:
>>> Prof. Dr. Heike Graßmann
>>> ---
>>>
>>>
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>   https://lists.galaxyproject.org/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>   http://galaxyproject.org/search/
>
>
> --
>
> ---
> Matthias Bernt
> Bioinformatics Service
> Molekulare Systembiologie (MOLSYB)
> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
> Helmholtz Centre for Environmental Research GmbH - UFZ
> Permoserstraße 15, 04318 Leipzig, Germany
> Phone +49 341 235 482296,
> m.be...@ufz.de, www.ufz.de
>
> Sitz der Gesellschaft/Registered Office: Leipzig
> Registergericht/Registration Office: Amtsgericht Leipzig
> Handelsregister Nr./Trade Register Nr.: B 4703
> Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig
> Wilfried Kraus
> Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
> Prof. Dr. Dr. h.c. Georg Teutsch
> Administrative Geschäftsführerin/ Administrative Managing Director:
> Prof. Dr. Heike Graßmann
> ---
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] cluster access

[Peter Cock]

This may depend on the cluster setup, and how Galaxy accesses it.
What are you using?

We use SGE (actually Univa Grid Engine) via DRMAA, and had to setup
usern...@example.org email aliases just for this.

Peter

On Mon, Apr 10, 2017 at 3:36 PM, Matthias Bernt  wrote:
> Dear list,
>
> I'm just starting to get jobs submitted to our cluster as real system user.
> I read in the documentation that the name of the system user is determined
> by "the Galaxy user's email address (with the @domain stripped off)". But
> this does not work on our system, rather the username stored in the
> galaxy_user table is the name of the user (this is correctly generated by
> the LDAP login).
>
> The job runner script seems to get the user id, so I guess I need to dig
> deeper. So my question is: Where could I change this behavior?
>
> Thanks a lot.
>
> Cheers,
> Matthias
>
>
> --
>
> ---
> Matthias Bernt
> Bioinformatics Service
> Molekulare Systembiologie (MOLSYB)
> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
> Helmholtz Centre for Environmental Research GmbH - UFZ
> Permoserstraße 15, 04318 Leipzig, Germany
> Phone +49 341 235 482296,
> m.be...@ufz.de, www.ufz.de
>
> Sitz der Gesellschaft/Registered Office: Leipzig
> Registergericht/Registration Office: Amtsgericht Leipzig
> Handelsregister Nr./Trade Register Nr.: B 4703
> Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig
> Wilfried Kraus
> Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
> Prof. Dr. Dr. h.c. Georg Teutsch
> Administrative Geschäftsführerin/ Administrative Managing Director:
> Prof. Dr. Heike Graßmann
> ---
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] git clone 16.10 bwa 0.7.15 data manager

[Peter Cock]

Hi Bryan,

I suspect the missing dependency BWA may reflect the switch away
from Galaxy's home-grown packaging system to using Conda -
enabled by default since v17.01,

https://docs.galaxyproject.org/en/master/admin/conda_faq.html

Newer dependencies will likely next get an old fashioned package
on the Galaxy Tool Shed.

Peter

On Fri, Apr 7, 2017 at 12:18 AM, Bryan Hepworth
 wrote:
> Hi All
>
> I've been trying to get a local galaxy running with an assortment of tools 
> for a local researcher. I'm a linux administrator rather than a 
> bioinformatics researcher and am familiar with how repo's work for RedHat, R 
> and the like. I've built tools for other groups at work and from my reading 
> it looked like galaxy was working on similar lines for keeping things in step.
>
> I've done a git clone of 16.10 into /galaxy altered the galaxy.ini to 
> postgresql and added me as admin. Done the sh run.sh to get a feel for how 
> everything goes together. I've pulled reference genomes from the source and 
> have had some success converting to different formats. I bumped up against a 
> few dependency issues so I've also enabled the test repo.
>
> The first stumbling block on the most recent build is getting bwa on there 
> with the data manager route - it has a dependency on bwa 0.7.15, but try as I 
> might I don't seem to be able to find an instance of package_bwa_0.7.15 
> anywhere on the devteam repo's to get over this hurdle. I'm guessing I'm 
> missing something really simple as it isn't pulling in dependancies as the 
> documentation suggests it should. I'd be really grateful for any pointers.
>
> Bryan
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

[galaxy-dev] Fwd: [Bosc] BOSC 2017: Call for Abstracts

[Peter Cock]

Dear all,

Recent BOSC meetings have had a strong Galaxy presence, so please do
consider submitting an abstract and/or attending BOSC 2017 in Prague.

Thank you,

Peter

Dr. Peter Cock,
Bioinformatician at The James Hutton Institute;
Open Bioinformatics Foundation, board of directors, treasurer;
BOSC 2017 organising committee member.

-- Forwarded message --
From: Nomi Harris <nlhar...@gmail.com>
Date: Mon, Mar 6, 2017 at 6:50 PM
Subject: [Bosc] BOSC 2017: Call for Abstracts
To: bosc-annou...@mailman.open-bio.org, bosc-review...@open-bio.org
Cc: Nomi Harris <nlhar...@gmail.com>, BOSC 2017 Organizing Committee <
b...@open-bio.org>


*Call for Abstracts for the 18th Annual Bioinformatics Open Source
Conference (BOSC 2017)*
An ISMB/ECCB Community of Special Interest (COSI)

Dates: July 22-23, 2017
Location: Prague, Czech Republic
Web site: http://www.open-bio.org/wiki/BOSC_2017
Email: b...@open-bio.org
BOSC announcements mailing list: http://lists.open-bio.
org/mailman/listinfo/bosc-announce
Twitter: @OBF_BOSC

*Important Dates*

   - Call for one-page abstracts
   <https://www.open-bio.org/wiki/BOSC_Abstract_Submission> opens: March 6,
   2017
   - Abstract submission
   <https://www.open-bio.org/wiki/BOSC_Abstract_Submission> deadline: April
   13, 2017
   - Travel fellowship
   <https://github.com/OBF/obf-docs/blob/master/Travel_fellowships.md>
application
   deadline: April 15, 2017
   - Authors notified: May 10, 2017
   - Codefest 2017 <https://www.open-bio.org/wiki/Codefest_2017>: July
   21-22, Prague
   - BOSC 2017 <https://www.open-bio.org/wiki/BOSC_2017>: July 22-23,
   Prague  (days 1 and 2 of ISMB/ECCB)
   - ISMB/ECCB 2017 <https://www.iscb.org/ismbeccb2017>: July 21-25, Prague

*About BOSC*
Since 2000, the yearly Bioinformatics Open Source Conference (BOSC) has
provided a forum for developers and users to interact and share research
results and ideas in open source bioinformatics. BOSC’s broad spectrum of
topics includes practical techniques for solving bioinformatics problems;
software development practices; standards and ontologies; approaches that
promote open science and sharing of data, results and software; and ways to
grow open source communities while promoting diversity within them.

In the past, BOSC has taken place the two days before ISMB as a Special
Interest Group (SIG). This year, ISMB is trying a new structure: the SIGs
(now called COSIs) are integrated into the main ISMB meeting. BOSC will
take place the first two full days of ISMB (July 22-23). Attendees will
have the option to register for the full ISMB/ECCB meeting (July 21-25) or
for just two days (there is no single-day registration option this year). A
limited number of partial travel fellowships will be granted to some
accepted speakers who would not otherwise be able to attend BOSC--please
see https://github.com/OBF/obf-docs/blob/master/Travel_fellowships.md for
more information.

We encourage you to submit one-page abstracts on any topic of relevance to
open source bioinformatics and open science. After review, some abstracts
will be selected for lightning talks, longer talks, and/or posters.
Abstract submission instructions and a link to the EasyChair portal can be
found on https://www.open-bio.org/wiki/BOSC_Abstract_Submission

*Session topics include:*

   - Open Science and Reproducible Research
   - Open Biomedical Data
   - Citizen/Participatory Science
   - Standards and Interoperability
   - Data Science
   - Workflows
   - Visualization
   - Medical and Translational Bioinformatics
   - Developer Tools and Libraries
   - Bioinformatics Open Source Project Progress Reports

*Sponsorship*
We gratefully accept sponsorships from relevant private companies. These
sponsorships enable us to offer free registration to some BOSC speakers to
help increase diversity at our meeting. Sponsors in 2016 included
Curoverse, the company behind the open source platform Arvados. Please
contact us if you are interested in being a sponsor of BOSC 2017!

Thank you,
  BOSC 2017 Organizing Committee:   Nomi Harris (chair), Brad Chapman,
Peter Cock, Christopher Fields, Bastian Greshake, Karsten Hokamp, Hilmar
Lapp, Mónica Muñoz-Torres, Heather Wiencko

P.S. Don't forget to submit your BOSC abstract by April 13 at
https://www.open-bio.org/wiki/BOSC_Abstract_Submission!

-- 
You received this message because you are subscribed to the Google Groups
"OBF-BOSC" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to obf-bosc+unsubscr...@googlegroups.com.
Visit this group at https://groups.google.com/group/obf-bosc.
For more options, visit https://groups.google.com/d/optout.
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists

Re: [galaxy-dev] Rg: Galaxy Version

[Peter Cock]

Thanks Martin & Eugen,

That's much easier (although Anamika will likely need to check if
their Galaxy was setup using hg or git anyway for updating it).

I'm not sure when it was introduced, but its been a while as even
the older Galaxy instance in my example under hg gives:

{"version_major": "15.05"}

Regards,

Peter


On Wed, Mar 8, 2017 at 2:06 PM, Martin Čech <mar...@bx.psu.edu> wrote:
> Hi Anamika,
>
> adding a detail to Peter's excellent answer:
>
> For newer versions of Galaxy (while it is running) you can go to the url
> `your.galaxy.com/api/version` to learn the release of Galaxy you have.
>
> Thanks for using Galaxy,
>
> Martin
>
> On Wed, Mar 8, 2017 at 7:39 AM Peter Cock <p.j.a.c...@googlemail.com> wrote:
>>
>> Galaxy was originally distributed via the mercurial (hg) version
>> control system, later the git version control system.
>>
>> Try this (which will work on a very old Galaxy using hg),
>>
>> $ hg log | head
>> changeset:   17125:e837a68e8d14
>> tag: tip
>> parent:  17121:7a4e3a661a76
>> parent:  17124:46a4ef8850b5
>> user:Nate Coraor <n...@bx.psu.edu>
>> date:Fri May 15 15:21:34 2015 -0400
>> summary: Merge stable to default
>> ...
>>
>> If that fails, try this which will work on a Galaxy distributed with git:
>>
>> $ git log | head
>> commit cbd527b9a4671c7abe9a22cf69e869b5d2de858a
>> Merge: 33dec01 d0f39ce
>> Author: Nicola Soranzo <nicola.sora...@earlham.ac.uk>
>> Date:   Wed Dec 7 20:35:40 2016 +
>>
>> Merge branch 'release_16.10' into dev
>>
>> Conflicts:
>> static/scripts/bundled/analysis.bundled.js
>> static/scripts/bundled/analysis.bundled.js.map
>> ...
>>
>>
>> If it isn't clear from the commit message, you can then match those
>> commits
>> to the relevant repository and hopefully work it out, here those were:
>>
>> https://bitbucket.org/galaxy/galaxy-central/commits/e837a68e8d14
>>
>> and
>>
>>
>> https://github.com/galaxyproject/galaxy/commits/cbd527b9a4671c7abe9a22cf69e869b5d2de858a
>>
>> If neither git nor hg work, then perhaps your Galaxy was installed via a
>> different route - and that will likely complicate updating it.
>>
>> Peter
>>
>>
>> On Wed, Mar 8, 2017 at 10:36 AM, Anamika Singh <anamikalibr...@gmail.com>
>> wrote:
>> > Dear galaxy team,
>> >
>> > I am using galaxy platform (old version) in Linux OS which was not
>> > installed
>> > by me. I would like to know 'how to check what is the galaxy version
>> > installed currently in my system.
>> >
>> > Regards
>> >
>> > --
>> > Anamika
>> >
>> > ___
>> > Please keep all replies on the list by using "reply all"
>> > in your mail client.  To manage your subscriptions to this
>> > and other Galaxy lists, please use the interface at:
>> >   https://lists.galaxyproject.org/
>> >
>> > To search Galaxy mailing lists use the unified search at:
>> >   http://galaxyproject.org/search/mailinglists/
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Execute command once job has completed

[Peter Cock]

I do not believe that is possible (nor desirable in general,
not everyone will need or want the converted file and the
overhead that comes with creating and storing it).

What do you want to do with the bigwig file, and is the
automatic conversion working for you? i.e. If you have
tool X in Galaxy which wants an bigwig input file, does
Galaxy offer an automated conversion of your bedgraph
file as an input in the drop down file list?

Peter


On Tue, Feb 28, 2017 at 5:27 PM, evan clark <eclar...@fau.edu> wrote:
> By automatic I mean that once any bedgraph file is finished writing it will
> automatically run the convert to bigwig command
>
>
> Peter Cock
> February 28, 2017 12:21 PM
> It depends on what you mean by automatic. You could
> create a Galaxy Workflow where a tool is run which
> generates the bedgraph file, and then as the next
> step it is converted to bigwig.
>
> For this specific task, it might be possible to exploit
> the Galaxy datatype definition for bedgraph to allow
> automatic conversion to bigwig - is this working for you?
>
> https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/datatypes/converters/bedgraph_to_bigwig_converter.xml
>
> Peter
> evan clark
> February 28, 2017 11:47 AM
> Is there a method to setup a command to run depending on the filetype of a
> job. Essentially I want to setup a task where everytime a bedgraph file is
> generated it will either automatically start a new job and convert it to
> bigwig or, within the same job file it will convert the bedgraph to bigwig.
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Execute command once job has completed

[Peter Cock]

It depends on what you mean by automatic. You could
create a Galaxy Workflow where a tool is run which
generates the bedgraph file, and then as the next
step it is converted to bigwig.

For this specific task, it might be possible to exploit
the Galaxy datatype definition for bedgraph to allow
automatic conversion to bigwig - is this working for you?

https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/datatypes/converters/bedgraph_to_bigwig_converter.xml

Peter

On Tue, Feb 28, 2017 at 4:47 PM, evan clark  wrote:
> Is there a method to setup a command to run depending on the filetype of a
> job. Essentially I want to setup a task where everytime a bedgraph file is
> generated it will either automatically start a new job and convert it to
> bigwig or, within the same job file it will convert the bedgraph to bigwig.
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Tools using Galaxy's Python library?

[Peter Cock]

To recap, in the next release of Galaxy, there will not be a copy of the
sequence parsing code like lib/galaxy_utils/sequence/fastq.py - see
this pull request: https://github.com/galaxyproject/galaxy/pull/3551

Both Galaxy itself, and any tools needing it, will depend on the now
separately packaged galaxy_sequence_utils library, source code now
here:

https://github.com/galaxyproject/sequence_utils

The original version of this is available on the Tool Shed:

https://toolshed.g2.bx.psu.edu

The library is now available on PyPI too:

https://pypi.python.org/pypi/galaxy_sequence_utils/

This code has been updated and now does "import six" as part of
work to make sure it can be used on Python 2 and Python 3, and
it includes support for compressed FASTQ files.

My TravisCI tool tests are passing again using both Galaxy's stable
and dev branches.

Peter

On Wed, Feb 1, 2017 at 11:20 AM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
> This went smoother today, and has served to refresh some of the
> details of how the Galaxy's venv etc all work.
>
> In my TravisCI setup for tool testing I currently "manually" install
> the tool dependencies, and thus am using pip with the same URL
> as the Tool Shed package:
>
> pip install 
> http://depot.galaxyproject.org/package/source/galaxy_sequence_utils/galaxy_sequence_utils-1.0.1-st.tgz
>
> Commit:
>
> https://github.com/peterjc/pico_galaxy/commit/c9b77b49281974988c790a54f05c4d1ff81b0904
>
> The TravisCI build is still failing, but on an unrelated change
> on the Galaxy dev branch.
>
> Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] ImportError and Project life cycle

[Peter Cock]

On Mon, Feb 13, 2017 at 9:17 AM, Matthias Bernt  wrote:
> Dear Martin,
>
> Thanks for your input. Before answering the raised questions I would like to
> add some more information that I just found:
>
> I've noticed something odd (to me) which is maybe the source of the problem.
> My galaxy installation has galaxy_utils installed twice:
>
> - lib/galaxy_utils/sequence/fastq.py
>
> -
> database/dependencies/galaxy_sequence_utils/1.0.0/devteam/package_galaxy_utils_1_0/8882f14715b5/lib/python/galaxy_sequence_utils-1.0.0-py2.7.egg/galaxy_utils/sequence/fastq.py
>

Something like that is to be expected right now as this is in
transition, see also:

http://dev.list.galaxyproject.org/Tools-using-Galaxy-s-Python-library-td4670472.html

In the next release of Galaxy, there will not be a copy of the code at
lib/galaxy_utils/sequence/fastq.py - see this pull request for the change
https://github.com/galaxyproject/galaxy/pull/3551

Originally tools were allowed to access Galaxy's internal Python code,
here lib/galaxy_utils/sequence/fastq.py - but that is changing. In the
next Galaxy release tools should explicitly depend on the new
packaged version of Galaxy's sequence_utils - that's where the
second copy you've seen comes from:

https://github.com/galaxyproject/sequence_utils

This does now import six as part of work to make sure it can be
used on Python 2 and Python 3:

https://github.com/galaxyproject/sequence_utils/blob/master/galaxy_utils/sequence/sequence.py

Galaxy itself will depend on the now separate sequence_utils
package.

--

As to what is going wrong in your situation, I'm hoping one of the
Galaxy developers working on this can help.

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] ImportError and Project life cycle

[Peter Cock]

On Fri, Feb 10, 2017 at 3:26 PM, Matthias Bernt  wrote:
> Dear galaxy-dev list,
>
> I'm currently trying to introduce myself to the galaxy server, i.e.,
> learning how to use and administer a galaxy server.
>
> I have installed galaxy following this excellent tutorial:
>
> https://github.com/martenson/dagobah-training
>
> Then I tried the tool "FASTQ Quality Trimmer by sliding window" which is
> mentioned in NGS101-6 tutorial. I installed the tool from tool shed, but I
> get a python error (the Admin panel shows that all dependencies are
> installed):
>
> Traceback (most recent call last):
>   File
> "/home/berntm/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/fastq_trimmer_by_quality/25c24379693a/fastq_trimmer_by_quality/fastq_trimmer_by_quality.py",
> line 3, in 
> from galaxy_utils.sequence.fastq import fastqReader, fastqWriter
>   File "/home/berntm/galaxy/lib/galaxy_utils/sequence/fastq.py", line 7, in
> 
> from six import Iterator, string_types
> ImportError: No module named six
>
> Do you have an idea what could be wrong? I guess that this is to little
> information, but I did not know which kind of additional information would
> be helpful.

This appears to be a missing dependency on the Python module "six"
which is used to help write code which will work under both Python 2
and Python 3.

This is likely a recent regression - hopefully one of the Galaxy team
can comment on this, likely the Tool Shed definition for this tool needs
a minor tweak (and then you can apply the update via the Galaxy
Admin controls within your browser).

> Another question: At my institution quite a lot of NGS projects are carried
> out. After a project is finished all methods and data are stored to a long
> term read only tape archive. For the future the idea is to use galaxy. Which
> options are there to implement such a project life cycle with galaxy?

Not that I personally am aware of, no. There are optional disk quotas to
encourage users to remove (delete) old data.

> A last general question: would it be better to send a separate mail to the
> list when I have multiple questions?

Yes please, with useful email subjects would be best.

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Create a new datatype svg

[Peter Cock]

Hmm. I wonder if a composite datatype would work here?

https://wiki.galaxyproject.org/Admin/Datatypes/Composite%20Datatypes

Peter

On Thu, Feb 9, 2017 at 9:25 AM, Jochen Bick  wrote:
> Hi Dan,
>
> I understand the option with this specific flag. What I mean is that for
> example the file format bam has two file in one save button. So if you
> click on save you can either save the bam file or the bai file. I would
> liek to have the same feature with my png and svg. So that I have just
> one item in the history showing the png and if you click on save you
> have the option to also save svg.
>
> Cheers Jochen
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Peter Cock <p.j.a.c...@googlemail.com>

[Peter Cock]

Thank you,

Peter

On Wed, Feb 1, 2017 at 3:11 PM, Eduardo de Paiva Alves
<alves.edua...@gmail.com> wrote:
> Hi Peter,
>
> I am wrapping https://rostlab.org/owiki/index.php/Tmseg which takes a
> PSSM matrix as one of the inputs so I will add the psiblast locally
> and test and let yo know how it goes.
>
> Thank you
>
> Eduardo
>
> On 01/02/2017 10:10, "Peter Cock" <p.j.a.c...@googlemail.com> wrote:
>
>
> Hi Eduardo,
>
>
> Certainly when Luobin Yang submitted the early work on the
> PSI BLAST wrapper we need the PSSM datatype to be defined,
> but as you note, that has been done:
>
> https://github.com/peterjc/galaxy_blast/commit/65252ee61fdd98e52454ff045fa069f04ec40c06
>
> According to the minimal notes I made on the issue for finishing
> the PSI BLAST wrapper, it is just help text and unit tests needed
> (which would include active user testing):
>
> https://github.com/peterjc/galaxy_blast/issues/19
>
> If you are a PSI BLAST user, it would be very helpful to have
> your input here.
>
> Right now the wrapper is on the TravisCI blacklist because
> it would fail testing. Likewise it is not included in the Tool Shed.
>
> Peter
>
> On Wed, Feb 1, 2017 at 9:47 AM, De paiva Alves, Eduardo
> <eduardoal...@abdn.ac.uk> wrote:
> Peter
>
> I noticed there is now a PSSM datatype which is used by NCBI BLAST+
> makeprofiledb. Does that allow you to remove psiblast from blacklist in
> your repository?
>
>
> https://github.com/peterjc/galaxy_blast/blob/13d5b6deca0663d7f1301428574720
> b19aea1380/.tt_blacklist
>
>
> Thank you
>
> Eduardo
>
>
> On Tue, Feb 5, 2013 at 4:28 PM, Peter Cock <[hidden email]> wrote:
> Thanks Luobin,
>
> I've checked that into my development repository (on my tools branch):
> https://bitbucket.org/peterjc/galaxy-central/commits/f8f43f8494abdd228998d4
> e9fe67b0f2378494e0
>
> That will allow me to track changes etc as we work on the datatypes.
> Note I am not yet including ncbi_psiblast_wrapper.xml on the Galaxy
> Tool Shed.
>
> Regards,
>
> Peter
>
>
>
> The University of Aberdeen is a charity registered in Scotland, No SC013683.
> Tha Oilthigh Obar Dheathain na charthannas clàraichte ann an Alba, Àir. 
> SC013683
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Tools using Galaxy's Python library?

[Peter Cock]

This went smoother today, and has served to refresh some of the
details of how the Galaxy's venv etc all work.

In my TravisCI setup for tool testing I currently "manually" install
the tool dependencies, and thus am using pip with the same URL
as the Tool Shed package:

pip install 
http://depot.galaxyproject.org/package/source/galaxy_sequence_utils/galaxy_sequence_utils-1.0.1-st.tgz

Commit:

https://github.com/peterjc/pico_galaxy/commit/c9b77b49281974988c790a54f05c4d1ff81b0904

The TravisCI build is still failing, but on an unrelated change
on the Galaxy dev branch.

Peter

On Wed, Feb 1, 2017 at 9:46 AM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
> Thanks John - that could explain what's going wrong and offers
> another approach to fixing my TravisCI testing - setting:
>
> preserve_python_environment  = legacy_and_local
>
> or:
>
> preserve_python_environment = always
>
> (Despite several attempts I didn't hit on a nice way to get the
> galaxy_sequence_utils installed nicely via a "manual install",
> one day I will switch my TravisCI testing to use BioConda)
>
> I am hoping to get my TravisCI builds passing again today...
>
> Peter
>
> On Tue, Jan 31, 2017 at 5:27 PM, John Chilton <jmchil...@gmail.com> wrote:
>> So the PR that broke your tools is probably here
>> https://github.com/galaxyproject/galaxy/pull/3364/files. That pull request
>> removed Galaxy from the Python path of Galaxy tools - this gives tools a
>> much cleaner environment and prevents certain conflicts between Conda and
>> Galaxy.
>>
>> As part of that PR, there is a list of Galaxy tools in
>> lib/galaxy/tools/__init__.py that still get setup with Galaxy's Python
>> environment. This includes many random devteam and even an IUC tool - I've
>> added something to the list even though one ancient version of tool shed
>> tool required it years ago and it has since been updated. I would open a PR
>> to add the tool id of any of your tools that have been published to the tool
>> shed and require these tools to this list. You can target it against 17.01
>> ideally and then we can merge that into dev.
>>
>> This behavior can be reverted - there is a config option that is documented
>> pretty well in the PR but I missed named it in the sample (fixing it with
>> https://github.com/galaxyproject/galaxy/pull/3521/files).
>>
>> Hopefully this helps and thanks for the bug report!
>>
>> -John
>>
>>
>> On Tue, Jan 31, 2017 at 7:08 AM Peter Cock <p.j.a.c...@googlemail.com>
>> wrote:
>>>
>>> Thanks Nicola,
>>>
>>> I didn't spot the missing slash, but planemo lint did - for anyone
>>> else copy-and-pasting:
>>>
>>> >> version="1.0.1">galaxy_sequence_utils
>>>
>>> For anyone not yet using BioConda with Galaxy, there is also a Tool
>>> Shed entry here:
>>>
>>>
>>> https://toolshed.g2.bx.psu.edu/view/iuc/package_galaxy_sequence_utils_1_0_1/
>>>
>>> I am therefore adding this to my tool_dependencies.xml files:
>>>
>>> 
>>> >> />
>>> 
>>>
>>> I still need to work out how best to make this available under TravisCI,
>>> given I am not currently using BioConda for the dependencies.
>>>
>>> @IUC: Should this also be released on PyPI for easy install via pip?
>>>
>>> Peter
>>>
>>>
>>> On Tue, Jan 31, 2017 at 11:15 AM, Nicola Soranzo <nsora...@tiscali.it>
>>> wrote:
>>> > Hi Peter,
>>> > adding
>>> >
>>> > >> > version="1.0.1">galaxy_sequence_utils
>>> >
>>> > to each of these tools should solve the problem, there is a Bioconda
>>> > package
>>> > which provides the Python library:
>>> >
>>> > https://anaconda.org/bioconda/galaxy_sequence_utils
>>> >
>>> > Cheers,
>>> > Nicola
>>> >
>>> >
>>> > On 31/01/17 10:39, Peter Cock wrote:
>>> >>
>>> >> Hi all,
>>> >>
>>> >> A few of my tools have for a long time used Galaxy's own parsing
>>> >> functionality in order to avoid an external dependency. Lately
>>> >> this has stopped working on my TravisCI testing with planemo
>>> >> using the Galaxy dev branch (the stable master branch is fine):
>>> >> e.g.
>>> >>
>>> >> https://travis-ci.org/peterjc/pico_galaxy/builds/196655736
>>> >>
>&g

Re: [galaxy-dev] psiblast

[Peter Cock]

Hi Eduardo,

Certainly when Luobin Yang submitted the early work on the
PSI BLAST wrapper we need the PSSM datatype to be defined,
but as you note, that has been done:

https://github.com/peterjc/galaxy_blast/commit/65252ee61fdd98e52454ff045fa069f04ec40c06

According to the minimal notes I made on the issue for finishing
the PSI BLAST wrapper, it is just help text and unit tests needed
(which would include active user testing):

https://github.com/peterjc/galaxy_blast/issues/19

If you are a PSI BLAST user, it would be very helpful to have
your input here.

Right now the wrapper is on the TravisCI blacklist because
it would fail testing. Likewise it is not included in the Tool Shed.

Peter

On Wed, Feb 1, 2017 at 9:47 AM, De paiva Alves, Eduardo
<eduardoal...@abdn.ac.uk> wrote:
> Peter
>
> I noticed there is now a PSSM datatype which is used by NCBI BLAST+
> makeprofiledb. Does that allow you to remove psiblast from blacklist in
> your repository?
>
> https://github.com/peterjc/galaxy_blast/blob/13d5b6deca0663d7f1301428574720
> b19aea1380/.tt_blacklist
>
>
> Thank you
>
> Eduardo
>
>
> On Tue, Feb 5, 2013 at 4:28 PM, Peter Cock <[hidden email]> wrote:
> Thanks Luobin,
>
> I've checked that into my development repository (on my tools branch):
> https://bitbucket.org/peterjc/galaxy-central/commits/f8f43f8494abdd228998d4
> e9fe67b0f2378494e0
>
> That will allow me to track changes etc as we work on the datatypes.
> Note I am not yet including ncbi_psiblast_wrapper.xml on the Galaxy
> Tool Shed.
>
> Regards,
>
> Peter
>
>
>
> The University of Aberdeen is a charity registered in Scotland, No SC013683.
> Tha Oilthigh Obar Dheathain na charthannas clàraichte ann an Alba, Àir. 
> SC013683.
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Tools using Galaxy's Python library?

[Peter Cock]

Thanks Nicola,

I didn't spot the missing slash, but planemo lint did - for anyone
else copy-and-pasting:

galaxy_sequence_utils

For anyone not yet using BioConda with Galaxy, there is also a Tool
Shed entry here:

https://toolshed.g2.bx.psu.edu/view/iuc/package_galaxy_sequence_utils_1_0_1/

I am therefore adding this to my tool_dependencies.xml files:





I still need to work out how best to make this available under TravisCI,
given I am not currently using BioConda for the dependencies.

@IUC: Should this also be released on PyPI for easy install via pip?

Peter


On Tue, Jan 31, 2017 at 11:15 AM, Nicola Soranzo <nsora...@tiscali.it> wrote:
> Hi Peter,
> adding
>
>  version="1.0.1">galaxy_sequence_utils
>
> to each of these tools should solve the problem, there is a Bioconda package
> which provides the Python library:
>
> https://anaconda.org/bioconda/galaxy_sequence_utils
>
> Cheers,
> Nicola
>
>
> On 31/01/17 10:39, Peter Cock wrote:
>>
>> Hi all,
>>
>> A few of my tools have for a long time used Galaxy's own parsing
>> functionality in order to avoid an external dependency. Lately
>> this has stopped working on my TravisCI testing with planemo
>> using the Galaxy dev branch (the stable master branch is fine):
>> e.g.
>>
>> https://travis-ci.org/peterjc/pico_galaxy/builds/196655736
>>
>> The tools fail with things like:
>>
>> |  from galaxy_utils.sequence.fasta import fastaReader, fastaWriter
>> |  ImportError: No module named galaxy_utils.sequence.fasta
>>
>>
>> or:
>>
>> |  from galaxy_utils.sequence.fastq import fastqReader
>> |  ImportError: No module named galaxy_utils.sequence.fastq
>>
>> Is this a temporary regression in Galaxy, or a deliberate change?
>> Do the tools need to do something to explicit have access to the
>> Galaxy Python library, or are they now considered private?
>> If so, I can update these tools to use an explicit dependency
>> for parsing FASTA and FASTQ (e.g. Biopython).
>>
>> Thanks,
>>
>> Peter
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>http://galaxyproject.org/search/mailinglists/
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Tools using Galaxy's Python library?

[Peter Cock]

Hi all,

A few of my tools have for a long time used Galaxy's own parsing
functionality in order to avoid an external dependency. Lately
this has stopped working on my TravisCI testing with planemo
using the Galaxy dev branch (the stable master branch is fine):
e.g.

https://travis-ci.org/peterjc/pico_galaxy/builds/196655736

The tools fail with things like:

|  from galaxy_utils.sequence.fasta import fastaReader, fastaWriter
|  ImportError: No module named galaxy_utils.sequence.fasta


or:

|  from galaxy_utils.sequence.fastq import fastqReader
|  ImportError: No module named galaxy_utils.sequence.fastq

Is this a temporary regression in Galaxy, or a deliberate change?
Do the tools need to do something to explicit have access to the
Galaxy Python library, or are they now considered private?
If so, I can update these tools to use an explicit dependency
for parsing FASTA and FASTQ (e.g. Biopython).

Thanks,

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Check running jobs

[Peter Cock]

Ah - I assumed you meant from the user-facing UI, or the reports page.

I would suggest exploring the advance job metrics options, see e.g.

https://github.com/galaxyproject/galaxy/blob/dev/config/job_metrics_conf.xml.sample

I've never used it, but perhaps the backend using Collectl
(http://collectl.sourceforge.net/) would cover this?

Peter

On Fri, Jan 27, 2017 at 2:50 PM, Katherine Beaulieu
 wrote:
> Ok so there is no way to access the galaxy object outside of galaxy? I'm
> looking for something like the 'trans' variable often seen throughout the
> Galaxy code base so that I can use sqlalchemy and make a database query.
> Thanks for replying.
> Katherine
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Check running jobs

[Peter Cock]

Hi Katherine,

I don't think Galaxy itself will tell you this, in part because the
details would be dependent on the Cluster being used.

I would do that via whatever cluster management system you are running
(e.g. qstat or qmon if using SGE).

By default all the jobs will be submitted by the Galaxy user account,
but if you are submitting jobs as the individual users then you can
still spot Galaxy jobs via their naming convention (starting "g",
number, underscore, ..., tool name, user's email).

Peter

On Thu, Jan 26, 2017 at 6:58 PM, Katherine Beaulieu
 wrote:
> Hi there,
> We are trying to build our own Galaxy cluster and I was wondering if there
> was a way to check if there are any running Galaxy jobs on a specific
> server? This would be to determine whether or not to remove the server from
> the cluster.
> Thanks!
> Katherine
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Blast db permission

[Peter Cock]

Hello Mohamed,

Right now this is not possible, and I can't think of any way to
restrict access to system level BLAST databases listed in
the blast*.loc files by user. However, since other Galaxy
tools use *.loc files, someone else may have some ideas.

You might be able to try using a BLAST database in a
shared data library (Galaxy has a role based permissions
system you could use), but the user would first have to
import the database into their current history.

I hope that helps,

Peter

On Thu, Jan 19, 2017 at 1:45 PM, Mohamed Kassam  wrote:
> Dear all,
>
> I would like to know if it is possible to put permission on the macro.xml
> for ncbi-blast + database.
> For example if I want to show the db to one user and not to all.
>
> Best reagards,
>
> Mohamed
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Testing updated NCBI BLAST+ wrappers for version 2.5.0

[Peter Cock]

Hello all,

I have updated the NCBI BLAST+ wrappers on the Test Tool Shed,
the wrapper is now at v0.2.00:

https://testtoolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus/

The main changes is this now depends on BLAST+ 2.5.0, and that is
available via either BioConda or the Tool Shed:

https://toolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_5_0/
https://testtoolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_5_0/

In order for the dependency to work smoothly on both BioConda
and the Tool Shed system, we have changed the package name
from "blast+" to just "blast". Given the NCBI stopped updated the
original "legacy" BLAST some time ago, when combined with the
version number this is no longer ambiguous.

Jumping from using BLAST+ 2.2.31 to using BLAST+ 2.5.0
required updating lots of the test files for NCBI changes, including
dropping the GI numbers in many outputs, expanding the percentage
identity field from 2dp to 3dp, and also changing how -parse_deflines
works with tabular output.

The wrappers (deliberately) do not yet offer any new functionality
added in the recent NCBI BLAST+ updates, in particular BLAST
XML v2 is not yet available as an output with a datatype in Galaxy.

At this point I would welcome feedback from those of you using the
BLAST+ wrappers - including if you were able to install this with the
dependencies from BioConda or the traditional Tool Shed packages.

Once I'm confident that this is all OK, I will update the main Tool Shed
(and think about adding new functionality in 2017).

Thank you all,

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Unable to upload bam files as links to my local galaxy instance - please help!

[Peter Cock]

Galaxy itself needs samtools, and this is not handled via the
Tool Shed which only does dependencies of Galaxy Tools.

I don't think the website is clear enough about this,
https://wiki.galaxyproject.org/Admin/GetGalaxy
just points at
https://wiki.galaxyproject.org/Admin/Tools/ToolDependencies

Which version of Galaxy are you using? Older versions had
to have an older samtools (which had a different API for
using samtools sort).

Peter

On Mon, Dec 5, 2016 at 12:36 PM, Konstantinos Voutetakis  wrote:
> Dear Sir(s),
>
> I have consumed a few days in order to resolve this issue but nothing! I
> have installed all the SAM tools of public Galaxy in my local galaxy via the
> Toolshed (installing automatically all the dependencies were needed). I have
> also export the path of the executable file SAMtools from bin folder to my
> Galaxy path and I take the following error during uploading:
>
> Traceback (most recent call last):
>   File "/home/user/galaxy/tools/data_source/upload.py", line 434, in
> 
> __main__()
>   File "/home/user/galaxy/tools/data_source/upload.py", line 423, in
> __main__
> add_file( dataset, registry, json_file, output_path )
>   File "/home/user/galaxy/tools/data_source/upload.py", line 317, in
> add_file
> if datatype.dataset_content_needs_grooming( dataset.path ):
>   File "/home/user/galaxy/lib/galaxy/datatypes/binary.py", line 251, in
> dataset_content_needs_grooming
> version = self._get_samtools_version()
>   File "/home/user/galaxy/lib/galaxy/datatypes/binary.py", line 212, in
> _get_samtools_version
> raise Exception(message)
> Exception: Attempting to use functionality requiring samtools, but it cannot
> be located on Galaxy's PATH.
>
> When I am in my galaxy folder (from where I connect to my galaxy server) and
> I type samtools in the terminal I take:
> Program: samtools (Tools for alignments in the SAM format)
> Version: 0.1.19-44428cd
>
> Usage:   samtools  [options]
>
> Command: viewSAM<->BAM conversion
>  sortsort alignment file
>  mpileup multi-way pileup
>  depth   compute the depth
>  faidx   index/extract FASTA
>  tview   text alignment viewer
>  index   index alignment
>  idxstatsBAM index stats (r595 or later)
>  fixmate fix mate information
>  flagstatsimple stats
>  calmd   recalculate MD/NM tags and '=' bases
>  merge   merge sorted alignments
>  rmdup   remove PCR duplicates
>  reheaderreplace BAM header
>  cat concatenate BAMs
>  bedcov  read depth per BED region
>  targetcut   cut fosmid regions (for fosmid pool only)
>  phase   phase heterozygotes
>  bamshuf shuffle and group alignments by name
>
> What can I do? I installed galaxy to my pc (ubuntu 16.04) before two weeks
> and it is up to date. I would be grateful for your help.
>
> Yours sincerely,
> Kostas
>
> --
> Konstantinos G. Voutetakis, MSc - Research Officer
> National Hellenic Research Foundation (N.H.R.F.)
> Institute of Biology, Medicinal Chemistry and Biotechnology
> 48 Vas. Constantinou Ave., Athens 11635
> Greece
> --
> 
> tel.: +30-210-7273894
> e-mail: kvou...@eie.gr
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Code File Execution

[Peter Cock]

Are you asking how Galaxy takes the user-entered parameters
(requested in the tool XML file with  tags) and builds a
command line string using the Cheetah-format template defined
in the  tag?

At least part of the answer to that is the ToolEvaluator object:

https://github.com/galaxyproject/galaxy/blob/71cea6604d43c5fe6215f5656462ba6c1af69bb6/lib/galaxy/tools/evaluation.py#L454

Peter

On Mon, Nov 21, 2016 at 6:37 PM, Katherine Beaulieu
 wrote:
> Hi Everyone,
> Does anyone know how execution of code files happens? Is there a function
> somewhere that fills the parameters in?
> Cheers,
> Katherine
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Reporting issues for the wiki? e.g. universe_wgsi.ini still widely used

[Peter Cock]

Thanks Dannon,

I've logged the universe_wsgi.ini issue there as
https://github.com/galaxyproject/galaxy-site/issues/16

Interesting to see yet another MediaWiki site being moved to
GitHub with Markdown - although not with GitHub pages in this
case. I'd guess you guys used the wonderful pandoc for the
conversion?

I assume you'll be able to add "edit" buttons to pages pointing
at the markdown pages on GitHub? While it could be prettier,
this works pretty well for us using GitHub Pages:

https://github.com/biopython/biopython.github.io/pull/78

Regards,

Peter

On Tue, Nov 15, 2016 at 3:10 PM, Dannon Baker <dannon.ba...@gmail.com> wrote:
> Hi Peter,
>
> We're in the middle of migrating the wiki right now -- you may notice that
> the current wiki is in read-only mode.  Going forward, the place to report
> (and fix) issues like that will be
> https://github.com/galaxyproject/galaxy-site.
>
> -Dannon
>
> On Tue, Nov 15, 2016 at 10:06 AM, Peter Cock <p.j.a.c...@googlemail.com>
> wrote:
>>
>> Hello all,
>>
>> Is there a specific issue tracker for the wiki? It seems wrong to
>> open an issue on the main GitHub code repository...
>>
>> e.g. Many of the documentation pages still refer to the legacy
>> configuration filename universe_wgsi.ini rather than the new
>> name of config/galaxy.ini
>>
>> Problem pages include:
>>
>> https://wiki.galaxyproject.org/Learn/SecurityFeatures
>> https://wiki.galaxyproject.org/Admin/DiskQuotas
>> https://wiki.galaxyproject.org/Admin/SampleTracking/Demo
>> https://wiki.galaxyproject.org/Admin/Tools/MultipleOutputFiles
>> https://wiki.galaxyproject.org/Admin/Config/GenomeSpace
>> https://wiki.galaxyproject.org/Admin/Config/ApacheExternalUserAuth
>> https://wiki.galaxyproject.org/VisualizationSetup
>> https://wiki.galaxyproject.org/ToolShedApi
>>
>> Other pages would benefit from minor edits to at least put the old form
>> second and in brackets:
>>
>> https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster
>> https://wiki.galaxyproject.org/Admin/Interface
>> https://wiki.galaxyproject.org/ToolShed/InstallingRepositoriesToGalaxy
>> Some pages look fine, e.g.
>>
>> https://wiki.galaxyproject.org/ToolShed/InstallingAndCompilingPackages
>>
>> Or are clearly dated and appropriate for the time, e.g. Dev news briefs
>>
>> https://wiki.galaxyproject.org/DevNewsBriefs/2010_07_16
>>
>> This means lots of pages with the term which are fine:
>>
>>
>> https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax?action=fullsearch=0=0=universe_wsgi.ini
>>
>> Regards,
>>
>> Peter
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Reporting issues for the wiki? e.g. universe_wgsi.ini still widely used

[Peter Cock]

Hello all,

Is there a specific issue tracker for the wiki? It seems wrong to
open an issue on the main GitHub code repository...

e.g. Many of the documentation pages still refer to the legacy
configuration filename universe_wgsi.ini rather than the new
name of config/galaxy.ini

Problem pages include:

https://wiki.galaxyproject.org/Learn/SecurityFeatures
https://wiki.galaxyproject.org/Admin/DiskQuotas
https://wiki.galaxyproject.org/Admin/SampleTracking/Demo
https://wiki.galaxyproject.org/Admin/Tools/MultipleOutputFiles
https://wiki.galaxyproject.org/Admin/Config/GenomeSpace
https://wiki.galaxyproject.org/Admin/Config/ApacheExternalUserAuth
https://wiki.galaxyproject.org/VisualizationSetup
https://wiki.galaxyproject.org/ToolShedApi

Other pages would benefit from minor edits to at least put the old form
second and in brackets:

https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster
https://wiki.galaxyproject.org/Admin/Interface
https://wiki.galaxyproject.org/ToolShed/InstallingRepositoriesToGalaxy
Some pages look fine, e.g.

https://wiki.galaxyproject.org/ToolShed/InstallingAndCompilingPackages

Or are clearly dated and appropriate for the time, e.g. Dev news briefs

https://wiki.galaxyproject.org/DevNewsBriefs/2010_07_16

This means lots of pages with the term which are fine:

https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax?action=fullsearch=0=0=universe_wsgi.ini

Regards,

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Installing ToolShed Tools into Galaxy via Browser Interface Exception Thrown

[Peter Cock]

Good news - this is the bug I though it was, the ampersand
should be escaped as  in the XML file:

https://github.com/galaxyproject/galaxy/issues/3084

More good news: This will likely be fixed in the next release
of Galaxy:

https://github.com/galaxyproject/galaxy/pull/3122

Right now I suggest you manually edit these XML files to
use the word "and" instead of "&".

Peter


On Thu, Nov 3, 2016 at 2:36 PM, Yip, Miu ki <m...@cshl.edu> wrote:
> Hi Peter,
>
> Line 44 is oddly specific. Nevertheless, we have this as the 44th line in the 
> xml file.
>
>  version="" />
>
> This tool is not the one trying to be installed.
>
> Thanks,
> Miuki
> On Nov 3, 2016, at 10:28 AM, Peter Cock 
> <p.j.a.c...@googlemail.com<mailto:p.j.a.c...@googlemail.com>> wrote:
>
> I think you may have hit the same bug I found recently:
>
> https://github.com/galaxyproject/galaxy/issues/3084
>
> What is line 44 of your integrated_tool_panel.xml file?
>
> Peter
>
> On Thu, Nov 3, 2016 at 2:16 PM, Yip, Miu ki <m...@cshl.edu> wrote:
> Hi all,
>
> While trying to install tools from the toolshed via the browser interface, an 
> error keeps appearing and nothing is abled to be installed. Currently using 
> the latest version of Galaxy and having tried to install several different 
> tools, the following error keeps appearing in the log file.
>
> galaxy.queue_worker DEBUG 2016-11-03 13:08:32,425 Executing toolbox reload on 
> 'main'
> Exception in thread Thread-2:
> Traceback (most recent call last):
>  File "/usr/lib64/python2.7/threading.py", line 811, in __bootstrap_inner
>self.run()
>  File "/usr/lib64/python2.7/threading.py", line 764, in run
>self.__target(*self.__args, **self.__kwargs)
>  File 
> "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/watcher.py", line 
> 147, in on_any_event
>self._handle(event)
>  File 
> "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/watcher.py", line 
> 150, in _handle
>self.reload_callback()
>  File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
> line 78, in 
>self._tool_conf_watcher = get_tool_conf_watcher( lambda: 
> reload_toolbox(app))
>  File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/queue_worker.py", line 
> 56, in reload_toolbox
>app.toolbox = _get_new_toolbox(app)
>  File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/queue_worker.py", line 
> 72, in _get_new_toolbox
>new_toolbox = tools.ToolBox(tool_configs, app.config.tool_path, app, 
> app.toolbox._tool_conf_watcher)
>  File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/__init__.py", line 
> 110, in __init__
>tool_conf_watcher=tool_conf_watcher
>  File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
> line 1037, in __init__
>super(BaseGalaxyToolBox, self).__init__(config_filenames, tool_root_dir, 
> app, tool_conf_watcher=tool_conf_watcher)
>  File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
> line 69, in __init__
>self._init_integrated_tool_panel( app.config )
>  File 
> "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/integrated_panel.py",
>  line 36, in _init_integrated_tool_panel
>self._load_integrated_tool_panel_keys()
>  File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
> line 368, in _load_integrated_tool_panel_keys
>tree = parse_xml( self._integrated_tool_panel_config )
>  File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/util/__init__.py", line 
> 211, in parse_xml
>root = tree.parse( fname, parser=ElementTree.XMLParser( 
> target=DoctypeSafeCallbackTarget() ) )
>  File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 656, in parse
>parser.feed(data)
>  File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1642, in feed
>self._raiseerror(v)
>  File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1506, in 
> _raiseerror
>raise err
> ParseError: not well-formed (invalid token): line 44, column 71
>
> Is there an XML or configuration file which needs to be changed for this to 
> work? Not sure how to proceed.
>
> Thanks!
> Miuki
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Installing ToolShed Tools into Galaxy via Browser Interface Exception Thrown

[Peter Cock]

I think you may have hit the same bug I found recently:

https://github.com/galaxyproject/galaxy/issues/3084

What is line 44 of your integrated_tool_panel.xml file?

Peter

On Thu, Nov 3, 2016 at 2:16 PM, Yip, Miu ki  wrote:
> Hi all,
>
> While trying to install tools from the toolshed via the browser interface, an 
> error keeps appearing and nothing is abled to be installed. Currently using 
> the latest version of Galaxy and having tried to install several different 
> tools, the following error keeps appearing in the log file.
>
> galaxy.queue_worker DEBUG 2016-11-03 13:08:32,425 Executing toolbox reload on 
> 'main'
> Exception in thread Thread-2:
> Traceback (most recent call last):
>   File "/usr/lib64/python2.7/threading.py", line 811, in __bootstrap_inner
> self.run()
>   File "/usr/lib64/python2.7/threading.py", line 764, in run
> self.__target(*self.__args, **self.__kwargs)
>   File 
> "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/watcher.py", line 
> 147, in on_any_event
> self._handle(event)
>   File 
> "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/watcher.py", line 
> 150, in _handle
> self.reload_callback()
>   File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
> line 78, in 
> self._tool_conf_watcher = get_tool_conf_watcher( lambda: 
> reload_toolbox(app))
>   File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/queue_worker.py", line 
> 56, in reload_toolbox
> app.toolbox = _get_new_toolbox(app)
>   File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/queue_worker.py", line 
> 72, in _get_new_toolbox
> new_toolbox = tools.ToolBox(tool_configs, app.config.tool_path, app, 
> app.toolbox._tool_conf_watcher)
>   File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/__init__.py", line 
> 110, in __init__
> tool_conf_watcher=tool_conf_watcher
>   File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
> line 1037, in __init__
> super(BaseGalaxyToolBox, self).__init__(config_filenames, tool_root_dir, 
> app, tool_conf_watcher=tool_conf_watcher)
>   File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
> line 69, in __init__
> self._init_integrated_tool_panel( app.config )
>   File 
> "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/integrated_panel.py",
>  line 36, in _init_integrated_tool_panel
> self._load_integrated_tool_panel_keys()
>   File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
> line 368, in _load_integrated_tool_panel_keys
> tree = parse_xml( self._integrated_tool_panel_config )
>   File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/util/__init__.py", line 
> 211, in parse_xml
> root = tree.parse( fname, parser=ElementTree.XMLParser( 
> target=DoctypeSafeCallbackTarget() ) )
>   File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 656, in parse
> parser.feed(data)
>   File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1642, in feed
> self._raiseerror(v)
>   File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1506, in 
> _raiseerror
> raise err
> ParseError: not well-formed (invalid token): line 44, column 71
>
> Is there an XML or configuration file which needs to be changed for this to 
> work? Not sure how to proceed.
>
> Thanks!
> Miuki
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] odose

[Peter Cock]

You might find this work useful for fetching bacteria from the NCBI:

https://github.com/kblin/ncbi-genome-download

Peter

On Tue, Oct 25, 2016 at 12:54 PM, Vos, Michiel  wrote:
>
>
> Dear Galaxy developers,
>
>
>
> We hereby want to ask if anyone is interested in helping out to repair and
> update our Galaxy tool ‘odose’:
>
>
>
> Webserver
>
>
>
> Publication
>
>
>
> Public server page
>
>
>
> Github
>
>
>
> Odose enables users to perform an array of population genetics tests on
> prokaryote genomes. Our tool is currently linked to NCBI to download various
> files, but continuous restructuring at NCBI has resulted in broken links
> which prevent the pipeline from working. Developer Tim te Beek has moved on
> outside academia and does not have the time to fix it, but is available to
> give answer question/give instructions. We believe our tool is highly useful
> for microbiologists and would greatly appreciate any help from the Galaxy
> community to get it working properly again! Please get in touch if you would
> like to know more.
>
>
>
> Best Wishes, Michiel Vos, Mark van Passel and Tim te Beek
>
>
>
>
>
>
>
> Michiel Vos
>
> European Centre for Environment and Human Health
>
> University of Exeter
>
> ESI Building, Penryn Campus
>
> TR10 9FE Penryn, UK
>
> 0044 (0)1326259464
>
> https://coastalpathogens.wordpress.com/
>
>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Tool needs a particular file extension -- now Datatype help

[Peter Cock]

Hi Steve,

You are on the right track, but something in the WAV file has
triggered one of Galaxy's security protections to try to block
uploading of potentially dangerous files. There may be some
settings here you can relax - I've not had to deal with this
myself.

Peter

On Fri, Oct 21, 2016 at 8:55 PM, Steve Cassidy  wrote:
> Thanks all,
>   it seems that my real problem is that the audio file (.wav) is not being
> identified as a valid datatype and ending up as a zero length text file. So,
> I need to start to explore the world of datatypes.
>
> Following the docs
> (https://wiki.galaxyproject.org/Admin/Datatypes/Adding%20Datatypes) I can
> modify datatypes_conf.xml in my Galaxy sources and add a new datatype for
> wav files:
>
>  display_in_upload="true" mimetype="audio/wav" subclass="True”/>
>
> but, I get a message "The uploaded binary file contains inappropriate
> content” and a zero length file just as I did before adding this - although
> the datatype is now set to ‘wav’.
>
> I didn’t add a sniffer for this and set the datatype explicitly on upload.
>
> Also, this doesn’t seem like a modular way to add datatypes - how do I
> include datatypes in my tool definition?  I can see from some other tools
> that I include a datatypes_conf.xml in my tool folder.   When I try that and
> test with planemo the new type isn’t found.
>
> Pointers welcome.
>
> Thanks,
>
> Steve
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Tool needs a particular file extension

[Peter Cock]

Using a soft link for this is a common pattern, and should be followed with &&
(ideally using XML CDATA to avoid escaping everything like  etc),
and quote the filenames just in case there are any spaces. e.g.

https://github.com/galaxyproject/tools-iuc/blob/master/tools/trinity/run_de_analysis.xml#L16


For reference, in tools-iuc there are over 400 soft link examples:

$ grep "ln -s" tools/*/*.xml | wc -l
 446

Peter

On Fri, Oct 21, 2016 at 5:48 PM, Steve Cassidy  wrote:
> Hi,
>  I’m wrapping a tool that needs it’s input to have a known file extension
> (an audio file, eg. .wav).  Since Galaxy stores all data as .dat files the
> tool is falling over since it doesn’t know what .dat is.
>
> I thought I’d be able to get around this by hard linking the .dat file to
> the same name with a .wav extension (dataset_1.dat.wav), this works when I
> try it with the tool on the command line but within Galaxy it fails, here’s
> my :
>
> ln $signal ${signal}.wav 
> /home/maus/maus OUTFORMAT=TextGrid LANGUAGE=$language
> BPF=$bpf INSKANTEXTGRID=$inskantextgrid
> INSORTTEXTGRID=$insorttextgrid
> MODUS=$modus MAUSSHIFT=$mausshift MINPAUSLEN=$minpauslen
> WEIGHT=$weight
> INSPROB=$insprob NOINITIALFINALSILENCE=$noinitialfinalsilence
> OUTSYMBOL=$outsymbol
> OUT=$output SIGNAL=${signal}.wav
>
> resulting in the job command line:
>
> ln /tmp/tmp7AZvx7/files/000/dataset_2.dat
> /tmp/tmp7AZvx7/files/000/dataset_2.dat.wav & /home/maus/maus
> OUTFORMAT=TextGrid LANGUAGE=aus BPF=/tmp/tmp7AZvx7/files/000/dataset_1.dat
> INSKANTEXTGRID=false INSORTTEXTGRID=false MODUS=standard MAUSSHIFT=10
> MINPAUSLEN=5 WEIGHT=7.0 INSPROB=0.0 NOINITIALFINALSILENCE=no OUTSYMBOL=sampa
> OUT=/tmp/tmp7AZvx7/files/000/dataset_3.dat
> SIGNAL=/tmp/tmp7AZvx7/files/000/dataset_2.dat.wav
>
> I’m getting an error message from the tool:
>
> sox FAIL formats: can't open input file
> `/tmp/tmp7AZvx7/files/000/dataset_2.dat.wav': WAVE: RIFF header not found
>
> this suggests that the hard link didn’t get made.  I tried copying the file
> instead but got the same result.
>
> I could go in and patch the tool script to be more forgiving but it would be
> good to find a solution that didn’t require that if possible.
>
> Any pointers appreciated.
>
> Steve
> —
> Department of Computing, Macquarie University
> http://web.science.mq.edu.au/~cassidy
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] galaxy - prokka - tbl2asn

[Peter Cock]

This is an annoyance from the NCBI team, and dealing with the
expired tbl2asn tool is a regular question from Prokka users, eg:

https://github.com/tseemann/prokka/issues/96
https://github.com/tseemann/prokka/issues/139

The Tool Shed package for Prokka will need updating - but
luckily Nicola is already working on that:

https://github.com/galaxyproject/tools-iuc/pull/985

I'm not sure if this will work, but you can try manually installing
the latest tbl2asn from the NCBI (and if need be, copy the binary
on top of the expired binary from the Tool Shed - messy!):

ftp://ftp.ncbi.nih.gov/toolbox/ncbi_tools/converters/by_program/tbl2asn/

Peter

On Tue, Oct 18, 2016 at 7:19 PM, Fernandez Edgar
 wrote:
> Hey Guys,
>
>
>
> I’ve fixed this problem by bypassing the system perl and compiling my own
> with all the right modules.
>
> So I don’t get any more perl errors.
>
>
>
> Also, I’ve realized that the latest prokka package (Revision: f5e44aad6498)
> has a dependency with the package_tbl2_asn_24_3 (Revision 41764d6a6a3c).
>
>
>
> However, when you execute prokka, tbl2asn gives you the following error:
>
> [tbl2asn] This copy of tbl2asn is more than a year old.  Please download the
> current version.
>
>
>
> What would you suggest I do?
>
>
>
> Edgar Fernandez
>
> System Administrator (Linux)
>
> Direction Générale des Technologies de l'Information et de la Communication
>
> Université de Montréal
>
>
>
> PAVILLON ROGER-GAUDRY, bureau X-210
>
> (  Bur. : 1-514-343-6111 poste 16568
>
>
>
>
>
> From: Nicola Soranzo [mailto:nicola.sora...@gmail.com] On Behalf Of Nicola
> Soranzo
> Sent: October-07-16 3:14 PM
> To: Fernandez Edgar ; galaxy-...@bx.psu.edu
> Subject: Re: [galaxy-dev] galaxy - bioperl - prokka
>
>
>
> Hi Edgar,
> presently the Prokka wrapper requires some dependencies (in particular some
> Perl modules) to be installed separetely, see the README file at
> https://toolshed.g2.bx.psu.edu/view/crs4/prokka/
>
> We are planning to release an updated Prokka wrapper which will use Conda
> dependencies, you can track the progress at:
>
> https://github.com/galaxyproject/tools-iuc/pull/985
>
> Cheers,
> Nicola
>
> On 07/10/16 15:06, Fernandez Edgar wrote:
>
> Hello,
>
>
>
> I have installed prokka, perl and bioperl in my Galaxy’s ToolShed.
>
> However, my users are getting this error during execution of prokka:
>
> Can't locate Bio/Root/Version.pm in @INC (@INC contains:
> /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl
> /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at
> /home/galaxy/galaxy-prod/tool-data/toolshed.dependency.dir/prokka/1.11/crs4/prokka/f5e44aad6498/bin/prokka
> line 29. BEGIN failed--compilation aborted at
> /home/galaxy/galaxy-prod/tool-data/toolshed.dependency.dir/prokka/1.11/crs4/prokka/f5e44aad6498/bin/prokka
> line 29.
>
>
>
> I was under the impression that the galaxy’s tools will use the internal
> perl.
>
> Do I just need to install Bio::Root:Version on my system installation of
> perl?
>
> Please give me whatever information you can on this issue.
>
>
>
> Regards,
>
>
>
> Edgar Fernandez
>
> System Administrator (Linux)
>
> Direction Générale des Technologies de l'Information et de la Communication
>
> Université de Montréal
>
>
>
> PAVILLON ROGER-GAUDRY, bureau X-210
>
> (  Bur. : 1-514-343-6111 poste 16568
>
>
>
>
>
>
>
>
> ___
>
> Please keep all replies on the list by using "reply all"
>
> in your mail client.  To manage your subscriptions to this
>
> and other Galaxy lists, please use the interface at:
>
>   https://lists.galaxyproject.org/
>
>
>
> To search Galaxy mailing lists use the unified search at:
>
>   http://galaxyproject.org/search/mailinglists/
>
>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] planemo test with

[Peter Cock]

Good, because you are using planemo test --galaxy_root ...
you can preset the *.loc files as needed (because currently
Planemo will not do this for you).

See also Nicola's reply.

Peter

On Fri, Oct 7, 2016 at 5:49 PM, D K <danielforti...@gmail.com> wrote:
> Hi Peter,
>
> This is the command I'm using to run planemo:
> . galaxy_test/.venv/bin/activate
> planemo test --test_data /home/galaxy-functional-testdata --galaxy_root
> galaxy-test/ --skip_venv test.xml
>
> I'm getting no errors when using the planemo lint tool.
>
> Thanks!
>
> On Fri, Oct 7, 2016 at 2:06 AM, Peter Cock <p.j.a.c...@googlemail.com>
> wrote:
>>
>> Hi DK,
>>
>> Looking at your XML that looks OK.
>>
>> It might not be important, but how exactly are you running planemo
>> test?
>>
>> I'm asking because currently it will not create tool-data/twobit.loc
>> for you from your sample file - although in that case I would have
>> expected a different error message:
>>
>> https://github.com/galaxyproject/planemo/issues/530
>> https://github.com/galaxyproject/planemo/issues/530
>>
>> Is planemo lint happy with your tool?
>>
>> Peter
>>
>>
>> On Thu, Oct 6, 2016 at 11:08 PM, D K <danielforti...@gmail.com> wrote:
>>>
>>> Hi galaxy-dev,
>>>
>>> I'm having a problem running a test using planemo where I would like the
>>> value of a parameter taken from one of the data tables. I get the following
>>> error in planemo:
>>>
>>> 'Error creating a job for these tool inputs - Parameter
>>> refGenomeSource_type requires a value, but has no legal values defined.\n
>>>
>>>
>>> From my script XML:
>>>>
>>>>  
>>>>   
>>>>   
>>>> 
>>>> ...
>>>>   
>>>>   
>>>> 
>>>>  
>>>>   
>>>>
>>> From my "tool_data_table_conf.xml.sample" and
>>> "tool_data_table_conf.xml.test" (mirrored)
>>>>
>>>> >>> allow_duplicate_entries="False">
>>>> value, name, path
>>>> 
>>>> 
>>>
>>>
>>> and from twobit.loc (where the columns are tab separated):
>>>>
>>>> hsapiensH. sapiens (hg38)
>>>> /remote/RMS/users/galaxy/reference_genomes/MFEprimer_index/Homo_sapiens.GRCh38.cdna.all.fa
>>>
>>>
>>> Any suggestions would be greatly appreciated!
>>>
>>>
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>   https://lists.galaxyproject.org/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>   http://galaxyproject.org/search/mailinglists/
>>
>>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] planemo test with

[Peter Cock]

Hi DK,

Looking at your XML that looks OK.

It might not be important, but how exactly are you running planemo
test?

I'm asking because currently it will not create tool-data/twobit.loc
for you from your sample file - although in that case I would have
expected a different error message:

https://github.com/galaxyproject/planemo/issues/530
https://github.com/galaxyproject/planemo/issues/530

Is planemo lint happy with your tool?

Peter


On Thu, Oct 6, 2016 at 11:08 PM, D K  wrote:

> Hi galaxy-dev,
>
> I'm having a problem running a test using planemo where I would like the
> value of a parameter taken from one of the data tables. I get the following
> error in planemo:
>
> 'Error creating a job for these tool inputs - Parameter refGenomeSource_type 
> requires a value, but has no legal values defined.\n
>
>
> From my script XML:
>
>>  
>>   
>>   
>> 
>> ...
>>   
>>   
>> 
>>  
>>   
>>
>> From my "tool_data_table_conf.xml.sample" and "tool_data_table_conf.xml.test"
> (mirrored)
>
>> 
>> value, name, path
>> 
>> 
>
>
> and from twobit.loc (where the columns are tab separated):
>
>> hsapiensH. sapiens (hg38)   /remote/RMS/users/galaxy/
>> reference_genomes/MFEprimer_index/Homo_sapiens.GRCh38.cdna.all.fa
>
>
> Any suggestions would be greatly appreciated!
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] [galaxy-iuc] GMAP tool_dependencies.xml package for ToolShed

[Peter Cock]

Hi again JJ,

Thanks again for agreeing that we (the IUC) can adopt your
GMAP wrappers under the MIT license.

Apologies for the extra question, but are you happy for the
GMAP datatypes originally defined in your ToolShed repository
https://toolshed.g2.bx.psu.edu/view/jjohnson/gmap/ to be
moved into the Galaxy core, under Galaxy's open source
licence: Academic Free License version 3.0
https://github.com/galaxyproject/galaxy/blob/dev/LICENSE.txt

Thanks,

Peter

On Tue, Sep 20, 2016 at 2:51 PM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
> Thanks JJ,
>
> That's great. We're hoping to use GMAP locally, so I have an incentive
> to tackle this and the dependency glitch with the current versions on
> the Tool Shed.
>
> Cheers,
>
> Peter
>
> On Tue, Sep 20, 2016 at 2:50 PM, Jim Johnson <johns...@umn.edu> wrote:
>> I would greatly appreciate the IUC adopting all gmap/gsnap work that I
>> originally worked on.
>> I do not currently have the time to maintain these tools.
>>
>> Administrative access granted to IUC for:
>>
>> https://toolshed.g2.bx.psu.edu/view/jjohnson/gmap/
>>
>> https://toolshed.g2.bx.psu.edu/view/jjohnson/package_gmap_2013_05_09/
>> https://testtoolshed.g2.bx.psu.edu/view/jjohnson/gmap/
>>
>> https://testtoolshed.g2.bx.psu.edu/view/jjohnson/package_gmap_2013_05_09/
>>
>> These can be copied to the iuc owned repositories and I could deprecate
>> those under my name.
>>
>>
>>
>> On 9/20/16 8:13 AM, Peter Cock wrote:
>>>
>>> Thanks JJ :)
>>>
>>> Could you explicitly confirm that we (the IUC) have your permission
>>> to adopt the gmap wrappers and packages (and any of your other
>>> Galaxy work you might want to suggest), to develop and maintain
>>> under the IUC banner, under the MIT license.
>>>
>>> (Ideally I'd suggest you state this in an email CC'ing the galaxy-dev
>>> list as a public record)
>>>
>>> For now this will mean tracking their development in:
>>> https://github.com/galaxyproject/tools-iuc
>>>
>>> Secondly, would you be willing to grant admin/write access
>>> to the IUC account on the (main and test) tool shed for these:
>>>
>>> https://toolshed.g2.bx.psu.edu/view/jjohnson/gmap/
>>> https://toolshed.g2.bx.psu.edu/view/jjohnson/package_gmap_2013_05_09/
>>>
>>> Thanks,
>>>
>>> Peter
>>>
>>> On Tue, Sep 20, 2016 at 12:47 PM, Jim Johnson <johns...@umn.edu> wrote:
>>>>
>>>> It would be great to move this to IUC.
>>>> I originally put a lot of effort into this for a researcher at MN; I
>>>> think
>>>> it was tried once.
>>>> I now use gmap as part a dependency of the deFuse application.
>>>> Thanks,
>>>> JJ
>>
>>
>>
>> --
>> James E. Johnson Minnesota Supercomputing Institute University of Minnesota
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] [galaxy-iuc] GMAP tool_dependencies.xml package for ToolShed

[Peter Cock]

Thanks JJ,

That's great. We're hoping to use GMAP locally, so I have an incentive
to tackle this and the dependency glitch with the current versions on
the Tool Shed.

Cheers,

Peter

On Tue, Sep 20, 2016 at 2:50 PM, Jim Johnson <johns...@umn.edu> wrote:
> I would greatly appreciate the IUC adopting all gmap/gsnap work that I
> originally worked on.
> I do not currently have the time to maintain these tools.
>
> Administrative access granted to IUC for:
>
> https://toolshed.g2.bx.psu.edu/view/jjohnson/gmap/
>
> https://toolshed.g2.bx.psu.edu/view/jjohnson/package_gmap_2013_05_09/
> https://testtoolshed.g2.bx.psu.edu/view/jjohnson/gmap/
>
> https://testtoolshed.g2.bx.psu.edu/view/jjohnson/package_gmap_2013_05_09/
>
> These can be copied to the iuc owned repositories and I could deprecate
> those under my name.
>
>
>
> On 9/20/16 8:13 AM, Peter Cock wrote:
>>
>> Thanks JJ :)
>>
>> Could you explicitly confirm that we (the IUC) have your permission
>> to adopt the gmap wrappers and packages (and any of your other
>> Galaxy work you might want to suggest), to develop and maintain
>> under the IUC banner, under the MIT license.
>>
>> (Ideally I'd suggest you state this in an email CC'ing the galaxy-dev
>> list as a public record)
>>
>> For now this will mean tracking their development in:
>> https://github.com/galaxyproject/tools-iuc
>>
>> Secondly, would you be willing to grant admin/write access
>> to the IUC account on the (main and test) tool shed for these:
>>
>> https://toolshed.g2.bx.psu.edu/view/jjohnson/gmap/
>> https://toolshed.g2.bx.psu.edu/view/jjohnson/package_gmap_2013_05_09/
>>
>> Thanks,
>>
>> Peter
>>
>> On Tue, Sep 20, 2016 at 12:47 PM, Jim Johnson <johns...@umn.edu> wrote:
>>>
>>> It would be great to move this to IUC.
>>> I originally put a lot of effort into this for a researcher at MN; I
>>> think
>>> it was tried once.
>>> I now use gmap as part a dependency of the deFuse application.
>>> Thanks,
>>> JJ
>
>
>
> --
> James E. Johnson Minnesota Supercomputing Institute University of Minnesota
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Send repeat parameter to code file for multiple select list

[Peter Cock]

Ah - I understand what you mean now. Using the code tag is
discouraged, but you may find some of these examples useful
for what you are trying to do:

https://github.com/galaxyproject/galaxy/issues/2714

I hope that helps,

Peter



On Thu, Sep 1, 2016 at 2:37 PM, Katherine Beaulieu
<katherine.beaulieu...@gmail.com> wrote:
>
>
> On Thu, Sep 1, 2016 at 9:36 AM, Katherine Beaulieu
> <katherine.beaulieu...@gmail.com> wrote:
>>
>> Hi Peter,
>> The code file tag is generally used in populating select lists. So
>> basically you have a function from a code file that is loaded before runtime
>> of the tool and you can dynamically load content based on that function and
>> you can pass it parameters from your tool but I'm wondering how you would
>> pass it the contents of a repeat parameter.
>> This is how the code file tag works:
>> https://wiki.galaxyproject.org/Admin/Tools/Custom%20Code
>> Thanks,
>> Katherine
>>
>> On Thu, Sep 1, 2016 at 8:23 AM, Peter Cock <p.j.a.c...@googlemail.com>
>> wrote:
>>>
>>> I'm not quite sure what you mean by a "code file", but if you
>>> mean using the  tag, here's an example:
>>>
>>>
>>> https://github.com/peterjc/galaxy_mira/blob/master/tools/mira4_0/mira4_de_novo.xml
>>>
>>> I am using a Cheetah syntax for loop to access each entry
>>> in a repeat parameter.
>>>
>>> Peter
>>>
>>> On Thu, Sep 1, 2016 at 12:58 PM, Katherine Beaulieu
>>> <katherine.beaulieu...@gmail.com> wrote:
>>> > Hello Galaxy Community,
>>> > I was wondering if anyone had ever come across a situation where they
>>> > have
>>> > had to pass the contents of a repeat parameter to a code file in a
>>> > Galaxy
>>> > tool config? Is there any way to do this?
>>> > Katherine
>>> >
>>> > ___
>>> > Please keep all replies on the list by using "reply all"
>>> > in your mail client.  To manage your subscriptions to this
>>> > and other Galaxy lists, please use the interface at:
>>> >   https://lists.galaxyproject.org/
>>> >
>>> > To search Galaxy mailing lists use the unified search at:
>>> >   http://galaxyproject.org/search/mailinglists/
>>
>>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Send repeat parameter to code file for multiple select list

[Peter Cock]

I'm not quite sure what you mean by a "code file", but if you
mean using the  tag, here's an example:

https://github.com/peterjc/galaxy_mira/blob/master/tools/mira4_0/mira4_de_novo.xml

I am using a Cheetah syntax for loop to access each entry
in a repeat parameter.

Peter

On Thu, Sep 1, 2016 at 12:58 PM, Katherine Beaulieu
 wrote:
> Hello Galaxy Community,
> I was wondering if anyone had ever come across a situation where they have
> had to pass the contents of a repeat parameter to a code file in a Galaxy
> tool config? Is there any way to do this?
> Katherine
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] how to create users through Galaxy API?

[Peter Cock]

Thanks Nicola & Peter B.,

I was hoping we could script new-user creation like this, and
that someone else had already done this with the Galaxy API
(and Bioblend). This looks great!

Thank you both,

Peter C.

On Tue, Aug 30, 2016 at 10:43 AM, Peter Briggs
<peter.bri...@manchester.ac.uk> wrote:
> Hello Peter
>
> Following on from Nicola's suggestion: a while ago I had a go at writing
> some command line tools using Bioblend to try and do some of my Galaxy admin
> tasks via the API, including creating new user accounts so I'd also
> recommend this route as much much easier than attempting to interact with
> the API directly.
>
> The tools I wrote are here if you'd like to look at some an example of using
> Bioblend for this, to get you started:
>
> https://github.com/pjbriggs/nebulizer/blob/master/nebulizer/users.py
>
> As long as the user that the API key belongs to is an admin user (or if
> you're using a master API key) this works fine.
>
> HTH
>
> Best wishes
>
> Peter
>
>
> On 29/08/16 17:37, Nicola Soranzo wrote:
>>
>> Hi Peter,
>> why not use BioBlend?
>>
>>
>> https://bioblend.readthedocs.io/en/latest/api_docs/galaxy/all.html#module-bioblend.galaxy.users
>>
>> Cheers,
>> Nicola
>>
>>  Peter Cock ha scritto 
>>
>>> Thanks Hans-Rudolf,
>>>
>>> Yes - you are right. The wiki does link there but it wasn't obvious to
>>> me.
>>> Reading the UserAPIController link source code suggests as long as
>>> I am authenticated as a Galaxy Admin, I could use this to create new
>>> user accounts:
>>>
>>>
>>> https://docs.galaxyproject.org/en/master/_modules/galaxy/webapps/galaxy/api/users.html#UserAPIController.create
>>>
>>> Thank looks very promising...
>>>
>>> Peter
>>>
>>> On Mon, Aug 29, 2016 at 4:36 PM, Hans-Rudolf Hotz <h...@fmi.ch> wrote:
>>>>
>>>> Hi Peter
>>>>
>>>> I guess it is here:
>>>>
>>>> https://docs.galaxyproject.org/en/master/api_doc.html
>>>>
>>>>
>>>> Regards,  Hans-Rudolf
>>>>
>>>> On 08/29/2016 05:27 PM, Peter Cock wrote:
>>>>>
>>>>>
>>>>> Hi Martin,
>>>>>
>>>>> I'd like to look into creating new users via the API (where we can
>>>>> control the email address and username format for integration with our
>>>>> cluster), and have the web-interface forbid users from
>>>>> self-registration.
>>>>>
>>>>> The read-the-docs site you mentioned in this old email (below) doesn't
>>>>> exist anymore, and the API wiki page doesn't seem to link to something
>>>>> similar:
>>>>>
>>>>> https://wiki.galaxyproject.org/Admin/API
>>>>>
>>>>> Where is the official API documentation now?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Peter
>>>>>
>>>>> On Mon, Dec 30, 2013 at 4:34 AM, Martin Čech <mar...@bx.psu.edu
>>>>> <mailto:mar...@bx.psu.edu>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> you specify username, password and email in the body (payload) of
>>>>> the POST as Key:Value pairs.
>>>>>
>>>>> Code from the API method: (
>>>>>
>>>>>
>>>>> https://galaxy-central.readthedocs.org/en/latest/_modules/galaxy/webapps/galaxy/api/users.html#UserAPIController.create
>>>>>
>>>>>
>>>>> <https://galaxy-central.readthedocs.org/en/latest/_modules/galaxy/webapps/galaxy/api/users.html#UserAPIController.create>
>>>>> )
>>>>>
>>>>>
>>>>>
>>>>> username=payload['username']email=payload['email']password=payload['password']
>>>>>
>>>>> There are also other conditions that need to be fulfilled (e.g.
>>>>> user
>>>>> creation has to be turned on in the configuration) - you will find
>>>>> these when you look at the source code of the method (because the
>>>>> documentation is not perfect yet, sorry).
>>>>>
>>>>> M.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ___
>>>>> Please keep all replies on the list

Re: [galaxy-dev] Introducing Conda as a new standard for Galaxy tool dependencies

[Peter Cock]

That's great Bjoern,

I was thinking we should mention this on the wiki (with a link
to the documentation rather than duplicating it). There are
some existing pages about the tool_dependencies.xml
system - but no obvious top level introduction that I saw?

https://wiki.galaxyproject.org/ToolDependencyRecipes
https://wiki.galaxyproject.org/ToolShed/PackageRecipes
https://wiki.galaxyproject.org/ToolDependenciesTagSets
https://wiki.galaxyproject.org/ToolsWithDependenciesInSeparateRepositories

Where on the wiki do people think would be best?

Peter

On Sun, Aug 28, 2016 at 9:41 PM, Léo Biscassi  wrote:
> Nice job! I was really anxious for some documentation about this theme!
> Thanks all involved!
>
> On Fri, Aug 26, 2016 at 3:38 PM Björn Grüning 
> wrote:
>>
>> Hi all,
>>
>> Galaxy 16.07 was just released:
>> https://docs.galaxyproject.org/en/master/releases/16.07_announce.html
>> and the IUC has some news to share with you.
>>
>> For a more readable version please see:
>> https://docs.galaxyproject.org/en/master/admin/conda_faq.html
>>
>> 
>>
>> Galaxy tools (also called wrappers) traditionally use Tool Shed package
>> recipes to install their dependencies. At the tool’s installation time
>> the recipe is downloaded and executed in order to provide the underlying
>> software executables. Introduction of these Galaxy-specific recipes was
>> a necessary step at the time, however nowadays there are other more
>> mature and stable options to install software in a similar manner. The
>> Galaxy community has taken steps to improve the tool dependency system
>> in order to enable new features and expand its reach. This document aims
>> to describe these and answer the FAQ.
>>
>> It is a pleasure to announce the adoption of a new standard for tool
>> dependencies in Galaxy which has been integrated over the last six
>> months: Conda packages!
>> Not only do Conda packages make tool dependencies more reliable and
>> stable, they are also easier to test and faster to develop than the
>> traditional Tool Shed package recipes .
>>
>> Conda is a package manager like apt-get, yum, pip, brew or guix. We
>> don’t want to argue about the relative merits of various package
>> managers here, in fact Galaxy supports multiple package managers and we
>> welcome community contributions (such as implementing a Guix package
>> manager or enhancing the existing brew support to bring it on par with
>> Conda).
>>
>> As a community, we have decided that Conda is the one that best fulfills
>> our needs. The following are some of the crucial Conda features that led
>> to this decision:
>>
>> * Installation of packages does not require root privileges
>> (Installation at any location  the Galaxy user has write access to)
>> * Multiple versions of software can be installed in parallel
>> * HPC-ready
>> * Faster and more robust package installations through pre-compiled
>> packages (no build environment complications)
>> * Independent of programming language (R, Perl, Python, Julia, Java,
>> pre-compiled binaries, and more)
>> * Easy to write recipes (1 YAML description file + 1 Bash install script)
>> * An active, large and growing community (with more and more software
>> authors managing their own recipes)
>> * Extensive documentation: Conda documentation
>> (http://conda.pydata.org/docs/building/build.html) and Conda quick-start
>> (http://conda.pydata.org/docs/get-started.html)
>>
>> Below we answer some common questions (collected by Lance Parsons):
>>
>> 1. How do I enable Conda dependency resolution for existing Galaxy
>> installations?
>> ---
>>
>> Most Galaxy administrators have not set up a dependency resolvers
>> configuration file (dependency_resolvers_conf.xml) which means they are
>> using Galaxy’s default (dependency_resolvers_conf.xml.sample). Galaxy
>> has enabled Conda dependency resolution by default since release 16.04
>> (if Conda was installed already), so many existing installations can use
>> Conda dependencies. Having Conda enabled in
>> dependency_resolvers_conf.xml means that Galaxy will look for
>> dependencies using the Conda system when it attempts to run tools. If
>> conda_auto_install is True, and a dependency is not found, Galaxy will
>> attempt to install it using the configured Conda channels. A graphical
>> user interface that allows administrators install Conda packages
>> directly from Galaxy when tools are installed from the Tool Shed is
>> available in the 16.07 release of Galaxy.
>>
>>
>> 2. How do Conda dependencies work? Where do things get installed?
>> -
>> In contrast to the old dependency system, which was used exclusively by
>> Galaxy, Conda is a pre-existing, independent project. With Conda it is
>> possible for an admin to install packages without touching Galaxy at all
>> – managing your dependencies 

Re: [galaxy-dev] Workflow Compatibility

[Peter Cock]

Hi Katherine,

Are you asking about compatibility staying on the same Galaxy
instance, or the harder problem of compatibility sharing workflows
between Galaxy servers?

Taking data from input Galaxy datasets should be fine, anything else
may not be portable. Even the relatively simple case of using
datafiles referenced via an example.loc file (e.g. BLAST databases)
would require the entries in the example.loc file be synchronised
between Galaxy instances, and the associated data files too.

Peter


On Fri, Aug 26, 2016 at 1:27 PM, Katherine Beaulieu
 wrote:
> Hi Everyone,
> Would anyone be able to tell me the conditions which would make a tool
> non-workflow compatible? I have a tool that imports files from a third party
> application and auto-detects the file format. There is also the option to
> upload multiple files at once so the tool always uploads at least two files.
> From what I have described can anyone see why this tool would not be able to
> send one of its files to the next tool in the chain, ex. a text manipulation
> tool?
> Thanks,
> Katherine
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Doubled arguments from loc files in tests

[Peter Cock]

Hello all,

Recently I've been debugging various problems with my Galaxy tool TravisCI
tests as part of moving to using "planemo test" rather than the fragile system
I first created which mimicked a pre-ToolShed manual install:

https://blastedbio.blogspot.co.uk/2013/09/using-travis-ci-for-testing-galaxy-tools.html

Progress:

https://travis-ci.org/peterjc/galaxy_mira - passing now :)
https://travis-ci.org/peterjc/pico_galaxy - probably a loc file problem
https://travis-ci.org/peterjc/galaxy_blast - different loc file problem

This email is about the problems testing the BLAST wrappers where I
am using a BLAST database defined in test-data/*.loc files.

(I wasted some time today trying moving/copying the loc files and
databases to different places before I noticed the issue below)

Problem lines quoted are from this run, galaxy master or dev branch:
https://travis-ci.org/peterjc/galaxy_blast/builds/150072050
as of this commit:
https://github.com/peterjc/galaxy_blast/commit/03f81e7fb1e0562b46b8b2a2079c94f04c1f3adf

Some of the command expect a single BLAST database from the
test-data/*.loc files, and the test should do that, but we get:

blastdbcmd ... -db path,path ...
rpsblast ... -db path,path ...
rpstblastn ... -db path,path ...

Here we should get ... -db path (once only), not repeated with a
comma, e.g.

galaxy.jobs.command_factory INFO 2016-08-05 14:55:08,543 Built script
[/tmp/tmp9NURC9/job_working_directory/000/58/tool_script.sh] for tool
command[blastdbcmd -version >
/tmp/tmp9NURC9/tmp/GALAXY_VERSION_STRING_58 2>&1; blastdbcmd -dbtype
prot -db 
"/home/travis/build/peterjc/galaxy_blast/test-data/four_human_proteins.fasta,/home/travis/build/peterjc/galaxy_blast/galaxy-dev/test-data/four_human_proteins.fasta"
-info -out "/tmp/tmp9NURC9/files/000/dataset_58.dat"]

For other tools you can select multiple databases as input, and
the blastn tests try that, e.g. a working test gave this (where the
space is deliberate as that's how the NCBI handle multiple DB
arguments):

blastn ... -db "path1 path2"

galaxy.jobs.command_factory INFO 2016-08-05 14:58:01,783 Built script
[/tmp/tmp9NURC9/job_working_directory/000/76/tool_script.sh] for tool
command[blastn -version > /tmp/tmp9NURC9/tmp/GALAXY_VERSION_STRING_76
2>&1; blastn -query "/tmp/tmp9NURC9/files/000/dataset_75.dat" -db
"/home/travis/build/peterjc/galaxy_blast/test-data/three_human_mRNA.fasta
/home/travis/build/peterjc/galaxy_blast/galaxy-dev/test-data/three_human_mRNA.fasta"
-task megablast -evalue 0.001 -out
"/tmp/tmp9NURC9/files/000/dataset_76.dat" -outfmt 6 -num_threads
"${GALAXY_SLOTS:-8}"]

But when a single database is given we get a repeated path:

blastn ... -db "path path"

galaxy.jobs.command_factory INFO 2016-08-05 14:58:22,901 Built script
[/tmp/tmp9NURC9/job_working_directory/000/78/tool_script.sh] for tool
command[blastn -version > /tmp/tmp9NURC9/tmp/GALAXY_VERSION_STRING_78
2>&1; blastn -query "/tmp/tmp9NURC9/files/000/dataset_77.dat" -db
"/home/travis/build/peterjc/galaxy_blast/test-data/rhodopsin_nucs.fasta
/home/travis/build/peterjc/galaxy_blast/galaxy-dev/test-data/rhodopsin_nucs.fasta"
-task megablast -evalue 0.001 -out
"/tmp/tmp9NURC9/files/000/dataset_78.dat" -outfmt 6 -num_threads
"${GALAXY_SLOTS:-8}"]

Any thoughts on where this is breaking? In planemo?

These tests are working locally via "planemo test" (older
version of planemo, older version of Galaxy, Python 2.6).

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Adding packages to depot.galaxyproject.org

[Peter Cock]

On Mon, Aug 8, 2016 at 3:27 PM, Björn Grüning <bjoern.gruen...@gmail.com> wrote:
> Hi Peter,
>
> Am 08.08.2016 um 15:54 schrieb Peter Cock:
>> On Mon, Aug 8, 2016 at 2:41 PM, Eric Rasche <e...@tamu.edu> wrote:
>>> Hi Peter,
>>>
>>> The cargo port is the ideal place for this, however it does not support
>>> manual uploads of files.
>>>
>>
>> Thanks Eric,
>>
>> You mean https://github.com/galaxyproject/cargo-port
>>
>> Is that something which might be addressed in future? The use case
>> is when a previously public file stops working (either temporarily or
>> permanently), so you can't use the official URL to make the cached
>> copy?
>>
>> Would it be OK to use a short lived URL like a personal dropbox URL,
>> and then once cached, update urls.tsv with the original official
>> (but broken) URL?
>
>
> Yes this is totally fine. We just need to have a way to verify and check
> tarballs before upload and we thought github and PR are a nice fit for this.
>
> Thanks,
> Bjoern

Let's try this then:

https://github.com/galaxyproject/cargo-port/pull/82

(I can't find the Mac binaries for MIRA 4.9.5, hopefully I might have
that on my home machine but it's a long shot.)

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/