Re: [galaxy-dev] contribute to tools without github

2017-10-20 Thread Peter Cock
I infer you are talking about this Tool Shed repository:

https://toolshed.g2.bx.psu.edu/view/fabad/sparql_uniprot/

Sadly there is no README file and no copyright statements.
The only clue to the author is one comment is written appears
to be written in Spanish.

Based on https://toolshed.g2.bx.psu.edu/view/fabad/ this is
their first and only Tool Shed release.

I have sent "fabad" a message via the Tool Shed (you need
to be logged into a Tool Shed account to do this) pointing
them at this thread.

Regards,

Peter

On Fri, Oct 20, 2017 at 2:34 PM, Matthias Bernt  wrote:
> I noticed that the sparql_uniprot tool does not run (python3
> incompatibility). Since there is no github page I do not know how to
> contribute to such a tool. I have a similar problem with the lefse tool.
>
> What would be the best way to fix sich tools?
>
> Best,
> Matthias
>
>
> --
>
> ---
> Matthias Bernt
> Bioinformatics Service
> Molekulare Systembiologie (MOLSYB)
> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
> Helmholtz Centre for Environmental Research GmbH - UFZ
> Permoserstraße 15, 04318 Leipzig, Germany
> Phone +49 341 235 482296,
> m.be...@ufz.de, www.ufz.de
>
> Sitz der Gesellschaft/Registered Office: Leipzig
> Registergericht/Registration Office: Amtsgericht Leipzig
> Handelsregister Nr./Trade Register Nr.: B 4703
> Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig
> Wilfried Kraus
> Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
> Prof. Dr. Dr. h.c. Georg Teutsch
> Administrative Geschäftsführerin/ Administrative Managing Director:
> Prof. Dr. Heike Graßmann
> ---
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Problems with BlastDB.loc

2017-10-18 Thread Peter Cock
What does your config/tool_data_table_conf.xml[.sample] file have?

Can you turn on the debugging as this will will give more information
about the *.loc files. e.g.

https://github.com/galaxyproject/galaxy/pull/3095
https://github.com/galaxyproject/galaxy/pull/3099

Peter

On Wed, Oct 18, 2017 at 8:02 AM, Enders Matthias
 wrote:
> Dear Galaxy Dev List,
>
>
>
> we currently facing a problem with integrating new Blast Databases in our
> Galaxy instance.
>
>
>
> We added a the new DB to the file:
>
> /galaxy-central/tool-data/blastdb.loc
>
>
>
> But the new Database didn’t show up in the respective tools, this situation
> remains also after restarting the galaxy instance and reloading the tool
> config (blasn for example).
>
>
>
> As we included several databases before (which worked fine) we tried to find
> the error an excluded (out-commented) an existing Db, but this database
> stayed in the tools. So the blastdb.loc file seems to be ignored by galaxy.
>
> (This is consistent for multiple users and browser, so we are sure this is
> not a client-sided Browser-Cache issue)
>
>
>
> So my questions are: Is there any kind of internal Cache in Galaxy which
> could explain this behavior? How can we track down / solve the issue?
>
>
>
> Our Galaxy Version is 17.05
>
>
>
> Thanks a lot in advance!
>
>
>
> Mit freundlichen Grüßen
>
>
>
> i.A. Matthias Enders
>
> **
>
>
>
> NPZ Innovation GmbH
>
> Hohenlieth-Hof, D-24363 Holtsee
>
> Tel.+49-4351-736 189
>
> Fax +49-4351-736 271
>
> Mobil:   +49-151-14247360
>
> Mail:  m.end...@npz-innovation.de
>
> Web: www.npz-innovation.de
>
>
>
> Firmensitz Hohenlieth, Holtsee
>
> Amtsgericht Kiel, HRB 15342 Kl
>
> Geschäftsführer: Dr. Gunhild Leckband, Thorsten Pommer
>
>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] upload file format is data - type not recognized in galaxy

2017-09-22 Thread Peter Cock
That's a shame - I don't see anything in the log either at
first reading - it was hoping to see problems with missing
datatypes or sniffers, which might explain this.

Sorry, I am out of ideas.

Peter

On Fri, Sep 22, 2017 at 2:41 PM, Pavel Fibich  wrote:
> Hi Hans-Rudolf and Peter,
>
> thank you for replies. I tried both suggestions, but no luck.
>
> I made empty database, let galaxy run from it. Than, uploading fasta file
> again finished in the format 'data', not recognized again.
>
> Also, I did not find anything interesting in the startup logs ( logs are
> here: https://repeatexplorer-elixir.cerit-sc.cz/galstart2.log ), there are
> just fine information about datatypes, eg. for fasta:
>
> galaxy.datatypes.registry DEBUG 2017-09-22 15:11:52,810 Retrieved datatype
> module galaxy.datatypes.sequence:Fasta from the datatype registry.
> ...
> galaxy.datatypes.registry DEBUG 2017-09-22 15:11:52,929 Loaded sniffer for
> datatype 'galaxy.datatypes.sequence:Fasta'
>
> I have only config/datatypes_conf.xml.sample, so I expect that it is
> default.
>
> If you have any other ideas, I will be glad for them.
>
> Best, Pavel
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] upload file format is data - type not recognized in galaxy

2017-09-22 Thread Peter Cock
My first thought would be to double check the Galaxy startup logs
(which normally tell you about the datatypes as they are setup).
That ought to give some clues.

Also check the XML files where the datatypes are defined are
all still fine - do you have any of these and do they look fine?

datatypes_conf.xml
config/datatypes_conf.xml
config/datatypes_conf.xml.sample

(I know you said the settings and code were not changed,
but it seems worth double checking)

Peter

On Fri, Sep 22, 2017 at 12:31 PM, Hans-Rudolf Hotz  wrote:
> Hi Pavel
>
>
> what happens if you connect your galaxy instance to an empty PostgreSQL
> database, let galaxy build all the tables and then try to upload files.
>
>
> Regards, Hans-Rudolf
>
>
>
>
> On 09/22/2017 10:51 AM, Pavel Fibich wrote:
>>
>> Hello,
>>
>> do you have any idea why my galaxy 16.07 stop recognizing uploaded files
>> and format of uploaded files is set to the format 'data'?
>>
>> After using older version of database backup, galaxy stop recognizing
>> format of uploaded files (e.g. fasta, fastqc) and everything is set to
>> format 'data'. Uploaded files are OK, I have tested them on other public
>> galaxy servers, where they were recognized well. Before changing
>> database (after the technical problems with the server), recognizing of
>> the files were OK. Even in debugging level I do not see any errors or
>> problems in the log (I have compared logs before changing database with
>> new ones, and there are no differences). From the investigating of
>> upload.py, I guess something happen with galaxy.datatypes.registry
>> (settings and code of galaxy were not changed). Do you have any ideas? I
>> cannot switch the database back and changing file format in Edit
>> attributes is annoying.
>>
>> Best, Pavel
>>
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>http://galaxyproject.org/search/
>>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Displaying tool select list according to user

2017-09-21 Thread Peter Cock
On Thu, Sep 21, 2017 at 12:50 PM, SAPET, Frederic <
frederic.sa...@biogemma.com> wrote:

> Hi Peter,
>
> Good idea, I will keep that in mind for other needs.
> I'm not sure that it will be suitable for what I want to do. Users will
> have to first import database to their history (yet another step as they
> ask me for less, even the upload process is sometimes one step they would
> like to avoid !)


>
But it could be a way of achieving what I want, if Galaxy could get files
> (or .loc files) from a data library, within the tool form itself !


>
Maybe I will create an issue on the Galaxy git in order to discuss about
> that. I will be glad to help if something can be done.
>

Good idea! I wonder if the Galaxy UI team can think of a good way to
integrate this, which would make Shared Data library files available to any
tool without the import to current history step? Can you reply with a link
to the new GitHub issue for this?


> Maybe some explanations about the fact that my users ask me for "less
> steps".
> I really want Galaxy to be the main tool of our analysis platform.
> Right now I still maintain a plain old perl/php/CGI web interface. With a
> blast tool (among a lot of others tools). This proprietary Blast interface
> is really like you will find on NCBI or EBI sites. And you can quickly
> paste your sequences, pick up a database, a Blast algorythm and then
> launch. Few seconds later I have a 'htmlized' Blast report, with a nice
> picture.
> This use case (is this sequence a part of this database ?) is still faster
> on this plain old interface. And I afraid that I will have to keep it until
> they found Galaxy better than it...
>

We used to run wwwBLAST locally for similar reasons. If I was trying to
setup up something similar now, I'd look at http://www.sequenceserver.com/
for this - but making the Galaxy alternative easier to use would be neater.


> Maybe it could be another (or two) idea(s)  : the user can drag and drop
> his file within the tool form, Galaxy will handle the creation of this file
> within the history and then launch the tool. Or user can copy/paste a
> sequence within a text-area, and Galaxy will handle the creation of a file
> from these data within the history and then launch the tool.
> This skips upload process.
>

The Galaxy upload does now support drag and drop, so again in principle the
Galaxy team might be able to extend the tool interface as well. The tool
input's expected file formats could be used to guide the data-type sniffer
and/or short list the choices shown to the use.

As well or instead, I could imagine having an upload icon associated with
each tool input (again, creating a history item and selecting it for you)?
This could take advantage of the fact that each tool's input parameter
specifies the expected input file formats, so that might be mapped to known
file extensions for a file picker?


>
> thank you again for sharing your ideas, it helps!
>
> Fred
>

These are great usability ideas you are putting forward here - as a Galaxy
Tool Author they sound positive, but I cannot comment on how complicated it
would be for the Galaxy UI team. Or indeed if the UI would actually be
practical - or just overly complicate things which is always a risk.

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Displaying tool select list according to user

2017-09-20 Thread Peter Cock
Have you tried creating the BLAST databases as files in your history
(by running makeblastdb within Galaxy), and then importing them into
a shared data library (which can then be configured with role based
permissions)?

Peter


On Wed, Sep 20, 2017 at 9:39 AM, SAPET, Frederic
<frederic.sa...@biogemma.com> wrote:
> Hello Laure
>
>
>
> this idea cannot be applied in my use case.
>
> indeed I don’t want that users can choose which data to display. I want to
> hide some data based on their name/group/address (anything that can be
> useful to clasify my users).
>
>
>
> Thank you for sharing your ideas :)
>
>
>
> Fred
>
> De : Laure QUINTRIC [mailto:laure.quint...@ifremer.fr]
> Envoyé : mercredi 13 septembre 2017 13:54
> À : Peter Cock <p.j.a.c...@googlemail.com>; SAPET, Frederic
> <frederic.sa...@biogemma.com>
>
>
> Cc : Galaxy-dev <galaxy-dev@lists.galaxyproject.org>
> Objet : Re: [galaxy-dev] Displaying tool select list according to user
>
>
>
> A workaround would be to do this :
>
> - first, create a tool which will output a tabular loc file with all the
> custom banks for the connected user.
> - then add in the ncbi_blast wrapper a select list with the attribute
> from_dataset to load the loc file generated by the previous tool  : see the
> example here :
> https://docs.galaxyproject.org/en/latest/dev/schema.html#from-dataset
>
> I have something like this in the ncbi_blast tool:
> 
>  multiple="true" display="checkboxes" label="Select our custom db">
> 
> 
> 
> 
> 
>
> and I set in param userdb_loc_file the loc file which is in my history and
> looks like this :
> NCBI_ntNCBI_nt
> 2017-05-19/path/to/biobanks/n/NCBI_nt/current/NCBI_nt/NCBI_nt
>
> Laure
>
> Le 12/09/2017 à 17:25, Peter Cock a écrit :
>
> Thanks Fred,
>
>
>
> That's the previous discussion I was trying to find -
>
> sadly no clear solution but some ideas to explore.
>
>
>
> It does seem filtering example.loc by user or role
>
> is not a niche request. I wonder if this could be
>
> linked to the Galaxy data library permissions
>
> structure as a way to manage the access rights?
>
>
>
> Peter
>
>
>
> On Tue, Sep 12, 2017 at 4:20 PM, SAPET, Frederic
> <frederic.sa...@biogemma.com> wrote:
>
> Hi Laure,
>
>
>
> There are others users (including me! ) that need this kind of function :
>
> http://dev.list.galaxyproject.org/Blast-db-permission-td4670440.html
>
>
>
> @Galaxy team:
>
> Do we need to add an issue in GitHub ?
>
> Do you think that it could be enabled one day ?
>
>
>
> I think this something that could be nice to develop !
>
> I would be glad to help.
>
>
>
> Thank a lot
>
>
>
> Fred
>
>
>
> De : galaxy-dev [mailto:galaxy-dev-boun...@lists.galaxyproject.org] De la
> part de Peter Cock
> Envoyé : mardi 12 septembre 2017 14:56
> À : Laure QUINTRIC <laure.quint...@ifremer.fr>
> Cc : Galaxy-dev <galaxy-dev@lists.galaxyproject.org>
> Objet : Re: [galaxy-dev] Displaying tool select list according to user
>
>
>
> Hi Laure,
>
>
>
> I don't think there is any easy to use mechanism for this in Galaxy.
>
> Unfortunately Galaxy loads all the *.loc files at startup and treats
>
> them as global values available to all users equally.
>
>
>
> I can't find the email thread, but I recall someone previously trying
>
> to do something with a single large example.loc file for all users
>
> but with an extra column for filtering by account name. I forget if
>
> they got these changes to work or not.
>
>
>
> Assuming you only have a few separate versions of the example.loc
>
> file, you could create copies of the BLAST wrapper XML file which
>
> use the specific *.loc file - and then restrict the tool by user?
>
>
>
> In either case at a minimum you would have to maintain local
>
> changes to the BLAST wrappers (and potentially changes to
>
> Galaxy itself), which is not a good situation.
>
>
>
> Peter
>
>
>
> On Tue, Sep 12, 2017 at 1:41 PM, Laure QUINTRIC <laure.quint...@ifremer.fr>
> wrote:
>
> Hello Galaxy users,
>
> I try to figure out how I can display a select list into a galaxy tool
> according to the connected user.
>
> I have several .loc files containing the paths for custom blast databases I
> have created for several users (user1.loc, user2.loc, and so on). I would
> like the blast tool to display the specific databases for the connected user
> (user1 or

Re: [galaxy-dev] Displaying tool select list according to user

2017-09-12 Thread Peter Cock
Thanks Fred,

That's the previous discussion I was trying to find -
sadly no clear solution but some ideas to explore.

It does seem filtering example.loc by user or role
is not a niche request. I wonder if this could be
linked to the Galaxy data library permissions
structure as a way to manage the access rights?

Peter

On Tue, Sep 12, 2017 at 4:20 PM, SAPET, Frederic <
frederic.sa...@biogemma.com> wrote:

> Hi Laure,
>
>
>
> There are others users (including me! ) that need this kind of function :
>
> http://dev.list.galaxyproject.org/Blast-db-permission-td4670440.html
>
>
>
> @Galaxy team:
>
> Do we need to add an issue in GitHub ?
>
> Do you think that it could be enabled one day ?
>
>
>
> I think this something that could be nice to develop !
>
> I would be glad to help.
>
>
>
> Thank a lot
>
>
>
> Fred
>
>
>
> *De :* galaxy-dev [mailto:galaxy-dev-boun...@lists.galaxyproject.org] *De
> la part de* Peter Cock
> *Envoyé :* mardi 12 septembre 2017 14:56
> *À :* Laure QUINTRIC <laure.quint...@ifremer.fr>
> *Cc :* Galaxy-dev <galaxy-dev@lists.galaxyproject.org>
> *Objet :* Re: [galaxy-dev] Displaying tool select list according to user
>
>
>
> Hi Laure,
>
>
>
> I don't think there is any easy to use mechanism for this in Galaxy.
>
> Unfortunately Galaxy loads all the *.loc files at startup and treats
>
> them as global values available to all users equally.
>
>
>
> I can't find the email thread, but I recall someone previously trying
>
> to do something with a single large example.loc file for all users
>
> but with an extra column for filtering by account name. I forget if
>
> they got these changes to work or not.
>
>
>
> Assuming you only have a few separate versions of the example.loc
>
> file, you could create copies of the BLAST wrapper XML file which
>
> use the specific *.loc file - and then restrict the tool by user?
>
>
>
> In either case at a minimum you would have to maintain local
>
> changes to the BLAST wrappers (and potentially changes to
>
> Galaxy itself), which is not a good situation.
>
>
>
> Peter
>
>
>
> On Tue, Sep 12, 2017 at 1:41 PM, Laure QUINTRIC <laure.quint...@ifremer.fr>
> wrote:
>
> Hello Galaxy users,
>
> I try to figure out how I can display a select list into a galaxy tool
> according to the connected user.
>
> I have several .loc files containing the paths for custom blast databases
> I have created for several users (user1.loc, user2.loc, and so on). I would
> like the blast tool to display the specific databases for the connected
> user (user1 or user2).
>
> Any idea on how to proceed ?
> Regards,
> Laure
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/
>
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Displaying tool select list according to user

2017-09-12 Thread Peter Cock
Hi Laure,

I don't think there is any easy to use mechanism for this in Galaxy.
Unfortunately Galaxy loads all the *.loc files at startup and treats
them as global values available to all users equally.

I can't find the email thread, but I recall someone previously trying
to do something with a single large example.loc file for all users
but with an extra column for filtering by account name. I forget if
they got these changes to work or not.

Assuming you only have a few separate versions of the example.loc
file, you could create copies of the BLAST wrapper XML file which
use the specific *.loc file - and then restrict the tool by user?

In either case at a minimum you would have to maintain local
changes to the BLAST wrappers (and potentially changes to
Galaxy itself), which is not a good situation.

Peter

On Tue, Sep 12, 2017 at 1:41 PM, Laure QUINTRIC 
wrote:

> Hello Galaxy users,
>
> I try to figure out how I can display a select list into a galaxy tool
> according to the connected user.
>
> I have several .loc files containing the paths for custom blast databases
> I have created for several users (user1.loc, user2.loc, and so on). I would
> like the blast tool to display the specific databases for the connected
> user (user1 or user2).
>
> Any idea on how to proceed ?
> Regards,
> Laure
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Samtools problem in Docker

2017-08-14 Thread Peter Cock
Galaxy itself uses samtools for many operations, which is different
from when samtools is called as a dependency of a tool run by the
Galaxy user.

However, the Docker image ought to take care of that - can you
give more details of which Docker image you are using (URL,
version, etc)?

Thanks,

Peter


On Mon, Aug 14, 2017 at 5:08 PM, Deepak Tanwar  wrote:
> Hello everyone,
>
> I am facing problem in uploading bam files on Galaxy. I am using Docker
> image of Galaxy.
>
> I have installed samtools version 0.1.19 and added it to the path
> /tool_deps/samtools/0.1.19/iuc/package_samtools_0_1_19/c9bd782f5342/bin/samtools
>
> Error: Exception: Attempting to use functionality requiring samtools, but it
> cannot be located on Galaxy's PATH.
>
> Thank you,
> Deepak
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Data manager for ncbi_blast_plus in the main tool shed?

2017-08-11 Thread Peter Cock
Hi David,

Daniel Blankenberg wrote one (CC'd):

https://github.com/peterjc/galaxy_blast/tree/master/data_managers/ncbi_blastdb

http://testtoolshed.g2.bx.psu.edu/view/blankenberg/data_manager_example_blastdb_ncbi_update_blastdb

I've never used it in production and don't have enough understanding
of it and the Data Manager framework to understand what if anything
needs working on.

Locally we have mirrors of the NCBI blast databases (updated with cron),
local databases for organisms of interest - these are all listed in the
*.loc
files by hand.

Separately, our users can and do make their own databases within
Galaxy using makeblastdb.

Regards,

Peter

On Thu, Aug 10, 2017 at 10:32 PM, David Kovalic  wrote:

> Hi,
>
> Does anyone know if a data manager exists in the tool shed for creating
> and integrating NCBI BLAST+ indices within galaxy?
>
> I can't seem to find it in the tools shed or searching the internet  Surly
> such a tool must exist.
>
> Any info would be greatly appreciated.
>
> Thanks,
>
> David
>
>
>
> David Kovalic Ph.D.   President, Analome
> phone: +314.496.8808
> web: www.analome.com
> email: kova...@analome.com
> 
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Image in tool XML "help" section only renders when installed from toolshed?

2017-08-09 Thread Peter Cock
I'm puzzled now too, cross reference

https://github.com/galaxy-iuc/standards/pull/46

and the original:

https://github.com/galaxy-iuc/standards/issues/12

Peter


On Wed, Aug 9, 2017 at 3:01 PM, Peter Briggs
<peter.bri...@manchester.ac.uk> wrote:
> Hello Peter
>
> Thanks for the fast response. From reading around elsewhere I'd understood
> that using $PATH_TO_IMAGES was now deprecated? I think I tried it anyway and
> it didn't seem to help. But I might have missed something, so I'd be
> interested to know if it works for you.
>
> Best wishes
>
> Peter
>
>
> On 09/08/17 14:57, Peter Cock wrote:
>>
>> It looks like something important is not showing in the best practise
>> website, special variable $PATH_TO_IMAGES as per the linked example:
>>
>> .. image:: $PATH_TO_IMAGES/slop-glyph.png
>>
>> Let me look into this,
>>
>> Peter
>>
>> On Wed, Aug 9, 2017 at 2:52 PM, Peter Briggs
>> <peter.bri...@manchester.ac.uk> wrote:
>>>
>>> Dear devs
>>>
>>> I've been updating the  section for a locally developed tool and
>>> want
>>> to include some images, but I've had some issues trying to see them when
>>> testing locally.
>>>
>>> I've followed the instructions outlined here:
>>>
>>>
>>> https://github.com/galaxy-iuc/standards/blob/master/docs/best_practices/tool_xml.rst#including-images
>>>
>>> (i.e. put the images files into a 'static/images/' subdir of the tool
>>> directory and reference them using ".. image:: picture.png")
>>>
>>> However when I load the tool into a Galaxy using "planemo serve ..." the
>>> images fail to render: I just get a line of text with the image filename.
>>> The same thing happens when I add the tool manually on our local server
>>> (i.e. by adding a  reference to the tool XML in the
>>> appropriate conf.xml file).
>>>
>>> If I upload the same tool files to a toolshed and then install from
>>> there, I
>>> do see the images rendered correctly.
>>>
>>> I'm a bit baffled as to why there should be a difference. Is there an
>>> option
>>> or setting I've missed to turn on the image rendering for
>>> non-shed-installed
>>> tools? Or have I made some other mistake?
>>>
>>> Thanks for your help,
>>>
>>> Best wishes
>>>
>>> Peter
>>>
>>> --
>>> Peter Briggs peter.bri...@manchester.ac.uk
>>> Bioinformatics Core Facility University of Manchester
>>> B.1083 Michael Smith Bldg Tel: (0161) 2751482
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>   https://lists.galaxyproject.org/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>   http://galaxyproject.org/search/
>
>
> --
> Peter Briggs peter.bri...@manchester.ac.uk
> Bioinformatics Core Facility University of Manchester
> B.1083 Michael Smith Bldg Tel: (0161) 2751482
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Image in tool XML "help" section only renders when installed from toolshed?

2017-08-09 Thread Peter Cock
It looks like something important is not showing in the best practise
website, special variable $PATH_TO_IMAGES as per the linked example:

.. image:: $PATH_TO_IMAGES/slop-glyph.png

Let me look into this,

Peter

On Wed, Aug 9, 2017 at 2:52 PM, Peter Briggs
 wrote:
> Dear devs
>
> I've been updating the  section for a locally developed tool and want
> to include some images, but I've had some issues trying to see them when
> testing locally.
>
> I've followed the instructions outlined here:
>
> https://github.com/galaxy-iuc/standards/blob/master/docs/best_practices/tool_xml.rst#including-images
>
> (i.e. put the images files into a 'static/images/' subdir of the tool
> directory and reference them using ".. image:: picture.png")
>
> However when I load the tool into a Galaxy using "planemo serve ..." the
> images fail to render: I just get a line of text with the image filename.
> The same thing happens when I add the tool manually on our local server
> (i.e. by adding a  reference to the tool XML in the
> appropriate conf.xml file).
>
> If I upload the same tool files to a toolshed and then install from there, I
> do see the images rendered correctly.
>
> I'm a bit baffled as to why there should be a difference. Is there an option
> or setting I've missed to turn on the image rendering for non-shed-installed
> tools? Or have I made some other mistake?
>
> Thanks for your help,
>
> Best wishes
>
> Peter
>
> --
> Peter Briggs peter.bri...@manchester.ac.uk
> Bioinformatics Core Facility University of Manchester
> B.1083 Michael Smith Bldg Tel: (0161) 2751482
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Venn diagram example data

2017-07-27 Thread Peter Cock
How does the plugin expect cases where the sets have a different
number of entries to be presented? Simply blank cells in the tab
separated table?

Peter

On Thu, Jul 27, 2017 at 4:43 PM, Aysam Guerler <aysam.guer...@gmail.com> wrote:
> Sorry for the late response. So the venn diagram visualizes the overlap
> between sets. Each set is represented by entries in a column e.g.
>
> #SET1 SET2 SET3
> A GA
> B HB
> C AG
> D BH
> E CX
> F DZ
>
> I hope this helps.
>
> Thanks
> Sam
>
> On Thu, Jul 27, 2017 at 11:33 AM, Peter Cock <p.j.a.c...@googlemail.com>
> wrote:
>>
>> Hi Steven,
>>
>> Just in case, you're not talking about my older simpler Galaxy Tool
>> for drawing simple Venn Diagrams, are you?
>>
>> https://toolshed.g2.bx.psu.edu/view/peterjc/venn_list
>> https://github.com/peterjc/pico_galaxy/tree/master/tools/venn_list
>>
>> This reminds me that I ought to update that tool to work with
>> dependencies via conda...
>>
>> Peter
>>
>> On Mon, Jul 24, 2017 at 2:16 PM, Steven Shen <ishenwei...@gmail.com>
>> wrote:
>> > Dear Galaxy list,
>> >
>> > I am interested in Galaxy Charts visualization.But venn diagram confuses
>> > me
>> > when I upload my tabular data, it seems venn diagram need  one column
>> > observations at least, actually I don't clear what should be contained
>> > in
>> > the one column observations.
>> >
>> > So is there any venn diagram example data or usage for this charts
>> > plugin ?
>> > Sad to say I am not good at JavaScript :)
>> >
>> > Thank you very much.
>> >
>> > Regards,
>> > Steven
>> >
>> > ___
>> > Please keep all replies on the list by using "reply all"
>> > in your mail client.  To manage your subscriptions to this
>> > and other Galaxy lists, please use the interface at:
>> >   https://lists.galaxyproject.org/
>> >
>> > To search Galaxy mailing lists use the unified search at:
>> >   http://galaxyproject.org/search/
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Venn diagram example data

2017-07-27 Thread Peter Cock
Hi Steven,

Just in case, you're not talking about my older simpler Galaxy Tool
for drawing simple Venn Diagrams, are you?

https://toolshed.g2.bx.psu.edu/view/peterjc/venn_list
https://github.com/peterjc/pico_galaxy/tree/master/tools/venn_list

This reminds me that I ought to update that tool to work with
dependencies via conda...

Peter

On Mon, Jul 24, 2017 at 2:16 PM, Steven Shen  wrote:
> Dear Galaxy list,
>
> I am interested in Galaxy Charts visualization.But venn diagram confuses me
> when I upload my tabular data, it seems venn diagram need  one column
> observations at least, actually I don't clear what should be contained in
> the one column observations.
>
> So is there any venn diagram example data or usage for this charts plugin ?
> Sad to say I am not good at JavaScript :)
>
> Thank you very much.
>
> Regards,
> Steven
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] tool and planemo problems

2017-07-12 Thread Peter Cock
And mere minutes later that change has been accepted,
and will be part of the next release of Planemo.

Thanks for the constructive feedback Matthias!

Peter

On Wed, Jul 12, 2017 at 4:24 PM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
> Good idea, pull request submitted:
>
> https://github.com/galaxyproject/planemo/pull/709
>
> Peter
>
> On Wed, Jul 12, 2017 at 4:03 PM, Matthias Bernt <m.be...@ufz.de> wrote:
>> Perfect. Thanks a lot. Maybe one could add a note to planemo --help that
>> command specific help can be obtained with planemo command --help.
>>
>> Best,
>> Matthias
>>
>>
>> On 12.07.2017 16:57, Peter Cock wrote:
>>>
>>> Try:
>>>
>>> planemo lint --fail_level error sleepwaste.xml
>>>
>>> The default is to fail if there are warnings.
>>>
>>> Peter
>>>
>>>
>>> On Wed, Jul 12, 2017 at 3:55 PM, Matthias Bernt <m.be...@ufz.de> wrote:
>>>>
>>>> Hi Peter,
>>>>
>>>> I just call planemo l sleepwaste.xml
>>>>
>>>> Best,
>>>> Matthias
>>>>
>>>>
>>>> On 12.07.2017 16:54, Peter Cock wrote:
>>>>>
>>>>>
>>>>> How are you running planemo lint - in particular is it set
>>>>> to fail if there are any warnings present (like your missing
>>>>> citation warning)?
>>>>>
>>>>> Peter
>>>>>
>>>>> On Wed, Jul 12, 2017 at 3:52 PM, Matthias Bernt <m.be...@ufz.de> wrote:
>>>>>>
>>>>>>
>>>>>> Dear list,
>>>>>>
>>>>>> In order to test the job resubmission feature I wrote a little tool for
>>>>>> galaxy that just wastes a specified (minimum) time and memory (using
>>>>>> different programming languages) for testing the new DRMAA job runner
>>>>>> that I
>>>>>> wrote.
>>>>>>
>>>>>> When linting it with planemo seemingly all tests are successful, but in
>>>>>> the
>>>>>> end I still get a fail -- verbose mode is also of no help here.
>>>>>>
>>>>>> Applying linter tests... CHECK
>>>>>> .. CHECK: 7 test(s) found.
>>>>>> Applying linter output... CHECK
>>>>>> .. INFO: 1 outputs found.
>>>>>> Applying linter inputs... CHECK
>>>>>> .. INFO: Found 3 input parameters.
>>>>>> Applying linter help... CHECK
>>>>>> .. CHECK: Tool contains help section.
>>>>>> .. CHECK: Help contains valid reStructuredText.
>>>>>> Applying linter general... CHECK
>>>>>> .. CHECK: Tool defines a version [0.1.0].
>>>>>> .. CHECK: Tool defines a name [Waste time and memory].
>>>>>> .. CHECK: Tool defines an id [sleepwaste].
>>>>>> .. CHECK: Tool targets 16.01 Galaxy profile.
>>>>>> Applying linter command... CHECK
>>>>>> .. INFO: Tool contains a command.
>>>>>> Applying linter citations... WARNING
>>>>>> .. WARNING: No citations found, consider adding citations to your tool.
>>>>>> Applying linter tool_xsd... CHECK
>>>>>> .. INFO: File validates against XML schema.
>>>>>> Failed linting
>>>>>>
>>>>>> The test of the tool are all successful. Any ideas what is wrong?
>>>>>>
>>>>>> Once the problems are solved, would such a tool also be useful in the
>>>>>> toolshed?
>>>>>>
>>>>>> Cheers,
>>>>>> Matthias
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> ---
>>>>>> Matthias Bernt
>>>>>> Bioinformatics Service
>>>>>> Molekulare Systembiologie (MOLSYB)
>>>>>> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
>>>>>> Helmholtz Centre for Environmental Research GmbH - UFZ
>>>>>> Permoserstraße 15, 04318 Leipzig, Germany
>>>>>> Phone +49 341 235 482296,
>>>>>> m.be...@ufz.de, www.ufz.de
>>>>>>
>>>>>> Sitz der Gesellschaft/Registered Office: Leipzig
>>>>>> Registergericht/Registration Office: Amtsgericht Leipzig
>>>>>>

Re: [galaxy-dev] tool and planemo problems

2017-07-12 Thread Peter Cock
Good idea, pull request submitted:

https://github.com/galaxyproject/planemo/pull/709

Peter

On Wed, Jul 12, 2017 at 4:03 PM, Matthias Bernt <m.be...@ufz.de> wrote:
> Perfect. Thanks a lot. Maybe one could add a note to planemo --help that
> command specific help can be obtained with planemo command --help.
>
> Best,
> Matthias
>
>
> On 12.07.2017 16:57, Peter Cock wrote:
>>
>> Try:
>>
>> planemo lint --fail_level error sleepwaste.xml
>>
>> The default is to fail if there are warnings.
>>
>> Peter
>>
>>
>> On Wed, Jul 12, 2017 at 3:55 PM, Matthias Bernt <m.be...@ufz.de> wrote:
>>>
>>> Hi Peter,
>>>
>>> I just call planemo l sleepwaste.xml
>>>
>>> Best,
>>> Matthias
>>>
>>>
>>> On 12.07.2017 16:54, Peter Cock wrote:
>>>>
>>>>
>>>> How are you running planemo lint - in particular is it set
>>>> to fail if there are any warnings present (like your missing
>>>> citation warning)?
>>>>
>>>> Peter
>>>>
>>>> On Wed, Jul 12, 2017 at 3:52 PM, Matthias Bernt <m.be...@ufz.de> wrote:
>>>>>
>>>>>
>>>>> Dear list,
>>>>>
>>>>> In order to test the job resubmission feature I wrote a little tool for
>>>>> galaxy that just wastes a specified (minimum) time and memory (using
>>>>> different programming languages) for testing the new DRMAA job runner
>>>>> that I
>>>>> wrote.
>>>>>
>>>>> When linting it with planemo seemingly all tests are successful, but in
>>>>> the
>>>>> end I still get a fail -- verbose mode is also of no help here.
>>>>>
>>>>> Applying linter tests... CHECK
>>>>> .. CHECK: 7 test(s) found.
>>>>> Applying linter output... CHECK
>>>>> .. INFO: 1 outputs found.
>>>>> Applying linter inputs... CHECK
>>>>> .. INFO: Found 3 input parameters.
>>>>> Applying linter help... CHECK
>>>>> .. CHECK: Tool contains help section.
>>>>> .. CHECK: Help contains valid reStructuredText.
>>>>> Applying linter general... CHECK
>>>>> .. CHECK: Tool defines a version [0.1.0].
>>>>> .. CHECK: Tool defines a name [Waste time and memory].
>>>>> .. CHECK: Tool defines an id [sleepwaste].
>>>>> .. CHECK: Tool targets 16.01 Galaxy profile.
>>>>> Applying linter command... CHECK
>>>>> .. INFO: Tool contains a command.
>>>>> Applying linter citations... WARNING
>>>>> .. WARNING: No citations found, consider adding citations to your tool.
>>>>> Applying linter tool_xsd... CHECK
>>>>> .. INFO: File validates against XML schema.
>>>>> Failed linting
>>>>>
>>>>> The test of the tool are all successful. Any ideas what is wrong?
>>>>>
>>>>> Once the problems are solved, would such a tool also be useful in the
>>>>> toolshed?
>>>>>
>>>>> Cheers,
>>>>> Matthias
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> ---
>>>>> Matthias Bernt
>>>>> Bioinformatics Service
>>>>> Molekulare Systembiologie (MOLSYB)
>>>>> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
>>>>> Helmholtz Centre for Environmental Research GmbH - UFZ
>>>>> Permoserstraße 15, 04318 Leipzig, Germany
>>>>> Phone +49 341 235 482296,
>>>>> m.be...@ufz.de, www.ufz.de
>>>>>
>>>>> Sitz der Gesellschaft/Registered Office: Leipzig
>>>>> Registergericht/Registration Office: Amtsgericht Leipzig
>>>>> Handelsregister Nr./Trade Register Nr.: B 4703
>>>>> Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board:
>>>>> MinDirig
>>>>> Wilfried Kraus
>>>>> Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
>>>>> Prof. Dr. Dr. h.c. Georg Teutsch
>>>>> Administrative Geschäftsführerin/ Administrative Managing Director:
>>>>> Prof. Dr. Heike Graßmann
>>>>> ---
>>>>>
>>>>> ___
>>>>> Please k

Re: [galaxy-dev] tool and planemo problems

2017-07-12 Thread Peter Cock
Try:

planemo lint --fail_level error sleepwaste.xml

The default is to fail if there are warnings.

Peter


On Wed, Jul 12, 2017 at 3:55 PM, Matthias Bernt <m.be...@ufz.de> wrote:
> Hi Peter,
>
> I just call planemo l sleepwaste.xml
>
> Best,
> Matthias
>
>
> On 12.07.2017 16:54, Peter Cock wrote:
>>
>> How are you running planemo lint - in particular is it set
>> to fail if there are any warnings present (like your missing
>> citation warning)?
>>
>> Peter
>>
>> On Wed, Jul 12, 2017 at 3:52 PM, Matthias Bernt <m.be...@ufz.de> wrote:
>>>
>>> Dear list,
>>>
>>> In order to test the job resubmission feature I wrote a little tool for
>>> galaxy that just wastes a specified (minimum) time and memory (using
>>> different programming languages) for testing the new DRMAA job runner
>>> that I
>>> wrote.
>>>
>>> When linting it with planemo seemingly all tests are successful, but in
>>> the
>>> end I still get a fail -- verbose mode is also of no help here.
>>>
>>> Applying linter tests... CHECK
>>> .. CHECK: 7 test(s) found.
>>> Applying linter output... CHECK
>>> .. INFO: 1 outputs found.
>>> Applying linter inputs... CHECK
>>> .. INFO: Found 3 input parameters.
>>> Applying linter help... CHECK
>>> .. CHECK: Tool contains help section.
>>> .. CHECK: Help contains valid reStructuredText.
>>> Applying linter general... CHECK
>>> .. CHECK: Tool defines a version [0.1.0].
>>> .. CHECK: Tool defines a name [Waste time and memory].
>>> .. CHECK: Tool defines an id [sleepwaste].
>>> .. CHECK: Tool targets 16.01 Galaxy profile.
>>> Applying linter command... CHECK
>>> .. INFO: Tool contains a command.
>>> Applying linter citations... WARNING
>>> .. WARNING: No citations found, consider adding citations to your tool.
>>> Applying linter tool_xsd... CHECK
>>> .. INFO: File validates against XML schema.
>>> Failed linting
>>>
>>> The test of the tool are all successful. Any ideas what is wrong?
>>>
>>> Once the problems are solved, would such a tool also be useful in the
>>> toolshed?
>>>
>>> Cheers,
>>> Matthias
>>>
>>>
>>>
>>> --
>>>
>>> ---
>>> Matthias Bernt
>>> Bioinformatics Service
>>> Molekulare Systembiologie (MOLSYB)
>>> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
>>> Helmholtz Centre for Environmental Research GmbH - UFZ
>>> Permoserstraße 15, 04318 Leipzig, Germany
>>> Phone +49 341 235 482296,
>>> m.be...@ufz.de, www.ufz.de
>>>
>>> Sitz der Gesellschaft/Registered Office: Leipzig
>>> Registergericht/Registration Office: Amtsgericht Leipzig
>>> Handelsregister Nr./Trade Register Nr.: B 4703
>>> Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board:
>>> MinDirig
>>> Wilfried Kraus
>>> Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
>>> Prof. Dr. Dr. h.c. Georg Teutsch
>>> Administrative Geschäftsführerin/ Administrative Managing Director:
>>> Prof. Dr. Heike Graßmann
>>> ---
>>>
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>https://lists.galaxyproject.org/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>http://galaxyproject.org/search/
>
>
> --
>
> ---
> Matthias Bernt
> Bioinformatics Service
> Molekulare Systembiologie (MOLSYB)
> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
> Helmholtz Centre for Environmental Research GmbH - UFZ
> Permoserstraße 15, 04318 Leipzig, Germany
> Phone +49 341 235 482296,
> m.be...@ufz.de, www.ufz.de
>
> Sitz der Gesellschaft/Registered Office: Leipzig
> Registergericht/Registration Office: Amtsgericht Leipzig
> Handelsregister Nr./Trade Register Nr.: B 4703
> Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig
> Wilfried Kraus
> Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
> Prof. Dr. Dr. h.c. Georg Teutsch
> Administrative Geschäftsführerin/ Administrative Managing Director:
> Prof. Dr. Heike Graßmann
> ---
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] tool and planemo problems

2017-07-12 Thread Peter Cock
How are you running planemo lint - in particular is it set
to fail if there are any warnings present (like your missing
citation warning)?

Peter

On Wed, Jul 12, 2017 at 3:52 PM, Matthias Bernt  wrote:
> Dear list,
>
> In order to test the job resubmission feature I wrote a little tool for
> galaxy that just wastes a specified (minimum) time and memory (using
> different programming languages) for testing the new DRMAA job runner that I
> wrote.
>
> When linting it with planemo seemingly all tests are successful, but in the
> end I still get a fail -- verbose mode is also of no help here.
>
> Applying linter tests... CHECK
> .. CHECK: 7 test(s) found.
> Applying linter output... CHECK
> .. INFO: 1 outputs found.
> Applying linter inputs... CHECK
> .. INFO: Found 3 input parameters.
> Applying linter help... CHECK
> .. CHECK: Tool contains help section.
> .. CHECK: Help contains valid reStructuredText.
> Applying linter general... CHECK
> .. CHECK: Tool defines a version [0.1.0].
> .. CHECK: Tool defines a name [Waste time and memory].
> .. CHECK: Tool defines an id [sleepwaste].
> .. CHECK: Tool targets 16.01 Galaxy profile.
> Applying linter command... CHECK
> .. INFO: Tool contains a command.
> Applying linter citations... WARNING
> .. WARNING: No citations found, consider adding citations to your tool.
> Applying linter tool_xsd... CHECK
> .. INFO: File validates against XML schema.
> Failed linting
>
> The test of the tool are all successful. Any ideas what is wrong?
>
> Once the problems are solved, would such a tool also be useful in the
> toolshed?
>
> Cheers,
> Matthias
>
>
>
> --
>
> ---
> Matthias Bernt
> Bioinformatics Service
> Molekulare Systembiologie (MOLSYB)
> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
> Helmholtz Centre for Environmental Research GmbH - UFZ
> Permoserstraße 15, 04318 Leipzig, Germany
> Phone +49 341 235 482296,
> m.be...@ufz.de, www.ufz.de
>
> Sitz der Gesellschaft/Registered Office: Leipzig
> Registergericht/Registration Office: Amtsgericht Leipzig
> Handelsregister Nr./Trade Register Nr.: B 4703
> Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig
> Wilfried Kraus
> Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
> Prof. Dr. Dr. h.c. Georg Teutsch
> Administrative Geschäftsführerin/ Administrative Managing Director:
> Prof. Dr. Heike Graßmann
> ---
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Issue with job_name, UnicodeDecodeError in drmaa

2017-05-29 Thread Peter Cock
Thanks, pull request submitted:

https://github.com/galaxyproject/galaxy/pull/4121

Peter


On Mon, May 29, 2017 at 5:23 PM, Saravanaraj Ayyampalayam <ra...@uga.edu> wrote:
> Peter,
>
> Here is the git diff output that shows the changes I made:
>
> diff --git a/lib/galaxy/jobs/runners/__init__.py 
> b/lib/galaxy/jobs/runners/__init__.py
> index ebf4859..6ee51c8 100644
> --- a/lib/galaxy/jobs/runners/__init__.py
> +++ b/lib/galaxy/jobs/runners/__init__.py
> @@ -419,7 +419,7 @@ class JobState( object ):
>  job_name += '_%s' % self.job_wrapper.tool.old_id
>  if self.job_wrapper.user:
>  job_name += '_%s' % self.job_wrapper.user
> -self.job_name = ''.join( map( lambda x: x if x in ( 
> string.letters + string.digits + '_' ) else '_', job_name ) )
> +self.job_name = ''.join( map( lambda x: x if x in ( 
> string.ascii_letters + string.digits + '_' ) else '_', job_name ) )
>
>  @staticmethod
>  def default_job_file( files_dir, id_tag ):
> diff --git a/lib/galaxy/jobs/runners/drmaa.py 
> b/lib/galaxy/jobs/runners/drmaa.py
> index 7c08984..5c5dc62 100644
> --- a/lib/galaxy/jobs/runners/drmaa.py
> +++ b/lib/galaxy/jobs/runners/drmaa.py
> @@ -394,7 +394,7 @@ class DRMAAJobRunner( AsynchronousJobRunner ):
>  job_name += '_%s' % job_wrapper.tool.old_id
>  if external_runjob_script is None:
>  job_name += '_%s' % job_wrapper.user
> -job_name = ''.join( x if x in ( string.letters + string.digits + '_' 
> ) else '_' for x in job_name )
> +job_name = ''.join( x if x in ( string.ascii_letters + string.digits 
> + '_' ) else '_' for x in job_name )
>  if self.restrict_job_name_length:
>  job_name = job_name[:self.restrict_job_name_length]
>  return job_name
>
> -Raj
>
>> On May 29, 2017, at 11:33 AM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
>>
>> Thanks Saravanaraj,
>>
>> You should not need to change the locale, after all many Galaxy servers and
>> their users will be labelling their files in non-English with
>> "special" characters -
>> but perhaps en_US.UTF-8 would be safer in your case as it seems that few
>> people use Galaxy with en_US.iso885915 otherwise this would have been
>> reported before?
>>
>> Can you tell us both places you changed string.letters to 
>> string.ascii_letters
>> please? I could then submit this as a pull request to get Galaxy itself 
>> updated
>> for future release.
>>
>> Peter
>>
>> On Mon, May 29, 2017 at 4:28 PM, Saravanaraj Ayyampalayam <ra...@uga.edu> 
>> wrote:
>>> Peter,
>>>
>>> You are correct. Changing string.letters to string.ascii_letters fixed it. 
>>> I had to change it in two places.
>>>
>>> My locale is :
>>>
>>> LANG=en_US.iso885915
>>> LC_CTYPE="en_US.iso885915"
>>> LC_NUMERIC="en_US.iso885915"
>>> LC_TIME="en_US.iso885915"
>>> LC_COLLATE="en_US.iso885915"
>>> LC_MONETARY="en_US.iso885915"
>>> LC_MESSAGES="en_US.iso885915"
>>> LC_PAPER="en_US.iso885915"
>>> LC_NAME="en_US.iso885915"
>>> LC_ADDRESS="en_US.iso885915"
>>> LC_TELEPHONE="en_US.iso885915"
>>> LC_MEASUREMENT="en_US.iso885915"
>>> LC_IDENTIFICATION="en_US.iso885915"
>>> LC_ALL=
>>>
>>> I will talk to our sysadmin and see if I can change the locale to 
>>> en_US.UTF-8.
>>>
>>> Thank you very much.
>>> -Raj
>>>
>>>
>>>> On May 29, 2017, at 10:58 AM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
>>>>
>>>> Curious, my new guess is this is position 52 a temporary string from
>>>> this expression:
>>>>
>>>> string.letters + string.digits + '_'
>>>>
>>>> This would fit with it being something unusual about the locale of
>>>> your Galaxy server, and my suggested hack may work - replace
>>>> string.letters with string.ascii_letters
>>>>
>>>> Separately at the Linux command line of the server running Galaxy, and
>>>> on a cluster node, what does the command "locale" give? e.g. I get:
>>>>
>>>> $ locale
>>>> LANG=en_US.UTF-8
>>>> LC_CTYPE="en_US.UTF-8"
>>>> LC_NUMERIC="en_US.UTF-8"
>>>> LC_TIME="en_US.UTF-8"
>>>> LC_COLLATE="en_US.UTF-8&qu

Re: [galaxy-dev] Issue with job_name, UnicodeDecodeError in drmaa

2017-05-29 Thread Peter Cock
Thanks Saravanaraj,

You should not need to change the locale, after all many Galaxy servers and
their users will be labelling their files in non-English with
"special" characters -
but perhaps en_US.UTF-8 would be safer in your case as it seems that few
people use Galaxy with en_US.iso885915 otherwise this would have been
reported before?

Can you tell us both places you changed string.letters to string.ascii_letters
please? I could then submit this as a pull request to get Galaxy itself updated
for future release.

Peter

On Mon, May 29, 2017 at 4:28 PM, Saravanaraj Ayyampalayam <ra...@uga.edu> wrote:
> Peter,
>
> You are correct. Changing string.letters to string.ascii_letters fixed it. I 
> had to change it in two places.
>
> My locale is :
>
> LANG=en_US.iso885915
> LC_CTYPE="en_US.iso885915"
> LC_NUMERIC="en_US.iso885915"
> LC_TIME="en_US.iso885915"
> LC_COLLATE="en_US.iso885915"
> LC_MONETARY="en_US.iso885915"
> LC_MESSAGES="en_US.iso885915"
> LC_PAPER="en_US.iso885915"
> LC_NAME="en_US.iso885915"
> LC_ADDRESS="en_US.iso885915"
> LC_TELEPHONE="en_US.iso885915"
> LC_MEASUREMENT="en_US.iso885915"
> LC_IDENTIFICATION="en_US.iso885915"
> LC_ALL=
>
> I will talk to our sysadmin and see if I can change the locale to en_US.UTF-8.
>
> Thank you very much.
> -Raj
>
>
>> On May 29, 2017, at 10:58 AM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
>>
>> Curious, my new guess is this is position 52 a temporary string from
>> this expression:
>>
>> string.letters + string.digits + '_'
>>
>> This would fit with it being something unusual about the locale of
>> your Galaxy server, and my suggested hack may work - replace
>> string.letters with string.ascii_letters
>>
>> Separately at the Linux command line of the server running Galaxy, and
>> on a cluster node, what does the command "locale" give? e.g. I get:
>>
>> $ locale
>> LANG=en_US.UTF-8
>> LC_CTYPE="en_US.UTF-8"
>> LC_NUMERIC="en_US.UTF-8"
>> LC_TIME="en_US.UTF-8"
>> LC_COLLATE="en_US.UTF-8"
>> LC_MONETARY="en_US.UTF-8"
>> LC_MESSAGES="en_US.UTF-8"
>> LC_PAPER="en_US.UTF-8"
>> LC_NAME="en_US.UTF-8"
>> LC_ADDRESS="en_US.UTF-8"
>> LC_TELEPHONE="en_US.UTF-8"
>> LC_MEASUREMENT="en_US.UTF-8"
>> LC_IDENTIFICATION="en_US.UTF-8"
>> LC_ALL=
>>
>>
>> Peter
>>
>> On Mon, May 29, 2017 at 3:43 PM, Saravanaraj Ayyampalayam <ra...@uga.edu> 
>> wrote:
>>> Peter,
>>>
>>> The job name looks OK. Here is the output of the debug statements:
>>>
>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,423 Making job_name, 
>>> tag was '15132', plan to use u'g15132_fastq_to_fasta_python_ra...@uga.edu' 
>>> with underscores replacements
>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,423 Making job_name, 
>>> character 0 in proposed job_name u'g' is 103
>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,423 Making job_name, 
>>> character 1 in proposed job_name u'1' is 49
>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,423 Making job_name, 
>>> character 2 in proposed job_name u'5' is 53
>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,424 Making job_name, 
>>> character 3 in proposed job_name u'1' is 49
>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,424 Making job_name, 
>>> character 4 in proposed job_name u'3' is 51
>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,424 Making job_name, 
>>> character 5 in proposed job_name u'2' is 50
>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,424 Making job_name, 
>>> character 6 in proposed job_name u'_' is 95
>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,424 Making job_name, 
>>> character 7 in proposed job_name u'f' is 102
>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,425 Making job_name, 
>>> character 8 in proposed job_name u'a' is 97
>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,425 Making job_name, 
>>> character 9 in proposed job_name u's' is 115
>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,425 Making job_name, 
>>> character 10 in proposed job_name u't' is 116
>>> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,425 Making job_name, 
>>> character 11 in proposed job_name u'q' is 113
>>> galaxy.

Re: [galaxy-dev] Issue with job_name, UnicodeDecodeError in drmaa

2017-05-29 Thread Peter Cock
cter 28 in proposed job_name u'_' is 95
> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,428 Making job_name, 
> character 29 in proposed job_name u'r' is 114
> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,429 Making job_name, 
> character 30 in proposed job_name u'a' is 97
> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,429 Making job_name, 
> character 31 in proposed job_name u'j' is 106
> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,429 Making job_name, 
> character 32 in proposed job_name u'7' is 55
> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,429 Making job_name, 
> character 33 in proposed job_name u'6' is 54
> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,429 Making job_name, 
> character 34 in proposed job_name u'@' is 64
> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,430 Making job_name, 
> character 35 in proposed job_name u'u' is 117
> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,430 Making job_name, 
> character 36 in proposed job_name u'g' is 103
> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,430 Making job_name, 
> character 37 in proposed job_name u'a' is 97
> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,430 Making job_name, 
> character 38 in proposed job_name u'.' is 46
> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,430 Making job_name, 
> character 39 in proposed job_name u'e' is 101
> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,431 Making job_name, 
> character 40 in proposed job_name u'd' is 100
> galaxy.jobs.runners.drmaa DEBUG 2017-05-29 10:37:21,431 Making job_name, 
> character 41 in proposed job_name u'u' is 117
> galaxy.jobs.runners ERROR 2017-05-29 10:37:21,431 (15132) Unhandled exception 
> calling queue_job
> Traceback (most recent call last):
>   File 
> "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/lib/galaxy/jobs/runners/__init__.py",
>  line 104, in run_next
> method(arg)
>   File 
> "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py",
>  line 132, in queue_job
> job_name = self._job_name(job_wrapper)
>   File 
> "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py",
>  line 400, in _job_name
> job_name = ''.join( x if x in ( string.letters + string.digits + '_' ) 
> else '_' for x in job_name )
>   File 
> "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py",
>  line 400, in 
> job_name = ''.join( x if x in ( string.letters + string.digits + '_' ) 
> else '_' for x in job_name )
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xa6 in position 52: 
> ordinal not in range(128)
>
>
> There is no sign of the 0xa6 at position 52 (the job name is only 42 chars 
> long) in the job name.
>
> Thanks!
> -Raj
>
>
>> On May 29, 2017, at 5:50 AM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
>>
>> Time for more debugging, try inserting something like this inserted
>> the line above into
>> /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py
>> (having backed up the file):
>>
>> log.debug( 'Making job_name, tag was %r, plan to use %r with
>> underscores replacements' % (galaxy_id_tag, job_name))
>>
>> And if that doesn't help, check all the characters:
>>
>> for i, letter in enumerate(job_name): log.debug( 'Making job_name,
>> character %i in proposed job_name %r is %i' % (i, letter,
>> ord(letter)))
>>
>> My hunch is that the locale of the Galaxy server differs from that on
>> the cluster, meaning DRMAA is stricter about allowed characters
>> because Python's string.letters would be whatever is allowed on the
>> Galaxy server's locale.
>>
>> If I am right, a crude fix would be switching to using
>> string.ascii_letters, i.e.
>>
>> job_name = ''.join( x if x in ( string.ascii_letters + string.digits +
>> '_' ) else '_' for x in job_name )
>>
>> Peter
>>
>> On Mon, May 29, 2017 at 6:54 AM, Saravanaraj Ayyampalayam <ra...@uga.edu> 
>> wrote:
>>> Hello,
>>>
>>> I updated to the latest version (17.05) galaxy code. I was updating from a 
>>> very old version.
>>> I fixed all the issues with except one. I am getting the following error 
>>> when I submit a job.
>>>
>>> galaxy.jobs.runners ERROR 2017-05-29 01:42:44,008 (15131) Unhandled 
>>> exception calling queue_job
>>> Traceback (most recent call last):
>>>  File 
>>> "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/lib/galaxy/jobs/runners/__init__.

Re: [galaxy-dev] Issue with job_name, UnicodeDecodeError in drmaa

2017-05-29 Thread Peter Cock
Time for more debugging, try inserting something like this inserted
the line above into
/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py
(having backed up the file):

log.debug( 'Making job_name, tag was %r, plan to use %r with
underscores replacements' % (galaxy_id_tag, job_name))

And if that doesn't help, check all the characters:

for i, letter in enumerate(job_name): log.debug( 'Making job_name,
character %i in proposed job_name %r is %i' % (i, letter,
ord(letter)))

My hunch is that the locale of the Galaxy server differs from that on
the cluster, meaning DRMAA is stricter about allowed characters
because Python's string.letters would be whatever is allowed on the
Galaxy server's locale.

If I am right, a crude fix would be switching to using
string.ascii_letters, i.e.

job_name = ''.join( x if x in ( string.ascii_letters + string.digits +
'_' ) else '_' for x in job_name )

Peter

On Mon, May 29, 2017 at 6:54 AM, Saravanaraj Ayyampalayam  wrote:
> Hello,
>
> I updated to the latest version (17.05) galaxy code. I was updating from a 
> very old version.
> I fixed all the issues with except one. I am getting the following error when 
> I submit a job.
>
> galaxy.jobs.runners ERROR 2017-05-29 01:42:44,008 (15131) Unhandled exception 
> calling queue_job
> Traceback (most recent call last):
>   File 
> "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/lib/galaxy/jobs/runners/__init__.py",
>  line 104, in run_next
> method(arg)
>   File 
> "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py",
>  line 132, in queue_job
> job_name = self._job_name(job_wrapper)
>   File 
> "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py",
>  line 397, in _job_name
> job_name = ''.join( x if x in ( string.letters + string.digits + '_' ) 
> else '_' for x in job_name )
>   File 
> "/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py",
>  line 397, in 
> job_name = ''.join( x if x in ( string.letters + string.digits + '_' ) 
> else '_' for x in job_name )
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xa6 in position 52: 
> ordinal not in range(128)
>
> I spent a day trying to debug this and didn’t get anywhere. The job_name 
> looks OK.
>
> I would really appreciate some help from the community.
>
> Thanks!
> -Raj
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Running jobs as real user and extra_file_path

2017-04-28 Thread Peter Cock
On Wed, Mar 20, 2013 at 3:51 PM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
> On Mon, Apr 30, 2012 at 4:47 PM, Nate Coraor <n...@bx.psu.edu> wrote:
>> On Apr 25, 2012, at 9:39 AM, Louise-Amélie Schmitt wrote:
>>
>>> Hi everyone,
>>>
>>> I just wanted to ask how the extra_file_path is handled in case of job
>>> running as the real user since the file_path is only writable by the
>>> galaxy user. Any clue?
>>
>> Hi L-A,
>>
>> There are actually two dataset attributes for accessing the extra files 
>> directory,
>> 'files_path' and 'extra_files_path'.
>>
>> 'extra_files_path' always points to the real path under the directory 
>> specified
>> in the config file's 'file_path' option and should be used when providing a
>> dataset's extra files directory as the input to a tool.
>>
>> 'files_path' points at the location that should be written to when used in a
>> job so that the job's finish method can find the output files.  This will be
>> under the job_working_directory and thus writable by the actual user.  So
>> any tool outputs that use the extra files directory should use the output's
>> 'files_path' attribute.
>>
>> Sorry for this confusion.  There was talk some time ago about merging both
>> attributes to work correctly in all cases, but this was never done.
>>
>> --nate
>
> Is this still the case?
>
> I'm guessing this is what Raj was alluding to with the BLAST+ wrapper
> for makeblastdb which creates a composite datatype as its output:
> http://lists.bx.psu.edu/pipermail/galaxy-dev/2013-March/013887.html
>
> If I'm using a single Linux/Unix galaxy user this doens't matter does it?
>
> Thanks,
>
> Peter

Some years later, this question has been resurrected as
https://github.com/galaxy-iuc/standards/issues/40

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] atlas version 3.10.2 installation error

2017-04-19 Thread Peter Cock
Good news: With Marius' help Biopython 1.67 should be in bioconda
shortly, https://github.com/bioconda/bioconda-recipes/pull/4462

That means any Galaxy package using Biopython needing/wanting
to work under both the legacy XML packaging system and bioconda
will be able to point at Biopython 1.67.

The newer Biopython releases are only in bioconda (and right now,
I have no compelling reason to write new legacy XML packages for
them).

Likewise any Galaxy package using NCBI BLAST+ needing/wanting
to work under both the legacy XML packaging system and bioconda
will be able to point at BLAST+ 2.5.0.

Peter

On Wed, Apr 19, 2017 at 1:17 PM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
> I've emailed the rest of the IUC for their input - adding the old versions
> in bioconda should work, or adding the recent Biopython releases as packages
> on the Tool Shed.
>
> For blast_rbh just need a semi-recent Biopython available in both systems
> (until everyone moves over to bioconda - we've not yet done this on our
> local Galaxy for example).
>
> Peter
>
> Peter
>
> On Wed, 19 Apr 2017 at 13:04, Matthias Bernt <m.be...@ufz.de> wrote:
>>
>> Dear Peter,
>>
>> thanks. Just for understanding: Why not just change the biopython
>> dependency from 1.67 to 1.69, if this is the one available in bioconda.
>> blast_rbh seems to use only the SeqIO fasta parser and writer. This
>> should be no problem.
>>
>> But I guess that for older versions of blast_rbh one might still need
>> legacy packages of biopython. Or could one change the dependencies in
>> older versions? Or would this be no good idea because of backward
>> compatibility?
>>
>> Cheers,
>> Matthias
>>
>>
>>
>>
>>
>> On 19.04.2017 13:50, Peter Cock wrote:
>> > Just lucky timing, could you try blast_rbh v0.1.11,
>> >
>> > https://toolshed.g2.bx.psu.edu/view/peterjc/blast_rbh/d8d9a9069586
>> >
>> > For BLAST+ this ought to work with either bioconda, or the legacy XML
>> > based packages, as both have BLAST+ 2.5.0.
>> >
>> > However, it looks like bioconda only has Biopython 1.68 and 1.69, while
>> > the legacy XML based packages only go up to Biopython 1.67 at the time
>> > of writing... it looks like we'll need to add a couple more legacy
>> > packages.
>> >
>> > Peter
>> >
>> > On Wed, Apr 19, 2017 at 12:01 PM, Matthias Bernt <m.be...@ufz.de> wrote:
>> >> Dear Peter,
>> >>
>> >> thanks for the info. Would be great to get the update, but since I have
>> >> a
>> >> method that is working for the moment there is no need to hurry.
>> >>
>> >> Best,
>> >> Matthias
>> >>
>> >>
>> >>
>> >> On 19.04.2017 12:57, Peter Cock wrote:
>> >>>
>> >>> Oh right - I've just been updating ncbi_blast_plus this morning
>> >>> to transition to BLAST+ 2.5.0 via either bioconda or the older
>> >>> legacy Tool Shed.
>> >>>
>> >>> I can try to update blast_rbh next, which may solve this by
>> >>> letting you use bioconda.
>> >>>
>> >>> Peter
>> >>>
>> >>> On Wed, Apr 19, 2017 at 11:52 AM, Matthias Bernt <m.be...@ufz.de>
>> >>> wrote:
>> >>>>
>> >>>> Dear Marius,
>> >>>>
>> >>>> thanks again for the help. I'm trying to install blast_rbh (owner is
>> >>>> peterjc). The dependencies look as follows:
>> >>>>
>> >>>> blast_rbh
>> >>>> - biopython
>> >>>>   - package_biopython_1_64
>> >>>> - package_numpy_1_8
>> >>>>   - package_atlas_3_10
>> >>>> - blast
>> >>>> - package_blast_plus_2_2_31
>> >>>>   - blast+
>> >>>>
>> >>>> With unchecked "When available, install Tool Shed managed tool
>> >>>> dependencies?" I get all as "Installed, missing tool dependencies".
>> >>>> Blast_rbh is not working since blast+ is not installed. Even if I
>> >>>> install
>> >>>> blast+ manually (in galaxy) its not working (makeblastdb is missing).
>> >>>>
>> >>>> When I leave "When available, install Tool Shed managed tool
>> >>>> dependencies?"
>> >>>> I see blast_rbh, package_blast_plus package_biopython as insta

Re: [galaxy-dev] atlas version 3.10.2 installation error

2017-04-19 Thread Peter Cock
I've emailed the rest of the IUC for their input - adding the old versions
in bioconda should work, or adding the recent Biopython releases as
packages on the Tool Shed.

For blast_rbh just need a semi-recent Biopython available in both systems
(until everyone moves over to bioconda - we've not yet done this on our
local Galaxy for example).

Peter

Peter

On Wed, 19 Apr 2017 at 13:04, Matthias Bernt <m.be...@ufz.de> wrote:

> Dear Peter,
>
> thanks. Just for understanding: Why not just change the biopython
> dependency from 1.67 to 1.69, if this is the one available in bioconda.
> blast_rbh seems to use only the SeqIO fasta parser and writer. This
> should be no problem.
>
> But I guess that for older versions of blast_rbh one might still need
> legacy packages of biopython. Or could one change the dependencies in
> older versions? Or would this be no good idea because of backward
> compatibility?
>
> Cheers,
> Matthias
>
>
>
>
>
> On 19.04.2017 13:50, Peter Cock wrote:
> > Just lucky timing, could you try blast_rbh v0.1.11,
> >
> > https://toolshed.g2.bx.psu.edu/view/peterjc/blast_rbh/d8d9a9069586
> >
> > For BLAST+ this ought to work with either bioconda, or the legacy XML
> > based packages, as both have BLAST+ 2.5.0.
> >
> > However, it looks like bioconda only has Biopython 1.68 and 1.69, while
> > the legacy XML based packages only go up to Biopython 1.67 at the time
> > of writing... it looks like we'll need to add a couple more legacy
> packages.
> >
> > Peter
> >
> > On Wed, Apr 19, 2017 at 12:01 PM, Matthias Bernt <m.be...@ufz.de> wrote:
> >> Dear Peter,
> >>
> >> thanks for the info. Would be great to get the update, but since I have
> a
> >> method that is working for the moment there is no need to hurry.
> >>
> >> Best,
> >> Matthias
> >>
> >>
> >>
> >> On 19.04.2017 12:57, Peter Cock wrote:
> >>>
> >>> Oh right - I've just been updating ncbi_blast_plus this morning
> >>> to transition to BLAST+ 2.5.0 via either bioconda or the older
> >>> legacy Tool Shed.
> >>>
> >>> I can try to update blast_rbh next, which may solve this by
> >>> letting you use bioconda.
> >>>
> >>> Peter
> >>>
> >>> On Wed, Apr 19, 2017 at 11:52 AM, Matthias Bernt <m.be...@ufz.de>
> wrote:
> >>>>
> >>>> Dear Marius,
> >>>>
> >>>> thanks again for the help. I'm trying to install blast_rbh (owner is
> >>>> peterjc). The dependencies look as follows:
> >>>>
> >>>> blast_rbh
> >>>> - biopython
> >>>>   - package_biopython_1_64
> >>>> - package_numpy_1_8
> >>>>   - package_atlas_3_10
> >>>> - blast
> >>>> - package_blast_plus_2_2_31
> >>>>   - blast+
> >>>>
> >>>> With unchecked "When available, install Tool Shed managed tool
> >>>> dependencies?" I get all as "Installed, missing tool dependencies".
> >>>> Blast_rbh is not working since blast+ is not installed. Even if I
> install
> >>>> blast+ manually (in galaxy) its not working (makeblastdb is missing).
> >>>>
> >>>> When I leave "When available, install Tool Shed managed tool
> >>>> dependencies?"
> >>>> I see blast_rbh, package_blast_plus package_biopython as installed and
> >>>> the
> >>>> others as Installed, missing tool dependencies.
> >>>> Here blast_rbh is working.
> >>>>
> >>>> What would be your suggestion? Live with the latter and file an issue
> at
> >>>> the
> >>>> tools repository (difficult to say which causes the problem(s))?
> >>>>
> >>>> I get similar behavior if I try to install from the test toolshed.
> >>>> (Only a different error from atlas: /bin/sh: line 3:
> >>>> /host/static_full_blas_lapack.diff: No such file or directory).
> >>>>
> >>>> But, actually I'm trying to automatize tool installation using the
> >>>> ansible
> >>>> role ansible-galaxy-tools
> >>>> (https://github.com/galaxyproject/ansible-galaxy-tools). With this
> method
> >>>> I
> >>>> get the latter behavior. The role uses uses ephemeris==0.4.0 to
> install
> >>>> tools. There I have seen three global variables t

Re: [galaxy-dev] atlas version 3.10.2 installation error

2017-04-19 Thread Peter Cock
Just lucky timing, could you try blast_rbh v0.1.11,

https://toolshed.g2.bx.psu.edu/view/peterjc/blast_rbh/d8d9a9069586

For BLAST+ this ought to work with either bioconda, or the legacy XML
based packages, as both have BLAST+ 2.5.0.

However, it looks like bioconda only has Biopython 1.68 and 1.69, while
the legacy XML based packages only go up to Biopython 1.67 at the time
of writing... it looks like we'll need to add a couple more legacy packages.

Peter

On Wed, Apr 19, 2017 at 12:01 PM, Matthias Bernt <m.be...@ufz.de> wrote:
> Dear Peter,
>
> thanks for the info. Would be great to get the update, but since I have a
> method that is working for the moment there is no need to hurry.
>
> Best,
> Matthias
>
>
>
> On 19.04.2017 12:57, Peter Cock wrote:
>>
>> Oh right - I've just been updating ncbi_blast_plus this morning
>> to transition to BLAST+ 2.5.0 via either bioconda or the older
>> legacy Tool Shed.
>>
>> I can try to update blast_rbh next, which may solve this by
>> letting you use bioconda.
>>
>> Peter
>>
>> On Wed, Apr 19, 2017 at 11:52 AM, Matthias Bernt <m.be...@ufz.de> wrote:
>>>
>>> Dear Marius,
>>>
>>> thanks again for the help. I'm trying to install blast_rbh (owner is
>>> peterjc). The dependencies look as follows:
>>>
>>> blast_rbh
>>> - biopython
>>>   - package_biopython_1_64
>>> - package_numpy_1_8
>>>   - package_atlas_3_10
>>> - blast
>>> - package_blast_plus_2_2_31
>>>   - blast+
>>>
>>> With unchecked "When available, install Tool Shed managed tool
>>> dependencies?" I get all as "Installed, missing tool dependencies".
>>> Blast_rbh is not working since blast+ is not installed. Even if I install
>>> blast+ manually (in galaxy) its not working (makeblastdb is missing).
>>>
>>> When I leave "When available, install Tool Shed managed tool
>>> dependencies?"
>>> I see blast_rbh, package_blast_plus package_biopython as installed and
>>> the
>>> others as Installed, missing tool dependencies.
>>> Here blast_rbh is working.
>>>
>>> What would be your suggestion? Live with the latter and file an issue at
>>> the
>>> tools repository (difficult to say which causes the problem(s))?
>>>
>>> I get similar behavior if I try to install from the test toolshed.
>>> (Only a different error from atlas: /bin/sh: line 3:
>>> /host/static_full_blas_lapack.diff: No such file or directory).
>>>
>>> But, actually I'm trying to automatize tool installation using the
>>> ansible
>>> role ansible-galaxy-tools
>>> (https://github.com/galaxyproject/ansible-galaxy-tools). With this method
>>> I
>>> get the latter behavior. The role uses uses ephemeris==0.4.0 to install
>>> tools. There I have seen three global variables that (I guess) control
>>> the
>>> installation of dependencies.
>>>
>>> INSTALL_TOOL_DEPENDENCIES = False
>>> INSTALL_REPOSITORY_DEPENDENCIES = True
>>> INSTALL_RESOLVER_DEPENDENCIES = True
>>>
>>> I guess I could control the behavior with the latter two variables.
>>>
>>> Best,
>>> Matthias
>>>
>>>
>>>
>>> On 18.04.2017 19:24, Marius van den Beek wrote:
>>>>
>>>>
>>>> Hi Matthias,
>>>>
>>>> it's me again!
>>>> What tool are you trying to install?
>>>>
>>>> My -perhaps- unhelpful suggestion is to not install anything that starts
>>>> with
>>>> package_* (also called tool shed packages), those should be exclusively
>>>> installed when you absolutely need
>>>> to install an old tool that can only function with packages.
>>>> We've switched to conda nowadays, which is much easier to install and to
>>>> maintain.
>>>> In fact the IUC has ported all their latest tools to conda, and
>>>> similarly most devteam
>>>> tools that are still in active use have also been ported to use Conda
>>>> dependencies.
>>>>
>>>> If you look at this
>>>> document
>>>> https://galaxyproject.org/admin/tools/add-tool-from-toolshed-tutorial/,
>>>> there is a part where you confirm dependencies. If you click on show
>>>> detail, and you unselect
>>>> When available, install Tool Shed managed tool dependencies?
>>>> Galaxy will not try to install these old

Re: [galaxy-dev] atlas version 3.10.2 installation error

2017-04-19 Thread Peter Cock
Oh right - I've just been updating ncbi_blast_plus this morning
to transition to BLAST+ 2.5.0 via either bioconda or the older
legacy Tool Shed.

I can try to update blast_rbh next, which may solve this by
letting you use bioconda.

Peter

On Wed, Apr 19, 2017 at 11:52 AM, Matthias Bernt  wrote:
> Dear Marius,
>
> thanks again for the help. I'm trying to install blast_rbh (owner is
> peterjc). The dependencies look as follows:
>
> blast_rbh
> - biopython
>   - package_biopython_1_64
> - package_numpy_1_8
>   - package_atlas_3_10
> - blast
> - package_blast_plus_2_2_31
>   - blast+
>
> With unchecked "When available, install Tool Shed managed tool
> dependencies?" I get all as "Installed, missing tool dependencies".
> Blast_rbh is not working since blast+ is not installed. Even if I install
> blast+ manually (in galaxy) its not working (makeblastdb is missing).
>
> When I leave "When available, install Tool Shed managed tool dependencies?"
> I see blast_rbh, package_blast_plus package_biopython as installed and the
> others as Installed, missing tool dependencies.
> Here blast_rbh is working.
>
> What would be your suggestion? Live with the latter and file an issue at the
> tools repository (difficult to say which causes the problem(s))?
>
> I get similar behavior if I try to install from the test toolshed.
> (Only a different error from atlas: /bin/sh: line 3:
> /host/static_full_blas_lapack.diff: No such file or directory).
>
> But, actually I'm trying to automatize tool installation using the ansible
> role ansible-galaxy-tools
> (https://github.com/galaxyproject/ansible-galaxy-tools). With this method I
> get the latter behavior. The role uses uses ephemeris==0.4.0 to install
> tools. There I have seen three global variables that (I guess) control the
> installation of dependencies.
>
> INSTALL_TOOL_DEPENDENCIES = False
> INSTALL_REPOSITORY_DEPENDENCIES = True
> INSTALL_RESOLVER_DEPENDENCIES = True
>
> I guess I could control the behavior with the latter two variables.
>
> Best,
> Matthias
>
>
>
> On 18.04.2017 19:24, Marius van den Beek wrote:
>>
>> Hi Matthias,
>>
>> it's me again!
>> What tool are you trying to install?
>>
>> My -perhaps- unhelpful suggestion is to not install anything that starts
>> with
>> package_* (also called tool shed packages), those should be exclusively
>> installed when you absolutely need
>> to install an old tool that can only function with packages.
>> We've switched to conda nowadays, which is much easier to install and to
>> maintain.
>> In fact the IUC has ported all their latest tools to conda, and
>> similarly most devteam
>> tools that are still in active use have also been ported to use Conda
>> dependencies.
>>
>> If you look at this
>> document
>> https://galaxyproject.org/admin/tools/add-tool-from-toolshed-tutorial/,
>> there is a part where you confirm dependencies. If you click on show
>> detail, and you unselect
>> When available, install Tool Shed managed tool dependencies?
>> Galaxy will not try to install these old packages. There are some tools
>> which come both with
>> old toolshed dependencies and that have Conda available. You may be
>> lucky and find that
>> the tool you are trying to install has Conda dependencies available.
>> There is a lot more background on the Toolshed and Conda in this
>> document here:
>> https://docs.galaxyproject.org/en/master/admin/conda_faq.html
>>
>> Best,
>> Marius
>>
>> On 18 April 2017 at 19:03, Matthias Bernt > > wrote:
>>
>> Dear list,
>>
>> again me :( stumbling from problem to problem ...
>>
>> I got compilation errors when installing
>>
>> package_atlas_3_10 Revision: 98c017ec230d
>>
>> I tried with gcc 4x and 6x without success. Do I need the intel C
>> compiler for this tool?
>>
>> If I got earlier posts
>> (http://dev.list.galaxyproject.org/atlas-3-10-IUC-error-tt4668236.html
>>
>> )
>>
>> correct a precompiled version should be used anyway?
>>
>> I'm running galaxy on CentOS 6.8 (64bit) (running in a VirtualBox).
>>
>> Best,
>> Matthias
>>
>>
>> Here is the output (which I guess is verbose output from configure):
>>
>>
>> /tmp/cctyMGAH.o: In function `ATL_tmpnam':
>>
>> /gpfs1/data/galaxy_server/galaxy-dev/database/tmp/tmp-toolshed-mtdAAKCLe/ATLAS/build/..//CONFIG/include/atlas_sys.h:224:
>> warning: the use of `tmpnam' is dangerous, better use `mkstemp'
>> probe_OS.o: In function `ATL_tmpnam':
>>
>> /gpfs1/data/galaxy_server/galaxy-dev/database/tmp/tmp-toolshed-mtdAAKCLe/ATLAS/build/..//CONFIG/include/atlas_sys.h:224:
>> warning: the use of `tmpnam' is dangerous, better use `mkstemp'
>> probe_asm.o: In function `ATL_tmpnam':
>>
>> /gpfs1/data/galaxy_server/galaxy-dev/database/tmp/tmp-toolshed-mtdAAKCLe/ATLAS/build/..//CONFIG/include/atlas_sys.h:224:
>> warning: the use of `tmpnam' is 

Re: [galaxy-dev] Testing updated NCBI BLAST+ wrappers for version 2.5.0

2017-04-19 Thread Peter Cock
This just went live on the main tool shed:

https://toolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus/

You can report issues here or on GitHub at:

https://github.com/peterjc/galaxy_blast/issues

Thanks,

Peter

On Thu, Dec 8, 2016 at 11:35 AM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
> Hello all,
>
> I have updated the NCBI BLAST+ wrappers on the Test Tool Shed,
> the wrapper is now at v0.2.00:
>
> https://testtoolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus/
>
> The main changes is this now depends on BLAST+ 2.5.0, and that is
> available via either BioConda or the Tool Shed:
>
> https://toolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_5_0/
> https://testtoolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_5_0/
>
> In order for the dependency to work smoothly on both BioConda
> and the Tool Shed system, we have changed the package name
> from "blast+" to just "blast". Given the NCBI stopped updated the
> original "legacy" BLAST some time ago, when combined with the
> version number this is no longer ambiguous.
>
> Jumping from using BLAST+ 2.2.31 to using BLAST+ 2.5.0
> required updating lots of the test files for NCBI changes, including
> dropping the GI numbers in many outputs, expanding the percentage
> identity field from 2dp to 3dp, and also changing how -parse_deflines
> works with tabular output.
>
> The wrappers (deliberately) do not yet offer any new functionality
> added in the recent NCBI BLAST+ updates, in particular BLAST
> XML v2 is not yet available as an output with a datatype in Galaxy.
>
> At this point I would welcome feedback from those of you using the
> BLAST+ wrappers - including if you were able to install this with the
> dependencies from BioConda or the traditional Tool Shed packages.
>
> Once I'm confident that this is all OK, I will update the main Tool Shed
> (and think about adding new functionality in 2017).
>
> Thank you all,
>
> Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] GCC2107 Update: Oral presentation abstracts due 15 April

2017-04-13 Thread Peter Cock
Hello all,

In addition to 15 April being the GCC2017 Oral Presentation
deadline, it is also the next Open Bioinformatics Foundation
(OBF) Travel Fellowship application deadline.

The program offers up to USD $1000 and is aimed at increasing
diverse participation at events promoting open source bioinformatics
software development and open science in the biological research
community (like the Galaxy Community Conference).

http://news.open-bio.org/2016/03/01/obf-travel-fellowship-program/
https://github.com/OBF/obf-docs/blob/master/Travel_fellowships.md

Regards,

Peter

(Dr Peter Cock, OBF Treasurer)


On Thu, Apr 13, 2017 at 10:02 PM, Dave Clements
<cleme...@galaxyproject.org> wrote:
> Hello all,
>
>
> This is just a reminder of some GCC2017 deadlines:
>
>
>   15 April:  Oral presentation abstracts are due this Saturday at 23:59
> Paris local time.
>
>   15 May:  Early registration ends
>
>   27 May:  Poster presentation abstracts are due.
>
>   27 May:  Visualization and Computer Demo presentation abstracts are due.
>
>   31 May:  Regular registration ends
>
>   23 June: Lightning Talk presentation abstracts are due.
>
>
> About GCC2017:
>
> GCC2017 will be in Montpellier, France, 26-30 June and will feature two days
> of presentations, discussions, poster sessions, lightning talks, computer
> demos, keynotes, and birds-of-a-feather meetups, all about data-intensive
> biology and the tools that support it. GCC2017 also includes data and coding
> hackathons, and two days of training covering 16 different topics.
>
> GCC2017 will be held at Le Corum Conference Centre in the heart of
> Montpellier, just 10km from the Mediterranean. This event will gather
> several hundred researchers addressing diverse questions and facing common
> challenges in data intensive life science research. GCC participants work
> across the tree of life, come from around the world, and work at
> universities, research organizations, industry, medical schools and research
> hospitals.  If you work in or support data intensive life science research
> then GCC2017 is an ideal opportunity to present your work.
>
>
> Early registration is starts at less than 55€ / day for post-docs and
> students.  You can also book low cost conference housing when you register.
>
>
> GCC2017 has sold out all premiere sponsorship slots (a first). If you are
> interested in sponsoring (or know someone who is) there are still Silver and
> Bronze and Hackathon sponsorships available. Contact the organisers if you
> are interested.
>
>
> About Galaxy
>
> Galaxy is an open, web-based platform for data-intensive biomedical analysis
> used by tens of thousands of researchers around the world.  It supports ad
> hoc exploration and analysis through scalable and repeatable data analysis
> pipelines for large research studies. Galaxy is available in over 90 free
> and publicly accessible web servers, on commercial and national cloud
> infrastructures, and is locally installed at hundreds, if not thousands, of
> research organisations around the world.
>
>
> We hope to see you this summer in Montpellier!
>
>
> Au revoir,
>
>
> The GCC2017 Organising Committee
>
>
>
> --
> https://galaxyproject.org/
> https://getgalaxy.org/
> https://usegalaxy.org/
> https://gcc2017.sciencesconf.org/
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] [Bosc] BOSC 2017: Call for Abstracts

2017-04-10 Thread Peter Cock
Hello all,

This is a reminder that the main ISMB/ECCB 2017 abstract submission
deadline is this Thursday 13 April - this is one of the biggest and
broadest commutational biology conferences and there will I'm sure be a
strong Galaxy presence.

https://www.iscb.org/ismbeccb2017-keydates

This deadline also applies to talks at all the former satellite or special
interest group (SIG) meetings, which are now called Communities of Special
Intererst (COSIs), and will this year run though the main meeting.

In particular, please hurry up and get your open source/open science
abstracts finished and submitted to BOSC, the annual Bioinformatics Open
Source Conference [*].

https://www.open-bio.org/wiki/BOSC_2017

Also, BOSC's patent organisation the Open Bioinformatics Foundation (OBF)
[**] has a travel fellowship available to help improve diversity at
Bioinformatics meetings including but not limited to BOSC. The next call
for this closes 15 April 2017.

https://github.com/OBF/obf-docs/blob/master/Travel_fellowships.md

Thanks for reading to the end.
Peter

[*] I'm on the BOSC committee and used to co-chair it, so clearly biased.

[**] I'm on the OBF board as treasurer.

On Wed, 8 Mar 2017 at 18:05, Peter Cock <p.j.a.c...@googlemail.com> wrote:

> Dear all,
>
> Recent BOSC meetings have had a strong Galaxy presence, so please do
> consider submitting an abstract and/or attending BOSC 2017 in Prague.
>
> Thank you,
>
> Peter
>
> Dr. Peter Cock,
> Bioinformatician at The James Hutton Institute;
> Open Bioinformatics Foundation, board of directors, treasurer;
> BOSC 2017 organising committee member.
>
> -- Forwarded message --
> From: *Nomi Harris* <nlhar...@gmail.com>
> Date: Mon, Mar 6, 2017 at 6:50 PM
> Subject: [Bosc] BOSC 2017: Call for Abstracts
> To: bosc-annou...@mailman.open-bio.org, bosc-review...@open-bio.org
> Cc: Nomi Harris <nlhar...@gmail.com>, BOSC 2017 Organizing Committee <
> b...@open-bio.org>
>
>
> *Call for Abstracts for the 18th Annual Bioinformatics Open Source
> Conference (BOSC 2017)*
> An ISMB/ECCB Community of Special Interest (COSI)
>
> Dates: July 22-23, 2017
> Location: Prague, Czech Republic
> Web site: http://www.open-bio.org/wiki/BOSC_2017
> Email: b...@open-bio.org
> BOSC announcements mailing list:
> http://lists.open-bio.org/mailman/listinfo/bosc-announce
> Twitter: @OBF_BOSC
>
> *Important Dates*
>
>- Call for one-page abstracts
><https://www.open-bio.org/wiki/BOSC_Abstract_Submission> opens: March
>6, 2017
>- Abstract submission
><https://www.open-bio.org/wiki/BOSC_Abstract_Submission> deadline:
>April 13, 2017
>- Travel fellowship
><https://github.com/OBF/obf-docs/blob/master/Travel_fellowships.md> 
> application
>deadline: April 15, 2017
>- Authors notified: May 10, 2017
>- Codefest 2017 <https://www.open-bio.org/wiki/Codefest_2017>: July
>21-22, Prague
>- BOSC 2017 <https://www.open-bio.org/wiki/BOSC_2017>: July 22-23,
>Prague  (days 1 and 2 of ISMB/ECCB)
>- ISMB/ECCB 2017 <https://www.iscb.org/ismbeccb2017>: July 21-25,
>Prague
>
> *About BOSC*
> Since 2000, the yearly Bioinformatics Open Source Conference (BOSC) has
> provided a forum for developers and users to interact and share research
> results and ideas in open source bioinformatics. BOSC’s broad spectrum of
> topics includes practical techniques for solving bioinformatics problems;
> software development practices; standards and ontologies; approaches that
> promote open science and sharing of data, results and software; and ways to
> grow open source communities while promoting diversity within them.
>
> In the past, BOSC has taken place the two days before ISMB as a Special
> Interest Group (SIG). This year, ISMB is trying a new structure: the SIGs
> (now called COSIs) are integrated into the main ISMB meeting. BOSC will
> take place the first two full days of ISMB (July 22-23). Attendees will
> have the option to register for the full ISMB/ECCB meeting (July 21-25) or
> for just two days (there is no single-day registration option this year). A
> limited number of partial travel fellowships will be granted to some
> accepted speakers who would not otherwise be able to attend BOSC--please
> see https://github.com/OBF/obf-docs/blob/master/Travel_fellowships.md for
> more information.
>
> We encourage you to submit one-page abstracts on any topic of relevance to
> open source bioinformatics and open science. After review, some abstracts
> will be selected for lightning talks, longer talks, and/or posters.
> Abstract submission instructions and a link to the EasyChair portal can be
> found on https://www.open-bio.org/wiki/BOSC_Abstra

Re: [galaxy-dev] cluster access

2017-04-10 Thread Peter Cock
That's interesting. I didn't set this up on our system, but if the LDAP can
tell Galaxy the real username, maybe we can avoid having to setup
usern...@example.org aliases and insisting our users do that for
logging into Galaxy.

Hopefully some other SGE users can comment on how they got the
run-as-user system working for them?

Peter

On Mon, Apr 10, 2017 at 3:43 PM, Matthias Bernt <m.be...@ufz.de> wrote:
> Hi,
>
> sorry, I forgot this central piece of information: We also use Univa Grid
> Engine.
>
> What would be easier adapting galaxy's mapping to users or setting up email
> aliases? I would need to ask our cluster admins if they could realize this.
>
> Best,
> Matthias
>
>
>
> On 10.04.2017 16:39, Peter Cock wrote:
>>
>> This may depend on the cluster setup, and how Galaxy accesses it.
>> What are you using?
>>
>> We use SGE (actually Univa Grid Engine) via DRMAA, and had to setup
>> usern...@example.org email aliases just for this.
>>
>> Peter
>>
>> On Mon, Apr 10, 2017 at 3:36 PM, Matthias Bernt <m.be...@ufz.de> wrote:
>>>
>>> Dear list,
>>>
>>> I'm just starting to get jobs submitted to our cluster as real system
>>> user.
>>> I read in the documentation that the name of the system user is
>>> determined
>>> by "the Galaxy user's email address (with the @domain stripped off)". But
>>> this does not work on our system, rather the username stored in the
>>> galaxy_user table is the name of the user (this is correctly generated by
>>> the LDAP login).
>>>
>>> The job runner script seems to get the user id, so I guess I need to dig
>>> deeper. So my question is: Where could I change this behavior?
>>>
>>> Thanks a lot.
>>>
>>> Cheers,
>>> Matthias
>>>
>>>
>>> --
>>>
>>> ---
>>> Matthias Bernt
>>> Bioinformatics Service
>>> Molekulare Systembiologie (MOLSYB)
>>> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
>>> Helmholtz Centre for Environmental Research GmbH - UFZ
>>> Permoserstraße 15, 04318 Leipzig, Germany
>>> Phone +49 341 235 482296,
>>> m.be...@ufz.de, www.ufz.de
>>>
>>> Sitz der Gesellschaft/Registered Office: Leipzig
>>> Registergericht/Registration Office: Amtsgericht Leipzig
>>> Handelsregister Nr./Trade Register Nr.: B 4703
>>> Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board:
>>> MinDirig
>>> Wilfried Kraus
>>> Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
>>> Prof. Dr. Dr. h.c. Georg Teutsch
>>> Administrative Geschäftsführerin/ Administrative Managing Director:
>>> Prof. Dr. Heike Graßmann
>>> ---
>>>
>>>
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>   https://lists.galaxyproject.org/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>   http://galaxyproject.org/search/
>
>
> --
>
> ---
> Matthias Bernt
> Bioinformatics Service
> Molekulare Systembiologie (MOLSYB)
> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
> Helmholtz Centre for Environmental Research GmbH - UFZ
> Permoserstraße 15, 04318 Leipzig, Germany
> Phone +49 341 235 482296,
> m.be...@ufz.de, www.ufz.de
>
> Sitz der Gesellschaft/Registered Office: Leipzig
> Registergericht/Registration Office: Amtsgericht Leipzig
> Handelsregister Nr./Trade Register Nr.: B 4703
> Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig
> Wilfried Kraus
> Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
> Prof. Dr. Dr. h.c. Georg Teutsch
> Administrative Geschäftsführerin/ Administrative Managing Director:
> Prof. Dr. Heike Graßmann
> ---
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] cluster access

2017-04-10 Thread Peter Cock
This may depend on the cluster setup, and how Galaxy accesses it.
What are you using?

We use SGE (actually Univa Grid Engine) via DRMAA, and had to setup
usern...@example.org email aliases just for this.

Peter

On Mon, Apr 10, 2017 at 3:36 PM, Matthias Bernt  wrote:
> Dear list,
>
> I'm just starting to get jobs submitted to our cluster as real system user.
> I read in the documentation that the name of the system user is determined
> by "the Galaxy user's email address (with the @domain stripped off)". But
> this does not work on our system, rather the username stored in the
> galaxy_user table is the name of the user (this is correctly generated by
> the LDAP login).
>
> The job runner script seems to get the user id, so I guess I need to dig
> deeper. So my question is: Where could I change this behavior?
>
> Thanks a lot.
>
> Cheers,
> Matthias
>
>
> --
>
> ---
> Matthias Bernt
> Bioinformatics Service
> Molekulare Systembiologie (MOLSYB)
> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
> Helmholtz Centre for Environmental Research GmbH - UFZ
> Permoserstraße 15, 04318 Leipzig, Germany
> Phone +49 341 235 482296,
> m.be...@ufz.de, www.ufz.de
>
> Sitz der Gesellschaft/Registered Office: Leipzig
> Registergericht/Registration Office: Amtsgericht Leipzig
> Handelsregister Nr./Trade Register Nr.: B 4703
> Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig
> Wilfried Kraus
> Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
> Prof. Dr. Dr. h.c. Georg Teutsch
> Administrative Geschäftsführerin/ Administrative Managing Director:
> Prof. Dr. Heike Graßmann
> ---
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] git clone 16.10 bwa 0.7.15 data manager

2017-04-10 Thread Peter Cock
Hi Bryan,

I suspect the missing dependency BWA may reflect the switch away
from Galaxy's home-grown packaging system to using Conda -
enabled by default since v17.01,

https://docs.galaxyproject.org/en/master/admin/conda_faq.html

Newer dependencies will likely next get an old fashioned package
on the Galaxy Tool Shed.

Peter

On Fri, Apr 7, 2017 at 12:18 AM, Bryan Hepworth
 wrote:
> Hi All
>
> I've been trying to get a local galaxy running with an assortment of tools 
> for a local researcher. I'm a linux administrator rather than a 
> bioinformatics researcher and am familiar with how repo's work for RedHat, R 
> and the like. I've built tools for other groups at work and from my reading 
> it looked like galaxy was working on similar lines for keeping things in step.
>
> I've done a git clone of 16.10 into /galaxy altered the galaxy.ini to 
> postgresql and added me as admin. Done the sh run.sh to get a feel for how 
> everything goes together. I've pulled reference genomes from the source and 
> have had some success converting to different formats. I bumped up against a 
> few dependency issues so I've also enabled the test repo.
>
> The first stumbling block on the most recent build is getting bwa on there 
> with the data manager route - it has a dependency on bwa 0.7.15, but try as I 
> might I don't seem to be able to find an instance of package_bwa_0.7.15 
> anywhere on the devteam repo's to get over this hurdle. I'm guessing I'm 
> missing something really simple as it isn't pulling in dependancies as the 
> documentation suggests it should. I'd be really grateful for any pointers.
>
> Bryan
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

[galaxy-dev] Fwd: [Bosc] BOSC 2017: Call for Abstracts

2017-03-08 Thread Peter Cock
Dear all,

Recent BOSC meetings have had a strong Galaxy presence, so please do
consider submitting an abstract and/or attending BOSC 2017 in Prague.

Thank you,

Peter

Dr. Peter Cock,
Bioinformatician at The James Hutton Institute;
Open Bioinformatics Foundation, board of directors, treasurer;
BOSC 2017 organising committee member.

-- Forwarded message --
From: Nomi Harris <nlhar...@gmail.com>
Date: Mon, Mar 6, 2017 at 6:50 PM
Subject: [Bosc] BOSC 2017: Call for Abstracts
To: bosc-annou...@mailman.open-bio.org, bosc-review...@open-bio.org
Cc: Nomi Harris <nlhar...@gmail.com>, BOSC 2017 Organizing Committee <
b...@open-bio.org>


*Call for Abstracts for the 18th Annual Bioinformatics Open Source
Conference (BOSC 2017)*
An ISMB/ECCB Community of Special Interest (COSI)

Dates: July 22-23, 2017
Location: Prague, Czech Republic
Web site: http://www.open-bio.org/wiki/BOSC_2017
Email: b...@open-bio.org
BOSC announcements mailing list: http://lists.open-bio.
org/mailman/listinfo/bosc-announce
Twitter: @OBF_BOSC

*Important Dates*

   - Call for one-page abstracts
   <https://www.open-bio.org/wiki/BOSC_Abstract_Submission> opens: March 6,
   2017
   - Abstract submission
   <https://www.open-bio.org/wiki/BOSC_Abstract_Submission> deadline: April
   13, 2017
   - Travel fellowship
   <https://github.com/OBF/obf-docs/blob/master/Travel_fellowships.md>
application
   deadline: April 15, 2017
   - Authors notified: May 10, 2017
   - Codefest 2017 <https://www.open-bio.org/wiki/Codefest_2017>: July
   21-22, Prague
   - BOSC 2017 <https://www.open-bio.org/wiki/BOSC_2017>: July 22-23,
   Prague  (days 1 and 2 of ISMB/ECCB)
   - ISMB/ECCB 2017 <https://www.iscb.org/ismbeccb2017>: July 21-25, Prague

*About BOSC*
Since 2000, the yearly Bioinformatics Open Source Conference (BOSC) has
provided a forum for developers and users to interact and share research
results and ideas in open source bioinformatics. BOSC’s broad spectrum of
topics includes practical techniques for solving bioinformatics problems;
software development practices; standards and ontologies; approaches that
promote open science and sharing of data, results and software; and ways to
grow open source communities while promoting diversity within them.

In the past, BOSC has taken place the two days before ISMB as a Special
Interest Group (SIG). This year, ISMB is trying a new structure: the SIGs
(now called COSIs) are integrated into the main ISMB meeting. BOSC will
take place the first two full days of ISMB (July 22-23). Attendees will
have the option to register for the full ISMB/ECCB meeting (July 21-25) or
for just two days (there is no single-day registration option this year). A
limited number of partial travel fellowships will be granted to some
accepted speakers who would not otherwise be able to attend BOSC--please
see https://github.com/OBF/obf-docs/blob/master/Travel_fellowships.md for
more information.

We encourage you to submit one-page abstracts on any topic of relevance to
open source bioinformatics and open science. After review, some abstracts
will be selected for lightning talks, longer talks, and/or posters.
Abstract submission instructions and a link to the EasyChair portal can be
found on https://www.open-bio.org/wiki/BOSC_Abstract_Submission

*Session topics include:*

   - Open Science and Reproducible Research
   - Open Biomedical Data
   - Citizen/Participatory Science
   - Standards and Interoperability
   - Data Science
   - Workflows
   - Visualization
   - Medical and Translational Bioinformatics
   - Developer Tools and Libraries
   - Bioinformatics Open Source Project Progress Reports

*Sponsorship*
We gratefully accept sponsorships from relevant private companies. These
sponsorships enable us to offer free registration to some BOSC speakers to
help increase diversity at our meeting. Sponsors in 2016 included
Curoverse, the company behind the open source platform Arvados. Please
contact us if you are interested in being a sponsor of BOSC 2017!

Thank you,
  BOSC 2017 Organizing Committee:   Nomi Harris (chair), Brad Chapman,
Peter Cock, Christopher Fields, Bastian Greshake, Karsten Hokamp, Hilmar
Lapp, Mónica Muñoz-Torres, Heather Wiencko

P.S. Don't forget to submit your BOSC abstract by April 13 at
https://www.open-bio.org/wiki/BOSC_Abstract_Submission!

-- 
You received this message because you are subscribed to the Google Groups
"OBF-BOSC" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to obf-bosc+unsubscr...@googlegroups.com.
Visit this group at https://groups.google.com/group/obf-bosc.
For more options, visit https://groups.google.com/d/optout.
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists

Re: [galaxy-dev] Rg: Galaxy Version

2017-03-08 Thread Peter Cock
Thanks Martin & Eugen,

That's much easier (although Anamika will likely need to check if
their Galaxy was setup using hg or git anyway for updating it).

I'm not sure when it was introduced, but its been a while as even
the older Galaxy instance in my example under hg gives:

{"version_major": "15.05"}

Regards,

Peter


On Wed, Mar 8, 2017 at 2:06 PM, Martin Čech <mar...@bx.psu.edu> wrote:
> Hi Anamika,
>
> adding a detail to Peter's excellent answer:
>
> For newer versions of Galaxy (while it is running) you can go to the url
> `your.galaxy.com/api/version` to learn the release of Galaxy you have.
>
> Thanks for using Galaxy,
>
> Martin
>
> On Wed, Mar 8, 2017 at 7:39 AM Peter Cock <p.j.a.c...@googlemail.com> wrote:
>>
>> Galaxy was originally distributed via the mercurial (hg) version
>> control system, later the git version control system.
>>
>> Try this (which will work on a very old Galaxy using hg),
>>
>> $ hg log | head
>> changeset:   17125:e837a68e8d14
>> tag: tip
>> parent:  17121:7a4e3a661a76
>> parent:  17124:46a4ef8850b5
>> user:Nate Coraor <n...@bx.psu.edu>
>> date:Fri May 15 15:21:34 2015 -0400
>> summary: Merge stable to default
>> ...
>>
>> If that fails, try this which will work on a Galaxy distributed with git:
>>
>> $ git log | head
>> commit cbd527b9a4671c7abe9a22cf69e869b5d2de858a
>> Merge: 33dec01 d0f39ce
>> Author: Nicola Soranzo <nicola.sora...@earlham.ac.uk>
>> Date:   Wed Dec 7 20:35:40 2016 +
>>
>> Merge branch 'release_16.10' into dev
>>
>> Conflicts:
>> static/scripts/bundled/analysis.bundled.js
>> static/scripts/bundled/analysis.bundled.js.map
>> ...
>>
>>
>> If it isn't clear from the commit message, you can then match those
>> commits
>> to the relevant repository and hopefully work it out, here those were:
>>
>> https://bitbucket.org/galaxy/galaxy-central/commits/e837a68e8d14
>>
>> and
>>
>>
>> https://github.com/galaxyproject/galaxy/commits/cbd527b9a4671c7abe9a22cf69e869b5d2de858a
>>
>> If neither git nor hg work, then perhaps your Galaxy was installed via a
>> different route - and that will likely complicate updating it.
>>
>> Peter
>>
>>
>> On Wed, Mar 8, 2017 at 10:36 AM, Anamika Singh <anamikalibr...@gmail.com>
>> wrote:
>> > Dear galaxy team,
>> >
>> > I am using galaxy platform (old version) in Linux OS which was not
>> > installed
>> > by me. I would like to know 'how to check what is the galaxy version
>> > installed currently in my system.
>> >
>> > Regards
>> >
>> > --
>> > Anamika
>> >
>> > ___
>> > Please keep all replies on the list by using "reply all"
>> > in your mail client.  To manage your subscriptions to this
>> > and other Galaxy lists, please use the interface at:
>> >   https://lists.galaxyproject.org/
>> >
>> > To search Galaxy mailing lists use the unified search at:
>> >   http://galaxyproject.org/search/mailinglists/
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Rg: Galaxy Version

2017-03-08 Thread Peter Cock
Galaxy was originally distributed via the mercurial (hg) version
control system, later the git version control system.

Try this (which will work on a very old Galaxy using hg),

$ hg log | head
changeset:   17125:e837a68e8d14
tag: tip
parent:  17121:7a4e3a661a76
parent:  17124:46a4ef8850b5
user:Nate Coraor 
date:Fri May 15 15:21:34 2015 -0400
summary: Merge stable to default
...

If that fails, try this which will work on a Galaxy distributed with git:

$ git log | head
commit cbd527b9a4671c7abe9a22cf69e869b5d2de858a
Merge: 33dec01 d0f39ce
Author: Nicola Soranzo 
Date:   Wed Dec 7 20:35:40 2016 +

Merge branch 'release_16.10' into dev

Conflicts:
static/scripts/bundled/analysis.bundled.js
static/scripts/bundled/analysis.bundled.js.map
...


If it isn't clear from the commit message, you can then match those commits
to the relevant repository and hopefully work it out, here those were:

https://bitbucket.org/galaxy/galaxy-central/commits/e837a68e8d14

and

https://github.com/galaxyproject/galaxy/commits/cbd527b9a4671c7abe9a22cf69e869b5d2de858a

If neither git nor hg work, then perhaps your Galaxy was installed via a
different route - and that will likely complicate updating it.

Peter


On Wed, Mar 8, 2017 at 10:36 AM, Anamika Singh  wrote:
> Dear galaxy team,
>
> I am using galaxy platform (old version) in Linux OS which was not installed
> by me. I would like to know 'how to check what is the galaxy version
> installed currently in my system.
>
> Regards
>
> --
> Anamika
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Execute command once job has completed

2017-02-28 Thread Peter Cock
I do not believe that is possible (nor desirable in general,
not everyone will need or want the converted file and the
overhead that comes with creating and storing it).

What do you want to do with the bigwig file, and is the
automatic conversion working for you? i.e. If you have
tool X in Galaxy which wants an bigwig input file, does
Galaxy offer an automated conversion of your bedgraph
file as an input in the drop down file list?

Peter


On Tue, Feb 28, 2017 at 5:27 PM, evan clark <eclar...@fau.edu> wrote:
> By automatic I mean that once any bedgraph file is finished writing it will
> automatically run the convert to bigwig command
>
>
> Peter Cock
> February 28, 2017 12:21 PM
> It depends on what you mean by automatic. You could
> create a Galaxy Workflow where a tool is run which
> generates the bedgraph file, and then as the next
> step it is converted to bigwig.
>
> For this specific task, it might be possible to exploit
> the Galaxy datatype definition for bedgraph to allow
> automatic conversion to bigwig - is this working for you?
>
> https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/datatypes/converters/bedgraph_to_bigwig_converter.xml
>
> Peter
> evan clark
> February 28, 2017 11:47 AM
> Is there a method to setup a command to run depending on the filetype of a
> job. Essentially I want to setup a task where everytime a bedgraph file is
> generated it will either automatically start a new job and convert it to
> bigwig or, within the same job file it will convert the bedgraph to bigwig.
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Execute command once job has completed

2017-02-28 Thread Peter Cock
It depends on what you mean by automatic. You could
create a Galaxy Workflow where a tool is run which
generates the bedgraph file, and then as the next
step it is converted to bigwig.

For this specific task, it might be possible to exploit
the Galaxy datatype definition for bedgraph to allow
automatic conversion to bigwig - is this working for you?

https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/datatypes/converters/bedgraph_to_bigwig_converter.xml

Peter

On Tue, Feb 28, 2017 at 4:47 PM, evan clark  wrote:
> Is there a method to setup a command to run depending on the filetype of a
> job. Essentially I want to setup a task where everytime a bedgraph file is
> generated it will either automatically start a new job and convert it to
> bigwig or, within the same job file it will convert the bedgraph to bigwig.
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Tools using Galaxy's Python library?

2017-02-13 Thread Peter Cock
To recap, in the next release of Galaxy, there will not be a copy of the
sequence parsing code like lib/galaxy_utils/sequence/fastq.py - see
this pull request: https://github.com/galaxyproject/galaxy/pull/3551

Both Galaxy itself, and any tools needing it, will depend on the now
separately packaged galaxy_sequence_utils library, source code now
here:

https://github.com/galaxyproject/sequence_utils

The original version of this is available on the Tool Shed:

https://toolshed.g2.bx.psu.edu

The library is now available on PyPI too:

https://pypi.python.org/pypi/galaxy_sequence_utils/

This code has been updated and now does "import six" as part of
work to make sure it can be used on Python 2 and Python 3, and
it includes support for compressed FASTQ files.

My TravisCI tool tests are passing again using both Galaxy's stable
and dev branches.

Peter

On Wed, Feb 1, 2017 at 11:20 AM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
> This went smoother today, and has served to refresh some of the
> details of how the Galaxy's venv etc all work.
>
> In my TravisCI setup for tool testing I currently "manually" install
> the tool dependencies, and thus am using pip with the same URL
> as the Tool Shed package:
>
> pip install 
> http://depot.galaxyproject.org/package/source/galaxy_sequence_utils/galaxy_sequence_utils-1.0.1-st.tgz
>
> Commit:
>
> https://github.com/peterjc/pico_galaxy/commit/c9b77b49281974988c790a54f05c4d1ff81b0904
>
> The TravisCI build is still failing, but on an unrelated change
> on the Galaxy dev branch.
>
> Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] ImportError and Project life cycle

2017-02-13 Thread Peter Cock
On Mon, Feb 13, 2017 at 9:17 AM, Matthias Bernt  wrote:
> Dear Martin,
>
> Thanks for your input. Before answering the raised questions I would like to
> add some more information that I just found:
>
> I've noticed something odd (to me) which is maybe the source of the problem.
> My galaxy installation has galaxy_utils installed twice:
>
> - lib/galaxy_utils/sequence/fastq.py
>
> -
> database/dependencies/galaxy_sequence_utils/1.0.0/devteam/package_galaxy_utils_1_0/8882f14715b5/lib/python/galaxy_sequence_utils-1.0.0-py2.7.egg/galaxy_utils/sequence/fastq.py
>

Something like that is to be expected right now as this is in
transition, see also:

http://dev.list.galaxyproject.org/Tools-using-Galaxy-s-Python-library-td4670472.html

In the next release of Galaxy, there will not be a copy of the code at
lib/galaxy_utils/sequence/fastq.py - see this pull request for the change
https://github.com/galaxyproject/galaxy/pull/3551

Originally tools were allowed to access Galaxy's internal Python code,
here lib/galaxy_utils/sequence/fastq.py - but that is changing. In the
next Galaxy release tools should explicitly depend on the new
packaged version of Galaxy's sequence_utils - that's where the
second copy you've seen comes from:

https://github.com/galaxyproject/sequence_utils

This does now import six as part of work to make sure it can be
used on Python 2 and Python 3:

https://github.com/galaxyproject/sequence_utils/blob/master/galaxy_utils/sequence/sequence.py

Galaxy itself will depend on the now separate sequence_utils
package.

--

As to what is going wrong in your situation, I'm hoping one of the
Galaxy developers working on this can help.

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] ImportError and Project life cycle

2017-02-10 Thread Peter Cock
On Fri, Feb 10, 2017 at 3:26 PM, Matthias Bernt  wrote:
> Dear galaxy-dev list,
>
> I'm currently trying to introduce myself to the galaxy server, i.e.,
> learning how to use and administer a galaxy server.
>
> I have installed galaxy following this excellent tutorial:
>
> https://github.com/martenson/dagobah-training
>
> Then I tried the tool "FASTQ Quality Trimmer by sliding window" which is
> mentioned in NGS101-6 tutorial. I installed the tool from tool shed, but I
> get a python error (the Admin panel shows that all dependencies are
> installed):
>
> Traceback (most recent call last):
>   File
> "/home/berntm/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/fastq_trimmer_by_quality/25c24379693a/fastq_trimmer_by_quality/fastq_trimmer_by_quality.py",
> line 3, in 
> from galaxy_utils.sequence.fastq import fastqReader, fastqWriter
>   File "/home/berntm/galaxy/lib/galaxy_utils/sequence/fastq.py", line 7, in
> 
> from six import Iterator, string_types
> ImportError: No module named six
>
> Do you have an idea what could be wrong? I guess that this is to little
> information, but I did not know which kind of additional information would
> be helpful.

This appears to be a missing dependency on the Python module "six"
which is used to help write code which will work under both Python 2
and Python 3.

This is likely a recent regression - hopefully one of the Galaxy team
can comment on this, likely the Tool Shed definition for this tool needs
a minor tweak (and then you can apply the update via the Galaxy
Admin controls within your browser).

> Another question: At my institution quite a lot of NGS projects are carried
> out. After a project is finished all methods and data are stored to a long
> term read only tape archive. For the future the idea is to use galaxy. Which
> options are there to implement such a project life cycle with galaxy?

Not that I personally am aware of, no. There are optional disk quotas to
encourage users to remove (delete) old data.

> A last general question: would it be better to send a separate mail to the
> list when I have multiple questions?

Yes please, with useful email subjects would be best.

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Create a new datatype svg

2017-02-09 Thread Peter Cock
Hmm. I wonder if a composite datatype would work here?

https://wiki.galaxyproject.org/Admin/Datatypes/Composite%20Datatypes

Peter

On Thu, Feb 9, 2017 at 9:25 AM, Jochen Bick  wrote:
> Hi Dan,
>
> I understand the option with this specific flag. What I mean is that for
> example the file format bam has two file in one save button. So if you
> click on save you can either save the bam file or the bai file. I would
> liek to have the same feature with my png and svg. So that I have just
> one item in the history showing the png and if you click on save you
> have the option to also save svg.
>
> Cheers Jochen
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Peter Cock <p.j.a.c...@googlemail.com>

2017-02-01 Thread Peter Cock
Thank you,

Peter

On Wed, Feb 1, 2017 at 3:11 PM, Eduardo de Paiva Alves
<alves.edua...@gmail.com> wrote:
> Hi Peter,
>
> I am wrapping https://rostlab.org/owiki/index.php/Tmseg which takes a
> PSSM matrix as one of the inputs so I will add the psiblast locally
> and test and let yo know how it goes.
>
> Thank you
>
> Eduardo
>
> On 01/02/2017 10:10, "Peter Cock" <p.j.a.c...@googlemail.com> wrote:
>
>
> Hi Eduardo,
>
>
> Certainly when Luobin Yang submitted the early work on the
> PSI BLAST wrapper we need the PSSM datatype to be defined,
> but as you note, that has been done:
>
> https://github.com/peterjc/galaxy_blast/commit/65252ee61fdd98e52454ff045fa069f04ec40c06
>
> According to the minimal notes I made on the issue for finishing
> the PSI BLAST wrapper, it is just help text and unit tests needed
> (which would include active user testing):
>
> https://github.com/peterjc/galaxy_blast/issues/19
>
> If you are a PSI BLAST user, it would be very helpful to have
> your input here.
>
> Right now the wrapper is on the TravisCI blacklist because
> it would fail testing. Likewise it is not included in the Tool Shed.
>
> Peter
>
> On Wed, Feb 1, 2017 at 9:47 AM, De paiva Alves, Eduardo
> <eduardoal...@abdn.ac.uk> wrote:
> Peter
>
> I noticed there is now a PSSM datatype which is used by NCBI BLAST+
> makeprofiledb. Does that allow you to remove psiblast from blacklist in
> your repository?
>
>
> https://github.com/peterjc/galaxy_blast/blob/13d5b6deca0663d7f1301428574720
> b19aea1380/.tt_blacklist
>
>
> Thank you
>
> Eduardo
>
>
> On Tue, Feb 5, 2013 at 4:28 PM, Peter Cock <[hidden email]> wrote:
> Thanks Luobin,
>
> I've checked that into my development repository (on my tools branch):
> https://bitbucket.org/peterjc/galaxy-central/commits/f8f43f8494abdd228998d4
> e9fe67b0f2378494e0
>
> That will allow me to track changes etc as we work on the datatypes.
> Note I am not yet including ncbi_psiblast_wrapper.xml on the Galaxy
> Tool Shed.
>
> Regards,
>
> Peter
>
>
>
> The University of Aberdeen is a charity registered in Scotland, No SC013683.
> Tha Oilthigh Obar Dheathain na charthannas clàraichte ann an Alba, Àir. 
> SC013683
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Tools using Galaxy's Python library?

2017-02-01 Thread Peter Cock
This went smoother today, and has served to refresh some of the
details of how the Galaxy's venv etc all work.

In my TravisCI setup for tool testing I currently "manually" install
the tool dependencies, and thus am using pip with the same URL
as the Tool Shed package:

pip install 
http://depot.galaxyproject.org/package/source/galaxy_sequence_utils/galaxy_sequence_utils-1.0.1-st.tgz

Commit:

https://github.com/peterjc/pico_galaxy/commit/c9b77b49281974988c790a54f05c4d1ff81b0904

The TravisCI build is still failing, but on an unrelated change
on the Galaxy dev branch.

Peter

On Wed, Feb 1, 2017 at 9:46 AM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
> Thanks John - that could explain what's going wrong and offers
> another approach to fixing my TravisCI testing - setting:
>
> preserve_python_environment  = legacy_and_local
>
> or:
>
> preserve_python_environment = always
>
> (Despite several attempts I didn't hit on a nice way to get the
> galaxy_sequence_utils installed nicely via a "manual install",
> one day I will switch my TravisCI testing to use BioConda)
>
> I am hoping to get my TravisCI builds passing again today...
>
> Peter
>
> On Tue, Jan 31, 2017 at 5:27 PM, John Chilton <jmchil...@gmail.com> wrote:
>> So the PR that broke your tools is probably here
>> https://github.com/galaxyproject/galaxy/pull/3364/files. That pull request
>> removed Galaxy from the Python path of Galaxy tools - this gives tools a
>> much cleaner environment and prevents certain conflicts between Conda and
>> Galaxy.
>>
>> As part of that PR, there is a list of Galaxy tools in
>> lib/galaxy/tools/__init__.py that still get setup with Galaxy's Python
>> environment. This includes many random devteam and even an IUC tool - I've
>> added something to the list even though one ancient version of tool shed
>> tool required it years ago and it has since been updated. I would open a PR
>> to add the tool id of any of your tools that have been published to the tool
>> shed and require these tools to this list. You can target it against 17.01
>> ideally and then we can merge that into dev.
>>
>> This behavior can be reverted - there is a config option that is documented
>> pretty well in the PR but I missed named it in the sample (fixing it with
>> https://github.com/galaxyproject/galaxy/pull/3521/files).
>>
>> Hopefully this helps and thanks for the bug report!
>>
>> -John
>>
>>
>> On Tue, Jan 31, 2017 at 7:08 AM Peter Cock <p.j.a.c...@googlemail.com>
>> wrote:
>>>
>>> Thanks Nicola,
>>>
>>> I didn't spot the missing slash, but planemo lint did - for anyone
>>> else copy-and-pasting:
>>>
>>> >> version="1.0.1">galaxy_sequence_utils
>>>
>>> For anyone not yet using BioConda with Galaxy, there is also a Tool
>>> Shed entry here:
>>>
>>>
>>> https://toolshed.g2.bx.psu.edu/view/iuc/package_galaxy_sequence_utils_1_0_1/
>>>
>>> I am therefore adding this to my tool_dependencies.xml files:
>>>
>>> 
>>> >> />
>>> 
>>>
>>> I still need to work out how best to make this available under TravisCI,
>>> given I am not currently using BioConda for the dependencies.
>>>
>>> @IUC: Should this also be released on PyPI for easy install via pip?
>>>
>>> Peter
>>>
>>>
>>> On Tue, Jan 31, 2017 at 11:15 AM, Nicola Soranzo <nsora...@tiscali.it>
>>> wrote:
>>> > Hi Peter,
>>> > adding
>>> >
>>> > >> > version="1.0.1">galaxy_sequence_utils
>>> >
>>> > to each of these tools should solve the problem, there is a Bioconda
>>> > package
>>> > which provides the Python library:
>>> >
>>> > https://anaconda.org/bioconda/galaxy_sequence_utils
>>> >
>>> > Cheers,
>>> > Nicola
>>> >
>>> >
>>> > On 31/01/17 10:39, Peter Cock wrote:
>>> >>
>>> >> Hi all,
>>> >>
>>> >> A few of my tools have for a long time used Galaxy's own parsing
>>> >> functionality in order to avoid an external dependency. Lately
>>> >> this has stopped working on my TravisCI testing with planemo
>>> >> using the Galaxy dev branch (the stable master branch is fine):
>>> >> e.g.
>>> >>
>>> >> https://travis-ci.org/peterjc/pico_galaxy/builds/196655736
>>> >>
>&g

Re: [galaxy-dev] psiblast

2017-02-01 Thread Peter Cock
Hi Eduardo,

Certainly when Luobin Yang submitted the early work on the
PSI BLAST wrapper we need the PSSM datatype to be defined,
but as you note, that has been done:

https://github.com/peterjc/galaxy_blast/commit/65252ee61fdd98e52454ff045fa069f04ec40c06

According to the minimal notes I made on the issue for finishing
the PSI BLAST wrapper, it is just help text and unit tests needed
(which would include active user testing):

https://github.com/peterjc/galaxy_blast/issues/19

If you are a PSI BLAST user, it would be very helpful to have
your input here.

Right now the wrapper is on the TravisCI blacklist because
it would fail testing. Likewise it is not included in the Tool Shed.

Peter

On Wed, Feb 1, 2017 at 9:47 AM, De paiva Alves, Eduardo
<eduardoal...@abdn.ac.uk> wrote:
> Peter
>
> I noticed there is now a PSSM datatype which is used by NCBI BLAST+
> makeprofiledb. Does that allow you to remove psiblast from blacklist in
> your repository?
>
> https://github.com/peterjc/galaxy_blast/blob/13d5b6deca0663d7f1301428574720
> b19aea1380/.tt_blacklist
>
>
> Thank you
>
> Eduardo
>
>
> On Tue, Feb 5, 2013 at 4:28 PM, Peter Cock <[hidden email]> wrote:
> Thanks Luobin,
>
> I've checked that into my development repository (on my tools branch):
> https://bitbucket.org/peterjc/galaxy-central/commits/f8f43f8494abdd228998d4
> e9fe67b0f2378494e0
>
> That will allow me to track changes etc as we work on the datatypes.
> Note I am not yet including ncbi_psiblast_wrapper.xml on the Galaxy
> Tool Shed.
>
> Regards,
>
> Peter
>
>
>
> The University of Aberdeen is a charity registered in Scotland, No SC013683.
> Tha Oilthigh Obar Dheathain na charthannas clàraichte ann an Alba, Àir. 
> SC013683.
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Tools using Galaxy's Python library?

2017-02-01 Thread Peter Cock
Thanks John - that could explain what's going wrong and offers
another approach to fixing my TravisCI testing - setting:

preserve_python_environment  = legacy_and_local

or:

preserve_python_environment = always

(Despite several attempts I didn't hit on a nice way to get the
galaxy_sequence_utils installed nicely via a "manual install",
one day I will switch my TravisCI testing to use BioConda)

I am hoping to get my TravisCI builds passing again today...

Peter

On Tue, Jan 31, 2017 at 5:27 PM, John Chilton <jmchil...@gmail.com> wrote:
> So the PR that broke your tools is probably here
> https://github.com/galaxyproject/galaxy/pull/3364/files. That pull request
> removed Galaxy from the Python path of Galaxy tools - this gives tools a
> much cleaner environment and prevents certain conflicts between Conda and
> Galaxy.
>
> As part of that PR, there is a list of Galaxy tools in
> lib/galaxy/tools/__init__.py that still get setup with Galaxy's Python
> environment. This includes many random devteam and even an IUC tool - I've
> added something to the list even though one ancient version of tool shed
> tool required it years ago and it has since been updated. I would open a PR
> to add the tool id of any of your tools that have been published to the tool
> shed and require these tools to this list. You can target it against 17.01
> ideally and then we can merge that into dev.
>
> This behavior can be reverted - there is a config option that is documented
> pretty well in the PR but I missed named it in the sample (fixing it with
> https://github.com/galaxyproject/galaxy/pull/3521/files).
>
> Hopefully this helps and thanks for the bug report!
>
> -John
>
>
> On Tue, Jan 31, 2017 at 7:08 AM Peter Cock <p.j.a.c...@googlemail.com>
> wrote:
>>
>> Thanks Nicola,
>>
>> I didn't spot the missing slash, but planemo lint did - for anyone
>> else copy-and-pasting:
>>
>> > version="1.0.1">galaxy_sequence_utils
>>
>> For anyone not yet using BioConda with Galaxy, there is also a Tool
>> Shed entry here:
>>
>>
>> https://toolshed.g2.bx.psu.edu/view/iuc/package_galaxy_sequence_utils_1_0_1/
>>
>> I am therefore adding this to my tool_dependencies.xml files:
>>
>> 
>> > />
>> 
>>
>> I still need to work out how best to make this available under TravisCI,
>> given I am not currently using BioConda for the dependencies.
>>
>> @IUC: Should this also be released on PyPI for easy install via pip?
>>
>> Peter
>>
>>
>> On Tue, Jan 31, 2017 at 11:15 AM, Nicola Soranzo <nsora...@tiscali.it>
>> wrote:
>> > Hi Peter,
>> > adding
>> >
>> > > > version="1.0.1">galaxy_sequence_utils
>> >
>> > to each of these tools should solve the problem, there is a Bioconda
>> > package
>> > which provides the Python library:
>> >
>> > https://anaconda.org/bioconda/galaxy_sequence_utils
>> >
>> > Cheers,
>> > Nicola
>> >
>> >
>> > On 31/01/17 10:39, Peter Cock wrote:
>> >>
>> >> Hi all,
>> >>
>> >> A few of my tools have for a long time used Galaxy's own parsing
>> >> functionality in order to avoid an external dependency. Lately
>> >> this has stopped working on my TravisCI testing with planemo
>> >> using the Galaxy dev branch (the stable master branch is fine):
>> >> e.g.
>> >>
>> >> https://travis-ci.org/peterjc/pico_galaxy/builds/196655736
>> >>
>> >> The tools fail with things like:
>> >>
>> >> |  from galaxy_utils.sequence.fasta import fastaReader, fastaWriter
>> >> |  ImportError: No module named galaxy_utils.sequence.fasta
>> >>
>> >>
>> >> or:
>> >>
>> >> |  from galaxy_utils.sequence.fastq import fastqReader
>> >> |  ImportError: No module named galaxy_utils.sequence.fastq
>> >>
>> >> Is this a temporary regression in Galaxy, or a deliberate change?
>> >> Do the tools need to do something to explicit have access to the
>> >> Galaxy Python library, or are they now considered private?
>> >> If so, I can update these tools to use an explicit dependency
>> >> for parsing FASTA and FASTQ (e.g. Biopython).
>> >>
>> >> Thanks,
>> >>
>> >> Peter
>> >>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Tools using Galaxy's Python library?

2017-01-31 Thread Peter Cock
Thanks Nicola,

I didn't spot the missing slash, but planemo lint did - for anyone
else copy-and-pasting:

galaxy_sequence_utils

For anyone not yet using BioConda with Galaxy, there is also a Tool
Shed entry here:

https://toolshed.g2.bx.psu.edu/view/iuc/package_galaxy_sequence_utils_1_0_1/

I am therefore adding this to my tool_dependencies.xml files:





I still need to work out how best to make this available under TravisCI,
given I am not currently using BioConda for the dependencies.

@IUC: Should this also be released on PyPI for easy install via pip?

Peter


On Tue, Jan 31, 2017 at 11:15 AM, Nicola Soranzo <nsora...@tiscali.it> wrote:
> Hi Peter,
> adding
>
>  version="1.0.1">galaxy_sequence_utils
>
> to each of these tools should solve the problem, there is a Bioconda package
> which provides the Python library:
>
> https://anaconda.org/bioconda/galaxy_sequence_utils
>
> Cheers,
> Nicola
>
>
> On 31/01/17 10:39, Peter Cock wrote:
>>
>> Hi all,
>>
>> A few of my tools have for a long time used Galaxy's own parsing
>> functionality in order to avoid an external dependency. Lately
>> this has stopped working on my TravisCI testing with planemo
>> using the Galaxy dev branch (the stable master branch is fine):
>> e.g.
>>
>> https://travis-ci.org/peterjc/pico_galaxy/builds/196655736
>>
>> The tools fail with things like:
>>
>> |  from galaxy_utils.sequence.fasta import fastaReader, fastaWriter
>> |  ImportError: No module named galaxy_utils.sequence.fasta
>>
>>
>> or:
>>
>> |  from galaxy_utils.sequence.fastq import fastqReader
>> |  ImportError: No module named galaxy_utils.sequence.fastq
>>
>> Is this a temporary regression in Galaxy, or a deliberate change?
>> Do the tools need to do something to explicit have access to the
>> Galaxy Python library, or are they now considered private?
>> If so, I can update these tools to use an explicit dependency
>> for parsing FASTA and FASTQ (e.g. Biopython).
>>
>> Thanks,
>>
>> Peter
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>http://galaxyproject.org/search/mailinglists/
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Tools using Galaxy's Python library?

2017-01-31 Thread Peter Cock
Hi all,

A few of my tools have for a long time used Galaxy's own parsing
functionality in order to avoid an external dependency. Lately
this has stopped working on my TravisCI testing with planemo
using the Galaxy dev branch (the stable master branch is fine):
e.g.

https://travis-ci.org/peterjc/pico_galaxy/builds/196655736

The tools fail with things like:

|  from galaxy_utils.sequence.fasta import fastaReader, fastaWriter
|  ImportError: No module named galaxy_utils.sequence.fasta


or:

|  from galaxy_utils.sequence.fastq import fastqReader
|  ImportError: No module named galaxy_utils.sequence.fastq

Is this a temporary regression in Galaxy, or a deliberate change?
Do the tools need to do something to explicit have access to the
Galaxy Python library, or are they now considered private?
If so, I can update these tools to use an explicit dependency
for parsing FASTA and FASTQ (e.g. Biopython).

Thanks,

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Check running jobs

2017-01-27 Thread Peter Cock
Ah - I assumed you meant from the user-facing UI, or the reports page.

I would suggest exploring the advance job metrics options, see e.g.

https://github.com/galaxyproject/galaxy/blob/dev/config/job_metrics_conf.xml.sample

I've never used it, but perhaps the backend using Collectl
(http://collectl.sourceforge.net/) would cover this?

Peter

On Fri, Jan 27, 2017 at 2:50 PM, Katherine Beaulieu
 wrote:
> Ok so there is no way to access the galaxy object outside of galaxy? I'm
> looking for something like the 'trans' variable often seen throughout the
> Galaxy code base so that I can use sqlalchemy and make a database query.
> Thanks for replying.
> Katherine
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Check running jobs

2017-01-27 Thread Peter Cock
Hi Katherine,

I don't think Galaxy itself will tell you this, in part because the
details would be dependent on the Cluster being used.

I would do that via whatever cluster management system you are running
(e.g. qstat or qmon if using SGE).

By default all the jobs will be submitted by the Galaxy user account,
but if you are submitting jobs as the individual users then you can
still spot Galaxy jobs via their naming convention (starting "g",
number, underscore, ..., tool name, user's email).

Peter

On Thu, Jan 26, 2017 at 6:58 PM, Katherine Beaulieu
 wrote:
> Hi there,
> We are trying to build our own Galaxy cluster and I was wondering if there
> was a way to check if there are any running Galaxy jobs on a specific
> server? This would be to determine whether or not to remove the server from
> the cluster.
> Thanks!
> Katherine
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Blast db permission

2017-01-19 Thread Peter Cock
Hello Mohamed,

Right now this is not possible, and I can't think of any way to
restrict access to system level BLAST databases listed in
the blast*.loc files by user. However, since other Galaxy
tools use *.loc files, someone else may have some ideas.

You might be able to try using a BLAST database in a
shared data library (Galaxy has a role based permissions
system you could use), but the user would first have to
import the database into their current history.

I hope that helps,

Peter

On Thu, Jan 19, 2017 at 1:45 PM, Mohamed Kassam  wrote:
> Dear all,
>
> I would like to know if it is possible to put permission on the macro.xml
> for ncbi-blast + database.
> For example if I want to show the db to one user and not to all.
>
> Best reagards,
>
> Mohamed
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Testing updated NCBI BLAST+ wrappers for version 2.5.0

2016-12-08 Thread Peter Cock
Hello all,

I have updated the NCBI BLAST+ wrappers on the Test Tool Shed,
the wrapper is now at v0.2.00:

https://testtoolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus/

The main changes is this now depends on BLAST+ 2.5.0, and that is
available via either BioConda or the Tool Shed:

https://toolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_5_0/
https://testtoolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_5_0/

In order for the dependency to work smoothly on both BioConda
and the Tool Shed system, we have changed the package name
from "blast+" to just "blast". Given the NCBI stopped updated the
original "legacy" BLAST some time ago, when combined with the
version number this is no longer ambiguous.

Jumping from using BLAST+ 2.2.31 to using BLAST+ 2.5.0
required updating lots of the test files for NCBI changes, including
dropping the GI numbers in many outputs, expanding the percentage
identity field from 2dp to 3dp, and also changing how -parse_deflines
works with tabular output.

The wrappers (deliberately) do not yet offer any new functionality
added in the recent NCBI BLAST+ updates, in particular BLAST
XML v2 is not yet available as an output with a datatype in Galaxy.

At this point I would welcome feedback from those of you using the
BLAST+ wrappers - including if you were able to install this with the
dependencies from BioConda or the traditional Tool Shed packages.

Once I'm confident that this is all OK, I will update the main Tool Shed
(and think about adding new functionality in 2017).

Thank you all,

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Unable to upload bam files as links to my local galaxy instance - please help!

2016-12-05 Thread Peter Cock
Galaxy itself needs samtools, and this is not handled via the
Tool Shed which only does dependencies of Galaxy Tools.

I don't think the website is clear enough about this,
https://wiki.galaxyproject.org/Admin/GetGalaxy
just points at
https://wiki.galaxyproject.org/Admin/Tools/ToolDependencies

Which version of Galaxy are you using? Older versions had
to have an older samtools (which had a different API for
using samtools sort).

Peter

On Mon, Dec 5, 2016 at 12:36 PM, Konstantinos Voutetakis  wrote:
> Dear Sir(s),
>
> I have consumed a few days in order to resolve this issue but nothing! I
> have installed all the SAM tools of public Galaxy in my local galaxy via the
> Toolshed (installing automatically all the dependencies were needed). I have
> also export the path of the executable file SAMtools from bin folder to my
> Galaxy path and I take the following error during uploading:
>
> Traceback (most recent call last):
>   File "/home/user/galaxy/tools/data_source/upload.py", line 434, in
> 
> __main__()
>   File "/home/user/galaxy/tools/data_source/upload.py", line 423, in
> __main__
> add_file( dataset, registry, json_file, output_path )
>   File "/home/user/galaxy/tools/data_source/upload.py", line 317, in
> add_file
> if datatype.dataset_content_needs_grooming( dataset.path ):
>   File "/home/user/galaxy/lib/galaxy/datatypes/binary.py", line 251, in
> dataset_content_needs_grooming
> version = self._get_samtools_version()
>   File "/home/user/galaxy/lib/galaxy/datatypes/binary.py", line 212, in
> _get_samtools_version
> raise Exception(message)
> Exception: Attempting to use functionality requiring samtools, but it cannot
> be located on Galaxy's PATH.
>
> When I am in my galaxy folder (from where I connect to my galaxy server) and
> I type samtools in the terminal I take:
> Program: samtools (Tools for alignments in the SAM format)
> Version: 0.1.19-44428cd
>
> Usage:   samtools  [options]
>
> Command: viewSAM<->BAM conversion
>  sortsort alignment file
>  mpileup multi-way pileup
>  depth   compute the depth
>  faidx   index/extract FASTA
>  tview   text alignment viewer
>  index   index alignment
>  idxstatsBAM index stats (r595 or later)
>  fixmate fix mate information
>  flagstatsimple stats
>  calmd   recalculate MD/NM tags and '=' bases
>  merge   merge sorted alignments
>  rmdup   remove PCR duplicates
>  reheaderreplace BAM header
>  cat concatenate BAMs
>  bedcov  read depth per BED region
>  targetcut   cut fosmid regions (for fosmid pool only)
>  phase   phase heterozygotes
>  bamshuf shuffle and group alignments by name
>
> What can I do? I installed galaxy to my pc (ubuntu 16.04) before two weeks
> and it is up to date. I would be grateful for your help.
>
> Yours sincerely,
> Kostas
>
> --
> Konstantinos G. Voutetakis, MSc - Research Officer
> National Hellenic Research Foundation (N.H.R.F.)
> Institute of Biology, Medicinal Chemistry and Biotechnology
> 48 Vas. Constantinou Ave., Athens 11635
> Greece
> --
> 
> tel.: +30-210-7273894
> e-mail: kvou...@eie.gr
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Moving Galaxy Install From One Server To Another

2016-11-23 Thread Peter Cock
On Wed, Nov 23, 2016 at 10:49 AM, Peter Briggs
 wrote:
> Hello Evan, Hans-Rudolf
>
> I'm just in the middle of doing a similar migration for our local production
> server, and Hans-Rudolf's advice seems sound to me. Definitely moving the
> core Galaxy server has been relatively straightforward.
>
> However: for the installed tools, I'm not sure that changing the paths in
> the env.sh files is sufficient - in our installation the absolute paths
> seemed to be baked into a lot of other files under the 'tool_dependencies'
> directory - including things like compiled files (e.g. static and shared
> libraries). So for many of the tools I wouldn't feel confident that they
> would still work after the move.

Yes, I've seen that with various third party tool installations (out side of
Galaxy), so sadly you cannot in general move the files to a different path :(

I don't know if we can do anything about this directly in Galaxy, or even
in BioConda?

> My plan has been to reinstall each of the tools from the toolshed (i.e.
> uninstall via the admin interface then reinstall the same tool repository
> revision(s) using the API), but I don't feel able to recommend this approach
> either as in my testing this has also had problems - ranging from some tool
> revisions no longer being available, through to more serious issues (such as
> tool dependencies which used to work but since become broken). I figured I'd
> just have to knuckle down and work through each problem as I encountered it.

One simple reason for this is stale URLs, where Galaxy's cache can help
but is another step for tool wrapper authors to do when setting up a new
Galaxy dependency: https://github.com/galaxyproject/cargo-port

However, even well intentioned Tool Shed updates could also break things :(

> If anyone else has experiences with these kinds of migrations then I'm also
> very interested to know what worked (and what didn't)!

Sorry - the closest I've been is setting up a new Galaxy server in
parallel with our old server, and manually installing "missing" tools
via the Tool Shed.

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Code File Execution

2016-11-21 Thread Peter Cock
Are you asking how Galaxy takes the user-entered parameters
(requested in the tool XML file with  tags) and builds a
command line string using the Cheetah-format template defined
in the  tag?

At least part of the answer to that is the ToolEvaluator object:

https://github.com/galaxyproject/galaxy/blob/71cea6604d43c5fe6215f5656462ba6c1af69bb6/lib/galaxy/tools/evaluation.py#L454

Peter

On Mon, Nov 21, 2016 at 6:37 PM, Katherine Beaulieu
 wrote:
> Hi Everyone,
> Does anyone know how execution of code files happens? Is there a function
> somewhere that fills the parameters in?
> Cheers,
> Katherine
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Reporting issues for the wiki? e.g. universe_wgsi.ini still widely used

2016-11-15 Thread Peter Cock
Thanks Dannon,

I've logged the universe_wsgi.ini issue there as
https://github.com/galaxyproject/galaxy-site/issues/16

Interesting to see yet another MediaWiki site being moved to
GitHub with Markdown - although not with GitHub pages in this
case. I'd guess you guys used the wonderful pandoc for the
conversion?

I assume you'll be able to add "edit" buttons to pages pointing
at the markdown pages on GitHub? While it could be prettier,
this works pretty well for us using GitHub Pages:

https://github.com/biopython/biopython.github.io/pull/78

Regards,

Peter

On Tue, Nov 15, 2016 at 3:10 PM, Dannon Baker <dannon.ba...@gmail.com> wrote:
> Hi Peter,
>
> We're in the middle of migrating the wiki right now -- you may notice that
> the current wiki is in read-only mode.  Going forward, the place to report
> (and fix) issues like that will be
> https://github.com/galaxyproject/galaxy-site.
>
> -Dannon
>
> On Tue, Nov 15, 2016 at 10:06 AM, Peter Cock <p.j.a.c...@googlemail.com>
> wrote:
>>
>> Hello all,
>>
>> Is there a specific issue tracker for the wiki? It seems wrong to
>> open an issue on the main GitHub code repository...
>>
>> e.g. Many of the documentation pages still refer to the legacy
>> configuration filename universe_wgsi.ini rather than the new
>> name of config/galaxy.ini
>>
>> Problem pages include:
>>
>> https://wiki.galaxyproject.org/Learn/SecurityFeatures
>> https://wiki.galaxyproject.org/Admin/DiskQuotas
>> https://wiki.galaxyproject.org/Admin/SampleTracking/Demo
>> https://wiki.galaxyproject.org/Admin/Tools/MultipleOutputFiles
>> https://wiki.galaxyproject.org/Admin/Config/GenomeSpace
>> https://wiki.galaxyproject.org/Admin/Config/ApacheExternalUserAuth
>> https://wiki.galaxyproject.org/VisualizationSetup
>> https://wiki.galaxyproject.org/ToolShedApi
>>
>> Other pages would benefit from minor edits to at least put the old form
>> second and in brackets:
>>
>> https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster
>> https://wiki.galaxyproject.org/Admin/Interface
>> https://wiki.galaxyproject.org/ToolShed/InstallingRepositoriesToGalaxy
>> Some pages look fine, e.g.
>>
>> https://wiki.galaxyproject.org/ToolShed/InstallingAndCompilingPackages
>>
>> Or are clearly dated and appropriate for the time, e.g. Dev news briefs
>>
>> https://wiki.galaxyproject.org/DevNewsBriefs/2010_07_16
>>
>> This means lots of pages with the term which are fine:
>>
>>
>> https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax?action=fullsearch=0=0=universe_wsgi.ini
>>
>> Regards,
>>
>> Peter
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Reporting issues for the wiki? e.g. universe_wgsi.ini still widely used

2016-11-15 Thread Peter Cock
Hello all,

Is there a specific issue tracker for the wiki? It seems wrong to
open an issue on the main GitHub code repository...

e.g. Many of the documentation pages still refer to the legacy
configuration filename universe_wgsi.ini rather than the new
name of config/galaxy.ini

Problem pages include:

https://wiki.galaxyproject.org/Learn/SecurityFeatures
https://wiki.galaxyproject.org/Admin/DiskQuotas
https://wiki.galaxyproject.org/Admin/SampleTracking/Demo
https://wiki.galaxyproject.org/Admin/Tools/MultipleOutputFiles
https://wiki.galaxyproject.org/Admin/Config/GenomeSpace
https://wiki.galaxyproject.org/Admin/Config/ApacheExternalUserAuth
https://wiki.galaxyproject.org/VisualizationSetup
https://wiki.galaxyproject.org/ToolShedApi

Other pages would benefit from minor edits to at least put the old form
second and in brackets:

https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster
https://wiki.galaxyproject.org/Admin/Interface
https://wiki.galaxyproject.org/ToolShed/InstallingRepositoriesToGalaxy
Some pages look fine, e.g.

https://wiki.galaxyproject.org/ToolShed/InstallingAndCompilingPackages

Or are clearly dated and appropriate for the time, e.g. Dev news briefs

https://wiki.galaxyproject.org/DevNewsBriefs/2010_07_16

This means lots of pages with the term which are fine:

https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax?action=fullsearch=0=0=universe_wsgi.ini

Regards,

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Installing ToolShed Tools into Galaxy via Browser Interface Exception Thrown

2016-11-03 Thread Peter Cock
Good news - this is the bug I though it was, the ampersand
should be escaped as  in the XML file:

https://github.com/galaxyproject/galaxy/issues/3084

More good news: This will likely be fixed in the next release
of Galaxy:

https://github.com/galaxyproject/galaxy/pull/3122

Right now I suggest you manually edit these XML files to
use the word "and" instead of "&".

Peter


On Thu, Nov 3, 2016 at 2:36 PM, Yip, Miu ki <m...@cshl.edu> wrote:
> Hi Peter,
>
> Line 44 is oddly specific. Nevertheless, we have this as the 44th line in the 
> xml file.
>
>  version="" />
>
> This tool is not the one trying to be installed.
>
> Thanks,
> Miuki
> On Nov 3, 2016, at 10:28 AM, Peter Cock 
> <p.j.a.c...@googlemail.com<mailto:p.j.a.c...@googlemail.com>> wrote:
>
> I think you may have hit the same bug I found recently:
>
> https://github.com/galaxyproject/galaxy/issues/3084
>
> What is line 44 of your integrated_tool_panel.xml file?
>
> Peter
>
> On Thu, Nov 3, 2016 at 2:16 PM, Yip, Miu ki <m...@cshl.edu> wrote:
> Hi all,
>
> While trying to install tools from the toolshed via the browser interface, an 
> error keeps appearing and nothing is abled to be installed. Currently using 
> the latest version of Galaxy and having tried to install several different 
> tools, the following error keeps appearing in the log file.
>
> galaxy.queue_worker DEBUG 2016-11-03 13:08:32,425 Executing toolbox reload on 
> 'main'
> Exception in thread Thread-2:
> Traceback (most recent call last):
>  File "/usr/lib64/python2.7/threading.py", line 811, in __bootstrap_inner
>self.run()
>  File "/usr/lib64/python2.7/threading.py", line 764, in run
>self.__target(*self.__args, **self.__kwargs)
>  File 
> "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/watcher.py", line 
> 147, in on_any_event
>self._handle(event)
>  File 
> "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/watcher.py", line 
> 150, in _handle
>self.reload_callback()
>  File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
> line 78, in 
>self._tool_conf_watcher = get_tool_conf_watcher( lambda: 
> reload_toolbox(app))
>  File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/queue_worker.py", line 
> 56, in reload_toolbox
>app.toolbox = _get_new_toolbox(app)
>  File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/queue_worker.py", line 
> 72, in _get_new_toolbox
>new_toolbox = tools.ToolBox(tool_configs, app.config.tool_path, app, 
> app.toolbox._tool_conf_watcher)
>  File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/__init__.py", line 
> 110, in __init__
>tool_conf_watcher=tool_conf_watcher
>  File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
> line 1037, in __init__
>super(BaseGalaxyToolBox, self).__init__(config_filenames, tool_root_dir, 
> app, tool_conf_watcher=tool_conf_watcher)
>  File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
> line 69, in __init__
>self._init_integrated_tool_panel( app.config )
>  File 
> "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/integrated_panel.py",
>  line 36, in _init_integrated_tool_panel
>self._load_integrated_tool_panel_keys()
>  File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
> line 368, in _load_integrated_tool_panel_keys
>tree = parse_xml( self._integrated_tool_panel_config )
>  File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/util/__init__.py", line 
> 211, in parse_xml
>root = tree.parse( fname, parser=ElementTree.XMLParser( 
> target=DoctypeSafeCallbackTarget() ) )
>  File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 656, in parse
>parser.feed(data)
>  File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1642, in feed
>self._raiseerror(v)
>  File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1506, in 
> _raiseerror
>raise err
> ParseError: not well-formed (invalid token): line 44, column 71
>
> Is there an XML or configuration file which needs to be changed for this to 
> work? Not sure how to proceed.
>
> Thanks!
> Miuki
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Installing ToolShed Tools into Galaxy via Browser Interface Exception Thrown

2016-11-03 Thread Peter Cock
I think you may have hit the same bug I found recently:

https://github.com/galaxyproject/galaxy/issues/3084

What is line 44 of your integrated_tool_panel.xml file?

Peter

On Thu, Nov 3, 2016 at 2:16 PM, Yip, Miu ki  wrote:
> Hi all,
>
> While trying to install tools from the toolshed via the browser interface, an 
> error keeps appearing and nothing is abled to be installed. Currently using 
> the latest version of Galaxy and having tried to install several different 
> tools, the following error keeps appearing in the log file.
>
> galaxy.queue_worker DEBUG 2016-11-03 13:08:32,425 Executing toolbox reload on 
> 'main'
> Exception in thread Thread-2:
> Traceback (most recent call last):
>   File "/usr/lib64/python2.7/threading.py", line 811, in __bootstrap_inner
> self.run()
>   File "/usr/lib64/python2.7/threading.py", line 764, in run
> self.__target(*self.__args, **self.__kwargs)
>   File 
> "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/watcher.py", line 
> 147, in on_any_event
> self._handle(event)
>   File 
> "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/watcher.py", line 
> 150, in _handle
> self.reload_callback()
>   File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
> line 78, in 
> self._tool_conf_watcher = get_tool_conf_watcher( lambda: 
> reload_toolbox(app))
>   File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/queue_worker.py", line 
> 56, in reload_toolbox
> app.toolbox = _get_new_toolbox(app)
>   File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/queue_worker.py", line 
> 72, in _get_new_toolbox
> new_toolbox = tools.ToolBox(tool_configs, app.config.tool_path, app, 
> app.toolbox._tool_conf_watcher)
>   File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/__init__.py", line 
> 110, in __init__
> tool_conf_watcher=tool_conf_watcher
>   File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
> line 1037, in __init__
> super(BaseGalaxyToolBox, self).__init__(config_filenames, tool_root_dir, 
> app, tool_conf_watcher=tool_conf_watcher)
>   File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
> line 69, in __init__
> self._init_integrated_tool_panel( app.config )
>   File 
> "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/integrated_panel.py",
>  line 36, in _init_integrated_tool_panel
> self._load_integrated_tool_panel_keys()
>   File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/tools/toolbox/base.py", 
> line 368, in _load_integrated_tool_panel_keys
> tree = parse_xml( self._integrated_tool_panel_config )
>   File "/mnt/sdd1/nondocker_galaxy/galaxy/lib/galaxy/util/__init__.py", line 
> 211, in parse_xml
> root = tree.parse( fname, parser=ElementTree.XMLParser( 
> target=DoctypeSafeCallbackTarget() ) )
>   File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 656, in parse
> parser.feed(data)
>   File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1642, in feed
> self._raiseerror(v)
>   File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1506, in 
> _raiseerror
> raise err
> ParseError: not well-formed (invalid token): line 44, column 71
>
> Is there an XML or configuration file which needs to be changed for this to 
> work? Not sure how to proceed.
>
> Thanks!
> Miuki
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] odose

2016-10-25 Thread Peter Cock
You might find this work useful for fetching bacteria from the NCBI:

https://github.com/kblin/ncbi-genome-download

Peter

On Tue, Oct 25, 2016 at 12:54 PM, Vos, Michiel  wrote:
>
>
> Dear Galaxy developers,
>
>
>
> We hereby want to ask if anyone is interested in helping out to repair and
> update our Galaxy tool ‘odose’:
>
>
>
> Webserver
>
>
>
> Publication
>
>
>
> Public server page
>
>
>
> Github
>
>
>
> Odose enables users to perform an array of population genetics tests on
> prokaryote genomes. Our tool is currently linked to NCBI to download various
> files, but continuous restructuring at NCBI has resulted in broken links
> which prevent the pipeline from working. Developer Tim te Beek has moved on
> outside academia and does not have the time to fix it, but is available to
> give answer question/give instructions. We believe our tool is highly useful
> for microbiologists and would greatly appreciate any help from the Galaxy
> community to get it working properly again! Please get in touch if you would
> like to know more.
>
>
>
> Best Wishes, Michiel Vos, Mark van Passel and Tim te Beek
>
>
>
>
>
>
>
> Michiel Vos
>
> European Centre for Environment and Human Health
>
> University of Exeter
>
> ESI Building, Penryn Campus
>
> TR10 9FE Penryn, UK
>
> 0044 (0)1326259464
>
> https://coastalpathogens.wordpress.com/
>
>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Tool needs a particular file extension -- now Datatype help

2016-10-22 Thread Peter Cock
Hi Steve,

You are on the right track, but something in the WAV file has
triggered one of Galaxy's security protections to try to block
uploading of potentially dangerous files. There may be some
settings here you can relax - I've not had to deal with this
myself.

Peter

On Fri, Oct 21, 2016 at 8:55 PM, Steve Cassidy  wrote:
> Thanks all,
>   it seems that my real problem is that the audio file (.wav) is not being
> identified as a valid datatype and ending up as a zero length text file. So,
> I need to start to explore the world of datatypes.
>
> Following the docs
> (https://wiki.galaxyproject.org/Admin/Datatypes/Adding%20Datatypes) I can
> modify datatypes_conf.xml in my Galaxy sources and add a new datatype for
> wav files:
>
>  display_in_upload="true" mimetype="audio/wav" subclass="True”/>
>
> but, I get a message "The uploaded binary file contains inappropriate
> content” and a zero length file just as I did before adding this - although
> the datatype is now set to ‘wav’.
>
> I didn’t add a sniffer for this and set the datatype explicitly on upload.
>
> Also, this doesn’t seem like a modular way to add datatypes - how do I
> include datatypes in my tool definition?  I can see from some other tools
> that I include a datatypes_conf.xml in my tool folder.   When I try that and
> test with planemo the new type isn’t found.
>
> Pointers welcome.
>
> Thanks,
>
> Steve
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Tool needs a particular file extension

2016-10-21 Thread Peter Cock
Using a soft link for this is a common pattern, and should be followed with &&
(ideally using XML CDATA to avoid escaping everything like  etc),
and quote the filenames just in case there are any spaces. e.g.

https://github.com/galaxyproject/tools-iuc/blob/master/tools/trinity/run_de_analysis.xml#L16


For reference, in tools-iuc there are over 400 soft link examples:

$ grep "ln -s" tools/*/*.xml | wc -l
 446

Peter

On Fri, Oct 21, 2016 at 5:48 PM, Steve Cassidy  wrote:
> Hi,
>  I’m wrapping a tool that needs it’s input to have a known file extension
> (an audio file, eg. .wav).  Since Galaxy stores all data as .dat files the
> tool is falling over since it doesn’t know what .dat is.
>
> I thought I’d be able to get around this by hard linking the .dat file to
> the same name with a .wav extension (dataset_1.dat.wav), this works when I
> try it with the tool on the command line but within Galaxy it fails, here’s
> my :
>
> ln $signal ${signal}.wav 
> /home/maus/maus OUTFORMAT=TextGrid LANGUAGE=$language
> BPF=$bpf INSKANTEXTGRID=$inskantextgrid
> INSORTTEXTGRID=$insorttextgrid
> MODUS=$modus MAUSSHIFT=$mausshift MINPAUSLEN=$minpauslen
> WEIGHT=$weight
> INSPROB=$insprob NOINITIALFINALSILENCE=$noinitialfinalsilence
> OUTSYMBOL=$outsymbol
> OUT=$output SIGNAL=${signal}.wav
>
> resulting in the job command line:
>
> ln /tmp/tmp7AZvx7/files/000/dataset_2.dat
> /tmp/tmp7AZvx7/files/000/dataset_2.dat.wav & /home/maus/maus
> OUTFORMAT=TextGrid LANGUAGE=aus BPF=/tmp/tmp7AZvx7/files/000/dataset_1.dat
> INSKANTEXTGRID=false INSORTTEXTGRID=false MODUS=standard MAUSSHIFT=10
> MINPAUSLEN=5 WEIGHT=7.0 INSPROB=0.0 NOINITIALFINALSILENCE=no OUTSYMBOL=sampa
> OUT=/tmp/tmp7AZvx7/files/000/dataset_3.dat
> SIGNAL=/tmp/tmp7AZvx7/files/000/dataset_2.dat.wav
>
> I’m getting an error message from the tool:
>
> sox FAIL formats: can't open input file
> `/tmp/tmp7AZvx7/files/000/dataset_2.dat.wav': WAVE: RIFF header not found
>
> this suggests that the hard link didn’t get made.  I tried copying the file
> instead but got the same result.
>
> I could go in and patch the tool script to be more forgiving but it would be
> good to find a solution that didn’t require that if possible.
>
> Any pointers appreciated.
>
> Steve
> —
> Department of Computing, Macquarie University
> http://web.science.mq.edu.au/~cassidy
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Tool.xml : correct way to define an environment_variable ?

2016-10-20 Thread Peter Cock
Hi Pierre,

The first example looks OK, very similar to this one of mine for example,
so I am not sure what is breaking:

https://github.com/peterjc/pico_galaxy/blob/133aea82c02adf5509134c415565ada99c519312/tools/effectiveT3/tool_dependencies.xml

I would try adding more shell actions to debug this with echo statements?

However, rather than using wget you could use the download_file
action which can verify the checksum, and will also work on a Mac
where wget is usually missing.

Peter

On Thu, Oct 20, 2016 at 8:03 AM, Pierre Lindenbaum
 wrote:
> (cross-posted at https://biostar.usegalaxy.org/p/20220/ )
>
>
> I'm trying to create some `tools.xml` that use java.
>
> Those tools need to download some library (*.jar) files from
> http://central.maven.org/ and I want to create a variable $MYTOOLCLASSPATH
> containing the jars. I've tried the following xml :
>
> 
> 
>   
> 
>   
> wget -P
> "$REPOSITORY_INSTALL_DIR" "http://central.maven.org/lib1.jar;
> "http://central.maven.org/lib2.jar;
> 
>  action="set_to">$REPOSITORY_INSTALL_DIR/lib1.jar:$REPOSITORY_INSTALL_DIR/lib2.jar
> 
>   
> 
>   
> 
>
> the base java command is defined as a token in macro.xml:
>
>   (...)
>   
> java   -cp "\${MYTOOLCLASSPATH}"
>   
>   (...)
>
> and called in my tool.xml:
>
> @MYJAVA@ tool.Tool ${input}  ${out} 
>
> but It seems that $MYTOOLCLASSPATH  is empty.
>
> I've also tried to create a set_environment under :
>
> 
> action="set_to">$REPOSITORY_INSTALL_DIR/lib1.jar:$REPOSITORY_INSTALL_DIR/lib2.jar
> 
>
>
> same problem: the $MYTOOLCLASSPATH remains empty
>
> Where Am i wrong ? What's the correct way to define MYTOOLCLASSPATH ?
>
>
>
> Thanks,
>
> Pierre
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] galaxy - prokka - tbl2asn

2016-10-19 Thread Peter Cock
This is an annoyance from the NCBI team, and dealing with the
expired tbl2asn tool is a regular question from Prokka users, eg:

https://github.com/tseemann/prokka/issues/96
https://github.com/tseemann/prokka/issues/139

The Tool Shed package for Prokka will need updating - but
luckily Nicola is already working on that:

https://github.com/galaxyproject/tools-iuc/pull/985

I'm not sure if this will work, but you can try manually installing
the latest tbl2asn from the NCBI (and if need be, copy the binary
on top of the expired binary from the Tool Shed - messy!):

ftp://ftp.ncbi.nih.gov/toolbox/ncbi_tools/converters/by_program/tbl2asn/

Peter

On Tue, Oct 18, 2016 at 7:19 PM, Fernandez Edgar
 wrote:
> Hey Guys,
>
>
>
> I’ve fixed this problem by bypassing the system perl and compiling my own
> with all the right modules.
>
> So I don’t get any more perl errors.
>
>
>
> Also, I’ve realized that the latest prokka package (Revision: f5e44aad6498)
> has a dependency with the package_tbl2_asn_24_3 (Revision 41764d6a6a3c).
>
>
>
> However, when you execute prokka, tbl2asn gives you the following error:
>
> [tbl2asn] This copy of tbl2asn is more than a year old.  Please download the
> current version.
>
>
>
> What would you suggest I do?
>
>
>
> Edgar Fernandez
>
> System Administrator (Linux)
>
> Direction Générale des Technologies de l'Information et de la Communication
>
> Université de Montréal
>
>
>
> PAVILLON ROGER-GAUDRY, bureau X-210
>
> (  Bur. : 1-514-343-6111 poste 16568
>
>
>
>
>
> From: Nicola Soranzo [mailto:nicola.sora...@gmail.com] On Behalf Of Nicola
> Soranzo
> Sent: October-07-16 3:14 PM
> To: Fernandez Edgar ; galaxy-...@bx.psu.edu
> Subject: Re: [galaxy-dev] galaxy - bioperl - prokka
>
>
>
> Hi Edgar,
> presently the Prokka wrapper requires some dependencies (in particular some
> Perl modules) to be installed separetely, see the README file at
> https://toolshed.g2.bx.psu.edu/view/crs4/prokka/
>
> We are planning to release an updated Prokka wrapper which will use Conda
> dependencies, you can track the progress at:
>
> https://github.com/galaxyproject/tools-iuc/pull/985
>
> Cheers,
> Nicola
>
> On 07/10/16 15:06, Fernandez Edgar wrote:
>
> Hello,
>
>
>
> I have installed prokka, perl and bioperl in my Galaxy’s ToolShed.
>
> However, my users are getting this error during execution of prokka:
>
> Can't locate Bio/Root/Version.pm in @INC (@INC contains:
> /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl
> /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at
> /home/galaxy/galaxy-prod/tool-data/toolshed.dependency.dir/prokka/1.11/crs4/prokka/f5e44aad6498/bin/prokka
> line 29. BEGIN failed--compilation aborted at
> /home/galaxy/galaxy-prod/tool-data/toolshed.dependency.dir/prokka/1.11/crs4/prokka/f5e44aad6498/bin/prokka
> line 29.
>
>
>
> I was under the impression that the galaxy’s tools will use the internal
> perl.
>
> Do I just need to install Bio::Root:Version on my system installation of
> perl?
>
> Please give me whatever information you can on this issue.
>
>
>
> Regards,
>
>
>
> Edgar Fernandez
>
> System Administrator (Linux)
>
> Direction Générale des Technologies de l'Information et de la Communication
>
> Université de Montréal
>
>
>
> PAVILLON ROGER-GAUDRY, bureau X-210
>
> (  Bur. : 1-514-343-6111 poste 16568
>
>
>
>
>
>
>
>
> ___
>
> Please keep all replies on the list by using "reply all"
>
> in your mail client.  To manage your subscriptions to this
>
> and other Galaxy lists, please use the interface at:
>
>   https://lists.galaxyproject.org/
>
>
>
> To search Galaxy mailing lists use the unified search at:
>
>   http://galaxyproject.org/search/mailinglists/
>
>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Python ValueError: too many values to unpack

2016-10-18 Thread Peter Cock
I just found this old email by a very similar exception logged as:

https://github.com/galaxyproject/galaxy/issues/2987

There do seem to be some corner cases where missing default
values are not handled nicely.

Peter


On Tue, Jan 26, 2016 at 2:38 PM, Matthias De Smet
 wrote:
> Wow, great tip! This fixed it!
>
> Thanks a bunch!
> Matthias
>
>
>> Op 26 jan. 2016, om 15:20 heeft Hans-Rudolf Hotz  het volgende 
>> geschreven:
>>
>> Hi Matthias
>>
>>
>> Make sure 'listing' is a list of (displayName,value,boolean)
>>
>> Though, there might be other errors as well
>>
>>
>> Regards, Hans-Rudolf
>>
>> I am not sure, whether this
>> On 01/26/2016 01:22 PM, Matthias De Smet wrote:
>>> Hi all,
>>>
>>> I’m trying to develop a tool that lets you select a file on the server.
>>> I’m using the dynamic_options setting to let Galaxy render a dropdown
>>> list of available file.
>>>
>>>
>>> However, when I try to test the tool in its current form, I get the
>>> following error:
>>>
>>> 
>>> galaxy.web.framework.decorators ERROR 2016-01-26 13:08:35,649 Uncaught
>>> exception in exposed API method:
>>> Traceback (most recent call last):
>>>   File "lib/galaxy/web/framework/decorators.py", line 260, in decorator
>>> rval = func( self, trans, *args, **kwargs)
>>>   File "lib/galaxy/webapps/galaxy/api/tools.py", line 98, in build
>>> return tool.to_json(trans, kwd.get('inputs', kwd))
>>>   File "lib/galaxy/tools/__init__.py", line 2602, in to_json
>>> populate_state(trans, self.inputs, state_inputs, state_errors,
>>> params.__dict__)
>>>   File "lib/galaxy/tools/__init__.py", line 2471, in populate_state
>>> state[input.name] = input.get_initial_value(trans, context,
>>> history=history)
>>>   File "lib/galaxy/tools/parameters/basic.py", line 984, in
>>> get_initial_value
>>> value = [ optval for _, optval, selected in options if selected ]
>>> ValueError: too many values to unpack
>>> 
>>>
>>> My tool xml looks like this:
>>>
>>> 
>>>
>>> 
>>>   from the archive to a data library
>>>   
>>>   
>>> test.sh > $log
>>>   
>>>   
>>> >> label="Choose source archive">
>>>   
>>> 
>>> 
>>>   
>>> 
>>> 
>>> >> dynamic_options="list_files(archive_source,query)"/>
>>> 
>>>   
>>> 
>>>  
>>>   
>>> 
>>> 
>>>   
>>>   
>>> This tool copies data from and archive folder to your history. Keep in
>>> mind this folder MUST be readable by Galaxy in order for this tool to work.
>>>   
>>> 
>>>
>>> 
>>>
>>> and the script called in the “code” tag looks like this
>>>
>>> 
>>> def list_files(dir,query): listing=[]
>>> for fname in listdir(dir):
>>>  fullpath = path.join(dir, fname) #only select datafiles, not md5
>>> checksums
>>>  if path.isfile(fullpath) and not fullpath.endswith(".md5"):
>>>  listing.append( fname )
>>> return listing
>>> 
>>>
>>> This script should return a list with filenames that should be displayed
>>> in the dropdown list.
>>>
>>> I’m tracking Galaxy release 15.10, running on a Ubuntu 14.04.3 server
>>>
>>> Thanks for your help!
>>>
>>> Matthias
>>>
>>>
>>>
>>>
>>>
>>>
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>   https://lists.galaxyproject.org/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>   http://galaxyproject.org/search/mailinglists/
>>>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] planemo test with

2016-10-07 Thread Peter Cock
Good, because you are using planemo test --galaxy_root ...
you can preset the *.loc files as needed (because currently
Planemo will not do this for you).

See also Nicola's reply.

Peter

On Fri, Oct 7, 2016 at 5:49 PM, D K <danielforti...@gmail.com> wrote:
> Hi Peter,
>
> This is the command I'm using to run planemo:
> . galaxy_test/.venv/bin/activate
> planemo test --test_data /home/galaxy-functional-testdata --galaxy_root
> galaxy-test/ --skip_venv test.xml
>
> I'm getting no errors when using the planemo lint tool.
>
> Thanks!
>
> On Fri, Oct 7, 2016 at 2:06 AM, Peter Cock <p.j.a.c...@googlemail.com>
> wrote:
>>
>> Hi DK,
>>
>> Looking at your XML that looks OK.
>>
>> It might not be important, but how exactly are you running planemo
>> test?
>>
>> I'm asking because currently it will not create tool-data/twobit.loc
>> for you from your sample file - although in that case I would have
>> expected a different error message:
>>
>> https://github.com/galaxyproject/planemo/issues/530
>> https://github.com/galaxyproject/planemo/issues/530
>>
>> Is planemo lint happy with your tool?
>>
>> Peter
>>
>>
>> On Thu, Oct 6, 2016 at 11:08 PM, D K <danielforti...@gmail.com> wrote:
>>>
>>> Hi galaxy-dev,
>>>
>>> I'm having a problem running a test using planemo where I would like the
>>> value of a parameter taken from one of the data tables. I get the following
>>> error in planemo:
>>>
>>> 'Error creating a job for these tool inputs - Parameter
>>> refGenomeSource_type requires a value, but has no legal values defined.\n
>>>
>>>
>>> From my script XML:
>>>>
>>>>  
>>>>   
>>>>   
>>>> 
>>>> ...
>>>>   
>>>>   
>>>> 
>>>>  
>>>>   
>>>>
>>> From my "tool_data_table_conf.xml.sample" and
>>> "tool_data_table_conf.xml.test" (mirrored)
>>>>
>>>> >>> allow_duplicate_entries="False">
>>>> value, name, path
>>>> 
>>>> 
>>>
>>>
>>> and from twobit.loc (where the columns are tab separated):
>>>>
>>>> hsapiensH. sapiens (hg38)
>>>> /remote/RMS/users/galaxy/reference_genomes/MFEprimer_index/Homo_sapiens.GRCh38.cdna.all.fa
>>>
>>>
>>> Any suggestions would be greatly appreciated!
>>>
>>>
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>   https://lists.galaxyproject.org/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>   http://galaxyproject.org/search/mailinglists/
>>
>>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] planemo test with

2016-10-07 Thread Peter Cock
Hi DK,

Looking at your XML that looks OK.

It might not be important, but how exactly are you running planemo
test?

I'm asking because currently it will not create tool-data/twobit.loc
for you from your sample file - although in that case I would have
expected a different error message:

https://github.com/galaxyproject/planemo/issues/530
https://github.com/galaxyproject/planemo/issues/530

Is planemo lint happy with your tool?

Peter


On Thu, Oct 6, 2016 at 11:08 PM, D K  wrote:

> Hi galaxy-dev,
>
> I'm having a problem running a test using planemo where I would like the
> value of a parameter taken from one of the data tables. I get the following
> error in planemo:
>
> 'Error creating a job for these tool inputs - Parameter refGenomeSource_type 
> requires a value, but has no legal values defined.\n
>
>
> From my script XML:
>
>>  
>>   
>>   
>> 
>> ...
>>   
>>   
>> 
>>  
>>   
>>
>> From my "tool_data_table_conf.xml.sample" and "tool_data_table_conf.xml.test"
> (mirrored)
>
>> 
>> value, name, path
>> 
>> 
>
>
> and from twobit.loc (where the columns are tab separated):
>
>> hsapiensH. sapiens (hg38)   /remote/RMS/users/galaxy/
>> reference_genomes/MFEprimer_index/Homo_sapiens.GRCh38.cdna.all.fa
>
>
> Any suggestions would be greatly appreciated!
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] [galaxy-iuc] GMAP tool_dependencies.xml package for ToolShed

2016-09-27 Thread Peter Cock
Hi again JJ,

Thanks again for agreeing that we (the IUC) can adopt your
GMAP wrappers under the MIT license.

Apologies for the extra question, but are you happy for the
GMAP datatypes originally defined in your ToolShed repository
https://toolshed.g2.bx.psu.edu/view/jjohnson/gmap/ to be
moved into the Galaxy core, under Galaxy's open source
licence: Academic Free License version 3.0
https://github.com/galaxyproject/galaxy/blob/dev/LICENSE.txt

Thanks,

Peter

On Tue, Sep 20, 2016 at 2:51 PM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
> Thanks JJ,
>
> That's great. We're hoping to use GMAP locally, so I have an incentive
> to tackle this and the dependency glitch with the current versions on
> the Tool Shed.
>
> Cheers,
>
> Peter
>
> On Tue, Sep 20, 2016 at 2:50 PM, Jim Johnson <johns...@umn.edu> wrote:
>> I would greatly appreciate the IUC adopting all gmap/gsnap work that I
>> originally worked on.
>> I do not currently have the time to maintain these tools.
>>
>> Administrative access granted to IUC for:
>>
>> https://toolshed.g2.bx.psu.edu/view/jjohnson/gmap/
>>
>> https://toolshed.g2.bx.psu.edu/view/jjohnson/package_gmap_2013_05_09/
>> https://testtoolshed.g2.bx.psu.edu/view/jjohnson/gmap/
>>
>> https://testtoolshed.g2.bx.psu.edu/view/jjohnson/package_gmap_2013_05_09/
>>
>> These can be copied to the iuc owned repositories and I could deprecate
>> those under my name.
>>
>>
>>
>> On 9/20/16 8:13 AM, Peter Cock wrote:
>>>
>>> Thanks JJ :)
>>>
>>> Could you explicitly confirm that we (the IUC) have your permission
>>> to adopt the gmap wrappers and packages (and any of your other
>>> Galaxy work you might want to suggest), to develop and maintain
>>> under the IUC banner, under the MIT license.
>>>
>>> (Ideally I'd suggest you state this in an email CC'ing the galaxy-dev
>>> list as a public record)
>>>
>>> For now this will mean tracking their development in:
>>> https://github.com/galaxyproject/tools-iuc
>>>
>>> Secondly, would you be willing to grant admin/write access
>>> to the IUC account on the (main and test) tool shed for these:
>>>
>>> https://toolshed.g2.bx.psu.edu/view/jjohnson/gmap/
>>> https://toolshed.g2.bx.psu.edu/view/jjohnson/package_gmap_2013_05_09/
>>>
>>> Thanks,
>>>
>>> Peter
>>>
>>> On Tue, Sep 20, 2016 at 12:47 PM, Jim Johnson <johns...@umn.edu> wrote:
>>>>
>>>> It would be great to move this to IUC.
>>>> I originally put a lot of effort into this for a researcher at MN; I
>>>> think
>>>> it was tried once.
>>>> I now use gmap as part a dependency of the deFuse application.
>>>> Thanks,
>>>> JJ
>>
>>
>>
>> --
>> James E. Johnson Minnesota Supercomputing Institute University of Minnesota
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] [galaxy-iuc] GMAP tool_dependencies.xml package for ToolShed

2016-09-20 Thread Peter Cock
Thanks JJ,

That's great. We're hoping to use GMAP locally, so I have an incentive
to tackle this and the dependency glitch with the current versions on
the Tool Shed.

Cheers,

Peter

On Tue, Sep 20, 2016 at 2:50 PM, Jim Johnson <johns...@umn.edu> wrote:
> I would greatly appreciate the IUC adopting all gmap/gsnap work that I
> originally worked on.
> I do not currently have the time to maintain these tools.
>
> Administrative access granted to IUC for:
>
> https://toolshed.g2.bx.psu.edu/view/jjohnson/gmap/
>
> https://toolshed.g2.bx.psu.edu/view/jjohnson/package_gmap_2013_05_09/
> https://testtoolshed.g2.bx.psu.edu/view/jjohnson/gmap/
>
> https://testtoolshed.g2.bx.psu.edu/view/jjohnson/package_gmap_2013_05_09/
>
> These can be copied to the iuc owned repositories and I could deprecate
> those under my name.
>
>
>
> On 9/20/16 8:13 AM, Peter Cock wrote:
>>
>> Thanks JJ :)
>>
>> Could you explicitly confirm that we (the IUC) have your permission
>> to adopt the gmap wrappers and packages (and any of your other
>> Galaxy work you might want to suggest), to develop and maintain
>> under the IUC banner, under the MIT license.
>>
>> (Ideally I'd suggest you state this in an email CC'ing the galaxy-dev
>> list as a public record)
>>
>> For now this will mean tracking their development in:
>> https://github.com/galaxyproject/tools-iuc
>>
>> Secondly, would you be willing to grant admin/write access
>> to the IUC account on the (main and test) tool shed for these:
>>
>> https://toolshed.g2.bx.psu.edu/view/jjohnson/gmap/
>> https://toolshed.g2.bx.psu.edu/view/jjohnson/package_gmap_2013_05_09/
>>
>> Thanks,
>>
>> Peter
>>
>> On Tue, Sep 20, 2016 at 12:47 PM, Jim Johnson <johns...@umn.edu> wrote:
>>>
>>> It would be great to move this to IUC.
>>> I originally put a lot of effort into this for a researcher at MN; I
>>> think
>>> it was tried once.
>>> I now use gmap as part a dependency of the deFuse application.
>>> Thanks,
>>> JJ
>
>
>
> --
> James E. Johnson Minnesota Supercomputing Institute University of Minnesota
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Send repeat parameter to code file for multiple select list

2016-09-01 Thread Peter Cock
Ah - I understand what you mean now. Using the code tag is
discouraged, but you may find some of these examples useful
for what you are trying to do:

https://github.com/galaxyproject/galaxy/issues/2714

I hope that helps,

Peter



On Thu, Sep 1, 2016 at 2:37 PM, Katherine Beaulieu
<katherine.beaulieu...@gmail.com> wrote:
>
>
> On Thu, Sep 1, 2016 at 9:36 AM, Katherine Beaulieu
> <katherine.beaulieu...@gmail.com> wrote:
>>
>> Hi Peter,
>> The code file tag is generally used in populating select lists. So
>> basically you have a function from a code file that is loaded before runtime
>> of the tool and you can dynamically load content based on that function and
>> you can pass it parameters from your tool but I'm wondering how you would
>> pass it the contents of a repeat parameter.
>> This is how the code file tag works:
>> https://wiki.galaxyproject.org/Admin/Tools/Custom%20Code
>> Thanks,
>> Katherine
>>
>> On Thu, Sep 1, 2016 at 8:23 AM, Peter Cock <p.j.a.c...@googlemail.com>
>> wrote:
>>>
>>> I'm not quite sure what you mean by a "code file", but if you
>>> mean using the  tag, here's an example:
>>>
>>>
>>> https://github.com/peterjc/galaxy_mira/blob/master/tools/mira4_0/mira4_de_novo.xml
>>>
>>> I am using a Cheetah syntax for loop to access each entry
>>> in a repeat parameter.
>>>
>>> Peter
>>>
>>> On Thu, Sep 1, 2016 at 12:58 PM, Katherine Beaulieu
>>> <katherine.beaulieu...@gmail.com> wrote:
>>> > Hello Galaxy Community,
>>> > I was wondering if anyone had ever come across a situation where they
>>> > have
>>> > had to pass the contents of a repeat parameter to a code file in a
>>> > Galaxy
>>> > tool config? Is there any way to do this?
>>> > Katherine
>>> >
>>> > ___
>>> > Please keep all replies on the list by using "reply all"
>>> > in your mail client.  To manage your subscriptions to this
>>> > and other Galaxy lists, please use the interface at:
>>> >   https://lists.galaxyproject.org/
>>> >
>>> > To search Galaxy mailing lists use the unified search at:
>>> >   http://galaxyproject.org/search/mailinglists/
>>
>>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Send repeat parameter to code file for multiple select list

2016-09-01 Thread Peter Cock
I'm not quite sure what you mean by a "code file", but if you
mean using the  tag, here's an example:

https://github.com/peterjc/galaxy_mira/blob/master/tools/mira4_0/mira4_de_novo.xml

I am using a Cheetah syntax for loop to access each entry
in a repeat parameter.

Peter

On Thu, Sep 1, 2016 at 12:58 PM, Katherine Beaulieu
 wrote:
> Hello Galaxy Community,
> I was wondering if anyone had ever come across a situation where they have
> had to pass the contents of a repeat parameter to a code file in a Galaxy
> tool config? Is there any way to do this?
> Katherine
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] how to create users through Galaxy API?

2016-08-30 Thread Peter Cock
Thanks Nicola & Peter B.,

I was hoping we could script new-user creation like this, and
that someone else had already done this with the Galaxy API
(and Bioblend). This looks great!

Thank you both,

Peter C.

On Tue, Aug 30, 2016 at 10:43 AM, Peter Briggs
<peter.bri...@manchester.ac.uk> wrote:
> Hello Peter
>
> Following on from Nicola's suggestion: a while ago I had a go at writing
> some command line tools using Bioblend to try and do some of my Galaxy admin
> tasks via the API, including creating new user accounts so I'd also
> recommend this route as much much easier than attempting to interact with
> the API directly.
>
> The tools I wrote are here if you'd like to look at some an example of using
> Bioblend for this, to get you started:
>
> https://github.com/pjbriggs/nebulizer/blob/master/nebulizer/users.py
>
> As long as the user that the API key belongs to is an admin user (or if
> you're using a master API key) this works fine.
>
> HTH
>
> Best wishes
>
> Peter
>
>
> On 29/08/16 17:37, Nicola Soranzo wrote:
>>
>> Hi Peter,
>> why not use BioBlend?
>>
>>
>> https://bioblend.readthedocs.io/en/latest/api_docs/galaxy/all.html#module-bioblend.galaxy.users
>>
>> Cheers,
>> Nicola
>>
>>  Peter Cock ha scritto 
>>
>>> Thanks Hans-Rudolf,
>>>
>>> Yes - you are right. The wiki does link there but it wasn't obvious to
>>> me.
>>> Reading the UserAPIController link source code suggests as long as
>>> I am authenticated as a Galaxy Admin, I could use this to create new
>>> user accounts:
>>>
>>>
>>> https://docs.galaxyproject.org/en/master/_modules/galaxy/webapps/galaxy/api/users.html#UserAPIController.create
>>>
>>> Thank looks very promising...
>>>
>>> Peter
>>>
>>> On Mon, Aug 29, 2016 at 4:36 PM, Hans-Rudolf Hotz <h...@fmi.ch> wrote:
>>>>
>>>> Hi Peter
>>>>
>>>> I guess it is here:
>>>>
>>>> https://docs.galaxyproject.org/en/master/api_doc.html
>>>>
>>>>
>>>> Regards,  Hans-Rudolf
>>>>
>>>> On 08/29/2016 05:27 PM, Peter Cock wrote:
>>>>>
>>>>>
>>>>> Hi Martin,
>>>>>
>>>>> I'd like to look into creating new users via the API (where we can
>>>>> control the email address and username format for integration with our
>>>>> cluster), and have the web-interface forbid users from
>>>>> self-registration.
>>>>>
>>>>> The read-the-docs site you mentioned in this old email (below) doesn't
>>>>> exist anymore, and the API wiki page doesn't seem to link to something
>>>>> similar:
>>>>>
>>>>> https://wiki.galaxyproject.org/Admin/API
>>>>>
>>>>> Where is the official API documentation now?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Peter
>>>>>
>>>>> On Mon, Dec 30, 2013 at 4:34 AM, Martin Čech <mar...@bx.psu.edu
>>>>> <mailto:mar...@bx.psu.edu>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> you specify username, password and email in the body (payload) of
>>>>> the POST as Key:Value pairs.
>>>>>
>>>>> Code from the API method: (
>>>>>
>>>>>
>>>>> https://galaxy-central.readthedocs.org/en/latest/_modules/galaxy/webapps/galaxy/api/users.html#UserAPIController.create
>>>>>
>>>>>
>>>>> <https://galaxy-central.readthedocs.org/en/latest/_modules/galaxy/webapps/galaxy/api/users.html#UserAPIController.create>
>>>>> )
>>>>>
>>>>>
>>>>>
>>>>> username=payload['username']email=payload['email']password=payload['password']
>>>>>
>>>>> There are also other conditions that need to be fulfilled (e.g.
>>>>> user
>>>>> creation has to be turned on in the configuration) - you will find
>>>>> these when you look at the source code of the method (because the
>>>>> documentation is not perfect yet, sorry).
>>>>>
>>>>> M.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ___
>>>>> Please keep all replies on the list

Re: [galaxy-dev] how to create users through Galaxy API?

2016-08-29 Thread Peter Cock
Thanks Hans-Rudolf,

Yes - you are right. The wiki does link there but it wasn't obvious to me.
Reading the UserAPIController link source code suggests as long as
I am authenticated as a Galaxy Admin, I could use this to create new
user accounts:

https://docs.galaxyproject.org/en/master/_modules/galaxy/webapps/galaxy/api/users.html#UserAPIController.create

Thank looks very promising...

Peter

On Mon, Aug 29, 2016 at 4:36 PM, Hans-Rudolf Hotz <h...@fmi.ch> wrote:
> Hi Peter
>
> I guess it is here:
>
> https://docs.galaxyproject.org/en/master/api_doc.html
>
>
> Regards,  Hans-Rudolf
>
> On 08/29/2016 05:27 PM, Peter Cock wrote:
>>
>> Hi Martin,
>>
>> I'd like to look into creating new users via the API (where we can
>> control the email address and username format for integration with our
>> cluster), and have the web-interface forbid users from self-registration.
>>
>> The read-the-docs site you mentioned in this old email (below) doesn't
>> exist anymore, and the API wiki page doesn't seem to link to something
>> similar:
>>
>> https://wiki.galaxyproject.org/Admin/API
>>
>> Where is the official API documentation now?
>>
>> Thanks,
>>
>> Peter
>>
>> On Mon, Dec 30, 2013 at 4:34 AM, Martin Čech <mar...@bx.psu.edu
>> <mailto:mar...@bx.psu.edu>> wrote:
>>
>> Hi,
>>
>> you specify username, password and email in the body (payload) of
>> the POST as Key:Value pairs.
>>
>> Code from the API method: (
>>
>> https://galaxy-central.readthedocs.org/en/latest/_modules/galaxy/webapps/galaxy/api/users.html#UserAPIController.create
>>
>> <https://galaxy-central.readthedocs.org/en/latest/_modules/galaxy/webapps/galaxy/api/users.html#UserAPIController.create>
>> )
>>
>>
>> username=payload['username']email=payload['email']password=payload['password']
>>
>> There are also other conditions that need to be fulfilled (e.g. user
>> creation has to be turned on in the configuration) - you will find
>> these when you look at the source code of the method (because the
>> documentation is not perfect yet, sorry).
>>
>> M.
>>
>>
>>
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>http://galaxyproject.org/search/mailinglists/
>>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] how to create users through Galaxy API?

2016-08-29 Thread Peter Cock
Hi Martin,

I'd like to look into creating new users via the API (where we can control
the email address and username format for integration with our cluster),
and have the web-interface forbid users from self-registration.

The read-the-docs site you mentioned in this old email (below) doesn't
exist anymore, and the API wiki page doesn't seem to link to something
similar:

https://wiki.galaxyproject.org/Admin/API

Where is the official API documentation now?

Thanks,

Peter

On Mon, Dec 30, 2013 at 4:34 AM, Martin Čech  wrote:

> Hi,
>
> you specify username, password and email in the body (payload) of the POST
> as Key:Value pairs.
>
> Code from the API method: ( https://galaxy-central.
> readthedocs.org/en/latest/_modules/galaxy/webapps/galaxy/api/users.html#
> UserAPIController.create )
>
> username = payload[ 'username' ] email = payload[ 'email' ] password =
> payload[ 'password' ]
>
> There are also other conditions that need to be fulfilled (e.g. user
> creation has to be turned on in the configuration) - you will find these
> when you look at the source code of the method (because the documentation
> is not perfect yet, sorry).
>
> M.
>
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Introducing Conda as a new standard for Galaxy tool dependencies

2016-08-29 Thread Peter Cock
That's great Bjoern,

I was thinking we should mention this on the wiki (with a link
to the documentation rather than duplicating it). There are
some existing pages about the tool_dependencies.xml
system - but no obvious top level introduction that I saw?

https://wiki.galaxyproject.org/ToolDependencyRecipes
https://wiki.galaxyproject.org/ToolShed/PackageRecipes
https://wiki.galaxyproject.org/ToolDependenciesTagSets
https://wiki.galaxyproject.org/ToolsWithDependenciesInSeparateRepositories

Where on the wiki do people think would be best?

Peter

On Sun, Aug 28, 2016 at 9:41 PM, Léo Biscassi  wrote:
> Nice job! I was really anxious for some documentation about this theme!
> Thanks all involved!
>
> On Fri, Aug 26, 2016 at 3:38 PM Björn Grüning 
> wrote:
>>
>> Hi all,
>>
>> Galaxy 16.07 was just released:
>> https://docs.galaxyproject.org/en/master/releases/16.07_announce.html
>> and the IUC has some news to share with you.
>>
>> For a more readable version please see:
>> https://docs.galaxyproject.org/en/master/admin/conda_faq.html
>>
>> 
>>
>> Galaxy tools (also called wrappers) traditionally use Tool Shed package
>> recipes to install their dependencies. At the tool’s installation time
>> the recipe is downloaded and executed in order to provide the underlying
>> software executables. Introduction of these Galaxy-specific recipes was
>> a necessary step at the time, however nowadays there are other more
>> mature and stable options to install software in a similar manner. The
>> Galaxy community has taken steps to improve the tool dependency system
>> in order to enable new features and expand its reach. This document aims
>> to describe these and answer the FAQ.
>>
>> It is a pleasure to announce the adoption of a new standard for tool
>> dependencies in Galaxy which has been integrated over the last six
>> months: Conda packages!
>> Not only do Conda packages make tool dependencies more reliable and
>> stable, they are also easier to test and faster to develop than the
>> traditional Tool Shed package recipes .
>>
>> Conda is a package manager like apt-get, yum, pip, brew or guix. We
>> don’t want to argue about the relative merits of various package
>> managers here, in fact Galaxy supports multiple package managers and we
>> welcome community contributions (such as implementing a Guix package
>> manager or enhancing the existing brew support to bring it on par with
>> Conda).
>>
>> As a community, we have decided that Conda is the one that best fulfills
>> our needs. The following are some of the crucial Conda features that led
>> to this decision:
>>
>> * Installation of packages does not require root privileges
>> (Installation at any location  the Galaxy user has write access to)
>> * Multiple versions of software can be installed in parallel
>> * HPC-ready
>> * Faster and more robust package installations through pre-compiled
>> packages (no build environment complications)
>> * Independent of programming language (R, Perl, Python, Julia, Java,
>> pre-compiled binaries, and more)
>> * Easy to write recipes (1 YAML description file + 1 Bash install script)
>> * An active, large and growing community (with more and more software
>> authors managing their own recipes)
>> * Extensive documentation: Conda documentation
>> (http://conda.pydata.org/docs/building/build.html) and Conda quick-start
>> (http://conda.pydata.org/docs/get-started.html)
>>
>> Below we answer some common questions (collected by Lance Parsons):
>>
>> 1. How do I enable Conda dependency resolution for existing Galaxy
>> installations?
>> ---
>>
>> Most Galaxy administrators have not set up a dependency resolvers
>> configuration file (dependency_resolvers_conf.xml) which means they are
>> using Galaxy’s default (dependency_resolvers_conf.xml.sample). Galaxy
>> has enabled Conda dependency resolution by default since release 16.04
>> (if Conda was installed already), so many existing installations can use
>> Conda dependencies. Having Conda enabled in
>> dependency_resolvers_conf.xml means that Galaxy will look for
>> dependencies using the Conda system when it attempts to run tools. If
>> conda_auto_install is True, and a dependency is not found, Galaxy will
>> attempt to install it using the configured Conda channels. A graphical
>> user interface that allows administrators install Conda packages
>> directly from Galaxy when tools are installed from the Tool Shed is
>> available in the 16.07 release of Galaxy.
>>
>>
>> 2. How do Conda dependencies work? Where do things get installed?
>> -
>> In contrast to the old dependency system, which was used exclusively by
>> Galaxy, Conda is a pre-existing, independent project. With Conda it is
>> possible for an admin to install packages without touching Galaxy at all
>> – managing your dependencies 

Re: [galaxy-dev] Workflow Compatibility

2016-08-26 Thread Peter Cock
Hi Katherine,

Are you asking about compatibility staying on the same Galaxy
instance, or the harder problem of compatibility sharing workflows
between Galaxy servers?

Taking data from input Galaxy datasets should be fine, anything else
may not be portable. Even the relatively simple case of using
datafiles referenced via an example.loc file (e.g. BLAST databases)
would require the entries in the example.loc file be synchronised
between Galaxy instances, and the associated data files too.

Peter


On Fri, Aug 26, 2016 at 1:27 PM, Katherine Beaulieu
 wrote:
> Hi Everyone,
> Would anyone be able to tell me the conditions which would make a tool
> non-workflow compatible? I have a tool that imports files from a third party
> application and auto-detects the file format. There is also the option to
> upload multiple files at once so the tool always uploads at least two files.
> From what I have described can anyone see why this tool would not be able to
> send one of its files to the next tool in the chain, ex. a text manipulation
> tool?
> Thanks,
> Katherine
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Doubled arguments from loc files in tests

2016-08-10 Thread Peter Cock
On Tue, Aug 9, 2016 at 6:33 PM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
> Hello all,
>
> Recently I've been debugging various problems with my Galaxy tool TravisCI
> tests as part of moving to using "planemo test" rather than the fragile system
> I first created which mimicked a pre-ToolShed manual install:
>
> https://blastedbio.blogspot.co.uk/2013/09/using-travis-ci-for-testing-galaxy-tools.html
>
> Progress:
>
> https://travis-ci.org/peterjc/galaxy_mira - passing now :)
> https://travis-ci.org/peterjc/pico_galaxy - probably a loc file problem

I now understand why this was breaking and have a workaround,
filed as:

https://github.com/galaxyproject/planemo/issues/530

> https://travis-ci.org/peterjc/galaxy_blast - different loc file problem
>
> This email is about the problems testing the BLAST wrappers where I
> am using a BLAST database defined in test-data/*.loc files.
>
> (I wasted some time today trying moving/copying the loc files and
> databases to different places before I noticed the issue below)
>
> Problem lines quoted are from this run, galaxy master or dev branch:
> https://travis-ci.org/peterjc/galaxy_blast/builds/150072050
> as of this commit:
> https://github.com/peterjc/galaxy_blast/commit/03f81e7fb1e0562b46b8b2a2079c94f04c1f3adf
>
> Some of the command expect a single BLAST database from the
> test-data/*.loc files, and the test should do that, but we get:
>
> blastdbcmd ... -db path,path ...
> rpsblast ... -db path,path ...
> rpstblastn ... -db path,path ...
>
> Here we should get ... -db path (once only), not repeated with a
> comma, e.g.

I think I now understand how this happens, see:

https://github.com/galaxyproject/galaxy/pull/2644

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Doubled arguments from loc files in tests

2016-08-09 Thread Peter Cock
Hello all,

Recently I've been debugging various problems with my Galaxy tool TravisCI
tests as part of moving to using "planemo test" rather than the fragile system
I first created which mimicked a pre-ToolShed manual install:

https://blastedbio.blogspot.co.uk/2013/09/using-travis-ci-for-testing-galaxy-tools.html

Progress:

https://travis-ci.org/peterjc/galaxy_mira - passing now :)
https://travis-ci.org/peterjc/pico_galaxy - probably a loc file problem
https://travis-ci.org/peterjc/galaxy_blast - different loc file problem

This email is about the problems testing the BLAST wrappers where I
am using a BLAST database defined in test-data/*.loc files.

(I wasted some time today trying moving/copying the loc files and
databases to different places before I noticed the issue below)

Problem lines quoted are from this run, galaxy master or dev branch:
https://travis-ci.org/peterjc/galaxy_blast/builds/150072050
as of this commit:
https://github.com/peterjc/galaxy_blast/commit/03f81e7fb1e0562b46b8b2a2079c94f04c1f3adf

Some of the command expect a single BLAST database from the
test-data/*.loc files, and the test should do that, but we get:

blastdbcmd ... -db path,path ...
rpsblast ... -db path,path ...
rpstblastn ... -db path,path ...

Here we should get ... -db path (once only), not repeated with a
comma, e.g.

galaxy.jobs.command_factory INFO 2016-08-05 14:55:08,543 Built script
[/tmp/tmp9NURC9/job_working_directory/000/58/tool_script.sh] for tool
command[blastdbcmd -version >
/tmp/tmp9NURC9/tmp/GALAXY_VERSION_STRING_58 2>&1; blastdbcmd -dbtype
prot -db 
"/home/travis/build/peterjc/galaxy_blast/test-data/four_human_proteins.fasta,/home/travis/build/peterjc/galaxy_blast/galaxy-dev/test-data/four_human_proteins.fasta"
-info -out "/tmp/tmp9NURC9/files/000/dataset_58.dat"]

For other tools you can select multiple databases as input, and
the blastn tests try that, e.g. a working test gave this (where the
space is deliberate as that's how the NCBI handle multiple DB
arguments):

blastn ... -db "path1 path2"

galaxy.jobs.command_factory INFO 2016-08-05 14:58:01,783 Built script
[/tmp/tmp9NURC9/job_working_directory/000/76/tool_script.sh] for tool
command[blastn -version > /tmp/tmp9NURC9/tmp/GALAXY_VERSION_STRING_76
2>&1; blastn -query "/tmp/tmp9NURC9/files/000/dataset_75.dat" -db
"/home/travis/build/peterjc/galaxy_blast/test-data/three_human_mRNA.fasta
/home/travis/build/peterjc/galaxy_blast/galaxy-dev/test-data/three_human_mRNA.fasta"
-task megablast -evalue 0.001 -out
"/tmp/tmp9NURC9/files/000/dataset_76.dat" -outfmt 6 -num_threads
"${GALAXY_SLOTS:-8}"]

But when a single database is given we get a repeated path:

blastn ... -db "path path"

galaxy.jobs.command_factory INFO 2016-08-05 14:58:22,901 Built script
[/tmp/tmp9NURC9/job_working_directory/000/78/tool_script.sh] for tool
command[blastn -version > /tmp/tmp9NURC9/tmp/GALAXY_VERSION_STRING_78
2>&1; blastn -query "/tmp/tmp9NURC9/files/000/dataset_77.dat" -db
"/home/travis/build/peterjc/galaxy_blast/test-data/rhodopsin_nucs.fasta
/home/travis/build/peterjc/galaxy_blast/galaxy-dev/test-data/rhodopsin_nucs.fasta"
-task megablast -evalue 0.001 -out
"/tmp/tmp9NURC9/files/000/dataset_78.dat" -outfmt 6 -num_threads
"${GALAXY_SLOTS:-8}"]

Any thoughts on where this is breaking? In planemo?

These tests are working locally via "planemo test" (older
version of planemo, older version of Galaxy, Python 2.6).

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Adding packages to depot.galaxyproject.org

2016-08-08 Thread Peter Cock
On Mon, Aug 8, 2016 at 3:27 PM, Björn Grüning <bjoern.gruen...@gmail.com> wrote:
> Hi Peter,
>
> Am 08.08.2016 um 15:54 schrieb Peter Cock:
>> On Mon, Aug 8, 2016 at 2:41 PM, Eric Rasche <e...@tamu.edu> wrote:
>>> Hi Peter,
>>>
>>> The cargo port is the ideal place for this, however it does not support
>>> manual uploads of files.
>>>
>>
>> Thanks Eric,
>>
>> You mean https://github.com/galaxyproject/cargo-port
>>
>> Is that something which might be addressed in future? The use case
>> is when a previously public file stops working (either temporarily or
>> permanently), so you can't use the official URL to make the cached
>> copy?
>>
>> Would it be OK to use a short lived URL like a personal dropbox URL,
>> and then once cached, update urls.tsv with the original official
>> (but broken) URL?
>
>
> Yes this is totally fine. We just need to have a way to verify and check
> tarballs before upload and we thought github and PR are a nice fit for this.
>
> Thanks,
> Bjoern

Let's try this then:

https://github.com/galaxyproject/cargo-port/pull/82

(I can't find the Mac binaries for MIRA 4.9.5, hopefully I might have
that on my home machine but it's a long shot.)

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Adding packages to depot.galaxyproject.org

2016-08-08 Thread Peter Cock
On Mon, Aug 8, 2016 at 2:41 PM, Eric Rasche  wrote:
> Hi Peter,
>
> The cargo port is the ideal place for this, however it does not support
> manual uploads of files.
>

Thanks Eric,

You mean https://github.com/galaxyproject/cargo-port

Is that something which might be addressed in future? The use case
is when a previously public file stops working (either temporarily or
permanently), so you can't use the official URL to make the cached
copy?

Would it be OK to use a short lived URL like a personal dropbox URL,
and then once cached, update urls.tsv with the original official
(but broken) URL?

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Adding packages to depot.galaxyproject.org

2016-08-08 Thread Peter Cock
Hello all,

How do tool wrapper developers/packagers go about adding entries
to the Galaxy depot? There isn't a GitHub repository for this is there?

Specifically I want to upload some older MIRA releases, e.g. these
URLs have broken, but I have the original files and checksums:

http://downloads.sourceforge.net/project/mira-assembler/MIRA/development/mira_4.9.5_2_darwin14.3.0_x86_64_static.tar.bz2
http://downloads.sourceforge.net/project/mira-assembler/MIRA/development/mira_4.9.5_2_linux-gnu_x86_64_static.tar.bz2

See also:

https://www.freelists.org/post/mira_talk/Stable-download-URLs-on-sourceforge
https://www.freelists.org/post/mira_talk/MIRA-495-now-missing-on-SourceForge

I'm now running "planemo lint --urls ..." under TravisCI which flags
the broken URLs immediately (rather than finding out when the
installation fails later):

https://github.com/peterjc/galaxy_mira/commit/4e26a4e3f3d69adaab657cb614e56ebcf9968592
https://travis-ci.org/peterjc/galaxy_mira/jobs/150025836

Regards,

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy Blast

2016-08-02 Thread Peter Cock
Cross referencing this open issue which looks like the same thing:

https://github.com/peterjc/galaxy_blast/issues/80

We're probably going to return the BLAST datatypes to the Galaxy core,
which will resolve this in a different way:

https://lists.galaxyproject.org/pipermail/galaxy-dev/2016-August/024696.html
https://github.com/galaxyproject/galaxy/pull/2696

Peter

On Wed, Jul 27, 2016 at 9:34 AM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
> Thank Dan, that's what I thought.
>
> So databases can added to any of the multiple blastdb.loc
> files which Galaxy ought to be combining. I'm not sure what
> went wrong for Harry (and therefore how to avoid it for
> others in future). :(
>
> (Separately I have raised the issue of returning the BLAST
> datatypes to the Galaxy core on the IUC mailing list)
>
> Peter
>
> On Tue, Jul 26, 2016 at 10:28 PM, Daniel Blankenberg <d...@bx.psu.edu> wrote:
>> Hi Peter and Harry,
>>
>> When a tool is installed from the toolshed, the tool data table loc files 
>> are installed to the configured location and are then referenced in 
>> shed_tool_data_table_conf.xml. You should be able to modify the .loc files 
>> as referenced in this file, or, as you did, add a new entry to the 
>> tool_data_table_conf.xml file. Each instance of the tool data table (one 
>> will exist per tool shed repository revision that has been installed) will 
>> be merged together into a single entity that can then be accessed by the 
>> tools. You can also find the paths to the .loc files for each tool data 
>> table under the admin interface. Data Managers will write to the specific 
>> .loc file that was created during their installation.
>>
>> Please let me know if I can provide additional clarification.
>>
>>
>>
>> Thanks for using Galaxy,
>>
>> Dan
>>
>>
>>> On Jul 26, 2016, at 6:08 AM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
>>>
>>> Hello all,
>>>
>>> Harry (CC'd) emailed me about some teething trouble getting
>>> nucleotide BLAST databases to appear in the drop down lists
>>> (see below).
>>>
>>> I suspect this is partly due to the partial data table sample that
>>> ships with Galaxy:
>>>
>>> https://github.com/galaxyproject/galaxy/blob/dev/config/tool_data_table_conf.xml.sample
>>>
>>> Can any of the Galaxy Data Manager expects advise please?
>>>
>>> Thanks,
>>>
>>> Peter
>>>
>>>
>>> -- Forwarded message --
>>> From: Harry Wright <harry.wri...@moredun.ac.uk>
>>> Date: Tue, Jul 26, 2016 at 9:25 AM
>>> Subject: RE: Galaxy Blast
>>> To: Peter Cock <p.j.a.c...@googlemail.com>
>>>
>>>
>>> Hi Peter,
>>>
>>> Sorry for the delay in response. If you think it would be useful please 
>>> post it.
>>>
>>> I have downloaded what I thought was the newest versions of NCIB
>>> blast+ 2.2.31 from the toolshed.
>>>
>>> The tool_data_table_conf.xml.sample I altered was in galaxy_dir/config/.
>>>
>>> If you need anything else please ask.
>>>
>>> Thanks
>>> Harry
>>>
>>> -Original Message-
>>> From: Peter Cock [mailto:p.j.a.c...@googlemail.com]
>>> Sent: 22 July 2016 16:28
>>> To: Harry Wright
>>> Subject: Re: Galaxy Blast
>>>
>>> Hi Harry,
>>>
>>> Hmm. I'm a little puzzled. That file looks fine on our GitHub and on
>>> the Tool Shed:
>>>
>>> https://github.com/peterjc/galaxy_blast/blob/master/tool-data/tool_data_table_conf.xml.sample
>>>
>>> http://toolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus (browse tip 
>>> files)
>>>
>>> Can you tell me exactly which version of things you got from the Tool
>>> Shed, and exactly which tool_data_table_conf.xml and
>>> tool_data_table_conf.xml.sample you had to edit?
>>>
>>> My guess is the problem is this partial example that ships with Galaxy:
>>>
>>> https://github.com/galaxyproject/galaxy/blob/dev/config/tool_data_table_conf.xml.sample
>>>
>>> (There is talk of moving datatypes like this back into the Galaxy
>>> core, which would simply this)
>>>
>>> May I forward this email to the public Galaxy developers mailing list 
>>> please?
>>>
>>> Thanks,
>>>
>>> Peter
>>>
>>> On Fri, Jul 22, 2016 at 3:59 PM, Harry Wright
>>> <harry.wri...@moredun.ac.u

Re: [galaxy-dev] Add url in help of tool xml

2016-08-02 Thread Peter Cock
Great - I'm glad it was an easy fix.

Peter

On Tue, Aug 2, 2016 at 10:36 AM, Mohamed Kassam <k.mam...@gmail.com> wrote:
> Hell Peter
> The CDATA works perfectly thanks again.
>
> Mohamed
>
> 2016-08-02 11:31 GMT+02:00 Peter Cock <p.j.a.c...@googlemail.com>:
>>
>> This could be as simple as the & needing to be
>> escaped as  or better escaping the whole
>> help text using CDATA:
>>
>>
>> https://galaxy-iuc-standards.readthedocs.io/en/latest/best_practices/tool_xml.html#help-tag
>>
>> Can you show us your tool XML file? Is it on GitHub?
>>
>> Peter
>>
>> On Tue, Aug 2, 2016 at 10:26 AM, Mohamed Kassam <k.mam...@gmail.com>
>> wrote:
>> > Dear all,
>> >
>> > I would like to add an url on my help section of a tool on my xml file
>> > however it does not work,
>> >
>> > https://10.38.14.14/api/fs/dir.html?path=/home/user/=galaxy
>> >
>> > I think it is the =galaxy doesn't work in the xml .
>> >
>> >
>> > Mohamed
>> >
>> > ___
>> > Please keep all replies on the list by using "reply all"
>> > in your mail client.  To manage your subscriptions to this
>> > and other Galaxy lists, please use the interface at:
>> >   https://lists.galaxyproject.org/
>> >
>> > To search Galaxy mailing lists use the unified search at:
>> >   http://galaxyproject.org/search/mailinglists/
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Add url in help of tool xml

2016-08-02 Thread Peter Cock
This could be as simple as the & needing to be
escaped as  or better escaping the whole
help text using CDATA:

https://galaxy-iuc-standards.readthedocs.io/en/latest/best_practices/tool_xml.html#help-tag

Can you show us your tool XML file? Is it on GitHub?

Peter

On Tue, Aug 2, 2016 at 10:26 AM, Mohamed Kassam  wrote:
> Dear all,
>
> I would like to add an url on my help section of a tool on my xml file
> however it does not work,
>
> https://10.38.14.14/api/fs/dir.html?path=/home/user/=galaxy
>
> I think it is the =galaxy doesn't work in the xml .
>
>
> Mohamed
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Returning BLAST datatypes and DB loc files to Galaxy core?

2016-08-01 Thread Peter Cock
Hi Gildas,

When datatypes were moved to the Tool Shed, I
think the idea was to eventually have only a core
set of topic-neutral datatypes (plain text, tabular,
etc) in Galaxy itself. That does seem sensible.

I'm not quite sure why policy has returned to a
larger centralised set - but limitations of the
ToolShed like dependencies (for Python code
imported directly into Galaxy), lack of versioning
of datatypes, and better control of namespaces
are likely part of this. I missed GCC2017 sadly.

Even with a large set of datatypes included with
Galaxy, it should be easy to hide/disable lots
(e.g. for an image analysis Galaxy you would
not want any of the sequence file formats).

Peter

P.S. Good point about EDAM, its been raised
on the BLAST datatypes pull request:
https://github.com/galaxyproject/galaxy/pull/2696


On Mon, Aug 1, 2016 at 9:27 PM, Gildas Le Corguillé
 wrote:
> Hi all,
>
> If indeed, datatypes return within the Galaxy distribution, can we imagine
> propose different datatypes_conf.xml?
>
> Galaxy isn’t anymore dedicated to NGS purpose. It is use also for
> metabolomics, proteomics, …
>
> So it could be great to propose 1 "common" list of datatypes (text, tabular,
> png, pdf, …) and n specific datatypes lists for the n scientific areas to
> reduce this huge list of datatypes proposed to the users.
> Maybe this selection should be based on edam ontology. As you know they are
> almost already annotated with edam_format and edam_data
>
>
> Gildas
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Returning BLAST datatypes and DB loc files to Galaxy core?

2016-08-01 Thread Peter Cock
I have a work-in-progress branch and pull request here,
https://github.com/peterjc/galaxy/tree/blast_datatypes
https://github.com/galaxyproject/galaxy/pull/2696

The Galaxy TravisCI tests looked fine.

Peter

On Mon, Aug 1, 2016 at 11:18 AM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
> Dear all,
>
> I ran this past the IUC first, and the only comments were
> positive.
>
> Although I wasn't at GCC2017 to discuss this in person,
> I understand that the Galaxy Team now encourages
> widely used datatypes to be included in the main Galaxy
> repository, rather than distributed via the Tool Shed.
>
> To that end, would a pull request returning the BLAST
> datatypes and associated database *.loc files be welcome?
>
> These are currently on my GitHub repository here:
> https://github.com/peterjc/galaxy_blast/
>
> And the datatypes are distributed via the Tool Shed here:
> http://toolshed.g2.bx.psu.edu/view/devteam/blast_datatypes
>
> Assuming this happens, we would need to phase out the
> tool shed version (but it will still be needed while older
> Galaxy instances are still running).
>
> Are there any pitfalls to worry about if the datatypes are
> already there with Galaxy and the tool shed version is
> installed on top? Or the tool shed version was installed
> but then Galaxy was updated to include the version
> bundled with Galaxy?
>
> Thanks,
>
> Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Fwd: Returning BLAST datatypes and DB loc files to Galaxy core?

2016-08-01 Thread Peter Cock
Dear all,

I ran this past the IUC first, and the only comments were
positive.

Although I wasn't at GCC2017 to discuss this in person,
I understand that the Galaxy Team now encourages
widely used datatypes to be included in the main Galaxy
repository, rather than distributed via the Tool Shed.

To that end, would a pull request returning the BLAST
datatypes and associated database *.loc files be welcome?

These are currently on my GitHub repository here:
https://github.com/peterjc/galaxy_blast/

And the datatypes are distributed via the Tool Shed here:
http://toolshed.g2.bx.psu.edu/view/devteam/blast_datatypes

Assuming this happens, we would need to phase out the
tool shed version (but it will still be needed while older
Galaxy instances are still running).

Are there any pitfalls to worry about if the datatypes are
already there with Galaxy and the tool shed version is
installed on top? Or the tool shed version was installed
but then Galaxy was updated to include the version
bundled with Galaxy?

Thanks,

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] TestToolShed down?

2016-07-28 Thread Peter Cock
For me too, thanks!

Peter

On Thu, Jul 28, 2016 at 11:06 AM, Nicola Soranzo <nsora...@tiscali.it> wrote:
> Works for me now.
>
> Cheers,
> Nicola
>
>
> On 28/07/16 09:44, Peter Cock wrote:
>>
>> Hello all,
>>
>> The main Tool Shed seems fine, but the Test Tool Shed is not:
>> https://testtoolshed.g2.bx.psu.edu/
>>
>>> Galaxy
>>>
>>> The Tool Shed could not be reached
>>>
>>> You are seeing this message because a request to the
>>> Tool Shed timed out or was refused. This may be a temporary
>>> issue which could be resolved by retrying the operation you
>>> were performing. If you receive this message repeatedly or
>>> for an extended amount of time, please report it to the
>>> Galaxy Team at galaxy-b...@bx.psu.edu with details on
>>> what you were trying to do and the URL in the address bar.
>>> More information on the current state of Galaxy services
>>> may be available on the Galaxy status page or the
>>> @galaxyproject Twitter feed.
>>
>> The https://status.galaxyproject.org/ page reports it should
>> be operational.
>>
>> It responds to ping (same server name and IP address as the
>> main Tool Shed):
>>
>>
>> $ ping -c 4 testtoolshed.g2.bx.psu.edu
>> PING radegast.galaxyproject.org (128.118.250.5) 56(84) bytes of data.
>> 64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=1
>> ttl=45 time=130 ms
>> 64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=2
>> ttl=45 time=129 ms
>> 64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=3
>> ttl=45 time=129 ms
>> 64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=4
>> ttl=45 time=129 ms
>> --- radegast.galaxyproject.org ping statistics ---
>> 4 packets transmitted, 4 received, 0% packet loss, time 3133ms
>> rtt min/avg/max/mdev = 129.744/129.955/130.358/0.348 ms
>>
>>
>> $ ping -c 4 toolshed.g2.bx.psu.edu
>> PING radegast.galaxyproject.org (128.118.250.5) 56(84) bytes of data.
>> 64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=1
>> ttl=45 time=130 ms
>> 64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=2
>> ttl=45 time=129 ms
>> 64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=3
>> ttl=45 time=129 ms
>> 64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=4
>> ttl=45 time=130 ms
>> --- radegast.galaxyproject.org ping statistics ---
>> 4 packets transmitted, 4 received, 0% packet loss, time 3135ms
>> rtt min/avg/max/mdev = 129.785/129.977/130.196/0.179 ms
>>
>>
>>
>> Peter
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>http://galaxyproject.org/search/mailinglists/
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] TestToolShed down?

2016-07-28 Thread Peter Cock
Hello all,

The main Tool Shed seems fine, but the Test Tool Shed is not:
https://testtoolshed.g2.bx.psu.edu/

> Galaxy
>
> The Tool Shed could not be reached
>
> You are seeing this message because a request to the
> Tool Shed timed out or was refused. This may be a temporary
> issue which could be resolved by retrying the operation you
> were performing. If you receive this message repeatedly or
> for an extended amount of time, please report it to the
> Galaxy Team at galaxy-b...@bx.psu.edu with details on
> what you were trying to do and the URL in the address bar.
> More information on the current state of Galaxy services
> may be available on the Galaxy status page or the
> @galaxyproject Twitter feed.

The https://status.galaxyproject.org/ page reports it should
be operational.

It responds to ping (same server name and IP address as the
main Tool Shed):


$ ping -c 4 testtoolshed.g2.bx.psu.edu
PING radegast.galaxyproject.org (128.118.250.5) 56(84) bytes of data.
64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=1
ttl=45 time=130 ms
64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=2
ttl=45 time=129 ms
64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=3
ttl=45 time=129 ms
64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=4
ttl=45 time=129 ms
--- radegast.galaxyproject.org ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3133ms
rtt min/avg/max/mdev = 129.744/129.955/130.358/0.348 ms


$ ping -c 4 toolshed.g2.bx.psu.edu
PING radegast.galaxyproject.org (128.118.250.5) 56(84) bytes of data.
64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=1
ttl=45 time=130 ms
64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=2
ttl=45 time=129 ms
64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=3
ttl=45 time=129 ms
64 bytes from radegast.galaxyproject.org (128.118.250.5): icmp_seq=4
ttl=45 time=130 ms
--- radegast.galaxyproject.org ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3135ms
rtt min/avg/max/mdev = 129.785/129.977/130.196/0.179 ms



Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy Blast

2016-07-27 Thread Peter Cock
Thank Dan, that's what I thought.

So databases can added to any of the multiple blastdb.loc
files which Galaxy ought to be combining. I'm not sure what
went wrong for Harry (and therefore how to avoid it for
others in future). :(

(Separately I have raised the issue of returning the BLAST
datatypes to the Galaxy core on the IUC mailing list)

Peter

On Tue, Jul 26, 2016 at 10:28 PM, Daniel Blankenberg <d...@bx.psu.edu> wrote:
> Hi Peter and Harry,
>
> When a tool is installed from the toolshed, the tool data table loc files are 
> installed to the configured location and are then referenced in 
> shed_tool_data_table_conf.xml. You should be able to modify the .loc files as 
> referenced in this file, or, as you did, add a new entry to the 
> tool_data_table_conf.xml file. Each instance of the tool data table (one will 
> exist per tool shed repository revision that has been installed) will be 
> merged together into a single entity that can then be accessed by the tools. 
> You can also find the paths to the .loc files for each tool data table under 
> the admin interface. Data Managers will write to the specific .loc file that 
> was created during their installation.
>
> Please let me know if I can provide additional clarification.
>
>
>
> Thanks for using Galaxy,
>
> Dan
>
>
>> On Jul 26, 2016, at 6:08 AM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
>>
>> Hello all,
>>
>> Harry (CC'd) emailed me about some teething trouble getting
>> nucleotide BLAST databases to appear in the drop down lists
>> (see below).
>>
>> I suspect this is partly due to the partial data table sample that
>> ships with Galaxy:
>>
>> https://github.com/galaxyproject/galaxy/blob/dev/config/tool_data_table_conf.xml.sample
>>
>> Can any of the Galaxy Data Manager expects advise please?
>>
>> Thanks,
>>
>> Peter
>>
>>
>> -- Forwarded message --
>> From: Harry Wright <harry.wri...@moredun.ac.uk>
>> Date: Tue, Jul 26, 2016 at 9:25 AM
>> Subject: RE: Galaxy Blast
>> To: Peter Cock <p.j.a.c...@googlemail.com>
>>
>>
>> Hi Peter,
>>
>> Sorry for the delay in response. If you think it would be useful please post 
>> it.
>>
>> I have downloaded what I thought was the newest versions of NCIB
>> blast+ 2.2.31 from the toolshed.
>>
>> The tool_data_table_conf.xml.sample I altered was in galaxy_dir/config/.
>>
>> If you need anything else please ask.
>>
>> Thanks
>> Harry
>>
>> -Original Message-
>> From: Peter Cock [mailto:p.j.a.c...@googlemail.com]
>> Sent: 22 July 2016 16:28
>> To: Harry Wright
>> Subject: Re: Galaxy Blast
>>
>> Hi Harry,
>>
>> Hmm. I'm a little puzzled. That file looks fine on our GitHub and on
>> the Tool Shed:
>>
>> https://github.com/peterjc/galaxy_blast/blob/master/tool-data/tool_data_table_conf.xml.sample
>>
>> http://toolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus (browse tip files)
>>
>> Can you tell me exactly which version of things you got from the Tool
>> Shed, and exactly which tool_data_table_conf.xml and
>> tool_data_table_conf.xml.sample you had to edit?
>>
>> My guess is the problem is this partial example that ships with Galaxy:
>>
>> https://github.com/galaxyproject/galaxy/blob/dev/config/tool_data_table_conf.xml.sample
>>
>> (There is talk of moving datatypes like this back into the Galaxy
>> core, which would simply this)
>>
>> May I forward this email to the public Galaxy developers mailing list please?
>>
>> Thanks,
>>
>> Peter
>>
>> On Fri, Jul 22, 2016 at 3:59 PM, Harry Wright
>> <harry.wri...@moredun.ac.uk> wrote:
>>>
>>>
>>> Dear Peter,
>>>
>>>
>>>
>>> I have just installed a new instance of Galaxy here at Moredun and
>>> have been busy setting up tools for us to use. One of the tools that I
>>> have set up is blast with the NCBI nr and nt databases sitting in the
>>> backend as a Locally installed database. I downloaded both files and
>>> formatted them into databases. I then add lines to the blastdb_p.loc
>>> and blastdb.loc. When I restarted the server I found that I could
>>> access nr through the interface, for blastx, but not nt, using blastn.
>>> For nt I was getting the message “No option available” in the
>>> Nucleotide BLAST database drop down when I selected “Locally installed 
>>> BLAST database”.
>>>
>>>
>>>
>>> After some searching I

[galaxy-dev] Fwd: Galaxy Blast

2016-07-26 Thread Peter Cock
Hello all,

Harry (CC'd) emailed me about some teething trouble getting
nucleotide BLAST databases to appear in the drop down lists
(see below).

I suspect this is partly due to the partial data table sample that
ships with Galaxy:

https://github.com/galaxyproject/galaxy/blob/dev/config/tool_data_table_conf.xml.sample

Can any of the Galaxy Data Manager expects advise please?

Thanks,

Peter


-- Forwarded message --
From: Harry Wright <harry.wri...@moredun.ac.uk>
Date: Tue, Jul 26, 2016 at 9:25 AM
Subject: RE: Galaxy Blast
To: Peter Cock <p.j.a.c...@googlemail.com>


Hi Peter,

Sorry for the delay in response. If you think it would be useful please post it.

I have downloaded what I thought was the newest versions of NCIB
blast+ 2.2.31 from the toolshed.

The tool_data_table_conf.xml.sample I altered was in galaxy_dir/config/.

If you need anything else please ask.

Thanks
Harry

-Original Message-
From: Peter Cock [mailto:p.j.a.c...@googlemail.com]
Sent: 22 July 2016 16:28
To: Harry Wright
Subject: Re: Galaxy Blast

Hi Harry,

Hmm. I'm a little puzzled. That file looks fine on our GitHub and on
the Tool Shed:

https://github.com/peterjc/galaxy_blast/blob/master/tool-data/tool_data_table_conf.xml.sample

http://toolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus (browse tip files)

Can you tell me exactly which version of things you got from the Tool
Shed, and exactly which tool_data_table_conf.xml and
tool_data_table_conf.xml.sample you had to edit?

My guess is the problem is this partial example that ships with Galaxy:

https://github.com/galaxyproject/galaxy/blob/dev/config/tool_data_table_conf.xml.sample

(There is talk of moving datatypes like this back into the Galaxy
core, which would simply this)

May I forward this email to the public Galaxy developers mailing list please?

Thanks,

Peter

On Fri, Jul 22, 2016 at 3:59 PM, Harry Wright
<harry.wri...@moredun.ac.uk> wrote:
>
>
> Dear Peter,
>
>
>
> I have just installed a new instance of Galaxy here at Moredun and
> have been busy setting up tools for us to use. One of the tools that I
> have set up is blast with the NCBI nr and nt databases sitting in the
> backend as a Locally installed database. I downloaded both files and
> formatted them into databases. I then add lines to the blastdb_p.loc
> and blastdb.loc. When I restarted the server I found that I could
> access nr through the interface, for blastx, but not nt, using blastn.
> For nt I was getting the message “No option available” in the
> Nucleotide BLAST database drop down when I selected “Locally installed BLAST 
> database”.
>
>
>
> After some searching I found reference to
> config/tool_data_table_conf.xml.sample. When I look there I could only
> find reference to blastdb_p and not blastdb.
>
>
>
> I added the lines below to the file and changed the file name to
> tool_data_table_conf.xml. I  restarted galaxy to find that I could now
> use nt in blastn.
>
>
>
> 
>
> 
>
> value, name, path
>
> 
>
> 
>
>
>
> I was wondering if there was something that I have missed or has this
> section of the code been missed out of the tool_data_table_conf.xml
> file in the installation download?
>
>
>
> Regards
>
> Harry
>
>
>
> Dr. Harry Wright
>
> Moredun Research Institute
>
> Pentlands Science Park
>
> Bush Loan
>
> Penicuik
>
> Midlothian EH26 0PZ
>
> Tel - 0131 4455111
>
> Fax - 0131 4456235
>
>
>
> The Moredun Research Institute is a limited company registered in Scotland.
> Registered number: SC149440
>
> The Moredun Research Institute is a charity registered in Scotland.
> Registered number: SC022353
>
> Registered office: Moredun Research Institute, Pentlands Science Park,
> Bush Loan, Penicuik, Midlothian, EH26 0PZ, UK
>
>



The Moredun Research Institute is a limited company registered
in Scotland. Registered number: SC149440

The Moredun Research Institute is a charity registered in
Scotland. Registered number: SC022353

Registered office: Moredun Research Institue, Pentlands
Science Park, Bush Loan, Penicuik, Midlothian, EH26 0PZ, UK
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Loading tool config file

2016-07-22 Thread Peter Cock
Sorry, I know relatively little about how Galaxy loads the tool XML
files and turns this into the HTML pages. I've mostly searched
the codebase with grep at the command line (also searching
directly on GitHub works pretty well) trying to trace functionality
when I was debugging something.

Hopefully one of the Galaxy developers will be able to answer.

Peter


On Fri, Jul 22, 2016 at 7:29 AM, Katherine Beaulieu wrote:
>
> No I'm really asking about which files in Galaxy, will load those
> parameters on the screen. I believe you were the one to give me a link to
> adding a new parameter type in Galaxy via the password parameter, but I
> haven't quite been able to implement that. The two files I have changed so
> far are basic.py and form_builder.py but using these two files it still only
> loads a text parameter.
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Loading tool config file

2016-07-22 Thread Peter Cock
Hi Katherine,

Are you asking about the XML files which described Galaxy Tools
by listing all the options to show the user, any help text to show the
user, and instructions for how to turn the options picked into a
command line to be executed:

https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax

Peter

On Fri, Jul 22, 2016 at 12:09 PM, Katherine Beaulieu
 wrote:
> Dear all,
> I am looking for the files/functions that get called from the point where
> you click on the tool name to the point where you get an html version of
> your tool on the screen. Could someone indicate what this path looks like? I
> would be really appreciative!
> Thanks,
> Katherine
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Papers that cite Galaxy workflows

2016-07-22 Thread Peter Cock
Hi Steve,

This isn't a very strong example, as it is self-reuse and self-citing,
but anyway - we described some workflows in this book chapter:

Cock and Pritchard (2014)
Galaxy as a Platform for Identifying Candidate Pathogen Effectors.
http://dx.doi.org/10.1007/978-1-62703-986-4_1

We re-used a workflow and cited the chapter in this paper where
I was a co-author:

Espada et al (2015)
Identification and characterization of parasitism genes from the
pinewood nematode Bursaphelenchus xylophilus reveals a multilayered
detoxification strategy
http://dx.doi.org/10./mpp.12280

(It was also cited by our associated tools paper, and a third-party
review paper, but those were not re-using the workflows)

Peter



On Fri, Jul 22, 2016 at 3:21 AM, Steve Cassidy  wrote:
> Hi all,
>  I’m writing a paper that will include some evangelism for Galaxy in
> language sciences and I’d like to be able to cite examples of papers in the
> Bio field that cite Galaxy workflows.  Even better would be the re-use of a
> Galaxy workflow cited in another paper.  Does anyone have any good example
> papers they could refer me to.
>
> Thanks,
>
> Steve
> —
> Department of Computing, Macquarie University
> http://web.science.mq.edu.au/~cassidy
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] BLAST only using single core

2016-07-21 Thread Peter Cock
On Thu, Jul 21, 2016 at 10:35 AM, Shane Sturrock  wrote:
>
> I’m just using the drmaa plugin so I’m guessing I need to either specify a
> job runner for BLAST which sets GALAXY_SLOTS. The documentation isn’t
> entirely clear although it seems I need to create a job_conf.xml which sets
> local_slots to a value for runner drmaa unless there’s a way to set that in
> the galaxy.ini file instead.

There is a legacy job setting via galaxy.ini aka universe_wsgi.ini where
we used to use this:

[galaxy:tool_runners]
ncbi_blastp_wrapper  = drmaa://-V -pe smp 4/
ncbi_blastn_wrapper  = drmaa://-V -pe smp 4/
ncbi_blastx_wrapper  = drmaa://-V -pe smp 4/
ncbi_tblastn_wrapper = drmaa://-V -pe smp 4/
ncbi_tblastx_wrapper = drmaa://-V -pe smp 4/
blast_reciprocal_best_hits = drmaa://-V -pe smp 4/

The -V was an SGE switch to copy the environment variables,
and -pe smp 4 is for a 4 core job.

For job_conf.xml I am trying something similar with less repetition:

...


-pe smp 4


...








...


i.e. I've set the computationally heavy BLAST+ tools to use a
destination configured to use just four cores. Because this is
SGE, we do this via the native syntax of -pe smp 4, i.e.
just like at the command line we'd do:

$ qsub -pe smp 4 ...

This detail will be different for your LSF cluster.

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] BLAST only using single core

2016-07-21 Thread Peter Cock
Hi Shane,

We've not touched anything on the BLAST+ wrapper here for
a while - the command line is always built using:

-num_threads "\${GALAXY_SLOTS:-8}"

That means use the environment variable $GALAXY_SLOTS
if set, defaulting to 8 threads if not. See:

https://github.com/peterjc/galaxy_blast/blob/master/tools/ncbi_blast_plus/ncbi_macros.xml#L352

Galaxy itself should be setting $GALAXY_SLOTS via your
cluster configuration - and from your description this seems
to be set to 1 thread/slot only.

Can you tell us more about how your Galaxy LSF is setup?
Have you got BLAST specific cluster job settings?

(I don't use LSF so don't have the details to hand)

Peter

On Wed, Jul 20, 2016 at 10:03 PM, Shane Sturrock  wrote:
> Previously, the BLAST wrapper was able to use multiple cores but recently
> users have started complaining it has got really slow and when I look at the
> cluster a job is only using a single core. I don’t want jobs split across
> multiple cluster nodes, but each node has 16 cores so it would be good if
> they could be used. I’m still using the older 16.01 release and this has
> been upgraded repeatedly over the last few years so I’m still using the
> universe_wsg.ini file (sym linked to config/galaxy.ini) and I don’t have a
> job_conf.xml set up. I’m using drmaa to drive an LSF cluster. BLAST is the
> main issue here so I was wondering if there’s a way to pass the -num_threads
> flag without breaking everything until I can build up a new server?
>
> Dr. Shane Sturrock
> NZGL BioIT Admin
> n...@biomatters.com
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] New galaxy parameter type

2016-07-15 Thread Peter Cock
Did you mean parameter as in input field shown
to the user? Like a combobox or text field?

Since this happened to come up earlier today, here's
a rejected pull request which added a password
parameter - that ought to give you a rough idea:

https://github.com/galaxyproject/galaxy/pull/393

Peter

On Fri, Jul 15, 2016 at 7:46 PM, Katherine Beaulieu
 wrote:
> Hi everyone,
> I am looking to implement a new parameter type for galaxy tool development.
> Does anyone know what would be involved in doing something like this or
> perhaps what files would need to be edited?
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Get the username and password of a user

2016-07-15 Thread Peter Cock
Hi Mohamed,

See https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax
which explains the $__user_id__ and $__user_email__ tokens
which can be used in the  definition of a tool
wrapper XML file.

From a security point of view, I don't think any tool should have
access to the user's Galaxy password. I don't know if there is
an API key or similar which might be used instead.

Peter


On Fri, Jul 15, 2016 at 10:14 AM, Mohamed Kassam  wrote:
> Dear all,
> I would like to know if there is ana api which I can get the password and
> username of a user directly for my python code. I would like to send these
> infromation to an another server for connection.
>
> Thanks in advance
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] hide password xml tag

2016-07-15 Thread Peter Cock
This has been suggested before (with code to do it),
but rejected on security grounds, see:

https://github.com/galaxyproject/galaxy/pull/393

What would you do with the password when calling
your tool? If you use it directly in the command line
string then it will appear in the Galaxy logs, but also
be visible in the history details within Galaxy.

Peter


On Fri, Jul 15, 2016 at 9:28 AM, Mohamed Kassam  wrote:
> Dear all,
> I would like to hide when typing a password in a tool I develloped.
> Do you have the tag I have to use, I tried type="password" but it does not
> work.
>
> Thanks in advance,
>
>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Auto detection of output format Galaxy custom tool

2016-07-07 Thread Peter Cock
I'm glad you could solve this - thanks for letting us know and
linking to the old email discussion which helped.

Peter

On Thu, Jul 7, 2016 at 4:35 PM, Katherine Beaulieu
 wrote:
> Hi Peter, I ending up finding a solution to my problem using this forum
> post:
> https://lists.galaxyproject.org/pipermail/galaxy-dev/2014-August/020361.html.
> I just ended up using the really specific file output path and it started
> auto-detecting the file format. Thanks for your help it was much
> appreciated.
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

  1   2   3   4   >