Re: [galaxy-dev] Request: Option to reduce server data transfer for big workflow in cluster

[Ben Gift]

This does mostly make sense, and is very illuminating. I appreciate all the
help and I'm sorry that I'm so new to this.

I'm not sure I fully understand though. Do you mean that I could have a
main Galaxy install setup for the 200 nodes, for general purpose use with a
shared file system, and a specialized setup that only uses 10 with its own
shared file system so that it doesn't bog down the main one? And to use the
specialized install when running huge workflows?


On Tue, Dec 17, 2013 at 10:11 AM, John Chilton  wrote:

> Hey Ben,
>
>   Hmmm... I don't think Galaxy is doing that - not directly anyway.
> Unless I am mistaken, Galaxy will put the file in one location on the
> web server node or VM. Typically, this location is on a filesystem
> that is shared between the web server and a cluster's compute nodes.
> So I wouldn't describe that as Galaxy copying the data to all of the
> compute nodes. If your cluster doesn't have a shared file system and
> to get around this someone has configured the Galaxy data to be synced
> across all nodes - that would be copying the data to all nodes but I
> don't think Galaxy is doing that. Alternatively, you might have condor
> configured to copy the galaxy data to the remote nodes in such a way
> that everytime a Galaxy job is run on a node, all the data is copied
> to that node?
>
>   Does that make sense?
>
>   So I still don't entirely understand your setup, but my advice is
> pretty general - for now you may want to solve this problem at the
> condor level. I am assuming this is a general purpose condor cluster
> and not setup explicitly for Galaxy? Lets say you have 200 nodes in
> your condor cluster and they cannot all mount the Galaxy filesystem -
> because it would overload the file server being used by Galaxy. I
> think you could setup a FileSystemDomain at the condor level that just
> 10 of nodes belonged to say (these 10 nodes can continue to run
> anything in general but Galaxy will only submit to these) This
> filesystemdomain could have a name like galaxy.example.com if
> example.com is your default filesystemdomain. Then you can setup the
> Galaxy condor runner with a requirement such that
> "FileSystemDomain=galaxy.example.com" and Galaxy jobs will only run on
> these 10 nodes then. Having 10 nodes mount a file server is much more
> manageable than 200.
>
> -John
>
>
> On Tue, Dec 17, 2013 at 11:52 AM, Ben Gift  wrote:
> > Hi John, thanks for the reply.
> >
> > Yes, I mean Galaxy's default behavior of keeping all the data on all
> nodes
> > of our condor cluster. So for instance if I run a job, then the output of
> > that job is copied to every node in the cluster. Is this not the normal
> > behavior?
> >
> >
> > On Tue, Dec 17, 2013 at 9:42 AM, John Chilton 
> wrote:
> >>
> >> Hey Ben,
> >>
> >> Thanks for the e-mail. I did not promise anything was coming soon, I
> >> only said people were working on parts of it. It is not a feature yet
> >> unfortunately - multiple people including myself are thinking about
> >> various parts of this problem though.
> >>
> >> I would like to respond, but I am trying to understand this line: "We
> >> can't do this because Galaxy copies all intermediate steps to all
> >> no(d)es, which would bog down the servers too much."
> >>
> >> Can you describe how you are doing this staging for me? Is data
> >> currently being copied around to all the nodes, if so how are you
> >> doing that? Or are you trying to say that Galaxy requires the data to
> >> be available on all of the nodes?
> >>
> >> -John
> >>
> >> On Tue, Dec 17, 2013 at 11:15 AM, Ben Gift  wrote:
> >> > We've run into a scenario lately where we need to run a very large
> >> > workflow
> >> > (huge data in intermediate steps) many times. We can't do this because
> >> > Galaxy copies all intermediate steps to all notes, which would bog
> down
> >> > the
> >> > servers too much.
> >> >
> >> > I asked about something similar before and John mentioned the feature
> to
> >> > automatically delete intermediate step data in a workflow once it
> >> > completed,
> >> > was coming soon. Is that a feature now? That would help.
> >> >
> >> > Ultimately though we can't be copying all this data around to all
> nodes.
> >> > The
> >> > network just isn't good enough, so I have an idea.
> >> >
> >> > What if we have an option on the 'run workflow' screen to only run on
> >> > one
> >> > node (eliminating the neat Galaxy concurrency ability for that
> workflow
> >> > unfortunately)? Then it just propagates the final step data.
> >> >
> >> > Or maybe only copy to a couple other nodes, to keep concurrency.
> >> >
> >> > If the job errored then in this case I think it should just throw out
> >> > all
> >> > the data, or propagate where it stopped.
> >> >
> >> > I've been trying to work on implementing this myself but it's taking
> me
> >> > a
> >> > long time. I only just started understanding the pyramid stack, and am
> >> > putting in the checkbox in the run.mako template. I still n

Re: [galaxy-dev] How to use Galaxy API?

[Martin Čech]

Hi,

the best start for using the API, that you found already, is the
documentation that, with a nice introduction, is nicely available here:
https://galaxy-central.readthedocs.org/en/latest/lib/galaxy.webapps.galaxy.api.html

Regarding libraries your main controllers of interest are: *libraries*,
*library_contents*, *folders, **folder_contents* - *please beware that
folders API is considered work-in-progress and will probably be deprecated
in future releases.*

If you want to test how the API behaves you can use something like Postman
extension (
https://chrome.google.com/webstore/detail/postman-rest-client/fdmmgilgnpjigdojojpjoooidkmcomcm?hl=en)
for chrome or other browser that allows you to nicely simulate API calls.

As of your error make sure that your local Galaxy, that you are trying to
connect to via API, runs on the 4096 port that you specified for the script
(by default local Galaxy runs at port 8080 - see universe_wsgi.ini for
config changes).

Thank you for using Galaxy and let us know how it goes!

Martin, Galaxy Team



On Tue, Dec 17, 2013 at 9:26 PM, xlwang  wrote:

> Hello,
> I want to know how to use Galaxy API(the path is
> /galaxy-dist/scripts/api). I follow the README but get errors. For
> instance, I have generated my_key and use this script:
> ./display.py my_key http://localhost:4096/api/libraries
> I get an error:
> 
> I do not know how it happened. Please tell me how to fix it. Thank you!
>
> In addition, my main purpose is to manage the data library in another
> system. Should I use the api such as library_create_library.py,
> library_upload_from_import_dir.py and so on?
>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] How to use Galaxy API?

[xlwang]

Hello,
I want to know how to use Galaxy API(the path is /galaxy-dist/scripts/api). I 
follow the README but get errors. For instance, I have generated my_key and use 
this script:
./display.py my_key http://localhost:4096/api/libraries
I get an error:

I do not know how it happened. Please tell me how to fix it. Thank you!
 
In addition, my main purpose is to manage the data library in another system. 
Should I use the api such as library_create_library.py, 
library_upload_from_import_dir.py and so on?___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Bug: Toolbox filters not applied in workflows

[Eric Rasche]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

John,

I've been using the dynamic jobs and they've been working beautifully!
(For context, I wrote the initial revision of the Access Control page
based off of my readings and experiences setting it up in galaxy).

I, for one, would be in favour of "stronger" filters in addition to the
current versions of filtering, mostly due to my/IT's paranoid nature :)
The current filters seem to accomplish exactly what they were designed
to, and I see no reason to replace them.

On 12/17/2013 06:15 PM, John Chilton wrote:
> Agreed with Dannon, it is a bug that they are showing up in the workflow
> editor but in general toolbox filters are not meant as a security
mechanism.
>
> You can block access to running specific tools (filtered or not) using
> dynamic job destinations. (Under dynamic job destinations documentation (
>
https://wiki.galaxyproject.org/Admin/Config/Jobs#Dynamic_Destination_Mapping)
> there is an example of enforcing "dev" only tools for a specific set of
> e-mails as well as an example of how to get a list of admin users in a
> dynamic job destination. These could be combined to enforce admin only
> tools. I think there has been some other posts on this topic... )
>
> It is true that one can still see the options of such tools, this is
> something we will have to think about... Perhaps a 'stronger' filter,
> though I think this variant of filtering should still be an option, there
> are many use cases for allowing users to run tools they cannot see.
>
> -John
>
>
>
> On Tue, Dec 17, 2013 at 5:39 PM, Dannon Baker
wrote:
>
>> This sounds like a bug; the primary toolbox and workflow editor toolbox
>> should reflect the same set of tools (exception being workflow-specific
>> control steps, etc).  I've created a trello card to track this issue
here:
>> https://trello.com/c/3TxFHkYR
>>
>> That said, do note the warning on the dynamic toolbox filters page:
>>
>> [image: ]  Filters will only hide Tools from the User Interface, they
>> are still available and can be made visible by means of HTML
manipulation.
>> That said these feature is not a security feature, it is intended to
>> separate multiple groups of Tools and simplify the
ToolBox
>> .
>>
>>
>> On Tue, Dec 17, 2013 at 6:27 PM, Eric Rasche
wrote:
>>
>>>  I'm running the stable copy of galaxy and noticed that some custom,
>>> administrative tools (and otherwise tools which should be restricted in
>>> access due to licensing/etc.) were showing up in normal user's toolboxes
>>> inside the workflow editor.
>>>
>>> I feel that this is a bug, as the tool filters should be applied
globally
>>> and not just in terms of what tools users are restricted from seeing  in
>>> the normal toolbox.
>>>
>>> For me, this presents a problem as I strongly believe any administrative
>>> tools that exist should leak as little information as possible--not
their
>>> entire set of options and associated documentation. Additionally,
that sort
>>> of information leakage isn't acceptable by my organisation's policies.
>>>
>>> Do I have my instance misconfigured or is this an actual bug?
>>> I have my galaxy configured according to
>>> https://wiki.galaxyproject.org/Admin/Config/Access%20Control
>>>
>>> $ hg head
>>> changeset:   11242:9d4cbf2a1c13
>>> branch:  stable
>>> tag: tip
>>> user:Nate Coraor  
>>> date:Fri Dec 06 16:28:31 2013 -0500
>>> summary: Add missing destination long arg to cli runner's Torque
>>> plugin and fix an incorrectly used PBS option in the sample job conf.
>>>
>>> changeset:   11216:c458a0fe1ba8
>>> parent:  11213:6d633418ecfa
>>> parent:  11215:f79149dd3d35
>>> user:Nate Coraor  
>>> date:Mon Nov 04 14:56:57 2013 -0500
>>> summary: Merge security fix for filtering tools from
>>> stable/next-stable.
>>>
>>> $ hg summary
>>> parent: 11242:9d4cbf2a1c13 tip
>>>  Add missing destination long arg to cli runner's Torque plugin and fix
>>> an incorrectly used PBS option in the sample job conf.
>>> branch: stable
>>> commit: 4 modified, 34 unknown
>>> update: (current)
>>>
>>>
>>> Thank you,
>>> Eric Rasche
>>>
>>> Programmer II
>>> Center for Phage Technology
>>> Texas A&M University
>>>
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>   http://lists.bx.psu.edu/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>   http://galaxyproject.org/search/mailinglists/
>>>
>>
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>

Re: [galaxy-dev] Bug: Toolbox filters not applied in workflows

[John Chilton]

Agreed with Dannon, it is a bug that they are showing up in the workflow
editor but in general toolbox filters are not meant as a security mechanism.

You can block access to running specific tools (filtered or not) using
dynamic job destinations. (Under dynamic job destinations documentation (
https://wiki.galaxyproject.org/Admin/Config/Jobs#Dynamic_Destination_Mapping)
there is an example of enforcing "dev" only tools for a specific set of
e-mails as well as an example of how to get a list of admin users in a
dynamic job destination. These could be combined to enforce admin only
tools. I think there has been some other posts on this topic... )

It is true that one can still see the options of such tools, this is
something we will have to think about... Perhaps a 'stronger' filter,
though I think this variant of filtering should still be an option, there
are many use cases for allowing users to run tools they cannot see.

-John



On Tue, Dec 17, 2013 at 5:39 PM, Dannon Baker wrote:

> This sounds like a bug; the primary toolbox and workflow editor toolbox
> should reflect the same set of tools (exception being workflow-specific
> control steps, etc).  I've created a trello card to track this issue here:
> https://trello.com/c/3TxFHkYR
>
> That said, do note the warning on the dynamic toolbox filters page:
>
> [image: ]  Filters will only hide Tools from the User Interface, they
> are still available and can be made visible by means of HTML manipulation.
> That said these feature is not a security feature, it is intended to
> separate multiple groups of Tools and simplify the 
> ToolBox
> .
>
>
> On Tue, Dec 17, 2013 at 6:27 PM, Eric Rasche wrote:
>
>>  I'm running the stable copy of galaxy and noticed that some custom,
>> administrative tools (and otherwise tools which should be restricted in
>> access due to licensing/etc.) were showing up in normal user's toolboxes
>> inside the workflow editor.
>>
>> I feel that this is a bug, as the tool filters should be applied globally
>> and not just in terms of what tools users are restricted from seeing  in
>> the normal toolbox.
>>
>> For me, this presents a problem as I strongly believe any administrative
>> tools that exist should leak as little information as possible--not their
>> entire set of options and associated documentation. Additionally, that sort
>> of information leakage isn't acceptable by my organisation's policies.
>>
>> Do I have my instance misconfigured or is this an actual bug?
>> I have my galaxy configured according to
>> https://wiki.galaxyproject.org/Admin/Config/Access%20Control
>>
>> $ hg head
>> changeset:   11242:9d4cbf2a1c13
>> branch:  stable
>> tag: tip
>> user:Nate Coraor  
>> date:Fri Dec 06 16:28:31 2013 -0500
>> summary: Add missing destination long arg to cli runner's Torque
>> plugin and fix an incorrectly used PBS option in the sample job conf.
>>
>> changeset:   11216:c458a0fe1ba8
>> parent:  11213:6d633418ecfa
>> parent:  11215:f79149dd3d35
>> user:Nate Coraor  
>> date:Mon Nov 04 14:56:57 2013 -0500
>> summary: Merge security fix for filtering tools from
>> stable/next-stable.
>>
>> $ hg summary
>> parent: 11242:9d4cbf2a1c13 tip
>>  Add missing destination long arg to cli runner's Torque plugin and fix
>> an incorrectly used PBS option in the sample job conf.
>> branch: stable
>> commit: 4 modified, 34 unknown
>> update: (current)
>>
>>
>> Thank you,
>> Eric Rasche
>>
>> Programmer II
>> Center for Phage Technology
>> Texas A&M University
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
>>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Request: Option to reduce server data transfer for big workflow in cluster

[Ben Gift]

How do you have it set up on the main public galaxy install? I imagine that
people run enough big jobs that there there is enormous use of your shared
file system. How did you scale that to so many nodes without bogging down
the file system with large dataset transfers?

It seems that for now the solution of having a second Galaxy instance will
work well, thank you very much John :) . But I'm still interested in a more
permanent scaled solution. After reading up more on our shared file system
it still seems like heavy traffic is bad, so could my initial idea still be
good?
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Bug: Toolbox filters not applied in workflows

[Eric Rasche]

Okay, thanks for confirming and creating the card.

Yes...it's unfortunate that tool filters will not block tools entirely,
but this will get it out of the users' view in one more place.


On 12/17/2013 05:39 PM, Dannon Baker wrote:
> This sounds like a bug; the primary toolbox and workflow editor toolbox
> should reflect the same set of tools (exception being workflow-specific
> control steps, etc).  I've created a trello card to track this issue here:
> https://trello.com/c/3TxFHkYR
>
> That said, do note the warning on the dynamic toolbox filters page:
>
> [image: ]  Filters will only hide Tools from the User Interface, they
> are still available and can be made visible by means of HTML manipulation.
> That said these feature is not a security feature, it is intended to
> separate multiple groups of Tools and simplify the
> ToolBox
> .
>
>
> On Tue, Dec 17, 2013 at 6:27 PM, Eric Rasche  wrote:
>
>>  I'm running the stable copy of galaxy and noticed that some custom,
>> administrative tools (and otherwise tools which should be restricted in
>> access due to licensing/etc.) were showing up in normal user's toolboxes
>> inside the workflow editor.
>>
>> I feel that this is a bug, as the tool filters should be applied globally
>> and not just in terms of what tools users are restricted from seeing  in
>> the normal toolbox.
>>
>> For me, this presents a problem as I strongly believe any administrative
>> tools that exist should leak as little information as possible--not their
>> entire set of options and associated documentation. Additionally, that sort
>> of information leakage isn't acceptable by my organisation's policies.
>>
>> Do I have my instance misconfigured or is this an actual bug?
>> I have my galaxy configured according to
>> https://wiki.galaxyproject.org/Admin/Config/Access%20Control
>>
>> $ hg head
>> changeset:   11242:9d4cbf2a1c13
>> branch:  stable
>> tag: tip
>> user:Nate Coraor  
>> date:Fri Dec 06 16:28:31 2013 -0500
>> summary: Add missing destination long arg to cli runner's Torque
>> plugin and fix an incorrectly used PBS option in the sample job conf.
>>
>> changeset:   11216:c458a0fe1ba8
>> parent:  11213:6d633418ecfa
>> parent:  11215:f79149dd3d35
>> user:Nate Coraor  
>> date:Mon Nov 04 14:56:57 2013 -0500
>> summary: Merge security fix for filtering tools from
>> stable/next-stable.
>>
>> $ hg summary
>> parent: 11242:9d4cbf2a1c13 tip
>>  Add missing destination long arg to cli runner's Torque plugin and fix an
>> incorrectly used PBS option in the sample job conf.
>> branch: stable
>> commit: 4 modified, 34 unknown
>> update: (current)
>>
>>
>> Thank you,
>> Eric Rasche
>>
>> Programmer II
>> Center for Phage Technology
>> Texas A&M University
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
>>


-- 
Eric Rasche
Programmer II
Center for Phage Technology
Texas A&M University
College Station, TX 77843
404-692-2048 
e...@tamu.edu 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Bug: Toolbox filters not applied in workflows

[Dannon Baker]

This sounds like a bug; the primary toolbox and workflow editor toolbox
should reflect the same set of tools (exception being workflow-specific
control steps, etc).  I've created a trello card to track this issue here:
https://trello.com/c/3TxFHkYR

That said, do note the warning on the dynamic toolbox filters page:

[image: ]  Filters will only hide Tools from the User Interface, they
are still available and can be made visible by means of HTML manipulation.
That said these feature is not a security feature, it is intended to
separate multiple groups of Tools and simplify the
ToolBox
.


On Tue, Dec 17, 2013 at 6:27 PM, Eric Rasche  wrote:

>  I'm running the stable copy of galaxy and noticed that some custom,
> administrative tools (and otherwise tools which should be restricted in
> access due to licensing/etc.) were showing up in normal user's toolboxes
> inside the workflow editor.
>
> I feel that this is a bug, as the tool filters should be applied globally
> and not just in terms of what tools users are restricted from seeing  in
> the normal toolbox.
>
> For me, this presents a problem as I strongly believe any administrative
> tools that exist should leak as little information as possible--not their
> entire set of options and associated documentation. Additionally, that sort
> of information leakage isn't acceptable by my organisation's policies.
>
> Do I have my instance misconfigured or is this an actual bug?
> I have my galaxy configured according to
> https://wiki.galaxyproject.org/Admin/Config/Access%20Control
>
> $ hg head
> changeset:   11242:9d4cbf2a1c13
> branch:  stable
> tag: tip
> user:Nate Coraor  
> date:Fri Dec 06 16:28:31 2013 -0500
> summary: Add missing destination long arg to cli runner's Torque
> plugin and fix an incorrectly used PBS option in the sample job conf.
>
> changeset:   11216:c458a0fe1ba8
> parent:  11213:6d633418ecfa
> parent:  11215:f79149dd3d35
> user:Nate Coraor  
> date:Mon Nov 04 14:56:57 2013 -0500
> summary: Merge security fix for filtering tools from
> stable/next-stable.
>
> $ hg summary
> parent: 11242:9d4cbf2a1c13 tip
>  Add missing destination long arg to cli runner's Torque plugin and fix an
> incorrectly used PBS option in the sample job conf.
> branch: stable
> commit: 4 modified, 34 unknown
> update: (current)
>
>
> Thank you,
> Eric Rasche
>
> Programmer II
> Center for Phage Technology
> Texas A&M University
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Bug: Toolbox filters not applied in workflows

[Eric Rasche]

I'm running the stable copy of galaxy and noticed that some custom,
administrative tools (and otherwise tools which should be restricted in
access due to licensing/etc.) were showing up in normal user's toolboxes
inside the workflow editor.

I feel that this is a bug, as the tool filters should be applied
globally and not just in terms of what tools users are restricted from
seeing  in the normal toolbox.

For me, this presents a problem as I strongly believe any administrative
tools that exist should leak as little information as possible--not
their entire set of options and associated documentation. Additionally,
that sort of information leakage isn't acceptable by my organisation's
policies.

Do I have my instance misconfigured or is this an actual bug?
I have my galaxy configured according to
https://wiki.galaxyproject.org/Admin/Config/Access%20Control

$ hg head
changeset:   11242:9d4cbf2a1c13
branch:  stable
tag: tip
user:Nate Coraor 
date:Fri Dec 06 16:28:31 2013 -0500
summary: Add missing destination long arg to cli runner's Torque
plugin and fix an incorrectly used PBS option in the sample job conf.

changeset:   11216:c458a0fe1ba8
parent:  11213:6d633418ecfa
parent:  11215:f79149dd3d35
user:Nate Coraor 
date:Mon Nov 04 14:56:57 2013 -0500
summary: Merge security fix for filtering tools from stable/next-stable.

$ hg summary
parent: 11242:9d4cbf2a1c13 tip
 Add missing destination long arg to cli runner's Torque plugin and fix
an incorrectly used PBS option in the sample job conf.
branch: stable
commit: 4 modified, 34 unknown
update: (current)


Thank you,
Eric Rasche

Programmer II
Center for Phage Technology
Texas A&M University
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] sort list of ftp uploaded files

[Curtis Hendrickson (Campus)]

John, 

I can't speak for Ulf, but a more general solution would be to allow sorting by 
a standard unix SORT set keys. 
That would allow things like - sort by the sample ID after the 3rd dash, then 
by the read direction. 
-t - -k1,1n -k2,2r # http://unixhelp.ed.ac.uk/CGI/man-cgi?sort

Regards, 
Curtis

PS - granted simple filename sort is a good place to start and  a lot better 
than nothing. Date sort is often useful, too. 


-Original Message-
From: galaxy-dev-boun...@lists.bx.psu.edu 
[mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of John Chilton
Sent: Tuesday, December 17, 2013 4:03 PM
To: Ulf Schaefer
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] sort list of ftp uploaded files

If Galaxy just sorted these files alphabetically before display/import would 
that fix your problem? Or do your users need to be able to modify the order?

-John

On Mon, Dec 16, 2013 at 9:19 AM, Ulf Schaefer  wrote:
> Dear all
>
> Is there a way to sort the list of files that a user has uploaded via 
> FTP? Currently they appear on the form of the tool 'Upload File' in 
> random order it seems. There is not much documentation on the  type="ftpfile" /> tag unfortunately.
>
> If the user just selects all files they end up in his/her history in 
> the same random order, which causes a slight inconvenience when using 
> these files as paired multiple input for a workflow.
>
> The obvious work-around is to put the files into the history one pair 
> at a time, but this becomes a bit onerous for larger number of files.
>
> Thanks a lot for your help.
>
> Cheers
> Ulf
>
> **
>  The information contained in the EMail and any attachments is 
> confidential and intended solely and for the attention and use of the 
> named addressee(s). It may not be disclosed to any other person 
> without the express authority of Public Health England, or the 
> intended recipient, or both. If you are not the intended recipient, 
> you must not disclose, copy, distribute or retain this message or any 
> part of it. This footnote also confirms that this EMail has been swept 
> for computer viruses by Symantec.Cloud, but please re-sweep any 
> attachments before opening or saving. http://www.gov.uk/PHE
> **
> 
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this and other 
> Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this and other Galaxy 
lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] sort list of ftp uploaded files

[John Chilton]

If Galaxy just sorted these files alphabetically before display/import
would that fix your problem? Or do your users need to be able to
modify the order?

-John

On Mon, Dec 16, 2013 at 9:19 AM, Ulf Schaefer  wrote:
> Dear all
>
> Is there a way to sort the list of files that a user has uploaded via
> FTP? Currently they appear on the form of the tool 'Upload File' in
> random order it seems. There is not much documentation on the  type="ftpfile" /> tag unfortunately.
>
> If the user just selects all files they end up in his/her history in the
> same random order, which causes a slight inconvenience when using these
> files as paired multiple input for a workflow.
>
> The obvious work-around is to put the files into the history one pair at
> a time, but this becomes a bit onerous for larger number of files.
>
> Thanks a lot for your help.
>
> Cheers
> Ulf
>
> **
> The information contained in the EMail and any attachments is confidential 
> and intended solely and for the attention and use of the named addressee(s). 
> It may not be disclosed to any other person without the express authority of 
> Public Health England, or the intended recipient, or both. If you are not the 
> intended recipient, you must not disclose, copy, distribute or retain this 
> message or any part of it. This footnote also confirms that this EMail has 
> been swept for computer viruses by Symantec.Cloud, but please re-sweep any 
> attachments before opening or saving. http://www.gov.uk/PHE
> **
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Trinity in local

[Jennifer Jackson]

Hello Jorge,

I just replied to your other question on the galaxy-user list with all 
of the details about configuring tools (wrappers vs binaries).


For this tool, the binary components most likely need to be installed 
and configured.


http://trinityrnaseq.sourceforge.net


Best,

Jen
Galaxy team

On 12/17/13 2:17 AM, Jorge Braun wrote:

hi mates,

I have problems to run trinity in local galaxy, because there is no 
executable script.


any ideas?

thank you very much


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
   http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/


--
Jennifer Hillman-Jackson
http://galaxyproject.org

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] "Skip automated testing of tools in this revision" setting?

[Greg Von Kuster]

Hi Nicola,

There's some bad news and good new regarding this issue.  

The bad news is that the old version of the tool shed's install and test 
framework required complete reengineering, and this resulted in it not being 
functional in the main tool shed's environment which runs the Galaxy stable 
branch. I've tried several approaches to committing and grafting changes to the 
stable branch in an attempt to piece together a functional environment for the 
main tool shed until the next Galaxy release, but the reengineering effort was 
so extensive that it made this extremely difficult.  

So the main tool shed cannot be tested until the next-stable branch is created 
in preparation for the next Galaxy release.  The next-stable branch is 
currently scheduled for mid January, 2014, with the Galaxy release currently 
scheduled for the first week of February, 2014.

The good news is that the reengineerig effort has resulted in a much more 
stable and functional install and test framework that will provide many more 
benefits than its previous incarnation.  I am now working on enhancing the 
framework to include installing and testing repositories of type 
tool_dependency_definition.  This enhancement will be included in the 
next-stable branch when it is created in mid-January.

Sorry for the inconvenience this may cause, but after mid-January things should 
be in good shape on the main tool shed.

Greg Von Kuster


On Dec 12, 2013, at 5:00 PM, Nicola Soranzo  wrote:

> Hi Greg,
> thanks for looking at this!
> Unfortunately still no test results displayed for
> http://toolshed.g2.bx.psu.edu/view/crs4/blat
> 
> Best,
> Nicola
> 
> Il giorno mar, 10/12/2013 alle 10.43 -0500, Greg Von Kuster ha scritto: 
>> Hello Nicola,
>> 
>> The main tool shed is running a bit behind the test tool shed since it
>> is on the stable branch. I've commited a fix for the main tool shed in
>> 11664:3bf805dfa14e that should correct the problem resulting in no
>> test results being displayed. Results should begin being displayed
>> with the test run tomorrow morning. I'll make sure to inspect the test
>> runs for the main tool shed more closely from here on (I've been
>> focusing more on the test tool shed) to make sure the different issues
>> that arise on the stable branch are handled more quickly. 
>> 
>> Thanks for letting me know about this.
>> 
>> Greg Von Kuster
>> 
>> On Dec 10, 2013, at 10:09 AM, Nicola Soranzo  wrote:
>> 
>>> Hi Greg!
>>> I don't know if it's the same problem, but I've not been able to see
>>> test results for this repository for a while:
>>> 
>>> http://toolshed.g2.bx.psu.edu/view/crs4/blat
>>> 
>>> Best,
>>> Nicola
>>> 
>>> Il giorno mar, 10/12/2013 alle 09.46 -0500, Greg Von Kuster ha scritto: 
 There are several "flags" for each repository revision that determine if 
 and how the revision is tested.  If tested, flags also detemine which 
 category the latest revision falls into when clicking on the links in the 
 Tool Shed's menu.  One of these flags is "tools_functionally_correct", 
 which was incorrectly set on 
 http://testtoolshed.g2.bx.psu.edu/view/peterjc/tmhmm_and_signalp/ee10017fcd80
  due to a bug introduced during the reengineering of the Topll Shed's 
 install and test framework that I recently did.  This bug was corrected in 
 the following change set.
 
 https://bitbucket.org/galaxy/galaxy-central/commits/ca25579a90e97157e32b527e7471cde90e9beed0
 
 I've corrected the setting for the tools_functionally_correct flag for 
 your repository so it should no longer be displayed in the "Latest 
 revision: failing tool tests" category.
 
 Thanks for letting us know about this, and please let us know if you 
 discover other reposiory revisions that require the same correction.
 
 Greg Von Kuster
 
 
 On Dec 9, 2013, at 5:06 AM, Peter Cock  wrote:
 
> Hi Dave & Greg,
> 
> Could you double check how the "Skip automated testing of tools in
> this revision" setting is being used on the Test Tool Shed?
> 
> e.g. 
> http://testtoolshed.g2.bx.psu.edu/view/peterjc/tmhmm_and_signalp/ee10017fcd80
> 
> Test runs
> Automated test environment
> Not tested
> Licensing issues restrict automated tool dependency installation.
> 2013-11-29 17:19:34
> Automated test environment
> Not tested
> Licensing issues restrict automated tool dependency installation.
> 
> (Ticked) Skip automated testing of tools in this revision
> 
> That all looks fine, and it is listed under "Latest revision: skip tool 
> tests".
> 
> However, this tool is also listed under "Latest revision: failing tool 
> tests".
> 
> Regards,
> 
> Peter
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and

Re: [galaxy-dev] Request: Option to reduce server data transfer for big workflow in cluster

[John Chilton]

Hey Ben,

  Hmmm... I don't think Galaxy is doing that - not directly anyway.
Unless I am mistaken, Galaxy will put the file in one location on the
web server node or VM. Typically, this location is on a filesystem
that is shared between the web server and a cluster's compute nodes.
So I wouldn't describe that as Galaxy copying the data to all of the
compute nodes. If your cluster doesn't have a shared file system and
to get around this someone has configured the Galaxy data to be synced
across all nodes - that would be copying the data to all nodes but I
don't think Galaxy is doing that. Alternatively, you might have condor
configured to copy the galaxy data to the remote nodes in such a way
that everytime a Galaxy job is run on a node, all the data is copied
to that node?

  Does that make sense?

  So I still don't entirely understand your setup, but my advice is
pretty general - for now you may want to solve this problem at the
condor level. I am assuming this is a general purpose condor cluster
and not setup explicitly for Galaxy? Lets say you have 200 nodes in
your condor cluster and they cannot all mount the Galaxy filesystem -
because it would overload the file server being used by Galaxy. I
think you could setup a FileSystemDomain at the condor level that just
10 of nodes belonged to say (these 10 nodes can continue to run
anything in general but Galaxy will only submit to these) This
filesystemdomain could have a name like galaxy.example.com if
example.com is your default filesystemdomain. Then you can setup the
Galaxy condor runner with a requirement such that
"FileSystemDomain=galaxy.example.com" and Galaxy jobs will only run on
these 10 nodes then. Having 10 nodes mount a file server is much more
manageable than 200.

-John


On Tue, Dec 17, 2013 at 11:52 AM, Ben Gift  wrote:
> Hi John, thanks for the reply.
>
> Yes, I mean Galaxy's default behavior of keeping all the data on all nodes
> of our condor cluster. So for instance if I run a job, then the output of
> that job is copied to every node in the cluster. Is this not the normal
> behavior?
>
>
> On Tue, Dec 17, 2013 at 9:42 AM, John Chilton  wrote:
>>
>> Hey Ben,
>>
>> Thanks for the e-mail. I did not promise anything was coming soon, I
>> only said people were working on parts of it. It is not a feature yet
>> unfortunately - multiple people including myself are thinking about
>> various parts of this problem though.
>>
>> I would like to respond, but I am trying to understand this line: "We
>> can't do this because Galaxy copies all intermediate steps to all
>> no(d)es, which would bog down the servers too much."
>>
>> Can you describe how you are doing this staging for me? Is data
>> currently being copied around to all the nodes, if so how are you
>> doing that? Or are you trying to say that Galaxy requires the data to
>> be available on all of the nodes?
>>
>> -John
>>
>> On Tue, Dec 17, 2013 at 11:15 AM, Ben Gift  wrote:
>> > We've run into a scenario lately where we need to run a very large
>> > workflow
>> > (huge data in intermediate steps) many times. We can't do this because
>> > Galaxy copies all intermediate steps to all notes, which would bog down
>> > the
>> > servers too much.
>> >
>> > I asked about something similar before and John mentioned the feature to
>> > automatically delete intermediate step data in a workflow once it
>> > completed,
>> > was coming soon. Is that a feature now? That would help.
>> >
>> > Ultimately though we can't be copying all this data around to all nodes.
>> > The
>> > network just isn't good enough, so I have an idea.
>> >
>> > What if we have an option on the 'run workflow' screen to only run on
>> > one
>> > node (eliminating the neat Galaxy concurrency ability for that workflow
>> > unfortunately)? Then it just propagates the final step data.
>> >
>> > Or maybe only copy to a couple other nodes, to keep concurrency.
>> >
>> > If the job errored then in this case I think it should just throw out
>> > all
>> > the data, or propagate where it stopped.
>> >
>> > I've been trying to work on implementing this myself but it's taking me
>> > a
>> > long time. I only just started understanding the pyramid stack, and am
>> > putting in the checkbox in the run.mako template. I still need to learn
>> > the
>> > database schema, message passing, and how jobs are stored, and how to
>> > tell
>> > condor to only use 1 node, (and more I'm sure) in Galaxy. (I'm drowning)
>> >
>> > This seems like a really important feature though as Galaxy gains more
>> > traction as a research tool for bigger projects that demand working with
>> > huge data, and running huge workflows many many times.
>> >
>> > ___
>> > Please keep all replies on the list by using "reply all"
>> > in your mail client.  To manage your subscriptions to this
>> > and other Galaxy lists, please use the interface at:
>> >   http://lists.bx.psu.edu/
>> >
>> > To search Galaxy mailing list

Re: [galaxy-dev] Request: Option to reduce server data transfer for big workflow in cluster

[Ben Gift]

Hi John, thanks for the reply.

Yes, I mean Galaxy's default behavior of keeping all the data on all nodes
of our condor cluster. So for instance if I run a job, then the output of
that job is copied to every node in the cluster. Is this not the normal
behavior?


On Tue, Dec 17, 2013 at 9:42 AM, John Chilton  wrote:

> Hey Ben,
>
> Thanks for the e-mail. I did not promise anything was coming soon, I
> only said people were working on parts of it. It is not a feature yet
> unfortunately - multiple people including myself are thinking about
> various parts of this problem though.
>
> I would like to respond, but I am trying to understand this line: "We
> can't do this because Galaxy copies all intermediate steps to all
> no(d)es, which would bog down the servers too much."
>
> Can you describe how you are doing this staging for me? Is data
> currently being copied around to all the nodes, if so how are you
> doing that? Or are you trying to say that Galaxy requires the data to
> be available on all of the nodes?
>
> -John
>
> On Tue, Dec 17, 2013 at 11:15 AM, Ben Gift  wrote:
> > We've run into a scenario lately where we need to run a very large
> workflow
> > (huge data in intermediate steps) many times. We can't do this because
> > Galaxy copies all intermediate steps to all notes, which would bog down
> the
> > servers too much.
> >
> > I asked about something similar before and John mentioned the feature to
> > automatically delete intermediate step data in a workflow once it
> completed,
> > was coming soon. Is that a feature now? That would help.
> >
> > Ultimately though we can't be copying all this data around to all nodes.
> The
> > network just isn't good enough, so I have an idea.
> >
> > What if we have an option on the 'run workflow' screen to only run on one
> > node (eliminating the neat Galaxy concurrency ability for that workflow
> > unfortunately)? Then it just propagates the final step data.
> >
> > Or maybe only copy to a couple other nodes, to keep concurrency.
> >
> > If the job errored then in this case I think it should just throw out all
> > the data, or propagate where it stopped.
> >
> > I've been trying to work on implementing this myself but it's taking me a
> > long time. I only just started understanding the pyramid stack, and am
> > putting in the checkbox in the run.mako template. I still need to learn
> the
> > database schema, message passing, and how jobs are stored, and how to
> tell
> > condor to only use 1 node, (and more I'm sure) in Galaxy. (I'm drowning)
> >
> > This seems like a really important feature though as Galaxy gains more
> > traction as a research tool for bigger projects that demand working with
> > huge data, and running huge workflows many many times.
> >
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client.  To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> >   http://lists.bx.psu.edu/
> >
> > To search Galaxy mailing lists use the unified search at:
> >   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Request: Option to reduce server data transfer for big workflow in cluster

[John Chilton]

Hey Ben,

Thanks for the e-mail. I did not promise anything was coming soon, I
only said people were working on parts of it. It is not a feature yet
unfortunately - multiple people including myself are thinking about
various parts of this problem though.

I would like to respond, but I am trying to understand this line: "We
can't do this because Galaxy copies all intermediate steps to all
no(d)es, which would bog down the servers too much."

Can you describe how you are doing this staging for me? Is data
currently being copied around to all the nodes, if so how are you
doing that? Or are you trying to say that Galaxy requires the data to
be available on all of the nodes?

-John

On Tue, Dec 17, 2013 at 11:15 AM, Ben Gift  wrote:
> We've run into a scenario lately where we need to run a very large workflow
> (huge data in intermediate steps) many times. We can't do this because
> Galaxy copies all intermediate steps to all notes, which would bog down the
> servers too much.
>
> I asked about something similar before and John mentioned the feature to
> automatically delete intermediate step data in a workflow once it completed,
> was coming soon. Is that a feature now? That would help.
>
> Ultimately though we can't be copying all this data around to all nodes. The
> network just isn't good enough, so I have an idea.
>
> What if we have an option on the 'run workflow' screen to only run on one
> node (eliminating the neat Galaxy concurrency ability for that workflow
> unfortunately)? Then it just propagates the final step data.
>
> Or maybe only copy to a couple other nodes, to keep concurrency.
>
> If the job errored then in this case I think it should just throw out all
> the data, or propagate where it stopped.
>
> I've been trying to work on implementing this myself but it's taking me a
> long time. I only just started understanding the pyramid stack, and am
> putting in the checkbox in the run.mako template. I still need to learn the
> database schema, message passing, and how jobs are stored, and how to tell
> condor to only use 1 node, (and more I'm sure) in Galaxy. (I'm drowning)
>
> This seems like a really important feature though as Galaxy gains more
> traction as a research tool for bigger projects that demand working with
> huge data, and running huge workflows many many times.
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Request: Option to reduce server data transfer for big workflow in cluster

[Ben Gift]

We've run into a scenario lately where we need to run a very large workflow
(huge data in intermediate steps) many times. We can't do this because
Galaxy copies all intermediate steps to all notes, which would bog down the
servers too much.

I asked about something similar before and John mentioned the feature to
automatically delete intermediate step data in a workflow once it
completed, was coming soon. Is that a feature now? That would help.

Ultimately though we can't be copying all this data around to all nodes.
The network just isn't good enough, so I have an idea.

What if we have an option on the 'run workflow' screen to only run on one
node (eliminating the neat Galaxy concurrency ability for that workflow
unfortunately)? Then it just propagates the final step data.

Or maybe only copy to a couple other nodes, to keep concurrency.

If the job errored then in this case I think it should just throw out all
the data, or propagate where it stopped.

I've been trying to work on implementing this myself but it's taking me a
long time. I only just started understanding the pyramid stack, and am
putting in the checkbox in the run.mako template. I still need to learn the
database schema, message passing, and how jobs are stored, and how to tell
condor to only use 1 node, (and more I'm sure) in Galaxy. (I'm drowning)

This seems like a really important feature though as Galaxy gains more
traction as a research tool for bigger projects that demand working with
huge data, and running huge workflows many many times.
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Count intervals in one file overlapping intervals in another file

[Javier Simon-Sanchez]

Ok,

So I guess I need to install it locally or in the cloud to use certain tools, 
right?

Thanks!

From: Jennifer Jackson [mailto:j...@bx.psu.edu]
Sent: Monday, December 16, 2013 6:55 PM
To: Javier Simon-Sanchez; galaxy-...@bx.psu.edu
Subject: Re: [galaxy-dev] Count intervals in one file overlapping intervals in 
another file

Javier,

This is part of a tool repository that can be installed into a local, cloud, or 
slipstream Galaxy and used there.

http://getgalaxy.org
http://usegalaxy.org/cloud
https://wiki.galaxyproject.org/BigPicture/Choices
http://usegalaxy.org/toolshed
https://wiki.galaxyproject.org/Tool%20Shed#Installing.2C_maintaining_and_uninstalling_tool_shed_repositories_within_a_Galaxy_instance

I am not sure of the size of your files, but if not too large, these tools may 
work just fine on a local install on a laptop. For anything larger, a server 
would be needed for a local or for the easiest set-up, use a cloud. For ongoing 
work, a slipstream Galaxy can be a great solution for labs.

Take care,

Jen
Galaxy team
On 12/16/13 7:30 AM, Javier Simon-Sanchez wrote:
Hi Jennifer,

Intersect_bed is exactly what I need. How can I include this on my workflows? 
(sorry for asking so many things. Im new with galaxy)

From: Jennifer Jackson [mailto:j...@bx.psu.edu]
Sent: Monday, December 16, 2013 4:03 PM
To: Javier Simon-Sanchez; galaxy-...@bx.psu.edu
Subject: Re: [galaxy-dev] Count intervals in one file overlapping intervals in 
another file

Hello Javier,

If you would like to see an enhancement for a tool that is sourced from the 
tool shed, then the repository author is the correct person to contact. Go to 
http://usegalaxy.org/toolshed and log in to see contact information. The repo 
name is "bedtools".

The Tool Shed also has an alternate versions of some BedTools, including this 
one. An alternative has an option to set overlap length: " bed_intersect".

Also, there are tools in the group "Operate on Genomic Intervals". I don't know 
the details of your query, but these tools may also be helpful.

Best,

Jen
Galaxy team
On 12/13/13 2:41 AM, Javier Simon-Sanchez wrote:
Hello,

I just noticed that when using the above mentioned option, it is not possible 
to set a minimum overlap required. This option can be run in the command line 
version of BEDtools and I think is crucial. Could you please implement it in 
Galaxy?

Thanks a lot.

-
Javier Simón-Sánchez, PhD
Research Scientist
MRC-Holland
Willem Schoutenstraat 6
1057 DN, Amsterdam
The Netherlands






___

Please keep all replies on the list by using "reply all"

in your mail client.  To manage your subscriptions to this

and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/



To search Galaxy mailing lists use the unified search at:

  http://galaxyproject.org/search/mailinglists/




--

Jennifer Hillman-Jackson

http://galaxyproject.org



--

Jennifer Hillman-Jackson

http://galaxyproject.org
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Trinity in local

[Jorge Braun]

hi mates,

I have problems to run trinity in local galaxy, because there is no executable 
script.

any ideas?

thank you very much   ___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/