[galaxy-dev] Trouble with Toolshed on MySQL

2013-04-11 Thread Rob Hooft 
To ease the maintenance of our Dutch National galaxy, we're trying to set
up our own toolshed on a system that happens to have a nicely large MySQL
database running. We'd like to make it use that database. The setup has
been ok, until we started running the toolshed on the MySQL server for the
first time, which resulted in the message in the community_webapp.log I
copied below. Is this an incompatibility?

Regards,

Rob Hooft

  File
"lib/galaxy/webapps/community/model/migrate/versions/0001_initial_tables.py",
line 150, in upgrade
metadata.create_all()
...
self._handle_dbapi_exception(e, statement, parameters, cursor, context)
  File
"/opt/toolshed/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py",
line 931, in _handle_dbapi_exception
raise exc.DBAPIError.instance(statement, parameters, e,
connection_invalidated=is_disconnect)
OperationalError: (OperationalError) (1170, "BLOB/TEXT column 'annotation'
used in key specification without a key length") u'CREATE INDEX
ix_tool_annotation_association_annotation ON tool_annotation_association
(annotation)' ()
Removing PID file community_webapp.pid


-- 
Rob W.W. Hooft
Chief Technology Officer BioAssist, Netherlands Bioinformatics Centre
http://www.nbic.nl/Skype: robhooftGSM: +31 6 27034319
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Local Galaxy Install - Multiple Job Handlers - Uploads waiting to run endlessly

2013-04-11 Thread Adam Brenner 
FYI: Here are the entries in paster.log. If you notice no job_handlers are
being sent. I performed tests on both nginx reverse proxy and direct
localhost:8080 (limited to our private network on campus)

128.200.34.227 - - [11/Apr/2013:13:24:40 -0700] "GET
/tool_runner?tool_id=upload1 HTTP/1.0" 200 - "
http://ghtf-hpc.oit.uci.edu/root/tool_menu"; "Mozilla/5.0 (Macintosh; Intel
Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3
Safari/536.28.10"
galaxy.tools.actions.upload_common INFO 2013-04-11 13:24:44,173 tool
upload1 created job id 8
128.200.34.227 - - [11/Apr/2013:13:24:43 -0700] "POST /tool_runner/index
HTTP/1.0" 200 - "http://ghtf-hpc.oit.uci.edu/tool_runner?tool_id=upload1";
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10
(KHTML, like Gecko) Version/6.0.3 Safari/536.28.10"
128.200.34.227 - - [11/Apr/2013:13:24:44 -0700] "GET /history HTTP/1.0" 200
- "http://ghtf-hpc.oit.uci.edu/tool_runner/index"; "Mozilla/5.0 (Macintosh;
Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko)
Version/6.0.3 Safari/536.28.10"
128.200.34.227 - - [11/Apr/2013:13:24:44 -0700] "GET
/api/histories/56959efe847bb360 HTTP/1.0" 200 - "
http://ghtf-hpc.oit.uci.edu/history"; "Mozilla/5.0 (Macintosh; Intel Mac OS
X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3
Safari/536.28.10"


--
Adam Brenner
Computer Science, Undergraduate Student
Donald Bren School of Information and Computer Sciences

Research Computing Support
Office of Information Technology
http://www.oit.uci.edu/rcs/

University of California, Irvine
www.ics.uci.edu/~aebrenne/
aebre...@uci.edu


On Thu, Apr 11, 2013 at 12:39 PM, Adam Brenner  wrote:

> Howdy Galaxy Admins,
>
> I am installing Galaxy on our HPC cluster and running into a strange issue
> that I can no longer debug further. I am running the latest galaxy build as
> of Tuesday April 9th.
> hg clone https://bitbucket.org/galaxy/galaxy-dist/
> hg update stable
>
> When a user tries to upload a file via the web interface, nginx reverse
> proxy setup as described here
> http://wiki.galaxyproject.org/Admin/Config/Performance/nginx%20Proxy, the
> upload stays in a state of "Job is waiting to run."
>
> The job stays like this for both FTP uploads, and uploads via browser. The
> dataset is a 319 bytes (both FTP and upload via browser)
>
> My universe_wsgi.ini variables are like so:
>  .. under [app:main] 
>  new_file_path = /som/galaxy/tmp
>  file_path = /som/galaxy/datasets/files
>  track_jobs_in_database = True
>  job_manager = manager
>  job_handlers = handler0,handler1
>  ... before [app:main] in [server:main] 
>
>  [server:main]
>  use = egg:Paste#http
>  port = 8080
>  host = 127.0.0.1
>  use_threadpool = true
>  threadpool_workers = 5
>
>  [server:manager]
>  use = egg:Paste#http
>  port = 8079
>  host = 127.0.0.1
>  use_threadpool = true
>  threadpool_workers = 5
>
>  [server:handler0]
>  use = egg:Paste#http
>  port = 8090
>  host = 127.0.0.1
>  use_threadpool = true
>  threadpool_workers = 5
>
>  [server:handler1]
>  use = egg:Paste#http
>  port = 8091
>  host = 127.0.0.1
>  use_threadpool = true
>  threadpool_workers = 5
>
>
> Galaxy, for now, is running under the root user and is installed in
> /data/apps/galaxy (NFS mount) and the tmp files are stored on a GlusterFS
> system under /som/galaxy/.
>
> I have tried removing the extra job managers and handlers so I just have
> server:main and this still continues to happen. The paster.log files (with
> DEBUG = True) does not show any errors from python or issues writing to
> file. All POST and GET responses are recorded and nothing else.
>
> What I find strange is that if I look in new_file_path I do see my dataset
> as
> upload_file_data_Zx_1rK
> and cat'ing the file shows the correct contents. The items listed in
> my file_path (the dataset_*.dat) files are empty/blank..but are created.
>
> Any ideas as to why its failing? Is it nginx? I only have it redirect
> 8080, and none of the other job managers / handlers...is that an issue? Do
> you think its permissions on the my file_path or new_file_path directories?
>
> A very stumped Adam.
>
> --
> Adam Brenner
> Computer Science, Undergraduate Student
> Donald Bren School of Information and Computer Sciences
>
> Research Computing Support
> Office of Information Technology
> http://www.oit.uci.edu/rcs/
>
> University of California, Irvine
> www.ics.uci.edu/~aebrenne/
> aebre...@uci.edu
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Local Galaxy Install - Multiple Job Handlers - Uploads waiting to run endlessly

2013-04-11 Thread Adam Brenner 
Howdy Galaxy Admins,

I am installing Galaxy on our HPC cluster and running into a strange issue
that I can no longer debug further. I am running the latest galaxy build as
of Tuesday April 9th.
hg clone https://bitbucket.org/galaxy/galaxy-dist/
hg update stable

When a user tries to upload a file via the web interface, nginx reverse
proxy setup as described here
http://wiki.galaxyproject.org/Admin/Config/Performance/nginx%20Proxy, the
upload stays in a state of "Job is waiting to run."

The job stays like this for both FTP uploads, and uploads via browser. The
dataset is a 319 bytes (both FTP and upload via browser)

My universe_wsgi.ini variables are like so:
 .. under [app:main] 
 new_file_path = /som/galaxy/tmp
 file_path = /som/galaxy/datasets/files
 track_jobs_in_database = True
 job_manager = manager
 job_handlers = handler0,handler1
 ... before [app:main] in [server:main] 

 [server:main]
 use = egg:Paste#http
 port = 8080
 host = 127.0.0.1
 use_threadpool = true
 threadpool_workers = 5

 [server:manager]
 use = egg:Paste#http
 port = 8079
 host = 127.0.0.1
 use_threadpool = true
 threadpool_workers = 5

 [server:handler0]
 use = egg:Paste#http
 port = 8090
 host = 127.0.0.1
 use_threadpool = true
 threadpool_workers = 5

 [server:handler1]
 use = egg:Paste#http
 port = 8091
 host = 127.0.0.1
 use_threadpool = true
 threadpool_workers = 5


Galaxy, for now, is running under the root user and is installed in
/data/apps/galaxy (NFS mount) and the tmp files are stored on a GlusterFS
system under /som/galaxy/.

I have tried removing the extra job managers and handlers so I just have
server:main and this still continues to happen. The paster.log files (with
DEBUG = True) does not show any errors from python or issues writing to
file. All POST and GET responses are recorded and nothing else.

What I find strange is that if I look in new_file_path I do see my dataset
as
upload_file_data_Zx_1rK
and cat'ing the file shows the correct contents. The items listed in
my file_path (the dataset_*.dat) files are empty/blank..but are created.

Any ideas as to why its failing? Is it nginx? I only have it redirect 8080,
and none of the other job managers / handlers...is that an issue? Do you
think its permissions on the my file_path or new_file_path directories?

A very stumped Adam.

--
Adam Brenner
Computer Science, Undergraduate Student
Donald Bren School of Information and Computer Sciences

Research Computing Support
Office of Information Technology
http://www.oit.uci.edu/rcs/

University of California, Irvine
www.ics.uci.edu/~aebrenne/
aebre...@uci.edu
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy hg repository problem "abort: path ... traverses symbolic link"

2013-04-11 Thread Nate Coraor 
On Apr 11, 2013, at 12:39 PM, Peter Cock wrote:

> On Thu, Apr 11, 2013 at 5:02 PM, John Chilton  wrote:
>> Hey Peter,
>> 
>>  Hopefully Nate will have some wisdom, all I have is a workaround (I
>> have fought with this on multiple forks and this is the best I have
>> come up with).
>> 
>> Check out a clean version of the tools branch somehow (it seems like
>> the working contents can get stuck in an invalid state that update and
>> checkout cannot recover from). Once you are back to a good state, it
>> works when you merge in two steps.
>> 
>> hg checkout tools
>> hg merge -r ee389d2
>> hg ci -m "Partial merge."  # Not sure if this step is needed.
>> hg merge default
>> 
>> -John
> 
> I don't understand quite what it is doing, but it seems to have worked
> on a quick test repository (I re-cloned Galaxy and my fork on bitbucket).
> I've not just push any of these changes to bitbucket - they're all still
> local to this experiment for now.
> 
> Unless I hit an issue or Nate has some alternative recommendations,
> I'll probably do this tomorrow (its home time here in the UK).
> 
> Thanks John,
> 
> Peter


Hi Peter,

I don't see any problems with this.  You are merging twice, but simply doing so 
from two different points in the timeline.  Thanks for the solution, John.

--nate
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy hg repository problem "abort: path ... traverses symbolic link"

2013-04-11 Thread Peter Cock 
On Thu, Apr 11, 2013 at 5:02 PM, John Chilton  wrote:
> Hey Peter,
>
>   Hopefully Nate will have some wisdom, all I have is a workaround (I
> have fought with this on multiple forks and this is the best I have
> come up with).
>
> Check out a clean version of the tools branch somehow (it seems like
> the working contents can get stuck in an invalid state that update and
> checkout cannot recover from). Once you are back to a good state, it
> works when you merge in two steps.
>
> hg checkout tools
> hg merge -r ee389d2
> hg ci -m "Partial merge."  # Not sure if this step is needed.
> hg merge default
>
> -John

I don't understand quite what it is doing, but it seems to have worked
on a quick test repository (I re-cloned Galaxy and my fork on bitbucket).
I've not just push any of these changes to bitbucket - they're all still
local to this experiment for now.

Unless I hit an issue or Nate has some alternative recommendations,
I'll probably do this tomorrow (its home time here in the UK).

Thanks John,

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Broken tools ids from ./run_functional_tests.sh -list

2013-04-11 Thread Peter Cock 
Hi all,

Does anyone else see this problem with broken tool ids in the
function test script? I found this because one of my own tools
wasn't being recognised because for some reason the id was
treated as _name= [sic].

This happens on the default branch:

$ ./run_functional_tests.sh -list | grep "_name="
_name=  RNA/DNA
_name=  Convert
_name=  Compute quality
statistics
_name=  Draw quality score
boxplot
_name=  DAVID

$ hg branch
default

 hg log -b default | head
changeset:   9349:199b62339f26
user:jeremy goecks 
date:Thu Apr 11 08:44:42 2013 -0400
summary: Whitespace fixes for Vcf datatype.

changeset:   9348:dbfc964167ae
user:guerler
date:Wed Apr 10 17:30:27 2013 -0400
summary: Enhance button for trackster visualization in data display viewer


Have I found a recent regression, or is my system messed up
somehow?

Thanks,

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] heatmap capabilities!

2013-04-11 Thread Carl Eberhard 
Hello, Hakeem
Although I've never used it, you might try the java_genomics_toolkit
available as a tool shed repository (or have your installation's admin
install it). It contains a tool called 'Align values in a matrix' which may
work for your needs.

Thanks,
Carl



On Tue, Apr 2, 2013 at 12:51 PM, Hakeem Almabrazi wrote:

>  Hi,
>
> ** **
>
> Is there a heatmap tool for Galaxy?  If so can someone send me a link to
> how to integrate it to a local galaxy?
>
> ** **
>
> I appreciate your help.
>
> ** **
>
>  
>
> ** **
>
> ** **
>
> ** **
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy hg repository problem "abort: path ... traverses symbolic link"

2013-04-11 Thread John Chilton 
Hey Peter,

  Hopefully Nate will have some wisdom, all I have is a workaround (I
have fought with this on multiple forks and this is the best I have
come up with).

Check out a clean version of the tools branch somehow (it seems like
the working contents can get stuck in an invalid state that update and
checkout cannot recover from). Once you are back to a good state, it
works when you merge in two steps.

hg checkout tools
hg merge -r ee389d2
hg ci -m "Partial merge."  # Not sure if this step is needed.
hg merge default

-John

On Thu, Apr 11, 2013 at 10:47 AM, Peter Cock  wrote:
> (Apologies if this comes twice - it seems I didn't press send when I wrote
> the draft earlier in the week)
>
> On Tue, Feb 19, 2013 at 5:38 PM, Peter Cock  wrote:
>> On Tue, Feb 19, 2013 at 5:26 PM, Nate Coraor  wrote:
>>> On Feb 19, 2013, at 12:18 PM, Peter Cock wrote:
>>> Nate wrote:
> Hi Peter,
>
> This is due to the annoying way in which Mercurial handles symlinks.  
> We're
> going to graft the changesets that caused this up to stable shortly, but 
> in the
> meantime, you can do:
>
> % rm static/june_2007_style
> % hg update stable --clean
>
> This would remove any uncommitted local changes, if you had any, so it's
> not safe for all scenarios.  But it's fine here since you're working on a 
> fresh
> clone with no changes.

 Cheers Nate, that trick seems to work.

 I'm guessing this recent commit (11 days ago) is part of the issue:
 https://bitbucket.org/galaxy/galaxy-central/commits/506484344db3a370f8ae24096041d38557d1967e
>>>
>>> Indeed, that, and the previous commit.
>>>
 I hope this won't require any 'grafting' for those of us working from the
 default branch? Will I be safe merging the 'default' into my own branch,
 or should I wait while you guys sort out whatever you need to do first?
>>>
>>> It shouldn't require you to do anything.  The only complications that
>>> should arise will be if you have changes to files under 
>>> static/june_2007_style/
>>
>> I've not touched that, so that's all good news - thank you :)
>>
>> Peter
>
> It seems I've not updated my 'tools' development branch with the
> 'default' branch changes since this sym link issue.
>
> So I've switched to the default branch, done a pull, and this is
> now all up to date.
>
> $ hg checkout default --clean
> 0 files updated, 0 files merged, 0 files removed, 0 files unresolved
>
> $ hg update --clean
> 0 files updated, 0 files merged, 0 files removed, 0 files unresolved
>
> $ hg tip
> changeset:   10037:ce3143a06f21
> tag: tip
> user:jeremy goecks 
> date:Fri Apr 05 09:39:05 2013 -0400
> summary: Implement a reference-based compression approach for
> encoding aligned reads' sequence and cigar. Move cigar functions into
> own file.
>
> $ hg diff
> (nothing)
>
> At this point I would normally do: hg checkout tools,  hg merge default
>
> $ hg checkout tools
> abort: path 'static/june_2007_style/Makefile' traverses symbolic link
> 'static/june_2007_style'
>
> So, trying the trick from before:
>
> $ rm static/june_2007_style
> $ hg checkout tools
> abort: crosses branches (merge branches or use --clean to discard changes)
> $ hg checkout tools --clean
> 769 files updated, 0 files merged, 413 files removed, 0 files unresolved
>
> And now, the stumbling block:
>
> $ hg merge default
> merging datatypes_conf.xml.sample
> merging lib/galaxy/datatypes/registry.py
> merging lib/galaxy/jobs/__init__.py
> merging run.sh
> merging static/june_2007_style/blue/base.css and
> static/style/blue/base.css to static/style/blue/base.css
> merging tool_conf.xml.sample
> merging tools/filters/cutWrapper.pl
> abort: No such file or directory:
> /mnt/galaxy/galaxy-central/static/style/blue/base.css
>
> Advice from an hg guru would be appreciated.
>
> Thanks,
>
> Peter
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Dataset diskspace useage

2013-04-11 Thread Carl Eberhard 
Hello, Alexander, and thanks for the report

Do any of the bugged histories contain shared data (such as library
datasets)?


On Wed, Apr 3, 2013 at 5:27 AM, Alexander Kurze <
alexander.ku...@bioch.ox.ac.uk> wrote:

>
> Hello,
>
> I am running a local galaxy server where I have restricted the amount of
> data for each user to a 100gb. Users are able to delete their own history
> to regain diskspace. However, for some users deleting the histories (all
> saved histories and datasets) only frees ~80% of their allowance. As an
> admin I ran the "clean-up" scripts which were recommended on the galaxy
> homepage. But this still didn't increase their data limit. Anyone knows
> what is going on?
>
> Best wishes,
>
> Alex
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy hg repository problem "abort: path ... traverses symbolic link"

2013-04-11 Thread Peter Cock 
(Apologies if this comes twice - it seems I didn't press send when I wrote
the draft earlier in the week)

On Tue, Feb 19, 2013 at 5:38 PM, Peter Cock  wrote:
> On Tue, Feb 19, 2013 at 5:26 PM, Nate Coraor  wrote:
>> On Feb 19, 2013, at 12:18 PM, Peter Cock wrote:
>> Nate wrote:
 Hi Peter,

 This is due to the annoying way in which Mercurial handles symlinks.  We're
 going to graft the changesets that caused this up to stable shortly, but 
 in the
 meantime, you can do:

 % rm static/june_2007_style
 % hg update stable --clean

 This would remove any uncommitted local changes, if you had any, so it's
 not safe for all scenarios.  But it's fine here since you're working on a 
 fresh
 clone with no changes.
>>>
>>> Cheers Nate, that trick seems to work.
>>>
>>> I'm guessing this recent commit (11 days ago) is part of the issue:
>>> https://bitbucket.org/galaxy/galaxy-central/commits/506484344db3a370f8ae24096041d38557d1967e
>>
>> Indeed, that, and the previous commit.
>>
>>> I hope this won't require any 'grafting' for those of us working from the
>>> default branch? Will I be safe merging the 'default' into my own branch,
>>> or should I wait while you guys sort out whatever you need to do first?
>>
>> It shouldn't require you to do anything.  The only complications that
>> should arise will be if you have changes to files under 
>> static/june_2007_style/
>
> I've not touched that, so that's all good news - thank you :)
>
> Peter

It seems I've not updated my 'tools' development branch with the
'default' branch changes since this sym link issue.

So I've switched to the default branch, done a pull, and this is
now all up to date.

$ hg checkout default --clean
0 files updated, 0 files merged, 0 files removed, 0 files unresolved

$ hg update --clean
0 files updated, 0 files merged, 0 files removed, 0 files unresolved

$ hg tip
changeset:   10037:ce3143a06f21
tag: tip
user:jeremy goecks 
date:Fri Apr 05 09:39:05 2013 -0400
summary: Implement a reference-based compression approach for
encoding aligned reads' sequence and cigar. Move cigar functions into
own file.

$ hg diff
(nothing)

At this point I would normally do: hg checkout tools,  hg merge default

$ hg checkout tools
abort: path 'static/june_2007_style/Makefile' traverses symbolic link
'static/june_2007_style'

So, trying the trick from before:

$ rm static/june_2007_style
$ hg checkout tools
abort: crosses branches (merge branches or use --clean to discard changes)
$ hg checkout tools --clean
769 files updated, 0 files merged, 413 files removed, 0 files unresolved

And now, the stumbling block:

$ hg merge default
merging datatypes_conf.xml.sample
merging lib/galaxy/datatypes/registry.py
merging lib/galaxy/jobs/__init__.py
merging run.sh
merging static/june_2007_style/blue/base.css and
static/style/blue/base.css to static/style/blue/base.css
merging tool_conf.xml.sample
merging tools/filters/cutWrapper.pl
abort: No such file or directory:
/mnt/galaxy/galaxy-central/static/style/blue/base.css

Advice from an hg guru would be appreciated.

Thanks,

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] negative user data usage

2013-04-11 Thread Carl Eberhard 
Thanks much for the information, Geert. I've created a card for this bug in
Trello: https://trello.com/c/KLmH1nag.


On Wed, Apr 3, 2013 at 5:39 AM, Geert Vandeweyer  wrote:

> Hi,
>
> In case anybody is wondering, the "set_user_disk_usage.py" script corrects
> this.
>
> Best,
>
> Geert
>
>
> On 03/28/2013 11:04 AM, Geert Vandeweyer wrote:
>
>> Hi,
>>
>> Today, I found out that one user in our local galaxy installation (the
>> administrator user) has a negative disk usage.
>> - Reports shows : -72780720701 bytes
>> - Galaxy history shows: -1%
>>
>> Does anybody have suggestions on what might be causing this and how to
>> solve it?
>> There is about 660 Gb of data in the histories of that user, but it was
>> more before.
>> I believe it happened after some histories were deleted and there was a
>> message of one of them being shared.
>>
>> Best,
>>
>> Geert
>>
>>
>
> --
>
> Geert Vandeweyer, Ph.D.
> Department of Medical Genetics
> University of Antwerp
> Prins Boudewijnlaan 43
> 2650 Edegem
> Belgium
> Tel: +32 (0)3 275 97 56
> E-mail: geert.vandewe...@ua.ac.be
> http://ua.ac.be/**cognitivegenetics 
> http://www.linkedin.com/pub/**geert-vandeweyer/26/457/726
>
> __**_
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>  
> http://galaxyproject.org/**search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Tophat didn't run

2013-04-11 Thread Jennifer Jackson 

Hi Sarah,

It still sounds like there is a path problem - this is why the tools are 
looking in the wrong loc file. When bowtie2/tophat2 installs, it will 
create a symbolic link that names itself as just "bowtie" or "tophat", 
pointing to the v2 binaries.


When you run these, what do you get?

   $ which bowtie

   $ which tophat

My guess is that these are symbolic links pointing to the v2 binaries. 
You will want to remove these. This is noted in the NGS set-up wiki, but 
easy to miss.


For the custom _reference genome _portion below, there is a mix-up here. 
A custom _reference genome_ is in fasta format, not GTF format. I think 
what you are doing is using a _reference annotation_ file with the 
process. Both can be used with RNA-seq tools, but the _reference genome_ 
is the one with the indexes. The link I sent about _custom reference 
genomes_ explains how to use one of these, if you still what want to try.


I think it is worth reviewing the path and loc info, plus the binary 
commands above. Unless this helps you to solve the problem on your own now.


Thanks!

Jen
Galaxy team

On 4/11/13 6:16 AM, Sarah Maman wrote:

Thanks a lot Jennifer,

Restart, full paths were OK.

I don't know why but the 2nd version of Tophat (so the tophat tool 
available from Galaxy) search indexs in bowtie-index.loc file and not 
in bowtie2-index.loc
So, I've added my bowtie2 index paths in bowtie-index. loc file and 
tophat run...



But when I want to run tophat with a reference genome from your 
history, here is the error message (my reference genome is a GFT file) :

Error in tophat:

[2013-04-11 14:57:12] Beginning TopHat run (v2.0.5)
---
[2013-04-11 14:57:12] Checking for Bowtie
  Bowtie version:2.0.0.7
[2013-04-11 14:57:12] Checking for Samtools
Samtools version:0.1.19.0
[2013-04-11 14:57:13] Checking for Bowtie index files
Error: Could not find Bowtie 2 index files 
(/tmp/1078173.1.workq/tmpzxEFNK/dataset_6485.*.bt2)

Settings:
  blablabla  OK.
Total time for backward call to driver() for mirror index: 00:00:57
TopHat v2.0.5
tophat -p 4  /tmp/1078173.1.workq/tmpzxEFNK/dataset_6485 
/work/galaxy/database/files/006/dataset_6528.dat
[bam_header_read] EOF marker is absent. The input is probably truncated.
[bam_header_read] invalid BAM binary header (this is not a BAM file).
[bam_index_core] Invalid BAM header.[bam_index_build2] fail to index the BAM 
file.
Epilog : job finished at jeu. avril 11 14:57:18 CEST 2013
Thanks in advance,
Sarah


Jennifer Jackson a écrit :

Hi Sarah,

Let's try to sort this out. Your problem does not seem to be the same 
as in the question referenced, but we can see. First - just to double 
check - since setting up the genome, you have restarted the server? 
If not, do that first and check to see if that fixes the problem. 
Basically, you want to follow this checklist and restarting is the 
final step:

http://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup

If the problem persists, then would you please send a few more details:

1 - full paths* on you system where you keep the .bt2 indexes, sam 
index, and .fa file. Maybe do an "ls -l" on these dirs so we can 
check the symbolic links are in place and named correctly.


* as a note, these should be "hard paths" and not symbolic (except 
for the .fa links), and must have permissions set to be accessible to 
the "galaxy user"


2 - lines from your bowtie2_indices.loc and sam_fa_indices.loc file 
for this genome. I may have you double check your builds.txt file 
later. if this doesn't sounds familiar, it could be the problem, the 
genome must be in there, too. - see this wiki: 
http://wiki.galaxyproject.org/Admin/Data%20Integration


3 - full error message you get when you try to run this using a 
genome in fasta format from your history. It really shouldn't be the 
same error - something is not right with the settings and a custom 
genome is not actually being used if that is the case. Give it 
another try and see what happens, then send that info. This is a bit 
of a side case, we should get your basic install correct, but knowing 
how to do this is a good thing and easy to learn.

http://wiki.galaxyproject.org/Support#Custom_reference_genome

It is OK to masked out anything like user names/groups you don't want 
to share. Please keep on the list in case we need other feedback.


Thanks!

Jen
Galaxy team

On 4/10/13 3:15 AM, Sarah Maman wrote:

Hello,


When I run tophat ("Tophat for Illumina Find splice junctions using 
RNA-seq data ), the job failed with truncated files. However, index 
files are available and I get exactly the same error message using 
built-in index or one of my history.


/
Tool execution generated the following error message:

Error in tophat:

[2013-04-10 09:17:07] Beginning TopHat run (v2.0.5)
---
[2013-04-10 09:17:07] Checking for Bowtie
  Bowtie version: 2.0.0.

Re: [galaxy-dev] Tophat didn't run

2013-04-11 Thread Sarah Maman 

Thanks a lot Jennifer,

Restart, full paths were OK.

I don't know why but the 2nd version of Tophat (so the tophat tool 
available from Galaxy) search indexs in bowtie-index.loc file and not in 
bowtie2-index.loc
So, I've added my bowtie2 index paths in bowtie-index. loc file and 
tophat run...



But when I want to run tophat with a reference genome from your history, 
here is the error message (my reference genome is a GFT file) :


Error in tophat:

[2013-04-11 14:57:12] Beginning TopHat run (v2.0.5)
---
[2013-04-11 14:57:12] Checking for Bowtie
  Bowtie version:2.0.0.7
[2013-04-11 14:57:12] Checking for Samtools
Samtools version:0.1.19.0
[2013-04-11 14:57:13] Checking for Bowtie index files
Error: Could not find Bowtie 2 index files 
(/tmp/1078173.1.workq/tmpzxEFNK/dataset_6485.*.bt2)


Settings:
blablabla  OK.
Total time for backward call to driver() for mirror index: 00:00:57
TopHat v2.0.5
tophat -p 4  /tmp/1078173.1.workq/tmpzxEFNK/dataset_6485 
/work/galaxy/database/files/006/dataset_6528.dat
[bam_header_read] EOF marker is absent. The input is probably truncated.
[bam_header_read] invalid BAM binary header (this is not a BAM file).
[bam_index_core] Invalid BAM header.[bam_index_build2] fail to index the BAM 
file.
Epilog : job finished at jeu. avril 11 14:57:18 CEST 2013

Thanks in advance,
Sarah


Jennifer Jackson a écrit :

Hi Sarah,

Let's try to sort this out. Your problem does not seem to be the same 
as in the question referenced, but we can see. First - just to double 
check - since setting up the genome, you have restarted the server? If 
not, do that first and check to see if that fixes the problem. 
Basically, you want to follow this checklist and restarting is the 
final step:

http://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup

If the problem persists, then would you please send a few more details:

1 - full paths* on you system where you keep the .bt2 indexes, sam 
index, and .fa file. Maybe do an "ls -l" on these dirs so we can check 
the symbolic links are in place and named correctly.


* as a note, these should be "hard paths" and not symbolic (except for 
the .fa links), and must have permissions set to be accessible to the 
"galaxy user"


2 - lines from your bowtie2_indices.loc and sam_fa_indices.loc file 
for this genome. I may have you double check your builds.txt file 
later. if this doesn't sounds familiar, it could be the problem, the 
genome must be in there, too. - see this wiki: 
http://wiki.galaxyproject.org/Admin/Data%20Integration


3 - full error message you get when you try to run this using a genome 
in fasta format from your history. It really shouldn't be the same 
error - something is not right with the settings and a custom genome 
is not actually being used if that is the case. Give it another try 
and see what happens, then send that info. This is a bit of a side 
case, we should get your basic install correct, but knowing how to do 
this is a good thing and easy to learn.

http://wiki.galaxyproject.org/Support#Custom_reference_genome

It is OK to masked out anything like user names/groups you don't want 
to share. Please keep on the list in case we need other feedback.


Thanks!

Jen
Galaxy team

On 4/10/13 3:15 AM, Sarah Maman wrote:

Hello,


When I run tophat ("Tophat for Illumina Find splice junctions using 
RNA-seq data ), the job failed with truncated files. However, index 
files are available and I get exactly the same error message using 
built-in index or one of my history.


/
Tool execution generated the following error message:

Error in tophat:

[2013-04-10 09:17:07] Beginning TopHat run (v2.0.5)
---
[2013-04-10 09:17:07] Checking for Bowtie
  Bowtie version: 2.0.0.7
[2013-04-10 09:17:07] Checking for Samtools
Samtools version: 0.1.19.0
[2013-04-10 09:17:07] Checking for Bowtie index files
Error: Could not find Bowtie 2 index files 
(/work/galaxy/Danio_rerio.Zv9.62.dna.chromosome.22.fa.*.bt2)



The tool produced the following additional output:

TopHat v2.0.5
tophat -p 4  /work/galaxy/Danio_rerio.Zv9.62.dna.chromosome.22.fa 
/work/galaxy/database/files/006/dataset_6528.dat

[bam_header_read] EOF marker is absent. The input is probably truncated.
[bam_header_read] invalid BAM binary header (this is not a BAM file).
[bam_index_core] Invalid BAM header.[bam_index_build2] fail to index 
the BAM file.

Epilog : job finished at mer. avril 10 09:17:12 CEST 2013 /

In this post 
(http://dev.list.galaxyproject.org/tophat-for-illumina-looking-in-wrong-directory-for-bowtie2-indexes-tt4658609.html#none), 
the solution isn't found.


Do you have any idea,
Sarah Maman
--
  --*--
Sarah Maman
INRA - LGC - SIGENAE
http://www.sigenae.org/
Chemin de Borde-Rouge - Auzeville - BP 52627
31326 Castanet-Tolosan cedex - FRANCE
Tel:   +33(0)5.61.28.57.08
Fax:   +33(0)5.61.28.57.53 
 --

Re: [galaxy-dev] Setting extension of file names generated by Galaxy

2013-04-11 Thread Peter Cock 
On Thu, Apr 11, 2013 at 7:54 AM, Holtgrewe, Manuel
 wrote:
> Dear all,
>
> is it possible to make Galaxy use a specific extension for a given data type 
> without a wrapper?
>
> I.e. can I make Galaxy use the extension .bam, .sam, .fasta etc. internally 
> instead of .dat?
>
> Thanks,
> Manuel

I think you must use a wrapper for dealing with inflexible tools with
hard coded filename/extension assumptions.

If you are wrapping a tool which insists on a particular input filename
or extension, you can try creating a symlink to the Galaxy *.dat file in
a tool wrapper script. Or if that fails, make a copy of the file.

Similarly, you can rename hard coded output filenames to the Galaxy
assigned *.dat filename once the tool has run using a wrapper script.

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] "Local install" of Galaxy Tool

2013-04-11 Thread Björn Grüning 
Hi Manuel,

collect your binaries in a separate location and add to PATH and your
cluster. If you want to ship your tool in the toolshed, the installation
and PATH will be automatically added once you installed it from the
toolshed. Have a look at:

http://wiki.galaxyproject.org/ToolShedToolFeatures#Automatic_third-party_tool_dependency_installation_and_compilation_with_installed_repositories

Regards,
Bjoern

> Adding galaxy-dist/tools/${category} to PATH before calling sh run.sh does 
> the trick. This is not ideal but works for now...
> 
> Thanks,
> Manuel
> 
> From: Björn Grüning [bjoern.gruen...@pharmazie.uni-freiburg.de]
> Sent: Thursday, April 11, 2013 12:19 AM
> To: Holtgrewe, Manuel
> Cc: galaxy-dev@lists.bx.psu.edu
> Subject: Re: [galaxy-dev] "Local install" of Galaxy Tool
> 
> Hi,
> 
> > I have:
> >
> > galaxy-dist/tools/${category}/BINARY
> > galaxy-dist/tools/${category}/BINARY.xml
> 
> Try to put the binary in your PATH.
> 
> > Where BINARY.xml contains the BINARY... XML code.
> >
> > Is there a good way to see what Galaxy is exactly executing for the tool?
> 
> In your webX.log. It is depending on your configuration, but in your log
> file the executed command is logged.
> 
> > I could not find a description on how to do that anywhere on the internet 
> > (*sigh* apparently one cannot find anything related to Galaxy, at least 
> > since the term Galaxy got "hijacked" by the smartphone of the same name ;)
> 
> Please use these URL for searching:
> 
> http://galaxyproject.org/search/web/
> 
> Cheers,
> Bjoern
> 
> >
> > Thanks,
> > Manuel
> > 
> > From: Björn Grüning [bjoern.gruen...@pharmazie.uni-freiburg.de]
> > Sent: Wednesday, April 10, 2013 11:37 PM
> > To: Holtgrewe, Manuel
> > Cc: galaxy-dev@lists.bx.psu.edu
> > Subject: Re: [galaxy-dev] "Local install" of Galaxy Tool
> >
> > Hi Manuel,
> >
> > is your XML file also under ${galaxy_root}/tools/${category}/ ? It
> > should be.
> >
> > Tata,
> > Bjoern
> >
> > > Dear all, I am trying to follow the Admin Add Tool Tutorial [1]. I put my 
> > > binary BINARY into the location ${galaxy_root}/tools/${category}/BINARY. 
> > > However, it is not found using if the  tag looks as follows:
> > >
> > > BINARY (arguments)
> > >
> > > When I put the following, it works:
> > >
> > > $__root_dir__/tools/CATEGORY/BINARY
> > >
> > > Am I doing something wrong? Is there a preferable way to such a "local" 
> > > installation than my second try?
> > >
> > > Thanks!
> > > Manuel
> > >
> > > [1] http://wiki.galaxyproject.org/Admin/Tools/Add%20Tool%20Tutorial
> > > ___
> > > Please keep all replies on the list by using "reply all"
> > > in your mail client.  To manage your subscriptions to this
> > > and other Galaxy lists, please use the interface at:
> > >   http://lists.bx.psu.edu/
> > >
> > > To search Galaxy mailing lists use the unified search at:
> > >   http://galaxy
> >
> >
> 
> 


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/