Re: [galaxy-dev] pass more information on a dataset merge

[Alex.Khassapov]

Hi John,

what I don't get - I specify the output format "m:grd", my tool generates 
multiple output files in dataset_id_files folder, but the dataset_id.dat file 
is empty. I need to call this regenerate_primary_file() to add the HTML with 
the file list to the dat file. But I'm not sure where?

-Alex

-Original Message-
From: jmchil...@gmail.com [mailto:jmchil...@gmail.com] On Behalf Of John Chilton
Sent: Friday, 19 October 2012 6:16 AM
To: Khassapov, Alex (CSIRO IM&T, Clayton)
Subject: Re: [galaxy-dev] pass more information on a dataset merge

On Tue, Oct 16, 2012 at 11:11 PM,   wrote:
> Hi John,
>
> I am definitely interested in this idea, not only me - we are currently 
> working on moving a few scientific tools (not related to genome) into cloud 
> using Galaxy.

Great. My interests in Galaxy are mostly outside of genomics as well, it is 
good to have more people utilizing Galaxy in this way because it will force the 
platform to become more generic and address more broader use cases.

>
> We will try it further and see if we need any changes. For now one 
> improvement would be nice, make dataset_id.dat contain list of paths to the 
> location of the uploaded files, so by displaying html page the user could 
> just click on the link and download the file.
>

Code that attempted to do this was in there, but didn't work obviously. I have 
now fixed it up.

Thanks for beta testing.

-John

> We are pretty new to Galaxy, so our understanding of Galaxy is pretty limited.
>
> Thanks again,
>
> Alex
>
>
> -Original Message-
> From: jmchil...@gmail.com [mailto:jmchil...@gmail.com] On Behalf Of 
> John Chilton
> Sent: Wednesday, 17 October 2012 3:21 AM
> To: Khassapov, Alex (CSIRO IM&T, Clayton)
> Subject: Re: [galaxy-dev] pass more information on a dataset merge
>
> Wow, thanks for the rapid feedback! I have made the changes you have 
> suggested. It seems you must be interested in this idea/implementation. Let 
> me know if you have specific use cases/requirements in mind and/or if you 
> would be interested in write access to the repository.
>
> -John
>
> On Mon, Oct 15, 2012 at 11:51 PM,   wrote:
>> Hi John,
>>
>> I tried your galaxy-central-homogeneous-composite-datatypes implementation, 
>> works great thank you (and Jorrit).
>>
>> A couple of fixes:
>>
>> 1. Add multi_upload.xml to too_conf.xml 2.
>> lib/galaxy/tools/parameters/grouping.py line 322 (in get_filenames( context 
>> )) -
>> "if ftp_files is not None:"
>>Remove "is not None" as ftp_files is empty [], but not None, then line 
>> 331 "user_ftp_dir = os.path.join( trans.app.config.ftp_upload_dir, 
>> trans.user.email )" throws an exeption if ftp_upload_dir isn't set.
>>
>> Alex
>>
>> -Original Message-
>> From: galaxy-dev-boun...@lists.bx.psu.edu
>> [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of John 
>> Chilton
>> Sent: Tuesday, 16 October 2012 1:07 AM
>> To: Jorrit Boekel
>> Cc: galaxy-dev@lists.bx.psu.edu
>> Subject: Re: [galaxy-dev] pass more information on a dataset merge
>>
>> Here is an implementation of the implicit multi-file composite datatypes 
>> piece of that idea. I think the implicit parallelism may be harder.
>>
>> https://bitbucket.org/galaxyp/galaxy-central-homogeneous-composite-da
>> t
>> atypes/compare
>>
>> Jorrit do you have any objection to me trying to get this included in 
>> galaxy-central (this is 95% code I stole from you)? I made the changes 
>> against a clean galaxy-central fork and included nothing proteomics specific 
>> in anticipation of trying to do that. I have talked with Jim Johnson about 
>> the idea and he believes it would be useful his mothur metagenomics tools, 
>> so the idea is valuable outside of proteomics.
>>
>> Galaxy team, would you be okay with including this and if so is there 
>> anything you would like to see either at a high level or at the level of the 
>> actual implementation.
>>
>> -John
>>
>> 
>> John Chilton
>> Senior Software Developer
>> University of Minnesota Supercomputing Institute
>> Office: 612-625-0917
>> Cell: 612-226-9223
>> Bitbucket: https://bitbucket.org/jmchilton
>> Github: https://github.com/jmchilton
>> Web: http://jmchilton.net
>>
>> On Mon, Oct 8, 2012 at 9:24 AM, John Chilton  wrote:
>>> Jim Johnson and I have been discussing that approach to handling 
>>> fractionated proteomics samples as well (composite datatypes, not 
>>> the specifics of the interface for parallelizing).
>>>
>>> My perspective has been that Galaxy should be augmented with better 
>>> native mechanisms for grouping objects in histories, operating over 
>>> those groups, building workflows that involve arbitrary numbers of 
>>> inputs, etc... Composite data types are kindof a kludge, I think 
>>> they are more useful for grouping HTML files together when you don't 
>>> care about operating on the constituent parts you just want to view 
>>> pages a as a report or something. With this proteomic data 

Re: [galaxy-dev] Retreiving a library data type in

[Alex.Khassapov]

$filename.ext

From: galaxy-dev-boun...@lists.bx.psu.edu 
[mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Simon Gladman
Sent: Friday, 19 October 2012 4:38 PM
To: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] Retreiving a library data type in 

oops, clicked send too early..
On 19 October 2012 16:33, Simon Gladman 
mailto:simon.glad...@monash.edu>> wrote:
Hi all,

Have searched mailing list to no avail. Therefore:

How can I get a data library's data_type in the  tagset?

Example:


dosomething.pl
filename.data_type
   
   
  


So is the file of type: fasta, fastq, sam or bam and how can I get that into 
the  tag?

Thanks in advance,

Simon Gladman.

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Retreiving a library data type in

[Simon Gladman]

oops, clicked send too early..

On 19 October 2012 16:33, Simon Gladman  wrote:

> Hi all,
>
> Have searched mailing list to no avail. Therefore:
>
> How can I get a data library's data_type in the  tagset?
>
> Example:
>
> 
> dosomething.pl
> filename.data_type
>
   
   
  


So is the file of type: fasta, fastq, sam or bam and how can I get that
into the  tag?

Thanks in advance,

Simon Gladman.
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Retreiving a library data type in

[Simon Gladman]

Hi all,

Have searched mailing list to no avail. Therefore:

How can I get a data library's data_type in the  tagset?

Example:


dosomething.pl
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] How to get a shedtool to run locally

[Fields, Christopher J]

On Oct 17, 2012, at 3:53 PM, Carlos Borroto  wrote:

> On Mon, Oct 15, 2012 at 11:49 AM, Liisa Koski  wrote:
>> Hello,
>> I've installed a tool to my local Galaxy installation via my local ToolShed.
>> I would like to run this tool locally, and not have it submitted to the
>> grid.
>> 
>> I have tried two ids in the universe_wsgi.ini [galaxy:tool_runners] section.
>> 
>> The tool id directly from the tool.xml file and the id from the
>> shed_tool_conf.xml (same id found in the integrated_tool_panel.xml)
>> 
>> tool = local:///
>> galaxy.server:9009/repos/user/tool/1.0 = local:///
>> 
>> 
>> My job still gets submitted to the grid.
>> 
> 
> Hi,
> 
> I would also like to specify a tool_runner for a tool installed from
> the shed_tool. As Liisa, I can not find the correct tool id to use.
> 
> Any help?
> Thanks,
> Carlos

My local one is set up like this and it works (for augustus):

toolshed.g2.bx.psu.edu/repos/bjoern-gruening/augustus/augustus/0.1 = 
pbs:///default/-l ncpus=1,mem=8gb,walltime=24:00:00/

The above is the path in within the shed repo.

chris
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Jobs crash only in workflow context

[Todd Oakley]

Hi James,
We have made some progress in understanding the workflow-specific 
job crashes.


It seems that 'parallel' workflows are sending jobs simultaneously, and 
this is problematic for torque.


We get this error:
10/18/2012 10:06:18;0080;PBS_Server;Req;req_reject;Reject reply 
code=15058(Bad DIS based Request Protocol MSG=cannot decode message), 
aux=0, type=Connect, from @



There is a thread here:
http://osdir.com/ml/galaxy-source-control/2011-08/msg00136.html

which is very similar to what we are experiencing.

In the post linked above, the author indicates he found a fix (pasted 
below). Would you recommend we make the same change?


Thanks!
Todd

"To deal with this I modified the lib/galaxy/jobs/runners/pbs.py script 
to make multiple attempts at submitting in the following way:
@@ -286,6 +286,12 @@ class PBSJobRunner( BaseJobRunner ): 
 log.debug("(%s) submitting file %s" % ( galaxy_job_id, job_file ) )
 log.debug("(%s) command is: %s" % ( galaxy_job_id, command_line ) 
)  job_id = pbs.pbs_submit(c, job_attrs, job_file, 
pbs_queue_name, None)+   ##Modified to give ten tries for qsubbing a 
job+   num_try=0+   while(not job_id and num_try<10): + 
  job_id = pbs.pbs_submit(c, job_attrs, job_file, pbs_queue_name, 
None)+   num_try+=1+pbs.pbs_disconnect(c)  # 
check to see if it submitted "


On 10/17/2012 9:40 AM, James Taylor wrote:

Todd, this is definitely unusual. Can you post (or send directly)
relevant sections from the Galaxy log?

-- jt


On Tue, Oct 16, 2012 at 8:15 PM, Todd Oakley
 wrote:

Hello,
 We just did a few tweaks to improve Galaxy performance, and a new issue
popped up that I would like advice on troubleshooting.

 When we run workflows, we see that tools later in the workflow run and
crash before the results they depend on have completed running.

 We can re-run the crashed jobs later and they work fine, suggesting that
they are only failing in the context of running workflows.

 I'd appreciate any advice on how to start troubleshooting this problem.

Thanks much!
Todd


--

***
Todd Oakley, Professor
Ecology Evolution and Marine Biology
University of California, Santa Barbara
Santa Barbara, CA 93106 USA
***

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


--

***
Todd Oakley, Professor
Ecology Evolution and Marine Biology
University of California, Santa Barbara
Santa Barbara, CA 93106 USA
***

Lab Website 
Twitter: @UCSB_OakleyLab

*Recent Papers: *

 * Pancrustacean Phylotranscriptomics
   MBE Paper
   


 * Convergent Evolution in Cephalopoda
   BMC Ev Biol 
 * Cnidocyte discharge regulated by opsin and light
   BMC Biology Paper  Scientific American
   Write-up
   


 * Sponge Larvae Could be Guided by Cryptochrome
   J Exp Biol. Paper  |
   Nature News
   

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Apache proxy to Galaxy: mod_rewrite or mod_proxy ?

[Oleksandr Moskalenko]

On Oct 18, 2012, at 12:28 PM, Christophe Antoniewski 
 wrote:

> Hi,
> 
> Currently, the guide lines for using Apache as a reverse proxy to galaxy are 
> to use the mod_rewrite module and the rewrite rules as explained in the 
> galaxy wiki.
> 
> I was wondering whether anybody has tried to use mod_proxy, mod_proxy-http 
> instead, with proxypass and proxypassreverse directives ? any technical 
> limitations ? could it be more efficient in a production configuration ?
> 
> Thanks for the info
> 
> Chris

Hi Chris,

Here's our production config. We use external auth, which is why REMOTE_USER is 
set. I hope it helps.

Regards,

Alex



BalancerMember http://localhost:8090
BalancerMember http://localhost:8091
BalancerMember http://localhost:8092
BalancerMember http://localhost:8093
BalancerMember http://localhost:8094
BalancerMember http://localhost:8095
BalancerMember http://localhost:8096

ProxyPass / balancer://galaxy/
RewriteCond %{IS_SUBREQ} ^false$ 
RewriteCond %{LA-U:REMOTE_USER} (.+) 
RewriteRule . - [E=RU:%1] 
RequestHeader set REMOTE_USER %{RU}e 
RequestHeader unset Authorization 
SetOutputFilter DEFLATE
SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.(?:t?gz|zip|bz2)$ no-gzip dont-vary
XSendFile on
XSendFilePath /
RewriteRule ^/static/style/(.*) /galaxy/run/prod/static/june_2007_style/blue/$1 
[L]
RewriteRule ^/static/scripts/(.*) /galaxy/run/prod/static/scripts/packed/$1 [L]
RewriteRule ^/static/(.*) /galaxy/run/prod/static/$1 [L]
RewriteRule ^/images/(.*) /galaxy/run/prod/static/images/$1 [L]
RewriteRule ^/favicon.ico /galaxy/run/prod/static/favicon.ico [L]
RewriteRule ^/robots.txt /galaxy/run/prod/static/robots.txt [L]
RewriteRule ^/errors/(.*) /var/www/html/galaxy/errors/$1 [L]



___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Fwd: 'UsesFormDefinitions' is not defined

[Hans-Rudolf Hotz]

Hi Scott

Thank you very much for your quick reply!

after fixing (ie replacing "galaxy.web.controllers.library_common" with 
"galaxy.webapps.galaxy.controllers.library_common") in the following 
four files:


~/templates/library/common/library_dataset_search_results.mako
   ldda_info.mako
   browse_library_opt.mako
   browse_library.mako


it works!  -  both on the fresh install and the upgraded server


@Alex:  I hope this solves your problem as well.



Regards Hans-Rudolf


Than

On 10/18/2012 06:12 PM, Scott McManus wrote:


This was fixed in galaxy-central last week and should be part of
galaxy-dist. What happened is that controllers were migrated and a
couple of modules were left out. See changeset 210c39f4bf7f or later,
which I believe was fixed on October 4th.

In particular, the problem you're seeing had to do with the
templates/library/common/library_common.mako pointing to
galaxy.web.controllers.library_common instead of
galaxy.webapps.galaxy.controllers.library_common.
A few other files had also been affected.

-Scott

- Original Message -

Hi Alex

I am in the same situation, updating our galaxy servers to the "Oct
5th"
release (changeset: 7824:b5bda7a5c345), and I have encountered the
same
problem. I guess, there are actually two problems:


First problem - this happens with a fresh install:
==

I can't create a library at all, I get the following error message:

Error attempting to display contents of library (foo): No
module named  controllers.library_common.



There have been a few commits on "library_common.py" over the last
few
days, which might solve this problem.?



Second problem - this happens after upgrading an existing
installation
==

Like the situation you reported earlier this morning, I get:


Error attempting to display contents of library (haruhotz):
name 'UsesFormDefinitions' is not defined.


I manage to track down the problem to the fact that

in the old version, we've had

~/lib/galaxy/web/controllers/


which apparently has been replaced by

~/lib/galaxy/webapps/galaxy/controllers/


if I remove ~/lib/galaxy/web/controllers/


I get a new error:

Error attempting to display contents of library (haruhotz): No
module named library_common.


which is very similar to the error you get with a fresh install


Obviously, old stuff (ie ~/lib/galaxy/web/controllers/) is still used
after the upgrade.


Sorry, no solution yet, but maybe it helps the developers to track
down
the problem and help us getting the servers running again.



Regards Hans-Rudolf


On 10/18/2012 10:48 AM, Alexander Graf wrote:

Hi,
yesterday I updated Galaxy to the latest revision and now it is not
possible to access any Data Libraries or to create a new one.
Galaxy shows me a read box with following error message:

*Error attempting to display contents of library
(100326_HWUSI-EAS632R_0002): name 'UsesFormDefinitions' is not
defined.*

The log file message is:

/"GET
/galaxy/library_common/browse_library?sort=name&webapp=galaxy&f-description=All&f-name=All&page=1&cntrller=library&show_item_checkboxes=false&async=false&operation=browse&id=9d04ee723dc2cac7
HTTP/1.1" 302 -
"http://x.x.x.x/galaxy/library/browse_libraries?status=error&use_panels=False&message=Error+attempting+to+display+contents+of+library+%100326_HWUSI-EAS632R_0002%29%3A+name+__sq__UsesFormDefinitions__sq__+is+not+defined.";
"Mozilla/5.0 (X11; Linux x86_64) /

Any ideas how to solve this problem or were to start
troubleshooting?

Thank you in advance

Alex




___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

   http://lists.bx.psu.edu/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/

[galaxy-dev] Apache proxy to Galaxy: mod_rewrite or mod_proxy ?

[Christophe Antoniewski]

Hi,

Currently, the guide lines for using Apache as a reverse proxy to galaxy
are to use the mod_rewrite module and the rewrite rules as explained in the
galaxy wiki.

I was wondering whether anybody has tried to use mod_proxy, mod_proxy-http
instead, with proxypass and proxypassreverse directives ? any technical
limitations ? could it be more efficient in a production configuration ?

Thanks for the info

Chris


-- 

¡ Note that my email address has changed !
¡ Please update your records with !
¡ 
*christophe.antoniew...@snv.jussieu.fr*!

--

Christophe Antoniewski
Drosophila Genetics and Epigenetics
Laboratoire de Biolologie du Développement

9, Quai St Bernard, Boîte courrier 24
75252 Paris Cedex 05

Tel +33 1 44 27 34 39
Fax +33 1 44 27 34 45
Mobile +33 6 68 60 51 50

http://drosophile.org
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Corner case in task splitter - merging zero files

[Scott McManus]

Hey Peter-

Thanks - I'll look into it. If you're able to reproduce the problem easily
and wouldn't mind crafting a pull request, then it would be much 
appreciated. Otherwise I'll put this on my to-do list to be done soon.
I or someone else may want to revisit the exception handling to prevent
that from happening.

Thanks!

-Scott

- Original Message -
> Hi Scott,
> 
> Following some failing hard drives, I'm rebuilding our Galaxy server.
> Something isn't quite right with our cluster integration yet, but it
> has
> exposed a problem in Galaxy's handling of task splitting - it can
> sometimes attempt to merge zero files.
> 
> Here is my fix for the BLAST XML format (now in the ToolShed),
> https://bitbucket.org/peterjc/galaxy-central/changeset/5cb6411bad19802ba4001a083164366b42850a48
> 
> Here's an example using the text format:
> 
> galaxy.jobs.splitters.multi ERROR 2012-10-18 16:26:21,330 Error
> merging files
> Traceback (most recent call last):
>   File
>   "/mnt/galaxy/galaxy-central/lib/galaxy/jobs/splitters/multi.py",
> line 133, in do_merge
> output_type.merge(output_files, output_file_name)
>   File "/mnt/galaxy/galaxy-central/lib/galaxy/datatypes/data.py",
>   line
> 545, in merge
> raise Exception('Result %s from %s' % (result, cmd))
> Exception: Result 2 from cat  >
> /mnt/galaxy/galaxy-central/database/files/000/dataset_304.dat
> 
> The problem obviously is that while "cat file1 ... fileN > merged"
> will
> work fine for one or more files, with no files it sits waiting for
> stdin
> (and from a user perspective stalls).
> 
> This logic error is in lib/galaxy/datatypes/data.py method merge,
> which could either treat zero files as an error, or a no-op:
> 
> if len(split_files) == 1:
> cmd = 'mv -f %s %s' % ( split_files[0], output_file )
> else:
> cmd = 'cat %s > %s' % ( ' '.join(split_files),
> output_file )
> result = os.system(cmd)
> 
> I think this should be something like this:
> 
> if not split_files:
> raise Exception('Asked to merge zero files')
> elif len(split_files) == 1:
> cmd = 'mv -f %s %s' % ( split_files[0], output_file )
> else:
> cmd = 'cat %s > %s' % ( ' '.join(split_files),
> output_file )
> result = os.system(cmd)
> 
> It might also make sense to check for zero files in the code which
> calls the merge, i.e. lib/galaxy/jobs/splitters/multi.py function
> do_merge
> I'm still investigating upstream how this comes about, one clue:
> 
> galaxy.jobs.runners.drmaa DEBUG 2012-10-18 16:25:01,930 (273/510)
> state change: job is running
> galaxy.jobs.runners.drmaa DEBUG 2012-10-18 16:25:03,040 (273/510)
> state change: job finished, but failed
> galaxy.jobs.runners.drmaa DEBUG 2012-10-18 16:25:03,074 Job output
> not
> returned from cluster
> galaxy.jobs DEBUG 2012-10-18 16:25:03,074 task 641 for job 273 ended;
> exit code: 0
> galaxy.jobs DEBUG 2012-10-18 16:25:03,148 task 641 ended
> galaxy.jobs.runners.tasks DEBUG 2012-10-18 16:25:05,169 execution
> finished - beginning merge: tblastx -query
> "/mnt/galaxy/galaxy-central/database/files/000/dataset_127.dat"   -db
> "/var/local/blast/ncbi/nt" -query_gencode 2 -evalue 0.001 -out
> /mnt/galaxy/galaxy-central/database/files/000/dataset_304.dat
> -outfmt 0 -num_threads 8
> galaxy.jobs.splitters.multi DEBUG 2012-10-18 16:25:05,181 files []
> 
> If you would prefer that small suggestion as a pull request, let me
> know.
> 
> Regards,
> 
> Peter
> 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Fwd: 'UsesFormDefinitions' is not defined

[Scott McManus]

Excuse me - I meant to say that this will part of the upcoming 
galaxy-dist.

-Scott

- Original Message -
> 
> This was fixed in galaxy-central last week and should be part of
> galaxy-dist. What happened is that controllers were migrated and a
> couple of modules were left out. See changeset 210c39f4bf7f or later,
> which I believe was fixed on October 4th.
> 
> In particular, the problem you're seeing had to do with the
> templates/library/common/library_common.mako pointing to
> galaxy.web.controllers.library_common instead of
> galaxy.webapps.galaxy.controllers.library_common.
> A few other files had also been affected.
> 
> -Scott
> 
> - Original Message -
> > Hi Alex
> > 
> > I am in the same situation, updating our galaxy servers to the "Oct
> > 5th"
> > release (changeset: 7824:b5bda7a5c345), and I have encountered the
> > same
> > problem. I guess, there are actually two problems:
> > 
> > 
> > First problem - this happens with a fresh install:
> > ==
> > 
> > I can't create a library at all, I get the following error message:
> > 
> > Error attempting to display contents of library (foo): No
> > module named  controllers.library_common.
> > 
> > 
> > 
> > There have been a few commits on "library_common.py" over the last
> > few
> > days, which might solve this problem.?
> > 
> > 
> > 
> > Second problem - this happens after upgrading an existing
> > installation
> > ==
> > 
> > Like the situation you reported earlier this morning, I get:
> > 
> > 
> > Error attempting to display contents of library (haruhotz):
> > name 'UsesFormDefinitions' is not defined.
> > 
> > 
> > I manage to track down the problem to the fact that
> > 
> > in the old version, we've had
> > 
> > ~/lib/galaxy/web/controllers/
> > 
> > 
> > which apparently has been replaced by
> > 
> > ~/lib/galaxy/webapps/galaxy/controllers/
> > 
> > 
> > if I remove ~/lib/galaxy/web/controllers/
> > 
> > 
> > I get a new error:
> > 
> > Error attempting to display contents of library (haruhotz): No
> > module named library_common.
> > 
> > 
> > which is very similar to the error you get with a fresh install
> > 
> > 
> > Obviously, old stuff (ie ~/lib/galaxy/web/controllers/) is still
> > used
> > after the upgrade.
> > 
> > 
> > Sorry, no solution yet, but maybe it helps the developers to track
> > down
> > the problem and help us getting the servers running again.
> > 
> > 
> > 
> > Regards Hans-Rudolf
> > 
> > 
> > On 10/18/2012 10:48 AM, Alexander Graf wrote:
> > > Hi,
> > > yesterday I updated Galaxy to the latest revision and now it is
> > > not
> > > possible to access any Data Libraries or to create a new one.
> > > Galaxy shows me a read box with following error message:
> > >
> > > *Error attempting to display contents of library
> > > (100326_HWUSI-EAS632R_0002): name 'UsesFormDefinitions' is not
> > > defined.*
> > >
> > > The log file message is:
> > >
> > > /"GET
> > > /galaxy/library_common/browse_library?sort=name&webapp=galaxy&f-description=All&f-name=All&page=1&cntrller=library&show_item_checkboxes=false&async=false&operation=browse&id=9d04ee723dc2cac7
> > > HTTP/1.1" 302 -
> > > "http://x.x.x.x/galaxy/library/browse_libraries?status=error&use_panels=False&message=Error+attempting+to+display+contents+of+library+%100326_HWUSI-EAS632R_0002%29%3A+name+__sq__UsesFormDefinitions__sq__+is+not+defined.";
> > > "Mozilla/5.0 (X11; Linux x86_64) /
> > >
> > > Any ideas how to solve this problem or were to start
> > > troubleshooting?
> > >
> > > Thank you in advance
> > >
> > > Alex
> > >
> > >
> > >
> > >
> > > ___
> > > Please keep all replies on the list by using "reply all"
> > > in your mail client.  To manage your subscriptions to this
> > > and other Galaxy lists, please use the interface at:
> > >
> > >http://lists.bx.psu.edu/
> > >
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client.  To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> > 
> >   http://lists.bx.psu.edu/
> > 
> 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Fwd: 'UsesFormDefinitions' is not defined

[Scott McManus]

This was fixed in galaxy-central last week and should be part of 
galaxy-dist. What happened is that controllers were migrated and a
couple of modules were left out. See changeset 210c39f4bf7f or later,
which I believe was fixed on October 4th.

In particular, the problem you're seeing had to do with the 
templates/library/common/library_common.mako pointing to 
galaxy.web.controllers.library_common instead of 
galaxy.webapps.galaxy.controllers.library_common.
A few other files had also been affected.

-Scott

- Original Message -
> Hi Alex
> 
> I am in the same situation, updating our galaxy servers to the "Oct
> 5th"
> release (changeset: 7824:b5bda7a5c345), and I have encountered the
> same
> problem. I guess, there are actually two problems:
> 
> 
> First problem - this happens with a fresh install:
> ==
> 
> I can't create a library at all, I get the following error message:
> 
>   Error attempting to display contents of library (foo): No
>   module named  controllers.library_common.
> 
> 
> 
> There have been a few commits on "library_common.py" over the last
> few
> days, which might solve this problem.?
> 
> 
> 
> Second problem - this happens after upgrading an existing
> installation
> ==
> 
> Like the situation you reported earlier this morning, I get:
> 
> 
>   Error attempting to display contents of library (haruhotz):
>   name 'UsesFormDefinitions' is not defined.
> 
> 
> I manage to track down the problem to the fact that
> 
> in the old version, we've had
> 
> ~/lib/galaxy/web/controllers/
> 
> 
> which apparently has been replaced by
> 
> ~/lib/galaxy/webapps/galaxy/controllers/
> 
> 
> if I remove ~/lib/galaxy/web/controllers/
> 
> 
> I get a new error:
> 
>   Error attempting to display contents of library (haruhotz): No
>   module named library_common.
> 
> 
> which is very similar to the error you get with a fresh install
> 
> 
> Obviously, old stuff (ie ~/lib/galaxy/web/controllers/) is still used
> after the upgrade.
> 
> 
> Sorry, no solution yet, but maybe it helps the developers to track
> down
> the problem and help us getting the servers running again.
> 
> 
> 
> Regards Hans-Rudolf
> 
> 
> On 10/18/2012 10:48 AM, Alexander Graf wrote:
> > Hi,
> > yesterday I updated Galaxy to the latest revision and now it is not
> > possible to access any Data Libraries or to create a new one.
> > Galaxy shows me a read box with following error message:
> >
> > *Error attempting to display contents of library
> > (100326_HWUSI-EAS632R_0002): name 'UsesFormDefinitions' is not
> > defined.*
> >
> > The log file message is:
> >
> > /"GET
> > /galaxy/library_common/browse_library?sort=name&webapp=galaxy&f-description=All&f-name=All&page=1&cntrller=library&show_item_checkboxes=false&async=false&operation=browse&id=9d04ee723dc2cac7
> > HTTP/1.1" 302 -
> > "http://x.x.x.x/galaxy/library/browse_libraries?status=error&use_panels=False&message=Error+attempting+to+display+contents+of+library+%100326_HWUSI-EAS632R_0002%29%3A+name+__sq__UsesFormDefinitions__sq__+is+not+defined.";
> > "Mozilla/5.0 (X11; Linux x86_64) /
> >
> > Any ideas how to solve this problem or were to start
> > troubleshooting?
> >
> > Thank you in advance
> >
> > Alex
> >
> >
> >
> >
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client.  To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> >
> >http://lists.bx.psu.edu/
> >
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>   http://lists.bx.psu.edu/
> 
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Fwd: 'UsesFormDefinitions' is not defined

[Hans-Rudolf Hotz]

Hi Alex

I am in the same situation, updating our galaxy servers to the "Oct 5th" 
release (changeset: 7824:b5bda7a5c345), and I have encountered the same 
problem. I guess, there are actually two problems:



First problem - this happens with a fresh install:
==

I can't create a library at all, I get the following error message:

Error attempting to display contents of library (foo): No
module named  controllers.library_common.



There have been a few commits on "library_common.py" over the last few 
days, which might solve this problem.?




Second problem - this happens after upgrading an existing installation
==

Like the situation you reported earlier this morning, I get:


Error attempting to display contents of library (haruhotz):
name 'UsesFormDefinitions' is not defined.


I manage to track down the problem to the fact that

in the old version, we've had

~/lib/galaxy/web/controllers/


which apparently has been replaced by

~/lib/galaxy/webapps/galaxy/controllers/


if I remove ~/lib/galaxy/web/controllers/


I get a new error:

Error attempting to display contents of library (haruhotz): No
module named library_common.


which is very similar to the error you get with a fresh install


Obviously, old stuff (ie ~/lib/galaxy/web/controllers/) is still used 
after the upgrade.



Sorry, no solution yet, but maybe it helps the developers to track down 
the problem and help us getting the servers running again.




Regards Hans-Rudolf


On 10/18/2012 10:48 AM, Alexander Graf wrote:

Hi,
yesterday I updated Galaxy to the latest revision and now it is not
possible to access any Data Libraries or to create a new one.
Galaxy shows me a read box with following error message:

*Error attempting to display contents of library
(100326_HWUSI-EAS632R_0002): name 'UsesFormDefinitions' is not defined.*

The log file message is:

/"GET
/galaxy/library_common/browse_library?sort=name&webapp=galaxy&f-description=All&f-name=All&page=1&cntrller=library&show_item_checkboxes=false&async=false&operation=browse&id=9d04ee723dc2cac7
HTTP/1.1" 302 -
"http://x.x.x.x/galaxy/library/browse_libraries?status=error&use_panels=False&message=Error+attempting+to+display+contents+of+library+%100326_HWUSI-EAS632R_0002%29%3A+name+__sq__UsesFormDefinitions__sq__+is+not+defined.";
"Mozilla/5.0 (X11; Linux x86_64) /

Any ideas how to solve this problem or were to start troubleshooting?

Thank you in advance

Alex




___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

   http://lists.bx.psu.edu/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/

[galaxy-dev] Corner case in task splitter - merging zero files

[Peter Cock]

Hi Scott,

Following some failing hard drives, I'm rebuilding our Galaxy server.
Something isn't quite right with our cluster integration yet, but it has
exposed a problem in Galaxy's handling of task splitting - it can
sometimes attempt to merge zero files.

Here is my fix for the BLAST XML format (now in the ToolShed),
https://bitbucket.org/peterjc/galaxy-central/changeset/5cb6411bad19802ba4001a083164366b42850a48

Here's an example using the text format:

galaxy.jobs.splitters.multi ERROR 2012-10-18 16:26:21,330 Error merging files
Traceback (most recent call last):
  File "/mnt/galaxy/galaxy-central/lib/galaxy/jobs/splitters/multi.py",
line 133, in do_merge
output_type.merge(output_files, output_file_name)
  File "/mnt/galaxy/galaxy-central/lib/galaxy/datatypes/data.py", line
545, in merge
raise Exception('Result %s from %s' % (result, cmd))
Exception: Result 2 from cat  >
/mnt/galaxy/galaxy-central/database/files/000/dataset_304.dat

The problem obviously is that while "cat file1 ... fileN > merged" will
work fine for one or more files, with no files it sits waiting for stdin
(and from a user perspective stalls).

This logic error is in lib/galaxy/datatypes/data.py method merge,
which could either treat zero files as an error, or a no-op:

if len(split_files) == 1:
cmd = 'mv -f %s %s' % ( split_files[0], output_file )
else:
cmd = 'cat %s > %s' % ( ' '.join(split_files), output_file )
result = os.system(cmd)

I think this should be something like this:

if not split_files:
raise Exception('Asked to merge zero files')
elif len(split_files) == 1:
cmd = 'mv -f %s %s' % ( split_files[0], output_file )
else:
cmd = 'cat %s > %s' % ( ' '.join(split_files), output_file )
result = os.system(cmd)

It might also make sense to check for zero files in the code which
calls the merge, i.e. lib/galaxy/jobs/splitters/multi.py function do_merge
I'm still investigating upstream how this comes about, one clue:

galaxy.jobs.runners.drmaa DEBUG 2012-10-18 16:25:01,930 (273/510)
state change: job is running
galaxy.jobs.runners.drmaa DEBUG 2012-10-18 16:25:03,040 (273/510)
state change: job finished, but failed
galaxy.jobs.runners.drmaa DEBUG 2012-10-18 16:25:03,074 Job output not
returned from cluster
galaxy.jobs DEBUG 2012-10-18 16:25:03,074 task 641 for job 273 ended;
exit code: 0
galaxy.jobs DEBUG 2012-10-18 16:25:03,148 task 641 ended
galaxy.jobs.runners.tasks DEBUG 2012-10-18 16:25:05,169 execution
finished - beginning merge: tblastx -query
"/mnt/galaxy/galaxy-central/database/files/000/dataset_127.dat"   -db
"/var/local/blast/ncbi/nt" -query_gencode 2 -evalue 0.001 -out
/mnt/galaxy/galaxy-central/database/files/000/dataset_304.dat
-outfmt 0 -num_threads 8
galaxy.jobs.splitters.multi DEBUG 2012-10-18 16:25:05,181 files []

If you would prefer that small suggestion as a pull request, let me know.

Regards,

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Fwd: 'UsesFormDefinitions' is not defined

[Alexander Graf]

Hi,
yesterday I updated Galaxy to the latest revision and now it is not
possible to access any Data Libraries or to create a new one.
Galaxy shows me a read box with following error message:

*Error attempting to display contents of library
(100326_HWUSI-EAS632R_0002): name 'UsesFormDefinitions' is not defined.*

The log file message is:

*"GET
/galaxy/library_common/browse_library?sort=name&webapp=galaxy&f-description=All&f-name=All&page=1&cntrller=library&show_item_checkboxes=false&async=false&operation=browse&id=9d04ee723dc2cac7
HTTP/1.1" 302 - "
http://x.x.x.x/galaxy/library/browse_libraries?status=error&use_panels=False&message=Error+attempting+to+display+contents+of+library+%100326_HWUSI-EAS632R_0002%29%3A+name+__sq__UsesFormDefinitions__sq__+is+not+defined.";
"Mozilla/5.0 (X11; Linux x86_64) *

Any ideas how to solve this problem or were to start troubleshooting?

Thank you in advance

Alex
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Display of HTML type broken in new Galaxy ?

[Lukasse, Pieter]

Thanks Joachim, that solved the problem!

-Original Message-
From: Joachim Jacob [mailto:joachim.ja...@vib.be] 
Sent: woensdag 17 oktober 2012 10:47
To: Lukasse, Pieter
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] Display of HTML type broken in new Galaxy ?

Hi,

Make sure that in your universe.wsgi is set:
sanitize_all_html = False

Hope this helps,
Joachim

Joachim Jacob, PhD

Rijvisschestraat 120, 9052 Zwijnaarde
Tel: +32 9 244.66.34
Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib

On 10/17/2012 10:09 AM, Lukasse, Pieter wrote:
>
> Hi,
>
> I noticed that in the new Galaxy version the preview of HTML files 
> seems to be broken. When I try to preview a HTML file generated by a 
> tool, it displays a HTML that has all style "stripped off". So the 
> HTML display is in fact changing the generated HTML before displaying it.
>
> Has anyone noticed this as well? What are the fixes/workarounds?
>
> Regards,
>
> Pieter Lukasse
>
> Wageningen UR, Plant Research International
>
> Departments of Bioscience and Bioinformatics
>
> Wageningen Campus, Building 107, Droevendaalsesteeg 1, 6708 PB, 
> Wageningen, the Netherlands
>
> +31-317480891; skype: pieter.lukasse.wur
>
> http://www.pri.wur.nl 
>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this and other 
> Galaxy lists, please use the interface at:
>
>http://lists.bx.psu.edu/






___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/