Re: [galaxy-dev] error with multi dataset tool run
yes, I was the only user but I guess it is too fragile for this kind of thing.
postgres wasn’t queuing age jobs correctly so I guess I should track down why.
From: Hans-Rudolf Hotz h...@fmi.ch
Subject: Re: [galaxy-dev] error with multi dataset tool run
Date: September 1, 2014 at 5:54:10 AM PDT
To: Robert Baertsch baert...@soe.ucsc.edu, galaxy-dev@lists.bx.psu.edu
Hi Robert
Are you using the built in SQLite database ?
Hans-Rudolf
On 08/31/2014 01:27 AM, Robert Baertsch wrote:
I submitted 13 fastq files to tophat2 using DRMAA and got this error.
Is it fatal? BTW: This is a super cool feature.
I’m running the following version of galaxy-dist.
changeset: 14212:91547729ffde
branch: stable
tag: tip
user:Nate Coraor n...@bx.psu.edu mailto:n...@bx.psu.edu
date:Fri Aug 29 14:00:23 2014 -0400
summary: Update tag latest_2014.08.11 for changeset ea12550fbc34
-Robert
There were errors setting up 2 submitted job(s):
* *Error executing tool: (OperationalError) database is locked
u'UPDATE history_dataset_association SET update_time=?, name=?,
blurb=? WHERE history_dataset_association.id = ?' ('2014-08-30
23:14:44.683957', 'Tophat2 on data 7: insertions', 'queued', 137)*
* *Error executing tool: (OperationalError) database is locked
u'UPDATE dataset SET update_time=?, state=? WHERE dataset.id = ?'
('2014-08-30 23:15:57.204718', 'queued', 446)*
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
From: Sandra Derozier sandra.deroz...@jouy.inra.fr
Subject: [galaxy-dev] Error with functional tests on cluster
Date: September 1, 2014 at 6:54:41 AM PDT
To: galaxy-...@bx.psu.edu
Hi all,
I try to set up functional tests on different tools on my Galaxy portal.
When I run functional tests locally everything works fine. But when I run
them on the cluster it failed with this message :
/bin/sh: module: line 1: syntax error: unexpected end of file
/bin/sh: error importing function definition for `module'
The execution on the cluster is ok but the dataset state is set to ERROR.
The DIFF between the expected result and the obtained result is null. Indeed,
this two files are the same.
As I do not know what the problem is: do you have an idea?
Thanks,
Sandra DEROZIER
Sandra DEROZIER
Unité Mathèmatique, Informatique et Génome (MIG)
Plateforme MIGALE
Bâtiment 233
Domaine de Vilvert
78352 Jouy-en-Josas Cedex
From: Hans-Rudolf Hotz h...@fmi.ch
Subject: [galaxy-dev] Solved - Re: testing the visualization plugins
Date: September 1, 2014 at 8:13:30 AM PDT
To: galaxy-...@bx.psu.edu galaxy-...@bx.psu.edu
Hi all
First of all, a big Thanks to Carl who helped me fixing this problem.
So as a summary for all, the problem was caused by a datatype (an extension
to tabular), I manually added to datatypes_conf.xml
Removing the datatype fixed the problem. I couldn't identify a syntax
problem, neither in datatypes_conf.xml nor in
~/lib/galaxy/datatypes/registry.py and ~/lib/galaxy/datatypes/tabular.py.
However, renaming it (in all three files) fixed it as well.
Regards, Hans-Rudolf
___
galaxy-dev mailing list
galaxy-dev@lists.bx.psu.edu
http://lists.bx.psu.edu/listinfo/galaxy-dev
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
[galaxy-dev] millions for rows in galaxy_session
Is there anyway to have galaxy not insert a row in the galaxy_session file for failed logins. Every ping from china seems to generate a row in galaxy_session. We have anonymous login turned off Robert ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] error with multi dataset tool run
I submitted 13 fastq files to tophat2 using DRMAA and got this error. Is it
fatal? BTW: This is a super cool feature.
I’m running the following version of galaxy-dist.
changeset: 14212:91547729ffde
branch: stable
tag: tip
user:Nate Coraor n...@bx.psu.edu
date:Fri Aug 29 14:00:23 2014 -0400
summary: Update tag latest_2014.08.11 for changeset ea12550fbc34
-Robert
There were errors setting up 2 submitted job(s):
Error executing tool: (OperationalError) database is locked u'UPDATE
history_dataset_association SET update_time=?, name=?, blurb=? WHERE
history_dataset_association.id = ?' ('2014-08-30 23:14:44.683957', 'Tophat2 on
data 7: insertions', 'queued', 137)
Error executing tool: (OperationalError) database is locked u'UPDATE dataset
SET update_time=?, state=? WHERE dataset.id = ?' ('2014-08-30 23:15:57.204718',
'queued', 446)
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] tiff files in galaxy
I was planning to use server code to tile the large images and then use one of a number of JavaScript libraries to render. Any browser should work without extensions . Jim did this with visigene in the genome browser years ago. - Robert On Dec 3, 2013, at 9:16 AM, Carl Eberhard carlfeberh...@gmail.com wrote: Hi, Robert As far as I know, all browsers besides Safari aren't set up to render tiff files by default. Are you using Safari to display the files in Galaxy? There are extensions for Firefox and Chrome that will allow you to display these files. Would that be an option? As a side note: creating, saving, and manipulating images and thumbnails would require Galaxy including a server side image processing library as a dependency. I'd like to do this at some point in the future as it would allow us other advantages too. Thanks, Carl On Thu, Nov 21, 2013 at 4:17 PM, Robert Baertsch baert...@soe.ucsc.edu wrote: Do you know a fast way to open large tiff files in galaxy ? The sniffer recognizes the type, but clicking on the eyeball on a large 82mb file just hangs. Downloading the file and opening in preview does work but it is clunky. Perhaps storing a low res version that would be display when clicking the eyeball would speed access. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] character set encoding metadata
Does galaxy have an official place to store the character set for a text file? I'm thinking of modifying the upload tool to prompt for the character set, since it cannot be sniffed. It should go in the hda/ldda metadata hash or extended_metadata for the dataset. Any opinions? BTW: everyone should watch this talk on unicode. http://bit.ly/unipain ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] latest galaxy-central version
Carl, Thanks for building a great tool. Can you fix the library performance problem? Steps to reproduce: 1. http://su2c-dev.ucsc.edu/library 2. open library TCGA Pancan. It takes more than 20 seconds! -Robert On Nov 5, 2013, at 1:29 PM, Carl Eberhard carlfeberh...@gmail.com wrote: Thanks for the thorough report, Robert. I've added some sanity checking to the panel in default:11240:c271cdb443c6 My only guess as to why the panel javascript is trying to load at all in the library section is a link to the controller/datasets methods (unhide, delete, etc.) that still assume histories are in their own frames. Still investigating. On Tue, Nov 5, 2013 at 12:06 PM, Robert Baertsch baert...@soe.ucsc.edu wrote: Carl, Just to make sure I just deleted my tree and did a fresh checkout and tried a third time. It happened all three times. I'm using a postgres database and this time I started with a fresh config files and just changed the setting below. I didn't see the javascript error in Chrome, perhaps it is some strangeness in Firebug. I've seen strange errors before in Firebug. Attached are two screen grab of the lastest try before and after browser refresh. changeset: 11227:151b7d3b2f1b branch: stable tag: tip parent: 11219:5c789ab4144a user:Carl Eberhard carlfeberh...@gmail.com date:Tue Nov 05 11:33:06 2013 -0500 summary: History panel: fix naming bug in purge async error handling -Robert 29c29 port = 8585 --- #port = 8080 34c34 host = 0.0.0.0 --- #host = 127.0.0.1 96d95 database_connection = postgresql://localhost/medbookgalaxycentral3 542c541 allow_library_path_paste = True --- #allow_library_path_paste = False 599c598 admin_users = baert...@soe.ucsc.edu --- #admin_users = None Screen Shot 2013-11-05 at 8.35.09 AM.png Screen Shot 2013-11-05 at 8.37.04 AM.png On Nov 5, 2013, at 8:21 AM, Carl Eberhard carlfeberh...@gmail.com wrote: Hello, Robert I'm having difficulty reproducing this with a fresh install of galaxy_central:default:11226:c67b9518c1e0. Is this an intermittent error or does it happen reliably each time with the steps above? Is it still the same javascript error you mentioned above? I'll investigate further. On Mon, Nov 4, 2013 at 7:29 PM, Robert Baertsch baert...@soe.ucsc.edu wrote: I updated to the stable release and reproduced the issue. Step to reproduce 1. go to admin 2. Manage data libraries 3. add dataset 4. select Upload files from filesystem paths 5. paste full path to any bam file. 6. leave defaults: auto-detect and copy files into galaxy 7. select role to restrict access 8. click upload to start Screen shows strange formatting and Job is running for Bam file. python /data/galaxy-central/tools/data_source/upload.py /data/galaxy-central /data/galaxy-central/database/tmp/tmpJoasJl /data/galaxy-central/database/tmp/tmpzZO8t1 8877:/data/galaxy-central/database/job_working_directory/004/4548/dataset_8877_files:/data/galaxy-central/database/files/008/dataset_8877.dat If I do a firefox refresh and go back to the library, the formatting is normal. I'm assuming the fix is to just render the page before waiting for the job to complete. -Robert On Nov 4, 2013, at 12:45 PM, Martin Čech mar...@bx.psu.edu wrote: Hello, I have also seen some of these errors while developing libraries. The library code is not in central however it might be related to recent changes to the history panel. Carl Eberhard might now more, adding him to the conversation. --Marten On Mon, Nov 4, 2013 at 2:45 PM, Robert Baertsch baert...@soe.ucsc.edu wrote: It keeps doing posts, and I'm not seeing any new errors. POST http://su2c-dev.ucsc.edu:8383/library_common/library_item_updates 200 OK 121ms When I did a browser refresh, I got the following javascript error: (I am logged in) Galaxy.currUser is undefined on Line 631 in history-panel.js When I opened the data library where the bam file was copying, everything is rendered ok. It seems the browser refresh fixed things. -Robert On Nov 4, 2013, at 11:14 AM, James Taylor ja...@jamestaylor.org wrote: Robert, I'm not sure what is going on here, other than that the javascript that converts buttons into dropdown menus has not fired. Are there any javascript errors? Marten is working on rewriting libraries, and we will be eliminating the progressive loading popupmenus for something much more efficient, but this also might indicate a bug so let us know if there is anything odd in the console. -- James Taylor, Associate Professor, Biology/CS, Emory University On Mon, Nov 4, 2013 at 1:58 PM, Robert Baertsch baert...@soe.ucsc.edu wrote: HI James, I just pulled in the latest code to see how you changed from iframe to divs. Very exciting update. I tried importing a bam file into the library using
Re: [galaxy-dev] latest galaxy-central version
It keeps doing posts, and I'm not seeing any new errors. POST http://su2c-dev.ucsc.edu:8383/library_common/library_item_updates 200 OK 121ms When I did a browser refresh, I got the following javascript error: (I am logged in) Galaxy.currUser is undefined on Line 631 in history-panel.js When I opened the data library where the bam file was copying, everything is rendered ok. It seems the browser refresh fixed things. -Robert On Nov 4, 2013, at 11:14 AM, James Taylor ja...@jamestaylor.org wrote: Robert, I'm not sure what is going on here, other than that the javascript that converts buttons into dropdown menus has not fired. Are there any javascript errors? Marten is working on rewriting libraries, and we will be eliminating the progressive loading popupmenus for something much more efficient, but this also might indicate a bug so let us know if there is anything odd in the console. -- James Taylor, Associate Professor, Biology/CS, Emory University On Mon, Nov 4, 2013 at 1:58 PM, Robert Baertsch baert...@soe.ucsc.edu wrote: HI James, I just pulled in the latest code to see how you changed from iframe to divs. Very exciting update. I tried importing a bam file into the library using the admin tool that reads the file directly from a path on the server. During the copy operation, the screen gets into a weird state. ps -ef shows these two processes running: python /data/galaxy-central/tools/data_source/upload.py /data/galaxy-central /data/galaxy-central/database/tmp/tmpywxnFE /data/galaxy-central/database/tmp/tmpXkEvSY 8876:/data/galaxy-central/database/job_working_directory/004/4547/dataset_8876_files:/data/galaxy-central/database/files/008/dataset_8876.dat samtools sort /data/galaxy-central/database/files/008/dataset_8876.dat /tmp/tmpNbcggh/sorted Do you finish rendering the screen after the copy operation is complete? -Robert Screen Shot 2013-11-04 at 10.53.02 AM.png ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] gzipped fastq reader
Peter and Dan,
I like the idea of replacing all open() with galaxy_open() in all tools. You
can tell the format by looking at the first 4 byes (see C code below from the
UCSC browser team). Is there some pythonic way of overriding open?
You need to read the first four bytes of the file to see if it is compressed
and call gzip.open inside of the function and pass pack the handle.
For now, it would require a global sweep through the tools to change open() to
galaxy_open(), but it is probably a good idea to have tool developers avoid
calling open directly.
You would have to have special handling if there are multiple files in the
compressed archive but that support could be added later.
-Robert
def galaxy_open(filename, mode=r):
compressor = getCompressor(filename, mode)
if compessor != NULL:
return openCompressed(filename, mode, compressor)
else:
return open(filename, mode)
def openCompressed(filename, mode):
4bytes = read4bytes(filename)
ext = getExtensionFromHdrSig(4bytes)
if ext == gz :
return gzip.open(filename, mode)
else if ext == bz2:
return bz2.BZ2File(filename, mode)
else if ext == zip:
return zipfile.ZipFile(filename, mode)
char *getExtensionFromHdrSig(char *first4bytes)
/* Check if header has signature of supported compression stream,
and return a phoney filename with extension for it, or NULL if no sig found.
*/
{
char buf[20];
char *ext=NULL;
if (startsWith(\x1f\x8b,first4bytes)) ext = gz;
else if (startsWith(\x1f\x9d\x90,first4bytes)) ext = Z;
else if (startsWith(BZ,first4bytes)) ext = bz2;
else if (startsWith(PK\x03\x04,first4bytes)) ext = zip;
if (ext==NULL)
return NULL;
}
On Jul 8, 2013, at 4:05 AM, Peter Cock wrote:
On Thu, Jul 4, 2013 at 9:49 PM, Robert Baertsch
robert.baert...@gmail.com wrote:
Dan,
Do these readers support gzip files?
reader = fastqVerboseErrorReader
reader = fastqReader
Presumably you are writing a Python script using this library?
The answer is a qualified yes. Instead of passing them a normal
file handle using open(example.fastq) you instead use
gzip.open(example.fastq) via import gzip.
Do I have to define a special type in galaxy for gzipped files or will the
fastq type be ok?
This needs a special file format - but you are not the first person to
look at this, some groups have defined custom gzipped variants of
the FASTQ formats within their own Galaxy instances. I've not
done this but there should be some useful emails in the archive.
Note you'd also need to modify any tool definitions to that they
can accept a gzipped FASTQ file.
Ideally, I would like to keep my files zipped and not have galaxy unzip
them, since they triple in size when unzipped.
I'm happy to do a push request if you don't support this but I want to make
sure I'm in line with your roadmap.
Personally I would like a more general system in Galaxy for
potentially any file type to be held compressed in a range of
formats (e.g. using gzip, bgzf, xy, bz2, etc), with exclusions
for things like BAM which are already compressed. This way
naive tools would get the gzipped file file uncompressed to a
temporary folder before use (i.e. no change for the tool wrapper),
but if a tool accepts a gzipped file it will get that (less disk IO
and CPU usage, but requires updating tool wrappers).
That idea is quite ambitious through ;)
I have written a simple tool to convert Illumina fastq to mapsplice fastq.
Does that already exist already somewhere?
I don't know.
Peter
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] gzipped fastq reader
great. Let's put the bx-python calls in a galaxy_open helper function. On Jul 8, 2013, at 3:20 PM, James Taylor wrote: open_compressed in bx-python does this already (for bz2 as well). On Jul 8, 2013, at 5:58 PM, Peter Cock p.j.a.c...@googlemail.com wrote: On Mon, Jul 8, 2013 at 10:24 PM, Robert Baertsch robert.baert...@gmail.com wrote: Peter and Dan, I like the idea of replacing all open() with galaxy_open() in all tools. You can tell the format by looking at the first 4 byes (see C code below from the UCSC browser team). Is there some pythonic way of overriding open? There is monkey patching (replace the current 'open' function with your modified version), but that is not a good idea in general. In any case, this would only affect the small number of Python tools which happen to use the Galaxy parsing libraries - which is a very small fraction of the tools in Galaxy. Most of the tools in Galaxy are compiled programs and are entirely separate. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] gzipped fastq reader
great. Let's put the bx-python calls in a galaxy_open helper function. On Jul 8, 2013, at 3:20 PM, James Taylor wrote: open_compressed in bx-python does this already (for bz2 as well). On Jul 8, 2013, at 5:58 PM, Peter Cock p.j.a.c...@googlemail.com wrote: On Mon, Jul 8, 2013 at 10:24 PM, Robert Baertsch robert.baert...@gmail.com wrote: Peter and Dan, I like the idea of replacing all open() with galaxy_open() in all tools. You can tell the format by looking at the first 4 byes (see C code below from the UCSC browser team). Is there some pythonic way of overriding open? There is monkey patching (replace the current 'open' function with your modified version), but that is not a good idea in general. In any case, this would only affect the small number of Python tools which happen to use the Galaxy parsing libraries - which is a very small fraction of the tools in Galaxy. Most of the tools in Galaxy are compiled programs and are entirely separate. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] gzipped fastq reader
I will implement this if the galaxy team likes the approach. We did this in ucsc genome browser code years ago: a single open_helper call handles, gzip, http, ftp and pipes. No need to care about how the data is compressed or where it data resides. wouldn't it be great to be able to pipe data between workflow steps rather than writing to disk? I admit that this will require some work but the first step is to abstract the open. On Jul 9, 2013, at 10:38 AM, Peter Cock wrote: On Tue, Jul 9, 2013 at 5:53 PM, Robert Baertsch rbaer...@ucsc.edu wrote: On Jul 8, 2013, at 3:33 PM, Peter Cock wrote: The tools available in Galaxy are written in a range of languages including C, Perl, R, etc. Yes, some are in Python, but of those most are independent of Galaxy and can be used separately from Galaxy. the helper function would have to ported to R. We are talking about how galaxy compressed data. Once we decide that, we can determine how to best implement it. Individual tools called from Galaxy read and create the files - and we can't usually control them at this level (modifying them all to call a Galaxy managed file open mechanism is not an option). Proposal: Do not treat compressed data as a separate data type. Treat it as an independent attribute that can be applied to any data. Otherwise you will have to create a gzipped , zip and bz2 type for every type that you want to compress. That's what I've been saying - the fact that some people are already using a new gzipped FASTQ format within their Galaxy instances is practical, but I view it as a short term solution only. Encoding the gzip status in the datatype will create an explosion of datatypes. Compression is not actually a datatype, it tells you nothing about the content data that is stored in the file. What we'd previously discussed was a dual system, holding the file type as now (e.g. FASTA, SAM, GFF3, etc) and any compression (e.g., None, normal GZIP, BGZF which is a GZIP variant, BZIP2, etc). What about tabular. Should we create tab.gz, tab.bz2 and tab.zip also? Note ZIP is a bit different, as it is often a multiple file bundle - it behaves differently from GZIP, BGZF, XY, BZIP2 etc in that regard. But otherwise, yes. As a specific example, the tabix tool used BGZF compressed tabular data to combine compression and efficient random access. This would be useful for many annotation files (e.g. GTF, GFF3). This will quickly get out of hand and create a mess for tool developers that need to support all thees types. Why? Individual tool developers don't need to know if Galaxy is keeping the original data file on disk compressed - unless the tool XML says otherwise, Galaxy would hide this detail and call the tool with an uncompressed input file. (Unix named pipe which decompresses the file on the file would be a potential alternative - but only if the tool XML was marked up to say that an input could be streamed. The default must be to assume potential random access to the input files) The tool code and tool xml should be written to handle uncompressed data and galaxy should handle the details of decompression. This is not hard to do. It isn't trivial either ;) Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] gzipped fastq reader
Dan, Do these readers support gzip files? reader = fastqVerboseErrorReader reader = fastqReader Do I have to define a special type in galaxy for gzipped files or will the fastq type be ok? Ideally, I would like to keep my files zipped and not have galaxy unzip them, since they triple in size when unzipped. I'm happy to do a push request if you don't support this but I want to make sure I'm in line with your roadmap. I have written a simple tool to convert Illumina fastq to mapsplice fastq. Does that already exist already somewhere? -Robert ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] history item status not updating
I submitted a workflow that in turn submits a drm job to sun grid engine. The queue had an error (probably due to a problem with automount). Traceback (most recent call last): File /inside/depot4/galaxy/lib/galaxy/jobs/runners/__init__.py, line 60, in run_next method(arg) File /inside/depot4/galaxy/lib/galaxy/jobs/runners/drmaa.py, line 169, in queue_job external_job_id = self.ds.runJob(jt) File /inside/depot4/galaxy/eggs/drmaa-0.4b3-py2.7.egg/drmaa/__init__.py, line 331, in runJob _h.c(_w.drmaa_run_job, jid, _ct.sizeof(jid), jobTemplate) File /inside/depot4/galaxy/eggs/drmaa-0.4b3-py2.7.egg/drmaa/helpers.py, line 213, in c return f(*(args + (error_buffer, sizeof(error_buffer File /inside/depot4/galaxy/eggs/drmaa-0.4b3-py2.7.egg/drmaa/errors.py, line 90, in error_check raise _ERRORS[code-1](code %s: %s % (code, error_buffer.value)) DeniedByDrmException: code 17: error: no suitable queues When the sysadmin cleared the error, the job started running normally after being in an error state for 10 minutes. The cool thing is that galaxy kept running without a problem. galaxy.jobs.runners.drmaa DEBUG 2013-06-27 14:51:22,968 (10481/4767487) state change: job is running galaxy.jobs.runners.drmaa DEBUG 2013-06-27 15:00:23,628 (10481/4767487) state change: job is queued and active galaxy.jobs.runners.drmaa DEBUG 2013-06-27 15:00:27,751 (10481/4767487) state change: job is running However in the history panel, the job shows as queued but not running, even if I refresh the history panel. Is this normal or should the status change to running? I'm using this version of straight galaxy: changeset: 10162:f295092476c7 branch: stable parent: 10160:2efb1083676b user:Dannon Baker dannonba...@me.com date:Sat Jun 15 09:08:09 2013 -0400 summary: Fix reports import issue reported by Lance, https://trello.com/card/bug-in-reports-webapp-imports/506338ce32ae458f6d15e4b3/943 -Robert Baertsch UC Santa Cruz ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] history item status not updating
ignore this last post On Jun 27, 2013, at 3:09 PM, Robert Baertsch wrote: I submitted a workflow that in turn submits a drm job to sun grid engine. The queue had an error (probably due to a problem with automount). Traceback (most recent call last): File /inside/depot4/galaxy/lib/galaxy/jobs/runners/__init__.py, line 60, in run_next method(arg) File /inside/depot4/galaxy/lib/galaxy/jobs/runners/drmaa.py, line 169, in queue_job external_job_id = self.ds.runJob(jt) File /inside/depot4/galaxy/eggs/drmaa-0.4b3-py2.7.egg/drmaa/__init__.py, line 331, in runJob _h.c(_w.drmaa_run_job, jid, _ct.sizeof(jid), jobTemplate) File /inside/depot4/galaxy/eggs/drmaa-0.4b3-py2.7.egg/drmaa/helpers.py, line 213, in c return f(*(args + (error_buffer, sizeof(error_buffer File /inside/depot4/galaxy/eggs/drmaa-0.4b3-py2.7.egg/drmaa/errors.py, line 90, in error_check raise _ERRORS[code-1](code %s: %s % (code, error_buffer.value)) DeniedByDrmException: code 17: error: no suitable queues When the sysadmin cleared the error, the job started running normally after being in an error state for 10 minutes. The cool thing is that galaxy kept running without a problem. galaxy.jobs.runners.drmaa DEBUG 2013-06-27 14:51:22,968 (10481/4767487) state change: job is running galaxy.jobs.runners.drmaa DEBUG 2013-06-27 15:00:23,628 (10481/4767487) state change: job is queued and active galaxy.jobs.runners.drmaa DEBUG 2013-06-27 15:00:27,751 (10481/4767487) state change: job is running However in the history panel, the job shows as queued but not running, even if I refresh the history panel. Is this normal or should the status change to running? I'm using this version of straight galaxy: changeset: 10162:f295092476c7 branch: stable parent: 10160:2efb1083676b user:Dannon Baker dannonba...@me.com date:Sat Jun 15 09:08:09 2013 -0400 summary: Fix reports import issue reported by Lance, https://trello.com/card/bug-in-reports-webapp-imports/506338ce32ae458f6d15e4b3/943 -Robert Baertsch UC Santa Cruz ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] bug in galaxy-central security
Dannon,
Can you also apply the same select2 fix to:
templates/webapps/galaxy/library/common/library_permissions.mako
It has the same problem.
-Robert
On Jun 6, 2013, at 10:42 AM, Dannon Baker wrote:
That'll work, but it'll deactivate select2 across the board for all pages in
your galaxy instance. I committed a fix for those specific admin pages
yesterday, available in galaxy-central (stable as well as default branch).
-Dannon
On Thu, Jun 6, 2013 at 1:37 PM, Robert Baertsch baert...@soe.ucsc.edu wrote:
Hi Dannon,
I deleted the declaration to select2 in base.mako (line 42) and that fixed
the problem for now. (thanks to Ted for the tip!)
-Robert
On Jun 4, 2013, at 1:30 PM, Dannon Baker wrote:
Hey Robert,
Thanks for catching this; I'm working on getting a fix out (this affects
multiple areas in the admin panel) but if you need to temporarily fix a
single page to get things working you can execute
$('select').select2(destroy) in your javascript console.
-Dannon
On Tue, Jun 4, 2013 at 3:20 PM, Robert Baertsch baert...@soe.ucsc.edu
wrote:
James,
I upgraded to the latest galaxy-central and starting with a blank database.
When I add more than 20 users, the security group admin screen doesn't
populate.
mako is pulling the correct data but there is some jquery code for
autocomplete or search that seems to be having trouble.
Here is a screen shot. Let me know if you want admin access to this server
and I'll setup a user for you.
-Robert
Screen Shot 2013-06-04 at 12.18.21 PM.png
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] bug in galaxy-central security
Hi Dannon,
I deleted the declaration to select2 in base.mako (line 42) and that fixed the
problem for now. (thanks to Ted for the tip!)
-Robert
On Jun 4, 2013, at 1:30 PM, Dannon Baker wrote:
Hey Robert,
Thanks for catching this; I'm working on getting a fix out (this affects
multiple areas in the admin panel) but if you need to temporarily fix a
single page to get things working you can execute
$('select').select2(destroy) in your javascript console.
-Dannon
On Tue, Jun 4, 2013 at 3:20 PM, Robert Baertsch baert...@soe.ucsc.edu wrote:
James,
I upgraded to the latest galaxy-central and starting with a blank database.
When I add more than 20 users, the security group admin screen doesn't
populate.
mako is pulling the correct data but there is some jquery code for
autocomplete or search that seems to be having trouble.
Here is a screen shot. Let me know if you want admin access to this server
and I'll setup a user for you.
-Robert
Screen Shot 2013-06-04 at 12.18.21 PM.png
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] bug in galaxy-central security
Ah, I see it now.
https://bitbucket.org/galaxy/galaxy-central/commits/c4096b030f10eb346b24da98ce16b72d23672f03
On Jun 6, 2013, at 10:42 AM, Dannon Baker wrote:
That'll work, but it'll deactivate select2 across the board for all pages in
your galaxy instance. I committed a fix for those specific admin pages
yesterday, available in galaxy-central (stable as well as default branch).
-Dannon
On Thu, Jun 6, 2013 at 1:37 PM, Robert Baertsch baert...@soe.ucsc.edu wrote:
Hi Dannon,
I deleted the declaration to select2 in base.mako (line 42) and that fixed
the problem for now. (thanks to Ted for the tip!)
-Robert
On Jun 4, 2013, at 1:30 PM, Dannon Baker wrote:
Hey Robert,
Thanks for catching this; I'm working on getting a fix out (this affects
multiple areas in the admin panel) but if you need to temporarily fix a
single page to get things working you can execute
$('select').select2(destroy) in your javascript console.
-Dannon
On Tue, Jun 4, 2013 at 3:20 PM, Robert Baertsch baert...@soe.ucsc.edu
wrote:
James,
I upgraded to the latest galaxy-central and starting with a blank database.
When I add more than 20 users, the security group admin screen doesn't
populate.
mako is pulling the correct data but there is some jquery code for
autocomplete or search that seems to be having trouble.
Here is a screen shot. Let me know if you want admin access to this server
and I'll setup a user for you.
-Robert
Screen Shot 2013-06-04 at 12.18.21 PM.png
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
[galaxy-dev] migration error
I upgraded to the latest galaxy-central and got an error when running migration script 115 which lengthens the password field from 40-255. It failed saying that the table migration_tmp already exists. I ran this without any existing database so I don't think it is anything on my end. Any pointers? .schema migration_tmp CREATE TABLE migration_tmp ( id INTEGER NOT NULL, create_time TIMESTAMP, update_time TIMESTAMP, tool_shed_repository_id INTEGER NOT NULL, name VARCHAR(255), version VARCHAR(40), type VARCHAR(40), uninstalled BOOLEAN, error_message TEXT, PRIMARY KEY (id), FOREIGN KEY(tool_shed_repository_id) REFERENCES tool_shed_repository (id) ); ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] symlink to june_2007_style
When merging in the latest galaxy-central, I'm getting this error: merging lib/galaxy/webapps/galaxy/controllers/library_common.py merging lib/galaxy/webapps/galaxy/controllers/page.py merging lib/galaxy/webapps/galaxy/controllers/user.py merging run.sh merging scripts/manage_db.py abort: could not symlink to 'style': File exists: /Users/robertbaertsch/src/medbook-merge/medbook-galaxy-central/static/june_2007_style Should I do the following: $ mv june_2007_style style $ ln -s style june_2007_style and then $ hg merge ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] reducing overhead for quick jobs
Hi All, After changing the sleep in manager.py from 5 seconds to 0.2 seconds, the job start is very snappy. I did not touch the various sleep(10)s that don't seem to be relevant. I'm testing how long it takes to sort a short bed file in galaxy. After the fix, the problem seems to be at the end of the process. In the timeline below the gap between the job ending on the server and the client getting the update is 5 seconds. Is that a built in sleep or something else? Can I turn off metadata for tools that don't generate it? BTW: thousands of seconds are on in the manager and handler logs and should be turned on in the main.log. Rather than 2 round trips to the server. I think we need a special class of tools that run in a single round trip (like the eyeball viz). The max run time for this class of jobs should be no more than a few seconds. Is that is the roadmap somewhere? -Robert Baertsch UC Santa Cruz https://medbook.ucsc.edu RUN1 - 9 seconds (1 sec to start, 0.2 sec to run , 2 sec metadata, 5.5 secs to update client) 12:08:12.? main: log job on main thread 12:08:12.8 manager : galaxy.jobs.manager DEBUG 2013-03-30 12:08:12,867 (2308) Job assigned to handler 'handler2' 12:08:13.26 handler:start handler 12:08:13.35 handler:job dispatched 12:08:13,44 handler:job runner started 12:08:13,72 handler:python tool started 12:08:13,91 handler: python tool finished 12:08:13,98 handler: set_metadata.sh started 12:08:15,70 handler:metadata ended 12:08:15,85 handler:job Ended 12:08:21main: /api/histories - update on client RUN2 - 9 seconds (1.8 sec to start, 0.2 sec to run, 2 sec metadata, 4.5 sec to update client) 12:23:00.? main: log job on main thread 12:23:00,60 manager : galaxy.jobs.manager DEBUG 2013-03-30 12:08:12,867 (2308) Job assigned to handler 'handler2' 12:23:01,63 handler:start handler 12:23:01,72 handler:job dispatched 12:23:01,81 handler:job runner started 12:23:02,44 handler:python tool started 12:23:02,64 handler: python tool finished 12:23:02,72 handler: set_metadata.sh started 12:23:04,64 handler:metadata ended 12:23:04,71 handler:job Ended 12:23:09main: /api/histories - update on client ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
