Hey Peter, Thanks for the contributions, I've committed the changes to -central. You're definitely right about the merge failure not setting the job status to error, and I'm looking into that now.
-Dannon On Thu, Apr 25, 2013 at 6:51 AM, Peter Cock <p.j.a.c...@googlemail.com>wrote: > Hi all, > > While testing a BLAST search using plain text output (the 'txt' > datatype in Galaxy) or subclasses (like 'blastxml') using galaxy-dist > with task splitting enabled, I found I new bug when the files are > merged: > > Error merging files global name 'shutil' is not defined > > This is a side effect of this commit due to a missing import > (perhaps the imports changed since Bjoern wrote the patch): > > https://bitbucket.org/galaxy/galaxy-central/commits/510b063792de89332428ed12ea1025155a7769c9 > > In addition to the missing import in data.py, there is a potential > similar issue with binary.py but this is masked by an evil > import * line ;) > > My suggested patch for the import issue below, however right > now even though the merge is failing, Galaxy failed to treat > the job as a failure - it was shown as a green successful job > with "Error merging files" as stdout, and for stderr: > "global name 'shutil' is not defined", but return code 0. > > So there is a deeper problem with the merge error not marking > the job as failed. > > Also there is a minor bug in that the stdout and stderr are > combined for the peep text without any white space - > presumably most of the time the stdout will have a trailing > new line, but not here - thus "filesglobal" [sic], not say > "files global" or a newline. There's a second patch below > for that minor issue too (although it seems my new lines > get turned into spaces by the time they are shown in > the history peep text). > > Regards, > > Peter > > -- > > $ hg diff > diff -r 8b9ca63f9128 lib/galaxy/datatypes/binary.py > --- a/lib/galaxy/datatypes/binary.py Wed Apr 24 16:24:41 2013 -0400 > +++ b/lib/galaxy/datatypes/binary.py Thu Apr 25 11:32:51 2013 +0100 > @@ -12,7 +12,7 @@ > from bx.seq.twobit import TWOBIT_MAGIC_NUMBER, > TWOBIT_MAGIC_NUMBER_SWAP, TWOBIT_MAGIC_SIZE > from urllib import urlencode, quote_plus > import zipfile, gzip > -import os, subprocess, tempfile > +import os, subprocess, tempfile, shutil > import struct > > log = logging.getLogger(__name__) > diff -r 8b9ca63f9128 lib/galaxy/datatypes/data.py > --- a/lib/galaxy/datatypes/data.py Wed Apr 24 16:24:41 2013 -0400 > +++ b/lib/galaxy/datatypes/data.py Thu Apr 25 11:32:51 2013 +0100 > @@ -3,6 +3,7 @@ > import mimetypes > import os > import sys > +import shutil > import tempfile > import zipfile > from cgi import escape > > > > > $ hg diff > diff -r 8b9ca63f9128 lib/galaxy/jobs/__init__.py > --- a/lib/galaxy/jobs/__init__.py Wed Apr 24 16:24:41 2013 -0400 > +++ b/lib/galaxy/jobs/__init__.py Thu Apr 25 11:43:30 2013 +0100 > @@ -924,7 +924,13 @@ > for dataset in > dataset_assoc.dataset.dataset.history_associations + > dataset_assoc.dataset.dataset.library_associations: #need to update > all associated output hdas, i.e. history was shared with job running > dataset.blurb = 'done' > dataset.peek = 'no peek' > - dataset.info = ( dataset.info or '' ) + > context['stdout'] + context['stderr'] > + dataset.info = (dataset.info or '') > + if context['stdout'].strip(): > + #Ensure white space between entries > + dataset.info = dataset.info.rstrip() + "\n" + > context['stdout'].strip() > + if context['stderr'].strip(): > + #Ensure white space between entries > + dataset.info = dataset.info.rstrip() + "\n" + > context['stderr'].strip() > dataset.tool_version = self.version_string > dataset.set_size() > if 'uuid' in context: > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ >
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/