Hey Peter,

It looks like ext will be undefined in your default implementation of
_archive_main_file - is this going to cause an error or am I missing
something?

If this is the interface you would like - I could rework it to handle
this ext issue and apply it directly to galaxy-central. Otherwise, I
will await a pull request.

Thanks for working on this!

-John

On Tue, Nov 26, 2013 at 10:55 AM, Peter Cock <p.j.a.c...@googlemail.com> wrote:
> On Mon, Nov 25, 2013 at 10:37 PM, Peter Cock <p.j.a.c...@googlemail.com> 
> wrote:
>> On Mon, Nov 25, 2013 at 7:47 PM, John Chilton <chil...@msi.umn.edu> wrote:
>>> On Mon, Nov 25, 2013 at 11:59 AM, Peter Cock <p.j.a.c...@googlemail.com> 
>>> wrote:
>>>> Hello all,
>>>>
>>>> I would like to modify the default ZIP bundle behaviour
>>>> for composite datatypes, which currently insists on
>>>> creating an HTML "main" file. To do this I think the current
>>>> monolithic _archive_composite_dataset function needs
>>>> to be refactored in file lib/galaxy/datatypes/data.py
>>>>
>>>> I would like to move the section which currently names
>>>> and populates the HTML index file into a sub-method,
>>>> i.e. this chunk:
>>>>
>>>>                 path = data.file_name
>>>>                 fname = os.path.split(path)[-1]
>>>>                 efp = data.extra_files_path
>>>>                 htmlname = os.path.splitext(outfname)[0]
>>>>                 if not htmlname.endswith(ext):
>>>>                     htmlname = '%s_%s' % (htmlname,ext)
>>>>                 archname = '%s.html' % htmlname # fake the real nature
>>>> of the html file
>>>>                 try:
>>>>                     archive.add(data.file_name,archname)
>>>>                 except IOError:
>>>>                     error = True
>>>>                     log.exception( "Unable to add composite parent %s
>>>> to temporary library download archive" % data.file_name)
>>>>                     msg = "Unable to create archive for download,
>>>> please report this error"
>>>>
>>>> Then by overriding this new method a subclass (custom
>>>> composite datatype) could for example set the filename
>>>> used inside the archive to be "index.html" or "index.htm"
>>>> or anything else like "README.txt" (doesn't have to be
>>>> HTML); alter the contents of the index file; or even not
>>>> include an index file in the archive.
>>>>
>>>> Would a pull request to do this be considered?
>>>
>>> Hey Peter,
>>>
>>> Definitely. This sounds like a great change - I am looking
>>> forward to merging it!
>>>
>>> -John
>>
>> I'll give that a go then, thanks.
>>
>> Peter
>
> Hi John,
>
> Here's a first cut - patch at end of email - and matching changes
> to the BLAST database class to use this:
>
> https://github.com/peterjc/galaxy_blast/commit/0a205ab98a639dccf29db72f81b02023798c1378
>
> Thoughts on the method signature welcome (should it get the
> full data object for example - for other more ambitious changes),
> and the return values (error condition signalling).
>
> With this change, rather than having a dummy HTML file
> (named after a truncated version of the dataset name),
> I can use a non-HTML filename like blastdb.log (or just
> README.txt etc), or have no dummy central file at all
> (useful for old BLAST databases where the central
> dummy file is empty).
>
> The logical next step would be to move the walk loop
> into a separate method as well (to allow that to be
> overridden to - for example if you want to enforce a
> subfolder structure inside the ZIP file).
>
> Regards,
>
> Peter
>
> --
>
> $ hg diff lib/galaxy/datatypes/data.py
> diff -r 359a822b2e5d lib/galaxy/datatypes/data.py
> --- a/lib/galaxy/datatypes/data.py    Tue Nov 26 08:04:27 2013 -0600
> +++ b/lib/galaxy/datatypes/data.py    Tue Nov 26 16:45:47 2013 +0000
> @@ -199,6 +199,29 @@
>              out = "Can't create peek %s" % str( exc )
>          return out
>
> +    def _archive_main_file(self, archive, outfname, data_filename):
> +        """Called from _archive_composite_dataset to add central file
> to archive.
> +
> +        Unless subclassed, this will add the main dataset file
> (argument data_filename)
> +        to the archive, as an HTML file with its filename derived
> from the dataset name
> +        (argument outfname).
> +
> +        Returns a tuple of boolean, string, string: (error, msg, messagetype)
> +        """
> +        error, msg, messagetype = False, "", ""
> +        htmlname = os.path.splitext(outfname)[0]
> +        if not htmlname.endswith(ext):
> +            htmlname = '%s_%s' % (htmlname, ext)
> +        archname = '%s.html' % htmlname # fake the real nature of the
> html file
> +        try:
> +            archive.add(data_filename, archname)
> +        except IOError:
> +            error = True
> +            log.exception("Unable to add composite parent %s to
> temporary library download archive" % data_filename)
> +            msg = "Unable to create archive for download, please
> report this error"
> +            messagetype = "error"
> +        return error, msg, messagetype
> +
>      def _archive_composite_dataset( self, trans, data=None, **kwd ):
>          # save a composite object into a compressed archive for downloading
>          params = util.Params( kwd )
> @@ -233,33 +256,25 @@
>                  messagetype = 'error'
>              if not error:
>                  current_user_roles = trans.get_current_user_roles()
> -                ext = data.extension
>                  path = data.file_name
>                  fname = os.path.split(path)[-1]
>                  efp = data.extra_files_path
> -                htmlname = os.path.splitext(outfname)[0]
> -                if not htmlname.endswith(ext):
> -                    htmlname = '%s_%s' % (htmlname,ext)
> -                archname = '%s.html' % htmlname # fake the real
> nature of the html file
> -                try:
> -                    archive.add(data.file_name,archname)
> -                except IOError:
> -                    error = True
> -                    log.exception( "Unable to add composite parent %s
> to temporary library download archive" % data.file_name)
> -                    msg = "Unable to create archive for download,
> please report this error"
> -                    messagetype = 'error'
> -                for root, dirs, files in os.walk(efp):
> -                    for fname in files:
> -                        fpath = os.path.join(root,fname)
> -                        rpath = os.path.relpath(fpath,efp)
> -                        try:
> -                            archive.add( fpath,rpath )
> -                        except IOError:
> -                            error = True
> -                            log.exception( "Unable to add %s to
> temporary library download archive" % rpath)
> -                            msg = "Unable to create archive for
> download, please report this error"
> -                            messagetype = 'error'
> -                            continue
> +                #Add any central file to the archive,
> +                error, msg, messagetype =
> self._archive_main_file(archive, outfname, path)
> +                if not error:
> +                    #Add any child files to the archive,
> +                    for root, dirs, files in os.walk(efp):
> +                        for fname in files:
> +                            fpath = os.path.join(root,fname)
> +                            rpath = os.path.relpath(fpath,efp)
> +                            try:
> +                                archive.add( fpath,rpath )
> +                            except IOError:
> +                                error = True
> +                                log.exception( "Unable to add %s to
> temporary library download archive" % rpath)
> +                                msg = "Unable to create archive for
> download, please report this error"
> +                                messagetype = 'error'
> +                                continue
>                  if not error:
>                      if params.do_action == 'zip':
>                          archive.close()
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Reply via email to