Re: [galaxy-dev] Support for subdirs in dataset extra_files_path

2011-06-02 Thread Daniel Blankenberg
Patch Imported in 5618:b9fdb88da530.

Thanks!

Dan


On Mar 16, 2011, at 11:50 AM, Jim Johnson wrote:

> Request is issue#494   
> https://bitbucket.org/galaxy/galaxy-central/issue/494/support-sub-dirs-in-extra_files_path-patch
> 
> I'm finding that some qiime metagenomics applications build HTML results with 
> an inherent directory structure.   For some other applications, e.g. FastQC, 
> I've been able to flatten the hierarchy and edit the html, but that appears 
> problematic for qiime.
> 
> Galaxy hasn't supported a dataset extra_files_path hierarchy, though the 
> developers don't seem opposed to the idea: 
> http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-October/003605.html
> 
> I added a route  in  lib/galaxy/web/buildapp.pyand modified the dataset 
> download code to traverse a hierarchy in lib/galaxy/web/controllers/dataset.py
> I don't think these add any security vulnerabilities, (I tried the obvious 
> ../../ ).
> 
> $ hg diff lib/galaxy/web/buildapp.py
> diff -r 6ae06d89fec7 lib/galaxy/web/buildapp.py
> --- a/lib/galaxy/web/buildapp.pyWed Mar 16 09:01:57 2011 -0400
> +++ b/lib/galaxy/web/buildapp.pyWed Mar 16 10:24:13 2011 -0500
> @@ -94,6 +94,8 @@
> webapp.add_route( '/async/:tool_id/:data_id/:data_secret', 
> controller='async', action='index', tool_id=None, data_id=None, 
> data_secret=None )
> webapp.add_route( '/:controller/:action', action='index' )
> webapp.add_route( '/:action', controller='root', action='index' )
> +# allow for subdirectories in extra_files_path
> +webapp.add_route( '/datasets/:dataset_id/display/{filename:.+?}', 
> controller='dataset', action='display', dataset_id=None, filename=None)
> webapp.add_route( '/datasets/:dataset_id/:action/:filename', 
> controller='dataset', action='index', dataset_id=None, filename=None)
> webapp.add_route( 
> '/display_application/:dataset_id/:app_name/:link_name/:user_id/:app_action/:action_param',
>  controller='dataset', action='display_application', dataset_id=None, 
> user_id=None, app_name = None, link_name = None, app_action = None, 
> action_param = None )
> webapp.add_route( '/u/:username/d/:slug', controller='dataset', 
> action='display_by_username_and_slug' )
> $
> $ hg diff lib/galaxy/web/controllers/dataset.py
> diff -r 6ae06d89fec7 lib/galaxy/web/controllers/dataset.py
> --- a/lib/galaxy/web/controllers/dataset.py Wed Mar 16 09:01:57 2011 -0400
> +++ b/lib/galaxy/web/controllers/dataset.py Wed Mar 16 10:24:29 2011 -0500
> @@ -266,17 +266,18 @@
> log.exception( "Unable to add composite parent %s to 
> temporary library download archive" % data.file_name)
> msg = "Unable to create archive for download, please 
> report this error"
> messagetype = 'error'
> -flist = glob.glob(os.path.join(efp,'*.*')) # glob returns 
> full paths
> -for fpath in flist:
> -efp,fname = os.path.split(fpath)
> -try:
> -archive.add( fpath,fname )
> -except IOError:
> -error = True
> -log.exception( "Unable to add %s to temporary 
> library download archive" % fname)
> -msg = "Unable to create archive for download, please 
> report this error"
> -messagetype = 'error'
> -continue
> +for root, dirs, files in os.walk(efp):
> +for fname in files:
> +fpath = os.path.join(root,fname)
> +rpath = os.path.relpath(fpath,efp)
> +try:
> +archive.add( fpath,rpath )
> +except IOError:
> +error = True
> +log.exception( "Unable to add %s to temporary 
> library download archive" % rpath)
> +msg = "Unable to create archive for download, 
> please report this error"
> +messagetype = 'error'
> +continue
> if not error:
> if params.do_action == 'zip':
> archive.close()
> 
> 
> 
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
> http://lists.bx.psu.edu/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


[galaxy-dev] Support for subdirs in dataset extra_files_path

2011-03-16 Thread Jim Johnson

Request is issue#494   
https://bitbucket.org/galaxy/galaxy-central/issue/494/support-sub-dirs-in-extra_files_path-patch

I'm finding that some qiime metagenomics applications build HTML results with 
an inherent directory structure.   For some other applications, e.g. FastQC, 
I've been able to flatten the hierarchy and edit the html, but that appears 
problematic for qiime.

Galaxy hasn't supported a dataset extra_files_path hierarchy, though the 
developers don't seem opposed to the idea: 
http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-October/003605.html

I added a route  in  lib/galaxy/web/buildapp.pyand modified the dataset 
download code to traverse a hierarchy in lib/galaxy/web/controllers/dataset.py
I don't think these add any security vulnerabilities, (I tried the obvious 
../../ ).

$ hg diff lib/galaxy/web/buildapp.py
diff -r 6ae06d89fec7 lib/galaxy/web/buildapp.py
--- a/lib/galaxy/web/buildapp.pyWed Mar 16 09:01:57 2011 -0400
+++ b/lib/galaxy/web/buildapp.pyWed Mar 16 10:24:13 2011 -0500
@@ -94,6 +94,8 @@
 webapp.add_route( '/async/:tool_id/:data_id/:data_secret', 
controller='async', action='index', tool_id=None, data_id=None, 
data_secret=None )
 webapp.add_route( '/:controller/:action', action='index' )
 webapp.add_route( '/:action', controller='root', action='index' )
+# allow for subdirectories in extra_files_path
+webapp.add_route( '/datasets/:dataset_id/display/{filename:.+?}', 
controller='dataset', action='display', dataset_id=None, filename=None)
 webapp.add_route( '/datasets/:dataset_id/:action/:filename', 
controller='dataset', action='index', dataset_id=None, filename=None)
 webapp.add_route( 
'/display_application/:dataset_id/:app_name/:link_name/:user_id/:app_action/:action_param',
 controller='dataset', action='display_application', dataset_id=None, 
user_id=None, app_name = None, link_name = None, app_action = None, 
action_param = None )
 webapp.add_route( '/u/:username/d/:slug', controller='dataset', 
action='display_by_username_and_slug' )
$
$ hg diff lib/galaxy/web/controllers/dataset.py
diff -r 6ae06d89fec7 lib/galaxy/web/controllers/dataset.py
--- a/lib/galaxy/web/controllers/dataset.py Wed Mar 16 09:01:57 2011 -0400
+++ b/lib/galaxy/web/controllers/dataset.py Wed Mar 16 10:24:29 2011 -0500
@@ -266,17 +266,18 @@
 log.exception( "Unable to add composite parent %s to temporary 
library download archive" % data.file_name)
 msg = "Unable to create archive for download, please report 
this error"
 messagetype = 'error'
-flist = glob.glob(os.path.join(efp,'*.*')) # glob returns full 
paths
-for fpath in flist:
-efp,fname = os.path.split(fpath)
-try:
-archive.add( fpath,fname )
-except IOError:
-error = True
-log.exception( "Unable to add %s to temporary library 
download archive" % fname)
-msg = "Unable to create archive for download, please report 
this error"
-messagetype = 'error'
-continue
+for root, dirs, files in os.walk(efp):
+for fname in files:
+fpath = os.path.join(root,fname)
+rpath = os.path.relpath(fpath,efp)
+try:
+archive.add( fpath,rpath )
+except IOError:
+error = True
+log.exception( "Unable to add %s to temporary library 
download archive" % rpath)
+msg = "Unable to create archive for download, please 
report this error"
+messagetype = 'error'
+continue
 if not error:
 if params.do_action == 'zip':
 archive.close()




___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/