Hi Christos,

Yes, I would be very interested by your modified file.

I'm forwarding to the mailing-list so this bug can be listed.

Thanks,

Pierre



Pierre Pericard
IE CDD - Projet Peptisan
Service Informatique et Bio-informatique (SIB)

Station Biologique de Roscoff
CNRS - UPMC
Place Georges Teissier
CS 90074
29688 ROSCOFF CEDEX
FRANCE
Tel : (+33) 2 98 29 56 46
http://abims.sb-roscoff.fr/

Le 01/02/2013 00:34, chriskan...@gmail.com a écrit :
Hi,

I'm working on the same thing, having a composite dataset with many files in 
many sub-folders.

Well during the development & testing of the tool that uses this composite 
datatype, I found out that Galaxy does not handle correctly sub-dirs, it works 
pretty good with files but if it finds a sub-dir then the copy messes it up.

To be more specific it uses shutil.copy without checking if it is file or 
directory, when coping dirs it has to use shutil.copytree.

I managed to make our local Galaxy installation to handle them quite ok with 
some modifications in method DiskObjectStor.update_from_file(...) located in 
lib/galaxy/objectstore/__init__.py

I can send you a copy of the file I've modified.

Regards,
Christos

------

Christos Kannas
Researcher
Ph.D Student

e-Health Laboratory
Dept. Computer Science,
University of Cyprus

<quote author='Pierre Pericard'>
Ok, thanks a lot, I'll try and get back to the mailing list if other
problems seem to occur.

Pierre.


Pierre Pericard
IE CDD - Projet Peptisan

Service Informatique et Bio-informatique (SIB)
Station Biologique de Roscoff
CNRS-UPMC
Place Georges Teissier
CS 90074
29688 Roscoff CEDEX
FRANCE
http://abims.sb-roscoff.fr/

Le 30/01/2013 11:45, Ross a écrit :
I'd suggest:
1) Make your new datatype a subclass of Html - it's a subclass of
composite that contains an HTML document as the object's native
display - so it can inform users what's there.

2) When constructing these new things, pass the file_path of the Html
(composite) dataset subclass to your wrapper on the command line

3) Your wrapper code can construct any arbitrary structure as long as
it's rooted in that directory - Galaxy stores it without any fuss. The
wrapper should also populate the Html file itself with nicely
laid annotation for the user to check out.

4) The key is that all tools that take this new datatype as input must
know how to decode this structure - they must be passed the
$input.extra_files_path which gives them that same path root.

5) Yes, it's odd and annoying that it's extra_files_path for
files_path. Go figure.

6) grep extra_files tools/*.xml to find some examples - I think the
velvetg one uses a complex subdirectory structure - but it doesn't
really matter - as long as your tools know how to deal with it, it's
just a directory to Galaxy!
I hope all this helps...


On Wed, Jan 30, 2013 at 8:22 PM, Pierre Pericard
<pierre.peric...@sb-roscoff.fr <mailto:pierre.peric...@sb-roscoff.fr>>
wrote:

     In that case, could anyone point me to an example of a Composite
     Datatype which could accept as input an unknown number of files in
     an unknown number of directories. I can't seem to understand how
     that would work based on the wiki.

     But maybe are we anticipating a near functionality of Galaxy.
     There were talks about changing the way Galaxy handle zip files,
     is it still on the table ?

     Thank in advance for any help,

     Pierre



     Pierre Pericard
     IE CDD - Projet Peptisan

     Service Informatique et Bio-informatique (SIB)
     Station Biologique de Roscoff
     CNRS-UPMC
     Place Georges Teissier
     CS 90074
     29688 Roscoff CEDEX
     FRANCE
     http://abims.sb-roscoff.fr/

     Le 29/01/2013 18:04, Peter Cock a écrit :

         On Tue, Jan 29, 2013 at 4:58 PM, Pierre Pericard
         <pierre.peric...@sb-roscoff.fr
         <mailto:pierre.peric...@sb-roscoff.fr>> wrote:

             If I'm not mistaking, Composite Datatypes allow for only
             one directory,
             whereas we need to keep a constant directory structure
             with 2 or more
             sub-directories containing our input files.

         I'm not sure if that is true - the example of HTML output with
         images
         comes to mind as a common use-case where subfolder(s) would be
         expected. I've only had limited first hand experience with
         Galaxy's
         composite datatypes myself though.

             We have no way to change these tools behavior (obviously
             not Galaxy-friendly
             ;-) ) and therefore need to maintain this structure in the
             job working
             directory.

         Perhaps a tool wrapper could create a dummy folder using symlinks
         (faster and less wasted disk than copying files), but that
         isn't ideal.

         Peter


     ___________________________________________________________


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

   http://lists.bx.psu.edu/
</quote>
Quoted from:
http://dev.list.galaxyproject.org/Best-way-to-work-with-one-directory-and-many-files-as-1-input-tp4658085p4658114.html

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/

Reply via email to