Duddy, John wrote:
> So, to summarize:
> 1 - I can upload files via the API now, which returns the id
> 2 - If I don't upload via directory paths, the file upload will not be 
> instant, so I'll have to use a poller to find out when the file upload is done
> 3 - I can write a new API to set metadata on a dataset and call that when the 
> file is done. If I don't, the only metadata file file will have is the stuff 
> created during set_meta().
> 
> Right?

I haven't worked on the API since it was originally written, so I am not
sure whether direct upload via the API was ever written/tested.  You can
upload via filesystem paths or the server directory methods, though.

None of the methods will be instant - once the upload request is made, a
dataset is created, but the upload will still need to run through the
upload tool and set required metadata.

3 is right.

--nate

> 
> 
> John Duddy
> Sr. Staff Software Engineer
> Illumina, Inc.
> 9885 Towne Centre Drive
> San Diego, CA 92121
> Tel: 858-736-3584
> E-mail: jdu...@illumina.com
> 
> 
> -----Original Message-----
> From: Nate Coraor [mailto:n...@bx.psu.edu] 
> Sent: Friday, May 06, 2011 8:12 AM
> To: Duddy, John
> Cc: galaxy-dev@lists.bx.psu.edu
> Subject: Re: [galaxy-dev] Question on setting metadata on upload via API
> 
> Duddy, John wrote:
> > I'm looking at extending the metadata fields for one of the supported file 
> > types. The files can get VERY large, and since I'm creating those files, 
> > I'd like to save as metadata some of the information I have on the contents.
> > 
> > Specifically, I'd like to tag the files with information about the sample 
> > as well as the byte offsets in the file for every millionth record (to 
> > facilitate fast file splitting). I saw the sample script for uploading a 
> > file to a library via the API. It looks to me like the most direct approach 
> > would be to modify LibrariesController.create() to handle arbitrary 
> > key-value pairs to be set on the data file.
> > 
> > However, that might not support things like having metadata values be 
> > arrays.
> > 
> > So, does my understanding of the problem and the areas involved seem 
> > reasonable? Is there something already out there that does what I need? I 
> > saw the set_metadata.py script, but that looks like it is meant to operate 
> > directly on the files, which might not make a running instance of Galaxy 
> > too happy. Is there an API version of that (or something in one of the web 
> > controllers I can repurpose to my needs)?
> 
> Hi John,
> 
> Galaxy will still set its own required metadata elements (determined by
> the file type) on upload, but if you create additional optional metadata
> elements and don't set them in your datatype's set_meta(), you could
> then set them after the job runs with a call to the API.  Editing
> metadata via the API is not yet implemented, but shouldn't be too
> difficult to add.
> 
> This does mean you'd have to get the newly created dataset ID and make
> second call to the API.  If that's not feasible it could probably be
> incorporated at creation time, but would be trickier since the upload
> code doesn't account for the possibility that metadata exists prior to
> running the upload tool itself.
> 
> --nate
> 
> > 
> > Thanks!
> > 
> > 
> > John Duddy
> > Illumina, Inc.
> > 
> > 
> 
> > ___________________________________________________________
> > Please keep all replies on the list by using "reply all"
> > in your mail client.  To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> > 
> >   http://lists.bx.psu.edu/
> 
> 
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Reply via email to