So, to summarize:
1 - I can upload files via the API now, which returns the id
2 - If I don't upload via directory paths, the file upload will not be instant, 
so I'll have to use a poller to find out when the file upload is done
3 - I can write a new API to set metadata on a dataset and call that when the 
file is done. If I don't, the only metadata file file will have is the stuff 
created during set_meta().

Right?


John Duddy
Sr. Staff Software Engineer
Illumina, Inc.
9885 Towne Centre Drive
San Diego, CA 92121
Tel: 858-736-3584
E-mail: jdu...@illumina.com


-----Original Message-----
From: Nate Coraor [mailto:n...@bx.psu.edu] 
Sent: Friday, May 06, 2011 8:12 AM
To: Duddy, John
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] Question on setting metadata on upload via API

Duddy, John wrote:
> I'm looking at extending the metadata fields for one of the supported file 
> types. The files can get VERY large, and since I'm creating those files, I'd 
> like to save as metadata some of the information I have on the contents.
> 
> Specifically, I'd like to tag the files with information about the sample as 
> well as the byte offsets in the file for every millionth record (to 
> facilitate fast file splitting). I saw the sample script for uploading a file 
> to a library via the API. It looks to me like the most direct approach would 
> be to modify LibrariesController.create() to handle arbitrary key-value pairs 
> to be set on the data file.
> 
> However, that might not support things like having metadata values be arrays.
> 
> So, does my understanding of the problem and the areas involved seem 
> reasonable? Is there something already out there that does what I need? I saw 
> the set_metadata.py script, but that looks like it is meant to operate 
> directly on the files, which might not make a running instance of Galaxy too 
> happy. Is there an API version of that (or something in one of the web 
> controllers I can repurpose to my needs)?

Hi John,

Galaxy will still set its own required metadata elements (determined by
the file type) on upload, but if you create additional optional metadata
elements and don't set them in your datatype's set_meta(), you could
then set them after the job runs with a call to the API.  Editing
metadata via the API is not yet implemented, but shouldn't be too
difficult to add.

This does mean you'd have to get the newly created dataset ID and make
second call to the API.  If that's not feasible it could probably be
incorporated at creation time, but would be trickier since the upload
code doesn't account for the possibility that metadata exists prior to
running the upload tool itself.

--nate

> 
> Thanks!
> 
> 
> John Duddy
> Illumina, Inc.
> 
> 

> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>   http://lists.bx.psu.edu/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Reply via email to