Duddy, John wrote:
> So, to summarize:
> 1 - I can upload files via the API now, which returns the id
> 2 - If I don't upload via directory paths, the file upload will not be
> instant, so I'll have to use a poller to find out when the file upload is done
> 3 - I can write a new API to set metadata on a dataset and call that when the
> file is done. If I don't, the only metadata file file will have is the stuff
> created during set_meta().
I haven't worked on the API since it was originally written, so I am not
sure whether direct upload via the API was ever written/tested. You can
upload via filesystem paths or the server directory methods, though.
None of the methods will be instant - once the upload request is made, a
dataset is created, but the upload will still need to run through the
upload tool and set required metadata.
3 is right.
> John Duddy
> Sr. Staff Software Engineer
> Illumina, Inc.
> 9885 Towne Centre Drive
> San Diego, CA 92121
> Tel: 858-736-3584
> E-mail: jdu...@illumina.com
> -----Original Message-----
> From: Nate Coraor [mailto:n...@bx.psu.edu]
> Sent: Friday, May 06, 2011 8:12 AM
> To: Duddy, John
> Cc: firstname.lastname@example.org
> Subject: Re: [galaxy-dev] Question on setting metadata on upload via API
> Duddy, John wrote:
> > I'm looking at extending the metadata fields for one of the supported file
> > types. The files can get VERY large, and since I'm creating those files,
> > I'd like to save as metadata some of the information I have on the contents.
> > Specifically, I'd like to tag the files with information about the sample
> > as well as the byte offsets in the file for every millionth record (to
> > facilitate fast file splitting). I saw the sample script for uploading a
> > file to a library via the API. It looks to me like the most direct approach
> > would be to modify LibrariesController.create() to handle arbitrary
> > key-value pairs to be set on the data file.
> > However, that might not support things like having metadata values be
> > arrays.
> > So, does my understanding of the problem and the areas involved seem
> > reasonable? Is there something already out there that does what I need? I
> > saw the set_metadata.py script, but that looks like it is meant to operate
> > directly on the files, which might not make a running instance of Galaxy
> > too happy. Is there an API version of that (or something in one of the web
> > controllers I can repurpose to my needs)?
> Hi John,
> Galaxy will still set its own required metadata elements (determined by
> the file type) on upload, but if you create additional optional metadata
> elements and don't set them in your datatype's set_meta(), you could
> then set them after the job runs with a call to the API. Editing
> metadata via the API is not yet implemented, but shouldn't be too
> difficult to add.
> This does mean you'd have to get the newly created dataset ID and make
> second call to the API. If that's not feasible it could probably be
> incorporated at creation time, but would be trickier since the upload
> code doesn't account for the possibility that metadata exists prior to
> running the upload tool itself.
> > Thanks!
> > John Duddy
> > Illumina, Inc.
> > ___________________________________________________________
> > Please keep all replies on the list by using "reply all"
> > in your mail client. To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> > http://lists.bx.psu.edu/
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at: