Re: [CODE4LIB] Any examples of using OAI-ORE for aggregation?

2010-03-19 Thread Ross Singer
Joe, I'm not sure if this conforms to what you're talking about, but
have you seen the Library of Congress' OAI-ORE implementation for
Chronicling America?

http://chroniclingamerica.loc.gov/

http://chroniclingamerica.loc.gov/lccn/sn83030214.rdf

-Ross.

On Wed, Mar 10, 2010 at 1:44 PM, Joe Hourcle
onei...@grace.nascom.nasa.gov wrote:
 Most of the examples I've seen of OAI-ORE seem to assume that you're
 ultimately interested in only one object within the resource map --
 effectively, it's content negotiation.

 Has anyone ever played with using ORE to point at an aggregation, with the
 expectation that the user will be interested in all parts, and automatically
 download them?

 ...

 Let me give a concrete example:

        A user searches for some data ... we find (x) number of records
        that match their criteria, and they then weed the list down to 10
        files of interest.

        We then save this request as a Resource Map, as part of an OAIS
        order.  I then want to be able to hand this off to a browser /
        downloader / whatever to try to obtain the individual files.

 Currently, I have something that can take the request, and create a tarball
 on the fly, but we have the unfortunate situation when some of the data is
 near-line and/or has to be regenerated -- I'm trying to find a good way to
 effectively fork the request into multiple smaller request, some of which I
 can service now, and some for which I can return an HTTP 503 status (service
 unavailable) w/ a retry-after header.

 ...

 Has anyone ever tried doing something like this?  Should I even be looking
 at ORE, or is there something that better fits with what I'm trying to do?

 Thanks for any advice / insight you can give

 -Joe

 -
 Joe Hourcle
 Programmer/Analyst
 Solar Data Analysis Center



[CODE4LIB] Any examples of using OAI-ORE for aggregation?

2010-03-10 Thread Joe Hourcle
Most of the examples I've seen of OAI-ORE seem to assume that you're 
ultimately interested in only one object within the resource map -- 
effectively, it's content negotiation.


Has anyone ever played with using ORE to point at an aggregation, with the 
expectation that the user will be interested in all parts, and 
automatically download them?


...

Let me give a concrete example:

A user searches for some data ... we find (x) number of records
that match their criteria, and they then weed the list down to 10
files of interest.

We then save this request as a Resource Map, as part of an OAIS
order.  I then want to be able to hand this off to a browser /
downloader / whatever to try to obtain the individual files.

Currently, I have something that can take the request, and create a 
tarball on the fly, but we have the unfortunate situation when some of the 
data is near-line and/or has to be regenerated -- I'm trying to find a 
good way to effectively fork the request into multiple smaller request, 
some of which I can service now, and some for which I can return an HTTP 
503 status (service unavailable) w/ a retry-after header.


...

Has anyone ever tried doing something like this?  Should I even be looking 
at ORE, or is there something that better fits with what I'm trying to do?


Thanks for any advice / insight you can give

-Joe

-
Joe Hourcle
Programmer/Analyst
Solar Data Analysis Center