Re: [CODE4LIB] OASIS SRU and CQL, access to most-current drafts
Hi,
Does the current draft include any support for asynchronous operation
of the protocol (either by status notifications and polling and/or
streaming), e.g some chunk of results coming back before others?
Sometime ago I read through an early draft published on the LOC site
and it mentioned support for federated search but it's hard to imagine
how could that be implemented without any async support.
On Tue, May 18, 2010 at 12:39 AM, Jonathan Rochkind rochk...@jhu.edu wrote:
Cool, we're on the same page.
This means that the asset parameter not only does not need to be, but
should NOT be, an actual parameterized value in the OpenSearch desc URL
template, in contrast to Tony's current SRU-in-opensearch-extension draft
spec.
At least that's my argument. Seems pretty clear to me.
Make sense:
Url type=application/rss+xml
template=http://my-sru-server.com?q={searchTerms}amp;accept=application%2Frss/
Does NOT make sense:
Url type=application/rss+xml
template=http://my-sru-server.com?q={searchTerms}amp;accept={sru:accept}/
[that parameter conflicts with the type in the Url template, does not make
sense to be parameterized in the OpenSearch URL template. Whether it's
parameterized to your SRU server is of no concern of OpenSearch, but to the
extent it's in the URL, it has to be FIXED in the OpenSearch URL template.
Is my argument. ]
Ray Denenberg, Library of Congress wrote:
Rather, OpenSearch descriptions provide a _different URL template_ for
every response IANA content type available
Yes, of course, that's what I meant, I said it somewhat slopily, but in
many of the examples we've looked at, it comes down to the same thing, that
the different templates differ only in a single (hard-coded) parameter.
--Ray
- Original Message - From: Jonathan Rochkind rochk...@jhu.edu
To: CODE4LIB@LISTSERV.ND.EDU
Sent: Monday, May 17, 2010 6:11 PM
Subject: Re: [CODE4LIB] OASIS SRU and CQL, access to most-current drafts
Ray Denenberg, Library of Congress wrote:
Ralph will probably be able to articulate this better than I can, but
the
accept parameter is driven by the requirement to be able to use
OpenSearch
(for example) to query an SRU server. The description document isn't
going
to provide templates that allow you to do this via content negotiation,
they
provide a parameter instead, to allow the client to tell the server that
it
wants, for example, an rss response.
No, they don't. I am having this same debate with Tony Hammond.
OpenSearch descriptions do NOT provide a parameter to allow the client to
tell the server what response it wants. They also don't easily provide for
content negotiation, it is true.
Rather, OpenSearch descriptions provide a _different URL template_ for
every response IANA content type available. Here is the example from the
OpenSearch documentation:
Url type=application/rss+xml
xmlns:example=http://example.com/opensearchextensions/1.0/;
template=http://example.com?q={searchTerms}amp;c={example:color?}/
Please note that application/rss+xml is an attribute of the URL template
itself, it is NOT a parameter in the template.
If SRU added an accept parameter to try and make OpenSearch happy, this
is a big mistake, because it in fact _conflicts_ with OpenSearch desc -- to
make it available as an actual parameter in the OpenSearch URL template.
If on the other hand, you just want to hard-code it in though, that could
make some sense.
Url type=application/rss+xml
xmlns:example=http://example.com/opensearchextensions/1.0/;
template=http://my-sru-server.com?q={searchTerms}amp;accept=application%2Frss/
That might make sense, if that's the use case. But actually trying to
provide a parameter for the _client_ to fill out in an OpenSearch desc in
fact _conflicts_ with OpenSearch, for better or for worse the respones type
is _hard coded_ into the template.
Jonathan
(I suggest, though, that you move further discussion of this to the SRU
list.)
--Ray
-Original Message-
From: Code for Libraries [mailto:code4...@listserv.nd.edu] On Behalf Of
Robert Sanderson
Sent: Monday, May 17, 2010 3:44 PM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: Re: [CODE4LIB] OASIS SRU and CQL, access to most-current drafts
In today's RESTful world, what's the requirement for the httpAccept
parameter? Isn't straight content negotiation sufficient rather than
pulling the headers into the URI?
What happens if the accept header and the httpAccept parameter say
different
things?
Rob
On Mon, May 17, 2010 at 1:37 PM, LeVan,Ralph le...@oclc.org wrote:
I'd code it. (I have already coded to it.) For me, the httpAccept
parameter and support for content negotiation on responses is a wonderful
addition to the standard. It lets us be OpenSearch compliant finally.
The virtue of coding to the draft is that there's a chance we can fix
any problems you encounter. While we consider the draft stable, that
doesn't
Re: [CODE4LIB] OASIS SRU and CQL, access to most-current drafts
There is no synchronous operation in SRU. As for federated search . To digress a moment, you may recall -- I believe it was on this list -- there was discussion (maybe a year ago?) of what that even means and whether it is the same or differs from metasearch, whatever that means. That discussion was inconclusive. Anyway, earlier drafts of SRU 2.0 describe a metasearch model. Recently, the committee decided that the terms metasearch and federated search are undefined jargon. We now choose to call it multi-server search. So to answer the question of whether there is federated search support: yes, limited support, if by federated search you mean multi server search. There is no multi-server support in terms of separate result sets for different servers. However, for (1) faceted search results, and (2) subquery results, these can be grouped according to server. --Ray -Original Message- From: Code for Libraries [mailto:code4...@listserv.nd.edu] On Behalf Of Kuba Sent: Tuesday, May 18, 2010 4:07 AM To: CODE4LIB@LISTSERV.ND.EDU Subject: Re: [CODE4LIB] OASIS SRU and CQL, access to most-current drafts Hi, Does the current draft include any support for asynchronous operation of the protocol (either by status notifications and polling and/or streaming), e.g some chunk of results coming back before others? Sometime ago I read through an early draft published on the LOC site and it mentioned support for federated search but it's hard to imagine how could that be implemented without any async support.
Re: [CODE4LIB] OASIS SRU and CQL, access to most-current drafts
On 18 May 2010 15:24, Ray Denenberg, Library of Congress r...@loc.gov wrote: There is no synchronous operation in SRU. As for federated search . To digress a moment, you may recall -- I believe it was on this list -- there was discussion (maybe a year ago?) of what that even means and whether it is the same or differs from metasearch, whatever that means. That discussion was inconclusive. Anyway, earlier drafts of SRU 2.0 describe a metasearch model. Recently, the committee decided that the terms metasearch and federated search are undefined jargon. We now choose to call it multi-server search. Way to go. Introducting yet ANOTHER synonym can only help! (And don't forget broadcast search.)
Re: [CODE4LIB] OASIS SRU and CQL, access to most-current drafts
What terms do you suggest, Mike? I think we're doomed no matter what with these, after certain communities started to use federated search and metasearch in directly opposite ways. I also was told recently that what is called an accordion in English is called a bandoneon in Spanish, and what is called a accordeon in Spanish is called a bandoneon in English. Hope this helps. Mike Taylor wrote: On 18 May 2010 15:24, Ray Denenberg, Library of Congress r...@loc.gov wrote: There is no synchronous operation in SRU. As for federated search . To digress a moment, you may recall -- I believe it was on this list -- there was discussion (maybe a year ago?) of what that even means and whether it is the same or differs from metasearch, whatever that means. That discussion was inconclusive. Anyway, earlier drafts of SRU 2.0 describe a metasearch model. Recently, the committee decided that the terms metasearch and federated search are undefined jargon. We now choose to call it multi-server search. Way to go. Introducting yet ANOTHER synonym can only help! (And don't forget broadcast search.)
Re: [CODE4LIB] OASIS SRU and CQL, access to most-current drafts
On 18 May 2010 15:24, Ray Denenberg, Library of Congress r...@loc.gov wrote: There is no synchronous operation in SRU. Sorry, meant to say no asynchronous . --Ray
Re: [CODE4LIB] OASIS SRU and CQL, access to most-current drafts
Mike Taylor wrote: What communities? All I know is we here on this very list had some people _insisting_ that federated search _really_ meant aggregated index, and meta search _really_ meant broadcast search, and other people insisting the opposite. Both sides had citations to the literature. I'm uninterested in arguing about what words really mean. Clearly these words have been used both ways in the profession and the literature. So I don't see any way around it by trying to find other words that attempt be harder to use more than one way. As you can see, I like broadcast search and aggregated index. Maybe we on the CODE4LIB list collectively carry enough weight that we could take the most prevalent meanings and propagate them? Ha, ha. Good luck with that. What we CAN do is spend a couple months arguing about what the words REALLY mean on the list (we're good at that), and then vote, and then have everyone else ignore the vote anyway. Jonathan
Re: [CODE4LIB] OASIS SRU and CQL, access to most-current drafts
On 18 May 2010 16:29, Jonathan Rochkind rochk...@jhu.edu wrote: Mike Taylor wrote: What communities? All I know is we here on this very list had some people _insisting_ that federated search _really_ meant aggregated index, and meta search _really_ meant broadcast search, and other people insisting the opposite. Both sides had citations to the literature. I'm uninterested in arguing about what words really mean. Clearly these words have been used both ways in the profession and the literature. So I don't see any way around it by trying to find other words that attempt be harder to use more than one way. As you can see, I like broadcast search and aggregated index. Maybe we on the CODE4LIB list collectively carry enough weight that we could take the most prevalent meanings and propagate them? Ha, ha. Good luck with that. What we CAN do is spend a couple months arguing about what the words REALLY mean on the list (we're good at that), and then vote, and then have everyone else ignore the vote anyway. Ha, ha indeed. You are of course right. You're also right that broadcast search and aggregated index are both explicit terms not susceptible to misinterpretation. We should try to remember to use those instead. Sign me up! Jonathan
Re: [CODE4LIB] OASIS SRU and CQL, access to most-current drafts
That is quite unfortunate, as we were looking at SRU 2.0 as a possible candidate for the front-end protocol for Index Data's pazpar2. The main problem with federate/broadcast/meta (however you want to call it ;) searching is that the back-end databases are scattered in different locations or simply slow in their response times and in order to provide decent user experience you need to be able to present some results sooner than others. Waiting for the slowest database to respond is usually not an option. On Tue, May 18, 2010 at 5:24 PM, Ray Denenberg, Library of Congress r...@loc.gov wrote: On 18 May 2010 15:24, Ray Denenberg, Library of Congress r...@loc.gov wrote: There is no synchronous operation in SRU. Sorry, meant to say no asynchronous . --Ray -- Cheers, Jakub
Re: [CODE4LIB] OASIS SRU and CQL, access to most-current drafts
What advantage do you see in having a concurrent operations feature (like Z39.50) versus opening several connections? (Concurrent operations introduced significant complexity into Z39.50 - including reference ids, operations, etc, and I'm not sure anyone ever really thought it was worth it.) --Ray -Original Message- From: Code for Libraries [mailto:code4...@listserv.nd.edu] On Behalf Of Kuba Sent: Tuesday, May 18, 2010 12:58 PM To: CODE4LIB@LISTSERV.ND.EDU Subject: Re: [CODE4LIB] OASIS SRU and CQL, access to most-current drafts That is quite unfortunate, as we were looking at SRU 2.0 as a possible candidate for the front-end protocol for Index Data's pazpar2. The main problem with federate/broadcast/meta (however you want to call it ;) searching is that the back-end databases are scattered in different locations or simply slow in their response times and in order to provide decent user experience you need to be able to present some results sooner than others. Waiting for the slowest database to respond is usually not an option. On Tue, May 18, 2010 at 5:24 PM, Ray Denenberg, Library of Congress r...@loc.gov wrote: On 18 May 2010 15:24, Ray Denenberg, Library of Congress r...@loc.gov wrote: There is no synchronous operation in SRU. Sorry, meant to say no asynchronous . --Ray -- Cheers, Jakub
Re: [CODE4LIB] OASIS SRU and CQL, access to most-current drafts
Jakub Skoczen wrote: I wonder if someone, like Kuba, could design an 'extended async SRU' on top of SRU, that is very SRU like, but builds on top of it to add just enough operations for Kuba's use case area. I think that's the right way to approach it. Is there a particular extensibility feature in the protocol that allows for this? I don't know, but that's not what I was suggesting. I was suggesting you read the SRU spec, and then design your own SRU-async spec, which is defined as exactly like SRU 2.0, except it also has the following operations, and is identified in an Explain document like X. Jonathan
Re: [CODE4LIB] OASIS SRU and CQL, access to most-current drafts
First, no. There are extensibility features in SRU but nothing that would help here. Actually, Jonathan, what I though you were suggesting was the creation of a (I hesitate to say it) metasearch engine. I use that term because it is what NISO called it, when they started their metasearch initiative five or so years ago, to create a standard for a metasearch engine, but they got distracted and the effort really came to nothing. The premise of the metasearch engine is that there exists a single-thread protocol, for example, SRU, and the need is to manage many threads, which is what the metasearch engine would have done if it had ever been defined. This is probably not an area for OASIS work, but if someone wanted to revive the effort in NISO (and put it on the right track) it could be useful. --Ray -Original Message- From: Code for Libraries [mailto:code4...@listserv.nd.edu] On Behalf Of Jonathan Rochkind Sent: Tuesday, May 18, 2010 2:56 PM To: CODE4LIB@LISTSERV.ND.EDU Subject: Re: [CODE4LIB] OASIS SRU and CQL, access to most-current drafts Jakub Skoczen wrote: I wonder if someone, like Kuba, could design an 'extended async SRU' on top of SRU, that is very SRU like, but builds on top of it to add just enough operations for Kuba's use case area. I think that's the right way to approach it. Is there a particular extensibility feature in the protocol that allows for this? I don't know, but that's not what I was suggesting. I was suggesting you read the SRU spec, and then design your own SRU-async spec, which is defined as exactly like SRU 2.0, except it also has the following operations, and is identified in an Explain document like X. Jonathan
Re: [CODE4LIB] OASIS SRU and CQL, access to most-current drafts
On Tue, May 18, 2010 at 9:17 PM, Ray Denenberg, Library of Congress r...@loc.gov wrote: First, no. There are extensibility features in SRU but nothing that would help here. Actually, Jonathan, what I though you were suggesting was the creation of a (I hesitate to say it) metasearch engine. I use that term because it is what NISO called it, when they started their metasearch initiative five or so years ago, to create a standard for a metasearch engine, but they got distracted and the effort really came to nothing. I'm not sure if Jonathan was suggesting that but that's exactly what I had in mind - using SRU 2.0 as a front-end protocol for a meta-search engine. And yes while creating a third-party, SRU-inspired protocol for that purpose could work, I see very little value in such exercise. I suspect that, as any standard, SRU has certain limitations and, as an implementer, you have to work around them but you do end up with an obvious gain: standards compliance. SRU-inspired protocol is not quite the same thing, and it's probably easier to go all the way and create a custom, proprietary protocol. The premise of the metasearch engine is that there exists a single-thread protocol, for example, SRU, and the need is to manage many threads, which is what the metasearch engine would have done if it had ever been defined. This is probably not an area for OASIS work, but if someone wanted to revive the effort in NISO (and put it on the right track) it could be useful. --Ray -Original Message- From: Code for Libraries [mailto:code4...@listserv.nd.edu] On Behalf Of Jonathan Rochkind Sent: Tuesday, May 18, 2010 2:56 PM To: CODE4LIB@LISTSERV.ND.EDU Subject: Re: [CODE4LIB] OASIS SRU and CQL, access to most-current drafts Jakub Skoczen wrote: I wonder if someone, like Kuba, could design an 'extended async SRU' on top of SRU, that is very SRU like, but builds on top of it to add just enough operations for Kuba's use case area. I think that's the right way to approach it. Is there a particular extensibility feature in the protocol that allows for this? I don't know, but that's not what I was suggesting. I was suggesting you read the SRU spec, and then design your own SRU-async spec, which is defined as exactly like SRU 2.0, except it also has the following operations, and is identified in an Explain document like X. Jonathan -- Cheers, Jakub
