Re: [CODE4LIB] linked data question
On Feb 26, 2015, at 9:48 AM, Owen Stephens o...@ostephens.com wrote: I highly recommend Chapter 6 of the Linked Data book which details different design approaches for Linked Data applications - sections 6.3 (http://linkeddatabook.com/editions/1.0/#htoc84) summarises the approaches as: 1. Crawling Pattern 2. On-the-fly dereferencing pattern 3. Query federation pattern Generally my view would be that (1) and (2) are viable approaches for different applications, but that (3) is generally a bad idea (having been through federated search before!) And at the risk of sounding like a broken record, owen++ because the Linked Data book” is a REALLY good read!! [0] While it is computer science-y, it is also authoritative, easy-to-read, full of examples, and just plain makes a whole lot of sense. [0] linked data book - http://linkeddatabook.com/ — Eric M.
Re: [CODE4LIB] linked data question
I apologize to both lists for this observation. I don't mean to offend anyone,
and now it's clear to me that this will potentially do so. I don't plan on
commenting further. I do hold both new technologists and traditional
librarians in respect - I just may generalize too much in trying to describe to
myself where the viewpoints differ.
Cindy Harper
-Original Message-
From: Harper, Cynthia
Sent: Thursday, February 26, 2015 10:22 AM
To: CODE4LIB@LISTSERV.ND.EDU
Cc: 'AUTOCAT'
Subject: RE: [CODE4LIB] linked data question
So the issue being discussed on AUTOCAT was the availability/fault tolerance of
the database, given that it's spread over numerous remote systems, and I
suppose local caching and mirroring are the answers there.
The other issue was skepticism about the feasibility of indexing all these
remote sources, which led me to thinking about remote indexes, but I see the
answer is that that's why we won't be using single-site local systems so much,
but instead using Google-like web-scale indexes. That's putting pressure on
the old vision of the library catalog as our database.
Is that a fair understanding?
Cindy Harper
char...@vts.edu
-Original Message-
From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of Eric
Lease Morgan
Sent: Thursday, February 26, 2015 9:44 AM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: Re: [CODE4LIB] linked data question
On Feb 25, 2015, at 3:12 PM, Sarah Weissman seweiss...@gmail.com wrote:
I am kind of new to this linked data thing, but it seems like the real
power of it is not full-text search, but linking through the use of
shared vocabularies. So if you have data about Jane Austen in your
database and you are using the same URI as other databases to
represent Jane Austen in your data (say
http://dbpedia.org/resource/Jane_Austen), then you (or rather, your
software) can do an exact search on that URI in remote resources vs. a
fuzzy text search. In other words, linked data is really
^
supposed to be linked by machines and discoverable through URIs. If
you
visit the URL: http://dbpedia.org/page/Jane_Austen you can see a
human-interpretable representation of the data a SPARQL endpoint would
return for a query for triples {http://dbpedia.org/page/Jane_Austen ?p ?o}.
This is essentially asking the database for all
subject-predicate-object facts it contains where Jane Austen is the subject.
Again, seweissman++ The implementation of linked data is VERY much like the
implementation of a relational database over HTTP, and in such a scenario, the
URIs are the database keys. —ELM
Re: [CODE4LIB] linked data question
So the issue being discussed on AUTOCAT was the availability/fault tolerance of
the database, given that it's spread over numerous remote systems, and I
suppose local caching and mirroring are the answers there.
The other issue was skepticism about the feasibility of indexing all these
remote sources, which led me to thinking about remote indexes, but I see the
answer is that that's why we won't be using single-site local systems so much,
but instead using Google-like web-scale indexes. That's putting pressure on
the old vision of the library catalog as our database.
Is that a fair understanding?
Cindy Harper
char...@vts.edu
-Original Message-
From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of Eric
Lease Morgan
Sent: Thursday, February 26, 2015 9:44 AM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: Re: [CODE4LIB] linked data question
On Feb 25, 2015, at 3:12 PM, Sarah Weissman seweiss...@gmail.com wrote:
I am kind of new to this linked data thing, but it seems like the real
power of it is not full-text search, but linking through the use of
shared vocabularies. So if you have data about Jane Austen in your
database and you are using the same URI as other databases to
represent Jane Austen in your data (say
http://dbpedia.org/resource/Jane_Austen), then you (or rather, your
software) can do an exact search on that URI in remote resources vs. a
fuzzy text search. In other words, linked data is really
^
supposed to be linked by machines and discoverable through URIs. If
you
visit the URL: http://dbpedia.org/page/Jane_Austen you can see a
human-interpretable representation of the data a SPARQL endpoint would
return for a query for triples {http://dbpedia.org/page/Jane_Austen ?p ?o}.
This is essentially asking the database for all
subject-predicate-object facts it contains where Jane Austen is the subject.
Again, seweissman++ The implementation of linked data is VERY much like the
implementation of a relational database over HTTP, and in such a scenario, the
URIs are the database keys. —ELM
Re: [CODE4LIB] linked data question
I highly recommend Chapter 6 of the Linked Data book which details different design approaches for Linked Data applications - sections 6.3 (http://linkeddatabook.com/editions/1.0/#htoc84) summarises the approaches as: 1. Crawling Pattern 2. On-the-fly dereferencing pattern 3. Query federation pattern Generally my view would be that (1) and (2) are viable approaches for different applications, but that (3) is generally a bad idea (having been through federated search before!) Owen Owen Stephens Owen Stephens Consulting Web: http://www.ostephens.com Email: o...@ostephens.com Telephone: 0121 288 6936 On 26 Feb 2015, at 14:40, Eric Lease Morgan emor...@nd.edu wrote: On Feb 25, 2015, at 2:48 PM, Esmé Cowles escow...@ticklefish.org wrote: In the non-techie library world, linked data is being talked about (perhaps only in listserv traffic) as if the data (bibliographic data, for instance) will reside on remote sites (as a SPARQL endpoint??? We don't know the technical implications of that), and be displayed by your local catalog/the centralized inter-national catalog by calling data from that remote site. But the original question was how the data on those remote sites would be access points - how can I start my search by searching for that remote content? I assume there has to be a database implementation that visits that data and pre-indexes it for it to be searchable, and therefore the index has to be local (or global a la Google or OCLC or its bibliographic-linked-data equivalent). I think there are several options for how this works, and different applications may take different approaches. The most basic approach would be to just include the URIs in your local system and retrieve them any time you wanted to work with them. But the performance of that would be terrible, and your application would stop working if it couldn't retrieve the URIs. So there are lots of different approaches (which could be combined): - Retrieve the URIs the first time, and then cache them locally. - Download an entire data dump of the remote vocabulary and host it locally. - Add text fields in parallel to the URIs, so you at least have a label for it. - Index the data in Solr, Elasticsearch, etc. and use that most of the time, esp. for read-only operations. Yes, exactly. I believe Esmé has articulated the possible solutions well. escowles++ —ELM
Re: [CODE4LIB] linked data question
On Feb 25, 2015, at 3:12 PM, Sarah Weissman seweiss...@gmail.com wrote:
I am kind of new to this linked data thing, but it seems like the real
power of it is not full-text search, but linking through the use of shared
vocabularies. So if you have data about Jane Austen in your database and
you are using the same URI as other databases to represent Jane Austen in
your data (say http://dbpedia.org/resource/Jane_Austen), then you (or
rather, your software) can do an exact search on that URI in remote
resources vs. a fuzzy text search. In other words, linked data is really
^
supposed to be linked by machines and discoverable through URIs. If you
visit the URL: http://dbpedia.org/page/Jane_Austen you can see a
human-interpretable representation of the data a SPARQL endpoint would
return for a query for triples {http://dbpedia.org/page/Jane_Austen ?p ?o}.
This is essentially asking the database for all subject-predicate-object
facts it contains where Jane Austen is the subject.
Again, seweissman++ The implementation of linked data is VERY much like the
implementation of a relational database over HTTP, and in such a scenario, the
URIs are the database keys. —ELM
Re: [CODE4LIB] linked data question
On Feb 25, 2015, at 2:48 PM, Esmé Cowles escow...@ticklefish.org wrote: In the non-techie library world, linked data is being talked about (perhaps only in listserv traffic) as if the data (bibliographic data, for instance) will reside on remote sites (as a SPARQL endpoint??? We don't know the technical implications of that), and be displayed by your local catalog/the centralized inter-national catalog by calling data from that remote site. But the original question was how the data on those remote sites would be access points - how can I start my search by searching for that remote content? I assume there has to be a database implementation that visits that data and pre-indexes it for it to be searchable, and therefore the index has to be local (or global a la Google or OCLC or its bibliographic-linked-data equivalent). I think there are several options for how this works, and different applications may take different approaches. The most basic approach would be to just include the URIs in your local system and retrieve them any time you wanted to work with them. But the performance of that would be terrible, and your application would stop working if it couldn't retrieve the URIs. So there are lots of different approaches (which could be combined): - Retrieve the URIs the first time, and then cache them locally. - Download an entire data dump of the remote vocabulary and host it locally. - Add text fields in parallel to the URIs, so you at least have a label for it. - Index the data in Solr, Elasticsearch, etc. and use that most of the time, esp. for read-only operations. Yes, exactly. I believe Esmé has articulated the possible solutions well. escowles++ —ELM
Re: [CODE4LIB] linked data question
Well, that's my question. I have the micro view of linked data, I think - it's a distribution/self-describing format. But I don't see the big picture. In the non-techie library world, linked data is being talked about (perhaps only in listserv traffic) as if the data (bibliographic data, for instance) will reside on remote sites (as a SPARQL endpoint??? We don't know the technical implications of that), and be displayed by your local catalog/the centralized inter-national catalog by calling data from that remote site. But the original question was how the data on those remote sites would be access points - how can I start my search by searching for that remote content? I assume there has to be a database implementation that visits that data and pre-indexes it for it to be searchable, and therefore the index has to be local (or global a la Google or OCLC or its bibliographic-linked-data equivalent). All of the above parenthesized or bracketed concepts are nebulous to me. Cindy -Original Message- From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of Sarah Weissman Sent: Tuesday, February 24, 2015 11:02 PM To: CODE4LIB@LISTSERV.ND.EDU Subject: Re: [CODE4LIB] linked data question I think Code4libbers will know more about my question about distributed INDEXES? This is my rudimentary knowledge of linked data - that the indexing process will have to transit the links, and build a local index to the data, even if in displaying the individual records, it goes again out to the source. But are there examples of distributed systems that have distributed INDEXES? Or Am I wrong in envisioning an index as a separate entity from the data in today's technology? I'm a little confused by what you mean by distributed index in a linked data context. I assume an index would have to be database implementation specific, while data is typically exposed for external consumption via implementation-agnostic protocols/formats, like a SPARQL endpoint or a REST API. How do you locally index something remote under these constraints? -Sarah Cindy Harper -Original Message- From: Harper, Cynthia Sent: Tuesday, February 24, 2015 1:20 PM To: auto...@listserv.syr.edu; 'Williams, Ann' Subject: RE: linked data question What I haven't read, but what I have wondered about, is whether so far, linked DATA is distributed, but the INDEXES are local? Is there any example of a system with distributed INDEXES? Cindy Harper char...@vts.edu -Original Message- From: AUTOCAT [mailto:auto...@listserv.syr.edu] On Behalf Of Williams, Ann Sent: Tuesday, February 24, 2015 10:26 AM To: auto...@listserv.syr.edu Subject: [ACAT] linked data question I was just wondering how linked data will affect OPAC searching and discovery vs. a record with text approach. For example, we have various 856 links to publisher, summary and biographical information in our OPAC as well as ISBNs linking to ContentCafe. But none of that content is discoverable in the OPAC and it requires a further click on the part of patrons (many of whom won't click). Ann Williams USJ -- ** * AUTOCAT quoting guide: http://www.cwu.edu/~dcc/Autocat/copyright.html E-mail AUTOCAT listowners: autocat-requ...@listserv.syr.edu Search AUTOCAT archives: http://listserv.syr.edu/archives/autocat.html By posting messages to AUTOCAT, the author does not cede copyright ** *
Re: [CODE4LIB] linked data question
Cindy- I think there are several options for how this works, and different applications may take different approaches. The most basic approach would be to just include the URIs in your local system and retrieve them any time you wanted to work with them. But the performance of that would be terrible, and your application would stop working if it couldn't retrieve the URIs. So there are lots of different approaches (which could be combined): - Retrieve the URIs the first time, and then cache them locally. - Download an entire data dump of the remote vocabulary and host it locally. - Add text fields in parallel to the URIs, so you at least have a label for it. - Index the data in Solr, Elasticsearch, etc. and use that most of the time, esp. for read-only operations. -Esme On 02/25/15, at 2:30 PM, Harper, Cynthia char...@vts.edu wrote: Well, that's my question. I have the micro view of linked data, I think - it's a distribution/self-describing format. But I don't see the big picture. In the non-techie library world, linked data is being talked about (perhaps only in listserv traffic) as if the data (bibliographic data, for instance) will reside on remote sites (as a SPARQL endpoint??? We don't know the technical implications of that), and be displayed by your local catalog/the centralized inter-national catalog by calling data from that remote site. But the original question was how the data on those remote sites would be access points - how can I start my search by searching for that remote content? I assume there has to be a database implementation that visits that data and pre-indexes it for it to be searchable, and therefore the index has to be local (or global a la Google or OCLC or its bibliographic-linked-data equivalent). All of the above parenthesized or bracketed concepts are nebulous to me. Cindy -Original Message- From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of Sarah Weissman Sent: Tuesday, February 24, 2015 11:02 PM To: CODE4LIB@LISTSERV.ND.EDU Subject: Re: [CODE4LIB] linked data question I think Code4libbers will know more about my question about distributed INDEXES? This is my rudimentary knowledge of linked data - that the indexing process will have to transit the links, and build a local index to the data, even if in displaying the individual records, it goes again out to the source. But are there examples of distributed systems that have distributed INDEXES? Or Am I wrong in envisioning an index as a separate entity from the data in today's technology? I'm a little confused by what you mean by distributed index in a linked data context. I assume an index would have to be database implementation specific, while data is typically exposed for external consumption via implementation-agnostic protocols/formats, like a SPARQL endpoint or a REST API. How do you locally index something remote under these constraints? -Sarah Cindy Harper -Original Message- From: Harper, Cynthia Sent: Tuesday, February 24, 2015 1:20 PM To: auto...@listserv.syr.edu; 'Williams, Ann' Subject: RE: linked data question What I haven't read, but what I have wondered about, is whether so far, linked DATA is distributed, but the INDEXES are local? Is there any example of a system with distributed INDEXES? Cindy Harper char...@vts.edu -Original Message- From: AUTOCAT [mailto:auto...@listserv.syr.edu] On Behalf Of Williams, Ann Sent: Tuesday, February 24, 2015 10:26 AM To: auto...@listserv.syr.edu Subject: [ACAT] linked data question I was just wondering how linked data will affect OPAC searching and discovery vs. a record with text approach. For example, we have various 856 links to publisher, summary and biographical information in our OPAC as well as ISBNs linking to ContentCafe. But none of that content is discoverable in the OPAC and it requires a further click on the part of patrons (many of whom won't click). Ann Williams USJ -- ** * AUTOCAT quoting guide: http://www.cwu.edu/~dcc/Autocat/copyright.html E-mail AUTOCAT listowners: autocat-requ...@listserv.syr.edu Search AUTOCAT archives: http://listserv.syr.edu/archives/autocat.html By posting messages to AUTOCAT, the author does not cede copyright ** *
Re: [CODE4LIB] linked data question
I am kind of new to this linked data thing, but it seems like the real
power of it is not full-text search, but linking through the use of shared
vocabularies. So if you have data about Jane Austen in your database and
you are using the same URI as other databases to represent Jane Austen in
your data (say http://dbpedia.org/resource/Jane_Austen), then you (or
rather, your software) can do an exact search on that URI in remote
resources vs. a fuzzy text search. In other words, linked data is really
supposed to be linked by machines and discoverable through URIs. If you
visit the URL: http://dbpedia.org/page/Jane_Austen you can see a
human-interpretable representation of the data a SPARQL endpoint would
return for a query for triples {http://dbpedia.org/page/Jane_Austen ?p ?o}.
This is essentially asking the database for all subject-predicate-object
facts it contains where Jane Austen is the subject. (Sorry if this is stuff
you already know.)
So yeah, to get full text search, I think you'd need to both cache and
index the data locally. I believe most triplestore implementations index on
subject and object URIs to make lookups like the one mentioned above
relatively efficient, but most would not have efficient full text search
unless through some external indexing application like Solr.
On Wed, Feb 25, 2015 at 2:30 PM, Harper, Cynthia char...@vts.edu wrote:
Well, that's my question. I have the micro view of linked data, I think -
it's a distribution/self-describing format. But I don't see the big picture.
In the non-techie library world, linked data is being talked about
(perhaps only in listserv traffic) as if the data (bibliographic data, for
instance) will reside on remote sites (as a SPARQL endpoint??? We don't
know the technical implications of that), and be displayed by your local
catalog/the centralized inter-national catalog by calling data from that
remote site. But the original question was how the data on those remote
sites would be access points - how can I start my search by searching for
that remote content? I assume there has to be a database implementation
that visits that data and pre-indexes it for it to be searchable, and
therefore the index has to be local (or global a la Google or OCLC or its
bibliographic-linked-data equivalent).
All of the above parenthesized or bracketed concepts are nebulous to me.
Cindy
-Original Message-
From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of
Sarah Weissman
Sent: Tuesday, February 24, 2015 11:02 PM
To: CODE4LIB@LISTSERV.ND.EDU
Subject: Re: [CODE4LIB] linked data question
I think Code4libbers will know more about my question about
distributed INDEXES? This is my rudimentary knowledge of linked data
- that the indexing process will have to transit the links, and build
a local index to the data, even if in displaying the individual
records, it goes again out to the source. But are there examples of
distributed systems that have distributed INDEXES? Or Am I wrong in
envisioning an index as a separate entity from the data in today's
technology?
I'm a little confused by what you mean by distributed index in a linked
data context. I assume an index would have to be database implementation
specific, while data is typically exposed for external consumption via
implementation-agnostic protocols/formats, like a SPARQL endpoint or a REST
API. How do you locally index something remote under these constraints?
-Sarah
Cindy Harper
-Original Message-
From: Harper, Cynthia
Sent: Tuesday, February 24, 2015 1:20 PM
To: auto...@listserv.syr.edu; 'Williams, Ann'
Subject: RE: linked data question
What I haven't read, but what I have wondered about, is whether so
far, linked DATA is distributed, but the INDEXES are local? Is there
any example of a system with distributed INDEXES?
Cindy Harper
char...@vts.edu
-Original Message-
From: AUTOCAT [mailto:auto...@listserv.syr.edu] On Behalf Of Williams,
Ann
Sent: Tuesday, February 24, 2015 10:26 AM
To: auto...@listserv.syr.edu
Subject: [ACAT] linked data question
I was just wondering how linked data will affect OPAC searching and
discovery vs. a record with text approach. For example, we have
various 856 links to publisher, summary and biographical information
in our OPAC as well as ISBNs linking to ContentCafe. But none of that
content is discoverable in the OPAC and it requires a further click on
the part of patrons (many of whom won't click).
Ann Williams
USJ
--
**
*
AUTOCAT quoting guide: http://www.cwu.edu/~dcc/Autocat/copyright.html
E-mail AUTOCAT listowners: autocat-requ...@listserv.syr.edu
Search AUTOCAT archives: http://listserv.syr.edu/archives/autocat.html
By posting messages to AUTOCAT, the author does not cede copyright
**
*
Re: [CODE4LIB] linked data question
Yes, I would expect each organization to fetch linked data resources and maintain their own local indexes, and probably also cache the remote resources to make it easier and faster to work with them. I've heard discussions of caching strategies, shared indexing tools, etc., but haven't heard about anyone distributing pre-indexed content. Many vocabularies are available as RDF data dumps, which can sometimes be very large and unwieldy. So I could imagine being able to download, e.g., a Solr index of the vocabulary instead of having to index it yourself. But I haven't heard of anybody doing that. -Esme On 02/24/15, at 10:56 AM, Harper, Cynthia char...@vts.edu wrote: Ann - I thought I'd refer part of your question to Code4lib. As far as having to click to get the linked data: systems that use linked data will be built to transit the link without the user being aware - it's the system that will follow that link and find the distributed data, then display it as it is programmed to do so. I think Code4libbers will know more about my question about distributed INDEXES? This is my rudimentary knowledge of linked data - that the indexing process will have to transit the links, and build a local index to the data, even if in displaying the individual records, it goes again out to the source. But are there examples of distributed systems that have distributed INDEXES? Or Am I wrong in envisioning an index as a separate entity from the data in today's technology? Cindy Harper -Original Message- From: Harper, Cynthia Sent: Tuesday, February 24, 2015 1:20 PM To: auto...@listserv.syr.edu; 'Williams, Ann' Subject: RE: linked data question What I haven't read, but what I have wondered about, is whether so far, linked DATA is distributed, but the INDEXES are local? Is there any example of a system with distributed INDEXES? Cindy Harper char...@vts.edu -Original Message- From: AUTOCAT [mailto:auto...@listserv.syr.edu] On Behalf Of Williams, Ann Sent: Tuesday, February 24, 2015 10:26 AM To: auto...@listserv.syr.edu Subject: [ACAT] linked data question I was just wondering how linked data will affect OPAC searching and discovery vs. a record with text approach. For example, we have various 856 links to publisher, summary and biographical information in our OPAC as well as ISBNs linking to ContentCafe. But none of that content is discoverable in the OPAC and it requires a further click on the part of patrons (many of whom won't click). Ann Williams USJ -- *** AUTOCAT quoting guide: http://www.cwu.edu/~dcc/Autocat/copyright.html E-mail AUTOCAT listowners: autocat-requ...@listserv.syr.edu Search AUTOCAT archives: http://listserv.syr.edu/archives/autocat.html By posting messages to AUTOCAT, the author does not cede copyright ***
Re: [CODE4LIB] [!!Mass Mail]Re: [CODE4LIB] linked data question
-Original Message- From: Code for Libraries [mailto:CODE4LIB@LISTSERV.ND.EDU] On Behalf Of Esmé Cowles Sent: Tuesday, February 24, 2015 12:10 PM To: CODE4LIB@LISTSERV.ND.EDU Subject: [!!Mass Mail]Re: [CODE4LIB] linked data question Yes, I would expect each organization to fetch linked data resources and maintain their own local indexes, and probably also cache the remote resources to make it easier and faster to work with them. I've heard discussions of caching strategies, shared indexing tools, etc., but haven't heard about anyone distributing pre-indexed content. Many vocabularies are available as RDF data dumps, which can sometimes be very large and unwieldy. So I could imagine being able to download, e.g., a Solr index of the vocabulary instead of having to index it yourself. But I haven't heard of anybody doing that. -Esme On 02/24/15, at 10:56 AM, Harper, Cynthia char...@vts.edu wrote: Ann - I thought I'd refer part of your question to Code4lib. As far as having to click to get the linked data: systems that use linked data will be built to transit the link without the user being aware - it's the system that will follow that link and find the distributed data, then display it as it is programmed to do so. I think Code4libbers will know more about my question about distributed INDEXES? This is my rudimentary knowledge of linked data - that the indexing process will have to transit the links, and build a local index to the data, even if in displaying the individual records, it goes again out to the source. But are there examples of distributed systems that have distributed INDEXES? Or Am I wrong in envisioning an index as a separate entity from the data in today's technology? Cindy Harper -Original Message- From: Harper, Cynthia Sent: Tuesday, February 24, 2015 1:20 PM To: auto...@listserv.syr.edu; 'Williams, Ann' Subject: RE: linked data question What I haven't read, but what I have wondered about, is whether so far, linked DATA is distributed, but the INDEXES are local? Is there any example of a system with distributed INDEXES? Cindy Harper char...@vts.edu -Original Message- From: AUTOCAT [mailto:auto...@listserv.syr.edu] On Behalf Of Williams, Ann Sent: Tuesday, February 24, 2015 10:26 AM To: auto...@listserv.syr.edu Subject: [ACAT] linked data question I was just wondering how linked data will affect OPAC searching and discovery vs. a record with text approach. For example, we have various 856 links to publisher, summary and biographical information in our OPAC as well as ISBNs linking to ContentCafe. But none of that content is discoverable in the OPAC and it requires a further click on the part of patrons (many of whom won't click). Ann Williams USJ -- *** AUTOCAT quoting guide: http://www.cwu.edu/~dcc/Autocat/copyright.html E-mail AUTOCAT listowners: autocat-requ...@listserv.syr.edu Search AUTOCAT archives: http://listserv.syr.edu/archives/autocat.html By posting messages to AUTOCAT, the author does not cede copyright ***
Re: [CODE4LIB] linked data question
There are a few issue here that might need to be parsed out. The first is indexing Linked Data. It seems to make sense from a performance perspective to have a local index for the URIs and their names. For example http://viaf.org/viaf/102333412 name: Austen, Jane Pretend 'name' is an index field and the URI is the index key or ID. If you are using a Lucene index, you can imagine having multiple names based on language variation, preferred label variation (i.e. 'Austen, Jane, 1775-1817') etc. Another issue has to do with what is cached in the index. I would argue that nothing other then the lookup values should be cached. The system should go off the the key/ID (i.e. the URI) and fetch the data from it. This is important because data can change all the time and you do not want to rely on having to download monthly data dumps to rebuild your index. Plus the idea of data dumps stands in opposition to the idea of Linked Data and the Web (i.e. its on the Web for a reason and that is to be accessed on the Web not downloaded and stored in a silo). The third issue has to do with using VIAF URIs or coining your own local URIs. This is a bit of a toss up but I would argue that it would be better if you could coin your own URI and simply use a sameAs link to other entities, such as VIAF, LCSH, FAST etc. This would allow you to have a localized world-view of the entity. Or, to explain it better, it would allow you to put a localized lens on the entity and show things like how does this entity relate to other things that I have, know about, vend to patrons, etc. There are also practice reasons for this. If I see a hot-link in my local Library OPAC for 'Jane Austen' I expect to stay within my local OPAC domain when I click on it. I do not want to be taken out to VIAF or another place. The reason for clicking it is to learn about it within the context of what I am doing on that website. Finally, coining your own URI allows you provide people with a bookmark-able URL. That is important for search engine visibility. The last issue would require the index example above to not have a VIAF URI but rather a local URI that could be retrieved from a local Triple Store. In the store you could provide sameAs links to VIAF as well as localized information about the entity such as what he/she has authored that you current have available. Thanks, Jeff Mixter Research Support Specialist OCLC Research 614-761-5159 mixt...@oclc.org From: Code for Libraries CODE4LIB@LISTSERV.ND.EDU on behalf of Esmé Cowles escow...@ticklefish.org Sent: Tuesday, February 24, 2015 3:09 PM To: CODE4LIB@LISTSERV.ND.EDU Subject: Re: [CODE4LIB] linked data question Yes, I would expect each organization to fetch linked data resources and maintain their own local indexes, and probably also cache the remote resources to make it easier and faster to work with them. I've heard discussions of caching strategies, shared indexing tools, etc., but haven't heard about anyone distributing pre-indexed content. Many vocabularies are available as RDF data dumps, which can sometimes be very large and unwieldy. So I could imagine being able to download, e.g., a Solr index of the vocabulary instead of having to index it yourself. But I haven't heard of anybody doing that. -Esme On 02/24/15, at 10:56 AM, Harper, Cynthia char...@vts.edu wrote: Ann - I thought I'd refer part of your question to Code4lib. As far as having to click to get the linked data: systems that use linked data will be built to transit the link without the user being aware - it's the system that will follow that link and find the distributed data, then display it as it is programmed to do so. I think Code4libbers will know more about my question about distributed INDEXES? This is my rudimentary knowledge of linked data - that the indexing process will have to transit the links, and build a local index to the data, even if in displaying the individual records, it goes again out to the source. But are there examples of distributed systems that have distributed INDEXES? Or Am I wrong in envisioning an index as a separate entity from the data in today's technology? Cindy Harper -Original Message- From: Harper, Cynthia Sent: Tuesday, February 24, 2015 1:20 PM To: auto...@listserv.syr.edu; 'Williams, Ann' Subject: RE: linked data question What I haven't read, but what I have wondered about, is whether so far, linked DATA is distributed, but the INDEXES are local? Is there any example of a system with distributed INDEXES? Cindy Harper char...@vts.edu -Original Message- From: AUTOCAT [mailto:auto...@listserv.syr.edu] On Behalf Of Williams, Ann Sent: Tuesday, February 24, 2015 10:26 AM To: auto...@listserv.syr.edu Subject: [ACAT] linked data question I was just wondering how linked data will affect OPAC searching and discovery vs. a record
Re: [CODE4LIB] linked data question
Ann - I thought I'd refer part of your question to Code4lib. As far as having to click to get the linked data: systems that use linked data will be built to transit the link without the user being aware - it's the system that will follow that link and find the distributed data, then display it as it is programmed to do so. I think Code4libbers will know more about my question about distributed INDEXES? This is my rudimentary knowledge of linked data - that the indexing process will have to transit the links, and build a local index to the data, even if in displaying the individual records, it goes again out to the source. But are there examples of distributed systems that have distributed INDEXES? Or Am I wrong in envisioning an index as a separate entity from the data in today's technology? Cindy Harper -Original Message- From: Harper, Cynthia Sent: Tuesday, February 24, 2015 1:20 PM To: auto...@listserv.syr.edu; 'Williams, Ann' Subject: RE: linked data question What I haven't read, but what I have wondered about, is whether so far, linked DATA is distributed, but the INDEXES are local? Is there any example of a system with distributed INDEXES? Cindy Harper char...@vts.edu -Original Message- From: AUTOCAT [mailto:auto...@listserv.syr.edu] On Behalf Of Williams, Ann Sent: Tuesday, February 24, 2015 10:26 AM To: auto...@listserv.syr.edu Subject: [ACAT] linked data question I was just wondering how linked data will affect OPAC searching and discovery vs. a record with text approach. For example, we have various 856 links to publisher, summary and biographical information in our OPAC as well as ISBNs linking to ContentCafe. But none of that content is discoverable in the OPAC and it requires a further click on the part of patrons (many of whom won't click). Ann Williams USJ -- *** AUTOCAT quoting guide: http://www.cwu.edu/~dcc/Autocat/copyright.html E-mail AUTOCAT listowners: autocat-requ...@listserv.syr.edu Search AUTOCAT archives: http://listserv.syr.edu/archives/autocat.html By posting messages to AUTOCAT, the author does not cede copyright ***
Re: [CODE4LIB] linked data question
We have taken a somewhat different approach to how we manage our RDF data: after years of using a native triple store, we found that it was actually extremely impractical for the way we actually used our data. Triple stores are fine for ad-hoc queries over arbitrary data, but that didn't reflect our usage. Our schema, while flexible, was still predefined and our joins were always on the same properties. We found it made a lot more sense to store all of our data about a given subject (concise bounded descriptions) as a document in a document-style database (we use MongoDB). This took care of our SPARQL DESCRIBE queries. However, since majority of the data we use are the equivalent of a SPARQL CONSTRUCT made up of fixed joins over these CBD graphs, we decided to cache these joined documents as their own documents in a read-only cache collection, which we refer to as views. Then, if any of the CBD graphs change, we invalidate the view and rebuild the cache documents. I think this usage pattern would also pretty closely reflect how a linked library data system might work, as well. The data doesn't actually change all that much, and there will likely be very few ad hoc queries. This also addresses a problem in the suggestion that Jeff made regarding using your own URIs with 3rd party data: it gets pretty complicated to manage changes to the graph in this scenario. By storing the original CBDs as is, and generating graphs with your URIs over the external data as a view, it's far easier to isolate what gets changed with particular updates (not to mention that large data updates from various sources in a native triple store is painful). You can take a look at what we wrote if you want more details: https://github.com/talis/tripod-php This still doesn't address how to deal with keeping your local copy of the external data up to date, and I don't know that there are a lot of good or standard answers to that yet. That said, I think it's a solvable problem: we just haven't gotten to that scale yet. -Ross. On Tuesday, February 24, 2015, Mixter,Jeff mixt...@oclc.org wrote: There are a few issue here that might need to be parsed out. The first is indexing Linked Data. It seems to make sense from a performance perspective to have a local index for the URIs and their names. For example http://viaf.org/viaf/102333412 name: Austen, Jane Pretend 'name' is an index field and the URI is the index key or ID. If you are using a Lucene index, you can imagine having multiple names based on language variation, preferred label variation (i.e. 'Austen, Jane, 1775-1817') etc. Another issue has to do with what is cached in the index. I would argue that nothing other then the lookup values should be cached. The system should go off the the key/ID (i.e. the URI) and fetch the data from it. This is important because data can change all the time and you do not want to rely on having to download monthly data dumps to rebuild your index. Plus the idea of data dumps stands in opposition to the idea of Linked Data and the Web (i.e. its on the Web for a reason and that is to be accessed on the Web not downloaded and stored in a silo). The third issue has to do with using VIAF URIs or coining your own local URIs. This is a bit of a toss up but I would argue that it would be better if you could coin your own URI and simply use a sameAs link to other entities, such as VIAF, LCSH, FAST etc. This would allow you to have a localized world-view of the entity. Or, to explain it better, it would allow you to put a localized lens on the entity and show things like how does this entity relate to other things that I have, know about, vend to patrons, etc. There are also practice reasons for this. If I see a hot-link in my local Library OPAC for 'Jane Austen' I expect to stay within my local OPAC domain when I click on it. I do not want to be taken out to VIAF or another place. The reason for clicking it is to learn about it within the context of what I am doing on that website. Finally, coining your own URI allows you provide people with a bookmark-able URL. That is important for search engine visibility. The last issue would require the index example above to not have a VIAF URI but rather a local URI that could be retrieved from a local Triple Store. In the store you could provide sameAs links to VIAF as well as localized information about the entity such as what he/she has authored that you current have available. Thanks, Jeff Mixter Research Support Specialist OCLC Research 614-761-5159 mixt...@oclc.org javascript:; From: Code for Libraries CODE4LIB@LISTSERV.ND.EDU javascript:; on behalf of Esmé Cowles escow...@ticklefish.org javascript:; Sent: Tuesday, February 24, 2015 3:09 PM To: CODE4LIB@LISTSERV.ND.EDU javascript:; Subject: Re: [CODE4LIB] linked data question Yes, I would expect each organization to fetch linked data resources
Re: [CODE4LIB] linked data question
I think Code4libbers will know more about my question about distributed INDEXES? This is my rudimentary knowledge of linked data - that the indexing process will have to transit the links, and build a local index to the data, even if in displaying the individual records, it goes again out to the source. But are there examples of distributed systems that have distributed INDEXES? Or Am I wrong in envisioning an index as a separate entity from the data in today's technology? I'm a little confused by what you mean by distributed index in a linked data context. I assume an index would have to be database implementation specific, while data is typically exposed for external consumption via implementation-agnostic protocols/formats, like a SPARQL endpoint or a REST API. How do you locally index something remote under these constraints? -Sarah Cindy Harper -Original Message- From: Harper, Cynthia Sent: Tuesday, February 24, 2015 1:20 PM To: auto...@listserv.syr.edu; 'Williams, Ann' Subject: RE: linked data question What I haven't read, but what I have wondered about, is whether so far, linked DATA is distributed, but the INDEXES are local? Is there any example of a system with distributed INDEXES? Cindy Harper char...@vts.edu -Original Message- From: AUTOCAT [mailto:auto...@listserv.syr.edu] On Behalf Of Williams, Ann Sent: Tuesday, February 24, 2015 10:26 AM To: auto...@listserv.syr.edu Subject: [ACAT] linked data question I was just wondering how linked data will affect OPAC searching and discovery vs. a record with text approach. For example, we have various 856 links to publisher, summary and biographical information in our OPAC as well as ISBNs linking to ContentCafe. But none of that content is discoverable in the OPAC and it requires a further click on the part of patrons (many of whom won't click). Ann Williams USJ -- *** AUTOCAT quoting guide: http://www.cwu.edu/~dcc/Autocat/copyright.html E-mail AUTOCAT listowners: autocat-requ...@listserv.syr.edu Search AUTOCAT archives: http://listserv.syr.edu/archives/autocat.html By posting messages to AUTOCAT, the author does not cede copyright ***
