GWAP: Evaluating Summaries
Dear all, We would like to invite you to try the latest creation of WhoKnows?Movies! [1] WhoKnows?Movies! is a GWAP (game with a purpose) that asks quiz questions about famous movies. The data comes from Freebase. We also have a high score list which we will keep online for eternity (you can mention it in your CV)! Game with a purpose - but what actually is the purpose?: Summarizing entities individually in accordance to the relevance of specific property-value pairs has gained focus of recent efforts by research (e.g. [2]) and also industry (e.g. [3]). Unfortunately, such summaries are hard to evaluate due to the lack of a gold standard. In our paper - accepted for this year's ISWC Experiments and Evaluation track - we address this issue with a GWAP. We have published a first (anonymized) excerpt of the data gathered via the game at [4]. However, such a dataset is never complete and we want to provide an enhanced version for the camera ready version of the paper. More information can be found on [5]. Kind Regards, Andreas [1] http://j.mp/WhoKnowsMovies [2] Gong Cheng, Thanh Tran and Yuzhong Qu. RELIN: Relatedness and Informativeness-based Centrality for Entity Summarization (http://iswc2011.semanticweb.org/fileadmin/iswc/Papers/Research_Paper/09/70310113.pdf) [3] http://googleblog.blogspot.com/2012/05/introducing-knowledge-graph-things-not.html (2. Get the best summary) [4] http://www.yovisto.com/labs/iswc2012/ [5] http://moresemantic.blogspot.co.at/2012/08/who-knows-movies-2nd-round.html -- Andreas Thalhammer PhD Student Semantic Technology Institute University of Innsbruck http://www.sti2.at/ phone: +43 (0) 512507 6454 email:andreas.thalham...@sti2.at
Re: GWAP: Evaluating Summaries
On 8/22/12 7:39 AM, Andreas Thalhammer wrote: Dear all, We would like to invite you to try the latest creation of WhoKnows?Movies! [1] WhoKnows?Movies! is a GWAP (game with a purpose) that asks quiz questions about famous movies. The data comes from Freebase. We also have a high score list which we will keep online for eternity (you can mention it in your CV)! Game with a purpose - but what actually is the purpose?: Summarizing entities individually in accordance to the relevance of specific property-value pairs has gained focus of recent efforts by research (e.g. [2]) and also industry (e.g. [3]). Unfortunately, such summaries are hard to evaluate due to the lack of a gold standard. In our paper - accepted for this year's ISWC Experiments and Evaluation track - we address this issue with a GWAP. We have published a first (anonymized) excerpt of the data gathered via the game at [4]. However, such a dataset is never complete and we want to provide an enhanced version for the camera ready version of the paper. More information can be found on [5]. Kind Regards, Andreas [1] http://j.mp/WhoKnowsMovies [2] Gong Cheng, Thanh Tran and Yuzhong Qu. RELIN: Relatedness and Informativeness-based Centrality for Entity Summarization (http://iswc2011.semanticweb.org/fileadmin/iswc/Papers/Research_Paper/09/70310113.pdf) [3] http://googleblog.blogspot.com/2012/05/introducing-knowledge-graph-things-not.html (2. Get the best summary) [4] http://www.yovisto.com/labs/iswc2012/ [5] http://moresemantic.blogspot.co.at/2012/08/who-knows-movies-2nd-round.html Andreas, Great stuff! I am assuming that the buy-product of this effort is LOD data cleansing? If so, is there an RDF based Linked Data archive (a linkset) that can then be uploaded to the LOD cloud etc? For instance, is the archive in question part of the resource at: http://www.yovisto.com/labs/iswc2012/google_summaries.tar.gz ? -- Regards, Kingsley Idehen Founder CEO OpenLink Software Company Web: http://www.openlinksw.com Personal Weblog: http://www.openlinksw.com/blog/~kidehen Twitter/Identi.ca handle: @kidehen Google+ Profile: https://plus.google.com/112399767740508618350/about LinkedIn Profile: http://www.linkedin.com/in/kidehen smime.p7s Description: S/MIME Cryptographic Signature
Re: GWAP: Evaluating Summaries
I am assuming that the buy-product of this effort is LOD data cleansing? If so, is there an RDF based Linked Data archive (a linkset) that can then be uploaded to the LOD cloud etc? For instance, is the archive in question part of the resource at: http://www.yovisto.com/labs/** iswc2012/google_summaries.tar.**gzhttp://www.yovisto.com/labs/iswc2012/google_summaries.tar.gz? Or written back to freebase?
Re: GWAP: Evaluating Summaries
On 8/22/12 8:08 AM, Daniel O'Connor wrote: I am assuming that the buy-product of this effort is LOD data cleansing? If so, is there an RDF based Linked Data archive (a linkset) that can then be uploaded to the LOD cloud etc? For instance, is the archive in question part of the resource at: http://www.yovisto.com/labs/iswc2012/google_summaries.tar.gz ? Or written back to freebase? Yes, that too. But, an RDF based Linked Data dump kills two birds with one stone :-) -- Regards, Kingsley Idehen Founder CEO OpenLink Software Company Web: http://www.openlinksw.com Personal Weblog: http://www.openlinksw.com/blog/~kidehen Twitter/Identi.ca handle: @kidehen Google+ Profile: https://plus.google.com/112399767740508618350/about LinkedIn Profile: http://www.linkedin.com/in/kidehen smime.p7s Description: S/MIME Cryptographic Signature
Re: GWAP: Evaluating Summaries
Hi Kingsley, Great stuff! Thanks, the major attribution w.r.t. the quiz game goes to Magnus Knuth and Harald Sack (and many others) from HPI Potsdam. I am assuming that the buy-product of this effort is LOD data cleansing? If so, is there an RDF based Linked Data archive (a linkset) that can then be uploaded to the LOD cloud etc? For instance, is the archive in question part of the resource at: http://www.yovisto.com/labs/iswc2012/google_summaries.tar.gz ? For what I understood, this was one of the initial goals of the WhoKnows? game. One of the publications on this was Collaboratively Patching Linked Data [1]. I hope this answers the question - otherwise feel free to contact Magnus (CCed) or Harald (CCed). Andreas [1] http://arxiv.org/abs/1204.2715 -- Andreas Thalhammer PhD Student Semantic Technology Institute University of Innsbruck http://www.sti2.at/ phone: +43 (0) 512507 6454 email: andreas.thalham...@sti2.at
Re: GWAP: Evaluating Summaries
On 8/22/12 8:18 AM, Andreas Thalhammer wrote: Hi Kingsley, Great stuff! Thanks, the major attribution w.r.t. the quiz game goes to Magnus Knuth and Harald Sack (and many others) from HPI Potsdam. I am assuming that the buy-product of this effort is LOD data cleansing? If so, is there an RDF based Linked Data archive (a linkset) that can then be uploaded to the LOD cloud etc? For instance, is the archive in question part of the resource at: http://www.yovisto.com/labs/iswc2012/google_summaries.tar.gz ? For what I understood, this was one of the initial goals of the WhoKnows? game. One of the publications on this was Collaboratively Patching Linked Data [1]. I hope this answers the question - otherwise feel free to contact Magnus (CCed) or Harald (CCed). Andreas [1] http://arxiv.org/abs/1204.2715 Andreas, I am asking about the location of the Collaboratively Patched Linked Data resources. I assume these exist in the form of RDF document (file) collections? -- Regards, Kingsley Idehen Founder CEO OpenLink Software Company Web: http://www.openlinksw.com Personal Weblog: http://www.openlinksw.com/blog/~kidehen Twitter/Identi.ca handle: @kidehen Google+ Profile: https://plus.google.com/112399767740508618350/about LinkedIn Profile: http://www.linkedin.com/in/kidehen smime.p7s Description: S/MIME Cryptographic Signature
SWIB12 - Registration Open (Semantic Web in Libraries, Cologne, 26 - 28 Nov)
The programme for SWIB12 (Semantic Web in Libraries) Conference in Cologne, 26 - 28 November 2012, has been published. Register now at http://swib.org/swib12 Towards an international LOD library ecosystem To an ever increasing extent Linked Open Data (LOD) is developing into a mainstream topic, with more and more organizations announcing LOD projects and services. Furthermore, and especially during the last two years, Linked Open Data has received a lot more attention from the library world. Examples ranging from the Library of Congress' initiative A Bibliographic Framework for the Digital Age, the Conference of European National Librarians and their vote to support the open licensing of their data, groups like LODLAM, IFLA'S Semantic Web Special Interest Group, to the point of library system vendors and providers discussing and experimenting with Linked Data technology - all these clearly reflect that LOD has gained a lot of momentum in library land. The question today is about how to ensure that LOD won't be a temporary hype but that it will gain ground in the scenery of future infrastructures. SWIB12 will focus on the adaption of Semantic Web approaches in applications for libraries and science. In the last years major efforts have been put into generating LOD datasets from legacy systems and into promoting the LOD approach towards a global and open information space. Upcoming challenges will involve the strategic and technical alignment of catalogues and legacy systems in libraries and the authoring environments for scholarly communication - both in a data and service infrastructure based on the principles of the Semantic Web. Full programme at http://swib.org/swib12/programme.php Offical Twitter Hashtag: #swib12 Further information and contact: Pascal Christoph North Rhine-Westphalian Library Service Center (hbz) Phone +49 221 400 75-139 E-Mail: swib(at)hbz-nrw.de Adrian Pohl North Rhine-Westphalian Library Service Center (hbz) Phone +49 221 400 75-235 E-Mail: swib(at)hbz-nrw.de Joachim Neubert German National Library of Economics Leibniz Information Centre for Economics (ZBW) Phone +49 40 428 34-462 E-Mail: j.neubert(at) zbw.eu Looking forward to meet you in Cologne - Joachim
Re: referencing a concept scheme as the code list of some referrer's property
Dear Thomas, I'm ccing public-esw-t...@w3.org. Perhaps this was the one you were looking for! (1) (2) You probably mean, if a ConceptScheme could be defined as a class, of which the concepts of a given concept scheme are instances? That would be the way to proceed, if you want to use the concept scheme directly as the range of a property. This is has never been suggested for inclusion in SKOS. In fact it is not forbidden, either. You can assert rdf:type statements between concepts and a concept scheme, if you want. You can also define an adhoc sub-class of skos:Concept (say, ex:ConceptOfSchemeX), which includes all concepts that related to a specific concept scheme (ex:SchemeX) by skos:inScheme statements. This is quite easy using OWL. And then you can use this new class as the rdf:range. The possibility of these two options makes it less obvious, why there should be a specific feature in SKOS to represent what you want. But more fundamentally, it was perhaps never discussed, because it's neither a 100% SKOS problem, nor a simple one. It's a bit like the link between a document and a subject concept: there could have been a skos:subject property, but it was argued that Dublin Core's dc:subject was good enough. But it's maybe even worse than that :-) There are indeed discussions in the Dublin Core Architecture community about represent the link between a property and a concept scheme directly, similar to what you want. This is what is called vocabulary/value encoding schemes there [1]. But the existence of this feature at a quite deep, data-model level, rather confirms for me that it is something that clearly couldn't be tackled at the time SKOS was made a standard. One can view this problem as one of modeling RDFS/OWL properties, rather than representing concepts, no? (3) I'm not sure I get the question. If they exist, such mapping properties could be very difficult to semantically define. Would a concept scheme be broader, equivalent, narrower than another one? Rather, I'd say that the property you're after indicates that some concepts from these two concept schemes are connected. For this I think one could use general linkage properties between datasets, such as voiD's linksets [2]. I hope that helps, Antoine [1] http://dublincore.org/documents/profile-guidelines/ , search Statement template: subject [2] http://vocab.deri.ie/void --- This is about SKOS usage in LOD. Yesterday I sent a post to public-swd...@w3.org, but obviously it has not been distributed, although it can be found in the archive. http://lists.w3.org/Archives/Public/public-swd-wg/2012Aug/.html public-swd...@w3.org isn't very active any more, so public-lod@w3.org might be a better place. Best regards, Thomas Original-Nachricht Betreff:referencing a concept scheme as the code list of some referrer's property Datum: Sun, 19 Aug 2012 10:50:05 +0200 Von:Thomas Bandholtz thomas.bandho...@innoq.com An: public-swd...@w3.org Hi SKOS, I came across several examples where concept schemes are referenced as the code list of some property. Usually this is done by two statements: ex:property rdfs:range skos:Concept ; ex:codeList [some concept scheme] . E.g. Data Cubes and geonames use this pattern, but one uses qb:codeList to point to the scheme, the other gn:featureClass. Regarding this, I have two questions: (1) does someone remember a discussion why concept schemes should not be expressed as subclasses of skos:Concept? If subclassing would have been used, any concept scheme could be referenced in a single rdfs:type statement of the concept and a single rdfs:range statement of the referrer. (2) if there are sufficient reasons to insist in the current patterns, shouldn't SKOS be extended by a standard property to be used by referrers when they point to a concept scheme along with a rdfs:range statement? And I add (3) why do we not have mapping properties to link concept schemes from different providers? This cannot be inferred from a given concept mapping, as mapping of some concepts does not imply mappings of their entire schemes. Best regards, Thomas -- Thomas Bandholtz Principal Consultant innoQ Deutschland GmbH Krischerstr. 100, D-40789 Monheim am Rhein, Germany http://www.innoq.com thomas.bandho...@innoq.com +49 178 4049387 http://innoq.com/de/themen/linked-data (German) https://github.com/innoq/iqvoc/wiki/Linked-Data (English)
Re: referencing a concept scheme as the code list of some referrer's property
Hi Antoine and CCs and everybody, nice answer, and I'm glad you have detected my question in this haystack. I think I have to tell more about the context of this question. We have a new RD project about Linking Open Environment Data [1]. Here we try to bring together Data Cubes (prefix qb:) [2], SKOS, DCAT, VoID, etc. In Data Cubes, dimension properties are defined having rdfs:range skos:Concept + qb:codeList rdfs:range skos:ConceptScheme. Fine so far. We have also developed iQvoc [3] in the previous years following the pattern described by SKOS in [4]: The notion of an individual SKOS concept scheme corresponds roughly to the notion of an individual thesaurus. The technical consequence has been (so far) serving a single concept scheme per iQvoc instance, and we may link multiple concept schemes by SKOS mapping properties between such instances. Fine as well so far. Now we have some quite large concept schemes, and a single dimension property cannot refer to entire the concept scheme (= thesaurus) as its value set, but only to a subset. So we have established the pattern of expressing such subsets by one skos:Collection per referring property. Fine again, but different from the qb: pattern (which is also used by geonames, but with a different property: gn:featureClass). If we have many dimension properties, each of them referring to a small subset of concepts as the value set, and we want to follow Data Cubes (or GeoNames), and use iQvoc, we would have to deploy many iQvoc instances each of them containing just a single value set. This would break the overall thesaurus into pieces. Of course we can write some lines of OWL code so that a reasoner can infer a dedicated concept scheme for all skos:member instances of each skos:Collection, but ... All this raised the question if it wouldn' have been better either to ... (a) define one single pattern as part of the SKOS standard how to specify any subset of skos:Concept individuals as the value set of a property, (b) strictly use subclasses of skos:Concept to describe any kind of subsets of skos:Concept individuals, so nobody would need any kind of attachment to the rdfs:range to refer to this subset. (b) is my preferred solution of (a). After thinking this over more and more, I find that the given definition of skos:ConceptScheme has introduced a fatal and completely needless structural redundancy. This could have been avoided by deciding for (b). To be more general: skos:ConceptScheme priotises domain conventions over common and shared (better: to be shared) RDFS/OWL patterns. I found something similar in the Data Structure Definition of Data Cubes. Dave (cc) will understand, as we had some discussion about this topic ;-) I strictly believe it is a better strategy to convince each domain inheritance of a single global standard with only few indispensable options. Following each of the aquainted domain pattern leeds to structural weakness of this one global standard, as everything can be expressed using multiple patterns even though one pattern can fit all. In the open world, each reasoner needs to understand all those domain patterns. This is a quite obscure requirement. Dave et al. is conciliatory with SDMX and weakens RDFS/OWL by this. SKOS is conciliatory with the ISO thesaurus people and weakens RDFS/OWL by this. The same happens more and more in any domain, and may be it is too late to stop this, or even roll this back. I am quite sad about this. Unfortunately, W3C has no clear governance in this question (@Sandro). Sometimes I feel a working draft becomes a recommendation only by public rating in some domain which has no understanding of the power of pure RDFS/OWL . Sorry if I have taken so much of your time, if you have read until here. Finally I quote some lines from Neill Young: Ambulance Blues which may talk about myself: And I still can hear him say: You're all just pissin' in the wind You don't know it but you are. Think I even know it. Best regards, Thomas [1] http://innoq.github.com/led/ [2] http://www.w3.org/TR/vocab-data-cube/ [3] https://github.com/innoq/iqvoc/ [4] http://www.w3.org/TR/2009/REC-skos-reference-20090818/#L1101 Am 22.08.2012 21:15, schrieb Antoine Isaac: Dear Thomas, I'm ccing public-esw-t...@w3.org. Perhaps this was the one you were looking for! (1) (2) You probably mean, if a ConceptScheme could be defined as a class, of which the concepts of a given concept scheme are instances? That would be the way to proceed, if you want to use the concept scheme directly as the range of a property. This is has never been suggested for inclusion in SKOS. In fact it is not forbidden, either. You can assert rdf:type statements between concepts and a concept scheme, if you want. You can also define an adhoc sub-class of skos:Concept (say, ex:ConceptOfSchemeX), which includes all concepts that related to a specific concept scheme (ex:SchemeX) by skos:inScheme statements. This is quite easy
RE: referencing a concept scheme as the code list of some referrer's property
Thomas - I've come to the same conclusion. my:property1 rdfs:range some:codeA . my:property2 rdfs:range some:codeB . some:codeA rdfs:subClassOf skos:Concept . some:codeB rdfs:subClassOf skos:Concept . some:item1 a some:codeA . # also a skos:Concept because of subclass relationship some:item2 a some:codeA . some:item3 a some:codeA . some:item4 a some:codeB , some:codeA . some:item5 a some:codeB . All of the items can be in the same ConceptScheme and in the same repo, but the ConceptScheme is not used explicitly in the ontology using the concepts. Simon Cox -Original Message- From: Thomas Bandholtz [mailto:thomas.bandho...@innoq.com] Sent: Thursday, 23 August 2012 8:07 AM To: public-esw-t...@w3.org; public-lod@w3.org Cc: dave.e.reyno...@gmail.com; san...@w3.org; Till Schulte-Coerne Subject: Re: referencing a concept scheme as the code list of some referrer's property Hi Antoine and CCs and everybody, nice answer, and I'm glad you have detected my question in this haystack. I think I have to tell more about the context of this question. We have a new RD project about Linking Open Environment Data [1]. Here we try to bring together Data Cubes (prefix qb:) [2], SKOS, DCAT, VoID, etc. In Data Cubes, dimension properties are defined having rdfs:range skos:Concept + qb:codeList rdfs:range skos:ConceptScheme. Fine so far. We have also developed iQvoc [3] in the previous years following the pattern described by SKOS in [4]: The notion of an individual SKOS concept scheme corresponds roughly to the notion of an individual thesaurus. The technical consequence has been (so far) serving a single concept scheme per iQvoc instance, and we may link multiple concept schemes by SKOS mapping properties between such instances. Fine as well so far. Now we have some quite large concept schemes, and a single dimension property cannot refer to entire the concept scheme (= thesaurus) as its value set, but only to a subset. So we have established the pattern of expressing such subsets by one skos:Collection per referring property. Fine again, but different from the qb: pattern (which is also used by geonames, but with a different property: gn:featureClass). If we have many dimension properties, each of them referring to a small subset of concepts as the value set, and we want to follow Data Cubes (or GeoNames), and use iQvoc, we would have to deploy many iQvoc instances each of them containing just a single value set. This would break the overall thesaurus into pieces. Of course we can write some lines of OWL code so that a reasoner can infer a dedicated concept scheme for all skos:member instances of each skos:Collection, but ... All this raised the question if it wouldn' have been better either to ... (a) define one single pattern as part of the SKOS standard how to specify any subset of skos:Concept individuals as the value set of a property, (b) strictly use subclasses of skos:Concept to describe any kind of subsets of skos:Concept individuals, so nobody would need any kind of attachment to the rdfs:range to refer to this subset. (b) is my preferred solution of (a). After thinking this over more and more, I find that the given definition of skos:ConceptScheme has introduced a fatal and completely needless structural redundancy. This could have been avoided by deciding for (b). To be more general: skos:ConceptScheme priotises domain conventions over common and shared (better: to be shared) RDFS/OWL patterns. I found something similar in the Data Structure Definition of Data Cubes. Dave (cc) will understand, as we had some discussion about this topic ;-) I strictly believe it is a better strategy to convince each domain inheritance of a single global standard with only few indispensable options. Following each of the aquainted domain pattern leeds to structural weakness of this one global standard, as everything can be expressed using multiple patterns even though one pattern can fit all. In the open world, each reasoner needs to understand all those domain patterns. This is a quite obscure requirement. Dave et al. is conciliatory with SDMX and weakens RDFS/OWL by this. SKOS is conciliatory with the ISO thesaurus people and weakens RDFS/OWL by this. The same happens more and more in any domain, and may be it is too late to stop this, or even roll this back. I am quite sad about this. Unfortunately, W3C has no clear governance in this question (@Sandro). Sometimes I feel a working draft becomes a recommendation only by public rating in some domain which has no understanding of the power of pure RDFS/OWL . Sorry if I have taken so much of your time, if you have read until here. Finally I quote some lines from Neill Young: Ambulance Blues which may talk about myself: And I still can hear him say: You're all just pissin' in the wind You don't know it but you are. Think I even know it. Best regards, Thomas [1] http://innoq.github.com/led/ [2]
