I was working with skos_categories but these are some reasons that I would
avoid using that dataset for parent-child relationship detection.
Reason 1
I need to get all leaf categories AND thier child- parent relationships.
Categories that don't have a broader category are not included in
skos_category dump. This claim is discussed here
https://github.com/dbpedia/dbpedia-links/issues/16
Reason 2
We need to concern about data freshness dealing with tasks related to
knowledge representation. Debpedia latest dumps (1.8) are nealy one year
older. This work also need to deal with other datasets such as Wikipedia
page_edit_history, interlaguage links ect. So there is the need that all
the datasets are in sync with each other, i.e. they have the same dates.
If I use dbpedia dumps there is a problem of finding synchronized datasets.
On Fri, Jun 28, 2013 at 3:33 PM, Alessio Palmero Aprosio <[email protected]>wrote:
> Dear Kasun,
> I'm investigating graph DBs in this period, but I haven't tried any yet.
> In my implementation, I'm using a Lucene index to store categories. I have
> two fields: category name and parent. The parent is null if there is no
> parent at all.
> Whenever I need a path, I start from the category and go for parents. If I
> encounter a category I already encountered before, I stop the loop
> (otherwise it will go on forever).
>
> You also can use a simple MySQL database with two fields, but I think
> Lucene is faster.
>
> Alessio
>
>
> Il 28/06/13 10:25, kasun perera ha scritto:
>
> Hi Alessio
>
> On Thu, Jun 27, 2013 at 1:42 PM, Alessio Palmero Aprosio
> <[email protected]>wrote:
>
>> Dear Kasun,
>> I had to deal with the same problem some months ago,
>>
>
> Just curious about how did you stored the edges and vertices
> relationships when processing the categories.
> In-memory processing would be difficult since it has a huge number
> of edges and vertices, so I think it's good to store them in a database.
> I have heard about graph databases[1], but haven't worked with them. Did
> you use something like that or simple mysql database?
>
> [1]http://en.wikipedia.org/wiki/Graph_database
>
>
>> and I managed to use the XML article file: you can intercept categories
>> using the "Category:" prefix, and you can infer father-son relation using
>> the <title> tag (if the <title> starts with "Category:", all the categories
>> for this page are possible ancestors).
>> The Wikipedia category taxonomy is quite a mess, so good luck!
>>
>> Alessio
>>
>>
>> Il 27/06/13 05:24, kasun perera ha scritto:
>>
>> As discussed with Marco these are the next tasks that i would be
>> working.
>>
>> 1. Identification of leaf categories
>> 2. Prominent leaves discovery
>> 3. Pages clustering based on prominent leaves
>>
>> For above task 1, I'm planing to use Wikipedia category and
>> category_links SQL tables available here.
>> http://dumps.wikimedia.org/enwiki/20130604/
>>
>> above dump files are somewhat larger 20mb and 1.2gb in size
>> respectively.
>> I'm thinking of putting these data in to a MySql database and do the
>> processing rather than process these files in-memory. Also the amount of
>> leaf categories and prominent nodes would be large and need to be push to a
>> MySql tables.
>>
>> I want to know whether this code should be write under
>> extraction-framwork code,if so where should I plug this code?
>> or whether is it good idea to write it separately, and push to a new
>> repo? If I write it separately can I use a language other than Scala?
>>
>>
>> --
>> Regards
>>
>> Kasun Perera
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> This SF.net email is sponsored by Windows:
>>
>> Build for Windows Store.
>> http://p.sf.net/sfu/windows-dev2dev
>>
>>
>>
>> _______________________________________________
>> Dbpedia-developers mailing
>> [email protected]https://lists.sourceforge.net/lists/listinfo/dbpedia-developers
>>
>>
>>
>
>
> --
> Regards
>
> Kasun Perera
>
>
>
--
Regards
Kasun Perera
------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:
Build for Windows Store.
http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Dbpedia-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/dbpedia-developers
