Our main interest is the text quality, if we get this right the shortening
/ tweaking should be the easy part :)
Could you please give us with some text quality feedback and if it is good
maybe we can start testing it to other languages as well
Best,
Dimitris
On Tue, Oct 2, 2012 at 11:11 PM, Piotr Jagielski <[email protected]>wrote:
> I haven't done extensive tests but one thing to improve for sure is the
> abstract shortening algorithm. You currently use a simple regex to solve a
> complex problem of breaking down natural language text into sentences.
> java.text.BreakIterator yields better results and is also locale sensitive.
> You might also want to take a look at more advanced boundary analysis
> library at http://userguide.icu-project.org/boundaryanalysis.
>
> Regards,
> Piotr
>
>
> On 2012-10-01 07:42, Dimitris Kontokostas wrote:
>
> Hi Piotr,
>
> Thank you for the patch, Although it catches an error case, it seems safe
> to be included in the framework.
> About the PageNode Abstracts, can you give us a quality feedback? It is
> something we always wanted to test but couldn't find the time.
>
> Best,
> Dimitris
>
> On Fri, Sep 28, 2012 at 5:57 PM, Piotr Jagielski <[email protected]>wrote:
>
>> OK, I submitted a bug with proposed fix and test cases at
>> https://sourceforge.net/tracker/?func=detail&aid=3572779&group_id=190976&atid=935521
>> .
>>
>> Thanks for the link to documentation. Now I know where the confusion came
>> from. I should have mentioned that I tweaked the code locally a little bit
>> in order to generate abstracts without a local MediaWiki instance :-) I
>> used SimpleWikiParser to create PageNode to pass to AbstractExctractor. The
>> issue is in SimpleWikiParser.
>>
>> Piotr
>>
>>
>> On 2012-09-13 11:51, Pablo N. Mendes wrote:
>>
>>
>> This question keeps coming up, so I added hints to the documentation.
>>
>> 4.3. Running Abstract Extraction
>> http://wiki.dbpedia.org/Documentation#h25-8
>>
>> Cheers,
>> Pablo
>>
>> On Thu, Sep 13, 2012 at 7:13 AM, Dimitris Kontokostas
>> <[email protected]>wrote:
>>
>>> Hi Piotr,
>>>
>>> We will happily accept you patch :)
>>> You can take a look at [1] & [2] for more details on abstract extraction.
>>>
>>> Best,
>>> Dimitris
>>>
>>> [1]
>>> http://dbpedia.hg.sourceforge.net/hgweb/dbpedia/extraction_framework/file/d580c99b5bbc/core/src/main/scala/org/dbpedia/extraction/mappings/AbstractExtractor.scala#l66
>>> [2]
>>> http://dbpedia.hg.sourceforge.net/hgweb/dbpedia/dbpedia/file/efc0afb0faa3/abstractExtraction/README.txt
>>>
>>>
>>> On Wed, Sep 12, 2012 at 10:37 PM, Piotr Jagielski <[email protected]
>>> > wrote:
>>>
>>>> Dimiris,
>>>>
>>>> I guess I'm confused about the project structure. I looked at
>>>> AbstractExtractor.scala. It clearly uses PageNode to figure out what the
>>>> abstract is and I figured out that PageNode is created by SimpleWikiParser.
>>>> I now see that there is some PHP code for a lot of stuff including abstract
>>>> extraction. I don't understand the relationship between Scala extraction
>>>> framework and PHP code and I'm wondering if you mean the latter when you
>>>> refer to "modified mediawiki installation". When I used
>>>> AbstractExtractor.scala to generate the abstract for
>>>> http://pl.dbpedia.org/page/Agnieszka_Rylik I got similar result
>>>> because of a strangely formatted template not parsed correctly.
>>>>
>>>> Anyway, I can now access the bug tracker so I will submit a patch there.
>>>>
>>>> Regards,
>>>> Piotr
>>>>
>>>>
>>>>
>>>> On 2012-09-11 08:39, Dimitris Kontokostas wrote:
>>>>
>>>> Hi Piotr,
>>>>
>>>> Any contribution is always welcome! However, the case you are referring
>>>> seems strange.
>>>> Abstracts are not generated by the SimpleWikiParser, they are produced
>>>> by a local wikipedia clone using a modified mediawiki installation.
>>>>
>>>> Best,
>>>> Dimitris
>>>>
>>>> On Mon, Sep 10, 2012 at 7:30 PM, Piotr Jagielski <[email protected]
>>>> > wrote:
>>>>
>>>>> Any thoughts on this? I wrote some test cases and a fix that I can
>>>>> contribute in case you are interested.
>>>>>
>>>>> Piotr
>>>>>
>>>>> On 2012-09-06 01:13, Piotr Jagielski wrote:
>>>>> > Hello,
>>>>> >
>>>>> > There is an issue with SimpleWikiParser in extraction framework
>>>>> > regarding template parsing. Strangely formatted templates like this
>>>>> one:
>>>>> > {{template | value |= }} are not parsed as templates nodes but text
>>>>> > nodes instead. Apart from preventing data extraction it results in
>>>>> > incorrect abstracts on Polish Dbpedia. For example on
>>>>> > http://pl.dbpedia.org/page/Agnieszka_Rylik the abstract contains
>>>>> infobox
>>>>> > parameter values.
>>>>> >
>>>>> > BTW, I noticed a couple of issues I when trying to report this issue.
>>>>> > 1) I couldn't submit a bug on SourceForge at
>>>>> > https://sourceforge.net/tracker/?group_id=190976&atid=935520. I got
>>>>> > permission denied error. Is there any reason to restrict bug
>>>>> reporting
>>>>> > to project members only?
>>>>> > 2) I wanted to created a test case for it but I couldn't find any
>>>>> tests
>>>>> > for the parser part in the repository. Are there any?
>>>>> >
>>>>> > Regards,
>>>>> > Piotr
>>>>> >
>>>>> >
>>>>> ------------------------------------------------------------------------------
>>>>> > Live Security Virtual Conference
>>>>> > Exclusive live event will cover all the ways today's security and
>>>>> > threat landscape has changed and how IT managers can respond.
>>>>> Discussions
>>>>> > will include endpoint security, mobile security and the latest in
>>>>> malware
>>>>> > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>>>> > _______________________________________________
>>>>> > Dbpedia-discussion mailing list
>>>>> > [email protected]
>>>>> > https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion
>>>>> >
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> Live Security Virtual Conference
>>>>> Exclusive live event will cover all the ways today's security and
>>>>> threat landscape has changed and how IT managers can respond.
>>>>> Discussions
>>>>> will include endpoint security, mobile security and the latest in
>>>>> malware
>>>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>>>> _______________________________________________
>>>>> Dbpedia-discussion mailing list
>>>>> [email protected]
>>>>> https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Kontokostas Dimitris
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Kontokostas Dimitris
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Live Security Virtual Conference
>>> Exclusive live event will cover all the ways today's security and
>>> threat landscape has changed and how IT managers can respond. Discussions
>>> will include endpoint security, mobile security and the latest in malware
>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>> _______________________________________________
>>> Dbpedia-discussion mailing list
>>> [email protected]
>>> https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion
>>>
>>>
>>
>>
>> --
>> ---
>> Pablo N. Mendes
>> http://pablomendes.com
>> Events: http://wole2012.eurecom.fr
>>
>>
>>
>
>
> --
> Kontokostas Dimitris
>
>
>
--
Kontokostas Dimitris
------------------------------------------------------------------------------
Don't let slow site performance ruin your business. Deploy New Relic APM
Deploy New Relic app performance management and know exactly
what is happening inside your Ruby, Python, PHP, Java, and .NET app
Try New Relic at no cost today and get our sweet Data Nerd shirt too!
http://p.sf.net/sfu/newrelic-dev2dev
_______________________________________________
Dbpedia-discussion mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion
