Hi Eric,

The paper of (Bar-Ilan et al., 2006) could be very useful to you. It
discusses various measures, including overlap, Spearman's footrule and
G-measure.
We have explored those in one study (Trigo et al., 2015), so you may have a
look at what we did.
There is also an interesting measure based on Spearman, but it giving a
greater weight to elements positioned higher in the ranking (Costa et al.,
2005).

Greetings
Pavel Brazdil
http://www.liaad.up.pt/area/pbrazdil/

Bar-Ilan, J., Mat-Hassan, M., and Levene, M. (2006). Methods for comparing
rankings of search engine results. Computer networks, 50(10):1448–1463.

L Trigo, M Víta, R Sarmento, P Brazdil, Retrieval, visualization and
validation of affinities between documents, in 1st Int. Workshop on the
design, development and use of Knowledge IT artifacts in professional
communities and aggregations - KITA 2015, 2015. SciTePress, 2015

Joaquim Pinto da Costa and Carlos Soares, A WEIGHTED RANK MEASURE OF
CORRELATION, Aust. N. Z. J. Stat. 47(4), 2005, 515–529

On Fri, Oct 14, 2016 at 11:25 AM, Hugo Gonçalo Oliveira <hro...@dei.uc.pt>
wrote:

> Hi,
>
> Would it make sense to compute the Minimum Edit Distance to transform one
> list into the other?
>
> Best,
> ----------
> Hugo Gonçalo Oliveira
> CISUC, Department of Informatics Engineering, University of Coimbra
> http://eden.dei.uc.pt/~hroliv
>
> On Fri, Oct 14, 2016 at 10:01 AM, Eric Atwell <e.s.atw...@leeds.ac.uk>
> wrote:
>
>> Thanks Olga, BUT I don't see how Spearman's (or other) rank correlation
>> formula measures overlap between 2 ranked lists containing some different
>> words.
>>
>>
>> For example, what is the similarity between 2 "distributional semantics"
>> representations of God?:
>>
>> (1 move, 2 bless, 3 and, 4 forbid, 5 have, 6 be, 7 create, 8 do, 9 know,
>> 10 want) and
>>
>> (1 bless, 2 blesses, 3 delusion, 4 incarnate, 5 hates, 6 himself, 7
>> exists, 8 almighty, 9 forbid, 10 rest)
>>
>>
>>  A simple measure involves count of overlapping words (bless, forbid)
>> i.e. this scores 2 (or 2/10, or 2/20)
>>
>> BUT I also want to take into account ranks: (bless 2,1), (forbid 4,9)
>>
>>
>> Does God shed light on my problems?
>>
>>
>> Eric
>>
>>
>> Eric Atwell, Asst Prof, Language@Leeds and Artificial Intelligence
>> groups,
>> School of Computing, University of Leeds, Times University of the Year
>> 2017
>>
>> ------------------------------
>> *From:* Ольга Ляшевская <ole...@yandex.ru>
>> *Sent:* 14 October 2016 08:09:36
>> *To:* Eric Atwell; CORPORA discussion forum
>> *Cc:* AbdulRahman AlOsaimy; Claire Brierley
>> *Subject:* Re: [Corpora-List] metric of overlap between two ranked lists?
>>
>> Dear Eric,
>>
>> Spearman's rank correlation coefficient will work in this case, I think.
>> https://en.wikipedia.org/wiki/Spearman%27s_rank_correlation_coefficient
>>
>> Best,
>> Olga
>>
>> 14.10.2016, 09:54, "Eric Atwell" <e.s.atw...@leeds.ac.uk>:
>>
>> > Is there a standard metric of overlap between two ranked lists?
>> > e.g. to measure/score the similarity between top 10 keywords extracted
>> > using 2 different formulae, such as LL v MI?
>> > OR e.g. to measure/score the similarity between top 10 hits from Google
>> > v top 10 hits from Bing for a give search phrase?
>> > OR e.g. to measure/score the similarity between ranked lists of PoS-tags
>> > predicted for a word by two rival PoS-taggers in an ensemble tagger?
>> >
>> > If these were unranked sets of keywords, i could simply count the
>> > intersection. But I want to take rank into account in some senible way.
>> >
>> > thanks for expert pointers to proven metrics ...
>> >
>> > Eric Atwell, Asst Prof, Language@Leeds and Artificial Intelligence
>> groups,
>> > School of Computing, University of Leeds, Times University of the Year
>> 2017
>> >
>> > _______________________________________________
>> > UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
>> > Corpora mailing list
>> > Corpora@uib.no
>> > http://mailman.uib.no/listinfo/corpora
>>
>> Olga Lyashevskaya
>>
>> School of Linguistics, Faculty of Humanities
>>  Higher School of Economics, Moscow
>>
>> _______________________________________________
>> UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
>> Corpora mailing list
>> Corpora@uib.no
>> http://mailman.uib.no/listinfo/corpora
>>
>>
>
> _______________________________________________
> UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
> Corpora mailing list
> Corpora@uib.no
> http://mailman.uib.no/listinfo/corpora
>
>
_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora@uib.no
http://mailman.uib.no/listinfo/corpora

Reply via email to