[
https://issues.apache.org/jira/browse/LUCENE-3662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir updated LUCENE-3662:
--------------------------------
Attachment: LUCENE-3662.patch
patch, hooking this into directspellchecker by default. I think its ready to
commit.
I did some rough perf tests, the transpositions costs us nothing. But the
suggestions are much more relevant in some situations:
for example, I typed "Wahsington" into the geonames database, asking for top-5
suggestions:
{panel:title=maxEdits=1}
before:
(no suggestions)
after:
||Suggestion||Score||DocFreq||
|Washington|0.9|61|
{panel:title=maxEdits=2}
before:
||Suggestion||Score||DocFreq||
|Washington|0.8|61|
|Warrington|0.8|13|
|Waddington|0.8|10|
|Wallington|0.8|7|
|Watlington|0.8|5|
after:
||Suggestion||Score||DocFreq||
|Washington|0.9|61|
|Warrington|0.8|13|
|Waddington|0.8|10|
|Wallington|0.8|7|
|Watlington|0.8|5|
About the beers: the story is that I made some small progress towards
implementing this (https://bitbucket.org/rmuir/moman) with many many beers, but
got stuck, Jean-Phillipe merged my commits, emailed me that he is confident he
can implement it in 15 hours, and did just that. I found a bug in the test, he
fixed it the next day as before... and go figure it looks like the bug might
have been in the part I did.
> extend LevenshteinAutomata to support transpositions as primitive edits
> -----------------------------------------------------------------------
>
> Key: LUCENE-3662
> URL: https://issues.apache.org/jira/browse/LUCENE-3662
> Project: Lucene - Java
> Issue Type: New Feature
> Affects Versions: 4.0
> Reporter: Robert Muir
> Attachments: LUCENE-3662.patch, LUCENE-3662.patch, LUCENE-3662.patch,
> LUCENE-3662.patch, LUCENE-3662.patch, LUCENE-3662_upgrade_moman.patch,
> lev1.rev115.txt, lev1.rev119.txt, lev1t.txt, update-moman.patch
>
>
> This would be a nice improvement for spell correction: currently a
> transposition counts as 2 edits,
> which means users of DirectSpellChecker must use larger values of n (e.g. 2
> instead of 1) and
> larger priority queue sizes, plus some sort of re-ranking with another
> distance measure for good results.
> Instead if we can integrate "chapter 7" of
> http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.16.652
> then you can just build an alternative DFA where a transposition is only a
> single edit
> (http://en.wikipedia.org/wiki/Damerau%E2%80%93Levenshtein_distance)
> According to the benchmarks in the original paper, the performance for LevT
> looks to be very similar to Lev.
> Support for this is now in moman (https://bitbucket.org/jpbarrette/moman/)
> thanks to Jean-Philippe
> Barrette-LaPierre.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
