[
https://issues.apache.org/jira/browse/LUCENE-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Rowe updated LUCENE-5753:
-------------------------------
Attachment: LUCENE-5753.patch
Patch updating TLDs for {{UAX29URLEmailTokenizer}}. Precommit and all tests
pass.
{{ASCIITLD.jflex-macro}} increases from 342 to 1543 TLDs.
I had to use {{ANT_OPTS=-Xmx8g ant regenerate}} to give enough memory to JFlex.
({{-Xmx4g}} didn't work, but maybe something between 4g and 8g would - I
didn't try.)
With this patch, the lucene-analyzers-common jar goes from 1.5MB to 1.6MB; I
think the size increase is acceptable.
Committing shortly.
> Domain lists for UAX_URL_EMAIL analyzer are incomplete - cannot recognize
> ".local" among others
> -----------------------------------------------------------------------------------------------
>
> Key: LUCENE-5753
> URL: https://issues.apache.org/jira/browse/LUCENE-5753
> Project: Lucene - Core
> Issue Type: Bug
> Components: modules/analysis
> Reporter: Steve Merritt
> Attachments: LUCENE-5753.patch
>
>
> uax_url_email analyzer appears unable to recognize the ".local" TLD among
> others. Bug can be reproduced by
> curl -XGET
> "ADDRESS/INDEX/_analyze?text=First%20Last%20lname@section.mycorp.local&pretty&analyzer=uax_url_email"
> will parse "ln...@section.my" and "corp.local" as separate tokens, as opposed
> to
> curl -XGET
> "ADDRESS/INDEX/_analyze?text=first%20last%20ln...@section.mycorp.org&pretty&analyzer=uax_url_email"
> which will recognize "ln...@section.mycorp.org".
> Can this be fixed by updating to a newer version? I am running ElasticSearch
> 0.90.5 and whatever Lucene version sits underneath that. My suspicion is that
> the TLD list the analyzer relies on (http://www.internic.net/zones/root.zone,
> I think?) is incomplete and needs updating.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
