[
https://issues.apache.org/jira/browse/LUCENE-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17318601#comment-17318601
]
Uwe Schindler edited comment on LUCENE-9924 at 4/10/21, 9:05 PM:
-----------------------------------------------------------------
Hi, yes that's a separate issue.
I just noticed that there are more and more puny code root domains. So a
further improvement would be to allow the tokenizer to match both variants.
Because in full text nobody would use the puny code one.
So in short: I would open issue to add both variants: ascii variant (puny code)
and decoded unicode version.
Puny code decoder is part of icu
was (Author: thetaphi):
Hi, yes that's a separate issue.
I just noticed that there are more and more puny code root domains. So a
further improvement would be to allow the tokenizer to match both variants.
Because in full text nobody would use the puny code one.
So in short: I would open issue to add both variants: ascii variant (puny code)
and decided unicode version.
Puny code decoder is part of icu
> regenerate TLD list from IANA TLD db, rather than root zone db
> --------------------------------------------------------------
>
> Key: LUCENE-9924
> URL: https://issues.apache.org/jira/browse/LUCENE-9924
> Project: Lucene - Core
> Issue Type: Task
> Reporter: Robert Muir
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently the TLD list comes from root zone database (DNS records) and these
> are parsed with regular expressions. Instead we can use
> https://data.iana.org/TLD/tlds-alpha-by-domain.txt which is a simple list.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
