[
https://issues.apache.org/jira/browse/SOLR-2839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128335#comment-13128335
]
Robert Muir commented on SOLR-2839:
-----------------------------------
ok, i'd like to add this basic implementation first.
later, we should add support for some advanced parameters and refactoring:
* whitelisting should not happen in the base class as a post-filter (though
this is fine as a default implementation), but subclasses should override i
think. For this detector, it could improve performance.
* for this detector whitelist should support priors too (e.g. en=0.5, fr=0.1).
* we should add support for configuring smoothing parameter and maxTextLength
(and, the base class's concat should respect that too).
* both this implementation and the tika implementation are copying objects
across lists of language information, i think this is not very efficient to do
per-document. So I think we should change the API from List<DetectedLanguage>
detectLanguage() to Iterable<DetectedLanguage> detectLanguage. It seems in
general it just wants the first one anyway.
> add alternative language detection impl
> ---------------------------------------
>
> Key: SOLR-2839
> URL: https://issues.apache.org/jira/browse/SOLR-2839
> Project: Solr
> Issue Type: Improvement
> Reporter: Robert Muir
> Attachments: SOLR-2839.patch
>
>
> based on http://code.google.com/p/language-detection (apache license),
> supports 53 languages.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
