[
https://issues.apache.org/jira/browse/LUCENE-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12784338#action_12784338
]
DM Smith commented on LUCENE-2034:
----------------------------------
Robert:
bq. DM, I think we can have both? A method to get the default stopword list,
but then they also happen to be in text files too?
Yes.
Uwe:
bq. Ideally he new methods should return Set<?> but implement this by a
CharArraySet (which would be possible then). At the moment the sets are always
copied to CharArraySet in each Analyzer.
I agree. That could also simplify some of what Simon is doing. However, the one
distinctive of CharArraySet is that it can take input that is not lowercase and
ignore the casing. This is what Simon's StopawareAnalyzer.loadStopwordSet(...)
allows.
BTW, in some of the analyzers sometimes it is a CharArraySet and other times it
is not (when it is via this class). This would make the treatment uniform.
> Massive Code Duplication in Contrib Analyzers - unifly the analyzer ctors
> -------------------------------------------------------------------------
>
> Key: LUCENE-2034
> URL: https://issues.apache.org/jira/browse/LUCENE-2034
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/analyzers
> Affects Versions: 2.9
> Reporter: Simon Willnauer
> Assignee: Robert Muir
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2034,patch, LUCENE-2034,patch, LUCENE-2034.patch,
> LUCENE-2034.patch, LUCENE-2034.patch, LUCENE-2034.patch, LUCENE-2034.patch,
> LUCENE-2034.txt
>
>
> Due to the variouse tokenStream APIs we had in lucene analyzer subclasses
> need to implement at least one of the methodes returning a tokenStream. When
> you look at the code it appears to be almost identical if both are
> implemented in the same analyzer. Each analyzer defnes the same inner class
> (SavedStreams) which is unnecessary.
> In contrib almost every analyzer uses stopwords and each of them creates his
> own way of loading them or defines a large number of ctors to load stopwords
> from a file, set, arrays etc.. those ctors should be removed / deprecated and
> eventually removed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
