[
https://issues.apache.org/jira/browse/LUCENE-3666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13188493#comment-13188493
]
Uwe Schindler commented on LUCENE-3666:
---------------------------------------
oh small changes needed:
This example consumer code is incomplete:
{noformat}
+<PRE class="prettyprint">
+ Analyzer analyzer = new StandardAnalyzer(Version.LUCENE_XY); // or any
other analyzer
+ TokenStream ts = analyzer.tokenStream("myfield",new StringReader("some
text goes here"));
+ while (ts.incrementToken()) {
+ System.out.println("token: "+ts));
+ }
+</PRE>
{noformat}
- TokenStream needs to call reset() before incrementing tokens (thats the
contract)
- It should call end() after incrementToken()
- It must call close finally (ideally in try/finally)
Finally TokenStream no longer is required to implement toString(), so this one
may produce useless standard toString() output (in 4.0 it does print
TokenStreamClass@hashcode, in 3.x for backwards compatibility it prints the
same like reflectAsString).
To get Token debug outbut, use
[http://lucene.apache.org/java/3_5_0/api/core/org/apache/lucene/util/AttributeSource.html#reflectAsString(boolean)],
e.g. {code}System.out.println("token: "+ts.reflectAsString(true)){code}.
Ideally the example code would use one attribute as example.
The example attribute impl's copyTo is using the actual Attribute (not the
impl) when casting, but the attribute has no fields, only methods. The copyTo
must call set setPos() method of the attribute interface.
Thats all.
> Update org.apache.lucene.analysis package summary
> -------------------------------------------------
>
> Key: LUCENE-3666
> URL: https://issues.apache.org/jira/browse/LUCENE-3666
> Project: Lucene - Java
> Issue Type: Improvement
> Components: general/javadocs
> Affects Versions: 3.5
> Reporter: Steven Rowe
> Assignee: Steven Rowe
> Priority: Minor
> Fix For: 3.6, 4.0
>
> Attachments: LUCENE-3666-branch_3x.patch,
> LUCENE-3666-branch_3x.patch, LUCENE-3666-branch_3x.patch,
> LUCENE-3666-branch_3x.patch, LUCENE-3666-trunk.patch, LUCENE-3666-trunk.patch
>
>
> {{package.html}} in {{lucene/src/java/org/apache/lucene/analysis/}} is out of
> date.
> It looks like the contents of the branch_3x version haven't changed
> substantially since the Lucene 2.9 release, e.g. it refers to
> {{TermAttribute}} instead of {{CharTermAttribute}}.
> The trunk version is more modern - it refers to {{CharTermAttribute}} - but
> it also has some issues. E.g., I can see that the {{LengthFilter}}
> discussion doesn't refer to {{FilteringTokenFilter}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
