siddharthteotia commented on a change in pull request #5297:
URL: https://github.com/apache/incubator-pinot/pull/5297#discussion_r414867401
##########
File path:
pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/text/LuceneTextIndexCreator.java
##########
@@ -50,6 +58,15 @@
private final Directory _indexDirectory;
private final IndexWriter _indexWriter;
+ public static final CharArraySet ENGLISH_STOP_WORDS_SET =
+ new CharArraySet(Arrays.asList(
Review comment:
StopAnalyzer can't be used either because it tokenizes input text at
each character. So we stick to StandardAnalyzer which uses StandardTokenizer
(based on unicode general purpose text segmentation algorithm)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
