I wonder if it would make sense to provide a TruncationFilter in addition to the LengthFilter. That way long tokens in source text could be better supported, albeit with some confusion if they share the same very long prefix...
On Fri, Sep 23, 2022 at 9:56 AM Scott Guthery <sguth...@gmail.com> wrote: > > Thanks much, Adrian. I hadn't realized that the size limit was on one > token in the text as opposed to being a limit on the length of the entire > text field. I'm loading patents, so I suspect that the very long word is a > DNA sequence. > > Thanks also for your guidance with regard to setting maximums. > > Cheers, Scott > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
