Re: Small Tokenization issue

[Shawn Heisey]

On 1/3/2018 1:56 PM, Nawab Zada Asad Iqbal wrote:
Thanks Emir, Erick.
What i want to do is remove empty tokens after WordDelimiterGraphFilter ?
Is there any such option in WordDelimiterGraphFilter to not generate empty
tokens?

I use LengthFilterFactory with a minimum of 1 and a maximum of 512 to 
remove empty tokens.

Thanks,
Shawn