Best way to fix Document contains at least one immense term?

2014-07-01 Thread Michael Ryan
In LUCENE-5472, Lucene was changed to throw an error if a term is too long, rather than just logging a message. I have fields with terms that are too long, but I don't care - I just want to ignore them and move on. The recommended solution in the docs is to use LengthFilterFactory, but this

Re: Best way to fix Document contains at least one immense term?

2014-07-01 Thread Jack Krupansky
anyway, so count of characters versus UTF-8 bytes may be a non-problem. -- Jack Krupansky -Original Message- From: Michael Ryan Sent: Tuesday, July 1, 2014 9:49 AM To: solr-user@lucene.apache.org Subject: Best way to fix Document contains at least one immense term? In LUCENE-5472, Lucene

RE: Best way to fix Document contains at least one immense term?

2014-07-01 Thread Michael Ryan
...@basetechnology.com] Sent: Tuesday, July 01, 2014 5:23 PM To: solr-user@lucene.apache.org Subject: Re: Best way to fix Document contains at least one immense term? You could develop an update processor to skip or trim long terms as you see fit. You can even code a script in JavaScruipt using the stateless