unsubscribe
On Sun, Jul 28, 2013 at 5:59 PM, Jan Høydahl <jan....@cominvent.com> wrote: > Hi, > > Looking at the code, you are right. Whitelist processing is only done on > detected languages, not on the fallback or fallbackFields languages, since > these are assumed to be correct. Thus you should not pass in a fallback > language, either in the input document or with langid.fallback which cannot > be handled by your schema. > > This is by design. However, I can also see an argument for making > fallbackFields subject to whitelist logic, especially if you do not control > the application that populates this field, to safeguard against exception. > Also, such a change woudl not harm any of the existing functionality, so it > would be safe to introduce. > > Feel free to write a JIRA issue for it. > > A workaround could be to write a simple UpdateProcessor which removes any > illegal value from langid.fallbackFields before the LangId processor. > > -- > Jan Høydahl, search solution architect > Cominvent AS - www.cominvent.com > > 7. juli 2013 kl. 18:05 skrev adfel70 <adfe...@gmail.com>: > > > Hi > > I'm trying to index a set of documents with solr's language detection > > component. > > I set > > <langid.fallbackFields>user_lan</langid.fallbackFields> > > <langid.whitelist>en,it</langid.whitelist> > > <langid.fallback>en</langid.fallback> > > > > In some documents user_lan has 'sk', solr falls-back to 'sk' ,which is > not > > in the whitelist, and instead of falling back to 'en' as stated here > > <http://wiki.apache.org/solr/LanguageDetection#langid.fallbackFields> > , I > > get an excpetion regarding not having a text_sk field in the schema. > > > > Anyone encountered this behavior? > > > > thanks. > > > > > > > > -- > > View this message in context: > http://lucene.472066.n3.nabble.com/lang-fallback-doesn-t-work-when-using-lang-fallbackFields-tp4076048.html > > Sent from the Solr - User mailing list archive at Nabble.com. > > -- Sam
