Robert: Thanks. I'm on my way out the door, so I'll have to put up a JIRA with your patch later if it hasn't been done already
Erick On Tue, Oct 29, 2013 at 10:14 PM, Robert Muir <rcm...@gmail.com> wrote: > I think its a bug, but thats just my opinion. i sent a patch to dev@ > for thoughts. > > On Tue, Oct 29, 2013 at 6:09 PM, Erick Erickson <erickerick...@gmail.com> > wrote: > > Hmmm, so you're saying that merging indexes where a field > > has been removed isn't handled. So you have some documents > > that do have a "what" field, but your schema doesn't have it, > > is that true? > > > > It _seems_ like you could get by by putting the _what_ field back > > into your schema, just not sending any data to it in new docs. > > > > I'll let others who understand merging better than me chime in on > > whether this is a case that should be handled or a bug. I pinged the > > dev list to see what the opinion is.... > > > > Best, > > Erick > > > > > > On Mon, Oct 28, 2013 at 6:39 PM, Matthew Shapiro <m...@mshapiro.net> > wrote: > > > >> Sorry for reposting after I just sent in a reply, but I just looked at > the > >> error trace closer and noticed > >> > >> > >> 1. Caused by: java.lang.IllegalArgumentException: no such field what > >> > >> > >> The 'what' field was removed by request of the customer as they wanted > the > >> logic behind what gets queried in the "what" field to be code side > instead > >> of solr side (for easier changing without having to re-index > everything. I > >> didn't feel strongly either way and since they are paying me, I took it > >> out). > >> > >> This makes me wonder if its crashing while merging because a field that > >> used to be there is now gone. However, this seems odd to me as Solr > >> doesn't even let me delete the old data and instead its leaving my > >> collection in an extremely bad state, with the only remedy I can think > of > >> is to nuke the index at the filesystem level. > >> > >> If this is indeed the cause of the crash, is the only way to delete a > field > >> to first completely empty your index first? > >> > >> > >> On Mon, Oct 28, 2013 at 6:34 PM, Matthew Shapiro <m...@mshapiro.net> > wrote: > >> > >> > Thanks for your response. > >> > > >> > You were right, solr is logging to the catalina.out file for tomcat. > >> When > >> > I click the optimize button in solr's admin interface the following > logs > >> > are written: http://apaste.info/laup > >> > > >> > About JVM memory, solr's admin interface is listing JVM memory at 3.1% > >> > (221.7MB is dark grey, 512.56MB light grey and 6.99GB total). > >> > > >> > > >> > On Mon, Oct 28, 2013 at 6:29 AM, Erick Erickson < > erickerick...@gmail.com > >> >wrote: > >> > > >> >> For Tomcat, the Solr is often put into catalina.out > >> >> as a default, so the output might be there. You can > >> >> configure Solr to send the logs most anywhere you > >> >> please, but without some specific setup > >> >> on your part the log output just goes to the default > >> >> for the servlet. > >> >> > >> >> I took a quick glance at the code but since the merges > >> >> are happening in the background, there's not much > >> >> context for where that error is thrown. > >> >> > >> >> How much memory is there for the JVM? I'm grasping > >> >> at straws a bit... > >> >> > >> >> Erick > >> >> > >> >> > >> >> On Sun, Oct 27, 2013 at 9:54 PM, Matthew Shapiro <m...@mshapiro.net> > >> wrote: > >> >> > >> >> > I am working at implementing solr to work as the search backend for > >> our > >> >> web > >> >> > system. So far things have been going well, but today I made some > >> >> schema > >> >> > changes and now things have broken. > >> >> > > >> >> > I updated the schema.xml file and reloaded the core (via the admin > >> >> > interface). No errors were reported in the logs. > >> >> > > >> >> > I then pushed 100 records to be indexed. A call to Commit > afterwards > >> >> > seemed fine, however my next call for Optimize caused the following > >> >> errors: > >> >> > > >> >> > java.io.IOException: background merge hit exception: > >> >> > _2n(4.4):C4263/154 _30(4.4):C134 _32(4.4):C10 _31(4.4):C10 into _37 > >> >> > [maxNumSegments=1] > >> >> > > >> >> > null:java.io.IOException: background merge hit exception: > >> >> > _2n(4.4):C4263/154 _30(4.4):C134 _32(4.4):C10 _31(4.4):C10 into _37 > >> >> > [maxNumSegments=1] > >> >> > > >> >> > > >> >> > Unfortunately, googling for background merge hit exception came up > >> >> > with 2 thing: a corrupt index or not enough free space. The host > >> >> > machine that's hosting solr has 227 out of 229GB free (according > to df > >> >> > -h), so that's not it. > >> >> > > >> >> > > >> >> > I then ran CheckIndex on the index, and got the following results: > >> >> > http://apaste.info/gmGU > >> >> > > >> >> > > >> >> > As someone who is new to solr and lucene, as far as I can tell this > >> >> > means my index is fine. So I am coming up at a loss. I'm somewhat > sure > >> >> > that I could probably delete my data directory and rebuild it but > I am > >> >> > more interested in finding out why is it having issues, what is the > >> >> > best way to fix it, and what is the best way to prevent it from > >> >> > happening when this goes into production. > >> >> > > >> >> > > >> >> > Does anyone have any advice that may help? > >> >> > > >> >> > > >> >> > As an aside, i do not have a stacktrace for you because the solr > admin > >> >> > page isn't giving me one. I tried looking in my logs file in my > solr > >> >> > directory, but it does not contain any logs. I opened up my > >> >> > ~/tomcat/lib/log4j.properties file and saw http://apaste.info/0rTL > , > >> >> > which didnt really help me find log files. Doing a 'find . | grep > >> >> > solr.log' didn't really help either. Any help for finding log > files > >> >> > (which may help find the actual cause of this) would also be > >> >> > appreciated. > >> >> > > >> >> > >> > > >> > > >> >
