All of the nodes now show as being Active. When starting the replicas I did receive the following message though. Not sure if this is expected or not.
INFO: Attempting to replicate from http://JamiesMac.local:8501/solr/slice2_shard2/ Feb 14, 2012 10:53:34 PM org.apache.solr.common.SolrException log SEVERE: Error while trying to recover:org.apache.solr.common.SolrException: null java.lang.NullPointerException at org.apache.solr.handler.admin.CoreAdminHandler.handlePrepRecoveryAction(CoreAdminHandler.java:646) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:358) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:172) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) null java.lang.NullPointerException at org.apache.solr.handler.admin.CoreAdminHandler.handlePrepRecoveryAction(CoreAdminHandler.java:646) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:358) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:172) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) request: http://JamiesMac.local:8501/solr/admin/cores?action=PREPRECOVERY&core=slice2_shard2&nodeName=JamiesMac.local:8502_solr&coreNodeName=JamiesMac.local:8502_solr_slice2_shard1&wt=javabin&version=2 at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:433) at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:251) at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:120) at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:208) Feb 14, 2012 10:53:34 PM org.apache.solr.update.UpdateLog dropBufferedUpdates INFO: Dropping buffered updates FSUpdateLog{state=BUFFERING, tlog=null} Feb 14, 2012 10:53:34 PM org.apache.solr.common.SolrException log SEVERE: Recovery failed - trying again... Feb 14, 2012 10:53:35 PM org.apache.solr.cloud.RecoveryStrategy run INFO: Finished recovery process Feb 14, 2012 10:53:35 PM org.apache.solr.cloud.RecoveryStrategy run INFO: Attempting to PeerSync from http://JamiesMac.local:8501/solr/slice2_shard2/ Feb 14, 2012 10:53:35 PM org.apache.solr.cloud.RecoveryStrategy run INFO: Sync Recovery was not successful - trying replication Feb 14, 2012 10:53:35 PM org.apache.solr.cloud.RecoveryStrategy run INFO: Begin buffering updates Feb 14, 2012 10:53:35 PM org.apache.solr.update.UpdateLog bufferUpdates INFO: Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null} Feb 14, 2012 10:53:35 PM org.apache.solr.cloud.RecoveryStrategy replicate INFO: Attempting to replicate from http://JamiesMac.local:8501/solr/slice2_shard2/ Feb 14, 2012 10:53:35 PM org.apache.solr.common.SolrException log SEVERE: Error while trying to recover:org.apache.solr.common.SolrException: null java.lang.NullPointerException at org.apache.solr.handler.admin.CoreAdminHandler.handlePrepRecoveryAction(CoreAdminHandler.java:646) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:358) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:172) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) null java.lang.NullPointerException at org.apache.solr.handler.admin.CoreAdminHandler.handlePrepRecoveryAction(CoreAdminHandler.java:646) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:358) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:172) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) request: http://JamiesMac.local:8501/solr/admin/cores?action=PREPRECOVERY&core=slice2_shard2&nodeName=JamiesMac.local:8502_solr&coreNodeName=JamiesMac.local:8502_solr_slice2_shard1&wt=javabin&version=2 at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:433) at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:251) at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:120) at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:208) On Tue, Feb 14, 2012 at 10:18 PM, Jamie Johnson <jej2...@gmail.com> wrote: > Doing so now, will let you know if I continue to see the same issues > > On Tue, Feb 14, 2012 at 4:59 PM, Mark Miller <markrmil...@gmail.com> wrote: >> Doh - looks like I was just seeing a test issue. Do you mind updating and >> trying the latest rev? At the least there should be some better logging >> around the recovery. >> >> I'll keep working on tests in the meantime. >> >> - Mark >> >> On Feb 14, 2012, at 3:15 PM, Jamie Johnson wrote: >> >>> Sounds good, if I pull the latest from trunk and rerun will that be >>> useful or were you able to duplicate my issue now? >>> >>> On Tue, Feb 14, 2012 at 3:00 PM, Mark Miller <markrmil...@gmail.com> wrote: >>>> Okay Jamie, I think I have a handle on this. It looks like an issue with >>>> what config files are being used by cores created with the admin core >>>> handler - I think it's just picking up default config and not the correct >>>> config for the collection. This means they end up using config that has no >>>> UpdateLog defined - and so recovery fails. >>>> >>>> I've added more logging around this so that it's easy to determine that. >>>> >>>> I'm investigating more and working on a test + fix. I'll file a JIRA issue >>>> soon as well. >>>> >>>> - Mark >>>> >>>> On Feb 14, 2012, at 11:39 AM, Jamie Johnson wrote: >>>> >>>>> Thanks Mark, not a huge rush, just me trying to get to use the latest >>>>> stuff on our project. >>>>> >>>>> On Tue, Feb 14, 2012 at 10:53 AM, Mark Miller <markrmil...@gmail.com> >>>>> wrote: >>>>>> Sorry, have not gotten it yet, but will be back trying later today - >>>>>> monday, tuesday tend to be slow for me (meetings and crap). >>>>>> >>>>>> - Mark >>>>>> >>>>>> On Feb 14, 2012, at 9:10 AM, Jamie Johnson wrote: >>>>>> >>>>>>> Has there been any success in replicating this? I'm wondering if it >>>>>>> could be something with my setup that is causing the issue... >>>>>>> >>>>>>> >>>>>>> On Mon, Feb 13, 2012 at 8:55 AM, Jamie Johnson <jej2...@gmail.com> >>>>>>> wrote: >>>>>>>> Yes, I have the following layout on the FS >>>>>>>> >>>>>>>> ./bootstrap.sh >>>>>>>> ./example (standard example directory from distro containing jetty >>>>>>>> jars, solr confs, solr war, etc) >>>>>>>> ./slice1 >>>>>>>> - start.sh >>>>>>>> -solr.xml >>>>>>>> - slice1_shard1 >>>>>>>> - data >>>>>>>> - slice2_shard2 >>>>>>>> -data >>>>>>>> ./slice2 >>>>>>>> - start.sh >>>>>>>> - solr.xml >>>>>>>> -slice2_shard1 >>>>>>>> -data >>>>>>>> -slice1_shard2 >>>>>>>> -data >>>>>>>> >>>>>>>> if it matters I'm running everything from localhost, zk and the solr >>>>>>>> shards >>>>>>>> >>>>>>>> On Mon, Feb 13, 2012 at 8:42 AM, Sami Siren <ssi...@gmail.com> wrote: >>>>>>>>> Do you have unique dataDir for each instance? >>>>>>>>> 13.2.2012 14.30 "Jamie Johnson" <jej2...@gmail.com> kirjoitti: >>>>>> >>>>>> - Mark Miller >>>>>> lucidimagination.com >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>> >>>> - Mark Miller >>>> lucidimagination.com >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >> >> - Mark Miller >> lucidimagination.com >> >> >> >> >> >> >> >> >> >> >>
