We have an architecture where we want to flip the solr data.dir (massive
dataset) while running and serving search requests with minimal downtime.
Some additional requirements.
* While ideally - we want the Solr Search clients to continue to serve
from the indices as soon as possible -the overriding requirement is that
the downtime for the Search Solr instances should be as less as
possible. So when a new set of (Lucene) indices come in - the
algorithm we are experimenting with:
- create a new solrcore instance with the revised data directory.
- warm up the solrcore instance with some test queries.
- register the new solrcore instance with the same name as the old one,
so that all new queries from the clients are to the new SolrCore instance.
- As part of register (String, SolrCore, boolean ) - the III parameter
when set to false , closes the core connection.
I am trying to understand more about the first and the fourth ( last)
steps.
1) What is the fastest / best possible way to get step 1 done ,through a
pluggable architecture.
Currently - I have written a request handler as follows, that takes care
of creating the core. What is the best way to change dataDir (got as
input from SolrQueryRequest) before creating SolrCore-s.
public class CustomRequestHandler extends RequestHandlerBase implements
SolrCoreAware
{
private CoreDescriptor coreDescriptor;
private String coreName;
@Override
public void handleRequestBody(SolrQueryRequest req,
SolrQueryResponse rsp) throws Exception
{
CoreContainer container = this.coreDescriptor.getCoreContainer();
// TODO: Parse XML to extract data
// container.reload(this.coreName);
// or
// 2.
// TODO: Set the new configuration for the data directory / before
creating the new core.
SolrCore newCore = container.create(this.coreDescriptor);
container.register(this.coreName, newCore, false);
}
@Override
public void inform(SolrCore core)
{
coreDescriptor = core.getCoreDescriptor();
coreName = core.getName();
}
}
2) When a close() happens on an existing SolrCore - what happens when
there is a long running IndexReader query on that SolrCore . Is that
terminated abruptly / would the close wait until the IndexReaders
completes the Query.
* The same process is repeated potentially for multiple SolrCores as
well, with additional closeHooks that might do some heavy i/o tasks -
talking over the network etc.
Right now - these long running processes are done in an independent
thread so that they do not block SolrCore.close() with the currently
nightly builds.
