[
https://issues.apache.org/jira/browse/SOLR-1306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erick Erickson updated SOLR-1306:
---------------------------------
Attachment: SOLR-1306.patch
MUST be applied _after_ SOLR-1028
OK, here's a preliminary cut at this, no tests yet, but I was looking at
logging and it seems to be doing what I want, putting up for inspection by the
curious...
A couple of notes:
1> It turns out that to make this work I needed to incorporate SOLR-4013 and
SOLR-3980. I'd appreciate anyone looking at the synchronization I did around
the member variable "loadingCores". The intent here is to keep two threads from
creating the same core at the same time.
1a> I'm assuming that there is exactly one CoreContainer per JVM. Otherwise I
don't understand how any of the synchronization works on the member vars....
1b> Running a simple test things went all to hell in JMX stuff without the
synchronization, apparently the multiple thread problem shows up early and
often.
1c> synchronization is always "interesting", so the more eyes the better.
1d> In particular, any good suggestions about bailing out of the sleep loop?
Since cores can take quite a while to warm, I'm having a hard time thinking of
a good default. I suppose another attribute where the provider is mentioned.
There's no reason a custom provider has to be present, so requiring a timeout
from the provider doesn't seem workable.
2> I've implemented a trivial CoreDescriptorProvider for a PoC, it's at the
bottom. It pre-supposes you have 4 collections, the accompanying Solr.xml is
also below.
3> I'm going to put this away for a couple of hours and come back to it with
fresh eyes, this copy is purely for the curious/critical...
*****sample custom descriptor provider
public class TestCoreContainerProvider implements CoreDescriptorProvider
{
@Override
public CoreDescriptor getDescriptor(CoreContainer container, String name) {
if (! "collection2".equals(name) && ! "collection3".equals(name) && !
"collection4".equals(name)) return null;
CoreDescriptor cd = new CoreDescriptor(container, name, name); // True hack
because I know the dirs are the same as the collection.
return cd;
}
@Override
public boolean isPersist(String s) {
return false;
}
@Override
public Collection<String> getCoreNames() {
return new ArrayList<String>(Arrays.asList("collection2", "collection3",
"collection4"));
}
}
**** solr.xml. Note no ZK stuff.
<solr persistent="false"
sharedLib="../../../../../blahblah/out/artifacts/provider_jar">
<cores adminPath="/admin/cores" defaultCoreName="collection1" host="${host:}"
hostPort="${jetty.port:}" hostContext="${hostContext:}"
zkClientTimeout="${zkClientTimeout:15000}"
coreDescriptorProviderClass="blah.TestCoreContainerProvider">
<core name="collection1" instanceDir="collection1" />
</cores>
</solr>
> Support pluggable persistence/loading of solr.xml details
> ---------------------------------------------------------
>
> Key: SOLR-1306
> URL: https://issues.apache.org/jira/browse/SOLR-1306
> Project: Solr
> Issue Type: New Feature
> Components: multicore
> Reporter: Noble Paul
> Assignee: Erick Erickson
> Fix For: 4.1
>
> Attachments: SOLR-1306.patch
>
>
> Persisting and loading details from one xml is fine if the no:of cores are
> small and the no:of cores are few/fixed . If there are 10's of thousands of
> cores in a single box adding a new core (with persistent=true) becomes very
> expensive because every core creation has to write this huge xml.
> Moreover , there is a good chance that the file gets corrupted and all the
> cores become unusable . In that case I would prefer it to be stored in a
> centralized DB which is backed up/replicated and all the information is
> available in a centralized location.
> We may need to refactor CoreContainer to have a pluggable implementation
> which can load/persist the details . The default implementation should
> write/read from/to solr.xml . And the class should be pluggable as follows in
> solr.xml
> {code:xml}
> <solr>
> <dataProvider class="com.foo.FooDataProvider" attr1="val1" attr2="val2"/>
> </solr>
> {code}
> There will be a new interface (or abstract class ) called SolrDataProvider
> which this class must implement
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
