[
https://issues.apache.org/jira/browse/SOLR-5316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14261477#comment-14261477
]
Erick Erickson commented on SOLR-5316:
--------------------------------------
Olivier:
I'm really sorry it's taken me this long to look more closely, well the heck
with the excuses...
Anyway, I think I'm going to pass on committing this one. It's not a problem
with the code, I looked it over (although not in excruciating detail), and it
looks quite good.
Rather, I think it introduces more complexity than I'm comfortable introducing
at this point for a not-very-common use-case AFAIK. Especially as Zookeeper as
"the one source of truth" is out there somewhere on the horizon.
So I'm making it unassigned, perhaps someone else might want to pick this up.
> Solr 4.2.1 LotsOfCores new options
> ----------------------------------
>
> Key: SOLR-5316
> URL: https://issues.apache.org/jira/browse/SOLR-5316
> Project: Solr
> Issue Type: Improvement
> Components: multicore
> Affects Versions: 4.2.1, 4.7
> Reporter: olivier soyez
> Assignee: Erick Erickson
> Priority: Minor
> Labels: patch
> Fix For: 4.2.1
>
> Attachments: SOLR-5316.patch, solr-4.2.1.patchLOTSOFCORES
>
>
> The SOLR-5316.patch is for the lotsofcores use case: a large number of
> homogeneous cores with frequent loading/unloading. The new Cores options are:
> - defaultLoadOnStartup : true by default
> - defaultTransient : false by default
> In this way, you can change the default attributes value for all cores.
> The lotsofcores use case requires to have defaultLoadOnStartup = false and
> defaultTransient = true
> - "numBuckets" to create a subdirectory based on a hash on the corename %
> numBuckets in the core Datadir, because all cores cannot live in the same
> directory
> - "auto" with 2 differents values :
> 1) createLoad: create, if not exist, and load the core on the fly on
> the first incoming request (update, select).
> 2) onlyLoad: load the core on the fly on the first incoming request
> (update, select), if exist on disk
> The auto option uses an additional cores option named baseDataDir to
> automatically generate the dataDir of each core and uses the "numBucket"
> option if exists. All the cores have the same solr config defined in the
> solr_home/conf.
> - discoverOnDemand : false by default. If enabled, the discovery and loading
> of cores at the solr startup are disabled and a new option named
> coreDiscovery in the STATUS action is available to discover on demand all the
> cores in the coreRootDirectory (only add the cores descriptions)
> - noCorePropertiesFile : false by default. If enabled, don't use anymore the
> core.properties file, a core will be detected on disk based on the file
> segments.gen
> The unload command was also modified in order to handle the non loaded
> transient cores, to be able to apply for example the deleteIndex,
> deleteDataDir options.
> I added some Junit tests.
> An example of solr.xml:
> <solr>
> <str name="shareSchema">schema.xml</str>
> <int name="transientCacheSize">5000</int>
> <str name="coreRootDirectory">solr_home/data</str>
> <str name="baseDataDir">solr_home/data</str>
> <int name="numBuckets">100</int>
> <str name="auto">createLoad</str>
> <str name="defaultLoadOnStartup">false</str>
> <str name="defaultTransient">true</str>
> <bool name="discoverOnDemand">true</bool>
> <bool name="noCorePropertiesFile">true</bool>
> </solr>
> The patch SOLR-5316.patch in attachment is for svn solr branch_4X (revision
> number 1556554)
> ---------------
> The patch solr-4.2.1.patchLOTSOFCORES is for the lotsofcores use case,
> including some modification :
> - by default, all cores have loadOnStartup="false" and transient="true"
> attributes
> - the create admin command can register a lazy core (to take into account the
> transientCacheSize option)
> - add transient cores persistency
> - handle unload admin command for never launched transient cores (non active
> cores)
> To improve performance, we use this Solr patched version with the persistence
> disabled.
> In this way, Solr is working with a solr.xml file without any core entries,
> because it's useless in our use case (with the new Auto option for the cores)
> The new Cores options :
> - "numBuckets" to create a subdirectory based on a hash on the corename
> % numBuckets in the core Datadir, because all cores cannot live in the
> same directory
> - "Auto" with 3 differents values :
> 1) false : default behaviour
> 2) createLoad : create, if not exist, and load the core on the fly on
> the first incoming request (update, select).
> 3) onlyLoad : load the core on the fly on the first incoming request
> (update, select), if exist on disk
> The Auto option uses an additional cores option named baseDataDir to
> automatically generate the dataDir of each core and uses the "numBucket"
> option if exists.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
