[ https://issues.apache.org/jira/browse/SOLR-5316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13903397#comment-13903397 ]
olivier soyez commented on SOLR-5316: ------------------------------------- I have run "ant precommit" after the changes in SOLR-5316.patch, and it's OK > Solr 4.2.1 LotsOfCores new options > ---------------------------------- > > Key: SOLR-5316 > URL: https://issues.apache.org/jira/browse/SOLR-5316 > Project: Solr > Issue Type: Improvement > Components: multicore > Affects Versions: 4.2.1, 4.7 > Reporter: olivier soyez > Assignee: Erick Erickson > Priority: Minor > Labels: patch > Fix For: 4.2.1 > > Attachments: SOLR-5316.patch, solr-4.2.1.patchLOTSOFCORES > > > The SOLR-5316.patch is for the lotsofcores use case: a large number of > homogeneous cores with frequent loading/unloading. The new Cores options are: > - defaultLoadOnStartup : true by default > - defaultTransient : false by default > In this way, you can change the default attributes value for all cores. > The lotsofcores use case requires to have defaultLoadOnStartup = false and > defaultTransient = true > - "numBuckets" to create a subdirectory based on a hash on the corename % > numBuckets in the core Datadir, because all cores cannot live in the same > directory > - "auto" with 2 differents values : > 1) createLoad: create, if not exist, and load the core on the fly on > the first incoming request (update, select). > 2) onlyLoad: load the core on the fly on the first incoming request > (update, select), if exist on disk > The auto option uses an additional cores option named baseDataDir to > automatically generate the dataDir of each core and uses the "numBucket" > option if exists. All the cores have the same solr config defined in the > solr_home/conf. > - discoverOnDemand : false by default. If enabled, the discovery and loading > of cores at the solr startup are disabled and a new option named > coreDiscovery in the STATUS action is available to discover on demand all the > cores in the coreRootDirectory (only add the cores descriptions) > - noCorePropertiesFile : false by default. If enabled, don't use anymore the > core.properties file, a core will be detected on disk based on the file > segments.gen > The unload command was also modified in order to handle the non loaded > transient cores, to be able to apply for example the deleteIndex, > deleteDataDir options. > I added some Junit tests. > An example of solr.xml: > <solr> > <str name="shareSchema">schema.xml</str> > <int name="transientCacheSize">5000</int> > <str name="coreRootDirectory">solr_home/data</str> > <str name="baseDataDir">solr_home/data</str> > <int name="numBuckets">100</int> > <str name="auto">createLoad</str> > <str name="defaultLoadOnStartup">false</str> > <str name="defaultTransient">true</str> > <bool name="discoverOnDemand">true</bool> > <bool name="noCorePropertiesFile">true</bool> > </solr> > The patch SOLR-5316.patch in attachment is for svn solr branch_4X (revision > number 1556554) > --------------- > The patch solr-4.2.1.patchLOTSOFCORES is for the lotsofcores use case, > including some modification : > - by default, all cores have loadOnStartup="false" and transient="true" > attributes > - the create admin command can register a lazy core (to take into account the > transientCacheSize option) > - add transient cores persistency > - handle unload admin command for never launched transient cores (non active > cores) > To improve performance, we use this Solr patched version with the persistence > disabled. > In this way, Solr is working with a solr.xml file without any core entries, > because it's useless in our use case (with the new Auto option for the cores) > The new Cores options : > - "numBuckets" to create a subdirectory based on a hash on the corename > % numBuckets in the core Datadir, because all cores cannot live in the > same directory > - "Auto" with 3 differents values : > 1) false : default behaviour > 2) createLoad : create, if not exist, and load the core on the fly on > the first incoming request (update, select). > 3) onlyLoad : load the core on the fly on the first incoming request > (update, select), if exist on disk > The Auto option uses an additional cores option named baseDataDir to > automatically generate the dataDir of each core and uses the "numBucket" > option if exists. -- This message was sent by Atlassian JIRA (v6.1.5#6160) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org