[
https://issues.apache.org/jira/browse/SOLR-7789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14704171#comment-14704171
]
Gregory Chanan commented on SOLR-7789:
--------------------------------------
Here is a patch that implements a ConfigSets API for SolrCloud with two entry
points:
- CREATE: Create a ConfigSet, based on an existing ConfigSet
Parameters:
||Key||Type||Required||Default||Description||
|name|String|Yes| |ConfigSet to be created|
|baseConfigSet|String|Yes| |ConfigSet to copy as a base|
|configSetProp._name=value_|String|No| |ConfigSet property from base to
override|
- DELETE: Delete a ConfigSet
Parameters:
||Key||Type||Required||Default||Description||
|name|String|Yes| |ConfigSet to be deleted|
The implementation is based on the Collections API and in fact uses the same
OverseerProcessor at this time. The handling code between the ConfigSets API
and the CollectionsAPI is completely separate so it should be easy to move to
its own OverseerProcessor in the future. Using one OverseerProcessor seemed
reasonable given the above operations should be fast.
Some notes:
1) DELETE does not check that a ConfigSet is not referenced. This is
equivalent to the existing method of deleting a ConfigSet via the zkcli. The
exception to this rule is with task exclusivity: a ConfigSet deletion won't go
through if there is an on going operation referencing that ConfigSet (e.g. as a
Base ConfigSet in a create). There is a bit of complexity in implementing that
efficiently (simulating a read/write lock for exclusivity so we don't block on
multiple ConfigSets being reated with the same Base ConfigSet).
2) The name for the ConfigSet properties file is always assumed to be
"configsetprops.json". This is necessary to handle deletions correctly; if we
allowed the user to specify in the API call what the file was called they could
point at another or non-existing file to be able to avoid the immutable check.
This is a little strange because the CoreDescriptor lets you override the name
of this file, but I guess this is equivalent to solrconfig.xml and schema.xml
being special in ZK even though they can also be overriden in the
CoreDescriptor.
3) The handling of the Overseer.getConfigSetQueue isn't as nice as I would
like. Because we use the same OverseerProcessor, we need some way of figuring
out which MessageHandler to dispatch to; we do this by prefixing the action in
ZK with "configsets:". Both the ConfigSetsHandler and the
OverseerConfigSetsMessageHandler need to be aware of this. Ideally, the queue
returned from Overseer.getConfigSetQueue would just handle this when an item
was inserted into the queue, so only the Overseer has to worry about how the
messages are actually handled. This was difficult to do given the
DistributedQueue API is just byte arrays. I think the correct thing to do here
is change at least the offer API to be JSON based -- all the non-test calls
already immediately convert JSON to bytes anyway -- but this seemed like a
larger change for this patch.
Some testing notes:
T1) I wanted to be able to test the case where the CREATE failed half way
through, so we end up with partial state in ZK that needed to be cleaned up. I
found the easiest way to do this was to override the ZKDatabase that the
ZKTestServer uses. So, you can now get/set the ZKDatabase in the ZKTestServer
and pass your own ZKTestServer to the MiniSolrCloudCluster.
T2) There's also an "exclusivity" test to ensure that actions on the same
ConfigSet don't trample each other. This is done by concurrently doing
DELETEs/CREATEs and ensuring the exceptions indicate that the operations ran
sequentially.
> Introduce a ConfigSet management API
> ------------------------------------
>
> Key: SOLR-7789
> URL: https://issues.apache.org/jira/browse/SOLR-7789
> Project: Solr
> Issue Type: New Feature
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: SOLR-7789.patch
>
>
> SOLR-5955 describes a feature to automatically create a ConfigSet, based on
> another one, from a collection API call (i.e. one step collection creation).
> Discussion there yielded SOLR-7742, Immutable ConfigSet support. To close
> the loop, we need support for a ConfigSet management API.
> The simplest ConfigSet API could have one operation:
> create a new config set, based on an existing one, possible modifying the
> ConfigSet properties. Note you need to be able to modify the ConfigSet
> properties at creation time because otherwise Immutable could not be changed.
> Another logical operation to support is ConfigSet deletion; that may be more
> complicated to implement than creation because you need to handle the case
> where a collection is already using the configuration.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
