[jira] [Updated] (SOLR-5308) Split all documents of a route key into another collection
[ https://issues.apache.org/jira/browse/SOLR-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-5308: Attachment: SOLR-5308-bitsep-fix.patch This patch fixes two bugs: # If migrate was invoked with a split.key containing a bit separator then the forwarding logic didn't kick in # When forwarding request to target collection, an intermediate forwarding node would cause the distrib.from.shard and distrib.from.collection params to be removed and the request would be rejected by the target shard. Now requests are routed directly to target shard instead of random assignment. Split all documents of a route key into another collection -- Key: SOLR-5308 URL: https://issues.apache.org/jira/browse/SOLR-5308 Project: Solr Issue Type: New Feature Components: SolrCloud Reporter: Shalin Shekhar Mangar Assignee: Shalin Shekhar Mangar Fix For: 5.0, 4.7 Attachments: SOLR-5308-bitsep-fix.patch, SOLR-5308-fixes.patch, SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch Enable SolrCloud users to split out a set of documents from a source collection into another collection. This will be useful in multi-tenant environments. This feature will make it possible to split a tenant out of a collection and put them into their own collection which can be scaled separately. -- This message was sent by Atlassian JIRA (v6.1.4#6159) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5308) Split all documents of a route key into another collection
[ https://issues.apache.org/jira/browse/SOLR-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-5308: Attachment: SOLR-5308-fixes.patch This patch fixes three bugs: # No collection.configName was specified while creating the temporary collection. This caused it to fail when multiple config sets exist and the name of the config set is different from the collection name. I had to move the readConfigName method from ZkController to ZkStateReader because the OverseerCollectionProcessor does not have access to the ZkController. # Fixed a NPE during the routing rule wait loop # Fixed a race condition where the split is performed before the temp collection leader becomes active. In this patch we wait for the leader to become active before invoking split. Split all documents of a route key into another collection -- Key: SOLR-5308 URL: https://issues.apache.org/jira/browse/SOLR-5308 Project: Solr Issue Type: New Feature Components: SolrCloud Reporter: Shalin Shekhar Mangar Assignee: Shalin Shekhar Mangar Fix For: 5.0, 4.7 Attachments: SOLR-5308-fixes.patch, SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch Enable SolrCloud users to split out a set of documents from a source collection into another collection. This will be useful in multi-tenant environments. This feature will make it possible to split a tenant out of a collection and put them into their own collection which can be scaled separately. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5308) Split all documents of a route key into another collection
[ https://issues.apache.org/jira/browse/SOLR-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-5308: Attachment: SOLR-5308.patch # Added routing rule expiry and test to assert behaviour. # Added a forward.timeout parameter to specify the number of seconds after which a routing rule is expired. The default is 10 minutes. Split all documents of a route key into another collection -- Key: SOLR-5308 URL: https://issues.apache.org/jira/browse/SOLR-5308 Project: Solr Issue Type: New Feature Components: SolrCloud Reporter: Shalin Shekhar Mangar Assignee: Shalin Shekhar Mangar Fix For: 4.6, 5.0 Attachments: SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch Enable SolrCloud users to split out a set of documents from a source collection into another collection. This will be useful in multi-tenant environments. This feature will make it possible to split a tenant out of a collection and put them into their own collection which can be scaled separately. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5308) Split all documents of a route key into another collection
[ https://issues.apache.org/jira/browse/SOLR-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-5308: Attachment: SOLR-5308.patch Removed unnecessary logging added to aid in debugging. Split all documents of a route key into another collection -- Key: SOLR-5308 URL: https://issues.apache.org/jira/browse/SOLR-5308 Project: Solr Issue Type: New Feature Components: SolrCloud Reporter: Shalin Shekhar Mangar Assignee: Shalin Shekhar Mangar Fix For: 4.6, 5.0 Attachments: SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch Enable SolrCloud users to split out a set of documents from a source collection into another collection. This will be useful in multi-tenant environments. This feature will make it possible to split a tenant out of a collection and put them into their own collection which can be scaled separately. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5308) Split all documents of a route key into another collection
[ https://issues.apache.org/jira/browse/SOLR-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-5308: Attachment: SOLR-5308.patch Changes: # Added add request forwarding to target collection. The incoming request on a target collection is handled according to the state of the node. If the update log is in buffering mode then the request is buffered otherwise the version set by the source leader is stripped and leader logic is invoked. # Added a test with request forwarding I'm still working on forwarding delete requests, remove routing rules after expiry and adding more/better tests. Split all documents of a route key into another collection -- Key: SOLR-5308 URL: https://issues.apache.org/jira/browse/SOLR-5308 Project: Solr Issue Type: New Feature Components: SolrCloud Reporter: Shalin Shekhar Mangar Assignee: Shalin Shekhar Mangar Fix For: 4.6, 5.0 Attachments: SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch Enable SolrCloud users to split out a set of documents from a source collection into another collection. This will be useful in multi-tenant environments. This feature will make it possible to split a tenant out of a collection and put them into their own collection which can be scaled separately. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5308) Split all documents of a route key into another collection
[ https://issues.apache.org/jira/browse/SOLR-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-5308: Attachment: SOLR-5308.patch This patch adds request forwarding for delete by ID and query requests. Split all documents of a route key into another collection -- Key: SOLR-5308 URL: https://issues.apache.org/jira/browse/SOLR-5308 Project: Solr Issue Type: New Feature Components: SolrCloud Reporter: Shalin Shekhar Mangar Assignee: Shalin Shekhar Mangar Fix For: 4.6, 5.0 Attachments: SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch Enable SolrCloud users to split out a set of documents from a source collection into another collection. This will be useful in multi-tenant environments. This feature will make it possible to split a tenant out of a collection and put them into their own collection which can be scaled separately. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5308) Split all documents of a route key into another collection
[
https://issues.apache.org/jira/browse/SOLR-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shalin Shekhar Mangar updated SOLR-5308:
Attachment: SOLR-5308.patch
Thanks Noble. That is certainly simpler. This patch has it as:
{code}
routingRules:{a!:{
routeRanges:3c25-3c25,
expireAt:1382535453866,
targetCollection:migrate_routekey_test_targetCollection}}
{code}
I won't keep the routeRanges as a json list but as a comma-separated string.
Split all documents of a route key into another collection
--
Key: SOLR-5308
URL: https://issues.apache.org/jira/browse/SOLR-5308
Project: Solr
Issue Type: New Feature
Components: SolrCloud
Reporter: Shalin Shekhar Mangar
Assignee: Shalin Shekhar Mangar
Fix For: 4.6, 5.0
Attachments: SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch,
SOLR-5308.patch
Enable SolrCloud users to split out a set of documents from a source
collection into another collection.
This will be useful in multi-tenant environments. This feature will make it
possible to split a tenant out of a collection and put them into their own
collection which can be scaled separately.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5308) Split all documents of a route key into another collection
[
https://issues.apache.org/jira/browse/SOLR-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shalin Shekhar Mangar updated SOLR-5308:
Attachment: SOLR-5308.patch
# Added a simple test that tests for migration of a route key from a static
index (i.e. no updates in progress)
# Cleaning up existing temporary collections was broken in the last patch. I
fixed it.
I started working on forwarding requests in this patch. A RoutingRule is added
to the clusterstate.json inside the source shard. Such a rule may be used by
the DistributedUpdateProcessor to forward requests from source shard to target
collection. Example below:
{code}
shard2:{
range:0-7fff,
state:active,
replicas:{
core_node2:{
state:active,
base_url:http://127.0.0.1:51321/ckvmo/du;,
core:collection1,
node_name:127.0.0.1:51321_ckvmo%2Fdu,
leader:true},
core_node4:{
state:active,
base_url:http://127.0.0.1:51328/ckvmo/du;,
core:collection1,
node_name:127.0.0.1:51328_ckvmo%2Fdu}},
routingRules:{a!:[{
targetShard:shard1,
routeRange:3c25-3c25,
expireAt:1382449837040,
targetCollection:migrate_routekey_test_targetCollection,
routeKey:a!}]}}}
{code}
The expireAt is hard coded to System.currentTimeMillis() + 10 minutes but may
be passed as a parameter in future. The intent is to forward update requests
for the given route key and the given hash range from a source shard to the
target collection (target shard doesn't matter) temporarily i.e. during the
migration and for some time afterwards. This is still a work in progress.
Split all documents of a route key into another collection
--
Key: SOLR-5308
URL: https://issues.apache.org/jira/browse/SOLR-5308
Project: Solr
Issue Type: New Feature
Components: SolrCloud
Reporter: Shalin Shekhar Mangar
Assignee: Shalin Shekhar Mangar
Fix For: 4.6, 5.0
Attachments: SOLR-5308.patch, SOLR-5308.patch, SOLR-5308.patch
Enable SolrCloud users to split out a set of documents from a source
collection into another collection.
This will be useful in multi-tenant environments. This feature will make it
possible to split a tenant out of a collection and put them into their own
collection which can be scaled separately.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5308) Split all documents of a route key into another collection
[ https://issues.apache.org/jira/browse/SOLR-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-5308: Attachment: SOLR-5308.patch # Added a REQUESTBUFFERUPDATES action in Core Admin API # Use the above to ask target shard to buffer updates # Call REQUESTAPPLYUPDATES action to replay buffered updates after index merge I'm still working through the nocommits. Split all documents of a route key into another collection -- Key: SOLR-5308 URL: https://issues.apache.org/jira/browse/SOLR-5308 Project: Solr Issue Type: New Feature Components: SolrCloud Reporter: Shalin Shekhar Mangar Assignee: Shalin Shekhar Mangar Fix For: 4.6, 5.0 Attachments: SOLR-5308.patch, SOLR-5308.patch Enable SolrCloud users to split out a set of documents from a source collection into another collection. This will be useful in multi-tenant environments. This feature will make it possible to split a tenant out of a collection and put them into their own collection which can be scaled separately. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5308) Split all documents of a route key into another collection
[ https://issues.apache.org/jira/browse/SOLR-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-5308: Attachment: SOLR-5308.patch This patch implements the MIGRATE Collection API # Identifies the source and target shards for the given split.key. The for each pair of (source,target) ## Calculates common hash range by intersecting hash range of source shard, target shard and split.key's hash range ## Creates a temporary collection with just one replica on the source shard leader node ## Splits the source shard using the common hash range and split.key into the temporary collection ## Creates a replica of the temporary collection on the target shard leader's node ## Waits for it to become active ## Merges the replica's index into the target shard leader's index This is a rough cut to show where we're going with this. There are plenty of nocommits and todos. I still need to figure out how the request forwarding should work and if/how we can make the replica recovery asynchronous as we did for the split shard API. Split all documents of a route key into another collection -- Key: SOLR-5308 URL: https://issues.apache.org/jira/browse/SOLR-5308 Project: Solr Issue Type: New Feature Components: SolrCloud Reporter: Shalin Shekhar Mangar Assignee: Shalin Shekhar Mangar Fix For: 4.6, 5.0 Attachments: SOLR-5308.patch Enable SolrCloud users to split out a set of documents from a source collection into another collection. This will be useful in multi-tenant environments. This feature will make it possible to split a tenant out of a collection and put them into their own collection which can be scaled separately. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5308) Split all documents of a route key into another collection
[ https://issues.apache.org/jira/browse/SOLR-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-5308: Attachment: (was: SOLR-5308.patch) Split all documents of a route key into another collection -- Key: SOLR-5308 URL: https://issues.apache.org/jira/browse/SOLR-5308 Project: Solr Issue Type: New Feature Components: SolrCloud Reporter: Shalin Shekhar Mangar Assignee: Shalin Shekhar Mangar Fix For: 4.6, 5.0 Enable SolrCloud users to split out a set of documents from a source collection into another collection. This will be useful in multi-tenant environments. This feature will make it possible to split a tenant out of a collection and put them into their own collection which can be scaled separately. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5308) Split all documents of a route key into another collection
[ https://issues.apache.org/jira/browse/SOLR-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-5308: Description: Enable SolrCloud users to split out a set of documents from a source collection into another collection. This will be useful in multi-tenant environments. This feature will make it possible to split a tenant out of a collection and put them into their own collection which can be scaled separately. was: Enable SolrCloud users to split out a set of documents from a single shard into a shard in another collection. This will be useful in multi-tenant environments. This feature will make it possible to split a tenant out of a shard and put it into its own collection which can be scaled separately. Summary: Split all documents of a route key into another collection (was: Split a shard into a shard in another collection) Split all documents of a route key into another collection -- Key: SOLR-5308 URL: https://issues.apache.org/jira/browse/SOLR-5308 Project: Solr Issue Type: New Feature Components: SolrCloud Reporter: Shalin Shekhar Mangar Assignee: Shalin Shekhar Mangar Fix For: 4.6, 5.0 Attachments: SOLR-5308.patch Enable SolrCloud users to split out a set of documents from a source collection into another collection. This will be useful in multi-tenant environments. This feature will make it possible to split a tenant out of a collection and put them into their own collection which can be scaled separately. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
