[jira] [Commented] (SOLR-2593) A new core admin command 'split' for splitting index
[ https://issues.apache.org/jira/browse/SOLR-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050236#comment-13050236 ] Noble Paul commented on SOLR-2593: -- bq. the fq type option is basically going to require making a full copy of hte index and then deleting by query... Lucene does it better. We can pass a Filtered Index to a new writer and it creates a new index w/ only those docs. I was surprised at the speed at which it split a dummy 1million doc index in 1 sec A new core admin command 'split' for splitting index Key: SOLR-2593 URL: https://issues.apache.org/jira/browse/SOLR-2593 Project: Solr Issue Type: New Feature Reporter: Noble Paul Fix For: 4.0 If an index is too large/hot it would be desirable to split it out to another core . This core may eventually be replicated out to another host. There can be to be multiple strategies * random split of x or x% * fq=user:johndoe example example : command=splitsplit=20percentnewcore=my_new_index or command=splitfq=user:johndoenewcore=john_doe_index -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2593) A new core admin command 'split' for splitting index
[ https://issues.apache.org/jira/browse/SOLR-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13049713#comment-13049713 ] Koji Sekiguchi commented on SOLR-2593: -- CoreAdminHandler uses action, not command. A new core admin command 'split' for splitting index Key: SOLR-2593 URL: https://issues.apache.org/jira/browse/SOLR-2593 Project: Solr Issue Type: New Feature Reporter: Noble Paul Fix For: 4.0 If an index is too large/hot it would be desirable to split it out to another core . This core may eventually be replicated out to another host. There can be to be multiple strategies * random split of x or x% * fq=user:johndoe example example : command=splitsplit=20percentnewcore=my_new_index or command=splitfq=user:johndoenewcore=john_doe_index -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2593) A new core admin command 'split' for splitting index
[ https://issues.apache.org/jira/browse/SOLR-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13049727#comment-13049727 ] Peter Sturge commented on SOLR-2593: This is a really great idea, thanks! If it's possible, it would be cool to have config parameters to: create a new core overwrite an existing core rename an existing core, then create (rolling backup) merge with an existing core (ever-growing, but kind of an accessible 'archive' index) A new core admin command 'split' for splitting index Key: SOLR-2593 URL: https://issues.apache.org/jira/browse/SOLR-2593 Project: Solr Issue Type: New Feature Reporter: Noble Paul Fix For: 4.0 If an index is too large/hot it would be desirable to split it out to another core . This core may eventually be replicated out to another host. There can be to be multiple strategies * random split of x or x% * fq=user:johndoe example example : command=splitsplit=20percentnewcore=my_new_index or command=splitfq=user:johndoenewcore=john_doe_index -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2593) A new core admin command 'split' for splitting index
[ https://issues.apache.org/jira/browse/SOLR-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050137#comment-13050137 ] Hoss Man commented on SOLR-2593: bq. If it's possible, it would be cool to have config parameters to: ...those seem like they should be discrete actions that can be taken after the split has happened. the simplest thing is to have a split action that _just_ creates a new core with the docs selected either using the fq (or randomly selection) and then use other CoreAdmin actions for the other stuff: rename, swap, swap+delete (the old one), merge ... merge is really the only one we don't have at a core level yet (i think) A new core admin command 'split' for splitting index Key: SOLR-2593 URL: https://issues.apache.org/jira/browse/SOLR-2593 Project: Solr Issue Type: New Feature Reporter: Noble Paul Fix For: 4.0 If an index is too large/hot it would be desirable to split it out to another core . This core may eventually be replicated out to another host. There can be to be multiple strategies * random split of x or x% * fq=user:johndoe example example : command=splitsplit=20percentnewcore=my_new_index or command=splitfq=user:johndoenewcore=john_doe_index -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2593) A new core admin command 'split' for splitting index
[ https://issues.apache.org/jira/browse/SOLR-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050139#comment-13050139 ] Hoss Man commented on SOLR-2593: one thing to think about when talking about the API is how the implementation will actually work. the fq type option is basically going to require making a full copy of hte index and then deleting by query. (unless i'm missing something) but for people who don't care how the index is partitioned a more efficient approach could probably happen by working at the segment level -- let the user say split off a hunk of at least 20% but no more then 50% and then you can look at individual segments and doc counts and see if it's possible to just move segments around (and maybe only do the copy+deleteByQuery logic on a single segment. A new core admin command 'split' for splitting index Key: SOLR-2593 URL: https://issues.apache.org/jira/browse/SOLR-2593 Project: Solr Issue Type: New Feature Reporter: Noble Paul Fix For: 4.0 If an index is too large/hot it would be desirable to split it out to another core . This core may eventually be replicated out to another host. There can be to be multiple strategies * random split of x or x% * fq=user:johndoe example example : command=splitsplit=20percentnewcore=my_new_index or command=splitfq=user:johndoenewcore=john_doe_index -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
