[jira] [Commented] (SOLR-9527) Solr RESTORE api doesn't distribute the replicas uniformly
[ https://issues.apache.org/jira/browse/SOLR-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004912#comment-16004912 ] ASF subversion and git services commented on SOLR-9527: --- Commit 201e2125f4db3b4fd1ecfe5ddef5471f305fc1f0 in lucene-solr's branch refs/heads/branch_6_6 from [~varunthacker] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=201e212 ] SOLR-9527: Improve distribution of replicas when restoring a collection > Solr RESTORE api doesn't distribute the replicas uniformly > --- > > Key: SOLR-9527 > URL: https://issues.apache.org/jira/browse/SOLR-9527 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.1 >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.6, master (7.0) > > Attachments: SOLR-9527.patch, SOLR-9527.patch, SOLR-9527.patch, > SOLR-9527.patch, SOLR-9527.patch, SOLR-9527.patch, Solr 9527.pdf > > > Please refer to this email thread for details, > http://lucene.markmail.org/message/ycun4x5nx7lwj5sk?q=solr+list:org%2Eapache%2Elucene%2Esolr-user+order:date-backward=1 -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9527) Solr RESTORE api doesn't distribute the replicas uniformly
[ https://issues.apache.org/jira/browse/SOLR-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004810#comment-16004810 ] Varun Thacker commented on SOLR-9527: - Thanks all for contributing! > Solr RESTORE api doesn't distribute the replicas uniformly > --- > > Key: SOLR-9527 > URL: https://issues.apache.org/jira/browse/SOLR-9527 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.1 >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.6, master (7.0) > > Attachments: SOLR-9527.patch, SOLR-9527.patch, SOLR-9527.patch, > SOLR-9527.patch, SOLR-9527.patch, SOLR-9527.patch, Solr 9527.pdf > > > Please refer to this email thread for details, > http://lucene.markmail.org/message/ycun4x5nx7lwj5sk?q=solr+list:org%2Eapache%2Elucene%2Esolr-user+order:date-backward=1 -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9527) Solr RESTORE api doesn't distribute the replicas uniformly
[ https://issues.apache.org/jira/browse/SOLR-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004807#comment-16004807 ] ASF subversion and git services commented on SOLR-9527: --- Commit 1adeebdd92a96233836f83f6144da9eead65a2cf in lucene-solr's branch refs/heads/branch_6x from [~varunthacker] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1adeebd ] SOLR-9527: Improve distribution of replicas when restoring a collection > Solr RESTORE api doesn't distribute the replicas uniformly > --- > > Key: SOLR-9527 > URL: https://issues.apache.org/jira/browse/SOLR-9527 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.1 >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.6, master (7.0) > > Attachments: SOLR-9527.patch, SOLR-9527.patch, SOLR-9527.patch, > SOLR-9527.patch, SOLR-9527.patch, SOLR-9527.patch, Solr 9527.pdf > > > Please refer to this email thread for details, > http://lucene.markmail.org/message/ycun4x5nx7lwj5sk?q=solr+list:org%2Eapache%2Elucene%2Esolr-user+order:date-backward=1 -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9527) Solr RESTORE api doesn't distribute the replicas uniformly
[ https://issues.apache.org/jira/browse/SOLR-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004794#comment-16004794 ] ASF subversion and git services commented on SOLR-9527: --- Commit b1efd37ba7b57cc01b3e2ac740b4f1d13fb86cd2 in lucene-solr's branch refs/heads/master from [~varunthacker] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b1efd37 ] SOLR-9527: Improve distribution of replicas when restoring a collection > Solr RESTORE api doesn't distribute the replicas uniformly > --- > > Key: SOLR-9527 > URL: https://issues.apache.org/jira/browse/SOLR-9527 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.1 >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Attachments: SOLR-9527.patch, SOLR-9527.patch, SOLR-9527.patch, > SOLR-9527.patch, SOLR-9527.patch, SOLR-9527.patch, Solr 9527.pdf > > > Please refer to this email thread for details, > http://lucene.markmail.org/message/ycun4x5nx7lwj5sk?q=solr+list:org%2Eapache%2Elucene%2Esolr-user+order:date-backward=1 -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9527) Solr RESTORE api doesn't distribute the replicas uniformly
[ https://issues.apache.org/jira/browse/SOLR-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002445#comment-16002445 ] Varun Thacker commented on SOLR-9527: - Hi Hrishikesh, Starting to look at it right now. > Solr RESTORE api doesn't distribute the replicas uniformly > --- > > Key: SOLR-9527 > URL: https://issues.apache.org/jira/browse/SOLR-9527 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.1 >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Attachments: SOLR-9527.patch, SOLR-9527.patch, SOLR-9527.patch, > SOLR-9527.patch, SOLR-9527.patch, Solr 9527.pdf > > > Please refer to this email thread for details, > http://lucene.markmail.org/message/ycun4x5nx7lwj5sk?q=solr+list:org%2Eapache%2Elucene%2Esolr-user+order:date-backward=1 -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9527) Solr RESTORE api doesn't distribute the replicas uniformly
[ https://issues.apache.org/jira/browse/SOLR-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999325#comment-15999325 ] Mikhail Khludnev commented on SOLR-9527: testRequestTimeUrp has been fixed. Just pull > Solr RESTORE api doesn't distribute the replicas uniformly > --- > > Key: SOLR-9527 > URL: https://issues.apache.org/jira/browse/SOLR-9527 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.1 >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Attachments: SOLR-9527.patch, SOLR-9527.patch, SOLR-9527.patch, > SOLR-9527.patch, SOLR-9527.patch, Solr 9527.pdf > > > Please refer to this email thread for details, > http://lucene.markmail.org/message/ycun4x5nx7lwj5sk?q=solr+list:org%2Eapache%2Elucene%2Esolr-user+order:date-backward=1 -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9527) Solr RESTORE api doesn't distribute the replicas uniformly
[
https://issues.apache.org/jira/browse/SOLR-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15981674#comment-15981674
]
Varun Thacker commented on SOLR-9527:
-
I reviewed the latest patch ( 6th April ) and here are a few things I believe
we should fix:
- Should we rename {{DocCollection#getReplicaCount}} to
{{DocCollection#getNodeCount}} ?
- RestoreCmd has an unused import. We should generally remove all unused
imports to make "ant precommit" happy
{code}
String createNodeArg = message.getStr(CREATE_NODE_SET, null);
if (createNodeArg == CREATE_NODE_SET_EMPTY) {
throw new SolrException(
SolrException.ErrorCode.BAD_REQUEST,
"Cannot restore with a CREATE_NODE_SET of CREATE_NODE_SET_EMPTY."
);
}
{code}
- Can we move this check from RestoreCmd to the CollectionsHandler#RESTORE_OP?
- In line 206 of RestoreCmd , shouldn't
{{int repFactor = backupCollectionState.getReplicaCount();}}
be
{{int repFactor = message.getInt(REPLICATION_FACTOR,
backupCollectionState.getReplicaCount());}}
- We should also allow users to pass {{createNodeSet.shuffle}}
> Solr RESTORE api doesn't distribute the replicas uniformly
> ---
>
> Key: SOLR-9527
> URL: https://issues.apache.org/jira/browse/SOLR-9527
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 6.1
>Reporter: Hrishikesh Gadre
>Assignee: Varun Thacker
> Attachments: SOLR-9527.patch, SOLR-9527.patch, SOLR-9527.patch,
> SOLR-9527.patch, Solr 9527.pdf
>
>
> Please refer to this email thread for details,
> http://lucene.markmail.org/message/ycun4x5nx7lwj5sk?q=solr+list:org%2Eapache%2Elucene%2Esolr-user+order:date-backward=1
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9527) Solr RESTORE api doesn't distribute the replicas uniformly
[ https://issues.apache.org/jira/browse/SOLR-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15979277#comment-15979277 ] Varun Thacker commented on SOLR-9527: - Thanks David for the review. I'll go through it and commit in on Monday > Solr RESTORE api doesn't distribute the replicas uniformly > --- > > Key: SOLR-9527 > URL: https://issues.apache.org/jira/browse/SOLR-9527 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.1 >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Attachments: SOLR-9527.patch, SOLR-9527.patch, SOLR-9527.patch, > SOLR-9527.patch, Solr 9527.pdf > > > Please refer to this email thread for details, > http://lucene.markmail.org/message/ycun4x5nx7lwj5sk?q=solr+list:org%2Eapache%2Elucene%2Esolr-user+order:date-backward=1 -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9527) Solr RESTORE api doesn't distribute the replicas uniformly
[ https://issues.apache.org/jira/browse/SOLR-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15979276#comment-15979276 ] David Smiley commented on SOLR-9527: +1 to the patch > Solr RESTORE api doesn't distribute the replicas uniformly > --- > > Key: SOLR-9527 > URL: https://issues.apache.org/jira/browse/SOLR-9527 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.1 >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Attachments: SOLR-9527.patch, SOLR-9527.patch, SOLR-9527.patch, > SOLR-9527.patch, Solr 9527.pdf > > > Please refer to this email thread for details, > http://lucene.markmail.org/message/ycun4x5nx7lwj5sk?q=solr+list:org%2Eapache%2Elucene%2Esolr-user+order:date-backward=1 -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9527) Solr RESTORE api doesn't distribute the replicas uniformly
[ https://issues.apache.org/jira/browse/SOLR-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15979271#comment-15979271 ] David Smiley commented on SOLR-9527: +1 to the patch > Solr RESTORE api doesn't distribute the replicas uniformly > --- > > Key: SOLR-9527 > URL: https://issues.apache.org/jira/browse/SOLR-9527 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.1 >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Attachments: SOLR-9527.patch, SOLR-9527.patch, SOLR-9527.patch, > SOLR-9527.patch, Solr 9527.pdf > > > Please refer to this email thread for details, > http://lucene.markmail.org/message/ycun4x5nx7lwj5sk?q=solr+list:org%2Eapache%2Elucene%2Esolr-user+order:date-backward=1 -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9527) Solr RESTORE api doesn't distribute the replicas uniformly
[ https://issues.apache.org/jira/browse/SOLR-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15979043#comment-15979043 ] Stephen Weiss commented on SOLR-9527: - Any chance we can get this patch merged in? We really need this issue fixed, and it's made upgrading a serious pain. Every new version we have to recompile our own Solr for this one little detail. If you need it to be tested... well... we've used it in production for months and it's great. What's actually in master is no good. Why didn't that need to be tested as much? > Solr RESTORE api doesn't distribute the replicas uniformly > --- > > Key: SOLR-9527 > URL: https://issues.apache.org/jira/browse/SOLR-9527 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.1 >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Attachments: SOLR-9527.patch, SOLR-9527.patch, SOLR-9527.patch, > SOLR-9527.patch, Solr 9527.pdf > > > Please refer to this email thread for details, > http://lucene.markmail.org/message/ycun4x5nx7lwj5sk?q=solr+list:org%2Eapache%2Elucene%2Esolr-user+order:date-backward=1 -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9527) Solr RESTORE api doesn't distribute the replicas uniformly
[ https://issues.apache.org/jira/browse/SOLR-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959792#comment-15959792 ] Rohit commented on SOLR-9527: - Trying to refine the replica placement to be more uniform across multiple nodes if we have sufficient amount of live_nodes in Solr available, 1. When we have live_nodes in excess of the replication factor which existed during the time to backup we can do away with creating multiple replicas for same shard on the same live_node to ensure better distribution. 2. User should be given warning or, option to go ahead with the Restore API if the number of live_nodes is less that what the collection was originally created. Details of the two issue in the attached PDF and initial patch attached with the ticket. > Solr RESTORE api doesn't distribute the replicas uniformly > --- > > Key: SOLR-9527 > URL: https://issues.apache.org/jira/browse/SOLR-9527 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.1 >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Attachments: SOLR-9527.patch, SOLR-9527.patch, SOLR-9527.patch, > SOLR-9527.patch, Solr 9527.pdf > > > Please refer to this email thread for details, > http://lucene.markmail.org/message/ycun4x5nx7lwj5sk?q=solr+list:org%2Eapache%2Elucene%2Esolr-user+order:date-backward=1 -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9527) Solr RESTORE api doesn't distribute the replicas uniformly
[ https://issues.apache.org/jira/browse/SOLR-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880507#comment-15880507 ] Dewald Viljoen commented on SOLR-9527: -- I've run smack dab straight into this issue recently. I was wondering what the progress is on this patch? I'm currently running Solr 6.4.1 and would really like to take advantage of the Collections Backup/Restore functionality in combination with HDFS. All works well until I restore the collection and all my shards end up on one of my SolrCloud nodes. I can specify a replicationFactor of 2 and then though some other API calls make the replica's the leaders and rebalance everything but it's a bit of a mess. I'm happy to lend my efforts to get this issue resolved. > Solr RESTORE api doesn't distribute the replicas uniformly > --- > > Key: SOLR-9527 > URL: https://issues.apache.org/jira/browse/SOLR-9527 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.1 >Reporter: Hrishikesh Gadre > Attachments: SOLR-9527.patch, SOLR-9527.patch, SOLR-9527.patch > > > Please refer to this email thread for details, > http://lucene.markmail.org/message/ycun4x5nx7lwj5sk?q=solr+list:org%2Eapache%2Elucene%2Esolr-user+order:date-backward=1 -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9527) Solr RESTORE api doesn't distribute the replicas uniformly
[ https://issues.apache.org/jira/browse/SOLR-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15648793#comment-15648793 ] Hrishikesh Gadre commented on SOLR-9527: Hi [~dsmiley] bq. I noticed this doesn't seem to be tested; right? I saw a test modification but it doesn't seem to test the distribution. I added assertion in the test to verify that the number of replicas per host are in-line with the expected value of max_shards_per_node param. I think this is sufficient to test the distribution (?). Although I have not added any test for "createNodeSet" functionality. Let me add that test as well. > Solr RESTORE api doesn't distribute the replicas uniformly > --- > > Key: SOLR-9527 > URL: https://issues.apache.org/jira/browse/SOLR-9527 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.1 >Reporter: Hrishikesh Gadre > Attachments: SOLR-9527.patch, SOLR-9527.patch, SOLR-9527.patch > > > Please refer to this email thread for details, > http://lucene.markmail.org/message/ycun4x5nx7lwj5sk?q=solr+list:org%2Eapache%2Elucene%2Esolr-user+order:date-backward=1 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9527) Solr RESTORE api doesn't distribute the replicas uniformly
[ https://issues.apache.org/jira/browse/SOLR-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15537773#comment-15537773 ] David Smiley commented on SOLR-9527: FYI When I originally worked on the backup capability, I didn't add support for createNodeSet because I wasn't yet sure how to resolve the possibility of duplicating logic with regular collection creation. I gave a cursory look at the patch here; it's interesting that the patch is actually kinda small. I should look at it more when I have time. I noticed this doesn't seem to be tested; right? I saw a test modification but it doesn't seem to test the distribution. > Solr RESTORE api doesn't distribute the replicas uniformly > --- > > Key: SOLR-9527 > URL: https://issues.apache.org/jira/browse/SOLR-9527 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.1 >Reporter: Hrishikesh Gadre > Attachments: SOLR-9527.patch, SOLR-9527.patch, SOLR-9527.patch > > > Please refer to this email thread for details, > http://lucene.markmail.org/message/ycun4x5nx7lwj5sk?q=solr+list:org%2Eapache%2Elucene%2Esolr-user+order:date-backward=1 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9527) Solr RESTORE api doesn't distribute the replicas uniformly
[ https://issues.apache.org/jira/browse/SOLR-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15537223#comment-15537223 ] Hrishikesh Gadre commented on SOLR-9527: [~varunthacker] [~dsmiley] Can you review this patch please? > Solr RESTORE api doesn't distribute the replicas uniformly > --- > > Key: SOLR-9527 > URL: https://issues.apache.org/jira/browse/SOLR-9527 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.1 >Reporter: Hrishikesh Gadre > Attachments: SOLR-9527.patch, SOLR-9527.patch, SOLR-9527.patch > > > Please refer to this email thread for details, > http://lucene.markmail.org/message/ycun4x5nx7lwj5sk?q=solr+list:org%2Eapache%2Elucene%2Esolr-user+order:date-backward=1 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
