[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15705501#comment-15705501 ] Mark Miller commented on SOLR-7280: --- It would seem the next step is probably to make the

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15705464#comment-15705464 ] Mark Miller commented on SOLR-7280: --- Hey [~dk], could you file a new JIRA issue with this info and link

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704322#comment-15704322 ] Damien Kamerman commented on SOLR-7280: --- I've just been testing this and found a couple of issues:

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392129#comment-15392129 ] Erick Erickson commented on SOLR-7280: -- Do note that all of my (non jUnit) tests were with 4 nodes. Of

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15391586#comment-15391586 ] Noble Paul commented on SOLR-7280: -- the zk register threads are still unbounded and that is done

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15390820#comment-15390820 ] Mark Miller commented on SOLR-7280: --- bq. But it sounds like you're getting more comfortable with the

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15390819#comment-15390819 ] Mark Miller commented on SOLR-7280: --- You only need one node because this issue is about loading cores on

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15390812#comment-15390812 ] Noble Paul commented on SOLR-7280: -- [~md...@cloudera.com] why do you want to test with

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15389669#comment-15389669 ] Erick Erickson commented on SOLR-7280: -- Tests ran find last night FWIW... > Load cores in sorted

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388992#comment-15388992 ] Noble Paul commented on SOLR-7280: -- It should work even if there is only one thread and many replicas. The

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388541#comment-15388541 ] Erick Erickson commented on SOLR-7280: -- Well, I'll still test it for fun, if _you're_worried I should

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388538#comment-15388538 ] Mike Drob commented on SOLR-7280: - If anybody wants to apply that, they should also add a call to

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388456#comment-15388456 ] Mark Miller commented on SOLR-7280: --- Okay, I think I got it. At some point after the bug I'm referring

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388442#comment-15388442 ] Mark Miller commented on SOLR-7280: --- [~mdrob] made a nice little test for this and the initial results

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388376#comment-15388376 ] Mark Miller commented on SOLR-7280: --- bq. Plus I'll vary the order of bring the JVMs up and loop all night

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388349#comment-15388349 ] Mark Miller commented on SOLR-7280: --- If it indeed remains an issue, another thing we might try is loading

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388323#comment-15388323 ] Mark Miller commented on SOLR-7280: --- I have no problem if it works. Perhaps some other unrelated changes

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388314#comment-15388314 ] Erick Erickson commented on SOLR-7280: -- Not in my testing, but I'll give it another whirl tonight just

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388312#comment-15388312 ] Mark Miller commented on SOLR-7280: --- Yeah, looks like I have the same issue as I mentioned. I've got to

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388304#comment-15388304 ] Mark Miller commented on SOLR-7280: --- Did this just reintroduce the old bug if you do > 8 replicas or it

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386729#comment-15386729 ] ASF subversion and git services commented on SOLR-7280: --- Commit

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386201#comment-15386201 ] ASF subversion and git services commented on SOLR-7280: --- Commit

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15384572#comment-15384572 ] Mike Drob commented on SOLR-7280: - {{+List l = new ArrayList<>();}} Could init this list to copy.size()

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15384263#comment-15384263 ] ASF subversion and git services commented on SOLR-7280: --- Commit

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15384125#comment-15384125 ] ASF subversion and git services commented on SOLR-7280: --- Commit

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15384095#comment-15384095 ] ASF subversion and git services commented on SOLR-7280: --- Commit

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383907#comment-15383907 ] Noble Paul commented on SOLR-7280: --  > Load cores in sorted order and tweak coreLoadThread counts to

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383138#comment-15383138 ] Mike Drob commented on SOLR-7280: - {code} - public int getCoreLoadThreadCount() { -return

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382916#comment-15382916 ] Mike Drob commented on SOLR-7280: - regarding the lambdas in the test, is there a functional difference

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382476#comment-15382476 ] Mike Drob commented on SOLR-7280: - minor nit: when compiling I see {{\[javac]

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15380860#comment-15380860 ] ASF subversion and git services commented on SOLR-7280: --- Commit

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15380855#comment-15380855 ] ASF subversion and git services commented on SOLR-7280: --- Commit

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15380791#comment-15380791 ] ASF subversion and git services commented on SOLR-7280: --- Commit

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15380759#comment-15380759 ] ASF subversion and git services commented on SOLR-7280: --- Commit

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15380491#comment-15380491 ] Shalin Shekhar Mangar commented on SOLR-7280: - Ah, right, I missed that. I'm reviewing the rest

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15380477#comment-15380477 ] Noble Paul commented on SOLR-7280: -- Shalin, that is the expected behavior. If there is only one replica

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15380105#comment-15380105 ] Shalin Shekhar Mangar commented on SOLR-7280: - Hmm, actually the logic we had discussed earlier

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15380062#comment-15380062 ] Shalin Shekhar Mangar commented on SOLR-7280: - What was the motivation behind changing the

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15374914#comment-15374914 ] Noble Paul commented on SOLR-7280: -- The sorting of shards to be loaded in the following order # The

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15374195#comment-15374195 ] damien kamerman commented on SOLR-7280: --- Or, ensure that the coreLoadThreads is >= max(collection's

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367410#comment-15367410 ] Noble Paul commented on SOLR-7280: -- Had a chat with [~shalinmangar] and came up with the following design.

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366706#comment-15366706 ] Mark Miller commented on SOLR-7280: --- bq. Unless we don't let users create shards with that many replicas

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366705#comment-15366705 ] Mark Miller commented on SOLR-7280: --- bq. That said, I think the default should be quite high in the cloud

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366703#comment-15366703 ] Mark Miller commented on SOLR-7280: --- bq. I'll emphasize though that the current code (without this patch)

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366690#comment-15366690 ] Erick Erickson commented on SOLR-7280: -- bq: I don't think it takes a weird topology - just more

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366618#comment-15366618 ] Mark Miller commented on SOLR-7280: --- One strategy might be to only require a certain number of replicas

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366614#comment-15366614 ] Mark Miller commented on SOLR-7280: --- bq. Sure, with weird enough topology I don't think it takes a weird

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366563#comment-15366563 ] Erick Erickson commented on SOLR-7280: -- The symptom is that OOM errors "unable to create native

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366417#comment-15366417 ] Noble Paul commented on SOLR-7280: -- Thanks [~markrmil...@gmail.com] for looking into this. The

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366168#comment-15366168 ] Mark Miller commented on SOLR-7280: --- bq. // This assumes replicaCount is less than coreLoadThreadCount?

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15364423#comment-15364423 ] Erick Erickson commented on SOLR-7280: -- Great! > Load cores in sorted order and tweak coreLoadThread

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[ https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15364212#comment-15364212 ] Noble Paul commented on SOLR-7280: -- [~erickerickson] planning to commit this soon > Load cores in sorted