[jira] [Commented] (SOLR-8722) Don't force a full ZkStateReader refresh on every Overseer operation
[ https://issues.apache.org/jira/browse/SOLR-8722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1512#comment-1512 ] ASF subversion and git services commented on SOLR-8722: --- Commit e4712bb028849f9a9b202651728c1f5c0a224374 in lucene-solr's branch refs/heads/branch_6x from [~shalinmangar] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e4712bb ] SOLR-8722: Don't force a full ZkStateReader refresh on every Overseer operation (cherry picked from commit 93133f5) > Don't force a full ZkStateReader refresh on every Overseer operation > > > Key: SOLR-8722 > URL: https://issues.apache.org/jira/browse/SOLR-8722 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Affects Versions: 5.4.1 >Reporter: Scott Blum >Assignee: Shalin Shekhar Mangar > Labels: patch, performance, solrcloud > Attachments: SOLR-8722.patch > > > We're doing an unnecessary ZkStateReader forced refresh on all Overseer > operations. This isn't necessary because ZkStateReader keeps itself up to > date. > According to [~shalinmangar]'s analysis, we just need to put a wait loop at > the end of addReplica to observe the state change. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8722) Don't force a full ZkStateReader refresh on every Overseer operation
[ https://issues.apache.org/jira/browse/SOLR-8722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15177768#comment-15177768 ] ASF subversion and git services commented on SOLR-8722: --- Commit 93133f54fd8cad47d7638c50a2360e3eb9daeb14 in lucene-solr's branch refs/heads/master from [~shalinmangar] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=93133f5 ] SOLR-8722: Don't force a full ZkStateReader refresh on every Overseer operation > Don't force a full ZkStateReader refresh on every Overseer operation > > > Key: SOLR-8722 > URL: https://issues.apache.org/jira/browse/SOLR-8722 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Affects Versions: 5.4.1 >Reporter: Scott Blum >Assignee: Shalin Shekhar Mangar > Labels: patch, performance, solrcloud > Attachments: SOLR-8722.patch > > > We're doing an unnecessary ZkStateReader forced refresh on all Overseer > operations. This isn't necessary because ZkStateReader keeps itself up to > date. > According to [~shalinmangar]'s analysis, we just need to put a wait loop at > the end of addReplica to observe the state change. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8722) Don't force a full ZkStateReader refresh on every Overseer operation
[ https://issues.apache.org/jira/browse/SOLR-8722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15175816#comment-15175816 ] Shalin Shekhar Mangar commented on SOLR-8722: - bq. In that case, should we land this patch as-is and then come back with a more holistic answer for async failures? +1, I'll commit after running tests. > Don't force a full ZkStateReader refresh on every Overseer operation > > > Key: SOLR-8722 > URL: https://issues.apache.org/jira/browse/SOLR-8722 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Affects Versions: 5.4.1 >Reporter: Scott Blum >Assignee: Shalin Shekhar Mangar > Labels: patch, performance, solrcloud > Attachments: SOLR-8722.patch > > > We're doing an unnecessary ZkStateReader forced refresh on all Overseer > operations. This isn't necessary because ZkStateReader keeps itself up to > date. > According to [~shalinmangar]'s analysis, we just need to put a wait loop at > the end of addReplica to observe the state change. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8722) Don't force a full ZkStateReader refresh on every Overseer operation
[ https://issues.apache.org/jira/browse/SOLR-8722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15171185#comment-15171185 ] Scott Blum commented on SOLR-8722: -- In that case, should we land this patch as-is and then come back with a more holistic answer for async failures? > Don't force a full ZkStateReader refresh on every Overseer operation > > > Key: SOLR-8722 > URL: https://issues.apache.org/jira/browse/SOLR-8722 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Affects Versions: 5.4.1 >Reporter: Scott Blum >Assignee: Shalin Shekhar Mangar > Labels: patch, performance, solrcloud > Attachments: SOLR-8722.patch > > > We're doing an unnecessary ZkStateReader forced refresh on all Overseer > operations. This isn't necessary because ZkStateReader keeps itself up to > date. > According to [~shalinmangar]'s analysis, we just need to put a wait loop at > the end of addReplica to observe the state change. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8722) Don't force a full ZkStateReader refresh on every Overseer operation
[ https://issues.apache.org/jira/browse/SOLR-8722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15170092#comment-15170092 ] Shalin Shekhar Mangar commented on SOLR-8722: - bq. What would you suggest in this case? Spinning for a very long time when the remote call might have failed quickly seems bad. I guess the only way is to modify the processResponses method to throw an exception in case the async request failed just like how we do it today for non-async requests. bq. What other handler methods do you think demonstrate the correct pattern? I don't think any of them do :( > Don't force a full ZkStateReader refresh on every Overseer operation > > > Key: SOLR-8722 > URL: https://issues.apache.org/jira/browse/SOLR-8722 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Affects Versions: 5.4.1 >Reporter: Scott Blum >Assignee: Shalin Shekhar Mangar > Labels: patch, performance, solrcloud > Attachments: SOLR-8722.patch > > > We're doing an unnecessary ZkStateReader forced refresh on all Overseer > operations. This isn't necessary because ZkStateReader keeps itself up to > date. > According to [~shalinmangar]'s analysis, we just need to put a wait loop at > the end of addReplica to observe the state change. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8722) Don't force a full ZkStateReader refresh on every Overseer operation
[ https://issues.apache.org/jira/browse/SOLR-8722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15167405#comment-15167405 ] Scott Blum commented on SOLR-8722: -- No problem! Welcome back, hope you had a good trip. What would you suggest in this case? Spinning for a very long time when the remote call might have failed quickly seems bad. What other handler methods do you think demonstrate the correct pattern? The code can be hard to follow. Sometime when you're on IRC I'd love to pick your brain about the code in OverseerCollectionMessageHandler & OverseerTaskProcessor. > Don't force a full ZkStateReader refresh on every Overseer operation > > > Key: SOLR-8722 > URL: https://issues.apache.org/jira/browse/SOLR-8722 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Affects Versions: 5.4.1 >Reporter: Scott Blum >Assignee: Mark Miller > Labels: patch, performance, solrcloud > Attachments: SOLR-8722.patch > > > We're doing an unnecessary ZkStateReader forced refresh on all Overseer > operations. This isn't necessary because ZkStateReader keeps itself up to > date. > According to [~shalinmangar]'s analysis, we just need to put a wait loop at > the end of addReplica to observe the state change. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8722) Don't force a full ZkStateReader refresh on every Overseer operation
[ https://issues.apache.org/jira/browse/SOLR-8722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166792#comment-15166792 ] Shalin Shekhar Mangar commented on SOLR-8722: - Hi Scott, yes this should work fine. Sorry for being out of the loop re: SOLR-8696. I was travelling back to India and had a terrible jet lag. bq. Also, what's the right way to handle things if the remote call fails? Will the code throw before it reaches the wait loop, or should I try to inspect the response object for errors before deciding to wait? I didn't see any special handling at other call sites. Hmm, good question. The processResponses method will throw an exception if abortOnError=true (which it is, in this case) but the 'async' handling does not throw any exceptions. So if the remote call fails and the user had specified the 'async' parameter, the wait loop will still be invoked. > Don't force a full ZkStateReader refresh on every Overseer operation > > > Key: SOLR-8722 > URL: https://issues.apache.org/jira/browse/SOLR-8722 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Affects Versions: 5.4.1 >Reporter: Scott Blum >Assignee: Mark Miller > Labels: patch, performance, solrcloud > Attachments: SOLR-8722.patch > > > We're doing an unnecessary ZkStateReader forced refresh on all Overseer > operations. This isn't necessary because ZkStateReader keeps itself up to > date. > According to [~shalinmangar]'s analysis, we just need to put a wait loop at > the end of addReplica to observe the state change. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8722) Don't force a full ZkStateReader refresh on every Overseer operation
[ https://issues.apache.org/jira/browse/SOLR-8722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15159408#comment-15159408 ] Scott Blum commented on SOLR-8722: -- [~shalinmangar] is this what you had in mind? We could probably optimize exactly what this code waits on, but I figured I'd just start by reusing the existing code. But looping 320 x 1 second seems a bit excessive for a new core creation. Also, what's the right way to handle things if the remote call fails? Will the code throw before it reaches the wait loop, or should I try to inspect the response object for errors before deciding to wait? I didn't see any special handling at other call sites. > Don't force a full ZkStateReader refresh on every Overseer operation > > > Key: SOLR-8722 > URL: https://issues.apache.org/jira/browse/SOLR-8722 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Affects Versions: 5.4.1 >Reporter: Scott Blum >Assignee: Mark Miller > Labels: patch, performance, solrcloud > Attachments: SOLR-8722.patch > > > We're doing an unnecessary ZkStateReader forced refresh on all Overseer > operations. This isn't necessary because ZkStateReader keeps itself up to > date. > According to [~shalinmangar]'s analysis, we just need to put a wait loop at > the end of addReplica to observe the state change. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
