[jira] [Updated] (SOLR-2822) don't run update processors twice
[ https://issues.apache.org/jira/browse/SOLR-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hoss Man updated SOLR-2822: --- Attachment: SOLR-2822.patch updated patch to trunk. don't run update processors twice - Key: SOLR-2822 URL: https://issues.apache.org/jira/browse/SOLR-2822 Project: Solr Issue Type: Sub-task Components: SolrCloud, update Reporter: Yonik Seeley Fix For: 4.0 Attachments: SOLR-2822.patch, SOLR-2822.patch, SOLR-2822.patch An update will first go through processors until it gets to the point where it is forwarded to the leader (or forwarded to replicas if already on the leader). We need a way to skip over the processors that were already run (perhaps by using a processor chain dedicated to sub-updates? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-2822) don't run update processors twice
[ https://issues.apache.org/jira/browse/SOLR-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hoss Man updated SOLR-2822: --- Attachment: SOLR-2822.patch patch i worked up during the last couple of weeks (but evidently forgot to upload) * implements the general idea i outlined before (using the interface name DistributingUpdateProcessorFactory instead of DistributedUpdateMarker) * includes a randomized test demonstrating the existing problem with doc adds and update processors both before and after the distrib handler -- but this was just shoehorned into an existing test and should be refactored * needs more unit tests - particularly something non randomized that does a straight forward verification of the processor skipping done by the chain * I want to add a NoOpDistributingUpdateProcessorFactory - both for testing and for easy use by any existing SolrCloud users who want absolute total control of their chains (ie: isn't using DistributedUpdateProcessorFactory and is building shards their own custom way) * has a few nocommits - other then things i've already mentioned, they are primarily related to docs and logging One interesting thing i didn't realize until i started implementing this is that the deleteByQuery code path already had something nearly identical to my update.distrib=NONE|TOLEADER|FROMLEADER param value (via a dqb_level=1|2|3) .. i take that as validation that generalizing it to replace the existing SEEN_LEADER logic in the other code paths isn't a bad idea. don't run update processors twice - Key: SOLR-2822 URL: https://issues.apache.org/jira/browse/SOLR-2822 Project: Solr Issue Type: Sub-task Components: SolrCloud, update Reporter: Yonik Seeley Fix For: 4.0 Attachments: SOLR-2822.patch An update will first go through processors until it gets to the point where it is forwarded to the leader (or forwarded to replicas if already on the leader). We need a way to skip over the processors that were already run (perhaps by using a processor chain dedicated to sub-updates? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-2822) don't run update processors twice
[ https://issues.apache.org/jira/browse/SOLR-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hoss Man updated SOLR-2822: --- Attachment: SOLR-2822.patch Updated patch dealing with all the nocommits and (i think) fixing all the TODO items that both i and andrzej mentioned, except for... bq. * includes a randomized test demonstrating the existing problem with doc adds and update processors both before and after the distrib handler – but this was just shoehorned into an existing test and should be refactored I started looking at this and realized that the overhead of creating the cloud instance in this test was somewhat significant (in terms of wall clock time), so it seemed better to leave the test method where it was (cleaned up a bit) then to refactor into another test with the same amount of setup. bq. * tests don't seem to cover any case that uses requests with TOLEADER set. the processor skipping code in the chain doesn't care what the value is, or even what type of requests are processed, so the next test i added in this version of the patch covers both cases of update.distrib being set or not set. The only code path that cares if the values is TOLEADER is in deleteByQuery -- all i did was ensure that the existing tests where changed to use the new update.distrib param instead of the old dbg_level If we need more deleteByQuery tests, that seems like it should be a distinct issue (or at the very least: an additional patch from someone who udnerstands the distrib tests better then me. writing new (whitebox) tests for distribted deleteByQuery is above my solrcloud foo at this point.) don't run update processors twice - Key: SOLR-2822 URL: https://issues.apache.org/jira/browse/SOLR-2822 Project: Solr Issue Type: Sub-task Components: SolrCloud, update Reporter: Yonik Seeley Fix For: 4.0 Attachments: SOLR-2822.patch, SOLR-2822.patch An update will first go through processors until it gets to the point where it is forwarded to the leader (or forwarded to replicas if already on the leader). We need a way to skip over the processors that were already run (perhaps by using a processor chain dedicated to sub-updates? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
