[
https://issues.apache.org/jira/browse/SOLR-10109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hoss Man updated SOLR-10109:
----------------------------
Attachment: SOLR-10109.testSoftAndHardCommitMaxTimeRapidAdds.patch
bq. ... I'll start working on a patch (and include some nocommit random sleeps
to help test for the possibility of this lag) ...
I'm attaching what I wrote up, but even as I was working on it I started having
doubts which were verified the first time i ran it with the random sleeps in
place: The {{minExpectedFoo}} calculations are fundementally assuming that the
"lag" is even between the updates. If we imagine a machine where the first 4
docs are added w/o any lag, but then the CPU stalls the main test thread for ~4
* the autoCommit value before adding the final document, the math i suggested
will assumethere must be a min of 3 commits, but in reality it makes sense that
only 2 commits would have happened.
I think the "correct" assertions we should be able to make are...
* there _must_ be a minimum of 1 each hard/soft commit
** the (non-null) timestamps of these commits reported by the monitor must be
less then the "after add" time (recorded by the test thread) of the first doc +
the autoCommit value + some fudge
* there _may_ be additional commits of each type...
** the upper limit on the # of additional commits allowed can be determined by
dividing the total add duration (recorded by the test thread) by the autoCommit
values for each type
** if we poll a monitor for that type of commit and find a _non-null_ timestamp:
*** the timestamp must be less then the _last_ timestamp (from the same
monitor) + the autoCommit value + some fudge
** if we poll a monitor for that type of commit and find a _null_ timestamp:
*** there better not be any more commits of that type later if we keep polling
This is sort of along the lines of what the existing (in master) loop logic for
{{i <= expectedSoft}} does -- in terms of treating {{i == 1}} as required --
except that:
* it only asserts that all commit timestamps are within the expected range of
the last doc added -- not the previous commit
* the existing {{i <= expectedHard}} loop doesn't have similar logic /
accounting.
I'll experiment with this idea.
> SoftAutoCommitTest is too fragile.
> ----------------------------------
>
> Key: SOLR-10109
> URL: https://issues.apache.org/jira/browse/SOLR-10109
> Project: Solr
> Issue Type: Test
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Mark Miller
> Assignee: Mark Miller
> Attachments: SoftAutoCommitTest.2017-02-25.2938.log.txt,
> SoftAutoCommitTest.2017-05-02.3983.log.txt,
> SOLR-10109.testSoftAndHardCommitMaxTimeRapidAdds.patch
>
>
> In my beast test runs it has failed 7% of the time, then 3% of the time, then
> when I started using hardware with 8 cores instead of 16, 40% of the time
> (unless something else also shifted). That gives it a test rating of 'screwy'.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
