Hi, On Thu, Aug 20, 2020 at 8:48 AM Jakub Kądziołka <[email protected]> wrote:
> We're observing nondeterministic test failures, like these: > > Failures: > From test_autocmd.vim: > Found errors in Test_CursorHold_autocmd(): > Run 1: > command line..script > /tmp/guix-build-vim-full-8.2.1471.drv-0/source/src/testdir/runtest.vim[454]..function > RunTheTest[39]..Test_CursorHold_autocmd line 21: Expected ['1', '2', '3'] > but got ['1', '3'] > Run 2: > command line..script > /tmp/guix-build-vim-full-8.2.1471.drv-0/source/src/testdir/runtest.vim[488]..function > RunTheTest[39]..Test_CursorHold_autocmd line 21: Expected ['1', '2', '3'] > but got ['1', '2'] > Run 3: > command line..script > /tmp/guix-build-vim-full-8.2.1471.drv-0/source/src/testdir/runtest.vim[488]..function > RunTheTest[39]..Test_CursorHold_autocmd line 21: Expected ['1', '2', '3'] > but got ['1', '2'] > Flaky test failed too often, giving up > From test_terminal3.vim: > Found errors in Test_term_mouse(): > Run 1: > Caught exception in Test_term_mouse(): Vim(call):E684: list index out > of range: 0 @ command line..script > /tmp/guix-build-vim-full-8.2.1471.drv-0/source/src/testdir/runtest.vim[454]..function > RunTheTest[39]..Test_term_mouse, line 73 > Run 2: > Caught exception in Test_term_mouse(): Vim(let):E684: list index out of > range: 0 @ command line..script > /tmp/guix-build-vim-full-8.2.1471.drv-0/source/src/testdir/runtest.vim[488]..function > RunTheTest[39]..Test_term_mouse, line 41 > Run 3: > Caught exception in Test_term_mouse(): Vim(call):E684: list index out > of range: 0 @ command line..script > /tmp/guix-build-vim-full-8.2.1471.drv-0/source/src/testdir/runtest.vim[488]..function > RunTheTest[39]..Test_term_mouse, line 64 > Run 4: > Caught exception in Test_term_mouse(): Vim(let):E684: list index out of > range: 0 @ command line..script > /tmp/guix-build-vim-full-8.2.1471.drv-0/source/src/testdir/runtest.vim[488]..function > RunTheTest[39]..Test_term_mouse, line 41 > Run 5: > Caught exception in Test_term_mouse(): Vim(call):E684: list index out > of range: 0 @ command line..script > /tmp/guix-build-vim-full-8.2.1471.drv-0/source/src/testdir/runtest.vim[488]..function > RunTheTest[39]..Test_term_mouse, line 64 > Flaky test failed too often, giving up > > A thing these tests have in common is that 50 ms sleep calls are used for > synchronization. It seems that this is too small a number, even when the > test is marked as flaky. Ideally, something like "wait until file is > written" or "wait until child vim is waiting for keystrokes again" would be > used instead of a hardcoded length of time, but I understand this is not > always possible. > > Perhaps it would be desirable to increase this delay? I understand that > this has an effect on test suite speed, but if the tests are always looped > due to flakiness, maybe this would help reduce that, and as a result speed > the test suite up? > > > Can you try changing the term_wait() calls to TermWait() in test_autocmd.vim and test_terminal3.vim and check whether that fixes this problem? Thanks, Yegappan -- -- You received this message from the "vim_dev" maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups "vim_dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/vim_dev/CAAW7x7%3DQcx_KyHLY6riuBBiGQe6P%3DSfyX1mbG-2R8Uhv2esgJQ%40mail.gmail.com.
