[
https://issues.apache.org/jira/browse/LUCENE-4456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469054#comment-13469054
]
Robert Muir commented on LUCENE-4456:
-------------------------------------
This is in 4.x now: just to try give some summary of this insanity...
# Uwe's original bug (LUCENE-4455) caused mike to add a sizeInBytes test to
TestRollingUpdates
# this test failed in jenkins overnight, because of some unreferenced files
leftover from indexwriter. so it was clear MockDirectoryWrapper's unrefed files
check has not really been working for some time! We now have a
"test-the-tester" test for this so we know its working in the future!
# when we turned that on, all kinds of test failures happened: some
exception/crash cases and what not, some real bugs like deleting non-index
files. some fake test failures because tests made multiple commits. other fake
test failures because IW couldnt delete files since MockDirectoryWrapper acts
like windows (and some reader had them open).
# We fixed MockDirectoryWrapper more so that it handles the openDeletedFiles
Windows case, and fixed it to handle indexes with multiple commits. More fake
test failures because of hard-to-reproduce act-like-Windows corner cases like
segments_N and segments.gen (its hard to reproduce the write at exactly the
same time as these are slurped).
# To try to make those more reproducible, I figured MockDir could sometimes
"hang onto" its IndexInputs a little bit longer so this is more likely to
happen in tests, by very rarely doing a sleep in its close(). But a side effect
of this, is testThreadInterruptDeadLock would sometimes interrupt this sleeping
in close, causing an exception to be thrown in an IndexInput's close(). So this
test (not intended for this purpose, but nice side effect) finds bugs where
exception handling of close() for indexinputs is wrong, because of its unclosed
files check.
And lots of pulling out hair and stuff in between.
Anyway it probably looks a lot worse than it is, because most of these failures
are silly bugs/test bugs/trying to get the test framework right.
I have two jenkins pointed at branch_4x now:
* http://sierranevada.servebeer.com/job/branch4x-beaster/ (Linux)
* http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ (Windows)
Combined these do 70 test-core builds per hour. So every day is like a month's
worth of test failures, which makes things look really unstable but its
intentional
to try to hunt everything down at once instead of onesy-twosey for a long time
:)
Lets give it a little time and see if it can flush anything out and then
backport this stuff to 4.0, beast a bit more, and respin.
> IndexWriter makes unrefed files, and MockDir cannot detect it
> -------------------------------------------------------------
>
> Key: LUCENE-4456
> URL: https://issues.apache.org/jira/browse/LUCENE-4456
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Robert Muir
> Assignee: Michael McCandless
> Attachments: LUCENE-4456_mdw_patch.txt, LUCENE-4456.patch,
> LUCENE-4456.patch
>
>
> Because MockDir calls crash() before it checks for unreferenced files,
> deletes are no longer allowed.
> this means the unreferenced files check is useless!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
