Hi Dawid, Rob, Sorry for the late reply!!
Yeah I agree: unit tests should strive to only and precisely fail for true bugs, not false alarms like this. I'm sorry this failure wasted your time Dawid! So we should fix this failure somehow: Remove all too-big terms from all test LineFileDocs sources (the small version in git and the large nightly version)?; Switch this unit test to not use LineFileDocs?; Make this exception a checked exception (not good to make all users pay the price of the rare exception)? Change IndexWriter to silently drop these terms? But then I don't think our testing of too-long terms, which happens to real users, is great. We have a dedicated unit test case (TestIndexWriter.testWickedLongTerm) which specifically confirms that the inverted index will be OK (and throw the right exception) if you attempt to index a massive term. But what about all our analyzers? Do they handle too-long terms? Does TestRandomChains sometimes inject massive terms? Or our random realistic Unicode string generation methods? Or we can just fallback to nightly benchmarks (closest thing we have to an "integration test"?) trying to catch such rare real-world problems that our users might hit? Mike McCandless http://blog.mikemccandless.com On Sat, Apr 23, 2022 at 2:05 PM Dawid Weiss <[email protected]> wrote: > > i don't think we should intentionally allow our tests to be flaky, i > > strongly disagree. > > I agree with Robert. Tests should pass. Every time a build fails it > takes some time to investigate the root cause and it is disheartening > when you discover that it's a failure that's "allowed" to happen. > > D. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
