On 23.10.25 15:00, Gary Gregory wrote:
I tested the src zip file:
- ASC OK
- SHA512 OK
- mvn clean verify
This causes errors in
org.apache.sshd.common.forward.ApacheServerJSchClientTest as "flakes"
warnings?. What does it mean?
Please see https://gist.github.com/garydgregory/622094942ed1d1a34e22e97c46a63c9a
TL;DR: yes, some tests are flaky. Ignore that as long as the full build
reports success.
Long version:
Flaky tests are a perennial pain point. I spent quite some time a while
back (I think in 2022) to get stable builds on my machine and in CI, but
that doesn't mean that all our tests are fully stable.
Also: re-trying surefire tests multiple times actually hides the
flakiness in CI (unless you'd go inspect every build output for flakes).
From time to time some tests still are flaky. I don't know what exactly
went wrong in your run; if the same test is frequently flaky, you'd
have to debug failing runs live to have a chance to figure out where
the problem is.
From my past experience, problems that cause flaky tests can be
- bugs in Apache MINA SSHD, typically a race condition somewhere.
- problems with sockets; sometimes in tests re-using sockets too
quickly, but sometimes also for unspecific reasons in the
environment or OS.
- bugs in Apache MINA SSHD test code, typically expecting some
assertion to be true synchronously when in fact the condition
is fulfilled only asynchronously.
- some tests are only flaky on some I/O transports. A test might
work perfectly on NIO2 or MINA, but be flaky on Netty. Or be
fine on NIO2 and Netty but be flaky on MINA. Or some other
combination.
- bugs and race conditions in third-party libraries used in tests.
(In particular JSch.)
(The list is not exhaustive!)
It's also possible that the refactorings done in 3.0.0 so far have
introduced a new problem that might make some tests flaky now, but
locally I have not observed anything like that. (Nor in CI).
However: I use a nearly 10 year old Intel Mac as development machine,
which is not the fastest. CI build in Github are also not exactly
fast. It's possible that running the tests on your much more powerful
ARM Mac uncovers new flakiness. You might hit race conditions that
just never occur on slower machines.
Cheers,
Thomas
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
