Hi Willy, On Mon, Jan 31, 2022 at 08:47:58AM +0100, Willy Tarreau wrote: > Just to be sure, what test suite are you talking about ? I guess you mean > the regression tests in the "reg-tests" directory, but I'm not sure. If so, > we're well aware that they are still fairly limited, and were introduced > relatively recently. They're progressively being completed as new features > are added, but it's for sure that a number of areas of the code are not yet > covered by them. However we tried to make sure that most of them depend on > the most sensitive areas that are easily subject to break (and during > development, there's no single day a developer doesn't break a few regtests > locally during testing, which indicates that they're representative enough > for our purpose for now).
I am indeed talking about the regression tests under "reg-tests".
> Also I don't know how the ratio of lines of code is counted in your case.
> We do have quite a bunch of code that is platform-specific or that depends
> on build options, and in this case I have no idea how that is counted. Also
The ratio only considers code compiled in, so platform code or optional
code that's not enabled in the build configuration will not be considered.
> a significant part of the code is dedicated to error handling and will never
> be triggered by regtests. Events like I/O errors, out-of-memory errors and
> unexpected conditions must never happen, yet collectively they probably
> represent about half of the code. So without more details it's hard to
> have a solid opinion on the subject.
This matches our observations. I'd say that the main causes for low
coverage are (in order of importance) the large number of features, many of
which (HAProxy CLI, Lua Scripting, many protocols, admin/debug features)
have low to nonexistent coverage, followed by error cases, and finally the
different implementations of same building blocks (load balancing algos,
polling backends, etc.).
Thanks for your detailed answer!
Best,
Hugo
--
Hugo Lefeuvre (hle) | www.owl.eu.com
RSA4096_ 360B 03B3 BF27 4F4D 7A3F D5E8 14AA 1EB8 A247 3DFD
ed25519_ 37B2 6D38 0B25 B8A2 6B9F 3A65 A36F 5357 5F2D DC4C
signature.asc
Description: PGP signature
