Hi Willy,
Op 7-1-2019 om 15:25 schreef Willy Tarreau:
Hi Pieter,

On Sun, Jan 06, 2019 at 04:38:21PM +0100, PiBa-NL wrote:
The 23654 mails received for a failed server is a bit much..
I agree. I really don't know much how the mails work to be honest, as
I have never used them. I remember that we reused a part of the tcp-check
infrastructure because by then it offered a convenient way to proceed with
send/expect sequences. Maybe there's something excessive in the sequence
there, such as a certain status code being expected at the end while the
mail succeeds, I don't know.

Given that this apparently has always been broken,
For 1 part its always been broken (needing the short mailer timeout to send all expected mails), for the other part, at least until 1.8.14 it used to NOT send thousands of mails so that would be a regression in the current 1.9 version that should get fixed on a shorter term.
I'm hesitant between
merging this in the slow category or the broken one. My goal with "broken"
was to keep the scripts that trigger broken behaviours that need to be
addressed, rather than keep broken scripts.
Indeed keeping broken scripts wouldn't be help-full in the long run, unless there is still the intent to fix them. It isn't what the makefile says about 'LEVEL 5' though. It says its for 'broken scripts' and to quickly disable them, not as you write here for scripts that show 'broken haproxy behavior'.
  My goal is to make sure we
never consider it normal to have failures in the regular test suite,
otherwise you know how it becomes, just like compiler warnings, people
say "oh I didn't notice this new error in the middle of all other ones".
Agreed, though i will likely fall into repeat some day, apology in advance ;).. I guess we could 'fix' the regtest by specifying the 'timeout mail 200', that would fix it for 1.7 and 1.8.. And might help for 1.9 regressiontests and to get it fixed to at least not send thousands of mails. We might forget about the short time requirement then though, which seems strange as well. And the test wouldn't be 1.6 compatible as it doesn't have that setting at all.
Thus probably the best thing to do is to use it at level 5 so that it's
easier to work on the bug without triggering false positives when doing
regression testing.

What's your opinion ?

With a changed description for 'level 5' being 'shows broken haproxy behavior, to be fixed in a future release' i think it would fit in there nicely. Can you change the starting letter of the .vtc test (and the .lua and reference to that) to 'k' during committing? Or shall i re-send it?

p.s. What do you think about the 'naming' of the test? 'k_healthcheckmail.vtc' or 'k00000.vtc' personally i don't think the 'numbering' of tests makes them easier to use.?.



PiBa-NL (Pieter)

Reply via email to