On Thu, Dec 19, 2019 at 4:32 PM Marion & Christophe JAILLET < [email protected]> wrote:
> Hi, > > I've received the attached patch against the test framework. > This is not the first time Petr propose such patches and most (or maybe > even all) have been merged. > > The goal is to avoid new warnings on suse tests when a recent test > framework is used on an older version of httpd. > > However, this particular case puzzles me. > I see r1868313 as a fix on a non-RFC compliant behavior. So I'm not sure > this should be "hidden" only because older versions are known to fail. > > In other word, what is the "semantic" behind the test framework? > - Testing against regression only? (in this case including Petr's patch > makes sense to me) - or - > - Checking that the behavior is correct (in this case, I'm not fan of > "hiding" things that are known to fail. A failure would mean: "you should > upgrade to have this fixed") > I think the tests should either skip or have changed expectations when run against an old build, not fail. In other words the tests are for us, not end users curious how correct what they have is.
