Le 23/04/2018 à 16:00, Jim Jagielski a écrit :
It seems that, IMO, if there was not so much concern about "regressions" in
releases, this whole revisit-versioning debate would not have come up. This implies, to
me at least, that the root cause (as I've said before) appears to be one related to QA
and testing more than anything. Unless we address this, then nothing else really matters.
We have a test framework. The questions are:
1. Are we using it?
2. Are we using it sufficiently well?
3. If not, what can we do to improve that?
4. Can we supplement/replace it w/ other frameworks?
It does seem to me that each time we patch something, there should be a test
added or extended which covers that bug. We have gotten lax in that. Same for
features. And the more substantial the change (ie, the more core code it
touches, or the more it refactors something), the more we should envision what
tests can be in place which ensure nothing breaks.
In other words: nothing backported unless it also involves some changes to the
Perl test framework or some pretty convincing reasons why it's not required.
Hi,
+1000 on my side for more tests.
But, IMHO, the perl framework is complex to understand for most of us.
Last week I tried to play with it. I tried to update proxy_balancer.t
because only lbmethod=byrequests is tested.
The current test itself is really simple. It just checks if the module
didn't crashed (i.e we receive 200).
I tried to extend it for the other lbmethod available. This looked as an
easy task. But figuring the relation between:
<VirtualHost proxy_http_bal1>
and
BalancerMember http://@SERVERNAME@:@PROXY_HTTP_BAL1_PORT@
still remains a mystery to me.
The ./test framework could be useful as well.
At least it is written in C, so the entry ticket should be cheaper for
most of us.
But every thing can't be done with it, I guess.
Maybe, we should at least have some unit testing for each ap_ function?
The behavior of these function should not change as it can be used by
3rd party modules.
The more tests, the better, but I believe that most regressions come
from interaction between all what is possible with httpd.
A test-suite is only a test-suite. Everything can't be tested.
IMHO, as a minimum, all CVE should have their dedicated test which
explicitly fails with version n, and succeeds with version n+1.
It would help to make sure than known security issues don't come back.
Another question with the perl framework.
Is there a way to send "invalid" data/request with it?
All, I see is some GET(...). I guess that it sends well formed date.
Checking the behavior when invalid queries are received would be great.
Some kind of RFC compliancy check.
just my 2c,
CJ
