On 12/30/2016 02:55 PM, Stefan Fritsch wrote:
Yes, httpd lacks unit tests. One problem is that many APIs depend on very
complex structs like request_rec, conn_rec, server_conf, etc. In order to
write unit tests for such APIs, one would need to write quite a bit of
infrastructure to set these things up. I think it would be worth the effort,
but it's not a small task. As there does not seem to be anybody with enough
spare time to do it, one could possibly ask someone (CII?) for funding.
A possible approach would be to compile the unit tests in the server and
execute them on startup if a special define is given (like the various DUMP_*
defines). Not sure how to get access to all the static helper function for unit
tests, though. Unless one would somehow include the tests in the same .c file.
That's an interesting idea. To riff on that a little bit: I've seen some
questions on #httpd recently about the shared-library build for
libhttpd, which IIUC only exists on Windows at the moment. It seems like
having a libhttpd would simplify building a unit test executable... can
anyone point me to the history behind the removal of that feature?
If the test suite would be easier to run, maybe more people would submit
tests. Is there a reason why the test suite is in a separate repository? Would
it help if it was moved into the normal httpd repo? Would it make sense to
include it in the release tarballs, possibly including the necessary non-
standard perl modules? And include it in the makefiles in a way that a user can
install a set of standard perl modules (from a distribution or cpan) and then
call "make test" to start it. What is in the test/ dir in the httpd repo right
now seems mostly useless and could probably be removed.
My personal end goals are
- to be able to perform the standard `make && make check` invocation
without installation (this was discussed with a user in another dev@ thread)
- to have a bugfix/feature *and* its related tests in the same commit or
backported patchset
So, to that end, I'd like to see the test suite eventually move into the
httpd repo. I think I can start on my first goal without that, though
(and I plan to start looking at that soon). That will hopefully give us
time to discuss any possible fallout of merging the two codebases, while
giving us some of the benefits in the meantime.
Another idea to make writing tests more attractive could be to somehow include
it in the backporting policy. For example, if there is a test for a new
feature (positive and error handling) or a bug fix, we could require only two
+1s for a backport.
I like this idea too.
Another thing that is missing: A buildbot that builds current trunk (and
possibly 2.x branches) and runs the test suite and alerts the dev list of
regressions. I guess this "just" needs a volunteer to set it up and document
it and the ASF would provide the infrastructure.
+1. This is a prerequisite to having a nice release cadence, IMHO.
--Jacob
