Hi Nik,
There is no formal workflow advised to follow.
However, it is the responsibility of every committer to keep the code
compile-able and run-able.
However, a few more explanations and precisions.
When some code is added/modified, it must be committed in trunk first.
trunk is a CTR branch. That means Commit Then Review.
You push first, then anyone can make some review and/or comment on the code.
If the code is considered good enough to be included in a new release,
it has to be proposed for backport (see the STATUS file
(http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/STATUS?view=markup)
for the current patches proposed for backport)
The other branches are RTC, that it to say Review Then Commit.
We need to have at least 3 +1 to backport the code.
Developers can comment, improve the proposal, raise potential issues, ...
One can also veto (-1) a patch and explain why.
Without any consensus, no backport is allowed.
You can also have some information at:
https://httpd.apache.org/dev/
https://httpd.apache.org/dev/guidelines.html#voting
In other words, before being included in a release, code:
- should be tested and reviewed by its author
- should be reviewed by other developers in trunk
- *IS* reviewed *AND* *IS* tested by at least 3 different committers
- *IS* tested by many people using the test framework discussed below
is usually run on some early adopter websites
is usually tested for some specific load/environment/configuration
by some developers
... before being released as part of a new release
It is also welcomed to add some new test-cases when changes are made,
bug fixed or new functionalities added.
Testing everything, every configuration, every environment and every
potential interaction is impossible.
However, the more test we have, the more confident we can be with each
new release.
We have 2 main test framework.
One is (should be) dedicated to unit test. It is available in the test
directory of the trunk branch. Up to now, it is mostly a PoC. Nothing
big is available yet.
At least an infrastructure is available if come contributors want to add
some useful stuff.
The other test framework is a perl framework located, as you noted, at:
- https://httpd.apache.org/test/
- http://svn.apache.org/viewvc/httpd/test/framework/trunk/
mod_h2 also has its own framework located at
http://svn.apache.org/viewvc/httpd/test/mod_h2/
This perl test framework is much more complete and useful.
Tests are run on several OS and configuration, depending of the
individual running the test.
They are run at least before each new release and results are posted.
To give you an idea, here is a pointer on the last voting / testing
thread before 2.4.37 release.
https://lists.apache.org/list.html?dev@httpd.apache.org:2018-10:httpd-2.4.37
I hope these few information are helpful for you and any contribution is
greatly appreciated.
If unclear or incomplete, do not hesitate to ask again, you are on the
right mailing list for that.
Best regards,
CJ
Le 13/01/2019 à 15:51, Nik Sultana a écrit :
Had sent this a month back but it might have languished in inboxes
during vacation season.
Basically is there any onboarding/advice/workflow for testing, or are
people left to their own (divergent/unchecked) devices?
Is there a more suitable httpd mailing list for this question?
Thanks,
Nik
On Thu, 13 Dec 2018, Nik Sultana wrote:
Hi all, I wonder how httpd devs go about testing their code before it's
committed -- is there a workflow that everybody's advised to follow, to
try and smoke out potential breakages early? I regularly see people
voting on this mailing list to make successive httpd releases, but it's
not clear to me what goes into the QA for release engineering.
I've been hacking some ideas locally into the Apache httpd source code
as part of a research project, and I usually test my changes by running
Apache Bench or other traffic generators on it. But I'm not sure how
well this compares with what Apache committers/testers do.
While looking around the Internet I came across this, which seems most
relevant:
https://httpd.apache.org/test/
I'm not sure how up-to-date it is (e.g., the SPECWeb99 tests it
references have long since been retired).
Are the programs linked from that page the state-of-the-art for httpd
testing?
I also came across various other disparate advice on load testing of web
servers. Are there any systems that httpd devs find especially useful?
And finally, does httpd testing tend to be done on one or two machines
(by firing a traffic generator at httpd), or are there any cluster-based
testing systems that you've found to be useful for more
rigorous/expansive testing? (e.g., firing traffic from various machines
into an httpd instance.)0
I'm happy to compile your answers into documentation if you wish, perhaps
as a refresh of https://httpd.apache.org/test/
Best,
Nik
