Hi,
On Tue, 2014-01-07 at 11:22 -0800, Matt Ficken wrote:
> Those summary reports show data from a new cross-platform test tool I have
> created called PFTT (PHP Full Test Tool), which can be run manually or
> unattended (automated).
>
> PFTT runs PHPT and PhpUnit tests (soon SimpleTest tests) across a variety
> of scenarios (CLI, Apache mod_php, Opcache, no code cache, different file
> systems, etc...). It can run PHPT and PhpUnit tests on Apache's mod_php,
> all of which has surfaced many PHP on Windows bugs which we have since
> fixed.
[...]
> The PFTT Unattended Test System I have running sends only a summary
> report to limit the amount of email messages.
I think sharing this is good. My peeve is that these summaries are
*looooooong*. Taking the length and amount makes it hard to digest
them.
What I'd like are three things which should also assist casual
contributors:
A. Instructions to reproduce the tests somewhere on qa.php.net or
such, so that when an issue is found I (or anybody else) can
reproduce it even their primary development system uses a
different setup
B. A short summary where I can see, with a single look, whether
there's something bad or not. With the current 78K I can scroll
over it and still now as much as before.
C. have a database with a nice report to get the details.
This wish is not limited to your tests, but also the "normal" tests we
already have, there https://qa.php.net/reports/ is a good start for
reporting, but still so much room for improvement (i.e. trend graphs or
"alerts" or something)
And to be absolutely clear: I thin it is *great* you're testing this,
and *great* you are sharing the results, but I guess currently the
results reach some void and we should improve that.
johannes
--
PHP Quality Assurance Mailing List <http://www.php.net/>
To unsubscribe, visit: http://www.php.net/unsub.php
