Hi Joanthan,
Valid points you are raising.
What's currently broken with tests is not, that developers don' care but
rather, that developers are not aware of failing tests due to the lack
of visibility. Also it's not possible currently to tell which commit is
responsible for a failing test.
Another thing is, that there is no reference platform yet. A test may
work on windows but fail on linux. Then somebody tries to "fix" it and
breaks it on the other platform. This method will introduce a
"reference" platform that will have to work. If further reference
platforms will be added later is open, but something to keep in mind.
These things will be fixed. And I don't assume that somebody will
disagree if a commit that breaks a test will be reverted as well as it
will make it easier to review pull requests because a "red" pull request
will not even have to be considered. But certainly, an official
statement of the PSC concerning this will be highly appreciated. There
is also a recent RFC/QEP from Nyall that deals with requirements for
unit tests. [1]
I hope I could address your concerns and if not, just ask again :-)
Kind regards,
Matthias
[1]
https://github.com/nyalldawson/QGIS-Enhancement-Proposals/blob/728f5ef8cfc6dd4c7ee830213339842db6891ee1/QEP-5-Unit-tests.rst
On 10/10/2014 02:10 PM, Jonathan Moules wrote:
Hi Matthias,
It’s good to see that some initiative is being taken on this, but if
this goes ahead, is there a requirement from the PSC that devs
actually do fix things that cause the tests to fail or not have their
commit accepted? Because otherwise I can see this devolving into the
situation that currently exists (devs don’t bother with unit tests).
Further thought – if this goes ahead it may be worth building on it
and requiring that unit tests be created for all new functionality as
part of acceptance of that functionality. Are there such plans?
(that’s more something for the dev list anyway).
Certainly a good start though!
Cheers,
Jonathan
*From:*[email protected]
[mailto:[email protected]] *On Behalf Of *Matthias Kuhn
*Sent:* Friday, October 10, 2014 12:58 PM
*To:* [email protected]
*Subject:* [Qgis-user] Crowdfunding Project: Automated tests for QGIS
Dear QGIS users,
I started a crowd-funding campaign for automated unit tests in QGIS.
Please consider to help funding this project in order to improve the
software and code quality in the long run. More details can be found
on the project page and in the text below.
http://blog.vitu.ch/10102014-1046/crowdfunding-initiative-automated-testing
I plan to improve the quality of QGIS sustainably by introducing the
automatic execution of test procedures which will inform responsible
developers automatically in case of failure. Your support for this
project is highly welcome and you can contribute to make QGIS more
stable in the long run. To implement the necessary changes as well as
repairing currently broken tests a minimal amount of CHF 5000.- will
be required. The following paragraphs will describe the project in
more detail.
----------
In the last years, QGIS matured from a hobby project to a serious
alternative with a broad scope of possibilities and a great potential.
With the constant addition of new features new possibilities have been
created and still are created. Every new feature means changes to the
source code of the software and is a potential source of errors.
There are possibilities to automatically detect errors. This is done
by executing a series of operations with known result. If a operation
does not return the desired (known) result something went wrong and it
needs to be evaulated why.
QGIS has currently over 100 different tests. However, these are not
run regularly and nobody realizes when tests fail. This led to the
situation, that at the moment approximately 2/3 of the tests do not
report success and nobody knows why and since when.
I intend to take the following measures to ensure that in the future
tests no longer can silently fail:
* tests will be run on all source code changes in master (main
development branch)
* tests will be run on all pull requests (changes being evaluated for
integration in master)
* if tests fail the responsible developers will be notified by mail
* with symbols, the current state of the tests will be indicated
With this system it is possible to react instantaneously when tests
fail and to revert the responsible changes or to not even integrate these.
To make sure that this system works and is accepted, it is important
to start with a working suite of tests. How much time it takes to fix
all tests is difficult to say. The plan is therefore to fix as many
tests with the available funds and to disable the rest of the tests in
the first time. Therefore additional funds that are provided on top of
the required minimum CHF 5000.- will be very welcome for additional
working tests.
It would be great if this crowd-funding effort could help to get this
project to fly.
Kind regards,
Matthias Kuhn
This message has been scanned for viruses by MailControl
<http://www.mailcontrol.com/>, a service from BlackSpider Technology
Click here
<https://www.mailcontrol.com/sr/h0opETQ15z7GX2PQPOmvUgny0%216zNmYYcr+CLPtUcwBrcg72XoIMJrYkKSbFUrl7COZlUrWwm5I94h7CChK4GQ==>
to report this email as spam.
------------------------------------------------------------------------
*HR Wallingford and its subsidiaries* uses faxes and emails for
confidential and legally privileged business communications. They do
not of themselves create legal commitments. Disclosure to parties
other than addressees requires our specific consent. We are not liable
for unauthorised disclosures nor reliance upon them.
If you have received this message in error please advise us
immediately and destroy all copies of it.
HR Wallingford Limited
Howbery Park, Wallingford, Oxfordshire, OX10 8BA, United Kingdom
Registered in England No. 02562099
------------------------------------------------------------------------
_______________________________________________
Qgis-user mailing list
[email protected]
http://lists.osgeo.org/mailman/listinfo/qgis-user
