On May 3, 2006, at 11:59 AM, Jean-Sebastien Delfino wrote:
Jim Marino wrote:
I tried to distinguish between commiters and non-commiters. For
*commiters* we should get into the habit of using unit tests to
demonstrate problems since I believe we want to encourage this
among non-comitters as well (note several Apache and many open
source projects do this, including many of the Geronimo
subprojects). I'm not saying showing how to reproduce the problem
using a sample is somehow "inappropriate", I'm just asking that
commiters adopt a default of using unit tests since it helps the
project. Of course, there are times when this is not easy to do,
but when it is (which was the case in the examples I referred to)
it seems like a logical request to make. Also, please note I did
not mean to imply that the bug fixer is free of the responsibility
to write testcases associated with the fix (actually I would argue
that if they must when checking in the fix).
Do you have an issue with this approach? If so, could you please
outline what specific concerns you have?
Jim
On May 3, 2006, at 8:43 AM, ant elder wrote:
I'm not sure I completely agree with this, if you make it hard to
report
problems then people don't report them. A bug reporter should
have to
describe/provide just enough of a way to easily recreate a
problem, its down
to the bug fixer to write whatever regression and unit tests are
required by
the fix. If you can describe how to demonstrate a bug just by
running one of
our samples that's fine in my book.
...ant
On 4/25/06, James F Marino <[EMAIL PROTECTED]> wrote:
I've noticed several issues have been logged against the Java
container where entire Maven projects or samples are used to
demonstrate a problem. Sometimes this may be unavoidable, but I
would
like to propose that we get into the habit of submitting failing
testcases as a default instead. This is common practice in many
projects, and my reasons for suggesting this are:
- it helps diagnose the problem
- it provides verification when the issue is resolved
- it builds up our test suite which is woefully inadequate
- it is often less work than creating entire projects
Obviously, this can't be a literal policy but I believe it
should be
"default" behavior for those directly involved in the project. For
non-commiters, we could encourage this behavior by noting that
logged
issues with testcases generally get attention quicker.
Also, I would like to propose that while we may tolerate an issue
logged with a project which requires a particular IDE such as
IntelliJ
or Eclipse, or requires "main" to run from those outside the
project,
commiters should never do this.
Comments?
Jim, could you please confirm what you mean by "test case" in this
context? Are you asking for a function test case? an integration
test case? or are you really talking about a unit test case?
This may depend on the issue but it is likely to be a unit or
integration test case. If we do not catch something there, it is
generally a problem with test coverage, although there are exceptions
to the rule (no pun intended). For example, if we can't use a unit or
integration test to check something, it's often (but not always)
reflects a design problem. An exception to this rule that comes to
mind are platform-related issues.
Also, we've been looking at adding to our web site some guideline
documents for contributors (exception handling, logging, coding
guidelines, submitting patches etc), can you point us to any
guideline document from the mentioned Apache projects describing
this approach?
Off the top of my head:
ActiveMQ: http://www.activemq.org/Contributing
Also, Spring (which obviously is not Apache).
I'm sure I can dig up others if I looked but I think those should
suffice. FYI on exception handling, Haleh was kind enough to revise
the original project exception handling document - I have a todo to
take a look at it.
Jim
--
Jean-Sebastien
