[resurrecting an old thread now that I've had time to put together a
proposal]
Sounds like folks are agreeing we need design docs and test plans for
features, with common sense prevailing on when to apply this rule to
smaller features.
So, let me propose some design doc and test plan templates.
Design Doc Template:
Problem Definition
what problem does the feature solve?
why is it needed?
Use Cases
sequence of simple steps on how users interacts with the feature
Interaction with Current System
does this feature build on an existing feature? if so, how?
how does the user interact with the new feature from outside the system
how does the new feature interact within the system
Requirements
focus on what functionality must exist and minimize discussion of how
that functionality will be implemented
list items out of scope
Design
give a high level description of the implementation
algorithms and/or data-structures involved, this may be in the form of
pseudo-code
failure analysis
Future Work
what additional work is related or could be done, but is not included
in this design
Test Plan Template:
Feature(s) Tested: enumerate the feature(s)
which Jira issue(s)?
what is the feature?
what is the externally visible view of the feature?
Risk Scenarios: enumerate the bad things that could happen in the
system that either:
could be caused by the feature
could have an effect on the feature
Test Cases: enumerate all tests in tables
using table headings Id, Type of Test, Description, Expected Behavior,
Is Automated?
enumerate positive and negative test case types
include functional, system, scale, and performance test case types
indicate which tests are automated in JUnit suite
Past Design Doc Examples
http://issues.apache.org/jira/secure/attachment/12348296/DFSUpgradeProposal3.html
Past Test Plan Examples
https://issues.apache.org/jira/secure/attachment/12373559/PermissionsTestPlan2.pdf
https://issues.apache.org/jira/secure/attachment/12363605/BlockCrcFeatureTestPlan.pdf
https://issues.apache.org/jira/secure/attachment/12351299/TestPlan-HdfsUpgrade.html
Some examples requiring the above:
HADOOP-1700 (file append)
HADOOP-702 (DFS upgrades)
HADOOP-3938 (disk quotas)
HADOOP-372 (input format per dir)
HADOOP-153 (skip bad records)
Thoughts? Is this something we can all commit to do?
Cheers,
Nige
Hadoop Committers,
The Hadoop project has grown to become a critical platform for many
companies. As we approach a 1.0 release (perhaps within the next 6
months) I think it is time we adopt some more rigor around designing
and testing new features.
As committers, we are already given a couple duties that relate to
quality. Fromhttp://wiki.apache.org/hadoop/HowToCommit :
If a committer reviews a patch they've not authored, and believe it
to be of sufficient quality, then they can commit the patch,
otherwise the patch should be cancelled with a clear explanation
for why it was rejected.
...
Patches should be rejected which do not adhere to the guidelines inhttp://wiki.apache.org/hadoop/HowToContribute
and to thehttp://wiki.apache.org/hadoop/CodeReviewChecklist.
Committers should always be polite to contributors and try to
instruct and encourage them to contribute better patches.
These have been a good starting point, but I'd like to propose more
rigor.
I propose that before we commit issues marked as "New Feature", they
must have:
1. a design doc attachment
2. a test plan attachment
(Templates to be provided for both)
I believe these 2 items will enhance the communication, design, and
quality of our new features.
Are committers open to adopting these further guidelines?
Thanks,
Nige
