On 4 October 2013 05:11, "Paweł Hajdan, Jr." <[email protected]> wrote:
> Even then, no amount of testing guarantees lack of problems. Indeed, but which is a better assurance, "5 testers tested this combination and nothing bad happened", or "5000 people tested this combination and nothing bad happened". Now, if you were to see "no people have successfully built combination X", that in itself is interesting, even if you don't have actual failure reports of that combination. Also, if "5 testers tested this combination and nothing bad happened" is combined with "however, we have 200 similar installation failures reported for this combination", you've got some context for research you need to do to understand why those failures exist ( even if none of them managed to file a bug report ). Essentially, I'm saying we need to lower the thresholds to providing reliable feedback about what is happening with packages in the field, ie: Diego's smoke boxes are very very useful, but thats *one* person. Imagine if we can get 500+ people running similar smoke operations with a manageable feedback system. That would give us fare more assurance than the arch testers are likely to be able to provide. -- Kent
