Hi all, After I managed to invoke the rspec test runner (see #1237), I realized with sadness that they are far from usable. I got: 3053 examples, 219 failures, 30 pending - when running as root and 3053 examples, 145 failures, 31 pending - when running as non-root This is on master branch.
Before you ask - no, the ones failing as non-root are not a subset of the ones failing as root. For me personally (and I think for anyone new to puppet, who would like to produce high-quality code) this is a huge barrier to puppet development. The reason is that until you gain intimate knowledge of how puppet works inside, there is no way you can be even remotely sure that a change that you are about to introduce won't break something. (Even with tests you cannot be certain, but at least you can be reasonably confident). So I have several questions: - do we at all agree that there should be NO failing tests at any point in time, and that any failing test is a bug? Even in master branch? And as a result, no commit should introduce a test failure? - is there anyone at all for whom all tests work at the moment? Or am I just very unlucky and it does work for everyone except for me? :) - how do we tackle this problem? I cannot go through 200 failing tests alone and fix them in reasonable time. We need to split the work somehow. - how do we make sure that this problem does not reappear? Should we set up some continuous integration environment and assume "project culture" would convince developers to fix the problems? Or should we go a step further and only ever release software tested and built by the continuous integration service? I know someone (Luke?) mentioned that it would be nice to have a build farm which would run the tests on all supported platforms. But I think that running them on a regular basis even on a single platform would be much better than the current situation. -- Marcin Owsiany <[EMAIL PROTECTED]> http://marcin.owsiany.pl/ GnuPG: 1024D/60F41216 FE67 DA2D 0ACA FC5E 3F75 D6F6 3A0D 8AA0 60F4 1216 "Every program in development at MIT expands until it can read mail." -- Unknown --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en -~----------~----~----~----~------~----~------~--~---
