On Tue, Nov 11, 2014 at 7:43 AM, David Lang <[email protected]> wrote: > On Mon, 10 Nov 2014, Thomas D. wrote: > > Hi, >> >> On 2014-11-06 08:10, Rainer Gerhards wrote: >> >>> Well, you can run the mysql test suite against any mysql server. It is >>>> independent from source. So that's not the best example at all... >>>> >>> >>> Don't you then need root (or similar) credentials to do some very >>> delicate >>> database operations like creating databases and tables? I wouldn't permit >>> this to be done to my production server. >>> >> >> That's why I asked for running the test suite in a 'sandbox'. >> > > you still need those credentials for your sandbox, right? > > If MySQL isn't installed on the system you will need to install the > package (requiring root + distro specific knowledge), if it is installed, > you need to configure users and permissions for the test user (requiring DB > root equivalent + distro specific knowlege) > > > > Well, finally I installed valgrind and had to re-install glibc with >>>> debugging symbols (another error the test suite logged but did not >>>> detect). >>>> >>> >>> why did you need the debugging symbols? >>> >> >> In Gentoo imdiag is currently only available when compiling with "debug" >> USE flag. This will also enable "--enable-valgrind".. >> >> The error message logged in the "udp-msgreduc-vg" log file was something >> like >> >> valgrind: Fatal error at startup: a function redirection >>> valgrind: which is mandatory for this platform-tool combination >>> valgrind: cannot be set up. Details of the redirection are: >>> valgrind: >>> valgrind: [...] >>> valgrind: >>> valgrind: Possible fixes: (1, short term): install glibc's debuginfo >>> valgrind: package on this machine. (2, longer term): ask the packagers >>> valgrind: for your Linux distribution to please in future ship a non- >>> valgrind: stripped ld.so (or whatever the dynamic linker .so is called) >>> valgrind: that exports the above-named function using the standard >>> valgrind: calling conventions for this platform. >>> valgrind: >>> valgrind: Cannot continue -- exiting now. Sorry. >>> >> >> >> >> But let me be clear. From this thread, it looks like these should be our >>> project priorities: >>> >>> 1. make the testbench fully automatic and self-contained >>> 2. create unit tests >>> 3. create more and better integration tests >>> 4. document >>> 5. fix bugs >>> 6. develop new features >>> >>> Given the fact that I am I and not a big team like at MySQL, I don't >>> believe this is the right order of priorities. >>> >>> In reality, it is me alone doing >95% of all work, and funding for this >>> work is almost solely by Adiscon (and only partially backed by paying >>> customers). >>> >> >> Asking this question shows me that you understand my point. Great! ;) >> >> Yes, I am voting for changing the priority order like you said. >> >> Don't get me wrong. I am not saying that you MUST change anything. I am >> only answering the question "If you could change something in the >> project, what would you change?" ... and I would change the priority to >> improve the current project quality using automatic tests. >> > > I disagree that 1, 2, and 3 are more important than 4 and 5 (where 6 lives > is subject to a lot of debate) > > you could completely halt all documentation, bugfixing, and new features > for a year while creating tests to cover everything. > > Testing is useful, but the end purpose of the software is not to pass > tests, but rather to provide capabilities to the user. > > As an extreme example, I know of a company that lost a month of > development and missed having the product ready for a national trade show > because they got so caught up in writing tests for everything that a test > that didn't work right when the software was being shutdown (after which it > would be reset for the next test) HAD to be written before anything more > could be done, and there was an upstream bug in one component that caused > this exit to never work cleanly. Don't fall into a similar trap. >
+1 tests are just means to an end, we should be disciplined to write them as we go, but every project has corners(and sometimes large portions) that are ill-tested or worse. Paralysing development completely to test-cover them in one go doesn't sound like the best plan. FWIW, here is something that has worked for teams I have been on in the past. We were aware of which areas were not test-covered, bug-prone, hard-to-read etc. When we were to touch anything in those areas (like adding new features that touched those parts of code, or fix a bug etc), we used to test-cover directly and indirectly affected portions of that code, and then refactor them heavily to bring it up to the mark. > > David Lang > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > -- Regards, Janmejay http://codehunk.wordpress.com _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
