On 10/27/14 11:41 AM, Robert Haas wrote: > Beyond all that, I have serious doubts about whether, even if we > eventually get these tests mostly working in most places, whether they > will actually catch any bugs.
Well, they caught the fact that pg_basebackup can't back up tablespaces with names longer than 99 characters, for example. But it's wrong to expect the primary value of tests to be to detect previously unknown bugs. Yes, that has been the experience in this project. We have software that we think works, and then we send it out to test on N obscure platforms, and we find interesting things. But the other dimension is that tests allow you to make changes with confidence. For example, the tablespace mapping logic in pg_basebackup has been whacked around about three times. I don't have any doubt that it still works correctly, because it has extensive test coverage. If someone sends in a patch to support cpio in pg_basebackup (hey, no symlink name length limits) and wants to refactor the entire source code to achieve that, I'll have no problem with that. Yes, some of the tests are low in value, perhaps a bit silly. But it's a foundation, and I expect to add more in the future. The good news is that because some of these "silly" tests actually exercise important internal functionality of the test mechanisms (e.g., create temporary directory, run program, record exit status, capture stdout and stderr separately, check output files), I am pretty confident that we can go far with the current infrastructure without needing any more external modules or something like that. So that also means that not all tests need to be run everywhere all the time. We have the option to run specific tests. It could be useful to have better groupings, but that has been discussed before and it hasn't come to a solution. > These are certainly good things to test, but I'd argue that once > you've verified that they are working, they're unlikely to get broken > again in the future. Famous last words ... ;-) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
