On 5/7/16 9:36 AM, Stephen Frost wrote:
Honestly, over the next couple of months between feature-freeze and release, I'd like to add even more tests, and not just to pg_dump but also to other commands that don't have very good testing today (psql, in particular, but pg_dumpall needs more also, and there's more to do with pg_dump too).
If we're going to do that, there will be no stopping it. I also have a bunch of code in this area lined up, but I was hoping to submit it to the commit fest process at the right time and order to get feedback and testing. If we're going to allow new tests being developed right up until release, I can just stop doing release preparation work right now and get back to coding and reviewing.
Ultimately, tests are code and should be treated as such.
When it comes to packaging, if adding tests using the existing test infrastructure (the TAP system) causes a problem there, then I think we need to address that issue rather than not add new tests. Packagers also always have the option to not enable the tap tests, if there really is an issue there which can't be addressed.
Well, that's like saying, if the feature I just committed the night before the release turns out to be problematic, you can just ifdef it out. I don't think we want that, and I think it simplifies the way the world works. Because new code interacts with old code, and then there will be even more new code which interacts with other code, and so it becomes harder and harder to isolate issues. Yeah, if adding more tests causes instabilities because of issues not related to the tests themselves, we should fix that. But that doesn't mean we should hand-wave past the fact that that potential exists. That's software development. If everything were perfect, we wouldn't need a beta period at all.
The configure flag to disable TAP tests isn't there so that we get license to play with them at random, it's because we wanted to make the software dependency optional. The psql tests run under pg_regress, so they can't be made optional anyway.
-- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
