On Tue, Aug 18, 2015 at 02:03:19PM +0100, Greg Stark wrote:
On Tue, Aug 18, 2015 at 6:57 AM, Noah Misch n...@leadboat.com wrote:
My own position is based on having maintained a pg_regress suite an order of
magnitude larger than that. I don't know why that outcome was so different.
And does
On Tue, Aug 18, 2015 at 3:32 PM, David Fetter da...@fetter.org wrote:
On Tue, Aug 18, 2015 at 04:54:07PM +0100, Greg Stark wrote:
On Tue, Aug 18, 2015 at 2:16 PM, David Fetter da...@fetter.org wrote:
I'm given to understand that this tight coupling is necessary for
performance. Are you
On Tue, Aug 18, 2015 at 02:03:19PM +0100, Greg Stark wrote:
On Tue, Aug 18, 2015 at 6:57 AM, Noah Misch n...@leadboat.com wrote:
I suspect any effort to significantly improve Postgres test
coverage is doomed until there's an alternative to pg_regress.
There is the
On Tue, Aug 18, 2015 at 6:57 AM, Noah Misch n...@leadboat.com wrote:
My own position is based on having maintained a pg_regress suite an order of
magnitude larger than that. I don't know why that outcome was so different.
Comparing the size of test suites by these numbers is impossible
because
On Tue, Aug 18, 2015 at 04:54:07PM +0100, Greg Stark wrote:
On Tue, Aug 18, 2015 at 2:16 PM, David Fetter da...@fetter.org wrote:
I'm given to understand that this tight coupling is necessary for
performance. Are you saying that it could be unwound, or that
testing strategies mostly need
On Tue, Aug 18, 2015 at 2:16 PM, David Fetter da...@fetter.org wrote:
I'm given to understand that this tight coupling is necessary for
performance. Are you saying that it could be unwound, or that testing
strategies mostly need to take it into account, or...?
I'm just saying that we
On Mon, Aug 17, 2015 at 02:04:40PM -0500, Jim Nasby wrote:
On 8/16/15 8:48 AM, Greg Stark wrote:
On Sun, Aug 16, 2015 at 7:33 AM, Noah Misch n...@leadboat.com wrote:
When I've just spent awhile implementing a behavior change, the test diffs
are
a comforting sight. They confirm that the test
On 8/15/15 4:45 AM, Petr Jelinek wrote:
We could fix a) by adding ORDER BY to those queries but I don't see how
to fix the rest easily or at all without sacrificing some test coverage.
Hopefully at some point we'll have test frameworks that don't depend on
capturing raw psql output, which
On 8/16/15 8:48 AM, Greg Stark wrote:
On Sun, Aug 16, 2015 at 7:33 AM, Noah Misch n...@leadboat.com wrote:
When I've just spent awhile implementing a behavior change, the test diffs are
a comforting sight. They confirm that the test suite exercises the topic I
just changed. Furthermore, most
On Wed, Aug 12, 2015 at 06:46:19PM +0100, Greg Stark wrote:
On Wed, Aug 12, 2015 at 3:10 AM, Noah Misch n...@leadboat.com wrote:
Committers press authors to delete tests more often than we press them to
resubmit with more tests. No wonder so many patches have insufficient
tests;
we
On Fri, Aug 14, 2015 at 12:47:49PM +0100, Simon Riggs wrote:
On 13 August 2015 at 00:31, Robert Haas robertmh...@gmail.com wrote:
On Wed, Aug 12, 2015 at 7:20 PM, Tom Lane t...@sss.pgh.pa.us wrote:
We've talked about having some sort of second rank of tests that
people wouldn't
On Sun, Aug 16, 2015 at 7:33 AM, Noah Misch n...@leadboat.com wrote:
When I've just spent awhile implementing a behavior change, the test diffs are
a comforting sight. They confirm that the test suite exercises the topic I
just changed. Furthermore, most tests today do not qualify under this
On 2015-08-15 03:35, Jim Nasby wrote:
I setup a simple example of this with 64 variations of TAP tests, BLKSZ
and WAL blocksize. Unfortunately to make this work you have to commit a
.travis.yml file to your fork.
build: https://travis-ci.org/decibel/postgres/builds/75692344
.travis.yml:
On 13 August 2015 at 00:31, Robert Haas robertmh...@gmail.com wrote:
On Wed, Aug 12, 2015 at 7:20 PM, Tom Lane t...@sss.pgh.pa.us wrote:
FWIW, I've objected in the past to tests that would significantly
increase the runtime of make check, unless I thought they were
especially valuable
On 8/14/15 12:11 AM, Jim Nasby wrote:
I favor splitting the regression tests to add all the time and
before commit targets as you describe. I think that once the
facility is there, we can determine over time how expansive that
second category gets to be.
I don't know how many folks work in a
On Thu, Aug 13, 2015 at 1:31 AM, Robert Haas robertmh...@gmail.com wrote:
On Wed, Aug 12, 2015 at 7:20 PM, Tom Lane t...@sss.pgh.pa.us wrote:
FWIW, I've objected in the past to tests that would significantly
increase the runtime of make check, unless I thought they were
especially valuable
FWIW, I've objected in the past to tests that would significantly
increase the runtime of make check, unless I thought they were
especially valuable (which enumerating every minor behavior of a
feature patch generally isn't IMO). I still think that that's an
important consideration: every
On Wed, Aug 12, 2015 at 11:23 PM, Magnus Hagander mag...@hagander.net wrote:
The value of a core regression suite that takes less time to run has
to be weighed against the possibility that a better core regression
suite might cause us to find more bugs before committing. That could
easily be
On 8/12/15 9:24 PM, Stephen Frost wrote:
* Michael Paquier (michael.paqu...@gmail.com) wrote:
On Thu, Aug 13, 2015 at 5:54 AM, Stephen Frost wrote:
The regression tests included in pgBackRest (available here:
https://github.com/pgmasters/backrest) go through a number of different
recovery
On 2015-08-13 09:32:02 -0400, David Steele wrote:
On 8/12/15 9:32 PM, Robert Haas wrote:
On Wed, Aug 12, 2015 at 9:24 PM, Stephen Frost sfr...@snowman.net wrote:
Certainly don't mind at all, entirely open source under the MIT
license.
Why not the PG license? It would be nicer if we didn't
On 8/12/15 9:32 PM, Robert Haas wrote:
On Wed, Aug 12, 2015 at 9:24 PM, Stephen Frost sfr...@snowman.net wrote:
* Michael Paquier (michael.paqu...@gmail.com) wrote:
Interesting. Do you mind if I pick up from it some ideas for the
in-core replication test suite based on TAP stuff? That's still
On 8/13/15 9:55 AM, Andres Freund wrote:
On 2015-08-13 09:32:02 -0400, David Steele wrote:
On 8/12/15 9:32 PM, Robert Haas wrote:
On Wed, Aug 12, 2015 at 9:24 PM, Stephen Frost sfr...@snowman.net wrote:
Certainly don't mind at all, entirely open source under the MIT
license.
Why not the PG
On 8/13/15 1:31 AM, Peter Geoghegan wrote:
On Wed, Aug 12, 2015 at 11:23 PM, Magnus Hagander mag...@hagander.net wrote:
The value of a core regression suite that takes less time to run has
to be weighed against the possibility that a better core regression
suite might cause us to find more bugs
* Robert Haas (robertmh...@gmail.com) wrote:
On Wed, Aug 12, 2015 at 2:04 PM, Peter Geoghegan p...@heroku.com wrote:
This resistance to adding tests seems quite short sighted to me,
especially when the concern is about queries that will each typically
take less than 1ms to execute. Like
On Wed, Aug 12, 2015 at 7:20 PM, Tom Lane t...@sss.pgh.pa.us wrote:
FWIW, I've objected in the past to tests that would significantly
increase the runtime of make check, unless I thought they were
especially valuable (which enumerating every minor behavior of a
feature patch generally isn't
Robert Haas robertmh...@gmail.com writes:
On Wed, Aug 12, 2015 at 2:04 PM, Peter Geoghegan p...@heroku.com wrote:
This resistance to adding tests seems quite short sighted to me,
especially when the concern is about queries that will each typically
take less than 1ms to execute. Like Noah, I
On Wed, Aug 12, 2015 at 2:04 PM, Peter Geoghegan p...@heroku.com wrote:
This resistance to adding tests seems quite short sighted to me,
especially when the concern is about queries that will each typically
take less than 1ms to execute. Like Noah, I think that it would be
very helpful to
On Thu, Aug 13, 2015 at 5:54 AM, Stephen Frost wrote:
The regression tests included in pgBackRest (available here:
https://github.com/pgmasters/backrest) go through a number of different
recovery tests. There's vagrant configs for a few different VMs too
(CentOS 6, CentOS 7, Ubuntu 12.04 and
On Wed, Aug 12, 2015 at 9:24 PM, Stephen Frost sfr...@snowman.net wrote:
* Michael Paquier (michael.paqu...@gmail.com) wrote:
On Thu, Aug 13, 2015 at 5:54 AM, Stephen Frost wrote:
The regression tests included in pgBackRest (available here:
https://github.com/pgmasters/backrest) go through a
* Michael Paquier (michael.paqu...@gmail.com) wrote:
On Thu, Aug 13, 2015 at 5:54 AM, Stephen Frost wrote:
The regression tests included in pgBackRest (available here:
https://github.com/pgmasters/backrest) go through a number of different
recovery tests. There's vagrant configs for a few
* Robert Haas (robertmh...@gmail.com) wrote:
On Wed, Aug 12, 2015 at 9:24 PM, Stephen Frost sfr...@snowman.net wrote:
* Michael Paquier (michael.paqu...@gmail.com) wrote:
On Thu, Aug 13, 2015 at 5:54 AM, Stephen Frost wrote:
The regression tests included in pgBackRest (available here:
On 12 August 2015 at 03:10, Noah Misch n...@leadboat.com wrote:
On another review I suggested we add a function to core to allow it to be
used in regression tests. A long debate ensued, deciding that we must be
consistent and put diagnostic functions in contrib. My understanding is
that
On Wed, Aug 12, 2015 at 10:46 AM, Greg Stark st...@mit.edu wrote:
The only time I've seen pushback against tests is when the test author
made valiant efforts to test every codepath and the expected output
embeds the precise behaviour of the current code as correct. Even
when patches have
On Wed, Aug 12, 2015 at 3:10 AM, Noah Misch n...@leadboat.com wrote:
Committers press authors to delete tests more often than we press them to
resubmit with more tests. No wonder so many patches have insufficient tests;
we treat those patches more favorably, on average. I have no objective
One trouble I face when adding tests is that sometimes they require
hooks in the code, to test for race conditions. In BRIN I cannot test
some code paths without resorting to adding breakpoints in GDB, for
instance. If there's no support for such in the core code, it's
essentially impossible to
On Mon, Aug 10, 2015 at 07:02:17AM +0100, Simon Riggs wrote:
Almost every patch I review has either zero or insufficient tests.
If we care about robustness, then we must discuss tests. Here are my two
recent experiences:
I agree we could do with x10 as many tests, but that doesn't mean all
On Mon, Aug 10, 2015 at 2:02 AM, Simon Riggs si...@2ndquadrant.com wrote:
Almost every patch I review has either zero or insufficient tests.
If we care about robustness, then we must discuss tests. Here are my two
recent experiences:
I agree we could do with x10 as many tests, but that
On 8 August 2015 at 17:47, Noah Misch n...@leadboat.com wrote:
We've too often criticized patches for carrying many lines/bytes of test
case
additions. Let's continue to demand debuggable, secure tests that fail
only
when something is wrong, but let's stop pushing for test minimalism. Such
On 10 August 2015 at 13:55, Robert Haas robertmh...@gmail.com wrote:
On Mon, Aug 10, 2015 at 2:02 AM, Simon Riggs si...@2ndquadrant.com
wrote:
On another review I suggested we add a function to core to allow it to be
used in regression tests. A long debate ensued, deciding that we must be
We've too often criticized patches for carrying many lines/bytes of test case
additions. Let's continue to demand debuggable, secure tests that fail only
when something is wrong, but let's stop pushing for test minimalism. Such
objections may improve the individual patch, but that doesn't make
On Sat, Aug 8, 2015 at 9:47 AM, Noah Misch n...@leadboat.com wrote:
We've too often criticized patches for carrying many lines/bytes of test case
additions. Let's continue to demand debuggable, secure tests that fail only
when something is wrong, but let's stop pushing for test minimalism.
On 08/08/2015 12:24 PM, Peter Geoghegan wrote:
I think that there needs to be a way of running an extended set of
regression tests. I could definitely respect the desire for minimalism
when it comes to adding tests to the regression tests proper if there
was an extended set of tests that could
On 08/08/2015 05:09 PM, Josh Berkus wrote:
On 08/08/2015 12:24 PM, Peter Geoghegan wrote:
I think that there needs to be a way of running an extended set of
regression tests. I could definitely respect the desire for minimalism
when it comes to adding tests to the regression tests proper if
On Sat, Aug 8, 2015 at 8:24 PM, Peter Geoghegan p...@heroku.com wrote:
I think that there needs to be a way of running an extended set of
regression tests. I could definitely respect the desire for minimalism
The larger expense in having extensive test suites is the cost to
maintain them. With
44 matches
Mail list logo
