On Tue, Aug 18, 2015 at 3:32 PM, David Fetter wrote:
> On Tue, Aug 18, 2015 at 04:54:07PM +0100, Greg Stark wrote:
> > On Tue, Aug 18, 2015 at 2:16 PM, David Fetter wrote:
> > > I'm given to understand that this tight coupling is necessary for
> > > performance. Are you saying that it could be
On Tue, Aug 18, 2015 at 02:03:19PM +0100, Greg Stark wrote:
> On Tue, Aug 18, 2015 at 6:57 AM, Noah Misch wrote:
> > My own position is based on having maintained a pg_regress suite an order of
> > magnitude larger than that. I don't know why that outcome was so different.
> And does your pg_reg
On Tue, Aug 18, 2015 at 04:54:07PM +0100, Greg Stark wrote:
> On Tue, Aug 18, 2015 at 2:16 PM, David Fetter wrote:
> > I'm given to understand that this tight coupling is necessary for
> > performance. Are you saying that it could be unwound, or that
> > testing strategies mostly need to take it
On Tue, Aug 18, 2015 at 2:16 PM, David Fetter wrote:
> I'm given to understand that this tight coupling is necessary for
> performance. Are you saying that it could be unwound, or that testing
> strategies mostly need to take it into account, or...?
I'm just saying that we shouldn't expect to fi
On Tue, Aug 18, 2015 at 02:03:19PM +0100, Greg Stark wrote:
> On Tue, Aug 18, 2015 at 6:57 AM, Noah Misch wrote:
> >> I suspect any effort to significantly improve Postgres test
> >> coverage is doomed until there's an alternative to pg_regress.
> >
> > There is the src/test/perl/TestLib.pm harne
On Tue, Aug 18, 2015 at 6:57 AM, Noah Misch wrote:
> My own position is based on having maintained a pg_regress suite an order of
> magnitude larger than that. I don't know why that outcome was so different.
Comparing the size of test suites by these numbers is impossible
because people put more
On Mon, Aug 17, 2015 at 02:04:40PM -0500, Jim Nasby wrote:
> On 8/16/15 8:48 AM, Greg Stark wrote:
> >On Sun, Aug 16, 2015 at 7:33 AM, Noah Misch wrote:
> >>When I've just spent awhile implementing a behavior change, the test diffs
> >>are
> >>a comforting sight. They confirm that the test suite
On 8/16/15 8:48 AM, Greg Stark wrote:
On Sun, Aug 16, 2015 at 7:33 AM, Noah Misch wrote:
When I've just spent awhile implementing a behavior change, the test diffs are
a comforting sight. They confirm that the test suite exercises the topic I
just changed. Furthermore, most tests today do not
On 8/15/15 4:45 AM, Petr Jelinek wrote:
We could fix a) by adding ORDER BY to those queries but I don't see how
to fix the rest easily or at all without sacrificing some test coverage.
Hopefully at some point we'll have test frameworks that don't depend on
capturing raw psql output, which pres
On Sun, Aug 16, 2015 at 7:33 AM, Noah Misch wrote:
> When I've just spent awhile implementing a behavior change, the test diffs are
> a comforting sight. They confirm that the test suite exercises the topic I
> just changed. Furthermore, most tests today do not qualify under this
> stringent met
On Fri, Aug 14, 2015 at 12:47:49PM +0100, Simon Riggs wrote:
> On 13 August 2015 at 00:31, Robert Haas wrote:
> > On Wed, Aug 12, 2015 at 7:20 PM, Tom Lane wrote:
> > > We've talked about having some sort of second rank of tests that
> > > people wouldn't necessarily run before committing, and th
On Wed, Aug 12, 2015 at 06:46:19PM +0100, Greg Stark wrote:
> On Wed, Aug 12, 2015 at 3:10 AM, Noah Misch wrote:
> > Committers press authors to delete tests more often than we press them to
> > resubmit with more tests. No wonder so many patches have insufficient
> > tests;
> > we treat those p
On 2015-08-15 03:35, Jim Nasby wrote:
I setup a simple example of this with 64 variations of TAP tests, BLKSZ
and WAL blocksize. Unfortunately to make this work you have to commit a
.travis.yml file to your fork.
build: https://travis-ci.org/decibel/postgres/builds/75692344
.travis.yml: https:/
On 8/14/15 12:11 AM, Jim Nasby wrote:
I favor splitting the regression tests to add "all the time" and
"before commit" targets as you describe. I think that once the
facility is there, we can determine over time how expansive that
second category gets to be.
I don't know how many folks work in
On 13 August 2015 at 00:31, Robert Haas wrote:
> On Wed, Aug 12, 2015 at 7:20 PM, Tom Lane wrote:
> > FWIW, I've objected in the past to tests that would significantly
> > increase the runtime of "make check", unless I thought they were
> > especially valuable (which enumerating every minor beha
On 8/13/15 1:31 AM, Peter Geoghegan wrote:
On Wed, Aug 12, 2015 at 11:23 PM, Magnus Hagander wrote:
The value of a core regression suite that takes less time to run has
to be weighed against the possibility that a better core regression
suite might cause us to find more bugs before committing.
On 8/13/15 9:55 AM, Andres Freund wrote:
On 2015-08-13 09:32:02 -0400, David Steele wrote:
On 8/12/15 9:32 PM, Robert Haas wrote:
On Wed, Aug 12, 2015 at 9:24 PM, Stephen Frost wrote:
Certainly don't mind at all, entirely open source under the MIT
license.
Why not the PG license? It would
On 2015-08-13 09:32:02 -0400, David Steele wrote:
> On 8/12/15 9:32 PM, Robert Haas wrote:
> >On Wed, Aug 12, 2015 at 9:24 PM, Stephen Frost wrote:
> >>Certainly don't mind at all, entirely open source under the MIT
> >>license.
> >
> >Why not the PG license? It would be nicer if we didn't have t
On 8/12/15 9:32 PM, Robert Haas wrote:
On Wed, Aug 12, 2015 at 9:24 PM, Stephen Frost wrote:
* Michael Paquier (michael.paqu...@gmail.com) wrote:
Interesting. Do you mind if I pick up from it some ideas for the
in-core replication test suite based on TAP stuff? That's still in the
works for th
On 8/12/15 9:24 PM, Stephen Frost wrote:
* Michael Paquier (michael.paqu...@gmail.com) wrote:
On Thu, Aug 13, 2015 at 5:54 AM, Stephen Frost wrote:
The regression tests included in pgBackRest (available here:
https://github.com/pgmasters/backrest) go through a number of different
recovery tests
On Wed, Aug 12, 2015 at 11:23 PM, Magnus Hagander wrote:
>> The value of a core regression suite that takes less time to run has
>> to be weighed against the possibility that a better core regression
>> suite might cause us to find more bugs before committing. That could
>> easily be worth the pr
On Thu, Aug 13, 2015 at 1:31 AM, Robert Haas wrote:
> On Wed, Aug 12, 2015 at 7:20 PM, Tom Lane wrote:
> > FWIW, I've objected in the past to tests that would significantly
> > increase the runtime of "make check", unless I thought they were
> > especially valuable (which enumerating every minor
FWIW, I've objected in the past to tests that would significantly
increase the runtime of "make check", unless I thought they were
especially valuable (which enumerating every minor behavior of a
feature patch generally isn't IMO). I still think that that's an
important consideration: every sec
* Robert Haas (robertmh...@gmail.com) wrote:
> On Wed, Aug 12, 2015 at 9:24 PM, Stephen Frost wrote:
> > * Michael Paquier (michael.paqu...@gmail.com) wrote:
> >> On Thu, Aug 13, 2015 at 5:54 AM, Stephen Frost wrote:
> >> > The regression tests included in pgBackRest (available here:
> >> > https:
On Wed, Aug 12, 2015 at 9:24 PM, Stephen Frost wrote:
> * Michael Paquier (michael.paqu...@gmail.com) wrote:
>> On Thu, Aug 13, 2015 at 5:54 AM, Stephen Frost wrote:
>> > The regression tests included in pgBackRest (available here:
>> > https://github.com/pgmasters/backrest) go through a number of
* Michael Paquier (michael.paqu...@gmail.com) wrote:
> On Thu, Aug 13, 2015 at 5:54 AM, Stephen Frost wrote:
> > The regression tests included in pgBackRest (available here:
> > https://github.com/pgmasters/backrest) go through a number of different
> > recovery tests. There's vagrant configs for
On Thu, Aug 13, 2015 at 5:54 AM, Stephen Frost wrote:
> The regression tests included in pgBackRest (available here:
> https://github.com/pgmasters/backrest) go through a number of different
> recovery tests. There's vagrant configs for a few different VMs too
> (CentOS 6, CentOS 7, Ubuntu 12.04 a
On Wed, Aug 12, 2015 at 7:20 PM, Tom Lane wrote:
> FWIW, I've objected in the past to tests that would significantly
> increase the runtime of "make check", unless I thought they were
> especially valuable (which enumerating every minor behavior of a
> feature patch generally isn't IMO). I still
Robert Haas writes:
> On Wed, Aug 12, 2015 at 2:04 PM, Peter Geoghegan wrote:
>> This resistance to adding tests seems quite short sighted to me,
>> especially when the concern is about queries that will each typically
>> take less than 1ms to execute. Like Noah, I think that it would be
>> very
* Robert Haas (robertmh...@gmail.com) wrote:
> On Wed, Aug 12, 2015 at 2:04 PM, Peter Geoghegan wrote:
> > This resistance to adding tests seems quite short sighted to me,
> > especially when the concern is about queries that will each typically
> > take less than 1ms to execute. Like Noah, I thin
On Wed, Aug 12, 2015 at 2:04 PM, Peter Geoghegan wrote:
> This resistance to adding tests seems quite short sighted to me,
> especially when the concern is about queries that will each typically
> take less than 1ms to execute. Like Noah, I think that it would be
> very helpful to simply be more i
One trouble I face when adding tests is that sometimes they require
hooks in the code, to test for race conditions. In BRIN I cannot test
some code paths without resorting to adding breakpoints in GDB, for
instance. If there's no support for such in the core code, it's
essentially impossible to a
On Wed, Aug 12, 2015 at 10:46 AM, Greg Stark wrote:
> The only time I've seen pushback against tests is when the test author
> made valiant efforts to test every codepath and the expected output
> embeds the precise behaviour of the current code as "correct". Even
> when patches have extensive tes
On Wed, Aug 12, 2015 at 3:10 AM, Noah Misch wrote:
> Committers press authors to delete tests more often than we press them to
> resubmit with more tests. No wonder so many patches have insufficient tests;
> we treat those patches more favorably, on average. I have no objective
> principles for
On 12 August 2015 at 03:10, Noah Misch wrote:
> > On another review I suggested we add a function to core to allow it to be
> > used in regression tests. A long debate ensued, deciding that we must be
> > consistent and put diagnostic functions in contrib. My understanding is
> > that we are not
On Mon, Aug 10, 2015 at 07:02:17AM +0100, Simon Riggs wrote:
> Almost every patch I review has either zero or insufficient tests.
>
> If we care about robustness, then we must discuss tests. Here are my two
> recent experiences:
>
> I agree we could do with x10 as many tests, but that doesn't mea
On 10 August 2015 at 13:55, Robert Haas wrote:
> On Mon, Aug 10, 2015 at 2:02 AM, Simon Riggs
> wrote:
> > On another review I suggested we add a function to core to allow it to be
> > used in regression tests. A long debate ensued, deciding that we must be
> > consistent and put diagnostic func
On Mon, Aug 10, 2015 at 2:02 AM, Simon Riggs wrote:
> Almost every patch I review has either zero or insufficient tests.
>
> If we care about robustness, then we must discuss tests. Here are my two
> recent experiences:
>
> I agree we could do with x10 as many tests, but that doesn't mean all test
On 8 August 2015 at 17:47, Noah Misch wrote:
> We've too often criticized patches for carrying many lines/bytes of test
> case
> additions. Let's continue to demand debuggable, secure tests that fail
> only
> when something is wrong, but let's stop pushing for test minimalism. Such
> objections
On Sat, Aug 8, 2015 at 8:24 PM, Peter Geoghegan wrote:
> I think that there needs to be a way of running an extended set of
> regression tests. I could definitely respect the desire for minimalism
The larger expense in having extensive test suites is the cost to
maintain them. With our current te
On 08/08/2015 05:09 PM, Josh Berkus wrote:
On 08/08/2015 12:24 PM, Peter Geoghegan wrote:
I think that there needs to be a way of running an extended set of
regression tests. I could definitely respect the desire for minimalism
when it comes to adding tests to the regression tests proper if the
On 08/08/2015 12:24 PM, Peter Geoghegan wrote:
> I think that there needs to be a way of running an extended set of
> regression tests. I could definitely respect the desire for minimalism
> when it comes to adding tests to the regression tests proper if there
> was an extended set of tests that co
On Sat, Aug 8, 2015 at 9:47 AM, Noah Misch wrote:
> We've too often criticized patches for carrying many lines/bytes of test case
> additions. Let's continue to demand debuggable, secure tests that fail only
> when something is wrong, but let's stop pushing for test minimalism. Such
> objections
We've too often criticized patches for carrying many lines/bytes of test case
additions. Let's continue to demand debuggable, secure tests that fail only
when something is wrong, but let's stop pushing for test minimalism. Such
objections may improve the individual patch, but that doesn't make up
44 matches
Mail list logo
