Hi,
Kevin Grittner wrote:
I just got to the point of having what appears to be a working but
poorly optimized version of serializable transactions,
Cool.
so it is
critical that I create a good set of tests to confirm correct
behavior and monitor for regressions as I optimize. I see that
[off-list to avoid distracting from the 9.0 wrap-up effort]
Markus Wanner mar...@bluegap.ch wrote:
Quoting Kevin Grittner kevin.gritt...@wicourts.gov:
I strongly encourage you to set that up on git.postgresql.org.
I'm about to provide git repositories for Postgres-R anyway, so
I've setup
I wrote:
[off-list to avoid distracting from the 9.0 wrap-up effort]
Arg. I really didn't mean that to go to the list. :-(
-Kevin
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Hi,
Jan Urbański wrote:
sorry to butt in to the conversation, but I have spent some time
wrapping/refining the concepts in dtester, and the results are here:
http://git.wulczer.org/?p=twisted-psql.git;a=summary
That seems to cover the concurrent psql part of dtester. But I don't see
how
Hi,
Quoting Kevin Grittner kevin.gritt...@wicourts.gov:
I strongly encourage you to set that up on git.postgresql.org.
I'm about to provide git repositories for Postgres-R anyway, so I've
setup two projects on git.postgres-r.org:
dtester: that's the driver/harness code
postgres-dtests: a
Markus Wanner wrote:
I do want to expand the tests quite a bit -- do I work them all into
this same file, or how would I proceed? I think I'll need about 20
more tests, but I don't want to get in the way of your work on the
framework which runs them.
Well, first of all, another piece of
Markus Wanner wrote:
Kevin Grittner wrote:
I differentiate tests and test suites. Tests mainly have a run
method, while test suites have setUp and tearDown ones.
I hadn't caught on to that distinction yet. That should help.
uses means that the referenced task has complimentary setUp and
Markus Wanner wrote:
Kevin Grittner wrote:
args=['psql', '-A', '--pset=pager=off',
That looks like a correct fix for psql, yes.
Other processes might be confused by (or at least act differently
with) a PAGER env variable, so that still needs to be cleared in
general.
I
Markus Wanner mar...@bluegap.ch wrote:
I must admit that I haven't ever tested on python 2.6 before. I'll
try that (especially as it's the staircase to 3.0, IIUC).
I don't use python much, so I can't comment on that. I do see that
my system has these two versions on it, with a symlink that
I wrote:
I'll also attach the results of the 2.6 attempt.
Let's try that again.
-Kevin
kgri...@kgrittn-desktop:~/git/postgresql/kgrittn$ make dcheck
make -C src/test dcheck
make[1]: Entering directory `/home/kgrittn/git/postgresql/kgrittn/src/test'
make
Hi,
Quoting Kevin Grittner kevin.gritt...@wicourts.gov:
I haven't quite gotten it to work yet; I'll start over with 3.0 and
see how it goes.
Let's stick to 2.x versions, first...
I'll also attach the results of the 2.6 attempt.
Thanks, that looks already pretty promising. ;-)
A few
Markus Wanner mar...@bluegap.ch wrote:
Quoting Kevin Grittner kevin.gritt...@wicourts.gov:
I haven't quite gotten it to work yet; I'll start over with 3.0
and see how it goes.
Let's stick to 2.x versions, first...
OK
It does: temp_install: creating temporary installation means
it's
Markus Wanner mar...@bluegap.ch wrote:
Second: at the very end of pg_dtester.py, you find the line:
reporter = StreamReporter()
Try a CursesReporter() instead, it gives much nicer output!
When I try to do that, Kate complains (I'm even copying their typo):
You are trying to save a
-BEGIN PGP SIGNED MESSAGE-
Hash: RIPEMD160
Dead or not, it still works, even against 8.4. I have many programs
that use it. It's simply a wrapper around the libpq interface and as
long as the libpq interface remains stable (which we go to great pains
to
Hi,
Kevin Grittner wrote:
Not sure what's relevant there. Entire file tarball attached.
Due to reasons mentioned in this thread as well, I've decided to use
psql to connect to the database. dtester is parsing its output and
checks that against expectations. Hawever, that has its own
Hi,
Kevin Grittner wrote:
You are trying to save a python file as non ASCII, without
specifiying a correct source encoding line for encoding utf-8
I wasn't aware I had non-ascii characters in there. Inserting an
encoding line seems fine. I'll fix that for the upcoming version 0.1.
Regards
Markus Wanner mar...@bluegap.ch wrote:
I wasn't aware I had non-ascii characters in there. Inserting an
encoding line seems fine. I'll fix that for the upcoming version
0.1.
Yeah, I couldn't find any, either. I just tried creating a minimal
python file in Kate, and it gave me that even
Markus Wanner mar...@bluegap.ch wrote:
Strangely, your log has escape codes in it, which I'm assuming
makes the parsing choke. Is that something special to your
installation? My psql never colors its outputs...
I haven't configured anything like that intentionally. I don't
*see* any colors
Hi,
Kevin Grittner wrote:
I haven't configured anything like that intentionally. I don't
*see* any colors when I use psql. Can you think of anywhere I
should check something which might be causing this?
No idea ATM.
However, just to make sure that has absolutely nothing to do with the
Markus Wanner mar...@bluegap.ch wrote:
Strangely, your log has escape codes in it, which I'm assuming
makes the parsing choke. Is that something special to your
installation?
My pager is less; could that cause it? Could the twisted
environment look like one where the pager should kick in?
Hi,
Kevin Grittner wrote:
My pager is less; could that cause it? Could the twisted
environment look like one where the pager should kick in?
Yes, that could be it. At least it fails here, too, if I set PAGER=less.
Try:
PAGER=more make dcheck
So, the solution probably lies in adjusting
Markus Wanner wrote:
Hi,
Kevin Grittner wrote:
My pager is less; could that cause it? Could the twisted
environment look like one where the pager should kick in?
Yes, that could be it. At least it fails here, too, if I set
PAGER=less. Try:
PAGER=more make dcheck
Surely for
Markus Wanner mar...@bluegap.ch wrote:
So, the solution probably lies in adjusting the environment,
before starting psql. (Maybe even dropping all existing
environment variables for better control of the situation). Will
add that for dtester 0.1.
Based on Andrew's suggestion, I changed
Kevin Grittner kevin.gritt...@wicourts.gov wrote:
Also, in looking closer at how you have the tests defined, it
doesn't look to me like you're carefully interleaving specific
sequences of statements on specific connections so much as opening
multiple connections and then for each statement
Markus Wanner mar...@bluegap.ch wrote:
Please recheck with the StreamReporter and try to grep the lines
starting with [psql0], [psql1] and [psql2]. Dtester simply
logs all and any output of all 3rd party processes started.
For me, all psql output seems to be [psql0]; no [psql1] or [psql2].
Markus Wanner mar...@bluegap.ch wrote:
Go try it, read the code and simply ask, if you get stuck. I'll
try to come up with some more documentation and such...
I'm a little unclear about the differences between uses,
depends, and onlyAfter. Here's what they *sound* like they
mean, to me;
Hi,
Kevin Grittner wrote:
I'm a little unclear about the differences between uses,
depends, and onlyAfter. Here's what they *sound* like they
mean, to me; although I don't think the code isn't entirely
consistent with this interpretation.
Wow, you are way ahead of me. I intended to write
Hi,
Kevin Grittner wrote:
Based on Andrew's suggestion, I changed line 276 to:
args=['psql', '-A', '--pset=pager=off',
That looks like a correct fix for psql, yes. Thanks for pointing that
out Andrew.
Other processes might be confused by (or at least act differently with)
a
Hi,
Markus Wanner wrote:
Sorry, if that didn't get clear. I'm trying to put together something I
can release real soon now (tm). I'll keep you informed.
Okay, here we go: dtester version 0.0.
This emerged out of Postgres-R, where I don't just need to test multiple
client connections, but
Markus Wanner mar...@bluegap.ch wrote:
Okay, here we go: dtester version 0.0.
C'mon, you could have tried to inspire a *bit* more confidence by
calling it version 0.1 or something! ;-)
It's certainly missing lots of things, mainly documentation.
However, I've attached a patch which
Hi,
Kevin Grittner wrote:
C'mon, you could have tried to inspire a *bit* more confidence by
calling it version 0.1 or something! ;-)
LOL
As David used to say: JFDI
I found that I just needed to ask for python-twisted.
Oh, sorry, yes, requirements: python, twisted.
I must admit that I
On Mon, Jan 11, 2010 at 04:17:42AM -, Greg Sabino Mullane wrote:
Because you'd have to build DBD::Pg against the new libpq, as you do
psql. That means you need DBD::Pg sources and the build environment for
Perl (headers etc) not just a working Perl runtime. Big difference.
Yes,
-BEGIN PGP SIGNED MESSAGE-
Hash: RIPEMD160
Is there a reason why you're suggesting using DBI? There is also the Pg
perl module which works as well and is one tenth of the size. It also
doesn't have external dependancies. It's just a plain wrapper around
libpq, which for the purposes
On Mon, Jan 11, 2010 at 05:42:08PM -, Greg Sabino Mullane wrote:
Is there a reason why you're suggesting using DBI? There is also the Pg
perl module which works as well and is one tenth of the size. It also
doesn't have external dependancies. It's just a plain wrapper around
libpq,
-BEGIN PGP SIGNED MESSAGE-
Hash: RIPEMD160
Using DBI/DBD::Pg would raise another issue - what version of libpq
would it be using? Not the one in the build being tested, that's for
On 8/01/2010 1:39 AM, Greg Sabino Mullane wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: RIPEMD160
Using DBI/DBD::Pg would raise another issue - what version of libpq
would it be using? Not the one in the build being tested, that's for
sure.
Er...why not? That's what psql uses.
Because
On 7/01/2010 9:15 AM, Robert Haas wrote:
On Wed, Jan 6, 2010 at 4:52 PM, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
David E. Wheelerda...@kineticode.com wrote:
Last I heard, Andrew was willing to require Test::More for
testing, so that a Perl script could handle multiple psql
-BEGIN PGP SIGNED MESSAGE-
Hash: RIPEMD160
Doing this without DBI is going to be ten times harder than doing it
with DBI. Are we really sure that's not a viable option?
In the buildfarm? Yes, I think so. The philosophy of the buildfarm is
that it should do what you would do
On 2010-01-07 11:50 +0200, Craig Ringer wrote:
On 7/01/2010 9:15 AM, Robert Haas wrote:
Doing this without DBI is going to be ten times harder than doing it
with DBI. Are we really sure that's not a viable option?
At this point, I'm personally wondering if it's worth putting together a
On 2010-01-07 18:13 +0200, Marko Tiikkaja wrote:
I had a similar syntax in mind, but instead of using threads, just
execute the file in order using asynchronous connections.
I completely failed to make the point here which was to somehow mark
which statements will (or, should) block. So here
On Jan 6, 2010, at 6:26 PM, Tom Lane wrote:
We have not yet fully accepted the notion that you must have Perl to
build (and, in fact, I am right now trying to verify that you don't).
I don't think that requiring Perl to test is going to fly.
I believe that the build farm already requires
Greg Sabino Mullane g...@turnstep.com writes:
We could even bundle DBI and DBD::Pg to ensure that the minimum versions
are there.
As a packager, my reaction to that is over my dead body. We have
enough trouble keeping our own software up to date, and pretty much
every external component that
David E. Wheeler wrote:
On Jan 6, 2010, at 6:26 PM, Tom Lane wrote:
We have not yet fully accepted the notion that you must have Perl to
build (and, in fact, I am right now trying to verify that you don't).
I don't think that requiring Perl to test is going to fly.
I believe that
On Thu, Jan 7, 2010 at 11:46 AM, Andrew Dunstan and...@dunslane.net wrote:
Using DBI/DBD::Pg would raise another issue - what version of libpq would it
be using? Not the one in the build being tested, that's for sure. If you
really want to use Perl then either a Pure Perl DBI driver (which Greg
On Wed, Jan 06, 2010 at 08:40:28PM -0500, Andrew Dunstan wrote:
A parallel psql seems to me a better way to go. We talked about that
a while ago, but I don't recall what happened to it.
Greg Stark had a patch a couple of years ago. Dunno what happened to
it since then.
Cheers,
David.
--
On Jan 6, 2010, at 6:31 PM, Kevin Grittner wrote:
As far as I've been able to determine so far, to call psql in a
relatively portable way would require something like this:
http://perldoc.perl.org/perlfork.html
Here's an example using IPC::Open3:
#!/usr/local/bin/perl -w
use
Andrew Dunstan and...@dunslane.net writes:
Unless I am mistaken, Perl is required in any case to build from CVS,
although not from a tarball.
Right, but to my mind building from a tarball needs to include the
ability to run the regression tests on what you built. So injecting
Perl into that
On Jan 7, 2010, at 9:08 AM, Tom Lane wrote:
Right, but to my mind building from a tarball needs to include the
ability to run the regression tests on what you built. So injecting
Perl into that is moving the goalposts on build requirements.
In that case, there's nothing for it except
David E. Wheeler da...@kineticode.com writes:
On Jan 7, 2010, at 9:08 AM, Tom Lane wrote:
Right, but to my mind building from a tarball needs to include the
ability to run the regression tests on what you built. So injecting
Perl into that is moving the goalposts on build requirements.
In
On Jan 7, 2010, at 9:19 AM, Tom Lane wrote:
In that case, there's nothing for it except concurrent psql.
Unless we are prepared to define concurrency testing as something
separate from the basic regression tests. Which is kind of annoying but
perhaps less so than the alternatives. It
-BEGIN PGP SIGNED MESSAGE-
Hash: RIPEMD160
Unless we are prepared to define concurrency testing as something
separate from the basic regression tests. Which is kind of annoying but
perhaps less so than the alternatives. It certainly seems to me to
be the kind of thing you wouldn't
-BEGIN PGP SIGNED MESSAGE-
Hash: RIPEMD160
Using DBI/DBD::Pg would raise another issue - what version of libpq
would it be using? Not the one in the build being tested, that's for
sure.
Er...why not? That's what psql uses. As for those advocating using a
custom C program written
-BEGIN PGP SIGNED MESSAGE-
Hash: RIPEMD160
We could even bundle DBI and DBD::Pg to ensure that the minimum versions
are there.
As a packager, my reaction to that is over my dead body. We have
enough trouble keeping our own software up to date, and pretty much
every external
Alvaro Herrera alvhe...@commandprompt.com wrote:
Kevin Grittner escribió:
Are we anywhere close to an agreement on what the multi-session
psql implementation would look like?
See
http://archives.postgresql.org/message-id/8204.1207689...@sss.pgh.pa.us
and followups.
Thanks, I had missed
On Jan 7, 2010, at 12:39 PM, Greg Sabino Mullane wrote:
I'm still *very* interested in making a libpq-less pure perl driver,
if anyone feels like funding it, let me know! :)
You mean this one:
http://search.cpan.org/~arc/DBD-PgPP-0.07/lib/DBD/PgPP.pm
?
Cheers,
M
--
Sent via pgsql-hackers
A.M. wrote:
On Jan 7, 2010, at 12:39 PM, Greg Sabino Mullane wrote:
I'm still *very* interested in making a libpq-less pure perl driver,
if anyone feels like funding it, let me know! :)
You mean this one:
http://search.cpan.org/~arc/DBD-PgPP-0.07/lib/DBD/PgPP.pm
?
It has a
David E. Wheeler da...@kineticode.com wrote:
Last I heard, Andrew was willing to require Test::More for
testing, so that a Perl script could handle multiple psql
connections (perhaps forked) and output test results based on
them. But he wasn't as interested in requiring DBI and DBD::Pg,
On ons, 2010-01-06 at 15:52 -0600, Kevin Grittner wrote:
David E. Wheeler da...@kineticode.com wrote:
Last I heard, Andrew was willing to require Test::More for
testing, so that a Perl script could handle multiple psql
connections (perhaps forked) and output test results based on
them.
On Jan 6, 2010, at 1:52 PM, Kevin Grittner wrote:
Last I heard, Andrew was willing to require Test::More for
testing, so that a Perl script could handle multiple psql
connections (perhaps forked) and output test results based on
them. But he wasn't as interested in requiring DBI and DBD::Pg,
On Jan 6, 2010, at 2:08 PM, Peter Eisentraut wrote:
Then I don't see much of a point in using Perl. You might as well fire
up a few psqls from a shell script
If you're more comfortable with shell, then yes. Although then it won't run on
Windows, will it?
Best,
David
--
Sent via
On 2010-01-07 00:08 +0200, Peter Eisentraut wrote:
On ons, 2010-01-06 at 15:52 -0600, Kevin Grittner wrote:
David E. Wheelerda...@kineticode.com wrote:
Last I heard, Andrew was willing to require Test::More for
testing, so that a Perl script could handle multiple psql
connections (perhaps
Marko Tiikkaja marko.tiikk...@cs.helsinki.fi wrote:
On 2010-01-07 00:08 +0200, Peter Eisentraut wrote:
Then I don't see much of a point in using Perl. You might as
well fire up a few psqls from a shell script.
I don't see how that would work, but I might have misunderstood
what we're
Marko Tiikkaja wrote:
On 2010-01-07 00:08 +0200, Peter Eisentraut wrote:
Then I don't see much of a point in using Perl. You might as well fire
up a few psqls from a shell script.
I don't see how that would work, but I might have misunderstood what
we're reaching for here. What I think
Alvaro Herrera alvhe...@commandprompt.com wrote:
Open a few psql with -f pointing to a pipe, and from the shell
write into the pipe? I don't think it's straightforward, but it
should be possible.
I'll play with it and see what I can do.
Thanks,
-Kevin
--
Sent via pgsql-hackers
On Wed, Jan 6, 2010 at 4:52 PM, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
David E. Wheeler da...@kineticode.com wrote:
Last I heard, Andrew was willing to require Test::More for
testing, so that a Perl script could handle multiple psql
connections (perhaps forked) and output test
Robert Haas wrote:
On Wed, Jan 6, 2010 at 4:52 PM, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
David E. Wheeler da...@kineticode.com wrote:
Last I heard, Andrew was willing to require Test::More for
testing, so that a Perl script could handle multiple psql
connections (perhaps
On Wed, Jan 6, 2010 at 8:40 PM, Andrew Dunstan and...@dunslane.net wrote:
Doing this without DBI is going to be ten times harder than doing it
with DBI. Are we really sure that's not a viable option?
In the buildfarm? Yes, I think so. The philosophy of the buildfarm is that
it should do what
Robert Haas robertmh...@gmail.com writes:
It just seems crazy to me to try to test anything without proper
language bindings. Opening a psql session and parsing the results
seems extraordinarily painful. I wonder if it would make sense write
a small wrapper program that uses libpq and dumps
Tom Lane wrote:
We have not yet fully accepted the notion that you must have Perl
to build (and, in fact, I am right now trying to verify that you
don't). I don't think that requiring Perl to test is going to fly.
Well, if that's the consensus, I have to choose between trying to
implement
On Wed, Jan 6, 2010 at 9:26 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
It just seems crazy to me to try to test anything without proper
language bindings. Opening a psql session and parsing the results
seems extraordinarily painful. I wonder if it would
Robert Haas wrote:
I'm not exactly sure what Kevin's goal is here.
I think it would be insane to try to do something like serializable
isolation mode without having regression tests. I would want more
tests than could reasonably go into the 'make check' suite to support
development, but it
Kevin Grittner escribió:
Well, if that's the consensus, I have to choose between trying to
implement multi-session psql and using testing which can't carry over
to long-term regular use. Are we anywhere close to an agreement on
what the multi-session psql implementation would look like? (If
On Wed, Jan 6, 2010 at 10:00 PM, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
For basic regression tests, yeah, we'd probably like to keep that
Perl-free.
OK. Is parallel psql the only reasonable option?
It seems so, assuming you're willing to concede that it is a
reasonable option in
Robert Haas wrote:
It just seems crazy to me to try to test anything without proper
language bindings. Opening a psql session and parsing the results
seems extraordinarily painful.
I've written a Python based program that spawns a captive psql and talks
to it--twice for different
On ons, 2010-01-06 at 14:10 -0800, David E. Wheeler wrote:
On Jan 6, 2010, at 2:08 PM, Peter Eisentraut wrote:
Then I don't see much of a point in using Perl. You might as well fire
up a few psqls from a shell script
If you're more comfortable with shell, then yes. Although then it
On ons, 2010-01-06 at 20:49 -0500, Robert Haas wrote:
Another idea would be to make a set of Perl libpq bindings that is
simpler than DBD::Pg and don't go through DBI. If we put those in the
main source tree (perhaps as a contrib module) they would be available
wherever we need them.
Hi,
Kevin Grittner wrote:
It's very soon going to be critical that I be able to test particular
interleavings of statements in particular concurrent transaction sets
to be able to make meaningful progress on the serializable
transaction work.
I've something in place for Postgres-R, as I also
Markus Wanner mar...@bluegap.ch wrote:
Kevin Grittner wrote:
It's very soon going to be critical that I be able to test
particular interleavings of statements in particular concurrent
transaction sets to be able to make meaningful progress on the
serializable transaction work.
I've
Hi,
Kevin Grittner wrote:
Where would I find this (and any related documentation)?
Sorry, if that didn't get clear. I'm trying to put together something I
can release real soon now (tm). I'll keep you informed.
Regards
Markus Wanner
--
Sent via pgsql-hackers mailing list
David E. Wheeler da...@kineticode.com wrote:
I think the consensus was, failing support for concurrent sessions
in psql, to use a Perl script to control multiple psql sessions
and perhaps use Test::More to do the testing.
Are there any examples of that? While I can hack my way through
On mån, 2010-01-04 at 17:10 -0600, Kevin Grittner wrote:
David E. Wheeler da...@kineticode.com wrote:
I think the consensus was, failing support for concurrent sessions
in psql, to use a Perl script to control multiple psql sessions
and perhaps use Test::More to do the testing.
Are
On Jan 4, 2010, at 3:29 PM, Peter Eisentraut wrote:
If you're not deep into Perl, perhaps ignore the Test::More comment for
the moment and just use DBI to connect to several database sessions,
execute your queries and check if the results are what you want. Once
you have gotten somewhere
On Jan 1, 2010, at 6:01 PM, Kevin Grittner wrote:
It's very soon going to be critical that I be able to test particular
interleavings of statements in particular concurrent transaction sets
to be able to make meaningful progress on the serializable
transaction work. It would be wonderful if
There has been periodic discussion here about allowing psql to deal
with multiple sessions, or possibly creating another tool to allow
this sort of test. Is anyone working on this?
It's very soon going to be critical that I be able to test particular
interleavings of statements in particular
84 matches
Mail list logo
