Re: [HACKERS] binary representation of datatypes

2008-10-22 Thread Jeroen Vermeulen
Matthieu Imbert wrote: scenario 1 - parse the textual representation of all results of requests to the database and convert textual timestamps to a binary format that i choose among those ones (number of microseconds since 2000-01-01, or a structure similar to pg_tm (but with microsecond

Re: [HACKERS] binary representation of datatypes

2008-10-22 Thread Matthieu Imbert
Jeroen Vermeulen wrote: Matthieu Imbert wrote: scenario 1 - parse the textual representation of all results of requests to the database and convert textual timestamps to a binary format that i choose among those ones (number of microseconds since 2000-01-01, or a structure similar to pg_tm

Re: [HACKERS] Bitmap Indexes: request for feedback

2008-10-22 Thread Simon Riggs
On Wed, 2008-10-22 at 00:00 +0100, Gregory Stark wrote: Actually as I recall the immediate issue was that the patch was more complex than necessary. In particular it reimplemented parts of the executor internally rather than figuring out what api was necessary to integrate it fully into the

Re: [HACKERS] binary representation of datatypes

2008-10-22 Thread Heikki Linnakangas
Matthieu Imbert wrote: You mean that when results are asked in textual representation (the default), data is sent on network directly as text? Yes. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] Bitmap Indexes: request for feedback

2008-10-22 Thread Simon Riggs
On Tue, 2008-10-21 at 14:26 -0700, Josh Berkus wrote: Gianni, me and Gabriele Bartolini have been working on Bitmap Indexes (BMI) in the last weeks, with advice and guidance from Simon Riggs. We feel that we are about to approach the point where it is appropriate to ask for feedback

Re: [HACKERS] So what's an empty array anyway?

2008-10-22 Thread Simon Riggs
On Tue, 2008-10-21 at 14:03 -0700, David E. Wheeler wrote: On Oct 21, 2008, at 13:58, Tom Lane wrote: If we got rid of zero dimension arrays, how would I declare a new empty array in a PL/pgSQL function? Same as before, I think: initialize it to '{}'. What's at stake here is exactly

Re: [HACKERS] Multi CPU Queries - Feedback and/or suggestions wanted!

2008-10-22 Thread Simon Riggs
On Mon, 2008-10-20 at 21:05 +0200, Julius Stroffek wrote: He is going to do some investigation in the methods and write down the possibilities and then he is going to implement something from that for PostgreSQL. When will this work be complete? We are days away from completing main work on

Re: [HACKERS] Withdraw PL/Proxy from commitfest

2008-10-22 Thread Martin Pihlak
Hannu Krosing wrote: It is cleverly hidden. The CREATE SERVER and CREATE USER MAPPING take generic options (list of key/value pairs). These can be used for defining the actual connection to the remote server. Are you sure this is how it is intended to be done ? Yes. In pl/proxy context

Re: [HACKERS] Deriving Recovery Snapshots

2008-10-22 Thread Heikki Linnakangas
Simon Riggs wrote: On Thu, 2008-10-16 at 18:52 +0300, Heikki Linnakangas wrote: Simon Riggs wrote: * The backend slot may not be reused for some time, so we should take additional actions to keep state current and true. So we choose to log a snapshot from the master into WAL after each

Re: [HACKERS] Bitmap Indexes: request for feedback

2008-10-22 Thread Mark Kirkwood
Josh Berkus wrote: The other major issue with the Bitmap index patch as it stood in 2007 was that performance just wasn't that much faster than a btree, except for specific corner cases. Otherwise, someone else would have been interested enough to pick it up and finish it. So performance

Re: [HACKERS] Withdraw PL/Proxy from commitfest

2008-10-22 Thread David Fetter
On Wed, Oct 22, 2008 at 11:02:34AM +0300, Martin Pihlak wrote: Hannu Krosing wrote: It is cleverly hidden. The CREATE SERVER and CREATE USER MAPPING take generic options (list of key/value pairs). These can be used for defining the actual connection to the remote server. Are you sure

Re: [HACKERS] Deriving Recovery Snapshots

2008-10-22 Thread Hannu Krosing
On Thu, 2008-10-16 at 18:52 +0300, Heikki Linnakangas wrote: Also, I can't help thinking that this would be a lot simpler if we just treated all subtransactions the same as top-level transactions. The only problem with that is that there can be a lot of subtransactions, which means that

Re: [HACKERS] Withdraw PL/Proxy from commitfest

2008-10-22 Thread Martin Pihlak
David Fetter wrote: DBI's DBD::Excel http://search.cpan.org/~kwitknr/DBD-Excel-0.06/Excel.pm is a more extreme example of this kind of thing. You can get connect strings that look like this: my $hDb = DBI-connect( DBI:Excel:file=dbdtest.xls, undef, undef, {

Re: [HACKERS] Deriving Recovery Snapshots

2008-10-22 Thread Simon Riggs
On Wed, 2008-10-22 at 12:29 +0300, Heikki Linnakangas wrote: Simon Riggs wrote: On Thu, 2008-10-16 at 18:52 +0300, Heikki Linnakangas wrote: Simon Riggs wrote: * The backend slot may not be reused for some time, so we should take additional actions to keep state current and true. So we

Re: [HACKERS] binary representation of datatypes

2008-10-22 Thread Merlin Moncure
On Wed, Oct 22, 2008 at 3:29 AM, Matthieu Imbert [EMAIL PROTECTED] wrote: Jeroen Vermeulen wrote: Matthieu Imbert wrote: scenario 1 - parse the textual representation of all results of requests to the database and convert textual timestamps to a binary format that i choose among those ones

Re: [HACKERS] Deriving Recovery Snapshots

2008-10-22 Thread Simon Riggs
On Wed, 2008-10-22 at 12:29 +0300, Heikki Linnakangas wrote: How about: 1. Keep all transactions and subtransactions in UnobservedXids. 2. If it fills up, remove all subtransactions from it, that the startup process knows to be subtransactions and knows the parents, and update subtrans.

Re: [HACKERS] crypt auth

2008-10-22 Thread Magnus Hagander
Peter Eisentraut wrote: Tom Lane wrote: Peter Eisentraut [EMAIL PROTECTED] writes: AFAICT, removing an authentication method requires a protocol version bump. Why would it require that? There would just be some auth method codes that remain reserved but aren't used anymore. Yeah, I was

Re: [HACKERS] binary representation of datatypes

2008-10-22 Thread Andrew Chernow
You mean that when results are asked in textual representation (the default), data is sent on network directly as text? yes. You should know that text/binary conversions rarely play a significant role in terms of performance. There are exceptions...large bytea columns, or enormous sets of

Re: [HACKERS] binary representation of datatypes

2008-10-22 Thread Merlin Moncure
On Wed, Oct 22, 2008 at 8:07 AM, Andrew Chernow [EMAIL PROTECTED] wrote: You mean that when results are asked in textual representation (the default), data is sent on network directly as text? You should know that text/binary conversions rarely play a significant role in terms of performance.

Re: [HACKERS] Bitmap Indexes: request for feedback

2008-10-22 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: On Wed, 2008-10-22 at 00:00 +0100, Gregory Stark wrote: When we last left our heros they were proposing ways to refactor the index api to allow index ams to stream results to the executor in bitmap form. The indexAM API has now been changed, so that is

Re: [HACKERS] Bitmap Indexes: request for feedback

2008-10-22 Thread Simon Riggs
On Wed, 2008-10-22 at 08:31 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: On Wed, 2008-10-22 at 00:00 +0100, Gregory Stark wrote: When we last left our heros they were proposing ways to refactor the index api to allow index ams to stream results to the executor in bitmap

Re: [HACKERS] Bitmap Indexes: request for feedback

2008-10-22 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: On Wed, 2008-10-22 at 08:31 -0400, Tom Lane wrote: No, that was merely one component of the problem. The APIs for tidbitmaps need revision too. You can't stream a bitmap yet. Please explain further. Which calls? Why do we need to stream them? Well, I

[HACKERS] GiST bug in 8.3.4, 8.2.10 and 8.1.18

2008-10-22 Thread Teodor Sigaev
Separate thread to draw attention. Unfortunately, I introduced new bug in GiST while fixing possible duplication of tuples during GiST's scan. Bug is a marking as dead wrong tuples in index and, as result, index could return less number results than it should be. Fixes are already committed

Re: [HACKERS] Bitmap Indexes: request for feedback

2008-10-22 Thread Simon Riggs
On Wed, 2008-10-22 at 09:13 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: On Wed, 2008-10-22 at 08:31 -0400, Tom Lane wrote: No, that was merely one component of the problem. The APIs for tidbitmaps need revision too. You can't stream a bitmap yet. Please explain

Re: [HACKERS] So what's an empty array anyway?

2008-10-22 Thread David E. Wheeler
On Oct 22, 2008, at 00:40, Simon Riggs wrote: An empty, single-dimension array. But I got the impression from Simon that he thought it should be NULL. I meant the dimension of {} should be NULL. To me that's 0. Best, David -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Bitmap Indexes: request for feedback

2008-10-22 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: I agree we could do the stream bitmap as well, but I'd suggest we should save that until the rest of the patch has been checked out. Well, that would be a reasonable implementation path, but it's got nothing to do with the patch as presented. The concern

Re: [HACKERS] Bitmap Indexes: request for feedback

2008-10-22 Thread Simon Riggs
On Wed, 2008-10-22 at 10:00 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: I agree we could do the stream bitmap as well, but I'd suggest we should save that until the rest of the patch has been checked out. Well, that would be a reasonable implementation path, but it's got

[HACKERS] SQL:2008 CURRENT_CATALOG and CURRENT_SCHEMA

2008-10-22 Thread Peter Eisentraut
Here is another patch for relatively redundant SQL:2008 functionality. They standardized in CURRENT_CATALOG and CURRENT_SCHEMA (without parentheses, as usual) for what we have as current_database() and current_schema(). They also added SET CATALOG and SET SCHEMA. SET CATALOG won't work in

Re: [HACKERS] Deriving Recovery Snapshots

2008-10-22 Thread Heikki Linnakangas
Simon Riggs wrote: On Wed, 2008-10-22 at 12:29 +0300, Heikki Linnakangas wrote: How about: 1. Keep all transactions and subtransactions in UnobservedXids. 2. If it fills up, remove all subtransactions from it, that the startup process knows to be subtransactions and knows the parents, and

Re: [HACKERS] Deriving Recovery Snapshots

2008-10-22 Thread Heikki Linnakangas
Simon Riggs wrote: On Wed, 2008-10-22 at 12:29 +0300, Heikki Linnakangas wrote: Simon Riggs wrote: On Thu, 2008-10-16 at 18:52 +0300, Heikki Linnakangas wrote: Simon Riggs wrote: * The backend slot may not be reused for some time, so we should take additional actions to keep state current

Re: [HACKERS] Deriving Recovery Snapshots

2008-10-22 Thread Simon Riggs
On Wed, 2008-10-22 at 17:18 +0300, Heikki Linnakangas wrote: It doesn't need to be in subtrans before it appears on disk, AFAICS. I don't make these rules. Read comments in AssignTransactionId(). -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support --

Re: [HACKERS] Deriving Recovery Snapshots

2008-10-22 Thread Heikki Linnakangas
Hannu Krosing wrote: On Thu, 2008-10-16 at 18:52 +0300, Heikki Linnakangas wrote: Also, I can't help thinking that this would be a lot simpler if we just treated all subtransactions the same as top-level transactions. The only problem with that is that there can be a lot of subtransactions,

Re: [HACKERS] Deriving Recovery Snapshots

2008-10-22 Thread Simon Riggs
On Wed, 2008-10-22 at 17:18 +0300, Heikki Linnakangas wrote: The list of xids in a snapshot is just a palloc'd array, in backend-local memory, so we can easily make it as large as we need to. It's malloc'd before we hold the lock. Yes, we can make it as large as we need to, but that would

[HACKERS] Fwd: [PATCHES] Auto Partitioning Patch - WIP version 1

2008-10-22 Thread Jaime Casanova
just remembering that -patches is a dead list, so i'm sending this to -hackers where it will have more visibility... -- Forwarded message -- From: Jaime Casanova [EMAIL PROTECTED] Date: Oct 22, 2008 9:43 AM Subject: Re: [PATCHES] Auto Partitioning Patch - WIP version 1 To: Nikhil

Re: [HACKERS] Deriving Recovery Snapshots

2008-10-22 Thread Simon Riggs
On Wed, 2008-10-22 at 17:29 +0300, Heikki Linnakangas wrote: Simon Riggs wrote: You may be missing my point. We need an initial state to work from. I am proposing we write a full snapshot after each checkpoint because it allows us to start recovery again from that point. If we

Re: [HACKERS] Deriving Recovery Snapshots

2008-10-22 Thread Simon Riggs
On Wed, 2008-10-22 at 17:40 +0300, Heikki Linnakangas wrote: Simon Riggs wrote: If that makes snapshot overflow, we have a big problem: we would be forced to say sorry snapshot cannot be issued at this time, please wait. Ugh! BTW, we'll need to do that anyway, if we guess the

Re: [HACKERS] Deriving Recovery Snapshots

2008-10-22 Thread Heikki Linnakangas
Simon Riggs wrote: If that makes snapshot overflow, we have a big problem: we would be forced to say sorry snapshot cannot be issued at this time, please wait. Ugh! BTW, we'll need to do that anyway, if we guess the max_connections setting in the master incorrectly. We should avoid having to

Re: [HACKERS] SE-PostgreSQL wiki updates, but ...

2008-10-22 Thread Simon Riggs
On Tue, 2008-10-21 at 18:48 +0900, KaiGai Kohei wrote: I started to rework the SE-PostgreSQL documentation to catch up the latest implementation, because the existing PDF documents are a bit legacy to be updated. In addition, I moved them to wiki site for easier future updates.

Re: [HACKERS] SE-PostgreSQL wiki updates, but ...

2008-10-22 Thread KaiGai Kohei
Simon Riggs wrote: On Tue, 2008-10-21 at 18:48 +0900, KaiGai Kohei wrote: I started to rework the SE-PostgreSQL documentation to catch up the latest implementation, because the existing PDF documents are a bit legacy to be updated. In addition, I moved them to wiki site for easier future

Re: [HACKERS] So what's an empty array anyway?

2008-10-22 Thread Josh Berkus
Simon, I meant the dimension of {} should be NULL. Oh, so array_dims(1, '{}') IS NULL? I'm not entirely satisfied with that, but other solutions are just as bad. --Josh -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

[HACKERS] Reminder: only 10 days until final CommitFest

2008-10-22 Thread Josh Berkus
Hackers, As a reminder, the November CommitFest is coming up in 10 days. That means you have just over a week to get your patch into shape where it's ready to be submitted and merged. As a rule, patches which still require discussion and/or refactoring on November 20th are not going into

Re: [HACKERS] [COMMITTERS] pgsql: Rework subtransaction commit protocol for hot standby.

2008-10-22 Thread Tom Lane
[EMAIL PROTECTED] (Alvaro Herrera) writes: Rework subtransaction commit protocol for hot standby. I think this patch broke something. In CVS HEAD, replay of a transaction commit record with a subtransaction dies with an assert failure: #0 0xc0141220 in ?? () from /usr/lib/libc.1 #1

[HACKERS] Re: [COMMITTERS] pgsql: Rework subtransaction commit protocol for hot standby.

2008-10-22 Thread Simon Riggs
On Wed, 2008-10-22 at 13:25 -0400, Tom Lane wrote: [EMAIL PROTECTED] (Alvaro Herrera) writes: Rework subtransaction commit protocol for hot standby. I think this patch broke something. In CVS HEAD, replay of a transaction commit record with a subtransaction dies with an assert failure:

Re: [HACKERS] Reminder: only 10 days until final CommitFest

2008-10-22 Thread Tom Lane
Josh Berkus [EMAIL PROTECTED] writes: As a reminder, the November CommitFest is coming up in 10 days. That means you have just over a week to get your patch into shape where it's ready to be submitted and merged. As a rule, patches which still require discussion and/or refactoring on

Re: [HACKERS] Deriving Recovery Snapshots

2008-10-22 Thread Heikki Linnakangas
Simon Riggs wrote: On Wed, 2008-10-22 at 17:18 +0300, Heikki Linnakangas wrote: It doesn't need to be in subtrans before it appears on disk, AFAICS. I don't make these rules. Read comments in AssignTransactionId(). You mean this one: /* * Generate a new Xid and record it

Re: [HACKERS] PGDay.it collation discussion notes

2008-10-22 Thread Dave Gudeman
On Mon, Oct 20, 2008 at 2:28 AM, Heikki Linnakangas [EMAIL PROTECTED] wrote: Tom Lane wrote: Another objection to this design is that it's completely unclear that functions from text to text should necessarily yield the same collation that went into them, but if you treat collation as a

[HACKERS] Re: [COMMITTERS] pgsql: Rework subtransaction commit protocol for hot standby.

2008-10-22 Thread Simon Riggs
On Wed, 2008-10-22 at 13:25 -0400, Tom Lane wrote: conditionName=0x6deb8 !(((*byteptr bshift) ((1 2) - 1)) == 0 || ((*byteptr bshift) ((1 2) - 1)) == 0x03 || ((*byteptr bshift) ((1 2) - 1)) == status), errorType=0x6dde4 FailedAssertion, fileName=0x6ddf4 clog.c, lineNumber=330)

[HACKERS] Re: [COMMITTERS] pgsql: Rework subtransaction commit protocol for hot standby.

2008-10-22 Thread Simon Riggs
On Wed, 2008-10-22 at 13:25 -0400, Tom Lane wrote: #5 0x171680 in TransactionIdSetStatusBit (xid=2063810256, status=2063810248, lsn={xlogid = 0, xrecoff = 0}, slotno=0) at clog.c:330 #6 0x1714e8 in TransactionIdSetPageStatus (xid=84, nsubxids=1, subxids=0x4008dd78,

Re: [HACKERS] minimal update

2008-10-22 Thread Andrew Dunstan
Andrew Dunstan wrote: Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: In that case, why not put the trigger in core so people can use it easily? One advantage of making it a contrib module is that discussing how/when to use it would fit more easily into the structure of

Re: [HACKERS] minimal update

2008-10-22 Thread Kevin Grittner
Andrew Dunstan [EMAIL PROTECTED] wrote: Here is a patch that does it that way, along with docs s/mare/more/ -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Deriving Recovery Snapshots

2008-10-22 Thread Simon Riggs
On Wed, 2008-10-22 at 20:42 +0300, Heikki Linnakangas wrote: Simon Riggs wrote: On Wed, 2008-10-22 at 17:18 +0300, Heikki Linnakangas wrote: It doesn't need to be in subtrans before it appears on disk, AFAICS. I don't make these rules. Read comments in AssignTransactionId().

Re: [HACKERS] minimal update

2008-10-22 Thread Andrew Dunstan
Kevin Grittner wrote: Andrew Dunstan [EMAIL PROTECTED] wrote: Here is a patch that does it that way, along with docs s/mare/more/ Thanks. fixed in my tree.. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] minimal update

2008-10-22 Thread Simon Riggs
On Wed, 2008-10-22 at 14:43 -0400, Andrew Dunstan wrote: There seems to be a preponderance of opinion for doing this as a builtin. Here is a patch that does it that way, along with docs and regression test. In your example you use an underscore as the first character. The way you have

Re: [HACKERS] [COMMITTERS] pgsql: Rework subtransaction commit protocol for hot standby.

2008-10-22 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: These traces look weird. Look at the way the xid changes value as we move from call to call. It looks like something is screwy there. If those values are correct we should have failed an earlier assertion. No, that's normal behavior on this platform +

Re: [HACKERS] minimal update

2008-10-22 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: Can we call the function minimal_update_trigger, rather than min_... Minimal really fails to convey the point here IMHO. How about something like suppress_no_op_updates_trigger? regards, tom lane -- Sent via pgsql-hackers mailing

Re: [HACKERS] minimal update

2008-10-22 Thread Andrew Dunstan
Simon Riggs wrote: On Wed, 2008-10-22 at 14:43 -0400, Andrew Dunstan wrote: There seems to be a preponderance of opinion for doing this as a builtin. Here is a patch that does it that way, along with docs and regression test. In your example you use an underscore as the first

Re: [HACKERS] Deriving Recovery Snapshots

2008-10-22 Thread Heikki Linnakangas
Simon Riggs wrote: If you don't mark subtrans, you will need to keep things in shared memory. I agree you can defer marking subtrans until your shared memory fills, but you will still need to keep track of the parent for each subtransaction until you do. If you don't *know* the parent, how will

[HACKERS] Re: [COMMITTERS] pgsql: Rework subtransaction commit protocol for hot standby.

2008-10-22 Thread Simon Riggs
On Wed, 2008-10-22 at 15:18 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: These traces look weird. Look at the way the xid changes value as we move from call to call. It looks like something is screwy there. If those values are correct we should have failed an earlier

[HACKERS] Re: [COMMITTERS] pgsql: Rework subtransaction commit protocol for hot standby.

2008-10-22 Thread Simon Riggs
On Wed, 2008-10-22 at 20:52 +0100, Simon Riggs wrote: Took a while, but yes, I can reproduce this now. Analysing... OK, I think I see what it's doing and why it fails the assert. It's nothing to do with confusing commit/abort. The new way of doing things on commit is to subcommit then

Re: [HACKERS] [COMMITTERS] pgsql: Rework subtransaction commit protocol for hot standby.

2008-10-22 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: It's nothing to do with confusing commit/abort. The new way of doing things on commit is to subcommit then commit. This sequence is repeated during WAL replay. If we crash, it will try to repeat the sequence, so in some cases it will try to set status to

Re: [HACKERS] Deriving Recovery Snapshots

2008-10-22 Thread Simon Riggs
On Wed, 2008-10-22 at 22:38 +0300, Heikki Linnakangas wrote: Simon Riggs wrote: If you don't mark subtrans, you will need to keep things in shared memory. I agree you can defer marking subtrans until your shared memory fills, but you will still need to keep track of the parent for each

[HACKERS] Re: [COMMITTERS] pgsql: Rework subtransaction commit protocol for hot standby.

2008-10-22 Thread Simon Riggs
On Wed, 2008-10-22 at 16:41 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: It's nothing to do with confusing commit/abort. The new way of doing things on commit is to subcommit then commit. This sequence is repeated during WAL replay. If we crash, it will try to repeat

Re: [HACKERS] [COMMITTERS] pgsql: Rework subtransaction commit protocol for hot standby.

2008-10-22 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: On Wed, 2008-10-22 at 16:41 -0400, Tom Lane wrote: Hmm, but then why did we not see the same thing before? The failure definitely came from trying to set SUBCOMMITTED on a transaction already committed. Ah, I see: prior versions did not bother to make a

Re: [HACKERS] minimal update

2008-10-22 Thread Robert Haas
On Wed, Oct 22, 2008 at 3:24 PM, Tom Lane [EMAIL PROTECTED] wrote: Simon Riggs [EMAIL PROTECTED] writes: Can we call the function minimal_update_trigger, rather than min_... Minimal really fails to convey the point here IMHO. How about something like suppress_no_op_updates_trigger? +1.

[HACKERS] Re: [COMMITTERS] pgsql: Rework subtransaction commit protocol for hot standby.

2008-10-22 Thread Simon Riggs
On Wed, 2008-10-22 at 17:16 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: On Wed, 2008-10-22 at 16:41 -0400, Tom Lane wrote: Hmm, but then why did we not see the same thing before? The failure definitely came from trying to set SUBCOMMITTED on a transaction already

Re: [HACKERS] Fwd: [PATCHES] Auto Partitioning Patch - WIP version 1

2008-10-22 Thread Emmanuel Cecchet
Another advantage of triggers over rules is that it would work with COPY which is probably a desired feature. Emmanuel Jaime Casanova wrote: just remembering that -patches is a dead list, so i'm sending this to -hackers where it will have more visibility... -- Forwarded message

Re: [HACKERS] minimal update

2008-10-22 Thread Simon Riggs
On Wed, 2008-10-22 at 17:24 -0400, Robert Haas wrote: On Wed, Oct 22, 2008 at 3:24 PM, Tom Lane [EMAIL PROTECTED] wrote: Simon Riggs [EMAIL PROTECTED] writes: Can we call the function minimal_update_trigger, rather than min_... Minimal really fails to convey the point here IMHO. How

Re: [HACKERS] minimal update

2008-10-22 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: On Wed, Oct 22, 2008 at 3:24 PM, Tom Lane [EMAIL PROTECTED] wrote: Minimal really fails to convey the point here IMHO. How about something like suppress_no_op_updates_trigger? I think it means something to us, but no op is a very technical phrase that

Re: [HACKERS] minimal update

2008-10-22 Thread Kenneth Marshall
On Wed, Oct 22, 2008 at 06:05:26PM -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: On Wed, Oct 22, 2008 at 3:24 PM, Tom Lane [EMAIL PROTECTED] wrote: Minimal really fails to convey the point here IMHO. How about something like suppress_no_op_updates_trigger? I think it

Re: [HACKERS] minimal update

2008-10-22 Thread Kevin Grittner
Simon Riggs [EMAIL PROTECTED] wrote: On Wed, Oct 22, 2008 at 3:24 PM, Tom Lane [EMAIL PROTECTED] wrote: How about something like suppress_no_op_updates_trigger? I think it means something to us, but no op is a very technical phrase that probably doesn't travel very well. Not everybody

Re: [HACKERS] Block level concurrency during recovery

2008-10-22 Thread Simon Riggs
On Mon, 2008-10-20 at 13:54 +0100, Simon Riggs wrote: I'm looking at how to make queries safe during recovery, in the presence of concurrent changes to blocks. In particular, concurrent removal of rows that might be read by queries. My thinking is * we ignore LockRelationForExtension().

Re: [HACKERS] Reminder: only 10 days until final CommitFest

2008-10-22 Thread Josh Berkus
Tom, November 20th? We intentionally set up the 8.4 schedule to allow a long time for the final commitfest. I agree that there needs to be a cutoff, but that one seems mighty early. Mid-December is what I was expecting. Sssshhh. Things always run 3 weeks late in this community. I'm

[HACKERS] psql Feature request \set query

2008-10-22 Thread Josh Berkus
Hackers, Just had a feature request from Wheeler, and I don't see why it shouldn't go on our todo list. Currently, in psql you can do: \set myvar 2008-10-23 and you can do \set myvar `date` but you can't do \set myvar Select now() even though you could, bizarrely, do: \set myvar `psql

Re: [HACKERS] psql Feature request \set query

2008-10-22 Thread Alvaro Herrera
Josh Berkus wrote: Hackers, Just had a feature request from Wheeler, and I don't see why it shouldn't go on our todo list. I think you're asking for more scriptability in psql. Personally I think that would be a great idea, but we need a lot more than what's being proposed here. We'll

Re: [HACKERS] Re: [COMMITTERS] pgsql: Rework subtransaction commit protocol for hot standby.

2008-10-22 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: My interest was really in maintaining ultra-correctness, while removing the need to WAL log subcommits for Hot Standby. I think I achieved that, almost, but if you see further optimizations that is good too. I'm not focusing on performance here --- I'm

Re: [HACKERS] minimal update

2008-10-22 Thread Robert Haas
How about one of these?: suppress_same_value_updates_trigger suppress_no_change_updates_trigger suppress_no_effect_updates_trigger I like the first one. A trigger firing would be an effect, and possibly a change, but same value seems very clear. ...Robert -- Sent via pgsql-hackers

Re: [HACKERS] psql Feature request \set query

2008-10-22 Thread Tom Lane
Josh Berkus [EMAIL PROTECTED] writes: but you can't do \set myvar Select now() That's a seriously ridiculous syntax. If you want something like this, a variant of \g that squirts the results to a variable instead of a file would be more usable: no quoting problems and no restriction to a

Re: [HACKERS] psql Feature request \set query

2008-10-22 Thread Robert Haas
I think you're asking for more scriptability in psql. Personally I think that would be a great idea, but we need a lot more than what's being proposed here. We'll also need loops, conditionals, etc. We've had patches for those submitted over the years, but one at a time they are easily

Re: [HACKERS] psql Feature request \set query

2008-10-22 Thread Joshua Drake
On Wed, 22 Oct 2008 19:06:59 -0400 Robert Haas [EMAIL PROTECTED] wrote: I think you're asking for more scriptability in psql. Personally I think that would be a great idea, but we need a lot more than what's being proposed here. We'll also need loops, conditionals, etc. We've had

Re: [HACKERS] psql Feature request \set query

2008-10-22 Thread Josh Berkus
Tom, If you want something like this, a variant of \g that squirts the results to a variable instead of a file would be more usable: no quoting problems and no restriction to a single-line query. I don't care about syntax, and neither does David, I think. If we use \g or \o, that's fine

Re: [HACKERS] psql Feature request \set query

2008-10-22 Thread Robert Haas
True enough, but a car doesn't roll without at least four wheels. True, but I'm not sure why we'd need three other wheels to make this feature roll, or what those three wheels would be. Personally, I would never write a complicated script in psql rather than perl, but I can imagine using this

Re: [HACKERS] psql Feature request \set query

2008-10-22 Thread Joshua Drake
On Wed, 22 Oct 2008 20:43:44 -0400 Robert Haas [EMAIL PROTECTED] wrote: True enough, but a car doesn't roll without at least four wheels. True, but I'm not sure why we'd need three other wheels to make this feature roll, or what those three wheels would be. Personally, I would never write

Re: [HACKERS] psql Feature request \set query

2008-10-22 Thread Robert Haas
So now that we have both agreed with each other, what do we do? :P Well, the original suggestion was to add it to the TODO list. That seems reasonable to me. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] pg_stat_statements in core

2008-10-22 Thread ITAGAKI Takahiro
Tom Lane [EMAIL PROTECTED] wrote: It needs to be a contrib or pgfoundry package for awhile, to shake out feature issues in a context where users will understand the API is subject to change. Agreed. That is what I want to do at first. It seems to me that all you're really missing is a

Re: [HACKERS] double-buffering page writes

2008-10-22 Thread ITAGAKI Takahiro
Alvaro Herrera [EMAIL PROTECTED] wrote: I'm trying to see if it makes sense to do the double-buffering of page writes before going further ahead with CRC checking. I came up with the attached patch; it does the double-buffering inconditionally, because as it was said, it allows releasing

[HACKERS] Can anyone explain to me how the ps_OuterTupleSlot in PlanState is being used in implementing HashJoin?

2008-10-22 Thread Ran Tang
Hello there, Can anyone explain to me how the ps_OuterTupleSlot in JoinState is being used in implementing HashJoin? I looked at the source code. When it find a tuple in outer relation. It store the outer tuple to field : node-js.ps.ps_OuterTupleSlot But I can't find how this field is

[HACKERS] Re: [COMMITTERS] pgsql: Rework subtransaction commit protocol for hot standby.

2008-10-22 Thread Simon Riggs
On Wed, 2008-10-22 at 13:25 -0400, Tom Lane wrote: I can't reproduce this 100% reliably, but the case where I saw it came from mistakenly inserting Assert(!IsA(node, AppendRelInfo)); into flatten_join_alias_vars_mutator and then running the serial regression tests. This case *can*

Re: [HACKERS] Re: [COMMITTERS] pgsql: Rework subtransaction commit protocol for hot standby.

2008-10-22 Thread Simon Riggs
On Wed, 2008-10-22 at 18:53 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: My interest was really in maintaining ultra-correctness, while removing the need to WAL log subcommits for Hot Standby. I think I achieved that, almost, but if you see further optimizations that is

[HACKERS] Making pg_standby compression-friendly

2008-10-22 Thread Charles Duffy
Howdy, all. I'm interested in compressing archived WAL segments in an environment set up for PITR in the interests of reducing both network traffic and storage requirements. However, pg_standby presently checks file sizes, requiring that an archive segment be exactly the right size to be

Re: [HACKERS] psql Feature request \set query

2008-10-22 Thread Tom Lane
Joshua Drake [EMAIL PROTECTED] writes: Robert Haas [EMAIL PROTECTED] wrote: We can't do anything unless we do everything is a recipe for failure. True enough, but a car doesn't roll without at least four wheels. Perhaps more to the point: you don't have to implement every part of a set of

Re: [HACKERS] Deriving Recovery Snapshots

2008-10-22 Thread Simon Riggs
On Wed, 2008-10-22 at 21:47 +0100, Simon Riggs wrote: But once you reach 64 transactions, you'll need to write an extra WAL record for every subtransaction, which currently I've managed to avoid. Yes, I've managed to avoid it, but it will simplify the patch if you think its not worth

Re: [HACKERS] Deriving Recovery Snapshots

2008-10-22 Thread Simon Riggs
On Wed, 2008-10-22 at 21:47 +0100, Simon Riggs wrote: But once you reach 64 transactions, you'll need to write an extra WAL record for every subtransaction, which currently I've managed to avoid. Some further notes/tests on the optimisation you discovered. Because of the way I have changed

Re: [HACKERS] psql Feature request \set query

2008-10-22 Thread Gregory Stark
Alvaro Herrera [EMAIL PROTECTED] writes: Josh Berkus wrote: Hackers, Just had a feature request from Wheeler, and I don't see why it shouldn't go on our todo list. I think you're asking for more scriptability in psql. Personally I think that would be a great idea, but we need a lot

Re: [HACKERS] Can anyone explain to me how the ps_OuterTupleSlot in PlanState is being used in implementing HashJoin?

2008-10-22 Thread Tom Lane
Ran Tang [EMAIL PROTECTED] writes: Can anyone explain to me how the ps_OuterTupleSlot in JoinState is being used in implementing HashJoin? Hmm ... [ sounds of grepping ] actually it looks to me like PlanState.ps_OuterTupleSlot is entirely dead code now. The thing that actually does something

Re: [HACKERS] Deriving Recovery Snapshots

2008-10-22 Thread Heikki Linnakangas
Simon Riggs wrote: On Wed, 2008-10-22 at 21:47 +0100, Simon Riggs wrote: But once you reach 64 transactions, you'll need to write an extra WAL record for every subtransaction, which currently I've managed to avoid. Yes, I've managed to avoid it, but it will simplify the patch if you think