Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Simon Riggs
On Tue, 2010-05-25 at 23:59 -0400, Robert Haas wrote: Quorum commit is definitely an extra knob, IMHO. No, its about three less, as I have explained. Explain your position, don't just demand others listen. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Simon Riggs
On Wed, 2010-05-26 at 13:03 +0900, Fujii Masao wrote: On Wed, May 26, 2010 at 1:04 AM, Simon Riggs si...@2ndquadrant.com wrote: On Tue, 2010-05-25 at 12:40 +0900, Fujii Masao wrote: On Tue, May 25, 2010 at 10:29 AM, Josh Berkus j...@agliodbs.com wrote: I agree that #4 should be done last,

Re: [HACKERS] ExecutorCheckPerms() hook

2010-05-26 Thread KaiGai Kohei
The attached patch is a revised one for DML permission checks. List of updates: - Source code comments in the patched functions were revised. - ExecCheckRTPerms() and ExecCheckRTEPerms() were moved to aclchk.c, and renamed to chkpriv_relation_perms() and chkpriv_rte_perms(). - It took the 2nd

Re: [HACKERS] fillfactor gets set to zero for toast tables

2010-05-26 Thread Takahiro Itagaki
Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Tom Lane's message of vie may 14 15:03:57 -0400 2010: Maybe a better solution is to have some kind of notion of a default-only entry, which is sufficient to insert the default into the struct but isn't accepted as a

[HACKERS] Open items related to SR

2010-05-26 Thread Fujii Masao
Hi, Many open items related to SR are listed on the wiki again. http://wiki.postgresql.org/wiki/PostgreSQL_9.0_Open_Items I clarify the status of those items. Smart shutdown gets stuck - patch to fix from Fuji Masao Robert is reviewing and testing the patch I submitted. I believe that the

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Simon Riggs
On Wed, 2010-05-26 at 12:36 +0900, Fujii Masao wrote: On Wed, May 26, 2010 at 2:10 AM, Simon Riggs si...@2ndquadrant.com wrote: My suggestion is simply to have a single parameter (name unimportant) number_of_synch_servers_we_wait_for = N which is much easier to understand because it is

Re: [HACKERS] mapping object names to role IDs

2010-05-26 Thread Peter Eisentraut
On sön, 2010-05-23 at 00:50 -0400, Robert Haas wrote: Oid get_object-type_oid(List *qualname, bool missingok); -or- Oid get_object-type_oid(char *name, bool missingok); Thus get_database_oid and get_tablespace_oid would remain unchanged except for taking a second argument, get_roleid and

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Fujii Masao
On Wed, May 26, 2010 at 5:02 PM, Simon Riggs si...@2ndquadrant.com wrote: Everything I've said about per-standby settings applies here, which was based upon having just 2 settings: sync and async. If you have four settings instead, things get even more complex. If we were going to reduce

Re: [HACKERS] Snapshot Materialized Views - GSoC

2010-05-26 Thread Peter Eisentraut
On fre, 2010-05-21 at 16:31 +0200, Florian Pflug wrote: I guess the justification is that with the same argument you could argue that a view should have relkind 'r', since it's just an empty table with a rewrite rule attached. It used to be that way, but now a view doesn't have an empty table

Re: [HACKERS] Regression testing for psql

2010-05-26 Thread Peter Eisentraut
On tis, 2010-05-25 at 06:23 -0400, Stephen Frost wrote: * Robert Haas (robertmh...@gmail.com) wrote: Of course, if people want to suggest tests that just shouldn't be included, I can go through and strip things out. Well... I'm a little reluctant to believe that we should have 3.3M of

Re: [HACKERS] [PATCH] Add XMLEXISTS function from the SQL/XML standard

2010-05-26 Thread Mike Fowler
Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Tue, May 25, 2010 at 1:09 PM, Mike Fowler m...@mlfowler.com wrote: We're unlikely to accept this patch if it changes the minimum version of libxml2 required to compile PostgreSQL Why? 2.6.27 is almost 4 years old.

Re: [HACKERS] [PATCH] Add XMLEXISTS function from the SQL/XML standard (was: Add xpath_exists Function)

2010-05-26 Thread Peter Eisentraut
On tis, 2010-05-25 at 15:31 +0100, Mike Fowler wrote: I've been reading the SQL/XML standard and discovered that it defines a function named XMLEXISTS that does exactly what the todo item xpath_exists defines. My original patch named the function as per the todo but I think using the

Re: [HACKERS] [PATCH] Add XMLEXISTS function from the SQL/XML standard

2010-05-26 Thread Mike Fowler
Peter Eisentraut wrote: On tis, 2010-05-25 at 15:31 +0100, Mike Fowler wrote: I've been reading the SQL/XML standard and discovered that it defines a function named XMLEXISTS that does exactly what the todo item xpath_exists defines. My original patch named the function as per the todo but

Re: [HACKERS] ExecutorCheckPerms() hook

2010-05-26 Thread Stephen Frost
Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: I haven't dug in the SQL spec to see if that addresses the point, but it wouldn't bother me in the least to insist that both REFERENCES and SELECT privilege are required to create an FK. Ok. If we require REFERENCES and SELECT privs to create an FK

Re: [HACKERS] Exposing the Xact commit order to the user

2010-05-26 Thread Jan Wieck
On 5/25/2010 3:18 PM, Kevin Grittner wrote: Jan Wieck janwi...@yahoo.com wrote: Have you ever looked at one of those queries, that Londiste or Slony issue against the provider DB in order to get all the log data that has been committed between two snapshots? Is that really the best you can

Re: [HACKERS] Exposing the Xact commit order to the user

2010-05-26 Thread Jan Wieck
On 5/26/2010 7:03 AM, Jan Wieck wrote: To replicate from one consistent state to the next, the replication system now selects all log rows between two snapshots. Between here means it simulates MVCC visibility in the sense of that the writing transaction was in progress when the first snapshot

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Robert Haas
On Wed, May 26, 2010 at 2:31 AM, Simon Riggs si...@2ndquadrant.com wrote: On Tue, 2010-05-25 at 23:59 -0400, Robert Haas wrote: Quorum commit is definitely an extra knob, IMHO. No, its about three less, as I have explained. Explain your position, don't just demand others listen. OK. In

Re: [HACKERS] mapping object names to role IDs

2010-05-26 Thread Robert Haas
On Wed, May 26, 2010 at 5:27 AM, Peter Eisentraut pete...@gmx.net wrote: On sön, 2010-05-23 at 00:50 -0400, Robert Haas wrote: Oid get_object-type_oid(List *qualname, bool missingok); -or- Oid get_object-type_oid(char *name, bool missingok); Thus get_database_oid and get_tablespace_oid would

Re: [HACKERS] Stefan's bug (was: max_standby_delay considered harmful)

2010-05-26 Thread Robert Haas
On Mon, May 24, 2010 at 10:35 AM, Fujii Masao masao.fu...@gmail.com wrote: On Mon, May 24, 2010 at 10:26 PM, Robert Haas robertmh...@gmail.com wrote: This looks pretty reasonable to me, but I guess I feel like it would be better to drive the CancelBackup() decision off of whether we've ever

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Alastair Turner
A suggestion, based on what I believe would be ideal default settings for a fully developed SR capability. The thought being that as long as the default behaviour was stable additional knobs could be added across version boundaries without causing trouble. Per slave the master needs to know: -

Re: [HACKERS] Stefan's bug (was: max_standby_delay considered harmful)

2010-05-26 Thread Robert Haas
On Tue, May 25, 2010 at 6:19 AM, Fujii Masao masao.fu...@gmail.com wrote: On Tue, May 18, 2010 at 3:09 PM, Fujii Masao masao.fu...@gmail.com wrote: (2) pg_ctl -ms stop emits the following warning whenever there is the backup_label file in $PGDATA.       WARNING: online backup mode is active

[HACKERS] out-of-date comment in CreateRestartPoint()

2010-05-26 Thread Robert Haas
This comment obviously requires adjustment now that HS is committed. The obvious way to change it is to replace when we get hot standby capability with when running in Hot Standby mode, but I'm not clear whether that's all that's required. /* * If the last checkpoint record we've

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Simon Riggs
On Wed, 2010-05-26 at 18:52 +0900, Fujii Masao wrote: I guess that dropping the support of #3 doesn't reduce complexity since the code of #3 is almost the same as that of #2. Like walreceiver sends the ACK after receiving the WAL in #2 case, it has only to do the same thing after the WAL

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Simon Riggs
On Wed, 2010-05-26 at 18:52 +0900, Fujii Masao wrote: To summarise, I think we can get away with just 3 parameters: synchronous_replication = N # similar in name to synchronous_commit synch_rep_timeout = T synch_rep_timeout_action = commit | abort I agree to add the latter two

Re: [HACKERS] ExecutorCheckPerms() hook

2010-05-26 Thread Stephen Frost
KaiGai, * KaiGai Kohei (kai...@ak.jp.nec.com) wrote: Yes, it is entirely separate issue. I don't intend to argue whether we can assume the default PG permission allows owner to SELECT on the table, or not. This actually isn't a separate issue. It's the whole crux of it, as a matter of fact.

Re: [HACKERS] libpq, PQexecPrepared, data size sent to FE vs. FETCH_COUNT

2010-05-26 Thread Giles Lean
Abhijit Menon-Sen a...@toroid.org wrote: Unless you explicitly declare and fetch from an SQL-level cursor, your many GBs of data are going to be transmitted to libpq, which will eat lots of memory. (The wire protocol does have something like cursors, but libpq does not use them, it retrieves

Re: [HACKERS] mapping object names to role IDs

2010-05-26 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I still feel that we'd be better off putting all the functions that use the same design pattern in a single file, rather than spreading them out all over the backend. It's true that that one file will then depend on all the catalog stuff, but it

Re: [HACKERS] ExecutorCheckPerms() hook

2010-05-26 Thread Tom Lane
KaiGai Kohei kai...@ak.jp.nec.com writes: Hmm. If both REFERENCES and SELECT privilege are required to create a new FK constraint, why RI_Initial_Check() need to check SELECT permission prior to SPI_execute()? It eventually checks SELECT privilege during execution of the secondary query. It

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Simon Riggs
On Wed, 2010-05-26 at 07:10 -0400, Robert Haas wrote: OK. In words of one syllable, your way still has all the same knobs, plus some more. I explained how the per-standby settings would take many parameters, whereas per-transaction settings take far fewer. You sketched out a design which

Re: [HACKERS] out-of-date comment in CreateRestartPoint()

2010-05-26 Thread Heikki Linnakangas
On 26/05/10 16:16, Robert Haas wrote: This comment obviously requires adjustment now that HS is committed. The obvious way to change it is to replace when we get hot standby capability with when running in Hot Standby mode, but I'm not clear whether that's all that's required. I think that's

Re: [HACKERS] Exposing the Xact commit order to the user

2010-05-26 Thread Greg Stark
On Sun, May 23, 2010 at 9:21 PM, Jan Wieck janwi...@yahoo.com wrote: Each record of the Transaction Commit Info consists of     txid          xci_transaction_id     timestamptz   xci_begin_timestamp     timestamptz   xci_commit_timestamp     int64         xci_total_rowcount So I think

Re: [HACKERS] mapping object names to role IDs

2010-05-26 Thread Robert Haas
On Wed, May 26, 2010 at 9:45 AM, Tom Lane t...@sss.pgh.pa.us wrote: This is nonsense You can assert that, but I don't agree. We certainly have places (comment.c being the obvious example) where we need to look up a name and map it to an OID without doing anything else, and actually I believe

Re: [HACKERS] libpq should not be using SSL_CTX_set_client_cert_cb

2010-05-26 Thread Garick Hamlin
On Tue, May 25, 2010 at 10:29:07PM -0400, Tom Lane wrote: I've been experimenting with SSL setups involving chains of CA certificates, ie, where the server or client cert itself is signed by an intermediate CA rather than a trusted root CA. This appears to work well enough on the server side

Re: [HACKERS] libpq should not be using SSL_CTX_set_client_cert_cb

2010-05-26 Thread Tom Lane
Garick Hamlin gham...@isc.upenn.edu writes: I am guessing the problem is that validating the presented chain is hard? No, the problem is that the current libpq code fails to present the chain at all. It will only load and send the first cert in the postgresql.crt file. This works only when

Re: [HACKERS] mapping object names to role IDs

2010-05-26 Thread alvherre
Excerpts from Robert Haas's message of mié may 26 07:20:30 -0400 2010: I still feel that we'd be better off putting all the functions that use the same design pattern in a single file, rather than spreading them out all over the backend. It's true that that one file will then depend on all

Re: [HACKERS] mapping object names to role IDs

2010-05-26 Thread Robert Haas
On Wed, May 26, 2010 at 11:01 AM, alvherre alvhe...@commandprompt.com wrote: Excerpts from Robert Haas's message of mié may 26 07:20:30 -0400 2010: I still feel that we'd be better off putting all the functions that use the same design pattern in a single file, rather than spreading them out

Re: [HACKERS] Show schema name on REINDEX DATABASE

2010-05-26 Thread Selena Deckelmann
On Mon, Apr 5, 2010 at 9:29 AM, Greg Sabino Mullane g...@turnstep.com wrote: Patch attached to show the schema *and* table name when doing a REINDEX DATABASE. Is this something that can be added to 9.1 commitfest? -selena -- http://chesnok.com/daily - me -- Sent via pgsql-hackers mailing

Re: [HACKERS] ExecutorCheckPerms() hook

2010-05-26 Thread Stephen Frost
KaiGai, * KaiGai Kohei (kai...@ak.jp.nec.com) wrote: The attached patch is a revised one for DML permission checks. This is certainly alot better. ToDo: - makeRangeTblEntry() stuff to allocate a RTE node with given parameter is not yet. I'd certainly like to see the above done, or to

Re: [HACKERS] Regression testing for psql

2010-05-26 Thread Stephen Frost
* Peter Eisentraut (pete...@gmx.net) wrote: Maybe pg_regress is not the right framework to test that sort of thing. Perhaps, but if not, then what? And how can we avoid writing a bunch of new code that would then need to be checked itself..? Thanks, Stephen

Re: [HACKERS] ExecutorCheckPerms() hook

2010-05-26 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Because the queries inside the triggers are done with a different current userid. Indeed, I figured that out eventually too. Sorry it took so long. :/ Thanks, Stephen signature.asc Description: Digital signature

Re: [HACKERS] Exposing the Xact commit order to the user

2010-05-26 Thread Jan Wieck
On 5/26/2010 10:04 AM, Greg Stark wrote: On Sun, May 23, 2010 at 9:21 PM, Jan Wieck janwi...@yahoo.com wrote: Each record of the Transaction Commit Info consists of txid xci_transaction_id timestamptz xci_begin_timestamp timestamptz xci_commit_timestamp int64

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Robert Haas
On Wed, May 26, 2010 at 9:54 AM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, 2010-05-26 at 07:10 -0400, Robert Haas wrote: OK.  In words of one syllable, your way still has all the same knobs, plus some more. I explained how the per-standby settings would take many parameters, whereas

Re: [HACKERS] Exposing the Xact commit order to the user

2010-05-26 Thread Steve Singer
Jan Wieck wrote: Since the actual row level change information and other event data is found inside of regular tables, identified by TXID and sequence number, I am pretty sure I want that data in a server-side query. What you are proposing is to read the xid's and timestamps with an external

Re: [HACKERS] libpq should not be using SSL_CTX_set_client_cert_cb

2010-05-26 Thread Garick Hamlin
On Wed, May 26, 2010 at 10:54:42AM -0400, Tom Lane wrote: Garick Hamlin gham...@isc.upenn.edu writes: I am guessing the problem is that validating the presented chain is hard? No, the problem is that the current libpq code fails to present the chain at all. It will only load and send the

Re: [HACKERS] Exposing the Xact commit order to the user

2010-05-26 Thread Robert Haas
On Wed, May 26, 2010 at 11:43 AM, Steve Singer ssin...@ca.afilias.info wrote: Jan Wieck wrote: Since the actual row level change information and other event data is found inside of regular tables, identified by TXID and sequence number, I am pretty sure I want that data in a server-side

Re: [HACKERS] Exposing the Xact commit order to the user

2010-05-26 Thread Jan Wieck
On 5/26/2010 10:04 AM, Greg Stark wrote: Instead of discussing implementation I think you should start with the API the replication system needs. ... but to answer that request, actually I don't even think we should be discussing API specifics. During PGCon, Marco Kreen, Jim Nasby and I

Re: [HACKERS] libpq should not be using SSL_CTX_set_client_cert_cb

2010-05-26 Thread Tom Lane
Garick Hamlin gham...@isc.upenn.edu writes: One could make it work with multiple TAs in a similar fashion if it also checked for the existence of a directory (like: ~/.postgresql/client_ta ) to store chains to each supported TA by fingerprint. That might not be worth the effort at this

Re: [HACKERS] Exposing the Xact commit order to the user

2010-05-26 Thread Greg Stark
On Wed, May 26, 2010 at 5:10 PM, Jan Wieck janwi...@yahoo.com wrote: ... but to answer that request, actually I don't even think we should be discussing API specifics. How about just API generalities? Like, where do you need this data, on the master or on the slave? Would PGXC like it on the

Re: [HACKERS] [PATCH] Add XMLEXISTS function from the SQL/XML standard

2010-05-26 Thread Peter Eisentraut
On ons, 2010-05-26 at 11:47 +0100, Mike Fowler wrote: The XMLEXISTS function works with XQuery expressions and doesn't have the call signature that your patch implements Looking at the manuals of Oracle, Derby and DB2 I see how the call signature differs. I also note that Oracle's

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Heikki Linnakangas
On 26/05/10 18:31, Robert Haas wrote: And frankly, I don't think it's possible for quorum commit to reduce the number of parameters. Even if we have that feature available, not everyone will want to use it. And the people who don't will presumably need whatever parameters they would have

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Kevin Grittner
Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: One way to do that would be to refrain from flushing the commit record to disk on the master until the standby has acknowledged it. I'm not clear on the benefit of doing that, versus flushing the commit record and then waiting

Re: [HACKERS] Exposing the Xact commit order to the user

2010-05-26 Thread Heikki Linnakangas
Could you generate the commit-order log by simply registering a commit hook (RegisterXactCallback(XACT_EVENT_COMMIT)) that writes such a log somewhere in the data directory? That would work with older versions too, no server changes required. It would not get called during recovery, but I

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Heikki Linnakangas
On 26/05/10 20:10, Kevin Grittner wrote: Heikki Linnakangasheikki.linnakan...@enterprisedb.com wrote: One way to do that would be to refrain from flushing the commit record to disk on the master until the standby has acknowledged it. I'm not clear on the benefit of doing that, versus

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Simon Riggs
On Wed, 2010-05-26 at 11:31 -0400, Robert Haas wrote: Your reply has again avoided the subject of how we would handle failure modes with per-standby settings. That is important. I don't think anyone is avoiding that, we just haven't discussed it. You haven't discussed it, but even before

Re: [HACKERS] mapping object names to role IDs

2010-05-26 Thread Tom Lane
alvherre alvhe...@commandprompt.com writes: Excerpts from Robert Haas's message of mié may 26 07:20:30 -0400 2010: I still feel that we'd be better off putting all the functions that use the same design pattern in a single file, rather than spreading them out all over the backend. This

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Simon Riggs
On Wed, 2010-05-26 at 12:10 -0500, Kevin Grittner wrote: Adding a synchronous standby should require some action in the master, since it affects the behavior on master. +1 +1 -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Kevin Grittner
Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Although, if the master crashes at that point, and quickly recovers, you could see the last transactions committed on the master before they're replicated to the standby. Versus having the transaction committed on one or more

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Simon Riggs
On Wed, 2010-05-26 at 19:55 +0300, Heikki Linnakangas wrote: Now you want to set up a temporary replica of the master at a development server, for testing purposes. If you set quorum to 2, your development server becomes critical infrastructure, which is not what you want. That's a good

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Robert Haas
On Wed, May 26, 2010 at 1:24 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 26/05/10 20:10, Kevin Grittner wrote: Heikki Linnakangasheikki.linnakan...@enterprisedb.com  wrote: One way to do that would be to refrain from flushing the commit record to disk on the master

Re: [HACKERS] WIP patch for serializable transactions with predicate locking

2010-05-26 Thread Selena Deckelmann
On Wed, May 19, 2010 at 5:37 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: I heard that others were considering work on predicate locks for 9.1. Since Dan Ports of MIT and I have been working on that for the serializable implementation for the last few weeks, I felt it would be good to

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Heikki Linnakangas
On 26/05/10 20:33, Kevin Grittner wrote: Heikki Linnakangasheikki.linnakan...@enterprisedb.com wrote: Although, if the master crashes at that point, and quickly recovers, you could see the last transactions committed on the master before they're replicated to the standby. Versus having the

[HACKERS] psql \? \daS

2010-05-26 Thread Stephen Frost
Greetings, Noticed this while playing around with psql regression tests. Thanks, Stephen commit 31bf61bab77ad991f2a67a41699384e57c021508 Author: Stephen Frost sfr...@snowman.net Date: Wed May 26 13:51:27 2010 -0400 Add 'S' to optional parameters for \da

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Kevin Grittner
Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Unless we have a transaction manager and do proper distributed transactions, how do you avoid edge conditions like that? Yeah, I guess you can't. You can guarantee that a commit is always safely flushed first in the master, or

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Heikki Linnakangas
On 26/05/10 20:40, Simon Riggs wrote: On Wed, 2010-05-26 at 19:55 +0300, Heikki Linnakangas wrote: If you set quorum to 1, it also becomes critical infrastructure, because it's possible that a transaction has been replicated to the test server but not the real production standby, and a meteor

[HACKERS] Fwd: Re: [BUGS] dividing money by money

2010-05-26 Thread Kevin Grittner
Hi Andy, Do you want to package this up as a patch for 9.1? If not, is it OK if I do? -Kevin Andy Balholm a...@balholm.com wrote: On Apr 1, 2010, at 7:57 AM, Kevin Grittner wrote: I'm inclined to think it's better to have an explicit cast from money to numeric, as long as it is exact,

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Robert Haas
On Wed, May 26, 2010 at 1:26 PM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, 2010-05-26 at 11:31 -0400, Robert Haas wrote: Your reply has again avoided the subject of how we would handle failure modes with per-standby settings. That is important. I don't think anyone is avoiding that,

Re: [HACKERS] Idea for getting rid of VACUUM FREEZE on cold pages

2010-05-26 Thread Josh Berkus
On 5/25/10 10:04 PM, Heikki Linnakangas wrote: On 25/05/10 23:56, Josh Berkus wrote: Do we get a bit in the visibility map for a page which has aborted transaction rows on it? If there's a tuple with an aborted xmin on a page, the bit in the visibility map is not set. A tuple with aborted

Re: [HACKERS] Exposing the Xact commit order to the user

2010-05-26 Thread Jan Wieck
On 5/26/2010 1:17 PM, Heikki Linnakangas wrote: Could you generate the commit-order log by simply registering a commit hook (RegisterXactCallback(XACT_EVENT_COMMIT)) that writes such a log somewhere in the data directory? That would work with older versions too, no server changes required.

Re: [HACKERS] Idea for getting rid of VACUUM FREEZE on cold pages

2010-05-26 Thread Heikki Linnakangas
On 26/05/10 21:35, Josh Berkus wrote: On 5/25/10 10:04 PM, Heikki Linnakangas wrote: On 25/05/10 23:56, Josh Berkus wrote: Do we get a bit in the visibility map for a page which has aborted transaction rows on it? If there's a tuple with an aborted xmin on a page, the bit in the visibility

Re: [HACKERS] Idea for getting rid of VACUUM FREEZE on cold pages

2010-05-26 Thread Josh Berkus
In theory, until any tuple on the page is inserted/updated/deleted again. However, we've been operating on the assumption that it's always safe to clear any bit in the visibility map, without affecting correctness. I would not like to give up that assumption, it makes life easier. It

Re: [HACKERS] mapping object names to role IDs

2010-05-26 Thread Robert Haas
On Wed, May 26, 2010 at 1:27 PM, Tom Lane t...@sss.pgh.pa.us wrote: alvherre alvhe...@commandprompt.com writes: Excerpts from Robert Haas's message of mié may 26 07:20:30 -0400 2010: I still feel that we'd be better off putting all the functions that use the same design pattern in a single

Re: [HACKERS] mapping object names to role IDs

2010-05-26 Thread alvherre
Excerpts from Robert Haas's message of mié may 26 10:34:00 -0400 2010: lsyscache.c might have no conceptual consistency but it's extremely useful, I know I've been annoyed by lsyscache: looking for accessors to catalog stuff, not finding them and so creating my own by using syscache directly,

Re: [HACKERS] Idea for getting rid of VACUUM FREEZE on cold pages

2010-05-26 Thread Robert Haas
On Wed, May 26, 2010 at 2:44 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 26/05/10 21:35, Josh Berkus wrote: On 5/25/10 10:04 PM, Heikki Linnakangas wrote: On 25/05/10 23:56, Josh Berkus wrote: Do we get a bit in the visibility map for a page which has aborted

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Simon Riggs
On Wed, 2010-05-26 at 14:30 -0400, Robert Haas wrote: On Wed, May 26, 2010 at 1:26 PM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, 2010-05-26 at 11:31 -0400, Robert Haas wrote: Your reply has again avoided the subject of how we would handle failure modes with per-standby settings.

Re: [HACKERS] Exposing the Xact commit order to the user

2010-05-26 Thread Heikki Linnakangas
On 26/05/10 21:43, Jan Wieck wrote: On 5/26/2010 1:17 PM, Heikki Linnakangas wrote: It would not get called during recovery, but I believe that would be sufficient for Slony. You could always batch commits that you don't know when they committed as if they committed simultaneously. Here you

Re: [HACKERS] Regression testing for psql

2010-05-26 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: Then, too, there's the fact that many of these tests fail on my machine because my username is not sfrost, I've updated the patch to address this, it's again at: http://snowman.net/~sfrost/psql-regress-help.patch If the size is still an issue, I

Re: [HACKERS] psql \? \daS

2010-05-26 Thread Robert Haas
On Wed, May 26, 2010 at 1:59 PM, Stephen Frost sfr...@snowman.net wrote:  Noticed this while playing around with psql regression tests. Good catch. It looks like the + option doesn't actually do anything for \da, though, so I'm inclined to just write \da[S] rather than \da[S+] (see: \dl, \dC).

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Robert Haas
On Wed, May 26, 2010 at 3:13 PM, Simon Riggs si...@2ndquadrant.com wrote: I don't really understand this comment.  I have said, and I believe, that a system without quorum commit is simpler than one with quorum commit.  I'd debate the point with you but I find the point so self-evident that I

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Joshua D. Drake
On Wed, 2010-05-26 at 15:37 -0400, Robert Haas wrote: On Wed, May 26, 2010 at 3:13 PM, Simon Riggs si...@2ndquadrant.com wrote: I don't really understand this comment. I have said, and I believe, that a system without quorum commit is simpler than one with quorum commit. I'd debate the

Re: [HACKERS] [PATCH] Add _PG_init to PL language handler documentation

2010-05-26 Thread Robert Haas
On Tue, May 25, 2010 at 4:34 AM, Jonathan Leto jal...@gmail.com wrote: This tiny doc patch adds _PG_init to the skeleton example code for a PL. The information is quite valuable to PL authors, who might miss it when it is described in the shared library documentation. I'm not sure it does much

Re: [HACKERS] Regression testing for psql

2010-05-26 Thread alvherre
Excerpts from Stephen Frost's message of mié may 26 15:19:59 -0400 2010: * Robert Haas (robertmh...@gmail.com) wrote: Then, too, there's the fact that many of these tests fail on my machine because my username is not sfrost, I've updated the patch to address this, it's again at:

Re: [HACKERS] Exposing the Xact commit order to the user

2010-05-26 Thread Dimitri Fontaine
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Perhaps I'm missing something, but I thought that Slony currently uses a heartbeat, and all transactions committed between two beats are banged together and committed as one in the slave so that their relative commit order doesn't

[HACKERS] cursor_to_xml: How to signal end?

2010-05-26 Thread Peter Eisentraut
Currently, cursor_to_xml returns an empty string when the end of the cursor is reached (meaning the fetch returned zero rows). As discussed on -general, that's kind of weird, because you'd have to do something like IF val::text = '' to test for the end, since there is no = operator for xml, and

Re: [HACKERS] Exposing the Xact commit order to the user

2010-05-26 Thread Robert Haas
On Wed, May 26, 2010 at 4:11 PM, Dimitri Fontaine dfonta...@hi-media.com wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Perhaps I'm missing something, but I thought that Slony currently uses a heartbeat, and all transactions committed between two beats are banged

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Dimitri Fontaine
Simon Riggs si...@2ndquadrant.com writes: On Wed, 2010-05-26 at 19:55 +0300, Heikki Linnakangas wrote: Now you want to set up a temporary replica of the master at a development server, for testing purposes. If you set quorum to 2, your development server becomes critical infrastructure, which

Re: [HACKERS] Exposing the Xact commit order to the user

2010-05-26 Thread Jan Wieck
On 5/26/2010 3:16 PM, Heikki Linnakangas wrote: On 26/05/10 21:43, Jan Wieck wrote: On 5/26/2010 1:17 PM, Heikki Linnakangas wrote: It would not get called during recovery, but I believe that would be sufficient for Slony. You could always batch commits that you don't know when they committed

Re: [HACKERS] Exposing the Xact commit order to the user

2010-05-26 Thread Kevin Grittner
Jan Wieck janwi...@yahoo.com wrote: Without this logic, the replication system could not combine multiple origin sessions into one replication session without risking to never find a state, in which it can commit. My latest idea for handling this in WAL-based replication involves

Re: [HACKERS] Exposing the Xact commit order to the user

2010-05-26 Thread Jan Wieck
On 5/26/2010 4:11 PM, Dimitri Fontaine wrote: So even ordering the txid and txid_snapshots with respect to WAL commit time (LSN) won't be the whole story, for any given transaction containing more than one event we also need to have them in order. I know Jan didn't forget about it so it must

[HACKERS] functional call named notation clashes with SQL feature

2010-05-26 Thread Peter Eisentraut
It turns out that the SQL standard uses the function call notation foo(this AS that) for something else: routine invocation ::= routine name SQL argument list routine name ::= [ schema name period ] qualified identifier SQL argument list ::= left paren [ SQL argument [ { comma SQL argument

Re: [HACKERS] Exposing the Xact commit order to the user

2010-05-26 Thread Heikki Linnakangas
On 26/05/10 23:32, Jan Wieck wrote: Consider two transactions A and B that due to transaction batching between snapshots get applied together. Let the order of actions be 1. A starts 2. B starts 3. B selects a row for update, then updates the row 4. A tries to do the same and blocks 5. B

Re: [HACKERS] Exposing the Xact commit order to the user

2010-05-26 Thread Jan Wieck
On 5/26/2010 12:38 PM, Greg Stark wrote: On Wed, May 26, 2010 at 5:10 PM, Jan Wieck janwi...@yahoo.com wrote: ... but to answer that request, actually I don't even think we should be discussing API specifics. How about just API generalities? Like, where do you need this data, on the master

Re: [HACKERS] Exposing the Xact commit order to the user

2010-05-26 Thread Heikki Linnakangas
On 26/05/10 23:45, Heikki Linnakangas wrote: On 26/05/10 23:32, Jan Wieck wrote: Consider two transactions A and B that due to transaction batching between snapshots get applied together. Let the order of actions be 1. A starts 2. B starts 3. B selects a row for update, then updates the row 4.

Re: [HACKERS] Idea for getting rid of VACUUM FREEZE on cold pages

2010-05-26 Thread Josh Berkus
What if we drove it off of the PD_ALL_VISIBLE bit on the page itself, rather than the visibility map bit? It would be safe to clear the visibility map bit without touching the page, but if you clear the PD_ALL_VISIBLE bit on the page itself then you set all the hint bits and freeze all the

Re: [HACKERS] Show schema name on REINDEX DATABASE

2010-05-26 Thread alvherre
Excerpts from Selena Deckelmann's message of mié may 26 11:07:40 -0400 2010: On Mon, Apr 5, 2010 at 9:29 AM, Greg Sabino Mullane g...@turnstep.com wrote: Patch attached to show the schema *and* table name when doing a REINDEX DATABASE. Is this something that can be added to 9.1 commitfest?

Re: [HACKERS] Exposing the Xact commit order to the user

2010-05-26 Thread Heikki Linnakangas
On 26/05/10 23:49, Jan Wieck wrote: In this implementation it wouldn't even matter if a transaction that was recorded actually never made it because it crashed before the WAL flush. It would be reported by this commit order feature, but there would be no traces of whatever it did to be found

Re: [HACKERS] Exposing the Xact commit order to the user

2010-05-26 Thread Jan Wieck
On 5/26/2010 4:52 PM, Heikki Linnakangas wrote: Ok, I think I understand it now. The commit order is enough, because replaying the actions in the order all actions of B, then all actions of A yields the same result. Precisely. Jan -- Anyone who trades liberty for security deserves neither

Re: [HACKERS] [ADMIN] command tag logging

2010-05-26 Thread alvherre
Excerpts from Ray Stell's message of mié may 26 17:08:33 -0400 2010: I just installed a compiled from src 8.3.11. I usually include %i, command tag, in the log_line_prefix setting. This causes some spewage I'd not seen before on connection received lines as if it is dumping the environment:

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Heikki Linnakangas
On 26/05/10 23:31, Dimitri Fontaine wrote: d. choice of commit or rollback at timeout Rollback is not an option. There is no going back after the commit record has been flushed to disk or sent to a standby. The choice is to either commit anyway after the timeout, or wait forever. --

Re: [HACKERS] Show schema name on REINDEX DATABASE

2010-05-26 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Patch attached to show the schema *and* table name when doing a REINDEX DATABASE. Is this something that can be added to 9.1 commitfest? Not in this form, apparently. Can we convince Greg or someone else to work on adding some more

Re: [HACKERS] Synchronization levels in SR

2010-05-26 Thread Jan Wieck
On 5/26/2010 12:55 PM, Heikki Linnakangas wrote: On 26/05/10 18:31, Robert Haas wrote: And frankly, I don't think it's possible for quorum commit to reduce the number of parameters. Even if we have that feature available, not everyone will want to use it. And the people who don't will

  1   2   >