[HACKERS] (auto)vacuum truncate exclusive lock

2013-04-11 Thread Jeff Janes
I guess I'm a couple releases late to review the autovacuum truncate exclusive lock patch (a79ae0bc0d454b9f2c95a), but this patch did not only affect autovac, it affects manual vacuum as well (as did the original behavior it is a modification of). So the compiler constants are misnamed, and the

Re: [HACKERS] Inconsistent DB data in Streaming Replication

2013-04-11 Thread Amit Kapila
On Wednesday, April 10, 2013 10:31 PM Fujii Masao wrote: On Thu, Apr 11, 2013 at 1:44 AM, Shaun Thomas stho...@optionshouse.com wrote: On 04/10/2013 11:40 AM, Fujii Masao wrote: Strange. If this is really true, shared disk failover solution is fundamentally broken because the standby

[HACKERS] Add regression tests for COLLATE

2013-04-11 Thread Robins Tharakan
Hi, Please find attached a patch to take 'make check' code-coverage of COLLATE (/src/backend/commands/collationcmds) from 0% to 96%. Any feedback is more than welcome. Also posting this to Commitfest-next. -- Robins Tharakan regress_collate_v1.patch Description: Binary data -- Sent via

Re: [HACKERS] [DOCS] synchronize_seqscans' description is a bit misleading

2013-04-11 Thread Gurjeet Singh
On Wed, Apr 10, 2013 at 11:56 PM, Tom Lane t...@sss.pgh.pa.us wrote: Gurjeet Singh gurj...@singh.im writes: So, again, it is not guaranteed that all the scans on a relation will synchronize with each other. Hence my proposal to include the term 'probability' in the definition. Yeah, it's

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-04-11 Thread Simon Riggs
On 11 April 2013 00:37, Robert Haas robertmh...@gmail.com wrote: On Sat, Apr 6, 2013 at 10:44 AM, Andres Freund and...@2ndquadrant.com wrote: I feel pretty strongly that we shouldn't add any such complications to XLogInsert() itself, its complicated enough already and it should be made

Re: [HACKERS] Inconsistent DB data in Streaming Replication

2013-04-11 Thread Ants Aasma
On Thu, Apr 11, 2013 at 10:09 AM, Amit Kapila amit.kap...@huawei.com wrote: Consider the case old-master crashed during flushing the data page, now you would need full page image from new-master. It might so happen that in new-master Checkpoint would have purged (reused) the log file's from

Re: [HACKERS] Inconsistent DB data in Streaming Replication

2013-04-11 Thread Sameer Thakur
Hello, The only potential use case for this that I can see, would be for system maintenance and a controlled failover. I agree: that's a major PITA when doing DR testing, but I personally don't think this is the way to fix that particular edge case. This is the use case we are trying to address

Re: [HACKERS] Inconsistent DB data in Streaming Replication

2013-04-11 Thread Hannu Krosing
On 04/11/2013 01:26 PM, Sameer Thakur wrote: Hello, The only potential use case for this that I can see, would be for system maintenance and a controlled failover. I agree: that's a major PITA when doing DR testing, but I personally don't think this is the way to fix that particular edge

Re: [HACKERS] [GSOC] questions about idea rewrite pg_dump as library

2013-04-11 Thread Tom Lane
Pavel Golub pa...@microolap.com writes: From my point of view the new library should export only two functions: 1. The execution function: ExecStatusType PGdumpdbParams(const char * const *keywords, const char * const *values); No, this is exactly *wrong*. You might as

Re: [HACKERS] Inconsistent DB data in Streaming Replication

2013-04-11 Thread Ants Aasma
On Thu, Apr 11, 2013 at 4:25 PM, Hannu Krosing ha...@2ndquadrant.com wrote: The proposed fix - halting all writes of data pages to disk and to WAL files while waiting ACK from standby - will tremendously slow down all parallel work on master. This is not what is being proposed. The proposed

Re: [HACKERS] Inconsistent DB data in Streaming Replication

2013-04-11 Thread Tom Lane
Ants Aasma a...@cybertec.at writes: On Thu, Apr 11, 2013 at 4:25 PM, Hannu Krosing ha...@2ndquadrant.com wrote: The proposed fix - halting all writes of data pages to disk and to WAL files while waiting ACK from standby - will tremendously slow down all parallel work on master. This is not

Re: [HACKERS] Inconsistent DB data in Streaming Replication

2013-04-11 Thread Hannu Krosing
On 04/11/2013 03:52 PM, Ants Aasma wrote: On Thu, Apr 11, 2013 at 4:25 PM, Hannu Krosing ha...@2ndquadrant.com wrote: The proposed fix - halting all writes of data pages to disk and to WAL files while waiting ACK from standby - will tremendously slow down all parallel work on master. This is

[HACKERS] Nearing beta?

2013-04-11 Thread Bruce Momjian
We are nearing April 15 --- are we nearing a time when we can close 9.3 development and start focusing on the beta? -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +

Re: [HACKERS] [GSOC] questions about idea rewrite pg_dump as library

2013-04-11 Thread Pavel Golub
Hello, Tom. You wrote: TL Pavel Golub pa...@microolap.com writes: From my point of view the new library should export only two functions: 1. The execution function: ExecStatusType PGdumpdbParams(const char * const *keywords, const char * const *values); TL No, this is

Re: [HACKERS] Inconsistent DB data in Streaming Replication

2013-04-11 Thread Ants Aasma
On Thu, Apr 11, 2013 at 5:33 PM, Hannu Krosing ha...@2ndquadrant.com wrote: On 04/11/2013 03:52 PM, Ants Aasma wrote: On Thu, Apr 11, 2013 at 4:25 PM, Hannu Krosing ha...@2ndquadrant.com wrote: The proposed fix - halting all writes of data pages to disk and to WAL files while waiting ACK

Re: [HACKERS] ObjectClass/ObjectType mixup

2013-04-11 Thread Alvaro Herrera
Peter Eisentraut wrote: src/backend/catalog/dependency.c:213: EventTriggerSupportsObjectType(getObjectClass(thisobj))) src/backend/commands/event_trigger.c:1014: Assert(EventTriggerSupportsObjectType(getObjectClass(object))); getObjectClass() returns type ObjectClass, but

Re: [HACKERS] Clang compiler warning on 9.3 HEAD

2013-04-11 Thread Alvaro Herrera
Tom Lane wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: Now, it annoys me that we now have three places that know about object types supported by event triggers: there's a large struct of command tag substrings (event_trigger_support), then there's these two functions. It might be

Re: [HACKERS] Nearing beta?

2013-04-11 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: We are nearing April 15 --- are we nearing a time when we can close 9.3 development and start focusing on the beta? It's time to start maintaining the list of open items for 9.3, which would help us figure out if we're ready for beta. The issue with event

Re: [HACKERS] Nearing beta?

2013-04-11 Thread Bruce Momjian
On Thu, Apr 11, 2013 at 11:27:34AM -0400, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: We are nearing April 15 --- are we nearing a time when we can close 9.3 development and start focusing on the beta? It's time to start maintaining the list of open items for 9.3, which would

Re: [HACKERS] Nearing beta?

2013-04-11 Thread Alvaro Herrera
Tom Lane wrote: Bruce Momjian br...@momjian.us writes: We are nearing April 15 --- are we nearing a time when we can close 9.3 development and start focusing on the beta? It's time to start maintaining the list of open items for 9.3, which would help us figure out if we're ready for beta.

Re: [HACKERS] Nearing beta?

2013-04-11 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: Tom Lane wrote: The issue with event triggers causing catalog access during START TRANSACTION is clearly a must fix before beta item. I'm afraid that bug #8049 is going to require some nontrivial planner changes (more about that in a bit),

Re: [HACKERS] Nearing beta?

2013-04-11 Thread Tom Lane
I wrote: It's time to start maintaining the list of open items for 9.3, which would help us figure out if we're ready for beta. I momentarily forgot about materialized views. We definitely need to decide whether we're going to yank unlogged matviews from 9.3. regards,

Re: [HACKERS] Nearing beta?

2013-04-11 Thread Bruce Momjian
On Thu, Apr 11, 2013 at 11:42:27AM -0400, Tom Lane wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: Tom Lane wrote: The issue with event triggers causing catalog access during START TRANSACTION is clearly a must fix before beta item. I'm afraid that bug #8049 is going to require

Re: [HACKERS] [GSOC] questions about idea rewrite pg_dump as library

2013-04-11 Thread Andrew Dunstan
On 04/11/2013 09:51 AM, Tom Lane wrote: Pavel Golub pa...@microolap.com writes: From my point of view the new library should export only two functions: 1. The execution function: ExecStatusType PGdumpdbParams(const char * const *keywords, const char * const *values); No,

Re: [HACKERS] [GSOC] questions about idea rewrite pg_dump as library

2013-04-11 Thread Stephen Frost
* Andrew Dunstan (and...@dunslane.net) wrote: On 04/11/2013 09:51 AM, Tom Lane wrote: No, this is exactly *wrong*. You might as well not bother to refactor, if the only API the library presents is exactly equivalent to what you could get with system(pg_dump ...). Agreed. Well, either they

Re: [HACKERS] Nearing beta?

2013-04-11 Thread Ants Aasma
On Thu, Apr 11, 2013 at 6:27 PM, Tom Lane t...@sss.pgh.pa.us wrote: Bruce Momjian br...@momjian.us writes: We are nearing April 15 --- are we nearing a time when we can close 9.3 development and start focusing on the beta? It's time to start maintaining the list of open items for 9.3, which

Re: [HACKERS] Nearing beta?

2013-04-11 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: I can have release notes ready for April 22 --- I will get started. I think all our major features are committed. Well, we still have an open commitfest to deal with. I have taken the liberty of marking as returned with feedback, or moving to 2013-Next,

Re: [HACKERS] Inconsistent DB data in Streaming Replication

2013-04-11 Thread Fujii Masao
On Thu, Apr 11, 2013 at 2:42 AM, Tom Lane t...@sss.pgh.pa.us wrote: Ants Aasma a...@cybertec.at writes: We already rely on WAL-before-data to ensure correct recovery. What is proposed here is to slightly redefine it to require WAL to be replicated before it is considered to be flushed. This

[HACKERS] Analyzing bug 8049

2013-04-11 Thread Tom Lane
I looked into the problem reported at http://www.postgresql.org/message-id/e1upa3b-0004k0...@wrigleys.postgresql.org which is that we're deriving a bogus plan for this query: SELECT * FROM ( SELECT (COALESCE(h_n || '/', '') || l_n)::text AS fault FROM ( SELECT _bug_header.h_n,

Re: [HACKERS] Inconsistent DB data in Streaming Replication

2013-04-11 Thread Fujii Masao
On Thu, Apr 11, 2013 at 10:25 PM, Hannu Krosing ha...@2ndquadrant.com wrote: You just shut down the old master and let the standby catch up (takas a few microseconds ;) ) before you promote it. After this you can start up the former master with recovery.conf and it will follow nicely. No.

Re: [HACKERS] Inconsistent DB data in Streaming Replication

2013-04-11 Thread Fujii Masao
On Fri, Apr 12, 2013 at 12:09 AM, Ants Aasma a...@cybertec.at wrote: On Thu, Apr 11, 2013 at 5:33 PM, Hannu Krosing ha...@2ndquadrant.com wrote: On 04/11/2013 03:52 PM, Ants Aasma wrote: On Thu, Apr 11, 2013 at 4:25 PM, Hannu Krosing ha...@2ndquadrant.com wrote: The proposed fix - halting

[HACKERS] Viewing new 9.3 error fields

2013-04-11 Thread Bruce Momjian
This blog entry displays the new 9.3 error fields, schema/table/constraint: http://www.depesz.com/2013/03/07/waiting-for-9-3-provide-database-object-names-as-separate-fields-in-error-messages/ $ INSERT INTO t (i) VALUES (1); psql:z.sql:16: ERROR: 23505:

Re: [HACKERS] Viewing new 9.3 error fields

2013-04-11 Thread Dickson S. Guedes
Em Qui, 2013-04-11 às 14:08 -0400, Bruce Momjian escreveu: This blog entry displays the new 9.3 error fields, schema/table/constraint: http://www.depesz.com/2013/03/07/waiting-for-9-3-provide-database-object-names-as-separate-fields-in-error-messages/ $ INSERT INTO t (i)

Re: [HACKERS] Viewing new 9.3 error fields

2013-04-11 Thread Alvaro Herrera
Bruce Momjian wrote: However, I am unable to see this in psql: CREATE TABLE ledger (id SERIAL, balance NUMERIC(10,2) unique); SET log_error_verbosity = 'verbose'; INSERT INTO ledger VALUES (DEFAULT, 1); INSERT INTO ledger VALUES (DEFAULT, 1);

Re: [HACKERS] Enabling Checksums

2013-04-11 Thread Simon Riggs
On 11 April 2013 04:27, Jeff Davis pg...@j-davis.com wrote: On Wed, 2013-04-10 at 20:17 +0100, Simon Riggs wrote: OK, so we have a single combined calculate a checksum for a block function. That uses Jeff's zeroing trick and Ants' bulk-oriented performance optimization. For buffer

Re: [HACKERS] Viewing new 9.3 error fields

2013-04-11 Thread Bruce Momjian
On Thu, Apr 11, 2013 at 03:31:17PM -0300, Alvaro Herrera wrote: Bruce Momjian wrote: However, I am unable to see this in psql: CREATE TABLE ledger (id SERIAL, balance NUMERIC(10,2) unique); SET log_error_verbosity = 'verbose'; INSERT INTO ledger VALUES

[HACKERS] Add SPI_gettypmod() fucntion

2013-04-11 Thread Miguel Angel de Blas Burdalo
Hi hackers, My name is Miguel Angel de Blas. I'm newbie and it's my first collaborationso I hope it's correct ;-) I created a function SPI_gettypmod: int32SPI_gettypmod(TupleDesc tupdesc, int fnumber); Return -1 when type not need type-specific data or error. ref TODO: Add SPI_gettypmod()

Re: [HACKERS] [ADMIN] after 9.2.4 patch vacuumdb -avz not analyzing all tables

2013-04-11 Thread Tom Lane
Mike Broers mbro...@gmail.com writes: After patching to 9.2.4 I am noticing some mysterious behavior in my nightly vacuumdb cron job. I have been running vacuumdb -avz nightly for a while now, and have a script that tells me the next day if all the tables in pg_stat_user_tables have been

Re: [HACKERS] (auto)vacuum truncate exclusive lock

2013-04-11 Thread Tom Lane
Jeff Janes jeff.ja...@gmail.com writes: I guess I'm a couple releases late to review the autovacuum truncate exclusive lock patch (a79ae0bc0d454b9f2c95a), but this patch did not only affect autovac, it affects manual vacuum as well (as did the original behavior it is a modification of). So

Re: [HACKERS] [GSOC] questions about idea rewrite pg_dump as library

2013-04-11 Thread Michael Paquier
On Fri, Apr 12, 2013 at 1:00 AM, Stephen Frost sfr...@snowman.net wrote: Well, either they want that or they want that output more accessibly, and without all the baggage that pg_dump necessarily brings to the table. pg_dump does a lot of stuff that's basically designed for bulk

Re: [HACKERS] [GSOC] questions about idea rewrite pg_dump as library

2013-04-11 Thread Tatsuo Ishii
Well, either they want that or they want that output more accessibly, and without all the baggage that pg_dump necessarily brings to the table. pg_dump does a lot of stuff that's basically designed for bulk operations, and often what people want is a way to get, say, the creation DDL for

Re: [HACKERS] [ADMIN] after 9.2.4 patch vacuumdb -avz not analyzing all tables

2013-04-11 Thread Jeff Janes
On Thursday, April 11, 2013, Tom Lane wrote: [ pokes around ... ] You certain 9.2.3 didn't do this too? This appears to be an intentional behavior of the 9.2.3 patch that made it cancel truncation when there were conflicting lock requests: /* * Report results to the stats

Re: [HACKERS] Add SPI_gettypmod() fucntion

2013-04-11 Thread Amit Kapila
On Friday, April 12, 2013 4:24 AM Miguel Angel de Blas Burdalo wrote: Hi hackers, My name is Miguel Angel de Blas. I'm newbie and it's my first collaboration so I hope it's correct ;-) I created a function SPI_gettypmod: int32SPI_gettypmod(TupleDesc tupdesc, int fnumber); Return -1

Re: [HACKERS] [ADMIN] after 9.2.4 patch vacuumdb -avz not analyzing all tables

2013-04-11 Thread Tom Lane
Jeff Janes jeff.ja...@gmail.com writes: I believe the rationale was so that an autovacuum would still look like it was needed, and get fired again the next naptime, so that it could continue with the truncation attempts. (Rather than waiting for 20% turnover in the table before trying again).

Re: [HACKERS] Add SPI_gettypmod() fucntion

2013-04-11 Thread Michael Paquier
On Fri, Apr 12, 2013 at 7:53 AM, Miguel Angel de Blas Burdalo migueldeb...@gmail.com wrote: Hi hackers, My name is Miguel Angel de Blas. I'm newbie and it's my first collaboration so I hope it's correct ;-) I created a function SPI_gettypmod: int32SPI_gettypmod(TupleDesc tupdesc, int

Re: [HACKERS] [ADMIN] after 9.2.4 patch vacuumdb -avz not analyzing all tables

2013-04-11 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: However I've got to say that both of those side-effects of exclusive-lock abandonment seem absolutely brain dead now that I see them.  Why would we not bother to tell the stats collector what we've done?  Why would we think we should not do ANALYZE when we

Re: [HACKERS] (auto)vacuum truncate exclusive lock

2013-04-11 Thread Kevin Grittner
Jeff Janes jeff.ja...@gmail.com wrote: I guess I'm a couple releases late to review the autovacuum truncate exclusive lock patch (a79ae0bc0d454b9f2c95a), but this patch did not only affect autovac, it affects manual vacuum as well (as did the original behavior it is a modification of).  So

Re: [HACKERS] Add regression tests for COLLATE

2013-04-11 Thread Michael Paquier
On Thu, Apr 11, 2013 at 4:14 PM, Robins Tharakan thara...@gmail.com wrote: Hi, Please find attached a patch to take 'make check' code-coverage of COLLATE (/src/backend/commands/collationcmds) from 0% to 96%. Any feedback is more than welcome. Also posting this to Commitfest-next. Just by

[HACKERS] (auto)vacuum truncate exclusive lock

2013-04-11 Thread Jeff Janes
On Thursday, April 11, 2013, Tom Lane wrote: Jeff Janes jeff.ja...@gmail.com writes: I guess I'm a couple releases late to review the autovacuum truncate exclusive lock patch (a79ae0bc0d454b9f2c95a), but this patch did not only affect autovac, it affects manual vacuum as well (as did the

Re: [HACKERS] Inconsistent DB data in Streaming Replication

2013-04-11 Thread Pavan Deolasee
On Thu, Apr 11, 2013 at 8:39 PM, Ants Aasma a...@cybertec.at wrote: On Thu, Apr 11, 2013 at 5:33 PM, Hannu Krosing ha...@2ndquadrant.com wrote: On 04/11/2013 03:52 PM, Ants Aasma wrote: On Thu, Apr 11, 2013 at 4:25 PM, Hannu Krosing ha...@2ndquadrant.com wrote: The proposed fix -