Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-12-01 Thread Amit Kapila
On Saturday, December 01, 2012 1:30 AM Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Nov 28, 2012 at 9:47 AM, Amit kapila amit.kap...@huawei.com wrote: 5. PERSISTENT Keyword is added to the reserved keyword list. As it was giving some errors given below while parsing

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-12-01 Thread Albe Laurenz
Magnus Hagander wrote: On 30.11.2012 21:02, Andres Freund wrote: There are workloads where its detrimental, but in general having it default to on improver experience tremendously because getting conflicts because of vacuum is rather confusing. In the workloads where it might not be a good

Re: [HACKERS] WIP: index support for regexp search

2012-12-01 Thread Erik Rijkers
On Fri, November 30, 2012 12:22, Alexander Korotkov wrote: Hi! On Thu, Nov 29, 2012 at 12:58 PM, er e...@xs4all.nl wrote: On Mon, November 26, 2012 20:49, Alexander Korotkov wrote: I ran the simple-minded tests against generated data (similar to the ones I did in January 2012). The

Re: [HACKERS] proposal: fix corner use case of variadic fuctions usage

2012-12-01 Thread Pavel Stehule
Hello Hi Pavel. I am trying to review this patch and on my work computer everything compiles and tests perfectly. However, on my laptop, the regression tests don't pass with cache lookup failed for type XYZ where XYZ is some number that does not appear to be any type oid. I don't really

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Andrew Dunstan
On 11/30/2012 11:10 PM, Tom Lane wrote: Some of the buildfarm members are failing the pg_upgrade regression test since commit 12ee6ec71f8754ff3573711032b9b4d5a764ba84. I can duplicate it here, and the symptom is: pg_restore: creating TYPE float8range pg_restore: creating TYPE insenum

[HACKERS] Tablespaces in the data directory

2012-12-01 Thread Magnus Hagander
Someone just reported a problem when they had created a new tablespace inside the old data directory. I'm sure there can be other issues caused by this as well, but this is mainly a confusing scenario for people now. As there isn't (as far as I know at least) any actual *point* in creating a

Re: [HACKERS] Tablespaces in the data directory

2012-12-01 Thread Simon Riggs
On 1 December 2012 13:45, Magnus Hagander mag...@hagander.net wrote: Someone just reported a problem when they had created a new tablespace inside the old data directory. I'm sure there can be other issues caused by this as well, but this is mainly a confusing scenario for people now. As

[HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-01 Thread Simon Riggs
It's hard to know whether your tables will be locked for long periods when implementing DDL changes. The NOREWRITE option would cause an ERROR if the table would be rewritten by the command. This would allow testing to highlight long running statements before code hits production. -- Simon

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Bruce Momjian
On Sat, Dec 1, 2012 at 07:43:17AM -0500, Andrew Dunstan wrote: On 11/30/2012 11:10 PM, Tom Lane wrote: Some of the buildfarm members are failing the pg_upgrade regression test since commit 12ee6ec71f8754ff3573711032b9b4d5a764ba84. I can duplicate it here, and the symptom is: pg_restore:

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Bruce Momjian
On Sat, Dec 1, 2012 at 10:25:10AM -0500, Bruce Momjian wrote: On Sat, Dec 1, 2012 at 07:43:17AM -0500, Andrew Dunstan wrote: On 11/30/2012 11:10 PM, Tom Lane wrote: Some of the buildfarm members are failing the pg_upgrade regression test since commit

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Bruce Momjian
On Sat, Dec 1, 2012 at 10:41:06AM -0500, Bruce Momjian wrote: OK, I found the problem, and it isn't good. Our manual clearly says: ALTER TYPE ... ADD VALUE (the form that adds a new value to an enum type) cannot be executed inside a transaction block. This also means it can't

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Andres Freund
On 2012-12-01 10:55:09 -0500, Bruce Momjian wrote: On Sat, Dec 1, 2012 at 10:41:06AM -0500, Bruce Momjian wrote: OK, I found the problem, and it isn't good. Our manual clearly says: ALTER TYPE ... ADD VALUE (the form that adds a new value to an enum type) cannot be executed

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Andres Freund
On 2012-12-01 10:55:09 -0500, Bruce Momjian wrote: This does make me wonder why pg_restore supports --single-transaction if it has known failure cases (that are not documented in the pg_restore manual page, only in the ALTER TYPE manual page). Are users really going to know if their database

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: This does make me wonder why pg_restore supports --single-transaction if it has known failure cases (that are not documented in the pg_restore manual page, only in the ALTER TYPE manual page). AFAIR, the ADD VALUE path is only taken with --binary-upgrade,

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Bruce Momjian
On Sat, Dec 1, 2012 at 10:55:09AM -0500, Bruce Momjian wrote: Scratch that idea. By definition, no matter how we modify pg_dump or pg_restore, ALTER TYPE ... ADD VALUE is never going to be able to be run in a multi-statement transaction, so we have to certainly remove --single-transction,

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Bruce Momjian
On Sat, Dec 1, 2012 at 11:11:31AM -0500, Bruce Momjian wrote: Shame --- pg_upgrade performance was improving so steadily, I was hoping to see negative duration times soon. ;-) Is that the definition of optimism? :-) -- Bruce Momjian br...@momjian.ushttp://momjian.us

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-12-01 Thread Tom Lane
Amit Kapila amit.kap...@huawei.com writes: On Saturday, December 01, 2012 1:30 AM Tom Lane wrote: But having said that, it's not apparent to me why inventing SET PERSISTENT should require reserving PERSISTENT. The problem is due to RESET PERSISTENT configuration_variable Syntax. I think the

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Andres Freund
On 2012-12-01 17:03:03 +0100, Andres Freund wrote: Could we possibly allow adding enum values to a type which was just created in this transaction? That shouldn't be too hard. At least easier than providing the capability to pre-assign the next N oids... The attached patch does just that. Its

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Andres Freund
On 2012-12-01 17:36:20 +0100, Andres Freund wrote: On 2012-12-01 17:03:03 +0100, Andres Freund wrote: Could we possibly allow adding enum values to a type which was just created in this transaction? That shouldn't be too hard. At least easier than providing the capability to pre-assign

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-01 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: It's hard to know whether your tables will be locked for long periods when implementing DDL changes. The NOREWRITE option would cause an ERROR if the table would be rewritten by the command. This would allow testing to highlight long running

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-12-01 Thread Tom Lane
I wrote: But even if we can't make that work, it's not grounds for reserving PERSISTENT. Instead I'd be inclined to forget about RESET PERSISTENT syntax and use, say, SET PERSISTENT var_name TO DEFAULT to mean that. (BTW, I wonder what behavior that syntax has now in your patch.) In fact,

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: The attached patch does just that. Its *not* ready yet though, as it will be apparent for everyone who reads it ;) ISTM this sort of thing ought to be safe enough, though you probably need to insist both that the pg_type row's xmin be current XID and

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Andrew Dunstan
On 12/01/2012 11:38 AM, Andres Freund wrote: On 2012-12-01 17:36:20 +0100, Andres Freund wrote: On 2012-12-01 17:03:03 +0100, Andres Freund wrote: Could we possibly allow adding enum values to a type which was just created in this transaction? That shouldn't be too hard. At least easier than

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Bruce Momjian
On Sat, Dec 1, 2012 at 05:36:20PM +0100, Andres Freund wrote: On 2012-12-01 17:03:03 +0100, Andres Freund wrote: Could we possibly allow adding enum values to a type which was just created in this transaction? That shouldn't be too hard. At least easier than providing the capability to

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: Does this actually get you over the problem identified in the comment?: * We disallow this in transaction blocks, because we can't cope * with enum OID values getting into indexes and then having their * defining pg_enum entries go away. Why

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Andrew Dunstan
On 12/01/2012 12:06 PM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: Does this actually get you over the problem identified in the comment?: * We disallow this in transaction blocks, because we can't cope * with enum OID values getting into indexes and then having their

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Andres Freund
On 2012-12-01 12:00:46 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: The attached patch does just that. Its *not* ready yet though, as it will be apparent for everyone who reads it ;) ISTM this sort of thing ought to be safe enough, though you probably need to insist

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Andres Freund
On 2012-12-01 12:01:17 -0500, Andrew Dunstan wrote: On 12/01/2012 11:38 AM, Andres Freund wrote: On 2012-12-01 17:36:20 +0100, Andres Freund wrote: On 2012-12-01 17:03:03 +0100, Andres Freund wrote: Could we possibly allow adding enum values to a type which was just created in this

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2012-12-01 12:00:46 -0500, Tom Lane wrote: ISTM this sort of thing ought to be safe enough, though you probably need to insist both that the pg_type row's xmin be current XID and that it not be HEAP_UPDATED. I was concerned about updated rows

Re: [HACKERS] Tablespaces in the data directory

2012-12-01 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: Someone just reported a problem when they had created a new tablespace inside the old data directory. I'm sure there can be other issues caused by this as well, but this is mainly a confusing scenario for people now. As there isn't (as far as I

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-01 Thread Simon Riggs
On 1 December 2012 16:38, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: It's hard to know whether your tables will be locked for long periods when implementing DDL changes. The NOREWRITE option would cause an ERROR if the table would be rewritten by the

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Andres Freund
On 2012-12-01 12:14:37 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2012-12-01 12:00:46 -0500, Tom Lane wrote: ISTM this sort of thing ought to be safe enough, though you probably need to insist both that the pg_type row's xmin be current XID and that it not be

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-01 Thread Andres Freund
On 2012-12-01 18:27:08 +, Simon Riggs wrote: On 1 December 2012 16:38, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: It's hard to know whether your tables will be locked for long periods when implementing DDL changes. The NOREWRITE option would cause

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Bruce Momjian
On Sat, Dec 1, 2012 at 07:32:48PM +0100, Andres Freund wrote: On 2012-12-01 12:14:37 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2012-12-01 12:00:46 -0500, Tom Lane wrote: ISTM this sort of thing ought to be safe enough, though you probably need to insist

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Andres Freund
On 2012-12-01 13:43:44 -0500, Bruce Momjian wrote: On Sat, Dec 1, 2012 at 07:32:48PM +0100, Andres Freund wrote: On 2012-12-01 12:14:37 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2012-12-01 12:00:46 -0500, Tom Lane wrote: ISTM this sort of thing ought to

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2012-12-01 13:43:44 -0500, Bruce Momjian wrote: I believe this text in alter_type.sgml need updating: commandALTER TYPE ... ADD VALUE/ (the form that adds a new value to an enum type) cannot be executed inside a transaction block. I

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2012-12-01 12:14:37 -0500, Tom Lane wrote: It could do with some comments ;-) Hehe, yes. Hopefully this version has enough of that. Hm, maybe too many --- I don't really think it's necessary for utility.c to provide a redundant explanation of

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Bruce Momjian
On Sat, Dec 1, 2012 at 02:31:03PM -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2012-12-01 12:14:37 -0500, Tom Lane wrote: It could do with some comments ;-) Hehe, yes. Hopefully this version has enough of that. Hm, maybe too many --- I don't really think it's

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Andres Freund
On 2012-12-01 14:31:03 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2012-12-01 12:14:37 -0500, Tom Lane wrote: It could do with some comments ;-) Hehe, yes. Hopefully this version has enough of that. Hm, maybe too many --- I don't really think it's necessary for

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-01 Thread Josh Berkus
I'm not thrilled about inventing YA keyword for this. If you have a problem with that sort of scenario, why aren't you testing your DDL on a test server before you do it on production? *I* do test my DDL. However, there are literally hundreds of thousands of Rails, Django and Hibernate

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-01 Thread Andres Freund
On 2012-12-01 12:24:57 -0800, Josh Berkus wrote: I'm not thrilled about inventing YA keyword for this. If you have a problem with that sort of scenario, why aren't you testing your DDL on a test server before you do it on production? *I* do test my DDL. However, there are literally

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-01 Thread Josh Berkus
If you have a framework and you want to test for this you can just compare relfilenodes before/after. You're targeting the wrong users. This feature isn't for helping you or me. It's for helping the Rails users. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-01 Thread Andres Freund
On 2012-12-01 12:35:15 -0800, Josh Berkus wrote: If you have a framework and you want to test for this you can just compare relfilenodes before/after. You're targeting the wrong users. This feature isn't for helping you or me. It's for helping the Rails users. I am not targeting

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Andrew Dunstan
On 12/01/2012 02:34 PM, Bruce Momjian wrote: On Sat, Dec 1, 2012 at 02:31:03PM -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2012-12-01 12:14:37 -0500, Tom Lane wrote: It could do with some comments ;-) Hehe, yes. Hopefully this version has enough of that. Hm,

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-01 Thread Petr Jelinek
-Original Message- From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers- ow...@postgresql.org] On Behalf Of Andres Freund Sent: 01 December 2012 21:40 To: Josh Berkus Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] ALTER TABLE ... NOREWRITE option On 2012-12-01

Re: [HACKERS] [WIP] pg_ping utility

2012-12-01 Thread Phil Sorber
On Tue, Nov 27, 2012 at 9:43 AM, Phil Sorber p...@omniti.com wrote: On Tue, Nov 27, 2012 at 8:45 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Tue, Nov 27, 2012 at 7:35 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Peter Eisentraut pete...@gmx.net writes: Sure, PQping is

Re: [HACKERS] [PATCH] make -jN check fails / unset MAKEFLAGS in pg_regress.c

2012-12-01 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: Trivially updated patch attached. Applied, thanks. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] WIP: store additional info in GIN index

2012-12-01 Thread Tomas Vondra
On 18.11.2012 22:54, Alexander Korotkov wrote: Hackers, Patch completely changes storage in posting lists and leaf pages of posting trees. It uses varbyte encoding for BlockNumber and OffsetNumber. BlockNumber are stored incremental in page. Additionally one bit of OffsetNumber is reserved

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-01 Thread Noah Misch
On Sat, Dec 01, 2012 at 07:34:51PM +0100, Andres Freund wrote: On 2012-12-01 18:27:08 +, Simon Riggs wrote: On 1 December 2012 16:38, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: It's hard to know whether your tables will be locked for long periods

[HACKERS] proposal: separate databases for contrib module testing

2012-12-01 Thread Andrew Dunstan
I'd like to change the way we set the CONTRIB_TESTDB name for contrib modules. so that each module doesn't wipe out the previous module's test db. The reason is that this will let us test upgrading them using pg_upgrade much more easily. Not testing this is a significant hole in the pg_upgrade

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-12-01 Thread Amit kapila
On Saturday, December 01, 2012 10:00 PM Tom Lane wrote: Amit Kapila amit.kap...@huawei.com writes : On Saturday, December 01, 2012 1:30 AM Tom Lane wrote: But having said that, it's not apparent to me why inventing SET PERSISTENT should require reserving PERSISTENT. The problem is due to

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-12-01 Thread Amit kapila
On Saturday, December 01, 2012 10:15 PM Tom Lane wrote: I wrote: But even if we can't make that work, it's not grounds for reserving PERSISTENT. Instead I'd be inclined to forget about RESET PERSISTENT syntax and use, say, SET PERSISTENT var_name TO DEFAULT to mean that. (BTW, I wonder what