Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-12-01 Thread Amit Kapila
On Saturday, December 01, 2012 1:30 AM Tom Lane wrote: > Robert Haas writes: > > On Wed, Nov 28, 2012 at 9:47 AM, Amit kapila > wrote: > >> 5. PERSISTENT Keyword is added to the reserved keyword list. As it > was giving some errors given below while parsing gram.y > >> 15 shift/reduce conflicts .

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-12-01 Thread Albe Laurenz
Magnus Hagander wrote: >> On 30.11.2012 21:02, Andres Freund wrote: >>> There are workloads where its detrimental, but in general having it >>> default to on improver experience tremendously because getting conflicts >>> because of vacuum is rather confusing. >>> >>> In the workloads where it might

Re: [HACKERS] WIP: index support for regexp search

2012-12-01 Thread Erik Rijkers
On Fri, November 30, 2012 12:22, Alexander Korotkov wrote: > Hi! > > On Thu, Nov 29, 2012 at 12:58 PM, er wrote: > >> On Mon, November 26, 2012 20:49, Alexander Korotkov wrote: >> >> >> I ran the simple-minded tests against generated data (similar to the ones >> I did in January 2012). >> The prob

Re: [HACKERS] proposal: fix corner use case of variadic fuctions usage

2012-12-01 Thread Pavel Stehule
Hello > > Hi Pavel. > > I am trying to review this patch and on my work computer everything compiles > and tests perfectly. However, on my laptop, the regression tests don't pass > with "cache lookup failed for type XYZ" where XYZ is some number that does > not appear to be any type oid. > > I don

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Andrew Dunstan
On 11/30/2012 11:10 PM, Tom Lane wrote: Some of the buildfarm members are failing the pg_upgrade regression test since commit 12ee6ec71f8754ff3573711032b9b4d5a764ba84. I can duplicate it here, and the symptom is: pg_restore: creating TYPE float8range pg_restore: creating TYPE insenum pg_restor

[HACKERS] Tablespaces in the data directory

2012-12-01 Thread Magnus Hagander
Someone just reported a problem when they had created a new tablespace inside the old data directory. I'm sure there can be other issues caused by this as well, but this is mainly a confusing scenario for people now. As there isn't (as far as I know at least) any actual *point* in creating a table

Re: [HACKERS] Tablespaces in the data directory

2012-12-01 Thread Simon Riggs
On 1 December 2012 13:45, Magnus Hagander wrote: > Someone just reported a problem when they had created a new tablespace > inside the old data directory. I'm sure there can be other issues > caused by this as well, but this is mainly a confusing scenario for > people now. > > As there isn't (as f

[HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-01 Thread Simon Riggs
It's hard to know whether your tables will be locked for long periods when implementing DDL changes. The NOREWRITE option would cause an ERROR if the table would be rewritten by the command. This would allow testing to highlight long running statements before code hits production. -- Simon Rig

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Bruce Momjian
On Sat, Dec 1, 2012 at 07:43:17AM -0500, Andrew Dunstan wrote: > > On 11/30/2012 11:10 PM, Tom Lane wrote: > >Some of the buildfarm members are failing the pg_upgrade regression test > >since commit 12ee6ec71f8754ff3573711032b9b4d5a764ba84. I can duplicate > >it here, and the symptom is: > > > >

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Bruce Momjian
On Sat, Dec 1, 2012 at 10:25:10AM -0500, Bruce Momjian wrote: > On Sat, Dec 1, 2012 at 07:43:17AM -0500, Andrew Dunstan wrote: > > > > On 11/30/2012 11:10 PM, Tom Lane wrote: > > >Some of the buildfarm members are failing the pg_upgrade regression test > > >since commit 12ee6ec71f8754ff357371103

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Bruce Momjian
On Sat, Dec 1, 2012 at 10:41:06AM -0500, Bruce Momjian wrote: > OK, I found the problem, and it isn't good. Our manual clearly says: > > ALTER TYPE ... ADD VALUE (the form that adds a new value > to an enum type) cannot be executed inside a transaction block. > > This also means it

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Andres Freund
On 2012-12-01 10:55:09 -0500, Bruce Momjian wrote: > On Sat, Dec 1, 2012 at 10:41:06AM -0500, Bruce Momjian wrote: > > OK, I found the problem, and it isn't good. Our manual clearly says: > > > > ALTER TYPE ... ADD VALUE (the form that adds a new value > > to an enum type) cannot be execu

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Andres Freund
On 2012-12-01 10:55:09 -0500, Bruce Momjian wrote: > This does make me wonder why pg_restore supports --single-transaction if > it has known failure cases (that are not documented in the pg_restore > manual page, only in the ALTER TYPE manual page). Are users really > going to know if their databa

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Tom Lane
Bruce Momjian writes: > This does make me wonder why pg_restore supports --single-transaction if > it has known failure cases (that are not documented in the pg_restore > manual page, only in the ALTER TYPE manual page). AFAIR, the ADD VALUE path is only taken with --binary-upgrade, which is just

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Bruce Momjian
On Sat, Dec 1, 2012 at 10:55:09AM -0500, Bruce Momjian wrote: > Scratch that idea. By definition, no matter how we modify pg_dump or > pg_restore, ALTER TYPE ... ADD VALUE is never going to be able to be run > in a multi-statement transaction, so we have to certainly remove > --single-transction,

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Bruce Momjian
On Sat, Dec 1, 2012 at 11:11:31AM -0500, Bruce Momjian wrote: > Shame --- pg_upgrade performance was improving so steadily, I was hoping > to see negative duration times soon. ;-) Is that the definition of optimism? :-) -- Bruce Momjian http://momjian.us EnterpriseDB

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-12-01 Thread Tom Lane
Amit Kapila writes: > On Saturday, December 01, 2012 1:30 AM Tom Lane wrote: >> But having said that, it's not apparent to me why inventing SET >> PERSISTENT should require reserving PERSISTENT. > The problem is due to RESET PERSISTENT configuration_variable Syntax. > I think the reason is that c

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Andres Freund
On 2012-12-01 17:03:03 +0100, Andres Freund wrote: > Could we possibly allow adding enum values to a type which was just created in > this transaction? That shouldn't be too hard. At least easier than providing > the capability to pre-assign the next N oids... The attached patch does just that. It

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Andres Freund
On 2012-12-01 17:36:20 +0100, Andres Freund wrote: > On 2012-12-01 17:03:03 +0100, Andres Freund wrote: > > Could we possibly allow adding enum values to a type which was just created > > in > > this transaction? That shouldn't be too hard. At least easier than providing > > the capability to pre-

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-01 Thread Tom Lane
Simon Riggs writes: > It's hard to know whether your tables will be locked for long periods > when implementing DDL changes. > The NOREWRITE option would cause an ERROR if the table would be > rewritten by the command. > This would allow testing to highlight long running statements before > code

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-12-01 Thread Tom Lane
I wrote: > But even if we can't make that work, it's not grounds for reserving > PERSISTENT. Instead I'd be inclined to forget about "RESET PERSISTENT" > syntax and use, say, SET PERSISTENT var_name TO DEFAULT to mean that. > (BTW, I wonder what behavior that syntax has now in your patch.) In fac

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Tom Lane
Andres Freund writes: >> The attached patch does just that. Its *not* ready yet though, as it >> will be apparent for everyone who reads it ;) ISTM this sort of thing ought to be safe enough, though you probably need to insist both that the pg_type row's xmin be current XID and that it not be HEA

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Andrew Dunstan
On 12/01/2012 11:38 AM, Andres Freund wrote: On 2012-12-01 17:36:20 +0100, Andres Freund wrote: On 2012-12-01 17:03:03 +0100, Andres Freund wrote: Could we possibly allow adding enum values to a type which was just created in this transaction? That shouldn't be too hard. At least easier than p

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Bruce Momjian
On Sat, Dec 1, 2012 at 05:36:20PM +0100, Andres Freund wrote: > On 2012-12-01 17:03:03 +0100, Andres Freund wrote: > > Could we possibly allow adding enum values to a type which was just created > > in > > this transaction? That shouldn't be too hard. At least easier than providing > > the capabi

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Tom Lane
Andrew Dunstan writes: > Does this actually get you over the problem identified in the comment?: > * We disallow this in transaction blocks, because we can't cope > * with enum OID values getting into indexes and then having their > * defining pg_enum entries go away. Why wouldn't it? If

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Andrew Dunstan
On 12/01/2012 12:06 PM, Tom Lane wrote: Andrew Dunstan writes: Does this actually get you over the problem identified in the comment?: * We disallow this in transaction blocks, because we can't cope * with enum OID values getting into indexes and then having their * defining pg_enum e

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Andres Freund
On 2012-12-01 12:00:46 -0500, Tom Lane wrote: > Andres Freund writes: > >> The attached patch does just that. Its *not* ready yet though, as it > >> will be apparent for everyone who reads it ;) > > ISTM this sort of thing ought to be safe enough, though you probably > need to insist both that the

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Andres Freund
On 2012-12-01 12:01:17 -0500, Andrew Dunstan wrote: > > On 12/01/2012 11:38 AM, Andres Freund wrote: > >On 2012-12-01 17:36:20 +0100, Andres Freund wrote: > >>On 2012-12-01 17:03:03 +0100, Andres Freund wrote: > >>>Could we possibly allow adding enum values to a type which was just > >>>created in

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Tom Lane
Andres Freund writes: > On 2012-12-01 12:00:46 -0500, Tom Lane wrote: >> ISTM this sort of thing ought to be safe enough, though you probably >> need to insist both that the pg_type row's xmin be current XID and >> that it not be HEAP_UPDATED. > I was concerned about updated rows but forgot about

Re: [HACKERS] Tablespaces in the data directory

2012-12-01 Thread Tom Lane
Magnus Hagander writes: > Someone just reported a problem when they had created a new tablespace > inside the old data directory. I'm sure there can be other issues > caused by this as well, but this is mainly a confusing scenario for > people now. > As there isn't (as far as I know at least) any

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-01 Thread Simon Riggs
On 1 December 2012 16:38, Tom Lane wrote: > Simon Riggs writes: >> It's hard to know whether your tables will be locked for long periods >> when implementing DDL changes. > >> The NOREWRITE option would cause an ERROR if the table would be >> rewritten by the command. > >> This would allow testin

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Andres Freund
On 2012-12-01 12:14:37 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2012-12-01 12:00:46 -0500, Tom Lane wrote: > >> ISTM this sort of thing ought to be safe enough, though you probably > >> need to insist both that the pg_type row's xmin be current XID and > >> that it not be HEAP_UPDATED

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-01 Thread Andres Freund
On 2012-12-01 18:27:08 +, Simon Riggs wrote: > On 1 December 2012 16:38, Tom Lane wrote: > > Simon Riggs writes: > >> It's hard to know whether your tables will be locked for long periods > >> when implementing DDL changes. > > > >> The NOREWRITE option would cause an ERROR if the table would

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Bruce Momjian
On Sat, Dec 1, 2012 at 07:32:48PM +0100, Andres Freund wrote: > On 2012-12-01 12:14:37 -0500, Tom Lane wrote: > > Andres Freund writes: > > > On 2012-12-01 12:00:46 -0500, Tom Lane wrote: > > >> ISTM this sort of thing ought to be safe enough, though you probably > > >> need to insist both that t

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Andres Freund
On 2012-12-01 13:43:44 -0500, Bruce Momjian wrote: > On Sat, Dec 1, 2012 at 07:32:48PM +0100, Andres Freund wrote: > > On 2012-12-01 12:14:37 -0500, Tom Lane wrote: > > > Andres Freund writes: > > > > On 2012-12-01 12:00:46 -0500, Tom Lane wrote: > > > >> ISTM this sort of thing ought to be safe

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Tom Lane
Andres Freund writes: > On 2012-12-01 13:43:44 -0500, Bruce Momjian wrote: >> I believe this text in alter_type.sgml need updating: >> >> ALTER TYPE ... ADD VALUE (the form that adds a new value to an >> enum type) cannot be executed inside a transaction block. > I purposefully didn't change tha

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Tom Lane
Andres Freund writes: > On 2012-12-01 12:14:37 -0500, Tom Lane wrote: >> It could do with some comments ;-) > Hehe, yes. Hopefully this version has enough of that. Hm, maybe too many --- I don't really think it's necessary for utility.c to provide a redundant explanation of what's happening. Co

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Bruce Momjian
On Sat, Dec 1, 2012 at 02:31:03PM -0500, Tom Lane wrote: > Andres Freund writes: > > On 2012-12-01 12:14:37 -0500, Tom Lane wrote: > >> It could do with some comments ;-) > > > Hehe, yes. Hopefully this version has enough of that. > > Hm, maybe too many --- I don't really think it's necessary f

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Andres Freund
On 2012-12-01 14:31:03 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2012-12-01 12:14:37 -0500, Tom Lane wrote: > >> It could do with some comments ;-) > > > Hehe, yes. Hopefully this version has enough of that. > > Hm, maybe too many --- I don't really think it's necessary for utility.c >

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-01 Thread Josh Berkus
>> I'm not thrilled about inventing YA keyword for this. If you have a >> problem with that sort of scenario, why aren't you testing your DDL >> on a test server before you do it on production? *I* do test my DDL. However, there are literally hundreds of thousands of Rails, Django and Hibernate

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-01 Thread Andres Freund
On 2012-12-01 12:24:57 -0800, Josh Berkus wrote: > > >> I'm not thrilled about inventing YA keyword for this. If you have a > >> problem with that sort of scenario, why aren't you testing your DDL > >> on a test server before you do it on production? > > *I* do test my DDL. However, there are l

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-01 Thread Josh Berkus
> If you have a framework and you want to test for this you can just > compare relfilenodes before/after. You're targeting the wrong users. This feature isn't for helping you or me. It's for helping the Rails users. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via p

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-01 Thread Andres Freund
On 2012-12-01 12:35:15 -0800, Josh Berkus wrote: > > > If you have a framework and you want to test for this you can just > > compare relfilenodes before/after. > > You're targeting the wrong users. This feature isn't for helping you or > me. It's for helping the Rails users. I am not targeting

Re: [HACKERS] --single-transaction hack to pg_upgrade does not work

2012-12-01 Thread Andrew Dunstan
On 12/01/2012 02:34 PM, Bruce Momjian wrote: On Sat, Dec 1, 2012 at 02:31:03PM -0500, Tom Lane wrote: Andres Freund writes: On 2012-12-01 12:14:37 -0500, Tom Lane wrote: It could do with some comments ;-) Hehe, yes. Hopefully this version has enough of that. Hm, maybe too many --- I don't

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-01 Thread Petr Jelinek
> -Original Message- > From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers- > ow...@postgresql.org] On Behalf Of Andres Freund > Sent: 01 December 2012 21:40 > To: Josh Berkus > Cc: pgsql-hackers@postgresql.org > Subject: Re: [HACKERS] ALTER TABLE ... NOREWRITE option > > On 201

Re: [HACKERS] [WIP] pg_ping utility

2012-12-01 Thread Phil Sorber
On Tue, Nov 27, 2012 at 9:43 AM, Phil Sorber wrote: > On Tue, Nov 27, 2012 at 8:45 AM, Michael Paquier > wrote: >> >> >> On Tue, Nov 27, 2012 at 7:35 PM, Dimitri Fontaine >> wrote: >>> >>> Peter Eisentraut writes: >>> > Sure, PQping is useful for this very specific use case of seeing whether >>

Re: [HACKERS] [PATCH] make -jN check fails / unset MAKEFLAGS in pg_regress.c

2012-12-01 Thread Tom Lane
Andres Freund writes: > Trivially updated patch attached. Applied, thanks. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WIP: store additional info in GIN index

2012-12-01 Thread Tomas Vondra
On 18.11.2012 22:54, Alexander Korotkov wrote: > Hackers, > > Patch completely changes storage in posting lists and leaf pages of > posting trees. It uses varbyte encoding for BlockNumber and > OffsetNumber. BlockNumber are stored incremental in page. Additionally > one bit of OffsetNumber is rese

Re: [HACKERS] ALTER TABLE ... NOREWRITE option

2012-12-01 Thread Noah Misch
On Sat, Dec 01, 2012 at 07:34:51PM +0100, Andres Freund wrote: > On 2012-12-01 18:27:08 +, Simon Riggs wrote: > > On 1 December 2012 16:38, Tom Lane wrote: > > > Simon Riggs writes: > > >> It's hard to know whether your tables will be locked for long periods > > >> when implementing DDL chang

[HACKERS] proposal: separate databases for contrib module testing

2012-12-01 Thread Andrew Dunstan
I'd like to change the way we set the CONTRIB_TESTDB name for contrib modules. so that each module doesn't wipe out the previous module's test db. The reason is that this will let us test upgrading them using pg_upgrade much more easily. Not testing this is a significant hole in the pg_upgrade

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-12-01 Thread Amit kapila
On Saturday, December 01, 2012 10:00 PM Tom Lane wrote: Amit Kapila writes : > On Saturday, December 01, 2012 1:30 AM Tom Lane wrote: >>> But having said that, it's not apparent to me why inventing SET >>> PERSISTENT should require reserving PERSISTENT. >> The problem is due to RESET PERSISTENT c

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-12-01 Thread Amit kapila
On Saturday, December 01, 2012 10:15 PM Tom Lane wrote: I wrote: >> But even if we can't make that work, it's not grounds for reserving >> PERSISTENT. Instead I'd be inclined to forget about "RESET PERSISTENT" >> syntax and use, say, SET PERSISTENT var_name TO DEFAULT to mean that. >> (BTW, I won