Re: [HACKERS] Move unused buffers to freelist

2013-08-05 Thread Amit Kapila
On Friday, June 28, 2013 6:20 PM Robert Haas wrote: > On Fri, Jun 28, 2013 at 12:52 AM, Amit Kapila > wrote: > > Currently it wakes up based on bgwriterdelay config parameter which > is by > > default 200ms, so you means we should > > think of waking up bgwriter based on allocations and number of

[HACKERS] System catalog vacuum issues

2013-08-05 Thread Vlad Arkhipov
Hello, We are suffering from a long-standing issue with autovacuuming/vacuuming system catalogs on the production server. We are actively using temporary tables in the legacy application, so system catalogs grows unbounded in time. Autovacuum does not remove dead tuples and neither do the man

Re: [HACKERS] Moving 'hot' pages from buffer pool to heap

2013-08-05 Thread Atri Sharma
Sent from my iPad On 06-Aug-2013, at 1:57, Tom Lane wrote: > Atri Sharma writes: >> Just experimenting though.I was thinking of scenarios where a page is pinned >> for long period of time.My concern was that it would lead to blocking of a >> buffer pool slot for that entire duration. The id

Re: [HACKERS] Should we remove "not fast" promotion at all?

2013-08-05 Thread Michael Paquier
On Tue, Aug 6, 2013 at 12:52 PM, Andres Freund wrote: > > > Fujii Masao schrieb: >>On Tue, Aug 6, 2013 at 11:40 AM, Andres Freund >>wrote: >>> Hi, >>> >>> On 2013-08-06 03:24:58 +0900, Fujii Masao wrote: Hi all, We discussed the $SUBJECT in the following threads: >>http://www

Re: [HACKERS] Should we remove "not fast" promotion at all?

2013-08-05 Thread Tom Lane
Fujii Masao writes: > On Tue, Aug 6, 2013 at 11:40 AM, Andres Freund wrote: >> FWIW I'd rather keep plain promotion for a release or two. TBH, I have a >> bit of trust issues regarding the new method, and I'd like to be able to >> test potential issues against a stock postgres by doing a normal i

Re: [HACKERS] Should we remove "not fast" promotion at all?

2013-08-05 Thread Michael Paquier
On Tue, Aug 6, 2013 at 12:41 PM, Fujii Masao wrote: >> - >> + unlink(PROMOTE_SIGNAL_FILE); >> Wouldn't it make sense to keep the call to stat() to check the file >> status before unlinking it? > > Why do we need to check the existence of the file before removing it > here? Forget wha

Re: [HACKERS] Should we remove "not fast" promotion at all?

2013-08-05 Thread Andres Freund
Fujii Masao schrieb: >On Tue, Aug 6, 2013 at 11:40 AM, Andres Freund >wrote: >> Hi, >> >> On 2013-08-06 03:24:58 +0900, Fujii Masao wrote: >>> Hi all, >>> >>> We discussed the $SUBJECT in the following threads: >>> >http://www.postgresql.org/message-id/ca+tgmozbr+wl8e7mf_krp6fy4fd2pmr11tpiuyjmf

Re: [HACKERS] Should we remove "not fast" promotion at all?

2013-08-05 Thread Fujii Masao
On Tue, Aug 6, 2013 at 11:40 AM, Andres Freund wrote: > Hi, > > On 2013-08-06 03:24:58 +0900, Fujii Masao wrote: >> Hi all, >> >> We discussed the $SUBJECT in the following threads: >> http://www.postgresql.org/message-id/ca+tgmozbr+wl8e7mf_krp6fy4fd2pmr11tpiuyjmfx_vtg1...@mail.gmail.com >> http:/

Re: [HACKERS] Should we remove "not fast" promotion at all?

2013-08-05 Thread Fujii Masao
On Tue, Aug 6, 2013 at 11:20 AM, Michael Paquier wrote: > On Tue, Aug 6, 2013 at 3:24 AM, Fujii Masao wrote: >> Hi all, >> >> We discussed the $SUBJECT in the following threads: >> http://www.postgresql.org/message-id/ca+tgmozbr+wl8e7mf_krp6fy4fd2pmr11tpiuyjmfx_vtg1...@mail.gmail.com >> http://ww

Re: [HACKERS] HeapTupleSatisfiesDirty fails to test HEAP_XMAX_IS_LOCKED_ONLY for TransactionIdIsInProgress(...)

2013-08-05 Thread Craig Ringer
On 08/06/2013 02:37 AM, Robert Haas wrote: > I haven't read this particular test, but I do think we could get a lot > of mileage out of applying the isolation tester stuff to more things, > and am generally in favor of that. It has the advantages of (1) > allowing tests that require more than one

Re: [HACKERS] mvcc catalo gsnapshots and TopTransactionContext

2013-08-05 Thread Andres Freund
On 2013-08-05 13:09:31 -0400, Noah Misch wrote: > On Fri, Jul 12, 2013 at 11:42:23AM +0200, Andres Freund wrote: > > On 2013-07-11 15:09:45 -0400, Tom Lane wrote: > > > It never has been, and never will be, allowed to call the catcache code > > > without being in a transaction. What do you think w

Re: [HACKERS] HeapTupleSatisfiesDirty fails to test HEAP_XMAX_IS_LOCKED_ONLY for TransactionIdIsInProgress(...)

2013-08-05 Thread Andres Freund
On 2013-08-06 04:48:09 +0200, Andres Freund wrote: > On 2013-08-05 14:37:34 -0400, Robert Haas wrote: > > On Fri, Aug 2, 2013 at 5:25 PM, Alvaro Herrera > > wrote: > > >> The attached test case runs under isolationtester to exersise the > > >> problem. I've tested it against 9.2, 9.3, and HEAD, b

Re: [HACKERS] make --enable-depend the default

2013-08-05 Thread Tom Lane
Andres Freund writes: > On 2013-08-05 17:59:39 -0400, Robert Haas wrote: >> Yeah, I have alias dev-configure='./configure >> --appropriate-stuff-for-this-machine' on every system where I work on >> PG. I imagine others do something similar. > I have that on machines I work on regularly as well,

Re: [HACKERS] HeapTupleSatisfiesDirty fails to test HEAP_XMAX_IS_LOCKED_ONLY for TransactionIdIsInProgress(...)

2013-08-05 Thread Andres Freund
On 2013-08-05 14:37:34 -0400, Robert Haas wrote: > On Fri, Aug 2, 2013 at 5:25 PM, Alvaro Herrera > wrote: > >> The attached test case runs under isolationtester to exersise the > >> problem. I've tested it against 9.2, 9.3, and HEAD, but Andres looked > >> over the code and says the underlying b

Re: [HACKERS] Should we remove "not fast" promotion at all?

2013-08-05 Thread Andres Freund
Hi, On 2013-08-06 03:24:58 +0900, Fujii Masao wrote: > Hi all, > > We discussed the $SUBJECT in the following threads: > http://www.postgresql.org/message-id/ca+tgmozbr+wl8e7mf_krp6fy4fd2pmr11tpiuyjmfx_vtg1...@mail.gmail.com > http://www.postgresql.org/message-id/CAHGQGwEBUvgcx8X+Z0Hh+VdwYcJ8KCuR

Re: [HACKERS] Should we remove "not fast" promotion at all?

2013-08-05 Thread Michael Paquier
On Tue, Aug 6, 2013 at 3:24 AM, Fujii Masao wrote: > Hi all, > > We discussed the $SUBJECT in the following threads: > http://www.postgresql.org/message-id/ca+tgmozbr+wl8e7mf_krp6fy4fd2pmr11tpiuyjmfx_vtg1...@mail.gmail.com > http://www.postgresql.org/message-id/CAHGQGwEBUvgcx8X+Z0Hh+VdwYcJ8KCuRuLt

Re: [HACKERS] Regarding BGworkers

2013-08-05 Thread Michael Paquier
On Fri, Aug 2, 2013 at 1:40 PM, Alvaro Herrera wrote: > That seems more mess than just keeping that function in postmaster.c. > I agree with moving the other one. Please find attached a patch for that can be applied on master branch. do_start_bgworker is renamed to StartBackgroundWorker and moved

Re: [HACKERS] File-per-GUC WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Josh Berkus
On 08/05/2013 05:12 PM, Andres Freund wrote: > I don't think we're designing a feature that's supposed to be used under > heavy concurrency here. If you have users/tools doing conflicting > actions as superusers you need to solve that by social means, not by > technical ones. +1 -- Josh Berkus P

Re: [HACKERS] make --enable-depend the default

2013-08-05 Thread Michael Paquier
On Tue, Aug 6, 2013 at 6:59 AM, Robert Haas wrote: > On Mon, Aug 5, 2013 at 4:18 PM, Peter Eisentraut wrote: > > On Thu, 2013-08-01 at 11:10 +0200, Andres Freund wrote: > >> People, including me, every now and then forget to pass --enable-depend > >> to configure (when not using my own environme

Re: [HACKERS] File-per-GUC WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Andres Freund
On 2013-08-05 13:56:40 -0400, Tom Lane wrote: > Josh Berkus writes: > > I'll also point out that some of our settings only really "work" in > > combinations of two or more settings. For example, one doesn't want to > > set archive_mode = on unless one is setting archive_command as well. > > And g

Re: [HACKERS] make --enable-depend the default

2013-08-05 Thread Andres Freund
On 2013-08-05 17:59:39 -0400, Robert Haas wrote: > On Mon, Aug 5, 2013 at 4:18 PM, Peter Eisentraut wrote: > > On Thu, 2013-08-01 at 11:10 +0200, Andres Freund wrote: > >> People, including me, every now and then forget to pass --enable-depend > >> to configure (when not using my own environment).

Re: [HACKERS] Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Greg Stark
On Mon, Aug 5, 2013 at 6:16 PM, Tom Lane wrote: > If it's file-per-GUC, then when two sessions try to write different GUCs, > there is no conflict Well it's not that there's no conflict. It's just that we don't detect the conflict. If you raise shared_buffers and I calculate the optimal work_mem

Re: [HACKERS] Unsafe GUCs and ALTER SYSTEM WAS: Re: ALTER SYSTEM SET

2013-08-05 Thread Greg Stark
On Mon, Aug 5, 2013 at 7:02 PM, Josh Berkus wrote: > Nope. ALTER SYSTEM, from my POV, is mainly for folks who *don't* use > Puppet/Chef/whatever. Here's where I see ALTER SYSTEM being useful: > > * invididually managed servers with out centralized management (i.e. one > DBA, one server). Well a

Re: [HACKERS] don't own lock of type?

2013-08-05 Thread Joshua D. Drake
On 08/05/2013 02:58 PM, Robert Haas wrote: On Mon, Aug 5, 2013 at 3:37 PM, Joshua D. Drake wrote: It means there's a bug. Either in PostgreSQL, or some loadable module you're using. I am getting this rather frequently, I will check modules but I don't believe we are running any. We fixed

Re: [HACKERS] make --enable-depend the default

2013-08-05 Thread Robert Haas
On Mon, Aug 5, 2013 at 4:18 PM, Peter Eisentraut wrote: > On Thu, 2013-08-01 at 11:10 +0200, Andres Freund wrote: >> People, including me, every now and then forget to pass --enable-depend >> to configure (when not using my own environment). Which then leads to >> strange errors that cost time to

Re: [HACKERS] don't own lock of type?

2013-08-05 Thread Robert Haas
On Mon, Aug 5, 2013 at 3:37 PM, Joshua D. Drake wrote: >> It means there's a bug. Either in PostgreSQL, or some loadable module >> you're using. > > I am getting this rather frequently, I will check modules but I don't > believe we are running any. We fixed a bug in this area a while back. Are

Re: [HACKERS] Disabling ALTER SYSTEM SET WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters

2013-08-05 Thread Bruce Momjian
On Mon, Aug 5, 2013 at 03:53:01PM -0400, Alvaro Herrera wrote: > The other issue is that currently you can only edit a server's config if > you are logged in to it. If we permit SQL-level access to that, and > somebody who doesn't have access to edit the files blocks themselves > out, there is no

Re: [HACKERS] File-per-GUC WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Bruce Momjian
On Mon, Aug 5, 2013 at 01:56:40PM -0400, Tom Lane wrote: > Josh Berkus writes: > > I'll also point out that some of our settings only really "work" in > > combinations of two or more settings. For example, one doesn't want to > > set archive_mode = on unless one is setting archive_command as wel

Re: [HACKERS] Disabling ALTER SYSTEM SET WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters

2013-08-05 Thread Josh Berkus
Stefan, > thinking more about that - is there _ANY_ prerequisite of an application > that can be completely reconfigured over a remote access protocol and > solved the reliability and security challenges of that to a reasonable > degree? Good question! I also think that, given the issues raised

Re: [HACKERS] Disabling ALTER SYSTEM SET WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters

2013-08-05 Thread Stefan Kaltenbrunner
On 08/05/2013 09:53 PM, Alvaro Herrera wrote: > Tom Lane escribió: > >> What Josh seems to be concerned with in this thread is the question of >> whether we should support an installation *policy decision* not to allow >> ALTER SYSTEM SET. Not because a particular set of parameters is broken, >>

Re: [HACKERS] Disabling ALTER SYSTEM SET WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters

2013-08-05 Thread Stefan Kaltenbrunner
On 08/05/2013 10:18 PM, Dimitri Fontaine wrote: > Hi, > > I'm now completely lost in the current threads. Is there a single valid > use case for the feature we're trying to design? Who is the target > audience of this patch? wonder about that myself... > > Josh Berkus writes: >> I don't see t

Re: [HACKERS] CREATE EVENT TRIGGER syntax

2013-08-05 Thread Dimitri Fontaine
Bruce Momjian writes: > So do we want to keep that "AND" in the 9.3beta and 9.4 documentation? The grammar as in gram.y still allows the AND form, and I think we're used to maintain documentation that matches the code here. So I think it makes sense to remove both capabilities as we failed to del

Re: [HACKERS] Disabling ALTER SYSTEM SET WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters

2013-08-05 Thread Rodrigo Gonzalez
On Mon, 5 Aug 2013 15:53:01 -0400 Alvaro Herrera wrote: > The other issue is that currently you can only edit a server's config > if you are logged in to it. If we permit SQL-level access to that, > and somebody who doesn't have access to edit the files blocks > themselves out, there is no way f

Re: [HACKERS] Moving 'hot' pages from buffer pool to heap

2013-08-05 Thread Tom Lane
Atri Sharma writes: > Just experimenting though.I was thinking of scenarios where a page is pinned > for long period of time.My concern was that it would lead to blocking of a > buffer pool slot for that entire duration. The idea is to allocate a separate > data structure for such hot pages in

Re: [HACKERS] Disabling ALTER SYSTEM SET WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters

2013-08-05 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane escribió: >> What Josh seems to be concerned with in this thread is the question of >> whether we should support an installation *policy decision* not to allow >> ALTER SYSTEM SET. Not because a particular set of parameters is broken, >> but just because somebody

Re: [HACKERS] Autovacuum different in 9.2.4?

2013-08-05 Thread Kevin Grittner
Alvaro Herrera wrote: > Joshua D. Drake escribió: >> That is what is confusing me, I could be cracked but messages like these: >> >> automatic vacuum of table "pg_catalog.pg_attribute": could not >> (re)acquire exclusive lock for truncate scan >> >> Seem to be new? > > Yeah, those are new. Yeah,

Re: [HACKERS] Disabling ALTER SYSTEM SET WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters

2013-08-05 Thread Dimitri Fontaine
Hi, I'm now completely lost in the current threads. Is there a single valid use case for the feature we're trying to design? Who is the target audience of this patch? Josh Berkus writes: > I don't see this as a solution at all. "Mr. Sysadmin, we've given the > DBAs a new tool which allows them

Re: [HACKERS] make --enable-depend the default

2013-08-05 Thread Peter Eisentraut
On Thu, 2013-08-01 at 11:10 +0200, Andres Freund wrote: > People, including me, every now and then forget to pass --enable-depend > to configure (when not using my own environment). Which then leads to > strange errors that cost time to track down... One argument against that is that we only suppo

Re: [HACKERS] Moving 'hot' pages from buffer pool to heap

2013-08-05 Thread Atri Sharma
Sent from my iPad On 06-Aug-2013, at 1:24, Jeff Janes wrote: > On Mon, Aug 5, 2013 at 12:36 PM, Atri Sharma wrote: >> Hi all, >> >> I was experimenting with the idea of moving hot buffer pages from the >> buffer pool to heap, > > Which heap do you mean here? Alas, half the data structures u

Re: [HACKERS] Moving 'hot' pages from buffer pool to heap

2013-08-05 Thread Atri Sharma
Sent from my iPad On 06-Aug-2013, at 1:14, Tom Lane wrote: > Atri Sharma writes: >> I was experimenting with the idea of moving hot buffer pages from the >> buffer pool to heap, thus allowing for normal removal of the hot >> buffer page from the buffer pool and freeing the corresponding buffe

Re: [HACKERS] [9.4 CF 1]Commitfest ... over!

2013-08-05 Thread Robert Haas
On Sun, Aug 4, 2013 at 11:41 PM, Amit Kapila wrote: > First of all Thank you very much for running a wonderful Commit Fest. Yes. Thanks, Josh. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@po

Re: [HACKERS] pg_basebackup vs. Windows and tablespaces

2013-08-05 Thread Noah Misch
On Thu, Aug 01, 2013 at 01:04:42PM -0400, Andrew Dunstan wrote: > On 08/01/2013 12:15 PM, Noah Misch wrote: >> 1. Include in the base backup a file listing symbolic links/junction points, >> then have archive recovery recreate them. This file would be managed like >> the >> backup label file; exc

Re: [HACKERS] pg_basebackup vs. Windows and tablespaces

2013-08-05 Thread Noah Misch
On Thu, Aug 01, 2013 at 06:44:41PM +0200, Dimitri Fontaine wrote: > Noah Misch writes: > > 2. Add a pg_basebackup option like "--destdir" or "--sysroot", meaningful > > only > > with -Fp; tablespace backups will be stored relative to it. So if the > > actual > > tablespace path is c:/foo, --des

Re: [HACKERS] Moving 'hot' pages from buffer pool to heap

2013-08-05 Thread Jeff Janes
On Mon, Aug 5, 2013 at 12:36 PM, Atri Sharma wrote: > Hi all, > > I was experimenting with the idea of moving hot buffer pages from the > buffer pool to heap, Which heap do you mean here? Alas, half the data structures used in CS are called "heap". I can't think of any of them that are good cand

Re: [HACKERS] Disabling ALTER SYSTEM SET WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters

2013-08-05 Thread Alvaro Herrera
Tom Lane escribió: > What Josh seems to be concerned with in this thread is the question of > whether we should support an installation *policy decision* not to allow > ALTER SYSTEM SET. Not because a particular set of parameters is broken, > but just because somebody is afraid the DBA might brea

Re: [HACKERS] Autovacuum different in 9.2.4?

2013-08-05 Thread Alvaro Herrera
Joshua D. Drake escribió: > On 08/05/2013 12:13 PM, Jeff Janes wrote: > > >There should be no change in usage, unless you were taking some heroic > >methods to overcome the problems and can now discontinue them. > > That is what is confusing me, I could be cracked but messages like these: > > au

Re: [HACKERS] Moving 'hot' pages from buffer pool to heap

2013-08-05 Thread Tom Lane
Atri Sharma writes: > I was experimenting with the idea of moving hot buffer pages from the > buffer pool to heap, thus allowing for normal removal of the hot > buffer page from the buffer pool and freeing the corresponding buffer > pool slot. Uh ... what? Why in the world would you want to forc

Re: [HACKERS] Disabling ALTER SYSTEM SET WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters

2013-08-05 Thread Tom Lane
Bruce Momjian writes: > Well, can you assume that if you have a problem with one of your ALTER > SYSTEM SET commands, that disabling _all_ of them is going to get you a > running system? I question that, e.g. port. With postgresql.conf, you > can modify the bad entry, but how would that happen w

Re: [HACKERS] don't own lock of type?

2013-08-05 Thread Joshua D. Drake
On 08/05/2013 11:38 AM, Robert Haas wrote: On Mon, Aug 5, 2013 at 2:32 PM, Joshua D. Drake wrote: Hello, What exactly causes this? WARNING: you don't own a lock of type ExclusiveLock Does this mean the user calling the lock doesn't own the object? It means there's a bug. Either in Post

[HACKERS] Moving 'hot' pages from buffer pool to heap

2013-08-05 Thread Atri Sharma
Hi all, I was experimenting with the idea of moving hot buffer pages from the buffer pool to heap, thus allowing for normal removal of the hot buffer page from the buffer pool and freeing the corresponding buffer pool slot. This shouldnt be too hard to implement, we just need to keep track of the

Re: [HACKERS] Autovacuum different in 9.2.4?

2013-08-05 Thread Joshua D. Drake
On 08/05/2013 12:13 PM, Jeff Janes wrote: On Mon, Aug 5, 2013 at 11:16 AM, Joshua D. Drake wrote: Hello, I seem to recall autovacuum changes landing for 9.2.4. Can someone please describe what those changes were and how they could affect usage? Those landed in 9.2.3, see release notes for

Re: [HACKERS] Autovacuum different in 9.2.4?

2013-08-05 Thread Kevin Grittner
Jeff Janes wrote: > On Mon, Aug 5, 2013 at 11:16 AM, Joshua D. Drake > wrote: >> I seem to recall autovacuum changes landing for 9.2.4. Can someone please >> describe what those changes were and how they could affect usage? > > Those landed in 9.2.3, see release notes for that version: > Fix per

Re: [HACKERS] pass-through queries to foreign servers

2013-08-05 Thread Tom Lane
David Gudeman writes: > For those who don't want to go to the link to see what I'm talking > about with query rewrites, I thought I'd give a brief description. > Foreign data wrappers currently do all of their work in the planning > phase but I claim that isn't the right place to optimize foreign

Re: [HACKERS] Autovacuum different in 9.2.4?

2013-08-05 Thread Jeff Janes
On Mon, Aug 5, 2013 at 11:16 AM, Joshua D. Drake wrote: > > Hello, > > I seem to recall autovacuum changes landing for 9.2.4. Can someone please > describe what those changes were and how they could affect usage? Those landed in 9.2.3, see release notes for that version: Fix performance problems

Re: [HACKERS] Disabling ALTER SYSTEM SET WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters

2013-08-05 Thread Bruce Momjian
On Mon, Aug 5, 2013 at 02:52:40PM -0400, Stephen Frost wrote: > > However, > > frankly, these are flat files, so I don't see a problem with having the > > administrator modify the flat file. > > Admins on Ubuntu or Debian or a host of their derivatives aren't going > to be looking in $PGDATA for

Re: [HACKERS] pass-through queries to foreign servers

2013-08-05 Thread Pavel Stehule
Hello 2013/8/5 David Gudeman : > For those who don't want to go to the link to see what I'm talking > about with query rewrites, I thought I'd give a brief description. > Foreign data wrappers currently do all of their work in the planning > phase but I claim that isn't the right place to optimize

Re: [HACKERS] pass-through queries to foreign servers

2013-08-05 Thread David Gudeman
For those who don't want to go to the link to see what I'm talking about with query rewrites, I thought I'd give a brief description. Foreign data wrappers currently do all of their work in the planning phase but I claim that isn't the right place to optimize foreign queries with aggregates and GRO

Re: [HACKERS] Disabling ALTER SYSTEM SET WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters

2013-08-05 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: > On Mon, Aug 5, 2013 at 11:16:24AM -0700, Josh Berkus wrote: > > I don't think this is a small thing. I really think we'll get a LOT of > > blowback from sysadmins -- and maybe even refusals to upgrade -- if we >

Re: [HACKERS] Unsafe GUCs and ALTER SYSTEM WAS: Re: ALTER SYSTEM SET

2013-08-05 Thread Stephen Frost
* Josh Berkus (j...@agliodbs.com) wrote: > On 08/05/2013 11:14 AM, Stefan Kaltenbrunner wrote: > > * in a few years from now people will just use superuser over the > > network for almost all stuff "because its easy and I can click around in > > $gui", having potential "unsafe" operations available

Re: [HACKERS] don't own lock of type?

2013-08-05 Thread Robert Haas
On Mon, Aug 5, 2013 at 2:32 PM, Joshua D. Drake wrote: > Hello, > > What exactly causes this? > > WARNING: you don't own a lock of type ExclusiveLock > > Does this mean the user calling the lock doesn't own the object? It means there's a bug. Either in PostgreSQL, or some loadable module you're

Re: [HACKERS] HeapTupleSatisfiesDirty fails to test HEAP_XMAX_IS_LOCKED_ONLY for TransactionIdIsInProgress(...)

2013-08-05 Thread Robert Haas
On Fri, Aug 2, 2013 at 5:25 PM, Alvaro Herrera wrote: >> The attached test case runs under isolationtester to exersise the >> problem. I've tested it against 9.2, 9.3, and HEAD, but Andres looked >> over the code and says the underlying bug goes back to the commit of >> MVCC, it's just become easi

Re: [HACKERS] Unsafe GUCs and ALTER SYSTEM WAS: Re: ALTER SYSTEM SET

2013-08-05 Thread Josh Berkus
On 08/05/2013 11:28 AM, Stephen Frost wrote: > * Josh Berkus (j...@agliodbs.com) wrote: >> Nope. ALTER SYSTEM, from my POV, is mainly for folks who *don't* use >> Puppet/Chef/whatever. > > Ok, that's fine, but let's try to avoid making life difficult for those > who *do* use puppet/chef/whateve

[HACKERS] don't own lock of type?

2013-08-05 Thread Joshua D. Drake
Hello, What exactly causes this? WARNING: you don't own a lock of type ExclusiveLock Does this mean the user calling the lock doesn't own the object? JD -- Command Prompt, Inc. - http://www.commandprompt.com/ 509-416-6579 PostgreSQL Support, Training, Professional Services and Development H

Re: [HACKERS] comment for "fast promote"

2013-08-05 Thread Fujii Masao
On Sat, Aug 3, 2013 at 4:31 PM, Tomonari Katsumata wrote: > Hi, > > I made a patch for REL9_3_STABLE which gets rid of > old promote processing. please check it. > This patch make PostgreSQL do fast promoting(*) always. > (*) which means skipping long checkpoint before increasing > timeline. Than

Re: [HACKERS] Unsafe GUCs and ALTER SYSTEM WAS: Re: ALTER SYSTEM SET

2013-08-05 Thread Stefan Kaltenbrunner
On 08/05/2013 08:21 PM, Josh Berkus wrote: > On 08/05/2013 11:14 AM, Stefan Kaltenbrunner wrote: >> * in a few years from now people will just use superuser over the >> network for almost all stuff "because its easy and I can click around in >> $gui", having potential "unsafe" operations available

Re: [HACKERS] Unsafe GUCs and ALTER SYSTEM WAS: Re: ALTER SYSTEM SET

2013-08-05 Thread Stephen Frost
* Josh Berkus (j...@agliodbs.com) wrote: > Nope. ALTER SYSTEM, from my POV, is mainly for folks who *don't* use > Puppet/Chef/whatever. Ok, that's fine, but let's try to avoid making life difficult for those who *do* use puppet/chef/whatever. This capability runs a very high risk of that by al

Re: [HACKERS] Disabling ALTER SYSTEM SET WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters

2013-08-05 Thread Bruce Momjian
On Mon, Aug 5, 2013 at 11:16:24AM -0700, Josh Berkus wrote: > I don't see this as a solution at all. "Mr. Sysadmin, we've given the > DBAs a new tool which allows them to override your version-controlled > database parameter settings. You can turn it off, though, by using this > incredibly compl

[HACKERS] Should we remove "not fast" promotion at all?

2013-08-05 Thread Fujii Masao
Hi all, We discussed the $SUBJECT in the following threads: http://www.postgresql.org/message-id/ca+tgmozbr+wl8e7mf_krp6fy4fd2pmr11tpiuyjmfx_vtg1...@mail.gmail.com http://www.postgresql.org/message-id/CAHGQGwEBUvgcx8X+Z0Hh+VdwYcJ8KCuRuLt1jSsxeLxPcX=0...@mail.gmail.com Our consensus seems to remov

Re: [HACKERS] Unsafe GUCs and ALTER SYSTEM WAS: Re: ALTER SYSTEM SET

2013-08-05 Thread Josh Berkus
On 08/05/2013 11:14 AM, Stefan Kaltenbrunner wrote: > * in a few years from now people will just use superuser over the > network for almost all stuff "because its easy and I can click around in > $gui", having potential "unsafe" operations available over the network > will in turn cause a lot of a

Re: [HACKERS] whichever ALTER SYSTEM thread is the right one

2013-08-05 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: > On Mon, Aug 5, 2013 at 02:02:50PM -0400, Stephen Frost wrote: > > * Josh Berkus (j...@agliodbs.com) wrote: > > > I'll also point out that some of our settings only really "work" in > > > combinations of two or more settings. For example, one doesn't wan

Re: [HACKERS] Unsafe GUCs and ALTER SYSTEM WAS: Re: ALTER SYSTEM SET

2013-08-05 Thread Stefan Kaltenbrunner
On 08/05/2013 07:01 PM, Josh Berkus wrote: > Stephen, all: > > (forked from main ALTER SYSTEM discussion. this thread is meant to > discuss only this question: > > E) whether "unsafe" settings or "restart" settings should be allowed in > ALTER SYSTEM SET.) > > On 08/02/2013 01:48 PM, Stephen Fr

Re: [HACKERS] Disabling ALTER SYSTEM SET WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters

2013-08-05 Thread Josh Berkus
All: (this thread concerns argument (F) whether or not admins need a way to disable ALTER SYSTEM SET) On 08/01/2013 04:03 PM, Dimitri Fontaine wrote:> If only we could trigger some actions when a command is about to be > executed, in a way that it's easy for the user to implement whatever > polic

[HACKERS] Autovacuum different in 9.2.4?

2013-08-05 Thread Joshua D. Drake
Hello, I seem to recall autovacuum changes landing for 9.2.4. Can someone please describe what those changes were and how they could affect usage? JD -- Command Prompt, Inc. - http://www.commandprompt.com/ 509-416-6579 PostgreSQL Support, Training, Professional Services and Development High

Re: [HACKERS] Unsafe GUCs and ALTER SYSTEM WAS: Re: ALTER SYSTEM SET

2013-08-05 Thread Stefan Kaltenbrunner
On 08/05/2013 08:02 PM, Josh Berkus wrote: > On 08/05/2013 10:49 AM, Stephen Frost wrote: >> Josh, I really have to ask- are these people who are implementing puppet >> to control these configs really clamoring to have an 'ALTER SYSTEM' PG >> command to have to code against instead of dealing with

Re: [HACKERS] File-per-GUC WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Josh Berkus
On 08/05/2013 11:02 AM, Stephen Frost wrote: > * Josh Berkus (j...@agliodbs.com) wrote: >> I'll also point out that some of our settings only really "work" in >> combinations of two or more settings. For example, one doesn't want to >> set archive_mode = on unless one is setting archive_command as

Re: [HACKERS] File-per-GUC WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Bruce Momjian
On Mon, Aug 5, 2013 at 02:02:50PM -0400, Stephen Frost wrote: > * Josh Berkus (j...@agliodbs.com) wrote: > > I'll also point out that some of our settings only really "work" in > > combinations of two or more settings. For example, one doesn't want to > > set archive_mode = on unless one is setti

Re: [HACKERS] Unsafe GUCs and ALTER SYSTEM WAS: Re: ALTER SYSTEM SET

2013-08-05 Thread Josh Berkus
On 08/05/2013 10:49 AM, Stephen Frost wrote: > Josh, I really have to ask- are these people who are implementing puppet > to control these configs really clamoring to have an 'ALTER SYSTEM' PG > command to have to code against instead of dealing with text files? I > feel like you're arguing for th

Re: [HACKERS] File-per-GUC WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Stephen Frost
* Josh Berkus (j...@agliodbs.com) wrote: > I'll also point out that some of our settings only really "work" in > combinations of two or more settings. For example, one doesn't want to > set archive_mode = on unless one is setting archive_command as well. And one doesn't want to set archive_comman

Re: [HACKERS] Re: File-per-GUC WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Bruce Momjian
On Mon, Aug 5, 2013 at 10:52:52AM -0700, David Johnston wrote: > Josh Berkus wrote > > (this discussion concerns issue (D), file-per-setting vs. one-big-file) > > > > So the case of two sessions both modifying ALTER SYSTEM SET, and one > > succeeding for some-but-all-GUCS, and the other succeedin

Re: [HACKERS] File-per-GUC WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Tom Lane
Josh Berkus writes: > I'll also point out that some of our settings only really "work" in > combinations of two or more settings. For example, one doesn't want to > set archive_mode = on unless one is setting archive_command as well. > And generally if one sets sequential_page_cost, one is changi

[HACKERS] Re: File-per-GUC WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread David Johnston
Josh Berkus wrote > (this discussion concerns issue (D), file-per-setting vs. one-big-file) > > So the case of two sessions both modifying ALTER SYSTEM SET, and one > succeeding for some-but-all-GUCS, and the other succeeding for > some-but-not-all-GUCs, would not be user-friendly or pretty, even

Re: [HACKERS] Unsafe GUCs and ALTER SYSTEM WAS: Re: ALTER SYSTEM SET

2013-08-05 Thread Stephen Frost
Josh, * Josh Berkus (j...@agliodbs.com) wrote: > Based on serving literally hundreds of clients, the below are the > settings we change on client servers 50% or more of the time. Other > settings I touch maybe 10% of the time. THese are also, in general, the > settings which I modify when we cre

Re: [HACKERS] File-per-GUC WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Bruce Momjian
On Mon, Aug 5, 2013 at 10:21:56AM -0700, Josh Berkus wrote: > (this discussion concerns issue (D), file-per-setting vs. one-big-file) > > On 08/05/2013 10:16 AM, Tom Lane wrote: > > Bruce Momjian writes: > >> So my larger question is why a single-guc-per-file avoids corruption > >> while having

Re: [HACKERS] Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Bruce Momjian
On Mon, Aug 5, 2013 at 01:16:10PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > So my larger question is why a single-guc-per-file avoids corruption > > while having all the gucs in a single file does not. > > If it's file-per-GUC, then when two sessions try to write different GUCs, > there

Re: [HACKERS] query_planner() API change

2013-08-05 Thread Atri Sharma
On Mon, Aug 5, 2013 at 6:19 PM, Tom Lane wrote: > Atri Sharma writes: >>> While we could complicate query_planner()'s API even more to add some >>> understanding of unnecessary resjunk items, I think this is probably >>> the straw that breaks the camel's back for the current approach here. >>> Th

Re: [HACKERS] File-per-GUC WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Josh Berkus
(this discussion concerns issue (D), file-per-setting vs. one-big-file) On 08/05/2013 10:16 AM, Tom Lane wrote: > Bruce Momjian writes: >> So my larger question is why a single-guc-per-file avoids corruption >> while having all the gucs in a single file does not. > > If it's file-per-GUC, then w

Re: [HACKERS] Comma Comma Comma 8601

2013-08-05 Thread David E. Wheeler
On Jul 23, 2013, at 6:24 PM, David E. Wheeler wrote: > I kind of suspect not, since this fails: > > david=# select '12:24:53 654'::time; > ERROR: invalid input syntax for type time: "12:24:53 654" > LINE 1: select '12:24:53 654'::time; > ^ > > I would have guessed that the time p

Re: [HACKERS] Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Tom Lane
Bruce Momjian writes: > So my larger question is why a single-guc-per-file avoids corruption > while having all the gucs in a single file does not. If it's file-per-GUC, then when two sessions try to write different GUCs, there is no conflict. When they try to write the same GUC, the end result

Re: [HACKERS] mvcc catalo gsnapshots and TopTransactionContext

2013-08-05 Thread Noah Misch
On Fri, Jul 12, 2013 at 11:42:23AM +0200, Andres Freund wrote: > On 2013-07-11 15:09:45 -0400, Tom Lane wrote: > > It never has been, and never will be, allowed to call the catcache code > > without being in a transaction. What do you think will happen if the > > requested row isn't in cache? A t

Re: [HACKERS] Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Andres Freund
On 2013-08-05 12:53:24 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2013-08-05 12:18:25 -0400, Bruce Momjian wrote: > >> I am unclear why we don't need a lock around _each_ GUC, i.e. if two > >> sessions try to modify the same GUC at the same time. And if we need a > >> lock, seems we ca

Re: [HACKERS] Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Bruce Momjian
On Mon, Aug 5, 2013 at 12:53:24PM -0400, Tom Lane wrote: > Andres Freund writes: > > On 2013-08-05 12:18:25 -0400, Bruce Momjian wrote: > >> I am unclear why we don't need a lock around _each_ GUC, i.e. if two > >> sessions try to modify the same GUC at the same time. And if we need a > >> lock,

Re: [HACKERS] Unsafe GUCs and ALTER SYSTEM WAS: Re: ALTER SYSTEM SET

2013-08-05 Thread Josh Berkus
Stephen, all: (forked from main ALTER SYSTEM discussion. this thread is meant to discuss only this question: E) whether "unsafe" settings or "restart" settings should be allowed in ALTER SYSTEM SET.) On 08/02/2013 01:48 PM, Stephen Frost wrote: > Reflecting on this a bit more, I'm curious what

Re: [HACKERS] Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Tom Lane
Andres Freund writes: > On 2013-08-05 12:18:25 -0400, Bruce Momjian wrote: >> I am unclear why we don't need a lock around _each_ GUC, i.e. if two >> sessions try to modify the same GUC at the same time. And if we need a >> lock, seems we can have just one and write all the settings to one file >

Re: [HACKERS] StrategyGetBuffer optimization, take 2

2013-08-05 Thread Atri Sharma
> optimization 2: refcount is examined during buffer allocation without > a lock. if it's > 0, buffer is assumed pinned (even though it may not > in fact be) and sweep continues +1. I think this shall not lead to much problems, since a lost update cannot,IMO, lead to disastrous result. At most,

Re: [HACKERS] Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Andres Freund
On 2013-08-05 12:18:25 -0400, Bruce Momjian wrote: > On Thu, Aug 1, 2013 at 03:40:22PM +0100, Greg Stark wrote: > > Why isn't it enough to just dump out all variables with a source of alter > > system to a text file? You can either have a single global lock around that > > operation or write it to

Re: [HACKERS] CREATE EVENT TRIGGER syntax

2013-08-05 Thread Bruce Momjian
On Fri, Jul 19, 2013 at 05:12:16PM -0700, Dimitri Fontaine wrote: > Joe Abbate writes: > > What is the purpose of the [ AND ... ] at the end of the WHEN clause? > > Is that for later releases, when presumably additional filter_variables > > will be introduced? Right now, if I add "AND tag IN ..."

Re: [HACKERS] Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Josh Berkus
All, To limit the argument here, let's please not argue about things which people already agree on. So: We seem to have consensus around: A) the inadvisability of storing GUCs in a system catalog. B) the utility of a conf.d in /etc/ which may have nothing to do with ALTER SYSTEM SET C) that a

Re: [HACKERS] StrategyGetBuffer optimization, take 2

2013-08-05 Thread Andres Freund
On 2013-08-05 10:49:08 -0500, Merlin Moncure wrote: > optimization 4: remove free list lock (via Jeff Janes). This is the > other optimization: one backend will no longer be able to shut down > buffer allocation I think splitting off the actual freelist checking into a spinlock makes quite a bit

Re: [HACKERS] Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Bruce Momjian
On Thu, Aug 1, 2013 at 03:40:22PM +0100, Greg Stark wrote: > Why isn't it enough to just dump out all variables with a source of alter > system to a text file? You can either have a single global lock around that > operation or write it to a new file and move it into place. > > -- > greg > > On

Re: [HACKERS] getting rid of SnapshotNow

2013-08-05 Thread Bruce Momjian
On Mon, Aug 5, 2013 at 05:22:28PM +0200, Andres Freund wrote: > On 2013-08-05 11:17:08 -0400, Bruce Momjian wrote: > > One interesting aspect of this patch is that the backend code is no > > longer even checking HEAP_MOVED_OFF and HEAP_MOVED_IN. However, we > > can't reuse those bits because they

  1   2   >