Re: [HACKERS] Patch for reserved connections for replication users

2013-08-05 Thread Andres Freund
On 2013-08-02 08:16:15 -0400, Robert Haas wrote: On Tue, Jul 30, 2013 at 3:10 AM, Gibheer gibh...@zero-knowledge.org wrote: here is an update off my patch based on the discussion with Marko Tiikkaja and Andres Freund. Marko and I had the idea of introducing reserved connections based on

Re: [HACKERS] Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Amit Kapila
On Saturday, August 03, 2013 12:53 AM Tom Lane wrote: Stephen Frost sfr...@snowman.net writes: * Josh Berkus (j...@agliodbs.com) wrote: A much simpler solution to the issue Stephen proposes is to have a way to start up the server with all settings from ALTER SYSTEM SET disabled, just

Re: [HACKERS] query_planner() API change

2013-08-05 Thread Etsuro Fujita
Robert Haas robertmh...@gmail.com writes: On Sun, Aug 4, 2013 at 6:20 PM, Tom Lane t...@sss.pgh.pa.us wrote: I think it's time to bite the bullet and *not* pass back completed paths. What's looking more attractive now is to just pass back the top-level RelOptInfo (final_rel in

[HACKERS] Bottlenecks with large number of relation segment files

2013-08-05 Thread Amit Langote
Hello, I am looking the effect of having large number of relation files under $PGDATA/base/ (for example, in cases where I choose lower segment size using --with-segsize). Consider a case where I am working with a large database with large relations, for example a database similar in size to what

Re: [HACKERS] Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Tom Lane
Amit Kapila amit.kap...@huawei.com writes: On Saturday, August 03, 2013 12:53 AM Tom Lane wrote: Yeah, this approach is a nonstarter because there's no reason to assume that a postmaster started with default parameters will start successfully, or will be connectable-to if it does start.

Re: [HACKERS] query_planner() API change

2013-08-05 Thread Atri Sharma
While we could complicate query_planner()'s API even more to add some understanding of unnecessary resjunk items, I think this is probably the straw that breaks the camel's back for the current approach here. There is already a comment like this in query_planner(): * This introduces

Re: [HACKERS] query_planner() API change

2013-08-05 Thread Ashutosh Bapat
On Mon, Aug 5, 2013 at 3:50 AM, Tom Lane t...@sss.pgh.pa.us wrote: I've been looking at what it would take to do proper cost estimation for the recently-discussed patch to suppress calculation of unnecessary ORDER BY expressions. Can you please mention the subject of the thread? I tried to

[HACKERS] FOR UPDATE/SHARE incompatibility with GROUP BY, DISTINCT, HAVING and window functions

2013-08-05 Thread Michael Paquier
Hi all, By having a look at the documentation of SELECT, it is not specified that FOR SHARE/UPDATE and friends are incompatible with the clauses in $subject http://www.postgresql.org/docs/9.2/static/sql-select.html This restriction is explicitly cited for INTERSECT and UNION though, so people can

Re: [HACKERS] query_planner() API change

2013-08-05 Thread Etsuro Fujita
I agree with the idea,but am trying to understand why adding understanding of resjunk columns is a bad idea. Just for understanding purpose, could you please elaborate a bit on it? Although I may not have understood your question correctly, I think it is good to see

Re: [HACKERS] query_planner() API change

2013-08-05 Thread Etsuro Fujita
On Mon, Aug 5, 2013 at 3:50 AM, Tom Lane t...@sss.pgh.pa.us wrote: I've been looking at what it would take to do proper cost estimation for the recently-discussed patch to suppress calculation of unnecessary ORDER BY expressions. Can you please mention the subject of the thread? I tried to

Re: [HACKERS] [GENERAL] Bottlenecks with large number of relation segment files

2013-08-05 Thread KONDO Mitsumasa
Hi Amit, (2013/08/05 15:23), Amit Langote wrote: May the routines in fd.c become bottleneck with a large number of concurrent connections to above database, say something like pgbench -j 8 -c 128? Is there any other place I should be paying attention to? What kind of file system did you use?

Re: [HACKERS] [GENERAL] Bottlenecks with large number of relation segment files

2013-08-05 Thread Amit Langote
On Mon, Aug 5, 2013 at 5:01 PM, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: Hi Amit, (2013/08/05 15:23), Amit Langote wrote: May the routines in fd.c become bottleneck with a large number of concurrent connections to above database, say something like pgbench -j 8 -c 128? Is there

Re: [HACKERS] Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Amit Kapila
On Monday, August 05, 2013 11:57 AM Tom Lane wrote: Amit Kapila amit.kap...@huawei.com writes: On Saturday, August 03, 2013 12:53 AM Tom Lane wrote: Yeah, this approach is a nonstarter because there's no reason to assume that a postmaster started with default parameters will start

Re: [HACKERS] [GENERAL] Bottlenecks with large number of relation segment files

2013-08-05 Thread KONDO Mitsumasa
(2013/08/05 17:14), Amit Langote wrote: So, within the limits of max_files_per_process, the routines of file.c should not become a bottleneck? It may not become bottleneck. 1 FD consumes 160 byte in 64bit system. See linux manual at epoll. Regards, -- Mitsumasa KONDO NTT Open Source Software

Re: [HACKERS] [GENERAL] Bottlenecks with large number of relation segment files

2013-08-05 Thread Andres Freund
On 2013-08-05 18:40:10 +0900, KONDO Mitsumasa wrote: (2013/08/05 17:14), Amit Langote wrote: So, within the limits of max_files_per_process, the routines of file.c should not become a bottleneck? It may not become bottleneck. 1 FD consumes 160 byte in 64bit system. See linux manual at epoll.

Re: [HACKERS] Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Tom Lane
Amit Kapila amit.kap...@huawei.com writes: On Monday, August 05, 2013 11:57 AM Tom Lane wrote: My point here is just that we should keep the parameter values in plain text files, Here by text files, do you mean to say you are expecting file-per-guc-setting? No, I don't think this argument

Re: [HACKERS] [GENERAL] Bottlenecks with large number of relation segment files

2013-08-05 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: ... Also, there are global limits to the amount of filehandles that can simultaneously opened on a system. Yeah. Raising max_files_per_process puts you at serious risk that everything else on the box will start falling over for lack of available FD

Re: [HACKERS] [v9.4] row level security

2013-08-05 Thread Bruce Momjian
On Tue, Jul 23, 2013 at 11:30:14AM -0700, Josh Berkus wrote: Greg, It's more than the available experienced reviewers are willing to chew on fully as volunteers. The reward for spending review time is pretty low right now. Short of paying for review time, I don't think we have another

Re: [HACKERS] query_planner() API change

2013-08-05 Thread Tom Lane
Atri Sharma atri.j...@gmail.com writes: While we could complicate query_planner()'s API even more to add some understanding of unnecessary resjunk items, I think this is probably the straw that breaks the camel's back for the current approach here. There is already a comment like this in

Re: [HACKERS] query_planner() API change

2013-08-05 Thread Tom Lane
Ashutosh Bapat ashutosh.ba...@enterprisedb.com writes: Can we change the query_planner() to return both the paths (presorted and unsorted) irrespective of the cost of presorted path, and let grouping_planner() (or any caller of query_planner()) handle which of them to pick up? That's exactly

[HACKERS] Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Greg Stark
On Fri, Aug 2, 2013 at 4:05 PM, Stephen Frost sfr...@snowman.net wrote: I'm not even clear we do want this in /etc since none of our GUC options are repeatable things like Apache virtual servers. It actually makes *more* sense for pg_hba than it does for gucs. I think we can assume that in the

Re: [HACKERS] Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Amit Kapila amit.kap...@huawei.com writes: On Monday, August 05, 2013 11:57 AM Tom Lane wrote: My point here is just that we should keep the parameter values in plain text files, Here by text files, do you mean to say you are expecting

[HACKERS] Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Stephen Frost
* Greg Stark (st...@mit.edu) wrote: On Fri, Aug 2, 2013 at 4:05 PM, Stephen Frost sfr...@snowman.net wrote: I'm not even clear we do want this in /etc since none of our GUC options are repeatable things like Apache virtual servers. It actually makes *more* sense for pg_hba than it does for

Re: [HACKERS] getting rid of SnapshotNow

2013-08-05 Thread Bruce Momjian
On Thu, Jul 25, 2013 at 07:24:53PM -0400, Robert Haas wrote: - /* Used by pre-9.0 binary upgrades */ - if (tuple-t_infomask HEAP_MOVED_OFF) - { - TransactionId xvac = HeapTupleHeaderGetXvac(tuple); - - if

Re: [HACKERS] getting rid of SnapshotNow

2013-08-05 Thread Andres Freund
On 2013-08-05 11:17:08 -0400, Bruce Momjian wrote: One interesting aspect of this patch is that the backend code is no longer even checking HEAP_MOVED_OFF and HEAP_MOVED_IN. However, we can't reuse those bits because they could be set from pre-9.0 rows. The other tqual.c .satisfies routines

Re: [HACKERS] [COMMITTERS] pgsql: Add support for REFRESH MATERIALIZED VIEW CONCURRENTLY.

2013-08-05 Thread Kevin Grittner
Some of the issues raised by Andres and Noah have been addressed. These all seemed simple and non-controversial, so I've just applied the suggested fixes. Some issues remain, such as how best to create the temp table used for the diff data, and the related simplification of the security context

[HACKERS] StrategyGetBuffer optimization, take 2

2013-08-05 Thread Merlin Moncure
My $company recently acquired another postgres based $company and migrated all their server operations into our datacenter. Upon completing the move, the newly migrated database server started experiencing huge load spikes. *) Environment description: Postgres 9.2.4 RHEL 6 32 cores virtualized

Re: [HACKERS] [COMMITTERS] pgsql: Add support for REFRESH MATERIALIZED VIEW CONCURRENTLY.

2013-08-05 Thread Andres Freund
Hi On 2013-08-05 08:37:57 -0700, Kevin Grittner wrote: Some of the issues raised by Andres and Noah have been addressed. These all seemed simple and non-controversial, so I've just applied the suggested fixes. Cool!    I'd even suggest using BuildIndexInfo() or such on the indexes,    then

Re: [HACKERS] getting rid of SnapshotNow

2013-08-05 Thread Bruce Momjian
On Mon, Aug 5, 2013 at 05:22:28PM +0200, Andres Freund wrote: On 2013-08-05 11:17:08 -0400, Bruce Momjian wrote: One interesting aspect of this patch is that the backend code is no longer even checking HEAP_MOVED_OFF and HEAP_MOVED_IN. However, we can't reuse those bits because they could

Re: [HACKERS] Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Bruce Momjian
On Thu, Aug 1, 2013 at 03:40:22PM +0100, Greg Stark wrote: Why isn't it enough to just dump out all variables with a source of alter system to a text file? You can either have a single global lock around that operation or write it to a new file and move it into place. -- greg On 1 Aug

Re: [HACKERS] StrategyGetBuffer optimization, take 2

2013-08-05 Thread Andres Freund
On 2013-08-05 10:49:08 -0500, Merlin Moncure wrote: optimization 4: remove free list lock (via Jeff Janes). This is the other optimization: one backend will no longer be able to shut down buffer allocation I think splitting off the actual freelist checking into a spinlock makes quite a bit of

Re: [HACKERS] Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Josh Berkus
All, To limit the argument here, let's please not argue about things which people already agree on. So: We seem to have consensus around: A) the inadvisability of storing GUCs in a system catalog. B) the utility of a conf.d in /etc/ which may have nothing to do with ALTER SYSTEM SET C) that

Re: [HACKERS] CREATE EVENT TRIGGER syntax

2013-08-05 Thread Bruce Momjian
On Fri, Jul 19, 2013 at 05:12:16PM -0700, Dimitri Fontaine wrote: Joe Abbate j...@freedomcircle.com writes: What is the purpose of the [ AND ... ] at the end of the WHEN clause? Is that for later releases, when presumably additional filter_variables will be introduced? Right now, if I add

Re: [HACKERS] Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Andres Freund
On 2013-08-05 12:18:25 -0400, Bruce Momjian wrote: On Thu, Aug 1, 2013 at 03:40:22PM +0100, Greg Stark wrote: Why isn't it enough to just dump out all variables with a source of alter system to a text file? You can either have a single global lock around that operation or write it to a new

Re: [HACKERS] StrategyGetBuffer optimization, take 2

2013-08-05 Thread Atri Sharma
optimization 2: refcount is examined during buffer allocation without a lock. if it's 0, buffer is assumed pinned (even though it may not in fact be) and sweep continues +1. I think this shall not lead to much problems, since a lost update cannot,IMO, lead to disastrous result. At most, a

Re: [HACKERS] Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-08-05 12:18:25 -0400, Bruce Momjian wrote: I am unclear why we don't need a lock around _each_ GUC, i.e. if two sessions try to modify the same GUC at the same time. And if we need a lock, seems we can have just one and write all the

Re: [HACKERS] Unsafe GUCs and ALTER SYSTEM WAS: Re: ALTER SYSTEM SET

2013-08-05 Thread Josh Berkus
Stephen, all: (forked from main ALTER SYSTEM discussion. this thread is meant to discuss only this question: E) whether unsafe settings or restart settings should be allowed in ALTER SYSTEM SET.) On 08/02/2013 01:48 PM, Stephen Frost wrote: Reflecting on this a bit more, I'm curious what your

Re: [HACKERS] Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Bruce Momjian
On Mon, Aug 5, 2013 at 12:53:24PM -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-08-05 12:18:25 -0400, Bruce Momjian wrote: I am unclear why we don't need a lock around _each_ GUC, i.e. if two sessions try to modify the same GUC at the same time. And if we

Re: [HACKERS] Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Andres Freund
On 2013-08-05 12:53:24 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-08-05 12:18:25 -0400, Bruce Momjian wrote: I am unclear why we don't need a lock around _each_ GUC, i.e. if two sessions try to modify the same GUC at the same time. And if we need a lock,

Re: [HACKERS] mvcc catalo gsnapshots and TopTransactionContext

2013-08-05 Thread Noah Misch
On Fri, Jul 12, 2013 at 11:42:23AM +0200, Andres Freund wrote: On 2013-07-11 15:09:45 -0400, Tom Lane wrote: It never has been, and never will be, allowed to call the catcache code without being in a transaction. What do you think will happen if the requested row isn't in cache? A table

Re: [HACKERS] Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: So my larger question is why a single-guc-per-file avoids corruption while having all the gucs in a single file does not. If it's file-per-GUC, then when two sessions try to write different GUCs, there is no conflict. When they try to write the same GUC,

Re: [HACKERS] Comma Comma Comma 8601

2013-08-05 Thread David E. Wheeler
On Jul 23, 2013, at 6:24 PM, David E. Wheeler da...@justatheory.com wrote: I kind of suspect not, since this fails: david=# select '12:24:53 654'::time; ERROR: invalid input syntax for type time: 12:24:53 654 LINE 1: select '12:24:53 654'::time; ^ I would have guessed

Re: [HACKERS] File-per-GUC WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Josh Berkus
(this discussion concerns issue (D), file-per-setting vs. one-big-file) On 08/05/2013 10:16 AM, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: So my larger question is why a single-guc-per-file avoids corruption while having all the gucs in a single file does not. If it's

Re: [HACKERS] query_planner() API change

2013-08-05 Thread Atri Sharma
On Mon, Aug 5, 2013 at 6:19 PM, Tom Lane t...@sss.pgh.pa.us wrote: Atri Sharma atri.j...@gmail.com writes: While we could complicate query_planner()'s API even more to add some understanding of unnecessary resjunk items, I think this is probably the straw that breaks the camel's back for the

Re: [HACKERS] Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Bruce Momjian
On Mon, Aug 5, 2013 at 01:16:10PM -0400, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: So my larger question is why a single-guc-per-file avoids corruption while having all the gucs in a single file does not. If it's file-per-GUC, then when two sessions try to write different

Re: [HACKERS] File-per-GUC WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Bruce Momjian
On Mon, Aug 5, 2013 at 10:21:56AM -0700, Josh Berkus wrote: (this discussion concerns issue (D), file-per-setting vs. one-big-file) On 08/05/2013 10:16 AM, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: So my larger question is why a single-guc-per-file avoids corruption while

Re: [HACKERS] Unsafe GUCs and ALTER SYSTEM WAS: Re: ALTER SYSTEM SET

2013-08-05 Thread Stephen Frost
Josh, * Josh Berkus (j...@agliodbs.com) wrote: Based on serving literally hundreds of clients, the below are the settings we change on client servers 50% or more of the time. Other settings I touch maybe 10% of the time. THese are also, in general, the settings which I modify when we create

[HACKERS] Re: File-per-GUC WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread David Johnston
Josh Berkus wrote (this discussion concerns issue (D), file-per-setting vs. one-big-file) So the case of two sessions both modifying ALTER SYSTEM SET, and one succeeding for some-but-all-GUCS, and the other succeeding for some-but-not-all-GUCs, would not be user-friendly or pretty, even if

Re: [HACKERS] File-per-GUC WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: I'll also point out that some of our settings only really work in combinations of two or more settings. For example, one doesn't want to set archive_mode = on unless one is setting archive_command as well. And generally if one sets sequential_page_cost,

Re: [HACKERS] Re: File-per-GUC WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Bruce Momjian
On Mon, Aug 5, 2013 at 10:52:52AM -0700, David Johnston wrote: Josh Berkus wrote (this discussion concerns issue (D), file-per-setting vs. one-big-file) So the case of two sessions both modifying ALTER SYSTEM SET, and one succeeding for some-but-all-GUCS, and the other succeeding for

Re: [HACKERS] File-per-GUC WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Stephen Frost
* Josh Berkus (j...@agliodbs.com) wrote: I'll also point out that some of our settings only really work in combinations of two or more settings. For example, one doesn't want to set archive_mode = on unless one is setting archive_command as well. And one doesn't want to set archive_command

Re: [HACKERS] Unsafe GUCs and ALTER SYSTEM WAS: Re: ALTER SYSTEM SET

2013-08-05 Thread Josh Berkus
On 08/05/2013 10:49 AM, Stephen Frost wrote: Josh, I really have to ask- are these people who are implementing puppet to control these configs really clamoring to have an 'ALTER SYSTEM' PG command to have to code against instead of dealing with text files? I feel like you're arguing for these

Re: [HACKERS] File-per-GUC WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Bruce Momjian
On Mon, Aug 5, 2013 at 02:02:50PM -0400, Stephen Frost wrote: * Josh Berkus (j...@agliodbs.com) wrote: I'll also point out that some of our settings only really work in combinations of two or more settings. For example, one doesn't want to set archive_mode = on unless one is setting

Re: [HACKERS] File-per-GUC WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-05 Thread Josh Berkus
On 08/05/2013 11:02 AM, Stephen Frost wrote: * Josh Berkus (j...@agliodbs.com) wrote: I'll also point out that some of our settings only really work in combinations of two or more settings. For example, one doesn't want to set archive_mode = on unless one is setting archive_command as well.

Re: [HACKERS] Unsafe GUCs and ALTER SYSTEM WAS: Re: ALTER SYSTEM SET

2013-08-05 Thread Stefan Kaltenbrunner
On 08/05/2013 08:02 PM, Josh Berkus wrote: On 08/05/2013 10:49 AM, Stephen Frost wrote: Josh, I really have to ask- are these people who are implementing puppet to control these configs really clamoring to have an 'ALTER SYSTEM' PG command to have to code against instead of dealing with text

[HACKERS] Autovacuum different in 9.2.4?

2013-08-05 Thread Joshua D. Drake
Hello, I seem to recall autovacuum changes landing for 9.2.4. Can someone please describe what those changes were and how they could affect usage? JD -- Command Prompt, Inc. - http://www.commandprompt.com/ 509-416-6579 PostgreSQL Support, Training, Professional Services and Development High

Re: [HACKERS] Disabling ALTER SYSTEM SET WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters

2013-08-05 Thread Josh Berkus
All: (this thread concerns argument (F) whether or not admins need a way to disable ALTER SYSTEM SET) On 08/01/2013 04:03 PM, Dimitri Fontaine wrote: If only we could trigger some actions when a command is about to be executed, in a way that it's easy for the user to implement whatever policy

Re: [HACKERS] Unsafe GUCs and ALTER SYSTEM WAS: Re: ALTER SYSTEM SET

2013-08-05 Thread Stefan Kaltenbrunner
On 08/05/2013 07:01 PM, Josh Berkus wrote: Stephen, all: (forked from main ALTER SYSTEM discussion. this thread is meant to discuss only this question: E) whether unsafe settings or restart settings should be allowed in ALTER SYSTEM SET.) On 08/02/2013 01:48 PM, Stephen Frost wrote:

Re: [HACKERS] whichever ALTER SYSTEM thread is the right one

2013-08-05 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: On Mon, Aug 5, 2013 at 02:02:50PM -0400, Stephen Frost wrote: * Josh Berkus (j...@agliodbs.com) wrote: I'll also point out that some of our settings only really work in combinations of two or more settings. For example, one doesn't want to

Re: [HACKERS] Unsafe GUCs and ALTER SYSTEM WAS: Re: ALTER SYSTEM SET

2013-08-05 Thread Josh Berkus
On 08/05/2013 11:14 AM, Stefan Kaltenbrunner wrote: * in a few years from now people will just use superuser over the network for almost all stuff because its easy and I can click around in $gui, having potential unsafe operations available over the network will in turn cause a lot of actual

[HACKERS] Should we remove not fast promotion at all?

2013-08-05 Thread Fujii Masao
Hi all, We discussed the $SUBJECT in the following threads: http://www.postgresql.org/message-id/ca+tgmozbr+wl8e7mf_krp6fy4fd2pmr11tpiuyjmfx_vtg1...@mail.gmail.com http://www.postgresql.org/message-id/CAHGQGwEBUvgcx8X+Z0Hh+VdwYcJ8KCuRuLt1jSsxeLxPcX=0...@mail.gmail.com Our consensus seems to

Re: [HACKERS] Disabling ALTER SYSTEM SET WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters

2013-08-05 Thread Bruce Momjian
On Mon, Aug 5, 2013 at 11:16:24AM -0700, Josh Berkus wrote: I don't see this as a solution at all. Mr. Sysadmin, we've given the DBAs a new tool which allows them to override your version-controlled database parameter settings. You can turn it off, though, by using this incredibly

Re: [HACKERS] Unsafe GUCs and ALTER SYSTEM WAS: Re: ALTER SYSTEM SET

2013-08-05 Thread Stephen Frost
* Josh Berkus (j...@agliodbs.com) wrote: Nope. ALTER SYSTEM, from my POV, is mainly for folks who *don't* use Puppet/Chef/whatever. Ok, that's fine, but let's try to avoid making life difficult for those who *do* use puppet/chef/whatever. This capability runs a very high risk of that by

Re: [HACKERS] Unsafe GUCs and ALTER SYSTEM WAS: Re: ALTER SYSTEM SET

2013-08-05 Thread Stefan Kaltenbrunner
On 08/05/2013 08:21 PM, Josh Berkus wrote: On 08/05/2013 11:14 AM, Stefan Kaltenbrunner wrote: * in a few years from now people will just use superuser over the network for almost all stuff because its easy and I can click around in $gui, having potential unsafe operations available over the

Re: [HACKERS] comment for fast promote

2013-08-05 Thread Fujii Masao
On Sat, Aug 3, 2013 at 4:31 PM, Tomonari Katsumata t.katsumata1...@gmail.com wrote: Hi, I made a patch for REL9_3_STABLE which gets rid of old promote processing. please check it. This patch make PostgreSQL do fast promoting(*) always. (*) which means skipping long checkpoint before

[HACKERS] don't own lock of type?

2013-08-05 Thread Joshua D. Drake
Hello, What exactly causes this? WARNING: you don't own a lock of type ExclusiveLock Does this mean the user calling the lock doesn't own the object? JD -- Command Prompt, Inc. - http://www.commandprompt.com/ 509-416-6579 PostgreSQL Support, Training, Professional Services and Development

Re: [HACKERS] Unsafe GUCs and ALTER SYSTEM WAS: Re: ALTER SYSTEM SET

2013-08-05 Thread Josh Berkus
On 08/05/2013 11:28 AM, Stephen Frost wrote: * Josh Berkus (j...@agliodbs.com) wrote: Nope. ALTER SYSTEM, from my POV, is mainly for folks who *don't* use Puppet/Chef/whatever. Ok, that's fine, but let's try to avoid making life difficult for those who *do* use puppet/chef/whatever.

Re: [HACKERS] HeapTupleSatisfiesDirty fails to test HEAP_XMAX_IS_LOCKED_ONLY for TransactionIdIsInProgress(...)

2013-08-05 Thread Robert Haas
On Fri, Aug 2, 2013 at 5:25 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: The attached test case runs under isolationtester to exersise the problem. I've tested it against 9.2, 9.3, and HEAD, but Andres looked over the code and says the underlying bug goes back to the commit of MVCC, it's

Re: [HACKERS] don't own lock of type?

2013-08-05 Thread Robert Haas
On Mon, Aug 5, 2013 at 2:32 PM, Joshua D. Drake j...@commandprompt.com wrote: Hello, What exactly causes this? WARNING: you don't own a lock of type ExclusiveLock Does this mean the user calling the lock doesn't own the object? It means there's a bug. Either in PostgreSQL, or some

Re: [HACKERS] Unsafe GUCs and ALTER SYSTEM WAS: Re: ALTER SYSTEM SET

2013-08-05 Thread Stephen Frost
* Josh Berkus (j...@agliodbs.com) wrote: On 08/05/2013 11:14 AM, Stefan Kaltenbrunner wrote: * in a few years from now people will just use superuser over the network for almost all stuff because its easy and I can click around in $gui, having potential unsafe operations available over the

Re: [HACKERS] Disabling ALTER SYSTEM SET WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters

2013-08-05 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: On Mon, Aug 5, 2013 at 11:16:24AM -0700, Josh Berkus wrote: I don't think this is a small thing. I really think we'll get a LOT of blowback from sysadmins -- and maybe even refusals to upgrade -- if we

Re: [HACKERS] pass-through queries to foreign servers

2013-08-05 Thread David Gudeman
For those who don't want to go to the link to see what I'm talking about with query rewrites, I thought I'd give a brief description. Foreign data wrappers currently do all of their work in the planning phase but I claim that isn't the right place to optimize foreign queries with aggregates and

Re: [HACKERS] pass-through queries to foreign servers

2013-08-05 Thread Pavel Stehule
Hello 2013/8/5 David Gudeman dave.gude...@gmail.com: For those who don't want to go to the link to see what I'm talking about with query rewrites, I thought I'd give a brief description. Foreign data wrappers currently do all of their work in the planning phase but I claim that isn't the

Re: [HACKERS] Disabling ALTER SYSTEM SET WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters

2013-08-05 Thread Bruce Momjian
On Mon, Aug 5, 2013 at 02:52:40PM -0400, Stephen Frost wrote: However, frankly, these are flat files, so I don't see a problem with having the administrator modify the flat file. Admins on Ubuntu or Debian or a host of their derivatives aren't going to be looking in $PGDATA for config

Re: [HACKERS] Autovacuum different in 9.2.4?

2013-08-05 Thread Jeff Janes
On Mon, Aug 5, 2013 at 11:16 AM, Joshua D. Drake j...@commandprompt.com wrote: Hello, I seem to recall autovacuum changes landing for 9.2.4. Can someone please describe what those changes were and how they could affect usage? Those landed in 9.2.3, see release notes for that version: Fix

Re: [HACKERS] pass-through queries to foreign servers

2013-08-05 Thread Tom Lane
David Gudeman dave.gude...@gmail.com writes: For those who don't want to go to the link to see what I'm talking about with query rewrites, I thought I'd give a brief description. Foreign data wrappers currently do all of their work in the planning phase but I claim that isn't the right place

Re: [HACKERS] Autovacuum different in 9.2.4?

2013-08-05 Thread Kevin Grittner
Jeff Janes jeff.ja...@gmail.com wrote: On Mon, Aug 5, 2013 at 11:16 AM, Joshua D. Drake j...@commandprompt.com wrote: I seem to recall autovacuum changes landing for 9.2.4. Can someone please describe what those changes were and how they could affect usage? Those landed in 9.2.3, see

Re: [HACKERS] Autovacuum different in 9.2.4?

2013-08-05 Thread Joshua D. Drake
On 08/05/2013 12:13 PM, Jeff Janes wrote: On Mon, Aug 5, 2013 at 11:16 AM, Joshua D. Drake j...@commandprompt.com wrote: Hello, I seem to recall autovacuum changes landing for 9.2.4. Can someone please describe what those changes were and how they could affect usage? Those landed in

[HACKERS] Moving 'hot' pages from buffer pool to heap

2013-08-05 Thread Atri Sharma
Hi all, I was experimenting with the idea of moving hot buffer pages from the buffer pool to heap, thus allowing for normal removal of the hot buffer page from the buffer pool and freeing the corresponding buffer pool slot. This shouldnt be too hard to implement, we just need to keep track of

Re: [HACKERS] don't own lock of type?

2013-08-05 Thread Joshua D. Drake
On 08/05/2013 11:38 AM, Robert Haas wrote: On Mon, Aug 5, 2013 at 2:32 PM, Joshua D. Drake j...@commandprompt.com wrote: Hello, What exactly causes this? WARNING: you don't own a lock of type ExclusiveLock Does this mean the user calling the lock doesn't own the object? It means there's

Re: [HACKERS] Disabling ALTER SYSTEM SET WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters

2013-08-05 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: Well, can you assume that if you have a problem with one of your ALTER SYSTEM SET commands, that disabling _all_ of them is going to get you a running system? I question that, e.g. port. With postgresql.conf, you can modify the bad entry, but how would

Re: [HACKERS] Moving 'hot' pages from buffer pool to heap

2013-08-05 Thread Tom Lane
Atri Sharma atri.j...@gmail.com writes: I was experimenting with the idea of moving hot buffer pages from the buffer pool to heap, thus allowing for normal removal of the hot buffer page from the buffer pool and freeing the corresponding buffer pool slot. Uh ... what? Why in the world would

Re: [HACKERS] Autovacuum different in 9.2.4?

2013-08-05 Thread Alvaro Herrera
Joshua D. Drake escribió: On 08/05/2013 12:13 PM, Jeff Janes wrote: There should be no change in usage, unless you were taking some heroic methods to overcome the problems and can now discontinue them. That is what is confusing me, I could be cracked but messages like these: automatic

Re: [HACKERS] Disabling ALTER SYSTEM SET WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters

2013-08-05 Thread Alvaro Herrera
Tom Lane escribió: What Josh seems to be concerned with in this thread is the question of whether we should support an installation *policy decision* not to allow ALTER SYSTEM SET. Not because a particular set of parameters is broken, but just because somebody is afraid the DBA might break

Re: [HACKERS] Moving 'hot' pages from buffer pool to heap

2013-08-05 Thread Jeff Janes
On Mon, Aug 5, 2013 at 12:36 PM, Atri Sharma atri.j...@gmail.com wrote: Hi all, I was experimenting with the idea of moving hot buffer pages from the buffer pool to heap, Which heap do you mean here? Alas, half the data structures used in CS are called heap. I can't think of any of them that

Re: [HACKERS] pg_basebackup vs. Windows and tablespaces

2013-08-05 Thread Noah Misch
On Thu, Aug 01, 2013 at 06:44:41PM +0200, Dimitri Fontaine wrote: Noah Misch n...@leadboat.com writes: 2. Add a pg_basebackup option like --destdir or --sysroot, meaningful only with -Fp; tablespace backups will be stored relative to it. So if the actual tablespace path is c:/foo,

Re: [HACKERS] pg_basebackup vs. Windows and tablespaces

2013-08-05 Thread Noah Misch
On Thu, Aug 01, 2013 at 01:04:42PM -0400, Andrew Dunstan wrote: On 08/01/2013 12:15 PM, Noah Misch wrote: 1. Include in the base backup a file listing symbolic links/junction points, then have archive recovery recreate them. This file would be managed like the backup label file; exclusive

Re: [HACKERS] [9.4 CF 1]Commitfest ... over!

2013-08-05 Thread Robert Haas
On Sun, Aug 4, 2013 at 11:41 PM, Amit Kapila amit.kap...@huawei.com wrote: First of all Thank you very much for running a wonderful Commit Fest. Yes. Thanks, Josh. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing

Re: [HACKERS] Moving 'hot' pages from buffer pool to heap

2013-08-05 Thread Atri Sharma
Sent from my iPad On 06-Aug-2013, at 1:14, Tom Lane t...@sss.pgh.pa.us wrote: Atri Sharma atri.j...@gmail.com writes: I was experimenting with the idea of moving hot buffer pages from the buffer pool to heap, thus allowing for normal removal of the hot buffer page from the buffer pool and

Re: [HACKERS] Moving 'hot' pages from buffer pool to heap

2013-08-05 Thread Atri Sharma
Sent from my iPad On 06-Aug-2013, at 1:24, Jeff Janes jeff.ja...@gmail.com wrote: On Mon, Aug 5, 2013 at 12:36 PM, Atri Sharma atri.j...@gmail.com wrote: Hi all, I was experimenting with the idea of moving hot buffer pages from the buffer pool to heap, Which heap do you mean here?

Re: [HACKERS] make --enable-depend the default

2013-08-05 Thread Peter Eisentraut
On Thu, 2013-08-01 at 11:10 +0200, Andres Freund wrote: People, including me, every now and then forget to pass --enable-depend to configure (when not using my own environment). Which then leads to strange errors that cost time to track down... One argument against that is that we only support

Re: [HACKERS] Disabling ALTER SYSTEM SET WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters

2013-08-05 Thread Dimitri Fontaine
Hi, I'm now completely lost in the current threads. Is there a single valid use case for the feature we're trying to design? Who is the target audience of this patch? Josh Berkus j...@agliodbs.com writes: I don't see this as a solution at all. Mr. Sysadmin, we've given the DBAs a new tool

Re: [HACKERS] Autovacuum different in 9.2.4?

2013-08-05 Thread Kevin Grittner
Alvaro Herrera alvhe...@2ndquadrant.com wrote: Joshua D. Drake escribió: That is what is confusing me, I could be cracked but messages like these: automatic vacuum of table pg_catalog.pg_attribute: could not (re)acquire exclusive lock for truncate scan Seem to be new? Yeah, those are

Re: [HACKERS] Disabling ALTER SYSTEM SET WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters

2013-08-05 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: Tom Lane escribió: What Josh seems to be concerned with in this thread is the question of whether we should support an installation *policy decision* not to allow ALTER SYSTEM SET. Not because a particular set of parameters is broken, but just

Re: [HACKERS] Moving 'hot' pages from buffer pool to heap

2013-08-05 Thread Tom Lane
Atri Sharma atri.j...@gmail.com writes: Just experimenting though.I was thinking of scenarios where a page is pinned for long period of time.My concern was that it would lead to blocking of a buffer pool slot for that entire duration. The idea is to allocate a separate data structure for

Re: [HACKERS] Disabling ALTER SYSTEM SET WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters

2013-08-05 Thread Rodrigo Gonzalez
On Mon, 5 Aug 2013 15:53:01 -0400 Alvaro Herrera alvhe...@2ndquadrant.com wrote: The other issue is that currently you can only edit a server's config if you are logged in to it. If we permit SQL-level access to that, and somebody who doesn't have access to edit the files blocks themselves

Re: [HACKERS] CREATE EVENT TRIGGER syntax

2013-08-05 Thread Dimitri Fontaine
Bruce Momjian br...@momjian.us writes: So do we want to keep that AND in the 9.3beta and 9.4 documentation? The grammar as in gram.y still allows the AND form, and I think we're used to maintain documentation that matches the code here. So I think it makes sense to remove both capabilities as we

Re: [HACKERS] Disabling ALTER SYSTEM SET WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters

2013-08-05 Thread Stefan Kaltenbrunner
On 08/05/2013 10:18 PM, Dimitri Fontaine wrote: Hi, I'm now completely lost in the current threads. Is there a single valid use case for the feature we're trying to design? Who is the target audience of this patch? wonder about that myself... Josh Berkus j...@agliodbs.com writes: I

Re: [HACKERS] Disabling ALTER SYSTEM SET WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters

2013-08-05 Thread Stefan Kaltenbrunner
On 08/05/2013 09:53 PM, Alvaro Herrera wrote: Tom Lane escribió: What Josh seems to be concerned with in this thread is the question of whether we should support an installation *policy decision* not to allow ALTER SYSTEM SET. Not because a particular set of parameters is broken, but just

Re: [HACKERS] Disabling ALTER SYSTEM SET WAS: Re: ALTER SYSTEM SET command to change postgresql.conf parameters

2013-08-05 Thread Josh Berkus
Stefan, thinking more about that - is there _ANY_ prerequisite of an application that can be completely reconfigured over a remote access protocol and solved the reliability and security challenges of that to a reasonable degree? Good question! I also think that, given the issues raised in

  1   2   >