Re: [HACKERS] error out when building pg_xlogdump with pgxs

2013-08-29 Thread Cédric Villemain
Le mercredi 28 août 2013 00:12:22 Andres Freund a écrit : Hi Alvaro, On 2013-08-27 14:47:49 -0400, Alvaro Herrera wrote: Andres Freund wrote: pg_xlogdump cannot properly be built with pgxs since it needs a sourcetree around. That already has confused some users... How about the

Re: [HACKERS] error out when building pg_xlogdump with pgxs

2013-08-29 Thread Andres Freund
On 2013-08-29 11:49:00 +0200, Cédric Villemain wrote: Le mercredi 28 août 2013 00:12:22 Andres Freund a écrit : Hi Alvaro, On 2013-08-27 14:47:49 -0400, Alvaro Herrera wrote: Andres Freund wrote: pg_xlogdump cannot properly be built with pgxs since it needs a sourcetree around.

Re: [HACKERS] error out when building pg_xlogdump with pgxs

2013-08-29 Thread Cédric Villemain
Le jeudi 29 août 2013 11:52:36 Andres Freund a écrit : On 2013-08-29 11:49:00 +0200, Cédric Villemain wrote: Le mercredi 28 août 2013 00:12:22 Andres Freund a écrit : Hi Alvaro, On 2013-08-27 14:47:49 -0400, Alvaro Herrera wrote: Andres Freund wrote: pg_xlogdump cannot

Re: [HACKERS] Extension Templates S03E11

2013-08-29 Thread Dimitri Fontaine
Dimitri Fontaine dimi...@2ndquadrant.fr writes: Alvaro Herrera alvhe...@2ndquadrant.com writes: make check in contrib/pg_upgrade should do the trick. PASSED Even after I added extension to the serial_schedule. I don't know if I need to do anything specific on that area, will wait about some

Re: [HACKERS] [v9.4] row level security

2013-08-29 Thread Oleg Bartunov
Any constraints could be covert channel. On Wed, Aug 28, 2013 at 4:17 PM, Kohei KaiGai kai...@kaigai.gr.jp wrote: 2013/8/28 Oleg Bartunov obartu...@gmail.com: btw, there is serious problem with row-level security and constraints. For example, user with low security level could use unique

Re: Spinlock implementation on x86_64 (was Re: [HACKERS] Better LWLocks with compare-and-swap (9.4))

2013-08-29 Thread Heikki Linnakangas
On 28.08.2013 20:21, Tom Lane wrote: Heikki Linnakangashlinnakan...@vmware.com writes: So, my plan is to apply the attached non-locked-tas-spin-x86_64.patch to master. But I would love to get feedback from people running different x86_64 hardware. Surely this patch should update the existing

Re: [HACKERS] [v9.4] row level security

2013-08-29 Thread Alexander Korotkov
On Wed, Aug 28, 2013 at 4:17 PM, Kohei KaiGai kai...@kaigai.gr.jp wrote: 2013/8/28 Oleg Bartunov obartu...@gmail.com: btw, there is serious problem with row-level security and constraints. For example, user with low security level could use unique constraint to know about existence of a

Re: [HACKERS] [9.3 doc fix] clarification of Solaris versions

2013-08-29 Thread MauMau
From: Alvaro Herrera alvhe...@2ndquadrant.com Robert Haas escribió: Just because OpenSolaris is discontinued doesn't mean we don't support it. It looks like it has been unsupported for ~3 years at this point - not sure if that is long enough to remove the documentation reference. That part

Re: [HACKERS] Properly initialize negative/empty cache entries in relfilenodemap

2013-08-29 Thread MauMau
Andres Freund wrote: Hi, Thanks to the valgrind instrumentation I found a small oversight in the relfilenodemap patch which is fixed in the attached patch. Great! Could anybody find the root cause for the following memory leak problem, and if possible, fix this?

Re: [HACKERS] Properly initialize negative/empty cache entries in relfilenodemap

2013-08-29 Thread Andres Freund
On 2013-08-29 21:35:13 +0900, MauMau wrote: Andres Freund wrote: Hi, Thanks to the valgrind instrumentation I found a small oversight in the relfilenodemap patch which is fixed in the attached patch. Great! Could anybody find the root cause for the following memory leak problem, and if

Re: [HACKERS] [v9.4] row level security

2013-08-29 Thread Tom Lane
Alexander Korotkov aekorot...@gmail.com writes: On Wed, Aug 28, 2013 at 4:17 PM, Kohei KaiGai kai...@kaigai.gr.jp wrote: It is out of scope for this feature. We usually calls this type of information leakage covert channel; that is not avoidable in principle. I think there is another covert

Re: [HACKERS] [v9.4] row level security

2013-08-29 Thread Kohei KaiGai
2013/8/29 Alexander Korotkov aekorot...@gmail.com: On Wed, Aug 28, 2013 at 4:17 PM, Kohei KaiGai kai...@kaigai.gr.jp wrote: 2013/8/28 Oleg Bartunov obartu...@gmail.com: btw, there is serious problem with row-level security and constraints. For example, user with low security level could

Re: [HACKERS] [v9.4] row level security

2013-08-29 Thread k...@rice.edu
On Thu, Aug 29, 2013 at 04:14:53PM +0200, Kohei KaiGai wrote: 2013/8/29 Alexander Korotkov aekorot...@gmail.com: On Wed, Aug 28, 2013 at 4:17 PM, Kohei KaiGai kai...@kaigai.gr.jp wrote: 2013/8/28 Oleg Bartunov obartu...@gmail.com: btw, there is serious problem with row-level security and

Re: [HACKERS] [v9.4] row level security

2013-08-29 Thread Kohei KaiGai
2013/8/29 k...@rice.edu k...@rice.edu: On Thu, Aug 29, 2013 at 04:14:53PM +0200, Kohei KaiGai wrote: 2013/8/29 Alexander Korotkov aekorot...@gmail.com: On Wed, Aug 28, 2013 at 4:17 PM, Kohei KaiGai kai...@kaigai.gr.jp wrote: 2013/8/28 Oleg Bartunov obartu...@gmail.com: btw, there is

Re: [HACKERS] Support for REINDEX CONCURRENTLY

2013-08-29 Thread Robert Haas
On Wed, Aug 28, 2013 at 9:02 AM, Andres Freund and...@2ndquadrant.com wrote: During swap phase, process was waiting for transactions with older snapshots than the one taken by transaction doing the swap as they might hold the old index information. I think that we can get rid of it thanks to

Re: [HACKERS] pg_system_identifier()

2013-08-29 Thread Jim Nasby
On 8/26/13 8:48 AM, Fujii Masao wrote: I think it's also noteworthy that Slony and londiste both rely on the user specifying node identifiers. They don't try to be magic about it. I think there's 2 advantages there: - Code is simpler - Users can choose a naming schema that makes sense for them

Re: [HACKERS] Master-slave visibility order

2013-08-29 Thread Robert Haas
On Wed, Aug 28, 2013 at 10:58 AM, Ants Aasma a...@cybertec.at wrote: I currently see the following courses of action: 1. Do nothing about the inconsistency, use a transient global counter for master commit order and commit record LSN for slaves. Pro: doesn't change any semantics Con:

Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-29 Thread Robert Haas
On Wed, Aug 28, 2013 at 3:10 PM, Stephen Frost sfr...@snowman.net wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: Stephen Frost sfr...@snowman.net writes: While I appreciate that there are bootstrap-type issues with this, I really don't like this idea of later stuff can just override earlier

Re: [HACKERS] [v9.4] row level security

2013-08-29 Thread Josh Berkus
Kaigai, Although I didn't touch this task by myself, my senior colleague said that we calculated some possible bandwidth of leakage as an evident when we ship supercomputer system with MAC feature for customer who worry about security. I'm not sure how to measure it. However, if we assume a

Re: [HACKERS] [v9.4] row level security

2013-08-29 Thread Stephen Frost
* Kohei KaiGai (kai...@kaigai.gr.jp) wrote: Although I didn't touch this task by myself, my senior colleague said that we calculated some possible bandwidth of leakage as an evident when we ship supercomputer system with MAC feature for customer who worry about security. I'm not sure how to

Re: [HACKERS] [v9.4] row level security

2013-08-29 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: It's one thing to day we can't solve this covert channel issue right now in this patch, but saying we don't plan to solve it at all is likely to doom the patch. I'm not sure what the solution would be, exactly. Deny permission for EXPLAIN on certain

Re: [HACKERS] [v9.4] row level security

2013-08-29 Thread Josh Berkus
I'm not sure what the solution would be, exactly. Deny permission for EXPLAIN on certain tables? That would close only one covert channel. Others were already pointed out upthread, and I'll bet there are more ... Mind you, fundamentally this is no different from allowing INSERT

Re: [HACKERS] [v9.4] row level security

2013-08-29 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: That would close only one covert channel. Others were already pointed out upthread, and I'll bet there are more ... Mind you, fundamentally this is no different from allowing INSERT permission on a table but denying SELECT, or denying SELECT on certain

Re: [HACKERS] [v9.4] row level security

2013-08-29 Thread David Fetter
On Thu, Aug 29, 2013 at 10:05:14AM -0400, Tom Lane wrote: Alexander Korotkov aekorot...@gmail.com writes: On Wed, Aug 28, 2013 at 4:17 PM, Kohei KaiGai kai...@kaigai.gr.jp wrote: It is out of scope for this feature. We usually calls this type of information leakage covert channel; that is

Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-29 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: The sorts of watered-down half-features being proposed here are not going to do anything to address that situation. If there are restrictions on what GUCs can be changed with this feature, or if the feature is disabled by default, or if

Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-29 Thread Robert Haas
On Thu, Aug 29, 2013 at 1:42 PM, Stephen Frost sfr...@snowman.net wrote: To be honest, I don't find the arguments of pgAdmin does it badly nor psql users want this ability (which I find difficult to believe) to be particularlly compelling reasons to have a dangerous implementation (even if

Re: [HACKERS] PL/pgSQL PERFORM with CTE

2013-08-29 Thread Pavel Stehule
2013/8/28 Hannu Krosing ha...@2ndquadrant.com On 08/28/2013 12:10 AM, Pavel Stehule wrote: so if I write on client side BEGIN; SELECT 1,2; SELECT 2; SELECT 3,4; END; then I expect results 1,2 2 3,4 And you are perfectly ok to discard the results Actually it

[HACKERS] Variadic aggregates vs. project policy

2013-08-29 Thread Tom Lane
So I was hacking away at supporting variadic aggregates (per an internal request at Salesforce), and had it pretty much working, when I came across this old comment in opr_sanity.sql: -- Check that there are not aggregates with the same name and different -- numbers of arguments. While not

Re: [HACKERS] PL/pgSQL PERFORM with CTE

2013-08-29 Thread Pavel Stehule
2013/8/28 Robert Haas robertmh...@gmail.com On Tue, Aug 27, 2013 at 6:10 PM, Pavel Stehule pavel.steh...@gmail.com wrote: what is magical? Stored procedures - we talk about this technology was a originally simple script moved from client side to server side. so if I write on client

Re: [HACKERS] Variadic aggregates vs. project policy

2013-08-29 Thread Pavel Stehule
2013/8/29 Tom Lane t...@sss.pgh.pa.us So I was hacking away at supporting variadic aggregates (per an internal request at Salesforce), and had it pretty much working, when I came across this old comment in opr_sanity.sql: -- Check that there are not aggregates with the same name and

Re: [HACKERS] Variadic aggregates vs. project policy

2013-08-29 Thread Tom Lane
Pavel Stehule pavel.steh...@gmail.com writes: 2013/8/29 Tom Lane t...@sss.pgh.pa.us So the question I'm now wondering about is whether this consideration makes variadic aggregates a bad idea all around, even if we don't have any built-in ones. Is the risk of user confusion (in the use of

Re: [HACKERS] Variadic aggregates vs. project policy

2013-08-29 Thread Pavel Stehule
2013/8/29 Tom Lane t...@sss.pgh.pa.us Pavel Stehule pavel.steh...@gmail.com writes: 2013/8/29 Tom Lane t...@sss.pgh.pa.us So the question I'm now wondering about is whether this consideration makes variadic aggregates a bad idea all around, even if we don't have any built-in ones. Is

Re: [HACKERS] PL/pgSQL PERFORM with CTE

2013-08-29 Thread David E. Wheeler
On Aug 29, 2013, at 1:11 PM, Pavel Stehule pavel.steh...@gmail.com wrote: I cannot to say what is good design for PL/pgSQL - only I feel so some variant of RETURN statement is not good, because semantic is significantly different. And I see a increasing inconsistency between a original ADA

Re: [HACKERS] PL/pgSQL PERFORM with CTE

2013-08-29 Thread Pavel Stehule
2013/8/29 David E. Wheeler da...@justatheory.com On Aug 29, 2013, at 1:11 PM, Pavel Stehule pavel.steh...@gmail.com wrote: I cannot to say what is good design for PL/pgSQL - only I feel so some variant of RETURN statement is not good, because semantic is significantly different. And I see

Re: [HACKERS] Master-slave visibility order

2013-08-29 Thread Ants Aasma
Hi, thanks for your reply. On Thu, Aug 29, 2013 at 6:40 PM, Robert Haas robertmh...@gmail.com wrote: I think approach #2 is dead on arrival, at least as a default policy. It essentially amounts to requiring two commit records per transaction rather than one, and I think that has no chance of

Re: [HACKERS] PL/pgSQL PERFORM with CTE

2013-08-29 Thread David E. Wheeler
On Aug 29, 2013, at 2:22 PM, Pavel Stehule pavel.steh...@gmail.com wrote: Still I don't think so correct solution is enabling a unbound SELECTs, but correct is a fix a PERFORM and remove a necessity to use a PERFORM for call of VOID functions. Well, in this thread, I believe you are the

Re: [HACKERS] Master-slave visibility order

2013-08-29 Thread Andres Freund
On 2013-08-30 00:22:49 +0300, Ants Aasma wrote: Hi, thanks for your reply. On Thu, Aug 29, 2013 at 6:40 PM, Robert Haas robertmh...@gmail.com wrote: I think approach #2 is dead on arrival, at least as a default policy. It essentially amounts to requiring two commit records per transaction

Re: [HACKERS] PL/pgSQL PERFORM with CTE

2013-08-29 Thread Andres Freund
On 2013-08-29 14:31:55 -0700, David E. Wheeler wrote: On Aug 29, 2013, at 2:22 PM, Pavel Stehule pavel.steh...@gmail.com wrote: Still I don't think so correct solution is enabling a unbound SELECTs, but correct is a fix a PERFORM and remove a necessity to use a PERFORM for call of VOID

Re: [HACKERS] Variadic aggregates vs. project policy

2013-08-29 Thread Josh Berkus
Tom, On further reflection, what the policy was actually about was not that we should forbid users from creating potentially-confusing aggregates themselves, but only that we'd avoid having any *built in* aggregates with this hazard. So maybe I'm overthinking this, and the correct reading is

Re: [HACKERS] PL/pgSQL PERFORM with CTE

2013-08-29 Thread Josh Berkus
On 08/29/2013 02:22 PM, Pavel Stehule wrote: Still I don't think so correct solution is enabling a unbound SELECTs, but correct is a fix a PERFORM and remove a necessity to use a PERFORM for call of VOID functions. You have yet to supply any arguments which support this position. Several

Re: [HACKERS] PL/pgSQL PERFORM with CTE

2013-08-29 Thread Pavel Stehule
2013/8/29 David E. Wheeler da...@justatheory.com On Aug 29, 2013, at 2:22 PM, Pavel Stehule pavel.steh...@gmail.com wrote: Still I don't think so correct solution is enabling a unbound SELECTs, but correct is a fix a PERFORM and remove a necessity to use a PERFORM for call of VOID

Re: [HACKERS] PL/pgSQL PERFORM with CTE

2013-08-29 Thread Hannu Krosing
On 08/29/2013 11:01 PM, David E. Wheeler wrote: On Aug 29, 2013, at 1:11 PM, Pavel Stehule pavel.steh...@gmail.com wrote: I cannot to say what is good design for PL/pgSQL - only I feel so some variant of RETURN statement is not good, because semantic is significantly different. And I see a

Re: [HACKERS] PL/pgSQL PERFORM with CTE

2013-08-29 Thread David E. Wheeler
On Aug 29, 2013, at 2:41 PM, Pavel Stehule pavel.steh...@gmail.com wrote: I am thinking, so I propose a enough solution for you - when you use CTE for execution of VOID function, then result vill be VOID set, what we can accept as undefined result, and in this case a PERFORM should not be

Re: [HACKERS] PL/pgSQL PERFORM with CTE

2013-08-29 Thread Andres Freund
On 2013-08-29 14:40:24 -0700, Josh Berkus wrote: On 08/29/2013 02:22 PM, Pavel Stehule wrote: Still I don't think so correct solution is enabling a unbound SELECTs, but correct is a fix a PERFORM and remove a necessity to use a PERFORM for call of VOID functions. You have yet to supply

Re: [HACKERS] PL/pgSQL PERFORM with CTE

2013-08-29 Thread Pavel Stehule
2013/8/29 Josh Berkus j...@agliodbs.com On 08/29/2013 02:22 PM, Pavel Stehule wrote: Still I don't think so correct solution is enabling a unbound SELECTs, but correct is a fix a PERFORM and remove a necessity to use a PERFORM for call of VOID functions. You have yet to supply any

Re: [HACKERS] PL/pgSQL PERFORM with CTE

2013-08-29 Thread David E. Wheeler
On Aug 29, 2013, at 2:48 PM, Andres Freund and...@2ndquadrant.com wrote: You have yet to supply any arguments which support this position. I am not convinced that's enough of a reason, but the requirement to use PERFORM for SELECTs that aren't stored anywhere actually has prevented bugs for

Re: [HACKERS] PL/pgSQL PERFORM with CTE

2013-08-29 Thread Pavel Stehule
2013/8/29 Pavel Stehule pavel.steh...@gmail.com 2013/8/29 Josh Berkus j...@agliodbs.com On 08/29/2013 02:22 PM, Pavel Stehule wrote: Still I don't think so correct solution is enabling a unbound SELECTs, but correct is a fix a PERFORM and remove a necessity to use a PERFORM for call

Re: [HACKERS] Master-slave visibility order

2013-08-29 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: But I think that actually coordinating a consistent visibility order between commit, wal insertion and the procarray would have bigger scalability impact than the second record. I might be missing some clever tricks here though. Yeah. ISTM the

Re: [HACKERS] PL/pgSQL PERFORM with CTE

2013-08-29 Thread Andres Freund
On 2013-08-29 23:54:28 +0200, Pavel Stehule wrote: No think so PERFORM is a significant problem. A mayor problem for beginners is usually a fact, so PL/pgSQL is ALGOL like languages - and they don't know with these languages. Second problem is missing a more dynamic data structures. Next a

Re: [HACKERS] Variadic aggregates vs. project policy

2013-08-29 Thread Alvaro Herrera
Josh Berkus wrote: Tom, On further reflection, what the policy was actually about was not that we should forbid users from creating potentially-confusing aggregates themselves, but only that we'd avoid having any *built in* aggregates with this hazard. So maybe I'm overthinking this,

Re: [HACKERS] PL/pgSQL PERFORM with CTE

2013-08-29 Thread Pavel Stehule
2013/8/29 David E. Wheeler da...@justatheory.com On Aug 29, 2013, at 2:48 PM, Andres Freund and...@2ndquadrant.com wrote: You have yet to supply any arguments which support this position. I am not convinced that's enough of a reason, but the requirement to use PERFORM for SELECTs that

Re: [HACKERS] PL/pgSQL PERFORM with CTE

2013-08-29 Thread Andrew Dunstan
On 08/29/2013 05:31 PM, David E. Wheeler wrote: On Aug 29, 2013, at 2:22 PM, Pavel Stehule pavel.steh...@gmail.com wrote: Still I don't think so correct solution is enabling a unbound SELECTs, but correct is a fix a PERFORM and remove a necessity to use a PERFORM for call of VOID functions.

Re: [HACKERS] PL/pgSQL PERFORM with CTE

2013-08-29 Thread Pavel Stehule
2013/8/29 Andres Freund and...@2ndquadrant.com On 2013-08-29 23:54:28 +0200, Pavel Stehule wrote: No think so PERFORM is a significant problem. A mayor problem for beginners is usually a fact, so PL/pgSQL is ALGOL like languages - and they don't know with these languages. Second

Re: [HACKERS] Master-slave visibility order

2013-08-29 Thread Ants Aasma
On Fri, Aug 30, 2013 at 12:33 AM, Andres Freund and...@2ndquadrant.com wrote: FWIW, WAL is still the major bottleneck for INSERT heavy workloads. The per CPU overhead actually minimally increased (at least in my tests), it just scales noticeably better than before. Interesting. Do you have any

Re: [HACKERS] Variadic aggregates vs. project policy

2013-08-29 Thread Andres Freund
On 2013-08-29 15:55:13 -0400, Tom Lane wrote: So I was hacking away at supporting variadic aggregates (per an internal request at Salesforce), and had it pretty much working, when I came across this old comment in opr_sanity.sql: -- Check that there are not aggregates with the same name and

Re: [HACKERS] Master-slave visibility order

2013-08-29 Thread Ants Aasma
On Fri, Aug 30, 2013 at 12:59 AM, Tom Lane t...@sss.pgh.pa.us wrote: Andres Freund and...@2ndquadrant.com writes: But I think that actually coordinating a consistent visibility order between commit, wal insertion and the procarray would have bigger scalability impact than the second record. I

Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-29 Thread Andres Freund
On 2013-08-29 15:07:35 -0400, Robert Haas wrote: On Thu, Aug 29, 2013 at 1:42 PM, Stephen Frost sfr...@snowman.net wrote: To be honest, I don't find the arguments of pgAdmin does it badly nor psql users want this ability (which I find difficult to believe) to be particularlly compelling

Re: [HACKERS] Variadic aggregates vs. project policy

2013-08-29 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-08-29 15:55:13 -0400, Tom Lane wrote: For context see the thread starting here: http://www.postgresql.org/message-id/aanlktikv5ok2ts8t6v+gsapte3n6tjq1jpphmzhg2...@mail.gmail.com In that thread we agreed that this policy might be rather

[HACKERS] WAL CPU overhead/optimization (was Master-slave visibility order)

2013-08-29 Thread Andres Freund
On 2013-08-30 01:10:40 +0300, Ants Aasma wrote: On Fri, Aug 30, 2013 at 12:33 AM, Andres Freund and...@2ndquadrant.com wrote: FWIW, WAL is still the major bottleneck for INSERT heavy workloads. The per CPU overhead actually minimally increased (at least in my tests), it just scales

Re: [HACKERS] Variadic aggregates vs. project policy

2013-08-29 Thread Andres Freund
On 2013-08-29 18:29:34 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-08-29 15:55:13 -0400, Tom Lane wrote: For context see the thread starting here: http://www.postgresql.org/message-id/aanlktikv5ok2ts8t6v+gsapte3n6tjq1jpphmzhg2...@mail.gmail.com In that

Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-29 Thread Stephen Frost
On Thursday, August 29, 2013, Andres Freund wrote: On 2013-08-29 15:07:35 -0400, Robert Haas wrote: I don't really see a compelling reason why it can't or shouldn't be in core. But having something better in contrib would still be an improvement on the status quo. I don't see much

Re: [HACKERS] Variadic aggregates vs. project policy

2013-08-29 Thread Andrew Dunstan
On 08/29/2013 05:37 PM, Josh Berkus wrote: Tom, On further reflection, what the policy was actually about was not that we should forbid users from creating potentially-confusing aggregates themselves, but only that we'd avoid having any *built in* aggregates with this hazard. So maybe I'm

Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-29 Thread Andres Freund
On 2013-08-29 18:42:13 -0400, Stephen Frost wrote: On Thursday, August 29, 2013, Andres Freund wrote: The energy wasted in a good part of this massive 550+ messages thread is truly saddening. We all (c|sh)ould have spent that time making PG more awesome instead. Perhaps not understood by

Re: [HACKERS] PL/pgSQL PERFORM with CTE

2013-08-29 Thread Hannu Krosing
On 08/30/2013 12:04 AM, Pavel Stehule wrote: 2013/8/29 David E. Wheeler da...@justatheory.com mailto:da...@justatheory.com On Aug 29, 2013, at 2:48 PM, Andres Freund and...@2ndquadrant.com mailto:and...@2ndquadrant.com wrote: You have yet to supply any arguments which

Re: [HACKERS] WAL CPU overhead/optimization (was Master-slave visibility order)

2013-08-29 Thread Ants Aasma
On Fri, Aug 30, 2013 at 1:30 AM, Andres Freund and...@2ndquadrant.com wrote: On 2013-08-30 01:10:40 +0300, Ants Aasma wrote: On Fri, Aug 30, 2013 at 12:33 AM, Andres Freund and...@2ndquadrant.com wrote: FWIW, WAL is still the major bottleneck for INSERT heavy workloads. The per CPU

Re: [HACKERS] WAL CPU overhead/optimization (was Master-slave visibility order)

2013-08-29 Thread Andres Freund
On 2013-08-30 02:53:54 +0300, Ants Aasma wrote: On Fri, Aug 30, 2013 at 1:30 AM, Andres Freund and...@2ndquadrant.com wrote: On 2013-08-30 01:10:40 +0300, Ants Aasma wrote: On Fri, Aug 30, 2013 at 12:33 AM, Andres Freund and...@2ndquadrant.com wrote: FWIW, WAL is still the major

Re: [HACKERS] Properly initialize negative/empty cache entries in relfilenodemap

2013-08-29 Thread didier
Hi, On Thu, Aug 29, 2013 at 2:35 PM, MauMau maumau...@gmail.com wrote: Great! Could anybody find the root cause for the following memory leak problem, and if possible, fix this? http://www.postgresql.org/**message-id/**214653D8DF574BFEAA6ED53E545E99**

Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-29 Thread Stephen Frost
On Thursday, August 29, 2013, Andres Freund wrote: To quote Robert two mails up: Huh? The problem with adminpack is that it doesn't let you modify individual configuration settings. All you can do is rewrite an entire file. That's clearly fixable. I guess somebody could write a

Re: [HACKERS] dynamic shared memory

2013-08-29 Thread Jim Nasby
On 8/13/13 8:09 PM, Robert Haas wrote: is removed, the segment automatically goes away (we could allow for server-lifespan segments as well with only trivial changes, but I'm not sure whether there are compelling use cases for that). To clarify... you're talking something that would

Re: [HACKERS] WAL CPU overhead/optimization (was Master-slave visibility order)

2013-08-29 Thread Ants Aasma
On Fri, Aug 30, 2013 at 3:02 AM, Andres Freund and...@2ndquadrant.com wrote: I am not sure hot cache large buffer performance is really the interesting case. Most of the XLogInsert()s are pretty small in the common workloads. I vaguely recall trying 8 and getting worse performance on many

Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-29 Thread Robert Haas
On Thu, Aug 29, 2013 at 8:07 PM, Stephen Frost sfr...@snowman.net wrote: Having 400 emails about it means it's contentious. That's quite different from having a large demand. It does speak to the author's persistence as well, but that shouldn't be a surprise. Yet you can't ignore the fact that

Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-29 Thread Stephen Frost
Robert, Was working on replying to this, but got distracted.. * Robert Haas (robertmh...@gmail.com) wrote: To be honest, I think the argument that this is dangerous is pretty ridiculous. AFAICS, the only argument anyone's advanced for this being dangerous is the theory that you might

Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-29 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Thu, Aug 29, 2013 at 8:07 PM, Stephen Frost sfr...@snowman.net wrote: I'm not talking about malicious DBAs but rather a generally knowledgable DBA who changed shared_buffers up too high and then leaves on vacation, while the OPs guys need to

Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-29 Thread Peter Geoghegan
On Thu, Aug 29, 2013 at 9:13 AM, Robert Haas robertmh...@gmail.com wrote: I think the goals of this patch should be to (1) let users of other clients manipulate the startup configuration just as easily as we can already do using pgAdmin and (2) remove some of the concurrency hazards that

[HACKERS] Compression of full-page-writes

2013-08-29 Thread Fujii Masao
Hi, Attached patch adds new GUC parameter 'compress_backup_block'. When this parameter is enabled, the server just compresses FPW (full-page-writes) in WAL by using pglz_compress() before inserting it to the WAL buffers. Then, the compressed FPW is decompressed in recovery. This is very simple

Re: [HACKERS] Compression of full-page-writes

2013-08-29 Thread Satoshi Nagayasu
(2013/08/30 11:55), Fujii Masao wrote: Hi, Attached patch adds new GUC parameter 'compress_backup_block'. When this parameter is enabled, the server just compresses FPW (full-page-writes) in WAL by using pglz_compress() before inserting it to the WAL buffers. Then, the compressed FPW is

Re: [HACKERS] Compression of full-page-writes

2013-08-29 Thread Satoshi Nagayasu
(2013/08/30 12:07), Satoshi Nagayasu wrote: (2013/08/30 11:55), Fujii Masao wrote: Hi, Attached patch adds new GUC parameter 'compress_backup_block'. When this parameter is enabled, the server just compresses FPW (full-page-writes) in WAL by using pglz_compress() before inserting it to the

Re: [HACKERS] Compression of full-page-writes

2013-08-29 Thread Peter Geoghegan
On Thu, Aug 29, 2013 at 7:55 PM, Fujii Masao masao.fu...@gmail.com wrote: [the amount of WAL generated during running pgbench] 4302 MB (compress_backup_block = off) 1521 MB (compress_backup_block = on) Interesting. I wonder, what is the impact on recovery time under the same conditions?

Re: [HACKERS] Compression of full-page-writes

2013-08-29 Thread Amit Kapila
On Fri, Aug 30, 2013 at 8:25 AM, Fujii Masao masao.fu...@gmail.com wrote: Hi, Attached patch adds new GUC parameter 'compress_backup_block'. When this parameter is enabled, the server just compresses FPW (full-page-writes) in WAL by using pglz_compress() before inserting it to the WAL

Re: [HACKERS] Variadic aggregates vs. project policy

2013-08-29 Thread Pavel Stehule
2013/8/30 Andrew Dunstan and...@dunslane.net On 08/29/2013 05:37 PM, Josh Berkus wrote: Tom, On further reflection, what the policy was actually about was not that we should forbid users from creating potentially-confusing aggregates themselves, but only that we'd avoid having any *built

Re: [HACKERS] Compression of full-page-writes

2013-08-29 Thread KONDO Mitsumasa
(2013/08/30 11:55), Fujii Masao wrote: * Benchmark pgbench -c 32 -j 4 -T 900 -M prepared scaling factor: 100 checkpoint_segments = 1024 checkpoint_timeout = 5min (every checkpoint during benchmark were triggered by checkpoint_timeout) Did you execute munual checkpoint before

Re: [HACKERS] Compression of full-page-writes

2013-08-29 Thread Nikhil Sontakke
Hi Fujii-san, I must be missing something really trivial, but why not try to compress all types of WAL blocks and not just FPW? Regards, Nikhils On Fri, Aug 30, 2013 at 8:25 AM, Fujii Masao masao.fu...@gmail.com wrote: Hi, Attached patch adds new GUC parameter 'compress_backup_block'.

[HACKERS] proposal: allow a execution of VOID function without PERFORM keyword.

2013-08-29 Thread Pavel Stehule
Hello A perform keyword is semantically useless for VOID functions. So it can be optional for start of any function. The function type cannot be checked in compile time - so result will be checked in execution time. I hope so 9.4 will be support deep checking in some step between compilation and

Re: [HACKERS] Compression of full-page-writes

2013-08-29 Thread Fujii Masao
On Fri, Aug 30, 2013 at 12:43 PM, Peter Geoghegan p...@heroku.com wrote: On Thu, Aug 29, 2013 at 7:55 PM, Fujii Masao masao.fu...@gmail.com wrote: [the amount of WAL generated during running pgbench] 4302 MB (compress_backup_block = off) 1521 MB (compress_backup_block = on)

Re: [HACKERS] Compression of full-page-writes

2013-08-29 Thread Michael Paquier
On Fri, Aug 30, 2013 at 11:55 AM, Fujii Masao masao.fu...@gmail.com wrote: My idea is very simple, just compress FPW because FPW is a big part of WAL. I used pglz_compress() as a compression method, but you might think that other method is better. We can add something like FPW-compression-hook