[HACKERS] Review of GetUserId() Usage

2014-09-23 Thread Stephen Frost
Greetings, While looking through the GetUserId() usage in the backend, I couldn't help but notice a few rather questionable cases that, in my view, should be cleaned up. As a reminder, GetUserId() returns the OID of the user we are generally operating as (eg: whatever the 'role' is, as

Re: [HACKERS] Commitfest status

2014-09-23 Thread Heikki Linnakangas
On 09/20/2014 06:54 AM, Michael Paquier wrote: CF3 is actually over for a couple of days, There are different opinions on when a commitfest is over. In my opinion, the point of a commitfest is that every patch that someone submits gets enough review so that the patch author knows what he

Re: [HACKERS] RLS feature has been committed

2014-09-23 Thread Heikki Linnakangas
On 09/23/2014 09:15 AM, Peter Geoghegan wrote: On Mon, Sep 22, 2014 at 8:25 PM, Robert Haas robertmh...@gmail.com wrote: Should RLS be reverted, and revisited in a future CF? IMHO, that would be entirely appropriate. That seems pretty straightforward, then. I think that it will have to be

[HACKERS] Extending COPY TO

2014-09-23 Thread Andrea Riciputi
Hi all, it’s my first time here, so please let me know if I’m doing something wrong. I’m a developer, heavy PG user, but I’ve never hacked it before. Last week at work we had to produce a quite big CSV data file which should be used as input by another piece of software. Since the file must be

Re: [HACKERS] RLS feature has been committed

2014-09-23 Thread Peter Geoghegan
On Mon, Sep 22, 2014 at 11:45 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: To make sure this doesn't just slip by without sufficient review, I'll add this to the next commitfest. It's a bit unusual to have a commitfest entry for something that's already been committed, but that's fine.

Re: [HACKERS] Should we excise the remnants of borland cc support?

2014-09-23 Thread Magnus Hagander
On Sep 23, 2014 2:51 AM, Tom Lane t...@sss.pgh.pa.us wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-09-20 10:03:43 -0400, Andrew Dunstan wrote: I thought the Borland stuff was there only so we could build client libraries for use with things like Delphi. FWIW I got offlist

Re: [HACKERS] RLS feature has been committed

2014-09-23 Thread Heikki Linnakangas
Some random comments after a quick read-through of the patch: * Missing documentation. For a major feature like this, reference pages for the CREATE/DROP POLICY statements are not sufficient. We'll need a whole new chapter for this. * In CREATE POLICY, the USING and WITH CHECK keywords are

Re: [HACKERS] better atomics - v0.6

2014-09-23 Thread Oskari Saarenmaa
23.09.2014, 00:01, Andres Freund kirjoitti: I've finally managed to incorporate (all the?) feedback I got for 0.5. Imo the current version looks pretty good. Most notable changes: * Lots of comment improvements * code moved out of storage/ into port/ * separated the s_lock.h changes into its

Re: [HACKERS] better atomics - v0.6

2014-09-23 Thread Andres Freund
On 2014-09-23 13:50:28 +0300, Oskari Saarenmaa wrote: 23.09.2014, 00:01, Andres Freund kirjoitti: I've finally managed to incorporate (all the?) feedback I got for 0.5. Imo the current version looks pretty good. Most notable changes: * Lots of comment improvements * code moved out of

Re: [HACKERS] better atomics - v0.6

2014-09-23 Thread Andres Freund
Hi, On 2014-09-23 13:50:28 +0300, Oskari Saarenmaa wrote: 23.09.2014, 00:01, Andres Freund kirjoitti: I've finally managed to incorporate (all the?) feedback I got for 0.5. Imo the current version looks pretty good. Most notable changes: * Lots of comment improvements * code moved out of

Re: [HACKERS] RLS feature has been committed

2014-09-23 Thread Stephen Frost
Heikki, * Heikki Linnakangas (hlinnakan...@vmware.com) wrote: Some random comments after a quick read-through of the patch: Glad you were able to find a bit of time to take a look, thanks! * Missing documentation. For a major feature like this, reference pages for the CREATE/DROP POLICY

Re: [HACKERS] RLS feature has been committed

2014-09-23 Thread Dave Page
On Tue, Sep 23, 2014 at 4:25 AM, Robert Haas robertmh...@gmail.com wrote: On Mon, Sep 22, 2014 at 7:22 PM, Peter Geoghegan p...@heroku.com wrote: On Mon, Sep 22, 2014 at 4:02 PM, Andres Freund and...@anarazel.de wrote: This patch has been pushed in a clear violation of established policy.

Re: [HACKERS] Index scan optimization

2014-09-23 Thread Rajeev rastogi
On 22 September 2014 19:17, Heikki Linnakangas wrote: On 09/22/2014 04:45 PM, Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: On 09/22/2014 07:47 AM, Rajeev rastogi wrote: So my proposal is to skip the condition check on the first scan key condition for every tuple.

[HACKERS] proposal: adding a GUC for BAS_BULKREAD strategy

2014-09-23 Thread didier
Hi, Currently the value is hard code to NBuffers / 4 but ISTM that with bigger shared_buffer it's too much, ie even with a DB 10 to 20 time the memory size there's a lot of tables under this limit and nightly batch reports are trashing the shared buffers cache as if there's no tomorrow.

Re: [HACKERS] proposal: adding a GUC for BAS_BULKREAD strategy

2014-09-23 Thread Andres Freund
Hi, On 2014-09-23 14:47:33 +0200, didier wrote: Currently the value is hard code to NBuffers / 4 but ISTM that with bigger shared_buffer it's too much, ie even with a DB 10 to 20 time the memory size there's a lot of tables under this limit and nightly batch reports are trashing the shared

Re: [HACKERS] RLS feature has been committed

2014-09-23 Thread Andres Freund
On 2014-09-23 13:23:32 +0100, Dave Page wrote: Just to be clear here, the *only* issue we should even be discussing is whether the patch should or should not have been committed in the face of those objections. As Josh has also noted, the commitfest process was never meant to constrain what

Re: [HACKERS] RLS feature has been committed

2014-09-23 Thread Andres Freund
On 2014-09-22 21:38:17 -0700, David G Johnston wrote: Robert Haas wrote It's difficult to imagine a more flagrant violation of process than committing a patch without any warning and without even *commenting* on the fact that clear objections to commit were made on a public mailing list.

Re: [HACKERS] RLS feature has been committed

2014-09-23 Thread Robert Haas
On Tue, Sep 23, 2014 at 9:00 AM, Andres Freund and...@anarazel.de wrote: I think it's obvious that a committer doesn't need to wait till some later commitfest to commit patches that have since gotten enough review or are uncontroversial. Neither is the case here. Right. I mean, the

Re: [HACKERS] Review of GetUserId() Usage

2014-09-23 Thread Alvaro Herrera
Stephen Frost wrote: pgstat_get_backend_current_activity() Used to decide if the current activity string should be returned or not. In my view, this is a clear case which should be addressed through has_privs_of_role() instead of requiring the user to SET ROLE to each role

Re: [HACKERS] RLS feature has been committed

2014-09-23 Thread Robert Haas
On Tue, Sep 23, 2014 at 12:38 AM, David G Johnston david.g.johns...@gmail.com wrote: I'm of a mind to agree that this shouldn't have been committed...but I'm not seeing where Stephen has done sufficient wrong to justify crucifixion. Especially since everything is being done publicly and you are

Re: [HACKERS] add modulo (%) operator to pgbench

2014-09-23 Thread Fabien COELHO
Hello Stephen, But this is not convincing. Adding a unary function with a clean syntax indeed requires doing something with the parser. Based on the discussion so far, it sounds like you're coming around to agree with Robert (as I'm leaning towards also) that we'd be better off building a

Re: [HACKERS] RLS feature has been committed

2014-09-23 Thread David Johnston
On Tue, Sep 23, 2014 at 9:09 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-22 21:38:17 -0700, David G Johnston wrote: Robert Haas wrote It's difficult to imagine a more flagrant violation of process than committing a patch without any warning and without even *commenting*

Re: [HACKERS] tick buildfarm failure

2014-09-23 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: To be a bit more clear- why is it safe to change the contents if the equal() function returns 'false'? I'm guessing the answer is locking- whereby such a change would imply a lock was acquired on the relation by another backend, which shouldn't

Re: [HACKERS] add modulo (%) operator to pgbench

2014-09-23 Thread Tom Lane
Fabien COELHO coe...@cri.ensmp.fr writes: So my opinion is that this small modulo operator patch is both useful and harmless, so it should be committed. You've really failed to make that case --- in particular, AFAICS there is not even consensus on the exact semantics that the operator should

Re: [HACKERS] tick buildfarm failure

2014-09-23 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Stephen Frost sfr...@snowman.net writes: To be a bit more clear- why is it safe to change the contents if the equal() function returns 'false'? I'm guessing the answer is locking- whereby such a change would imply a lock was acquired on the

Re: [HACKERS] add modulo (%) operator to pgbench

2014-09-23 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Fabien COELHO coe...@cri.ensmp.fr writes: So my opinion is that this small modulo operator patch is both useful and harmless, so it should be committed. You've really failed to make that case --- in particular, AFAICS there is not even consensus on

[HACKERS] Refactoring code for sync node detection (was: Support for N synchronous standby servers)

2014-09-23 Thread Michael Paquier
On Sat, Sep 20, 2014 at 1:16 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Fri, Sep 19, 2014 at 12:18 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Sep 16, 2014 at 2:19 PM, Michael Paquier michael.paqu...@gmail.com wrote: - A patch refactoring code for pg_stat_get_wal_senders

Re: [HACKERS] RLS feature has been committed

2014-09-23 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: I'd be happy to discuss this with Stephen, either in person, by phone, or over public or private email. Please understand that I'm not ignoring you, the conversation, or the concern. I was asked (by other community members) to not comment

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-23 Thread Robert Haas
On Tue, Sep 23, 2014 at 1:23 AM, Stephen Frost sfr...@snowman.net wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: Stephen Frost sfr...@snowman.net writes: To clarify- we'll simply swap from (essentially) floor() to ceil() for handling all GUC_with_unit to internal_unit conversions, document

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-23 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Three people have voted for making it an *error* to supply a value that needs to be rounded, instead of changing the rounding behavior. Votes or no votes, that's a horrible idea; it breaks the design goal that users shouldn't need to remember the

Re: [HACKERS] Scaling shared buffer eviction

2014-09-23 Thread Robert Haas
On Fri, Sep 19, 2014 at 7:21 AM, Amit Kapila amit.kapil...@gmail.com wrote: Specific numbers of both the configurations for which I have posted data in previous mail are as follows: Scale Factor - 800 Shared_Buffers - 12286MB (Total db size is 12288MB) Client and Thread Count = 64

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-23 Thread Robert Haas
On Tue, Sep 23, 2014 at 10:29 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Three people have voted for making it an *error* to supply a value that needs to be rounded, instead of changing the rounding behavior. Votes or no votes, that's a horrible idea; it

Re: [HACKERS] Scaling shared buffer eviction

2014-09-23 Thread Andres Freund
Hi, On 2014-09-23 10:31:24 -0400, Robert Haas wrote: I suggest we count these things: 1. The number of buffers the reclaimer has put back on the free list. 2. The number of times a backend has run the clocksweep. 3. The number of buffers past which the reclaimer has advanced the clock

Re: [HACKERS] Scaling shared buffer eviction

2014-09-23 Thread Robert Haas
On Tue, Sep 23, 2014 at 10:31 AM, Robert Haas robertmh...@gmail.com wrote: [ review ] Oh, by the way, I noticed that this patch breaks pg_buffercache. If we're going to have 128 lock partitions, we need to bump MAX_SIMUL_LWLOCKS. But this gets at another point: the way we're benchmarking this

Re: [HACKERS] RLS feature has been committed

2014-09-23 Thread Dave Page
On Tue, Sep 23, 2014 at 2:00 PM, Andres Freund and...@anarazel.de wrote: On 2014-09-23 13:23:32 +0100, Dave Page wrote: Just to be clear here, the *only* issue we should even be discussing is whether the patch should or should not have been committed in the face of those objections. As Josh

Re: [HACKERS] RLS feature has been committed

2014-09-23 Thread Robert Haas
On Tue, Sep 23, 2014 at 11:16 AM, Dave Page dp...@pgadmin.org wrote: Regardless of what Robert may feel, review should only generally be *expected* during a commitfest, but it can be done at any time. Committers are free to commit at any time. The process was never intended to restrict what

Re: [HACKERS] RLS feature has been committed

2014-09-23 Thread Dave Page
On Tue, Sep 23, 2014 at 4:19 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Sep 23, 2014 at 11:16 AM, Dave Page dp...@pgadmin.org wrote: Regardless of what Robert may feel, review should only generally be *expected* during a commitfest, but it can be done at any time. Committers are free

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-23 Thread Andrew Dunstan
On 09/23/2014 10:29 AM, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: Three people have voted for making it an *error* to supply a value that needs to be rounded, instead of changing the rounding behavior. Votes or no votes, that's a horrible idea; it breaks the design goal that

Re: [HACKERS] RLS feature has been committed

2014-09-23 Thread Andres Freund
On 2014-09-23 16:16:18 +0100, Dave Page wrote: On Tue, Sep 23, 2014 at 2:00 PM, Andres Freund and...@anarazel.de wrote: On 2014-09-23 13:23:32 +0100, Dave Page wrote: Just to be clear here, the *only* issue we should even be discussing is whether the patch should or should not have been

Re: [HACKERS] RLS feature has been committed

2014-09-23 Thread Andrew Dunstan
On 09/23/2014 11:19 AM, Robert Haas wrote: On Tue, Sep 23, 2014 at 11:16 AM, Dave Page dp...@pgadmin.org wrote: Regardless of what Robert may feel, review should only generally be *expected* during a commitfest, but it can be done at any time. Committers are free to commit at any time. The

Re: [HACKERS] RLS feature has been committed

2014-09-23 Thread Robert Haas
On Tue, Sep 23, 2014 at 11:23 AM, Dave Page dp...@pgadmin.org wrote: On Tue, Sep 23, 2014 at 4:19 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Sep 23, 2014 at 11:16 AM, Dave Page dp...@pgadmin.org wrote: Regardless of what Robert may feel, review should only generally be *expected*

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-23 Thread David Johnston
On Tue, Sep 23, 2014 at 1:30 AM, Tom Lane t...@sss.pgh.pa.us wrote: David Johnston david.g.johns...@gmail.com writes: My original concern was things that are rounded to zero now will not be in 9.5 if the non-error solution is chosen. The risk profile is extremely small but it is not

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-23 Thread Heikki Linnakangas
On 09/23/2014 06:23 PM, Andrew Dunstan wrote: On 09/23/2014 10:29 AM, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: Three people have voted for making it an *error* to supply a value that needs to be rounded, instead of changing the rounding behavior. Votes or no votes, that's a

[HACKERS] JsonbValue to Jsonb conversion

2014-09-23 Thread Dmitry Dolgov
Hi all, I'm faced with some troubles about the jsonb implementation, and I hope I'll get little advice =) If I understand correctly, an abstract function for jsonb modification should have the following stages: Jsonb - JsonbValue - Modification - JsonbValue - Jsonb One can convert the

Re: [HACKERS] Hide 'Execution time' in EXPLAIN (COSTS OFF)

2014-09-23 Thread Robert Haas
On Sat, Sep 20, 2014 at 4:13 PM, Christoph Berg c...@df7cb.de wrote: there's another problem in this area: 9.4 adds Planning time to the EXPLAIN output. If you don't want to see that, you need to use (costs off), but this, well, also disables the costs. If you are running regression tests to

Re: [HACKERS] delta relations in AFTER triggers

2014-09-23 Thread Heikki Linnakangas
On 09/15/2014 05:25 PM, Kevin Grittner wrote: Tom Lane t...@sss.pgh.pa.us wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: On 08/30/2014 12:15 AM, Kevin Grittner wrote: If we were to go with the hooks as you propose, we would still need to take the information from TriggerData and

Re: [HACKERS] delta relations in AFTER triggers

2014-09-23 Thread Robert Haas
On Tue, Sep 23, 2014 at 12:46 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 09/15/2014 05:25 PM, Kevin Grittner wrote: Now, how do we make the tuplestores work similarly? Here's what I think we should do: Add a new p_tableref_hook function pointer, similar to p_paramref_hook.

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-23 Thread Florian Weimer
* Ants Aasma: CRC has exactly one hardware implementation in general purpose CPU's I'm pretty sure that's not true. Many general purpose CPUs have CRC circuity, and there must be some which also expose them as instructions. and Intel has a patent on the techniques they used to implement it.

Re: [HACKERS] Hide 'Execution time' in EXPLAIN (COSTS OFF)

2014-09-23 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Sat, Sep 20, 2014 at 4:13 PM, Christoph Berg c...@df7cb.de wrote: Can we have EXPLAIN (timing off) in 9.4+ hide the Planning time line? That would even be backwards compatible with 9.x where it would be a no-op. I don't think that'll work becuase:

Re: [HACKERS] delta relations in AFTER triggers

2014-09-23 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Sep 23, 2014 at 12:46 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Now, how do we make the tuplestores work similarly? Here's what I think we should do: Add a new p_tableref_hook function pointer, similar to p_paramref_hook.

Re: [HACKERS] Hide 'Execution time' in EXPLAIN (COSTS OFF)

2014-09-23 Thread Robert Haas
On Tue, Sep 23, 2014 at 1:32 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Sat, Sep 20, 2014 at 4:13 PM, Christoph Berg c...@df7cb.de wrote: Can we have EXPLAIN (timing off) in 9.4+ hide the Planning time line? That would even be backwards compatible with

Re: [HACKERS] add modulo (%) operator to pgbench

2014-09-23 Thread Fabien COELHO
So my opinion is that this small modulo operator patch is both useful and harmless, so it should be committed. You've really failed to make that case --- in particular, AFAICS there is not even consensus on the exact semantics that the operator should have. There is. Basically whatever with

[HACKERS] Replication identifiers, take 3

2014-09-23 Thread Andres Freund
Hi, I've previously started two threads about replication identifiers. Check http://archives.postgresql.org/message-id/20131114172632.GE7522%40alap2.anarazel.de and http://archives.postgresql.org/message-id/20131211153833.GB25227%40awork2.anarazel.de . The've also been discussed in the course of

Re: [HACKERS] RLS feature has been committed

2014-09-23 Thread Gregory Smith
On 9/23/14, 9:00 AM, Andres Freund wrote: I also think committers need to be much more careful when committing patches which they (or their employer) appear to have a business interest in. Rushing ahead to commit the patch of somebody 'unrelated' leaves a completely different taste than

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-09-23 Thread Alvaro Herrera
Alvaro Herrera wrote: Heikki Linnakangas wrote: If you add a new datatype, and define b-tree operators for it, what is required to create a minmax opclass for it? Would it be possible to generalize the functions in brin_minmax.c so that they can be reused for any datatype (with b-tree

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-23 Thread Greg Stark
Fwiw I agree with TL2. The simplest, least surprising behaviour to explain to users is to say we round to nearest and if that means we rounded to zero (or another special value) we throw an error. We could list the minimum value in pg_settings and maybe in documentation. -- greg

Re: [HACKERS] add modulo (%) operator to pgbench

2014-09-23 Thread Heikki Linnakangas
On 09/23/2014 09:15 PM, Fabien COELHO wrote: So I'm inclined to reject rather than put in something that may cause surprises down the road. The usefulness doesn't seem great enough to take that risk. Marked as Returned with feedback. If you reject it, you can also remove the gaussian and

Re: [HACKERS] delta relations in AFTER triggers

2014-09-23 Thread Heikki Linnakangas
On 09/23/2014 08:51 PM, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Sep 23, 2014 at 12:46 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Now, how do we make the tuplestores work similarly? Here's what I think we should do: Add a new p_tableref_hook function

Re: [HACKERS] RLS feature has been committed

2014-09-23 Thread Josh Berkus
On 09/22/2014 08:23 PM, Amit Kapila wrote: Who decides if the patch is adequately reviewed? Author, Committer or Reviewer? In CF, that is comparatively clear that once Reviewer is satisfied, he marks the patch as Ready For Committer and then Committer picks up and if he is satisfied with

Re: [HACKERS] Hide 'Execution time' in EXPLAIN (COSTS OFF)

2014-09-23 Thread Christoph Berg
Re: Tom Lane 2014-09-23 15155.1411493...@sss.pgh.pa.us Robert Haas robertmh...@gmail.com writes: On Sat, Sep 20, 2014 at 4:13 PM, Christoph Berg c...@df7cb.de wrote: Can we have EXPLAIN (timing off) in 9.4+ hide the Planning time line? That would even be backwards compatible with 9.x where

Re: [HACKERS] Scaling shared buffer eviction

2014-09-23 Thread Robert Haas
On Tue, Sep 23, 2014 at 10:55 AM, Robert Haas robertmh...@gmail.com wrote: But this gets at another point: the way we're benchmarking this right now, we're really conflating the effects of three different things: 1. Changing the locking regimen around the freelist and clocksweep. 2. Adding a

Re: [HACKERS] RLS feature has been committed

2014-09-23 Thread Stephen Frost
All, Robert and I had a great discussion on the phone where we were both able to voice our concerns and feelings about RLS being pushed. By way of summary, we agree that it was pushed ahead of its time and that it should have waited for at least another review, which likely would have

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-23 Thread David Johnston
On Tue, Sep 23, 2014 at 3:05 PM, Greg Stark st...@mit.edu wrote: Fwiw I agree with TL2. The simplest, least surprising behaviour to explain to users is to say we round to nearest and if that means we rounded to zero (or another special value) we throw an error. We could list the minimum value

Re: [HACKERS] Scaling shared buffer eviction

2014-09-23 Thread Robert Haas
On Tue, Sep 23, 2014 at 4:29 PM, Robert Haas robertmh...@gmail.com wrote: I did some more experimentation on this. Attached is a patch that JUST does #1, and, ... ...and that was the wrong patch. Thanks to Heikki for point that out. Second try. -- Robert Haas EnterpriseDB:

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-23 Thread Peter Eisentraut
On 9/23/14 10:29 AM, Tom Lane wrote: Votes or no votes, that's a horrible idea; it breaks the design goal that users shouldn't need to remember the precise unit size when making postgresql.conf entries. I think this is not historically correct. The original motivation was that you had to

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-23 Thread Peter Eisentraut
On 9/23/14 1:13 AM, Stephen Frost wrote: To clarify- we'll simply swap from (essentially) floor() to ceil() for handling all GUC_with_unit to internal_unit conversions, document that, and note it in the release notes as a possible behavior change and move on. That'll probably work, as long as

Re: [HACKERS] delta relations in AFTER triggers

2014-09-23 Thread Kevin Grittner
Thanks for reviewing this. I will spend some time looking at your recommendations in detail and seeing what it would take to implement them, and whether I agree that is better; but I wanted to point out a couple things regarding the SPI interface where I'm not sure you realize what's currently

Re: [HACKERS] better atomics - v0.6

2014-09-23 Thread Oskari Saarenmaa
23.09.2014, 15:18, Andres Freund kirjoitti: On 2014-09-23 13:50:28 +0300, Oskari Saarenmaa wrote: 23.09.2014, 00:01, Andres Freund kirjoitti: The patches: 0001: The actual atomics API I tried building PG on Solaris 10/Sparc using GCC 4.9.0 (buildfarm animal dingo) with this patch but

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-23 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On 9/23/14 10:29 AM, Tom Lane wrote: Votes or no votes, that's a horrible idea; it breaks the design goal that users shouldn't need to remember the precise unit size when making postgresql.conf entries. I think this is not historically correct. The

Re: [HACKERS] Scaling shared buffer eviction

2014-09-23 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Sep 23, 2014 at 4:29 PM, Robert Haas robertmh...@gmail.com wrote: I did some more experimentation on this. Attached is a patch that JUST does #1, and, ... ...and that was the wrong patch. Thanks to Heikki for point that out. Second try.

Re: [HACKERS] Scaling shared buffer eviction

2014-09-23 Thread Robert Haas
On Tue, Sep 23, 2014 at 5:43 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Sep 23, 2014 at 4:29 PM, Robert Haas robertmh...@gmail.com wrote: I did some more experimentation on this. Attached is a patch that JUST does #1, and, ... ...and that was

Re: [HACKERS] Scaling shared buffer eviction

2014-09-23 Thread Gregory Smith
On 9/23/14, 10:31 AM, Robert Haas wrote: I suggest we count these things: 1. The number of buffers the reclaimer has put back on the free list. 2. The number of times a backend has run the clocksweep. 3. The number of buffers past which the reclaimer has advanced the clock sweep (i.e. the

Re: [HACKERS] better atomics - v0.6

2014-09-23 Thread Andres Freund
On 2014-09-24 00:27:25 +0300, Oskari Saarenmaa wrote: 23.09.2014, 15:18, Andres Freund kirjoitti: On 2014-09-23 13:50:28 +0300, Oskari Saarenmaa wrote: 23.09.2014, 00:01, Andres Freund kirjoitti: The patches: 0001: The actual atomics API I tried building PG on Solaris 10/Sparc using GCC

Re: [HACKERS] Scaling shared buffer eviction

2014-09-23 Thread Andres Freund
On 2014-09-23 16:29:16 -0400, Robert Haas wrote: On Tue, Sep 23, 2014 at 10:55 AM, Robert Haas robertmh...@gmail.com wrote: But this gets at another point: the way we're benchmarking this right now, we're really conflating the effects of three different things: 1. Changing the locking

Re: [HACKERS] Scaling shared buffer eviction

2014-09-23 Thread Robert Haas
On Tue, Sep 23, 2014 at 6:02 PM, Gregory Smith gregsmithpg...@gmail.com wrote: On 9/23/14, 10:31 AM, Robert Haas wrote: I suggest we count these things: 1. The number of buffers the reclaimer has put back on the free list. 2. The number of times a backend has run the clocksweep. 3. The

Re: [HACKERS] Scaling shared buffer eviction

2014-09-23 Thread Robert Haas
On Tue, Sep 23, 2014 at 6:54 PM, Andres Freund and...@2ndquadrant.com wrote: Am I understanding you correctly that you also measured context switches for spinlocks? If so, I don't think that's a valid comparison. LWLocks explicitly yield the CPU as soon as there's any contention while

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-09-23 Thread Robert Haas
On Tue, Sep 23, 2014 at 3:04 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Alvaro Herrera wrote: Heikki Linnakangas wrote: If you add a new datatype, and define b-tree operators for it, what is required to create a minmax opclass for it? Would it be possible to generalize the

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-09-23 Thread Michael Paquier
On Wed, Sep 24, 2014 at 8:23 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, Sep 23, 2014 at 3:04 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Alvaro Herrera wrote: I will look into adding some testing mechanism for the union support proc; with that I will just consider the patch

[HACKERS] “Core” function in Postgres

2014-09-23 Thread Mingzhe Li
Hi experts, I want to know what's the core function used in Postgres server? I am looking for something corresponding to main() in a simple C program. I want to know the file path and the function name. I am using Postgres 9.3.5, however I assume the core function will be unchanged between

Re: [HACKERS] Scaling shared buffer eviction

2014-09-23 Thread Andres Freund
On 2014-09-23 19:21:10 -0400, Robert Haas wrote: On Tue, Sep 23, 2014 at 6:54 PM, Andres Freund and...@2ndquadrant.com wrote: I think it might be possible to construct some cases where the spinlock performs worse than the lwlock. But I think those will be clearly in the minority. And at

[HACKERS] Re: [HACKERS] “Core” function in Postgres

2014-09-23 Thread Peter Geoghegan
On Tue, Sep 23, 2014 at 4:29 PM, Mingzhe Li mingzhe0...@gmail.com wrote: I want to know what's the core function used in Postgres server? I am looking for something corresponding to main() in a simple C program. I want to know the file path and the function name. I am using Postgres 9.3.5,

Re: [HACKERS] JsonbValue to Jsonb conversion

2014-09-23 Thread Andrew Dunstan
On 09/23/2014 12:23 PM, Dmitry Dolgov wrote: Hi all, I'm faced with some troubles about the jsonb implementation, and I hope I'll get little advice =) If I understand correctly, an abstract function for jsonb modification should have the following stages: Jsonb - JsonbValue - Modification

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-23 Thread Peter Geoghegan
On Fri, Sep 19, 2014 at 5:40 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I think we should bite the bullet and break compatibility with 9.4beta2 format, even if we go with my patch. In a jsonb object, it makes sense to store all the keys first, like Tom did, because of cache benefits,

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-23 Thread Tom Lane
Peter Geoghegan p...@heroku.com writes: On Fri, Sep 19, 2014 at 5:40 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I think we should bite the bullet and break compatibility with 9.4beta2 format, even if we go with my patch. In a jsonb object, it makes sense to store all the keys

Re: [HACKERS] Core function in Postgres

2014-09-23 Thread Gregory Smith
On 9/23/14, 8:02 PM, Michael Paquier wrote: pgsql-hackers is as well a mailing list where people have technical discussions about features and patches, hence your question would be more adapted for pgsql-novice or pgsql-general. Let's be fair and get the details perfect if we're going to

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-09-23 Thread Alvaro Herrera
Robert Haas wrote: With all respect, I think this is a bad idea. I know you've put a lot of energy into this patch and I'm confident it's made a lot of progress. But as with Stephen's patch, the final form deserves a thorough round of looking over by someone else before it goes in. As you

Re: [HACKERS] “Core” function in Postgres

2014-09-23 Thread Mark Kirkwood
On 24/09/14 11:29, Mingzhe Li wrote: Hi experts, I want to know what's the core function used in Postgres server? I am looking for something corresponding to main() in a simple C program. I want to know the file path and the function name. I am using Postgres 9.3.5, however I assume the core

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-09-23 Thread Robert Haas
On Tue, Sep 23, 2014 at 7:35 PM, Michael Paquier michael.paqu...@gmail.com wrote: Would this person be it an extra committer or an simple reviewer? It would give more insurance if such huge patches (couple of thousands of lines) get an extra +1 from another committer, proving that the code has

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-09-23 Thread Robert Haas
On Tue, Sep 23, 2014 at 9:23 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Robert Haas wrote: With all respect, I think this is a bad idea. I know you've put a lot of energy into this patch and I'm confident it's made a lot of progress. But as with Stephen's patch, the final form

Re: [HACKERS] Scaling shared buffer eviction

2014-09-23 Thread Robert Haas
On Tue, Sep 23, 2014 at 7:42 PM, Andres Freund and...@2ndquadrant.com wrote: It will actually be far worse than that, because we'll acquire and release the spinlock for every buffer over which we advance the clock sweep, instead of just once for the whole thing. I said double, because we

Re: PENDING_LIST_CLEANUP_SIZE - maximum size of GIN pending list Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

2014-09-23 Thread Etsuro Fujita
(2014/09/13 2:42), Fujii Masao wrote: On Wed, Sep 10, 2014 at 10:37 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Fujii Masao wrote: On Wed, Sep 10, 2014 at 12:15 PM, Etsuro Fujita fujita.ets...@lab.ntt.co.jp wrote: PENDING_LIST_CLEANUP_SIZE and work_mem, for this setting. Wouldn't it

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-23 Thread Gregory Smith
On 9/23/14, 1:21 AM, David Johnston wrote: This patch should fix the round-to-zero issue. If someone wants to get rid of zero as a special case let them supply a separate patch for that improvement. I am going to wander into this fresh after just reading everything once (but with more than

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-23 Thread Peter Geoghegan
On Wed, Aug 27, 2014 at 7:43 PM, Peter Geoghegan p...@heroku.com wrote: Omission === The patch currently lacks a way of referencing datums rejected for insertion when updating. Attached revision of the patch set (which I'll call v1.2) adds this capability in a separate commit. It now

Re: [HACKERS] LIMIT for UPDATE and DELETE

2014-09-23 Thread Etsuro Fujita
(2014/09/17 1:58), Robert Haas wrote: On Tue, Sep 16, 2014 at 11:31 AM, Kevin Grittner kgri...@ymail.com wrote: Etsuro Fujita fujita.ets...@lab.ntt.co.jp wrote: (2014/08/15 6:18), Rukh Meski wrote: Based on the feedback on my previous patch, I've separated only the LIMIT part into its own

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-23 Thread David Johnston
On Tue, Sep 23, 2014 at 10:11 PM, Gregory Smith gregsmithpg...@gmail.com wrote: On 9/23/14, 1:21 AM, David Johnston wrote: This patch should fix the round-to-zero issue. If someone wants to get rid of zero as a special case let them supply a separate patch for that improvement. I am

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-09-23 Thread Alvaro Herrera
Robert Haas wrote: On Tue, Sep 23, 2014 at 9:23 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: If you or somebody else want to give it a look, I have no problem waiting a bit longer. I don't want to delay indefinitely, though, because I think it's better shipped early in the release

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-23 Thread Gregory Smith
On 9/23/14, 10:52 PM, David Johnston wrote: ​Given the following why not just pick ms for log_rotation_age now? Right now there are people out there who have configurations that look like this: log_rotation_age=60 In order to get hourly rotation. These users will suffer some trauma

Re: [HACKERS] Scaling shared buffer eviction

2014-09-23 Thread Gregory Smith
On 9/23/14, 7:13 PM, Robert Haas wrote: I think we expose far too little information in our system views. Just to take one example, we expose no useful information about lwlock acquire or release, but a lot of real-world performance problems are caused by lwlock contention. I sent over a

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-23 Thread Jan Wieck
On 09/15/2014 09:46 PM, Craig Ringer wrote: On 09/16/2014 07:44 AM, Peter Geoghegan wrote: FWIW, I am slightly concerned about weighing use cases around very large JSON documents too heavily. Having enormous jsonb documents just isn't going to work out that well, but neither will equivalent

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-23 Thread Peter Geoghegan
On Tue, Sep 23, 2014 at 10:02 PM, Jan Wieck j...@wi3ck.info wrote: Is it worth paying a byte per value to save on possible upgrade pain? This comment seems to have drowned in the discussion. If there indeed has to be a catversion bump in the process of this, then I agree with Craig. -1. We

  1   2   >