Re: [HACKERS] Minmax indexes

2013-09-27 Thread Amit Kapila
On Thu, Sep 26, 2013 at 1:46 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Amit Kapila escribió: On Sun, Sep 15, 2013 at 5:44 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: On Windows, patch gives below compilation errors: src\backend\access\minmax\mmtuple.c(96): error C2057:

Re: [HACKERS] Minmax indexes

2013-09-27 Thread Amit Kapila
On Fri, Sep 27, 2013 at 11:49 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Thu, Sep 26, 2013 at 1:46 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Amit Kapila escribió: On Sun, Sep 15, 2013 at 5:44 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: On Windows, patch gives

Re: [HACKERS] FW: REVIEW: Allow formatting in log_line_prefix

2013-09-27 Thread David Rowley
On Fri, Sep 27, 2013 at 10:28 AM, Robert Haas robertmh...@gmail.com wrote: On Thu, Sep 26, 2013 at 5:18 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Sep 26, 2013 at 3:55 PM, David Rowley dgrowle...@gmail.com wrote: I think I must have forgot to save it before I emailed it...

Re: [HACKERS] pgbench filler columns

2013-09-27 Thread Pavan Deolasee
On Thu, Sep 26, 2013 at 7:20 PM, Noah Misch n...@leadboat.com wrote: On Thu, Sep 26, 2013 at 03:23:30PM +0530, Pavan Deolasee wrote: Should we just fix the comment and say its applicable for all tables except accounts ? Please do. How about something like this ? Patch attached.

Re: [HACKERS] Some interesting news about Linux 3.12 OOM

2013-09-27 Thread Daniel Farina
On Wed, Sep 25, 2013 at 8:00 AM, Greg Stark st...@mit.edu wrote: On Wed, Sep 25, 2013 at 12:15 AM, Daniel Farina dan...@heroku.com wrote: Enable the memcg OOM killer only for user faults, where it's really the only option available. Is this really a big deal? I would expect most faults to

Re: [HACKERS] Wait free LW_SHARED acquisition

2013-09-27 Thread Heikki Linnakangas
On 27.09.2013 01:55, Andres Freund wrote: Hello, We have had several customers running postgres on bigger machines report problems on busy systems. Most recently one where a fully cached workload completely stalled in s_lock()s due to the *shared* lwlock acquisition in BufferAlloc() around the

Re: [HACKERS] Wait free LW_SHARED acquisition

2013-09-27 Thread Andres Freund
Hi, On 2013-09-27 10:14:46 +0300, Heikki Linnakangas wrote: On 27.09.2013 01:55, Andres Freund wrote: We have had several customers running postgres on bigger machines report problems on busy systems. Most recently one where a fully cached workload completely stalled in s_lock()s due to the

Re: [HACKERS] Wait free LW_SHARED acquisition

2013-09-27 Thread Andres Freund
On 2013-09-27 09:21:05 +0200, Andres Freund wrote: So the goal is to have LWLockAcquire(LW_SHARED) never block unless somebody else holds an exclusive lock. To produce enough appetite for reading the rest of the long mail, here's some numbers on a pgbench -j 90 -c 90 -T 60 -S (-i -s 10) on

Re: [HACKERS] Patch for fail-back without fresh backup

2013-09-27 Thread Sawada Masahiko
On Thu, Sep 26, 2013 at 8:54 PM, Pavan Deolasee pavan.deola...@gmail.com wrote: On Thu, Sep 19, 2013 at 4:02 PM, Fujii Masao masao.fu...@gmail.com wrote: Hmm... when synchronous_transfer is set to data_flush, IMO the intuitive behaviors are (1) synchronous_commit = on A data flush should

Re: [HACKERS] Wait free LW_SHARED acquisition

2013-09-27 Thread Heikki Linnakangas
On 27.09.2013 10:21, Andres Freund wrote: Hi, On 2013-09-27 10:14:46 +0300, Heikki Linnakangas wrote: On 27.09.2013 01:55, Andres Freund wrote: We have had several customers running postgres on bigger machines report problems on busy systems. Most recently one where a fully cached workload

Re: [HACKERS] Wait free LW_SHARED acquisition

2013-09-27 Thread Andres Freund
On 2013-09-27 09:57:07 +0200, Andres Freund wrote: On 2013-09-27 09:21:05 +0200, Andres Freund wrote: So the goal is to have LWLockAcquire(LW_SHARED) never block unless somebody else holds an exclusive lock. To produce enough appetite for reading the rest of the long mail, here's some

Re: [HACKERS] Patch for fail-back without fresh backup

2013-09-27 Thread Pavan Deolasee
On Fri, Sep 27, 2013 at 1:28 PM, Sawada Masahiko sawada.m...@gmail.comwrote: Thank you for comment. I think it is good simple idea. In your opinion, if synchronous_transfer is set 'all' and synchronous_commit is set 'on', the master wait for data flush eve if user sets synchronous_commit

Re: [HACKERS] Wait free LW_SHARED acquisition

2013-09-27 Thread Andres Freund
On 2013-09-27 11:11:56 +0300, Heikki Linnakangas wrote: On 27.09.2013 10:21, Andres Freund wrote: Heaps better. In the case causing this investigation lots of the pages with hot spinlocks were the simply the same ones over and over again, partitioning the lockspace won't help much there.

Re: [HACKERS] Patch for fast gin cache performance improvement

2013-09-27 Thread Etsuro Fujita
I wrote: I had a look over this patch. I think this patch is interesting and very useful. Here are my review points: 8. I think there are no issues in this patch. However, I have one question: how this patch works in the case where gin_fast_limit/fast_cache_size = 0? In this case, in my

Re: [HACKERS] insert throw error when year field len 4 for timestamptz datatype

2013-09-27 Thread Rushabh Lathia
Sorry for delay in reply. On Tue, Sep 17, 2013 at 6:23 PM, Haribabu kommi haribabu.ko...@huawei.comwrote: On Tue, 17 September 2013 14:33 Rushabh Lathia wrote: On Mon, Sep 16, 2013 at 7:22 PM, Haribabu kommi haribabu.ko...@huawei.com wrote: ***On *14 August 2013 Rushabh Lathia

Re: [HACKERS] Patch for fail-back without fresh backup

2013-09-27 Thread Sawada Masahiko
On Fri, Sep 27, 2013 at 5:18 PM, Pavan Deolasee pavan.deola...@gmail.com wrote: On Fri, Sep 27, 2013 at 1:28 PM, Sawada Masahiko sawada.m...@gmail.com wrote: Thank you for comment. I think it is good simple idea. In your opinion, if synchronous_transfer is set 'all' and

Re: [HACKERS] insert throw error when year field len 4 for timestamptz datatype

2013-09-27 Thread Haribabu kommi
On 27 September 2013 15:04 Rushabh Lathia wrote: On Tue, Sep 17, 2013 at 6:23 PM, Haribabu kommi haribabu.ko...@huawei.commailto:haribabu.ko...@huawei.com wrote: I feel changing the year value to accept the length (4) is not simple. So many places the year length crossing more than length 4 is

Re: [HACKERS] backup.sgml-cmd-v003.patch

2013-09-27 Thread Robert Haas
On Thu, Sep 26, 2013 at 1:15 PM, Ivan Lezhnjov IV i...@commandprompt.com wrote: Thanks for a detailed response. I attached a patch file that builds on your corrections and introduces some of the edits discussed above. This patch makes at least five unrelated sets of changes: 1. Attempting to

Re: [HACKERS] Extra functionality to createuser

2013-09-27 Thread Robert Haas
On Thu, Sep 26, 2013 at 1:04 PM, Christopher Browne cbbro...@gmail.com wrote: Sitting on my todo list for a while has been to consider the idea of adding a bit of additional functionality to createuser. One of the functions of CREATE ROLE is to associate the role with other roles, thus...

Re: [HACKERS] Re: Request for Patch Feedback: Lag Lead Window Functions Can Ignore Nulls

2013-09-27 Thread Robert Haas
On Thu, Sep 26, 2013 at 4:20 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: But how do we ensure that the BMS is allocated in a context? You'd have to switch contexts each time you call bms_add_member. I don't have a good answer to this. The coding of bms_add_member is pretty funky. Why

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2013-09-27 Thread Robert Haas
On Thu, Sep 26, 2013 at 11:58 PM, Peter Geoghegan p...@heroku.com wrote: On Thu, Sep 26, 2013 at 12:15 PM, Robert Haas robertmh...@gmail.com wrote: Well, I think we can rule out value locks that are held for the duration of a transaction right away. That's just not going to fly. I think I

Re: [HACKERS] Wait free LW_SHARED acquisition

2013-09-27 Thread Florian Pflug
On Sep27, 2013, at 00:55 , Andres Freund and...@2ndquadrant.com wrote: So the goal is to have LWLockAcquire(LW_SHARED) never block unless somebody else holds an exclusive lock. To produce enough appetite for reading the rest of the long mail, here's some numbers on a pgbench -j 90 -c 90 -T 60

Re: [HACKERS] backup.sgml-cmd-v003.patch

2013-09-27 Thread Karl O. Pinc
Hi Robert, On 09/27/2013 05:56:52 AM, Robert Haas wrote: 1. Attempting to encourage people to consider custom format dumps. What's important is what you can do... Your critique seems obvious in retrospect. Sorry you had to step in here and do my job. The above point is particularly

Re: [HACKERS] logical changeset generation v6.1

2013-09-27 Thread Thom Brown
On 27 September 2013 16:14, Andres Freund and...@2ndquadrant.com wrote: Hi, Attached you can find an updated version of the series taking in some of the review comments (the others are queued, not ignored), including: * split of things from the big Introduce wal decoding via ... patch * fix

Re: [HACKERS] logical changeset generation v6.1

2013-09-27 Thread Andres Freund
On 2013-09-27 16:35:53 +0100, Thom Brown wrote: On 27 September 2013 16:14, Andres Freund and...@2ndquadrant.com wrote: Hi, Attached you can find an updated version of the series taking in some of the review comments (the others are queued, not ignored), including: * split of things

[HACKERS] hstore extension version screwup

2013-09-27 Thread Andrew Dunstan
When adding json support to hstore, I made a major blunder and added the new functionality to the existing sql script instead of bumping the version, renaming the script and adding an update script. This was lazy and there's no real excuse, although I will note that it was a mistake far too

Re: [HACKERS] backup.sgml-cmd-v003.patch

2013-09-27 Thread Robert Haas
On Fri, Sep 27, 2013 at 9:49 AM, Karl O. Pinc k...@meme.com wrote: Hi Robert, On 09/27/2013 05:56:52 AM, Robert Haas wrote: 1. Attempting to encourage people to consider custom format dumps. What's important is what you can do... Your critique seems obvious in retrospect. Sorry you had

Re: [HACKERS] Minmax indexes

2013-09-27 Thread Jim Nasby
On 9/26/13 2:46 PM, Robert Haas wrote: On Thu, Sep 26, 2013 at 2:58 PM, Jim Nasby j...@nasby.net wrote: Why would we add additional code complexity when forks do the trick? That seems like a step backwards, not forward. Well, they sorta do the trick, but see e.g. commit

Re: [HACKERS] Extra functionality to createuser

2013-09-27 Thread Jim Nasby
On 9/27/13 6:01 AM, Robert Haas wrote: Thus, the above commands might be replaced by: PGUSER=postgres createuser -D -S -l -g app_readonly_role,app2_writer_role my_new_user Would this be worth adding to the ToDo list? I'd be inclined to favor a patch implementing this. +1 -- Jim C. Nasby,

Re: [HACKERS] Minmax indexes

2013-09-27 Thread Greg Stark
On Fri, Sep 27, 2013 at 7:22 PM, Jim Nasby j...@nasby.net wrote: Yeah, we obviously kept things simpler when adding forks in order to get the feature out the door. There's improvements that need to be made. But IMHO that's not reason to automatically avoid forks; we need to consider the cost

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-09-27 Thread Merlin Moncure
On Fri, Sep 27, 2013 at 7:56 AM, Merlin Moncure mmonc...@gmail.com wrote: On Thu, Sep 26, 2013 at 10:14 PM, Merlin Moncure mmonc...@gmail.com wrote: On Thu, Sep 26, 2013 at 6:08 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-08-27 12:17:55 -0500, Merlin Moncure wrote: On Tue, Aug

Re: [HACKERS] Minmax indexes

2013-09-27 Thread Jim Nasby
On 9/27/13 1:43 PM, Greg Stark wrote: Honestly, I think we actually need more obfuscation between what happens on the filesystem and the rest of postgres... we're starting to look at areas where that would help. For example, the recent idea of being able to truncate individual relation files

Re: [HACKERS] Re: Request for Patch Feedback: Lag Lead Window Functions Can Ignore Nulls

2013-09-27 Thread Heikki Linnakangas
On 27.09.2013 14:12, Robert Haas wrote: On Thu, Sep 26, 2013 at 4:20 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: But how do we ensure that the BMS is allocated in a context? You'd have to switch contexts each time you call bms_add_member. I don't have a good answer to this. The

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-09-27 Thread Heikki Linnakangas
On 27.09.2013 22:00, Merlin Moncure wrote: On Fri, Sep 27, 2013 at 7:56 AM, Merlin Moncuremmonc...@gmail.com wrote: On Thu, Sep 26, 2013 at 10:14 PM, Merlin Moncuremmonc...@gmail.com wrote: On Thu, Sep 26, 2013 at 6:08 PM, Andres Freundand...@2ndquadrant.com wrote: On 2013-08-27 12:17:55

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-09-27 Thread Peter Geoghegan
On Fri, Sep 27, 2013 at 1:28 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Yep. I only added the first caller of the barriers altogether in the xlog-insertion scaling patch. Robert wrote the infrastructure in 9.3, but it wasn't used until now, in 9.4. FWIW, it was actually during 9.2

Re: [HACKERS] logical changeset generation v6

2013-09-27 Thread Steve Singer
On 09/26/2013 02:47 PM, Steve Singer wrote: I've determined that when in this test the walsender seems to be hitting this when it is decode the transactions that are behind the slonik commands to add tables to replication (set add table, set add sequence). This is before the SUBSCRIBE SET

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-09-27 Thread Andres Freund
On 2013-09-27 23:28:37 +0300, Heikki Linnakangas wrote: On 27.09.2013 22:00, Merlin Moncure wrote: Attached is simplified patch that replaces the spinlock with a read barrier based on a suggestion made by Andres offlist. The approach has different performance characteristics -- a barrier call

Re: [HACKERS] logical changeset generation v6

2013-09-27 Thread Andres Freund
Hi Steve, On 2013-09-27 17:06:59 -0400, Steve Singer wrote: I've determined that when in this test the walsender seems to be hitting this when it is decode the transactions that are behind the slonik commands to add tables to replication (set add table, set add sequence). This is before the

Re: [HACKERS] Wait free LW_SHARED acquisition

2013-09-27 Thread Andres Freund
On 2013-09-27 14:46:50 +0200, Florian Pflug wrote: On Sep27, 2013, at 00:55 , Andres Freund and...@2ndquadrant.com wrote: So the goal is to have LWLockAcquire(LW_SHARED) never block unless somebody else holds an exclusive lock. To produce enough appetite for reading the rest of the long

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-09-27 Thread Andres Freund
Hi, What confuses me is that pg_read_barrier() is just a compiler barrier on x86[-64] in barrier.h. According to my knowledge it needs to be an lfence or the full barrier? The linked papers from Paul McKenney - which are a great read - seem to agree? On 2013-09-27 23:12:17 +0200, Andres Freund

Re: [HACKERS] Extra functionality to createuser

2013-09-27 Thread Christopher Browne
Attached is a patch implementing the -g / --roles option for createuser. I'll be attaching it to the open CommitFest shortly. createuser.diff Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] record identical operator - Review

2013-09-27 Thread Kevin Grittner
Bruce Momjian br...@momjian.us wrote: On Thu, Sep 19, 2013 at 06:31:38PM -0400, Steve Singer wrote: On 09/12/2013 06:27 PM, Kevin Grittner wrote: Attached is a patch for a bit of infrastructure I believe to be necessary for correct behavior of REFRESH MATERIALIZED VIEW CONCURRENTLY as well as

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2013-09-27 Thread Peter Geoghegan
On Fri, Sep 27, 2013 at 5:36 AM, Robert Haas robertmh...@gmail.com wrote: I don't have another idea either. In fact, I'd go so far as to say that doing any third thing that's better than those two to any reasonable person is obviously impossible. But I'd add that we simple cannot rollback at

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE - visibility semantics

2013-09-27 Thread Peter Geoghegan
On Tue, Sep 24, 2013 at 2:14 AM, Andres Freund and...@2ndquadrant.com wrote: Various messages are discussing semantics around visibility. I by now have a hard time keeping track. So let's keep the discussion of the desired semantics to this thread. Yes, it's pretty complicated. I meant to