Re: [HACKERS] Better support of exported snapshots with pg_dump

2014-10-14 Thread Michael Paquier
On Wed, Oct 15, 2014 at 2:46 PM, Andres Freund wrote: > > This seems more user-friendly. But well I agree that we could do > > a larger set of things that could be used for even other purposes: > > - Ability to define snapshot name with pg_dump > > - Take system or database-wide lock > > - Extra

Re: [HACKERS] WIP: dynahash replacement for buffer table

2014-10-14 Thread Andres Freund
On 2014-10-14 17:53:10 -0400, Robert Haas wrote: > On Tue, Oct 14, 2014 at 4:42 PM, Andres Freund wrote: > >> The code in CHashSearch shows the problem there: you need to STORE the > >> hazard pointer before you begin to do the LOAD operations to scan the > >> bucket, and you must finish all of th

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-10-14 Thread Craig Ringer
On 10/15/2014 12:53 PM, Noah Misch wrote: > Windows Server 2003 isn't even EOL yet. I'd welcome a buildfarm member with > that OS and a modern toolchain. It's possible to run multiple buildfarm animals on a single Windows instance, each with a different toolchain. There's the chance of interacti

Re: [HACKERS] Better support of exported snapshots with pg_dump

2014-10-14 Thread Andres Freund
On 2014-10-15 14:28:16 +0900, Michael Paquier wrote: > On Wed, Oct 15, 2014 at 2:06 PM, Andres Freund > wrote: > > > I think that's completely the wrong way to go at this. The time it takes > > to create a replication slot under write load is far larger than the > > time it takes to start pg_dump

Re: [HACKERS] Better support of exported snapshots with pg_dump

2014-10-14 Thread Michael Paquier
On Wed, Oct 15, 2014 at 2:06 PM, Andres Freund wrote: > I think that's completely the wrong way to go at this. The time it takes > to create a replication slot under write load is far larger than the > time it takes to start pg_dump and load. This really doesn't add any > actual safety. Also, the

[HACKERS] Additional role attributes && superuser review

2014-10-14 Thread Stephen Frost
Greetings, The attached patch for review implements a few additional role attributes (all requested by users or clients in various forums) for common operations which currently require superuser privileges. This is not a complete solution for all of the superuser-only privileges we have

Re: [HACKERS] Better support of exported snapshots with pg_dump

2014-10-14 Thread Andres Freund
On 2014-10-15 07:09:10 +0900, Michael Paquier wrote: > > whatever replication solution you use and just have pg_dump accept the > > snapshot as input parameter? I am not sure how much I like pg_dump creating > > the slot. I am aware that you need to have the replication connection open > > but that

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-10-14 Thread Noah Misch
On Tue, Oct 14, 2014 at 07:07:17PM -0400, Tom Lane wrote: > Dave Page writes: > > On Tue, Oct 14, 2014 at 11:38 PM, Tom Lane wrote: > >> I think we're hoping that somebody will step up and investigate how > >> narwhal's problem might be fixed. I have planned to look at reproducing narwhal's prob

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-10-14 Thread Robert Haas
On Wed, Oct 15, 2014 at 12:13 AM, Tom Lane wrote: > What is "timely service lock requests"? You got the lock before firing > off the background workers, you hold it till they're done. If you want to run any non-trivial (read: useful) code in the background workers, it rapidly gets very hard to p

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-10-14 Thread Tom Lane
Robert Haas writes: > For parallelism, I think we need a concept of group locking. That is, > suppose we have a user backend and N worker backends collaborating to > execute some query. For the sake of argument, let's say it's a > parallel CLUSTER, requiring a full table lock. We need all of th

[HACKERS] group locking: incomplete patch, just for discussion

2014-10-14 Thread Robert Haas
For parallelism, I think we need a concept of group locking. That is, suppose we have a user backend and N worker backends collaborating to execute some query. For the sake of argument, let's say it's a parallel CLUSTER, requiring a full table lock. We need all of the backends to be able to lock

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.9

2014-10-14 Thread Amit Kapila
On Wed, Oct 15, 2014 at 12:06 AM, Merlin Moncure wrote: > > A while back, I submitted a minor tweak to the clock sweep so that, > instead of spinlocking every single buffer header as it swept it just > did a single TAS as a kind of a trylock and punted to the next buffer > if the test failed on th

Re: [HACKERS] [PATCH] Cleanup: unify checks for catalog modification

2014-10-14 Thread Alvaro Herrera
Tom Lane wrote: > Marti Raudsepp writes: > > I happened to notice that there are no less than 14 places in the code > > that check whether a relation is a system catalog and throwing the > > error "permission denied: "foo" is a system catalog" > > > The attached patch factors all of those into a

Re: [HACKERS] [PATCH] Cleanup: unify checks for catalog modification

2014-10-14 Thread Tom Lane
Marti Raudsepp writes: > I happened to notice that there are no less than 14 places in the code > that check whether a relation is a system catalog and throwing the > error "permission denied: "foo" is a system catalog" > The attached patch factors all of those into a single > ForbidSystemTableMo

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-10-14 Thread Tom Lane
Dave Page writes: > On Tue, Oct 14, 2014 at 11:38 PM, Tom Lane wrote: >> I think we're hoping that somebody will step up and investigate how >> narwhal's problem might be fixed. However, the machine's owner (Dave) >> doesn't appear to have the time/interest to do that. > It's a time issue right

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-10-14 Thread Andrew Dunstan
On 10/14/2014 06:44 PM, Dave Page wrote: On Tue, Oct 14, 2014 at 11:38 PM, Tom Lane wrote: Alvaro Herrera writes: It seems we left this in broken state. Do we need to do more here to fix narwhal, or do we want to retire narwhal now? Something else? Are we waiting on someone in particular

[HACKERS] [PATCH] Cleanup: unify checks for catalog modification

2014-10-14 Thread Marti Raudsepp
Hi list, I happened to notice that there are no less than 14 places in the code that check whether a relation is a system catalog and throwing the error "permission denied: "foo" is a system catalog" The attached patch factors all of those into a single ForbidSystemTableMods() function. Is this c

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-10-14 Thread Dave Page
On Tue, Oct 14, 2014 at 11:38 PM, Tom Lane wrote: > Alvaro Herrera writes: >> It seems we left this in broken state. Do we need to do more here to >> fix narwhal, or do we want to retire narwhal now? Something else? Are >> we waiting on someone in particular to do something specific? > > I thi

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-10-14 Thread Tom Lane
Alvaro Herrera writes: > It seems we left this in broken state. Do we need to do more here to > fix narwhal, or do we want to retire narwhal now? Something else? Are > we waiting on someone in particular to do something specific? I think we're hoping that somebody will step up and investigate

Re: [HACKERS] Proposal for better support of time-varying timezone abbreviations

2014-10-14 Thread Tom Lane
I wrote: > I got interested in the problem discussed in > http://www.postgresql.org/message-id/20714.1412456...@sss.pgh.pa.us > to wit: >> It's becoming clear to me that our existing design whereby zone >> abbreviations represent fixed GMT offsets isn't really good enough. >> I've been wondering wh

Re: [HACKERS] replicating DROP commands across servers

2014-10-14 Thread Alvaro Herrera
Robert Haas wrote: > On Fri, Oct 3, 2014 at 4:58 PM, Alvaro Herrera > wrote: > > Robert Haas wrote: > >> I'm not really very convinced that it's a good idea to expose this > >> instead of just figuring out a way to parse the object identity. > > > > That's the first thing I tried. But it's not p

Re: [HACKERS] Better support of exported snapshots with pg_dump

2014-10-14 Thread Michael Paquier
On Tue, Oct 14, 2014 at 11:55 PM, Petr Jelinek wrote: > On 22/09/14 02:24, Michael Paquier wrote: >> >> On Thu, Sep 4, 2014 at 11:33 PM, Michael Paquier >> >> Taking a dump consistent with a replication slot is useful for online >> upgrade cases first, because you can simply run pg_dump, have a sl

Re: [HACKERS] WIP: dynahash replacement for buffer table

2014-10-14 Thread Robert Haas
On Tue, Oct 14, 2014 at 4:42 PM, Andres Freund wrote: >> The code in CHashSearch shows the problem there: you need to STORE the >> hazard pointer before you begin to do the LOAD operations to scan the >> bucket, and you must finish all of those LOADs before you STORE the >> NULL hazard pointer. A

Re: [HACKERS] narwhal and PGDLLIMPORT

2014-10-14 Thread Alvaro Herrera
It seems we left this in broken state. Do we need to do more here to fix narwhal, or do we want to retire narwhal now? Something else? Are we waiting on someone in particular to do something specific? -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Su

Re: [HACKERS] proposal: plpgsql - Assert statement

2014-10-14 Thread Petr Jelinek
On 09/09/14 17:37, Pavel Stehule wrote: Ada is language with strong character, and PLpgSQL is little bit strange fork - so it isn't easy to find some form, how to solve all requirements. My requests: * be consistent with current PLpgSQL syntax and logic * allow some future extensibility * allow

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-10-14 Thread Petr Jelinek
On 09/10/14 00:32, Andres Freund wrote: From c835a06f20792556d35a0eee4c2fa21f5f23e8a3 Mon Sep 17 00:00:00 2001 From: Robert Haas Date: Fri, 11 Jul 2014 09:53:40 -0400 Subject: [PATCH 6/6] pg_background: Run commands in a background worker, and get the results. The currently-active GUC values

Re: [HACKERS] WIP: dynahash replacement for buffer table

2014-10-14 Thread Andres Freund
On 2014-10-14 11:19:16 -0400, Robert Haas wrote: > On Tue, Oct 14, 2014 at 10:25 AM, Andres Freund > wrote: > >> The key idea here is that lookups are done without any locks, only > >> memory barriers; and inserts and deletes are done using atomic ops. > > > > Hm. I quickly looked and I see that

[HACKERS] New Event Trigger: table_rewrite

2014-10-14 Thread Dimitri Fontaine
Hi fellow hackers, Please find attached to this email a patch to implement a new Event Trigger, fired on the the "table_rewrite" event. As attached, it's meant as a discussion enabler and only supports ALTER TABLE (and maybe not in all forms of it). It will need to grow support for VACUUM FULL and

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2014-10-14 Thread Abhijit Menon-Sen
At 2014-10-14 20:09:50 +0100, si...@2ndquadrant.com wrote: > > I think that's a good idea. > > We could have pg_audit.roles = 'audit1, audit2' Yes, it's a neat idea, and we could certainly do that. But why is it any better than "ALTER ROLE audit_rw SET pgaudit.log = …" and granting that role to t

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2014-10-14 Thread Stephen Frost
* Simon Riggs (si...@2ndquadrant.com) wrote: > On 14 October 2014 13:57, Stephen Frost wrote: > > > Create an 'audit' role. > > > > Every command run by roles which are granted to the 'audit' role are > > audited. > > > > Every 'select' against tables which the 'audit' role has 'select' rights >

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2014-10-14 Thread Simon Riggs
On 14 October 2014 13:57, Stephen Frost wrote: > Create an 'audit' role. > > Every command run by roles which are granted to the 'audit' role are > audited. > > Every 'select' against tables which the 'audit' role has 'select' rights > on are audited. Similairly for every insert, update, delete.

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.9

2014-10-14 Thread Merlin Moncure
On Tue, Oct 14, 2014 at 8:58 AM, Andres Freund wrote: > On 2014-10-14 08:40:49 -0500, Merlin Moncure wrote: >> On Fri, Oct 10, 2014 at 11:00 AM, Andres Freund >> wrote: >> > Which is nearly trivial now that atomics are in. Check out the attached >> > WIP patch which eliminates the spinlock from

Re: [HACKERS] pg_dump refactor patch to remove global variables

2014-10-14 Thread Alvaro Herrera
I have pushed this, thanks. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsq

Re: [HACKERS] Buffer Requests Trace

2014-10-14 Thread Stephen Frost
* Lucas Lersch (lucasler...@gmail.com) wrote: > I see this... but ReleaseBuffer() simply decrements the reference count of > page the buffer currently holds. Assuming that a ReadBuffer() - > ReleaseBuffer() pattern is used for interacting with the shared_buffers, > there will be a ReleaseBuffer() c

Re: [HACKERS] Buffer Requests Trace

2014-10-14 Thread Lucas Lersch
I see this... but ReleaseBuffer() simply decrements the reference count of page the buffer currently holds. Assuming that a ReadBuffer() - ReleaseBuffer() pattern is used for interacting with the shared_buffers, there will be a ReleaseBuffer() call for any page (heap or index) "loaded" into the sha

Re: [HACKERS] Expose options to explain? (track_io_timing)

2014-10-14 Thread Robert Haas
On Tue, Oct 14, 2014 at 1:04 PM, Joshua D. Drake wrote: > On 10/14/2014 10:01 AM, Robert Haas wrote: >> Hmm. IIRC, there are only two use cases for I/O timing at present: >> pg_stat_statements (which really only makes sense if it's turned on or >> off system-wide) and EXPLAIN. Rather than invent

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-10-14 Thread Robert Haas
On Wed, Oct 8, 2014 at 6:32 PM, Andres Freund wrote: >> /* >> + * Arrange to remove a dynamic shared memory mapping at cleanup time. >> + * >> + * dsm_keep_mapping() can be used to preserve a mapping for the entire >> + * lifetime of a process; this function reverses that decision, making >> + *

Re: [HACKERS] Maximum number of WAL files in the pg_xlog directory

2014-10-14 Thread Bruce Momjian
On Tue, Oct 14, 2014 at 09:20:22AM -0700, Jeff Janes wrote: > On Mon, Oct 13, 2014 at 12:11 PM, Bruce Momjian wrote: > > > I looked into this, and came up with more questions.  Why is > checkpoint_completion_target involved in the total number of WAL > segments?  If checkpoint_comple

Re: [HACKERS] Buffer Requests Trace

2014-10-14 Thread Stephen Frost
* Lucas Lersch (lucasler...@gmail.com) wrote: > Aren't heap and index requests supposed to go through the shared buffers > anyway? Sure they do, but a given page in shared_buffers can be used over and over again for different heap and index pages.. Thanks, Stephen signa

[HACKERS] [RFC] Incremental backup v3: incremental PoC

2014-10-14 Thread Marco Nenciarini
Hi Hackers, following the advices gathered on the list I've prepared a third partial patch on the way of implementing incremental pg_basebackup as described here https://wiki.postgresql.org/wiki/Incremental_backup == Changes Compared to the previous version I've made the following changes: * T

Re: [HACKERS] Buffer Requests Trace

2014-10-14 Thread Lucas Lersch
Aren't heap and index requests supposed to go through the shared buffers anyway? On Tue, Oct 14, 2014 at 7:02 PM, Stephen Frost wrote: > * Lucas Lersch (lucasler...@gmail.com) wrote: > > shared_buffers is 128MB and the version of pgsql is 9.3.5 > > I suspect you're not tracking what you think yo

Re: [HACKERS] Expose options to explain? (track_io_timing)

2014-10-14 Thread Joshua D. Drake
On 10/14/2014 10:01 AM, Robert Haas wrote: Hmm. IIRC, there are only two use cases for I/O timing at present: pg_stat_statements (which really only makes sense if it's turned on or off system-wide) and EXPLAIN. Rather than inventing more GUC machinery, I think we could just add an explain fla

Re: [HACKERS] Buffer Requests Trace

2014-10-14 Thread Stephen Frost
* Lucas Lersch (lucasler...@gmail.com) wrote: > shared_buffers is 128MB and the version of pgsql is 9.3.5 I suspect you're not tracking what you think you're tracking, which is why I brought up shared_buffers. ~14k * 8192 (page size) = ~110MB What it sounds like you're actually tracking are shar

Re: [HACKERS] Expose options to explain? (track_io_timing)

2014-10-14 Thread Robert Haas
On Thu, Oct 9, 2014 at 2:41 PM, Jeff Janes wrote: > I think the theory for track_io_timing being PGC_SUSET is that if the > superuser turned it on, no one should be able to turn it off. > > But I don't see an argument for the other way around, that no one should be > able to turn it on locally of

Re: [HACKERS] Buffer Requests Trace

2014-10-14 Thread Lucas Lersch
shared_buffers is 128MB and the version of pgsql is 9.3.5 On Tue, Oct 14, 2014 at 6:31 PM, Lucas Lersch wrote: > Sorry, I do not understand the question. > > But I forgot to give an additional information: I am printing the page id > for the trace file in ReleaseBuffer() only if it is a shared b

Re: [HACKERS] CINE in CREATE TABLE AS ... and CREATE MATERIALIZED VIEW ...

2014-10-14 Thread Fabrízio de Royes Mello
On Wed, Oct 1, 2014 at 9:17 AM, Fabrízio de Royes Mello < fabriziome...@gmail.com> wrote: > > Hi all, > > We already have IF NOT EXISTS for CREATE TABLE. There are some reason to don't have to CREATE TABLE AS and CREATE MATERIALIZED VIEW?? > Patch attached to add CINE support to: - CREATE TABLE A

Re: [HACKERS] Column Redaction

2014-10-14 Thread Robert Haas
On Sat, Oct 11, 2014 at 3:40 AM, Simon Riggs wrote: > As soon as you issue the above query, you have clearly indicated your > intention to steal. Receiving information is no longer accidental, it > is an explicit act that is logged in the auditing system against your > name. This is sufficient to

Re: [HACKERS] Buffer Requests Trace

2014-10-14 Thread Lucas Lersch
Sorry, I do not understand the question. But I forgot to give an additional information: I am printing the page id for the trace file in ReleaseBuffer() only if it is a shared buffer, I am not considering local buffers. I assumed that local buffers were used only for temporary tables. On Tue, Oct

Re: [HACKERS] Buffer Requests Trace

2014-10-14 Thread Stephen Frost
* Lucas Lersch (lucasler...@gmail.com) wrote: > Unfortunately, in the generated trace with over 2 million buffer requests, > only ~14k different pages are being accessed, out of the 800k of the whole > database. Am I missing something here? What do you have shared_buffers set to..? Thanks

Re: [HACKERS] Maximum number of WAL files in the pg_xlog directory

2014-10-14 Thread Jeff Janes
On Mon, Oct 13, 2014 at 12:11 PM, Bruce Momjian wrote: > > I looked into this, and came up with more questions. Why is > checkpoint_completion_target involved in the total number of WAL > segments? If checkpoint_completion_target is 0.5 (the default), the > calculation is: > > (2 + 0.5)

[HACKERS] Buffer Requests Trace

2014-10-14 Thread Lucas Lersch
Hello, I changed the buffer manager code in order to generate a trace of page requests from the buffer manager perspective. In summary, whenever ReleaseBuffer() or ReleaseAndReadBuffer() are called, I print the page currently being released which is identified by the tuple (tableSpace, dbNode, rel

Re: [HACKERS] WIP: dynahash replacement for buffer table

2014-10-14 Thread Robert Haas
On Tue, Oct 14, 2014 at 11:31 AM, Andres Freund wrote: >> It doesn't look particularly dangerous to me. Famous last words. > >> Basically, I think what we're doing right now is holding the buffer >> mapping lock so that the buffer can't be renamed under us while we're >> pinning it. > > What I'm

Re: [HACKERS] WIP: dynahash replacement for buffer table

2014-10-14 Thread Andres Freund
On 2014-10-14 11:08:08 -0400, Robert Haas wrote: > On Tue, Oct 14, 2014 at 10:47 AM, Andres Freund > wrote: > > On 2014-10-14 09:30:58 -0400, Robert Haas wrote: > >> I took the basic infrastructure from before and used it to replace the > >> buffer table. Patch is attached. > > > > Hm. Unless I

Re: [HACKERS] WIP: dynahash replacement for buffer table

2014-10-14 Thread Robert Haas
On Tue, Oct 14, 2014 at 10:25 AM, Andres Freund wrote: >> The key idea here is that lookups are done without any locks, only >> memory barriers; and inserts and deletes are done using atomic ops. > > Hm. I quickly looked and I see that you use two full barriers for every > lookup. That's pretty ex

Re: [HACKERS] Drop any statistics of table after it's truncated

2014-10-14 Thread Sawada Masahiko
On Tue, Oct 14, 2014 at 11:20 PM, Tom Lane wrote: > Sawada Masahiko writes: >> I found that the statistics are still remained after it's truncated. >> In addition, the analyzing ignores table does not have any tuple. >> After table truncated, the entry of statistics continues to remain >> unless

Re: [HACKERS] WIP: dynahash replacement for buffer table

2014-10-14 Thread Amit Kapila
On Tue, Oct 14, 2014 at 8:38 PM, Andres Freund wrote: > > On 2014-10-14 20:30:45 +0530, Amit Kapila wrote: > > On Tue, Oct 14, 2014 at 7:55 PM, Andres Freund > > wrote: > > > > > > On 2014-10-14 09:30:58 -0400, Robert Haas wrote: > > > > A few years ago I started working on a concurrent hash tabl

Re: [HACKERS] WIP: dynahash replacement for buffer table

2014-10-14 Thread Robert Haas
On Tue, Oct 14, 2014 at 10:47 AM, Andres Freund wrote: > On 2014-10-14 09:30:58 -0400, Robert Haas wrote: >> I took the basic infrastructure from before and used it to replace the >> buffer table. Patch is attached. > > Hm. Unless I missed something you pretty much completely eradicated > locks f

Re: [HACKERS] WIP: dynahash replacement for buffer table

2014-10-14 Thread Andres Freund
On 2014-10-14 20:30:45 +0530, Amit Kapila wrote: > On Tue, Oct 14, 2014 at 7:55 PM, Andres Freund > wrote: > > > > On 2014-10-14 09:30:58 -0400, Robert Haas wrote: > > > A few years ago I started working on a concurrent hash table for > > > PostgreSQL. The hash table part of it worked, but I neve

Re: [HACKERS] pg_get_indexdef() doesn't quote string reloptions

2014-10-14 Thread Eric B. Ridge
> If this communication is in fact intended to be protected by some > legal privilege, or to remain company confidential, you have > definitely sent it to the wrong place. Sadly I don't control my company's email server. They however don't control my gmail account. I'll switch to that. eri

Re: [HACKERS] Shapes on the regression test for polygon

2014-10-14 Thread Bruce Momjian
On Tue, Oct 14, 2014 at 05:40:06PM +0300, Emre Hasegeli wrote: > > I extracted Emre's diagram adjustments from the patch and applied it, > > and no tabs now. Emre, I assume your regression changes did not affect > > the diagram contents. > > Thank you for looking at it. I wanted to make the test

Re: [HACKERS] WIP: dynahash replacement for buffer table

2014-10-14 Thread Amit Kapila
On Tue, Oct 14, 2014 at 7:55 PM, Andres Freund wrote: > > On 2014-10-14 09:30:58 -0400, Robert Haas wrote: > > A few years ago I started working on a concurrent hash table for > > PostgreSQL. The hash table part of it worked, but I never did > > anything with it, really. Amit mentioned to me ear

Re: [HACKERS] Better support of exported snapshots with pg_dump

2014-10-14 Thread Petr Jelinek
On 22/09/14 02:24, Michael Paquier wrote: On Thu, Sep 4, 2014 at 11:33 PM, Michael Paquier Taking a dump consistent with a replication slot is useful for online upgrade cases first, because you can simply run pg_dump, have a slot created, and get as well a state of the database consistent with t

Re: [HACKERS] WIP: dynahash replacement for buffer table

2014-10-14 Thread Andres Freund
On 2014-10-14 09:30:58 -0400, Robert Haas wrote: > I took the basic infrastructure from before and used it to replace the > buffer table. Patch is attached. Hm. Unless I missed something you pretty much completely eradicated locks from the buffer mapping code? That'd be pretty cool, but it's also

Re: [HACKERS] Shapes on the regression test for polygon

2014-10-14 Thread Emre Hasegeli
> I extracted Emre's diagram adjustments from the patch and applied it, > and no tabs now. Emre, I assume your regression changes did not affect > the diagram contents. Thank you for looking at it. I wanted to make the tests consistent with the diagrams. Now they look better but they still don'

Re: [HACKERS] WIP: dynahash replacement for buffer table

2014-10-14 Thread Andres Freund
On 2014-10-14 09:30:58 -0400, Robert Haas wrote: > A few years ago I started working on a concurrent hash table for > PostgreSQL. The hash table part of it worked, but I never did > anything with it, really. Amit mentioned to me earlier this week that > he was seeing contention inside the dynahas

Re: [HACKERS] Drop any statistics of table after it's truncated

2014-10-14 Thread Tom Lane
Sawada Masahiko writes: > I found that the statistics are still remained after it's truncated. > In addition, the analyzing ignores table does not have any tuple. > After table truncated, the entry of statistics continues to remain > unless insertion and analyzing are executed. > There is reason w

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.9

2014-10-14 Thread Andres Freund
On 2014-10-14 08:40:49 -0500, Merlin Moncure wrote: > On Fri, Oct 10, 2014 at 11:00 AM, Andres Freund > wrote: > > On 2014-10-10 16:41:39 +0200, Andres Freund wrote: > >> FWIW, the profile always looks like > >> - 48.61% postgres postgres [.] s_lock > >>- s_lock > >>

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.9

2014-10-14 Thread Merlin Moncure
On Fri, Oct 10, 2014 at 11:00 AM, Andres Freund wrote: > On 2014-10-10 16:41:39 +0200, Andres Freund wrote: >> FWIW, the profile always looks like >> - 48.61% postgres postgres [.] s_lock >>- s_lock >> + 96.67% StrategyGetBuffer >> + 1.19% UnpinBuffer >> +

[HACKERS] Drop any statistics of table after it's truncated

2014-10-14 Thread Sawada Masahiko
Hi all, I found that the statistics are still remained after it's truncated. In addition, the analyzing ignores table does not have any tuple. After table truncated, the entry of statistics continues to remain unless insertion and analyzing are executed. There is reason why statistics are remained

[HACKERS] WIP: dynahash replacement for buffer table

2014-10-14 Thread Robert Haas
A few years ago I started working on a concurrent hash table for PostgreSQL. The hash table part of it worked, but I never did anything with it, really. Amit mentioned to me earlier this week that he was seeing contention inside the dynahash machinery, which inspired me to go back and update the

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2014-10-14 Thread Stephen Frost
Abhijit, * Abhijit Menon-Sen (a...@2ndquadrant.com) wrote: > As before, the pgaudit code is at https://github.com/2ndQuadrant/pgaudit > I did a quick round of testing to make sure things still work. Thanks! I had a very interesting discussion about auditing rules and the need to limit what gets

Re: [HACKERS] [9.4 bug] The database server hangs with write-heavy workload on Windows

2014-10-14 Thread MauMau
From: "Heikki Linnakangas" Committed this. Thank you very much. I didn't anticipate such a difficult complicated cause. The user agreed to try the patch tonight. I'll report back the result as soon as I got it from him. BTW, in LWLockWaitForVar(), the first line of the following code fr

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2014-10-14 Thread Abhijit Menon-Sen
At 2014-10-14 18:05:00 +0530, a...@2ndquadrant.com wrote: > > As before, the pgaudit code is at > https://github.com/2ndQuadrant/pgaudit Sorry, I forgot to attach the tarball. -- Abhijit pgaudit.tar.gz Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.o

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2014-10-14 Thread Abhijit Menon-Sen
Hi. As before, the pgaudit code is at https://github.com/2ndQuadrant/pgaudit I did a quick round of testing to make sure things still work. I'll re-add it to the next CF now. -- Abhijit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] inherit support for foreign tables

2014-10-14 Thread Etsuro Fujita
(2014/09/12 16:30), Etsuro Fujita wrote: (2014/09/11 20:51), Heikki Linnakangas wrote: On 09/11/2014 02:30 PM, Etsuro Fujita wrote: So, should I split the patch into the two? Yeah, please do. OK, Will do. Here are separated patches. fdw-chk.patch - CHECK constraints on foreign tables fd

Re: [HACKERS] Scaling shared buffer eviction

2014-10-14 Thread Andres Freund
On 2014-10-14 15:24:57 +0530, Amit Kapila wrote: > After that I observed that contention for LW_SHARED has reduced > for this load, but it didn't help much in terms of performance, so I again > rechecked the profile and this time most of the contention is moved > to spinlock used in dynahash for bu

Re: [HACKERS] Scaling shared buffer eviction

2014-10-14 Thread Amit Kapila
On Thu, Oct 9, 2014 at 6:17 PM, Amit Kapila wrote: > > On Fri, Sep 26, 2014 at 7:04 PM, Robert Haas wrote: > > > > On another point, I think it would be a good idea to rebase the > > bgreclaimer patch over what I committed, so that we have a > > clean patch against master to test with. > > Please

Re: [HACKERS] pg_receivexlog --status-interval add fsync feedback

2014-10-14 Thread furuyao
> >> >>> If we remove --fsync-interval, resoponse time to user will not > be > >> delay. > >> >>> Although, fsync will be executed multiple times in a short period. > >> >>> And there is no way to solve the problem without > >> >>> --fsync-interval, what > >> >> should we do about it? > >> >> > >>

Re: [HACKERS] Commitfest progress, or lack thereof

2014-10-14 Thread Andres Freund
On 2014-10-13 21:01:57 +0300, Heikki Linnakangas wrote: > The August commitfest is still Open, with a few more patches left. The > patches that remain have stayed in limbo for a long time. It's not realistic > to expect anything to happen to them. > > I'm going to move the remaining patches to the

Re: [HACKERS] Commitfest progress, or lack thereof

2014-10-14 Thread Amit Kapila
On Mon, Oct 13, 2014 at 11:31 PM, Heikki Linnakangas < hlinnakan...@vmware.com> wrote: > > I'm going to move the remaining patches to the next commitfest, and close the August one. Many thanks for managing commit fest in a best possible way. I think it is big bonanza for all the authors who have

Re: [HACKERS] split builtins.h to quote.h

2014-10-14 Thread Michael Paquier
On Mon, Oct 13, 2014 at 11:14 PM, Robert Haas wrote: > > IMHO, putting some prototypes for a .c file in one header and others > in another header is going to make it significantly harder to figure > out which files you need to #include when. Keeping a simple rule there > seems essential to me. OK

Re: [HACKERS] [9.4 bug] The database server hangs with write-heavy workload on Windows

2014-10-14 Thread Heikki Linnakangas
On 10/13/2014 06:57 PM, Heikki Linnakangas wrote: Hmm, we could set releaseOK in LWLockWaitForVar(), though, when it (re-)queues the backend. That would be less invasive, for sure (attached). I like this better. Committed this. - Heikki -- Sent via pgsql-hackers mailing list (pgsql-hackers@

Re: [HACKERS] Hide 'Execution time' in EXPLAIN (COSTS OFF)

2014-10-14 Thread David Rowley
On Tue, Oct 14, 2014 at 6:04 AM, Tom Lane wrote: > Andres Freund writes: > > Well. Unless I miss something it doesn't resolve the problem that > > started this thread. Namely that it's currently impossible to write > > regression using EXPLAIN (ANALYZE, TIMING OFF. COSTS OFF). Which is > > worth