Re: [HACKERS] PATCH: Reducing lock strength of trigger and foreign key DDL

2014-12-24 Thread Michael Paquier
On Sun, Dec 14, 2014 at 5:45 AM, Andreas Karlsson andr...@proxel.se wrote: get_rule_expr() relies heavily on the catcache which to me does not look like it could easily be (and probably not even should be) made to use the current snapshot. Refactoring ruleutils.c to rely less no the catcache

Re: hash_create API changes (was Re: [HACKERS] speedup tidbitmap patch: hash BlockNumber)

2014-12-24 Thread Andres Freund
On 2014-12-24 00:27:39 -0600, Jim Nasby wrote: pgbench -S -T10 -c 4 -j 4 master: tps = 9556.356145 (excluding connections establishing) tps = 9897.324917 (excluding connections establishing) tps = 9287.286907 (excluding connections establishing) tps = 10210.130833 (excluding connections

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-12-24 Thread Dilip kumar
On 19 December 2014 16:41, Amit Kapila Wrote, One idea is to send all the stages and corresponding Analyze commands to server in one go which means something like BEGIN; SET default_statistics_target=1; SET vacuum_cost_delay=0; Analyze t1; COMMIT; BEGIN; SET default_statistics_target=10; RESET

Re: [HACKERS] replicating DROP commands across servers

2014-12-24 Thread Andres Freund
On 2014-12-24 21:54:20 +1300, David Rowley wrote: On 24 December 2014 at 07:41, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I have pushed patches 0001 through 0004, with some revisions. Only the final part is missing. Hi Alvaro, Would you be able to commit the attached? It just

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-24 Thread Fujii Masao
On Fri, Dec 19, 2014 at 12:19 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Thu, Dec 18, 2014 at 5:27 PM, Fujii Masao masao.fu...@gmail.com wrote: Thanks! Thanks for your input. +else +memcpy(compression_scratch, page, page_len); I don't think

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-24 Thread Michael Paquier
On Wed, Dec 24, 2014 at 8:44 PM, Fujii Masao masao.fu...@gmail.com wrote: On Fri, Dec 19, 2014 at 12:19 AM, Michael Paquier michael.paqu...@gmail.com wrote: Firstly I'm thinking to commit the 0001-Move-pg_lzcompress.c-to-src-common.patch. pg_lzcompress.h still exists in include/utils, but it

Re: [HACKERS] speedup tidbitmap patch: cache page

2014-12-24 Thread David Rowley
On 24 December 2014 at 00:24, Teodor Sigaev teo...@sigaev.ru wrote: I've also attached some benchmark results using your original table from up-thread. It seems that the caching if the page was seen as lossy is not much of a help in this test case. Did you find another one where you saw some

[HACKERS] Moving RestoreBlockImage from xlogreader.c to xlogutils.c

2014-12-24 Thread Michael Paquier
Hi all, Commit 2c03216d has introduced RestoreBlockImage to restore a page from a given decoding state. ISTM that this is a backend-only operation but it has been added in xlogreader.c which could be used as well by frontend utilities like pg_xlogdump. Wouldn't it be better to declare it as a

Re: [HACKERS] Commit timestamp abbreviations

2014-12-24 Thread Alvaro Herrera
Bruce Momjian wrote: I noticed this when looking at the allocated shared memory structures in head: shared memory alignment 64-byte of CommitTs Ctl: 0 shared memory alignment 64-byte of CommitTs shared: 0 I thought we got rid of the idea that 'Ts' means timestamp. Was this

Re: [HACKERS] pg_basebackup fails with long tablespace paths

2014-12-24 Thread Peter Eisentraut
On 12/22/14 5:40 PM, Oskari Saarenmaa wrote: I think we should just use the UStar tar format (http://en.wikipedia.org/wiki/Tar_%28computing%29#UStar_format) and allow long file names; all actively used tar implementations should be able to handle them. I'll try to write a patch for that

Re: [HACKERS] pg_basebackup fails with long tablespace paths

2014-12-24 Thread Peter Eisentraut
On 12/22/14 10:00 PM, Amit Kapila wrote: There is already a patch [1] in this CF which will handle both cases, so I am not sure if it is very good idea to go with a new tar format to handle this issue. I think it would still make sense to have proper symlinks in the basebackup if possible,

Re: [HACKERS] Moving RestoreBlockImage from xlogreader.c to xlogutils.c

2014-12-24 Thread Fujii Masao
On Wed, Dec 24, 2014 at 9:42 PM, Michael Paquier michael.paqu...@gmail.com wrote: Hi all, Commit 2c03216d has introduced RestoreBlockImage to restore a page from a given decoding state. ISTM that this is a backend-only operation but it has been added in xlogreader.c which could be used as

Re: [HACKERS] Moving RestoreBlockImage from xlogreader.c to xlogutils.c

2014-12-24 Thread Michael Paquier
On Wed, Dec 24, 2014 at 10:16 PM, Fujii Masao masao.fu...@gmail.com wrote: On Wed, Dec 24, 2014 at 9:42 PM, Michael Paquier michael.paqu...@gmail.com wrote: Wouldn't it be better to declare it as a static routine in xlogutils.c? If we keep it in xlogreader.c, I think that we should at least

Re: [HACKERS] Moving RestoreBlockImage from xlogreader.c to xlogutils.c

2014-12-24 Thread Fujii Masao
On Wed, Dec 24, 2014 at 10:41 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Wed, Dec 24, 2014 at 10:16 PM, Fujii Masao masao.fu...@gmail.com wrote: On Wed, Dec 24, 2014 at 9:42 PM, Michael Paquier michael.paqu...@gmail.com wrote: Wouldn't it be better to declare it as a static

Re: [HACKERS] replicating DROP commands across servers

2014-12-24 Thread David Rowley
On 25 December 2014 at 00:34, Andres Freund and...@2ndquadrant.com wrote: On 2014-12-24 21:54:20 +1300, David Rowley wrote: On 24 December 2014 at 07:41, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I have pushed patches 0001 through 0004, with some revisions. Only the final part

Re: [HACKERS] Commit timestamp abbreviations

2014-12-24 Thread Bruce Momjian
On Tue, Dec 23, 2014 at 06:00:21PM -0300, Alvaro Herrera wrote: Bruce Momjian wrote: I noticed this when looking at the allocated shared memory structures in head: shared memory alignment 64-byte of CommitTs Ctl: 0 shared memory alignment 64-byte of CommitTs shared: 0 I

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

2014-12-24 Thread Bruce Momjian
On Wed, Dec 24, 2014 at 10:30:19AM +0100, Andres Freund wrote: On 2014-12-23 22:51:22 -0500, Bruce Momjian wrote: Many of these are 64-byte aligned, including Buffer Descriptors. In that case you need to change max_connections, some settings will lead to unaligned BufferDescriptors. Well,

Re: [HACKERS] replicating DROP commands across servers

2014-12-24 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: I really wonder if we can't make msvc reliably recognize this kind of scenario - especially this case is pretty trivial? Even if MSVC did understand pg_unreachable(), there would be other compilers that didn't, so we'd need to worry about suppressing

Re: [HACKERS] replicating DROP commands across servers

2014-12-24 Thread Tom Lane
David Rowley dgrow...@gmail.com writes: On 25 December 2014 at 00:34, Andres Freund and...@2ndquadrant.com wrote: I really wonder if we can't make msvc reliably recognize this kind of scenario - especially this case is pretty trivial? The attached patch removes the warning, but likely can't

Re: [HACKERS] Commit timestamp abbreviations

2014-12-24 Thread Petr Jelinek
On 24/12/14 15:15, Bruce Momjian wrote: On Tue, Dec 23, 2014 at 06:00:21PM -0300, Alvaro Herrera wrote: Bruce Momjian wrote: I noticed this when looking at the allocated shared memory structures in head: shared memory alignment 64-byte of CommitTs Ctl: 0 shared memory

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

2014-12-24 Thread Andres Freund
On 2014-12-24 10:00:05 -0500, Bruce Momjian wrote: On Wed, Dec 24, 2014 at 10:30:19AM +0100, Andres Freund wrote: On 2014-12-23 22:51:22 -0500, Bruce Momjian wrote: Many of these are 64-byte aligned, including Buffer Descriptors. In that case you need to change max_connections, some

Re: hash_create API changes (was Re: [HACKERS] speedup tidbitmap patch: hash BlockNumber)

2014-12-24 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-12-24 00:27:39 -0600, Jim Nasby wrote: pgbench -S -T10 -c 4 -j 4 master: tps = 9556.356145 (excluding connections establishing) tps = 9897.324917 (excluding connections establishing) tps = 9287.286907 (excluding connections establishing)

Re: [HACKERS] Additional role attributes superuser review

2014-12-24 Thread Adam Brightwell
All, I want to revive this thread and continue to move these new role attributes forward. In summary, the ultimate goal is to include new role attributes for common operations which currently require superuser privileges. Initially proposed were the following attributes: * BACKUP - allows role

[HACKERS] nls and server log

2014-12-24 Thread Euler Taveira
Hi, Currently the same message goes to server log and client app. Sometimes it bothers me since I have to analyze server logs and discovered that lc_messages is set to pt_BR and to worse things that stup^H^H^H application parse some error messages in portuguese. My solution has been a modified

Re: [HACKERS] Proposal VACUUM SCHEMA

2014-12-24 Thread Jim Nasby
On 12/23/14, 8:49 PM, Fabrízio de Royes Mello wrote: Em terça-feira, 23 de dezembro de 2014, Jim Nasby jim.na...@bluetreble.com mailto:jim.na...@bluetreble.com escreveu: On 12/23/14, 8:54 AM, Fabrízio de Royes Mello wrote: Right now a lot of people just work around this with

Re: [HACKERS] Turning recovery.conf into GUCs

2014-12-24 Thread Alex Shulgin
Alex Shulgin a...@commandprompt.com writes: Petr Jelinek p...@2ndquadrant.com writes: I had a quick look, the patch does not apply cleanly anymore but it's just release notes so nothing too bad. Yes, there were some ongoing changes that touched some parts of this and I must have missed the

Re: hash_create API changes (was Re: [HACKERS] speedup tidbitmap patch: hash BlockNumber)

2014-12-24 Thread Jim Nasby
On 12/24/14, 10:58 AM, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-12-24 00:27:39 -0600, Jim Nasby wrote: pgbench -S -T10 -c 4 -j 4 master: tps = 9556.356145 (excluding connections establishing) tps = 9897.324917 (excluding connections establishing) tps = 9287.286907

Re: [HACKERS] replicating DROP commands across servers

2014-12-24 Thread David Rowley
On 25 December 2014 at 04:47, Tom Lane t...@sss.pgh.pa.us wrote: David Rowley dgrow...@gmail.com writes: On 25 December 2014 at 00:34, Andres Freund and...@2ndquadrant.com wrote: I really wonder if we can't make msvc reliably recognize this kind of scenario - especially this case is

Re: hash_create API changes (was Re: [HACKERS] speedup tidbitmap patch: hash BlockNumber)

2014-12-24 Thread Tom Lane
Jim Nasby jim.na...@bluetreble.com writes: On 12/24/14, 10:58 AM, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: FWIW, I don't believe these results for one second. It's quite plausible that there's a noticeable performance benefit, but a factor of three is completely

Re: [HACKERS] Moving RestoreBlockImage from xlogreader.c to xlogutils.c

2014-12-24 Thread Michael Paquier
On Wed, Dec 24, 2014 at 10:51 PM, Fujii Masao masao.fu...@gmail.com wrote: Fair enough. Anyway I wait for applying the patch which moves pg_lzcompress.c until we will have reached any consensus about this. Just to be clear (after sleeping on it), we still need pglz stuff in src/common to offer

Re: ctidscan as an example of custom-scan (Re: [HACKERS] [v9.5] Custom Plan API)

2014-12-24 Thread Kouhei Kaigai
On Mon, Dec 15, 2014 at 4:55 PM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: I'm not certain whether we should have this functionality in contrib from the perspective of workload that can help, but its major worth is for an example of custom-scan interface. worker_spi is now in

Re: [HACKERS] Securing make check (CVE-2014-0067)

2014-12-24 Thread David Rowley
On 30 November 2014 at 15:02, Noah Misch n...@leadboat.com wrote: On Sun, Sep 21, 2014 at 02:31:15AM -0400, Noah Misch wrote: It then dawned on me that every Windows build of PostgreSQL already has a way to limit connections to a particular OS user. SSPI authentication is essentially the

Re: [HACKERS] Securing make check (CVE-2014-0067)

2014-12-24 Thread Michael Paquier
On Thu, Dec 25, 2014 at 11:55 AM, David Rowley dgrowle...@gmail.com wrote: f6dc6dd seems to have broken vcregress check for me: Having a look at the pg_hba.conf that's been generated by pgregress, it looks like it only adds a line for IPv4 addresses. Indeed. I can see this problem as well on my

Re: [HACKERS] Securing make check (CVE-2014-0067)

2014-12-24 Thread Noah Misch
On Thu, Dec 25, 2014 at 03:55:02PM +1300, David Rowley wrote: f6dc6dd seems to have broken vcregress check for me: FATAL: no pg_hba.conf entry for host ::1, user David, database postgres ... FATAL: no pg_hba.conf entry for host ::1, user David, database postgres Thanks. I bet this is

Re: [HACKERS] What exactly is our CRC algorithm?

2014-12-24 Thread Abhijit Menon-Sen
Hi. Here's a proposed patch to use CPUID at startup to determine if the SSE4.2 CRC instructions are available, to use them instead of the slice-by-8 implementation (posted earlier). A few notes: 1. GCC has included cpuid.h since 4.3.0, so I figured it was safe to use. It can be replaced with