Re: [HACKERS] What exactly is our CRC algorithm?

2014-12-24 Thread Abhijit Menon-Sen
Hi. Here's a proposed patch to use CPUID at startup to determine if the SSE4.2 CRC instructions are available, to use them instead of the slice-by-8 implementation (posted earlier). A few notes: 1. GCC has included cpuid.h since 4.3.0, so I figured it was safe to use. It can be replaced with

Re: [HACKERS] Securing "make check" (CVE-2014-0067)

2014-12-24 Thread Noah Misch
On Thu, Dec 25, 2014 at 03:55:02PM +1300, David Rowley wrote: > f6dc6dd seems to have broken vcregress check for me: > FATAL: no pg_hba.conf entry for host "::1", user "David", database > "postgres" > ... > FATAL: no pg_hba.conf entry for host "::1", user "David", database > "postgres" Thanks.

Re: [HACKERS] Securing "make check" (CVE-2014-0067)

2014-12-24 Thread Michael Paquier
On Thu, Dec 25, 2014 at 11:55 AM, David Rowley wrote: > f6dc6dd seems to have broken vcregress check for me: > Having a look at the pg_hba.conf that's been generated by pgregress, it > looks like it only adds a line for IPv4 addresses. Indeed. I can see this problem as well on my win7 box, and you

Re: [HACKERS] Securing "make check" (CVE-2014-0067)

2014-12-24 Thread David Rowley
On 30 November 2014 at 15:02, Noah Misch wrote: > On Sun, Sep 21, 2014 at 02:31:15AM -0400, Noah Misch wrote: > > It then dawned on me that every Windows build of PostgreSQL already has > a way > > to limit connections to a particular OS user. SSPI authentication is > > essentially the Windows e

Re: ctidscan as an example of custom-scan (Re: [HACKERS] [v9.5] Custom Plan API)

2014-12-24 Thread Kouhei Kaigai
> > On Mon, Dec 15, 2014 at 4:55 PM, Kouhei Kaigai > wrote: > > > I'm not certain whether we should have this functionality in contrib > > > from the perspective of workload that can help, but its major worth > > > is for an example of custom-scan interface. > > worker_spi is now in src/test/modul

Re: [HACKERS] Moving RestoreBlockImage from xlogreader.c to xlogutils.c

2014-12-24 Thread Michael Paquier
On Wed, Dec 24, 2014 at 10:51 PM, Fujii Masao wrote: > Fair enough. Anyway I wait for applying the patch which moves pg_lzcompress.c > until we will have reached any consensus about this. Just to be clear (after sleeping on it), we still need pglz stuff in src/common to offer to the frontends the

Re: hash_create API changes (was Re: [HACKERS] speedup tidbitmap patch: hash BlockNumber)

2014-12-24 Thread Tom Lane
Jim Nasby writes: > On 12/24/14, 10:58 AM, Tom Lane wrote: >> Andres Freund writes: >>> FWIW, I don't believe these results for one second. It's quite plausible >>> that there's a noticeable performance benefit, but a factor of three is >>> completely unrealistic... Could you please recheck? >>

Re: [HACKERS] replicating DROP commands across servers

2014-12-24 Thread David Rowley
On 25 December 2014 at 04:47, Tom Lane wrote: > David Rowley writes: > > On 25 December 2014 at 00:34, Andres Freund > wrote: > >> I really wonder if we can't make msvc reliably recognize this kind of > >> scenario - especially this case is pretty trivial? > > > The attached patch removes the w

Re: hash_create API changes (was Re: [HACKERS] speedup tidbitmap patch: hash BlockNumber)

2014-12-24 Thread Jim Nasby
On 12/24/14, 10:58 AM, Tom Lane wrote: Andres Freund writes: On 2014-12-24 00:27:39 -0600, Jim Nasby wrote: pgbench -S -T10 -c 4 -j 4 master: tps = 9556.356145 (excluding connections establishing) tps = 9897.324917 (excluding connections establishing) tps = 9287.286907 (excluding connections e

Re: [HACKERS] Turning recovery.conf into GUCs

2014-12-24 Thread Alex Shulgin
Alex Shulgin writes: > Petr Jelinek writes: >> >> I had a quick look, the patch does not apply cleanly anymore but it's >> just release notes so nothing too bad. > > Yes, there were some ongoing changes that touched some parts of this and > I must have missed the latest one (or maybe I was waiti

Re: [HACKERS] Proposal "VACUUM SCHEMA"

2014-12-24 Thread Jim Nasby
On 12/23/14, 8:49 PM, Fabrízio de Royes Mello wrote: Em terça-feira, 23 de dezembro de 2014, Jim Nasby mailto:jim.na...@bluetreble.com>> escreveu: On 12/23/14, 8:54 AM, Fabrízio de Royes Mello wrote: > Right now a lot of people just work around this with things like DO blocks, bu

[HACKERS] nls and server log

2014-12-24 Thread Euler Taveira
Hi, Currently the same message goes to server log and client app. Sometimes it bothers me since I have to analyze server logs and discovered that lc_messages is set to pt_BR and to worse things that stup^H^H^H application parse some error messages in portuguese. My solution has been a modified ver

Re: [HACKERS] Additional role attributes && superuser review

2014-12-24 Thread Adam Brightwell
All, I want to revive this thread and continue to move these new role attributes forward. In summary, the ultimate goal is to include new role attributes for common operations which currently require superuser privileges. Initially proposed were the following attributes: * BACKUP - allows role

Re: hash_create API changes (was Re: [HACKERS] speedup tidbitmap patch: hash BlockNumber)

2014-12-24 Thread Tom Lane
Andres Freund writes: > On 2014-12-24 00:27:39 -0600, Jim Nasby wrote: >> pgbench -S -T10 -c 4 -j 4 >> master: >> tps = 9556.356145 (excluding connections establishing) >> tps = 9897.324917 (excluding connections establishing) >> tps = 9287.286907 (excluding connections establishing) >> tps = 1021

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

2014-12-24 Thread Andres Freund
On 2014-12-24 10:00:05 -0500, Bruce Momjian wrote: > On Wed, Dec 24, 2014 at 10:30:19AM +0100, Andres Freund wrote: > > On 2014-12-23 22:51:22 -0500, Bruce Momjian wrote: > > > Many of these are 64-byte aligned, including Buffer Descriptors. > > > > In that case you need to change max_connections,

Re: [HACKERS] Commit timestamp abbreviations

2014-12-24 Thread Petr Jelinek
On 24/12/14 15:15, Bruce Momjian wrote: On Tue, Dec 23, 2014 at 06:00:21PM -0300, Alvaro Herrera wrote: Bruce Momjian wrote: I noticed this when looking at the allocated shared memory structures in head: shared memory alignment 64-byte of CommitTs Ctl: 0 shared memory alignmen

Re: [HACKERS] replicating DROP commands across servers

2014-12-24 Thread Tom Lane
David Rowley writes: > On 25 December 2014 at 00:34, Andres Freund wrote: >> I really wonder if we can't make msvc reliably recognize this kind of >> scenario - especially this case is pretty trivial? > The attached patch removes the warning, but likely can't be used in case > someone somewhere

Re: [HACKERS] replicating DROP commands across servers

2014-12-24 Thread Tom Lane
Andres Freund writes: > I really wonder if we can't make msvc reliably recognize this kind of > scenario - especially this case is pretty trivial? Even if MSVC did understand pg_unreachable(), there would be other compilers that didn't, so we'd need to worry about suppressing such warnings anyhow

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

2014-12-24 Thread Bruce Momjian
On Wed, Dec 24, 2014 at 10:30:19AM +0100, Andres Freund wrote: > On 2014-12-23 22:51:22 -0500, Bruce Momjian wrote: > > Many of these are 64-byte aligned, including Buffer Descriptors. > > In that case you need to change max_connections, some settings will lead > to unaligned BufferDescriptors. W

Re: [HACKERS] Commit timestamp abbreviations

2014-12-24 Thread Bruce Momjian
On Tue, Dec 23, 2014 at 06:00:21PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > I noticed this when looking at the allocated shared memory structures in > > head: > > > > shared memory alignment 64-byte of CommitTs Ctl: 0 > > shared memory alignment 64-byte of CommitTs shared:

Re: [HACKERS] replicating DROP commands across servers

2014-12-24 Thread David Rowley
On 25 December 2014 at 00:34, Andres Freund wrote: > On 2014-12-24 21:54:20 +1300, David Rowley wrote: > > On 24 December 2014 at 07:41, Alvaro Herrera > > wrote: > > > > > I have pushed patches 0001 through 0004, with some revisions. Only the > > > final part is missing. > > > > > > > > Hi Alv

Re: [HACKERS] Moving RestoreBlockImage from xlogreader.c to xlogutils.c

2014-12-24 Thread Fujii Masao
On Wed, Dec 24, 2014 at 10:41 PM, Michael Paquier wrote: > On Wed, Dec 24, 2014 at 10:16 PM, Fujii Masao wrote: >> On Wed, Dec 24, 2014 at 9:42 PM, Michael Paquier >> wrote: >>> Wouldn't it be better to declare it as a static routine in >>> xlogutils.c? If we keep it in xlogreader.c, I think tha

Re: [HACKERS] Moving RestoreBlockImage from xlogreader.c to xlogutils.c

2014-12-24 Thread Michael Paquier
On Wed, Dec 24, 2014 at 10:16 PM, Fujii Masao wrote: > On Wed, Dec 24, 2014 at 9:42 PM, Michael Paquier > wrote: >> Wouldn't it be better to declare it as a static routine in >> xlogutils.c? If we keep it in xlogreader.c, I think that we should at >> least wrap it with ifndef FRONTEND. > > If we

Re: [HACKERS] Moving RestoreBlockImage from xlogreader.c to xlogutils.c

2014-12-24 Thread Fujii Masao
On Wed, Dec 24, 2014 at 9:42 PM, Michael Paquier wrote: > Hi all, > > Commit 2c03216d has introduced RestoreBlockImage to restore a page > from a given decoding state. ISTM that this is a backend-only > operation but it has been added in xlogreader.c which could be used as > well by frontend utili

Re: [HACKERS] pg_basebackup fails with long tablespace paths

2014-12-24 Thread Peter Eisentraut
On 12/22/14 10:00 PM, Amit Kapila wrote: > There is already a patch [1] in this CF which will handle both cases, so > I am > not sure if it is very good idea to go with a new tar format to handle this > issue. I think it would still make sense to have proper symlinks in the basebackup if possib

Re: [HACKERS] pg_basebackup fails with long tablespace paths

2014-12-24 Thread Peter Eisentraut
On 12/22/14 5:40 PM, Oskari Saarenmaa wrote: > I think we should just use the UStar tar format > (http://en.wikipedia.org/wiki/Tar_%28computing%29#UStar_format) and > allow long file names; all actively used tar implementations should be > able to handle them. I'll try to write a patch for that so

Re: [HACKERS] Commit timestamp abbreviations

2014-12-24 Thread Alvaro Herrera
Bruce Momjian wrote: > I noticed this when looking at the allocated shared memory structures in > head: > > shared memory alignment 64-byte of CommitTs Ctl: 0 > shared memory alignment 64-byte of CommitTs shared: 0 > > I thought we got rid of the idea that 'Ts' means timestamp. Was

[HACKERS] Moving RestoreBlockImage from xlogreader.c to xlogutils.c

2014-12-24 Thread Michael Paquier
Hi all, Commit 2c03216d has introduced RestoreBlockImage to restore a page from a given decoding state. ISTM that this is a backend-only operation but it has been added in xlogreader.c which could be used as well by frontend utilities like pg_xlogdump. Wouldn't it be better to declare it as a stat

Re: [HACKERS] speedup tidbitmap patch: cache page

2014-12-24 Thread David Rowley
On 24 December 2014 at 00:24, Teodor Sigaev wrote: > I've also attached some benchmark results using your original table from >> > up-thread. It seems that the caching if the page was seen as lossy is not >> much >> of a help in this test case. Did you find another one where you saw some >> bette

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-24 Thread Michael Paquier
On Wed, Dec 24, 2014 at 8:44 PM, Fujii Masao wrote: > On Fri, Dec 19, 2014 at 12:19 AM, Michael Paquier > wrote: > Firstly I'm thinking to commit the > 0001-Move-pg_lzcompress.c-to-src-common.patch. > > pg_lzcompress.h still exists in include/utils, but it should be moved to > include/common? You

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-24 Thread Fujii Masao
On Fri, Dec 19, 2014 at 12:19 AM, Michael Paquier wrote: > On Thu, Dec 18, 2014 at 5:27 PM, Fujii Masao wrote: >> Thanks! > Thanks for your input. > >> +else >> +memcpy(compression_scratch, page, page_len); >> >> I don't think the block image needs to be copied

Re: [HACKERS] replicating DROP commands across servers

2014-12-24 Thread Andres Freund
On 2014-12-24 21:54:20 +1300, David Rowley wrote: > On 24 December 2014 at 07:41, Alvaro Herrera > wrote: > > > I have pushed patches 0001 through 0004, with some revisions. Only the > > final part is missing. > > > > > Hi Alvaro, > > Would you be able to commit the attached? It just fixes a ne

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-12-24 Thread Dilip kumar
On 19 December 2014 16:41, Amit Kapila Wrote, >One idea is to send all the stages and corresponding Analyze commands >to server in one go which means something like >"BEGIN; SET default_statistics_target=1; SET vacuum_cost_delay=0; > Analyze t1; COMMIT;" >"BEGIN; SET default_statistics_target=10;

Re: hash_create API changes (was Re: [HACKERS] speedup tidbitmap patch: hash BlockNumber)

2014-12-24 Thread Andres Freund
On 2014-12-24 00:27:39 -0600, Jim Nasby wrote: > pgbench -S -T10 -c 4 -j 4 > master: > tps = 9556.356145 (excluding connections establishing) > tps = 9897.324917 (excluding connections establishing) > tps = 9287.286907 (excluding connections establishing) > tps = 10210.130833 (excluding connections

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

2014-12-24 Thread Andres Freund
On 2014-12-23 22:51:22 -0500, Bruce Momjian wrote: > Many of these are 64-byte aligned, including Buffer Descriptors. In that case you need to change max_connections, some settings will lead to unaligned BufferDescriptors. > I > tested pgbench with these commands: > > $ pgbench -i -s 95 pg

Re: [HACKERS] replicating DROP commands across servers

2014-12-24 Thread David Rowley
On 24 December 2014 at 07:41, Alvaro Herrera wrote: > I have pushed patches 0001 through 0004, with some revisions. Only the > final part is missing. > > Hi Alvaro, Would you be able to commit the attached? It just fixes a new compiler warning that I'm seeing on MSVC. src\backend\parser\parse_

Re: [HACKERS] PATCH: Reducing lock strength of trigger and foreign key DDL

2014-12-24 Thread Michael Paquier
On Sun, Dec 14, 2014 at 5:45 AM, Andreas Karlsson wrote: > get_rule_expr() relies heavily on the catcache which to me does not look > like it could easily be (and probably not even should be) made to use the > current snapshot. Refactoring ruleutils.c to rely less no the catcache seems > like a re