Re: [HACKERS] Parallel Seq Scan

2015-01-29 Thread Jeff Janes
On Tue, Jan 27, 2015 at 11:08 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 01/28/2015 04:16 AM, Robert Haas wrote: On Tue, Jan 27, 2015 at 6:00 PM, Robert Haas robertmh...@gmail.com wrote: Now, when you did what I understand to be the same test on the same machine, you got

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread Bruce Momjian
On Wed, Jan 28, 2015 at 09:26:11PM -0800, Josh Berkus wrote: 3. Check that the replica is not very lagged. If it is, wait for traffic to die down and for it to catch up. Is this necessary. It seems quite imprecise too. 4. Shut down the master using -m fast or -m smart for a clean shutdown.

Re: [HACKERS] Parallel Seq Scan

2015-01-29 Thread Tom Lane
Jeff Janes jeff.ja...@gmail.com writes: On Tue, Jan 27, 2015 at 11:08 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: OTOH, spreading the I/O across multiple files is not a good thing, if you don't have a RAID setup like that. With a single spindle, you'll just induce more seeks.

Re: [HACKERS] TABLESAMPLE patch

2015-01-29 Thread Bruce Momjian
On Thu, Jan 29, 2015 at 11:08:55AM -0500, Robert Haas wrote: On Wed, Jan 28, 2015 at 5:19 AM, Petr Jelinek p...@2ndquadrant.com wrote: Yes, that's my view too. I would generally be for that change also and it would be worth it if the code was used in more than one place, but as it is it

Re: [HACKERS] Memory leak in gingetbitmap

2015-01-29 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: [ assorted GIN leaks ] I think we need a more whole-sale approach. I'm thinking of adding a new memory context to contain everything related to the scan keys, which can then be destroyed in whole. We haven't heard any complaints about

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread Bruce Momjian
On Thu, Jan 29, 2015 at 10:21:30AM -0500, Andrew Dunstan wrote: On 01/29/2015 12:26 AM, Josh Berkus wrote: So, for my 2c, I'm on the fence about it. On the one hand, I agree, it's a bit of a complex process to get right. On the other hand, it's far better if we put something out there

Re: [HACKERS] Exposing the stats snapshot timestamp to SQL

2015-01-29 Thread Robert Haas
On Thu, Jan 29, 2015 at 12:18 AM, Matt Kelly mkell...@gmail.com wrote: In a previous thread Tom Lane said: (I'm also wondering if it'd make sense to expose the stats timestamp as a callable function, so that the case could be dealt with programmatically as well. But that's future-feature

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread Bruce Momjian
On Tue, Jan 27, 2015 at 10:16:48PM -0500, David Steele wrote: This is definitely an edge case. Not only does the file have to be modified in the same second *after* rsync has done the copy, but the file also has to not be modified in *any other subsequent second* before the next incremental

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread Andrew Dunstan
On 01/29/2015 12:26 AM, Josh Berkus wrote: So, for my 2c, I'm on the fence about it. On the one hand, I agree, it's a bit of a complex process to get right. On the other hand, it's far better if we put something out there along the lines of if you really want to, this is how to do it than

Re: [HACKERS] Parallel Seq Scan

2015-01-29 Thread Robert Haas
On Thu, Jan 29, 2015 at 11:40 AM, Tom Lane t...@sss.pgh.pa.us wrote: In my experience with RAID, it is smart enough to take advantage of that. If the raid controller detects a sequential access pattern read, it initiates a read ahead on each disk to pre-position the data it will need (or at

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread Andrew Dunstan
On 01/29/2015 11:34 AM, Bruce Momjian wrote: On Thu, Jan 29, 2015 at 10:21:30AM -0500, Andrew Dunstan wrote: On 01/29/2015 12:26 AM, Josh Berkus wrote: So, for my 2c, I'm on the fence about it. On the one hand, I agree, it's a bit of a complex process to get right. On the other hand, it's

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-29 Thread Robert Haas
On Wed, Jan 28, 2015 at 12:48 PM, Tom Lane t...@sss.pgh.pa.us wrote: The cause of this bug is commit 0ad1a816320a2b539a51628e2a0b1e83ff096b1d, which I'm inclined to think we need to simply revert, not render even more squirrely. Yes, that commit looks broken. If you convert from text to JSON

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread Bruce Momjian
On Thu, Jan 29, 2015 at 12:09:58PM -0500, Andrew Dunstan wrote: 7. shut down postgres on the replica. 8. rsync both the old and new data directories from the master to the replica, using the --size-only and -H hard links options. For example, if both 9.3 and 9.4 are in /var/lib/postgresql,

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2015-01-29 Thread Pavel Stehule
2015-01-29 10:28 GMT+01:00 Fabrízio de Royes Mello fabriziome...@gmail.com : Em quinta-feira, 29 de janeiro de 2015, Pavel Stehule pavel.steh...@gmail.com escreveu: Hi I am testing this feature on relative complex schema (38619 tables in db) and I got deadlock [pavel@localhost bin]$

Re: [HACKERS] Possible typo in create_policy.sgml

2015-01-29 Thread Dean Rasheed
On 29 January 2015 at 04:00, Stephen Frost sfr...@snowman.net wrote: * Robert Haas (robertmh...@gmail.com) wrote: On Wed, Jan 7, 2015 at 3:06 PM, Stephen Frost sfr...@snowman.net wrote: If I'm following correctly, Peter's specifically talking about: [ USING ( replaceable

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2015-01-29 Thread Fabrízio de Royes Mello
Em quinta-feira, 29 de janeiro de 2015, Pavel Stehule pavel.steh...@gmail.com escreveu: Hi I am testing this feature on relative complex schema (38619 tables in db) and I got deadlock [pavel@localhost bin]$ /usr/local/pgsql/bin/vacuumdb test2 -fz -j 4 vacuumdb: vacuuming database test2

[HACKERS] small postgresql.conf.sample bugfix

2015-01-29 Thread Pavel Stehule
Hello a max_locks_per_transaction is twice times in postgresql.conf.sample Attached simple bugfix Regards Pavel commit 89ee480408e63fc39fb6b7a23871c4f46438b7a2 Author: Pavel Stehule pavel.steh...@gooddata.com Date: Thu Jan 29 12:21:07 2015 +0100 remove twice max_locks_per_transaction

Re: [HACKERS] small postgresql.conf.sample bugfix

2015-01-29 Thread Thom Brown
On 29 January 2015 at 11:35, Pavel Stehule pavel.steh...@gmail.com wrote: Hello a max_locks_per_transaction is twice times in postgresql.conf.sample Attached simple bugfix I only see one entry, which is the one you're removing. Where's the other? Thom

Re: [HACKERS] Providing catalog view to pg_hba.conf file - Patch submission

2015-01-29 Thread Fabrízio de Royes Mello
On Wed, Jan 28, 2015 at 5:27 PM, Tom Lane t...@sss.pgh.pa.us wrote: =?UTF-8?Q?Fabr=C3=ADzio_de_Royes_Mello?= fabriziome...@gmail.com writes: But I'm thinking about this patch and would not be interesting to have a FDW to manipulate the hba file? Imagine if we are able to manipulate the HBA

Re: [HACKERS] Safe memory allocation functions

2015-01-29 Thread Robert Haas
On Wed, Jan 28, 2015 at 9:34 AM, Michael Paquier michael.paqu...@gmail.com wrote: As a result of all the comments on this thread, here are 3 patches implementing incrementally the different ideas from everybody: 1) 0001 modifies aset.c to return unconditionally NULL in case of an OOM instead

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-29 Thread Noah Misch
On Wed, Jan 28, 2015 at 12:48:45PM -0500, Tom Lane wrote: Noah Misch n...@leadboat.com writes: On Tue, Jan 27, 2015 at 03:56:22PM -0500, Tom Lane wrote: So at this point I propose that we reject \u when de-escaping JSON. I would have agreed on 2014-12-09, and this release is the last

Re: [HACKERS] small postgresql.conf.sample bugfix

2015-01-29 Thread Pavel Stehule
2015-01-29 12:52 GMT+01:00 Thom Brown t...@linux.com: On 29 January 2015 at 11:35, Pavel Stehule pavel.steh...@gmail.com wrote: Hello a max_locks_per_transaction is twice times in postgresql.conf.sample Attached simple bugfix I only see one entry, which is the one you're removing.

Re: [HACKERS] Parallel Seq Scan

2015-01-29 Thread Amit Kapila
On Wed, Jan 28, 2015 at 8:59 PM, Amit Kapila amit.kapil...@gmail.com wrote: I have tried the tests again and found that I have forgotten to increase max_worker_processes due to which the data is so different. Basically at higher client count it is just scanning lesser number of blocks in

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-29 Thread Bruce Momjian
On Wed, Jan 28, 2015 at 03:13:47PM -0600, Jim Nasby wrote: While I sympathize with Noah's sentiments, the only thing that makes sense to me is that a JSON text field is treated the same way as we treat text. Right now, that means NUL is not allowed, period. +1 -- Bruce Momjian

Re: [HACKERS] Possible typo in create_policy.sgml

2015-01-29 Thread Robert Haas
On Wed, Jan 28, 2015 at 10:45 PM, Stephen Frost sfr...@snowman.net wrote: I agree, especially after going back and re-reading this while fixing the issue mentioned earlier by Peter (which was an orthogonal complaint about the shadowing of WITH CHECK by USING, if WITH CHECK isn't specified).

Re: [HACKERS] Possible typo in create_policy.sgml

2015-01-29 Thread Stephen Frost
* Dean Rasheed (dean.a.rash...@gmail.com) wrote: On 29 January 2015 at 04:00, Stephen Frost sfr...@snowman.net wrote: * Robert Haas (robertmh...@gmail.com) wrote: On Wed, Jan 7, 2015 at 3:06 PM, Stephen Frost sfr...@snowman.net wrote: If I'm following correctly, Peter's specifically

Re: [HACKERS] pg_check_dir comments and implementation mismatch

2015-01-29 Thread Marco Nenciarini
Il 29/01/15 18:37, Robert Haas ha scritto: On Thu, Jan 29, 2015 at 11:00 AM, Marco Nenciarini marco.nenciar...@2ndquadrant.it wrote: reading pgcheckdir.c code I noticed that the function comment was outdated. The code now can return values from 0 to 4 while the comment explains only values

Re: [HACKERS] pg_check_dir comments and implementation mismatch

2015-01-29 Thread Robert Haas
On Thu, Jan 29, 2015 at 11:00 AM, Marco Nenciarini marco.nenciar...@2ndquadrant.it wrote: reading pgcheckdir.c code I noticed that the function comment was outdated. The code now can return values from 0 to 4 while the comment explains only values 0,1,2. This is not just a comment fix; you are

Re: [HACKERS] pg_check_dir comments and implementation mismatch

2015-01-29 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Jan 29, 2015 at 11:00 AM, Marco Nenciarini marco.nenciar...@2ndquadrant.it wrote: reading pgcheckdir.c code I noticed that the function comment was outdated. The code now can return values from 0 to 4 while the comment explains only values

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread Josh Berkus
On 01/29/2015 09:11 AM, Bruce Momjian wrote: On Thu, Jan 29, 2015 at 12:09:58PM -0500, Andrew Dunstan wrote: Then step 2 should specify that it's for the master. Right. Josh is just listing all the steps --- the pg_upgrade docs already have that spelled out in detail. What I'm also saying

Re: [HACKERS] File based Incremental backup v8

2015-01-29 Thread Robert Haas
On Thu, Jan 29, 2015 at 9:47 AM, Marco Nenciarini marco.nenciar...@2ndquadrant.it wrote: The current implementation of copydir function is incompatible with LSN based incremental backups. The problem is that new files are created, but their blocks are still with the old LSN, so they will not be

Re: [HACKERS] GetLockConflicts() and thus recovery conflicts seem pretty broken

2015-01-29 Thread Simon Riggs
On 27 January 2015 at 14:27, Andres Freund and...@2ndquadrant.com wrote: While investigating other bugs I noticed that ResolveRecoveryConflictWithLock() wasn't really working. Turns out GetLockConflicts() violates it's API contract which says: * The result array is palloc'd and is

[HACKERS] File based Incremental backup v8

2015-01-29 Thread Marco Nenciarini
The current implementation of copydir function is incompatible with LSN based incremental backups. The problem is that new files are created, but their blocks are still with the old LSN, so they will not be backed up because they are looking old enough. copydir function is used in: CREATE

[HACKERS] Memory leak in gingetbitmap

2015-01-29 Thread Heikki Linnakangas
While looking at the segfault that Olaf Gawenda reported (bug #12694), I realized that the GIN fast scan patch introduced a small memory leak to re-scanning a GIN index. In a nutshell, freeScanKeys() needs to pfree() the two new arrays, requiredEntries and additionalEntries. After fixing

Re: [HACKERS] pg_dump with both --serializable-deferrable and -j

2015-01-29 Thread Tom Lane
Kevin Grittner kgri...@ymail.com writes: I propose to apply the attached to master and back-patch to 9.3, and follow that with a patch (for master only) along the lines suggested by Andres. Since *that* change is more invasive and changes existing behavior I will submit it to the open CF

Re: [HACKERS] Hot Standby WAL reply uses heavyweight session locks, but doesn't have enough infrastructure set up

2015-01-29 Thread Robert Haas
On Wed, Jan 28, 2015 at 2:41 AM, Michael Paquier michael.paqu...@gmail.com wrote: Andres Freund wrote: I think this isn't particularly pretty, but it seems to be working well enough, and changing it would be pretty invasive. So keeping in line with all that code seems to be easier. OK, I'm

Re: [HACKERS] pg_dump with both --serializable-deferrable and -j

2015-01-29 Thread Kevin Grittner
Kevin Grittner kgri...@ymail.com wrote: Tom Lane t...@sss.pgh.pa.us wrote: A comment seems essential here, because as written anybody would think the test for a snapshot is a bug. Good point. I propose to apply the attached to master and back-patch to 9.3, and follow that with a patch

[HACKERS] pg_check_dir comments and implementation mismatch

2015-01-29 Thread Marco Nenciarini
Hi, reading pgcheckdir.c code I noticed that the function comment was outdated. The code now can return values from 0 to 4 while the comment explains only values 0,1,2. Patch attached. Regards, Marco -- Marco Nenciarini - 2ndQuadrant Italy PostgreSQL Training, Services and Support

Re: [HACKERS] TABLESAMPLE patch

2015-01-29 Thread Robert Haas
On Wed, Jan 28, 2015 at 5:19 AM, Petr Jelinek p...@2ndquadrant.com wrote: Yes, that's my view too. I would generally be for that change also and it would be worth it if the code was used in more than one place, but as it is it seems like it will just add code/complexity for no real benefit. It

Re: [HACKERS] Providing catalog view to pg_hba.conf file - Patch submission

2015-01-29 Thread Jim Nasby
On 1/29/15 6:19 AM, Fabrízio de Royes Mello wrote: On Wed, Jan 28, 2015 at 5:27 PM, Tom Lane t...@sss.pgh.pa.us mailto:t...@sss.pgh.pa.us wrote: =?UTF-8?Q?Fabr=C3=ADzio_de_Royes_Mello?= fabriziome...@gmail.com mailto:fabriziome...@gmail.com writes: But I'm thinking about this patch and

Re: [HACKERS] Proposal: two new role attributes and/or capabilities?

2015-01-29 Thread Robert Haas
On Thu, Jan 29, 2015 at 4:09 PM, Jim Nasby jim.na...@bluetreble.com wrote: The difference between the autovacuum-run vacuum and the cron-run vacuum is that the one running out of cron will just keep holding the lock until it's actually able to truncate the end of the relation, no? I recall

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-29 Thread Andrew Dunstan
On 01/29/2015 04:59 PM, Robert Haas wrote: On Thu, Jan 29, 2015 at 4:33 PM, Andrew Dunstan and...@dunslane.net wrote: On 01/29/2015 12:10 PM, Robert Haas wrote: On Wed, Jan 28, 2015 at 12:48 PM, Tom Lane t...@sss.pgh.pa.us wrote: The cause of this bug is commit

Re: [HACKERS] Small bug on CREATE EXTENSION pgq...

2015-01-29 Thread Jim Nasby
On 1/28/15 10:25 PM, Tom Lane wrote: Stephen Frost sfr...@snowman.net writes: * David Johnston (david.g.johns...@gmail.com) wrote: Fair enough but reset to what? I don't know the internal mechanics but if the session default is warning and a local change sets it to notice then an

Re: [HACKERS] [COMMITTERS] pgsql: Fix column-privilege leak in error-message paths

2015-01-29 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: Fix column-privilege leak in error-message paths This patch is at least one brick shy of a load: regression=# create table t1 (f1 int); CREATE TABLE regression=# create unique index on t1 (abs(f1)); CREATE INDEX regression=# create user joe; CREATE ROLE

[HACKERS] Re: [COMMITTERS] pgsql: Fix column-privilege leak in error-message paths

2015-01-29 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Stephen Frost sfr...@snowman.net writes: Fix column-privilege leak in error-message paths This patch is at least one brick shy of a load: regression=# create table t1 (f1 int); CREATE TABLE regression=# create unique index on t1 (abs(f1)); CREATE

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-29 Thread Robert Haas
On Thu, Jan 29, 2015 at 4:33 PM, Andrew Dunstan and...@dunslane.net wrote: On 01/29/2015 12:10 PM, Robert Haas wrote: On Wed, Jan 28, 2015 at 12:48 PM, Tom Lane t...@sss.pgh.pa.us wrote: The cause of this bug is commit 0ad1a816320a2b539a51628e2a0b1e83ff096b1d, which I'm inclined to think we

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-29 Thread Andrew Dunstan
On 01/29/2015 12:10 PM, Robert Haas wrote: On Wed, Jan 28, 2015 at 12:48 PM, Tom Lane t...@sss.pgh.pa.us wrote: The cause of this bug is commit 0ad1a816320a2b539a51628e2a0b1e83ff096b1d, which I'm inclined to think we need to simply revert, not render even more squirrely. Yes, that commit

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-29 Thread Andres Freund
On 2015-01-29 16:33:36 -0500, Andrew Dunstan wrote: On 01/29/2015 12:10 PM, Robert Haas wrote: On Wed, Jan 28, 2015 at 12:48 PM, Tom Lane t...@sss.pgh.pa.us wrote: The cause of this bug is commit 0ad1a816320a2b539a51628e2a0b1e83ff096b1d, which I'm inclined to think we need to simply revert,

Re: [HACKERS] PATCH: decreasing memory needlessly consumed by array_agg

2015-01-29 Thread Jim Nasby
On 1/28/15 4:25 PM, Tomas Vondra wrote: + * It's possible to choose whether to create a separate memory context for the + * array builde state, or whether to allocate it directly within rcontext Typo. -- Jim Nasby, Data Architect, Blue Treble Consulting Data in Trouble? Get it in Treble!

Re: [HACKERS] Proposal: two new role attributes and/or capabilities?

2015-01-29 Thread Jim Nasby
On 1/28/15 7:45 PM, Stephen Frost wrote: Jim, * Jim Nasby (jim.na...@bluetreble.com) wrote: On 12/23/14 12:52 PM, Stephen Frost wrote: Autovacuum can certainly run vacuum/analyze on a few tables every 12 hours, so I'm not really following where you see autovacuum being unable to cope. I

Re: [HACKERS] File based Incremental backup v8

2015-01-29 Thread Andres Freund
On 2015-01-29 12:57:22 -0500, Robert Haas wrote: The issues here are similar to those in http://www.postgresql.org/message-id/20150120152819.gc24...@alap3.anarazel.de - basically, I think we need to make CREATE DATABASE and ALTER DATABASE .. SET TABLESPACE fully WAL-logged operations, or this

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-29 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Jan 29, 2015 at 4:33 PM, Andrew Dunstan and...@dunslane.net wrote: I'm coming down more and more on the side of Tom's suggestion just to ban \u in jsonb. I have yet to understand what we fix by banning \u. How is different from

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

2015-01-29 Thread Andres Freund
Pushed a version (hopefully) fixing Tom's complaints. On 2015-01-28 13:52:30 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2015-01-28 13:38:51 -0500, Tom Lane wrote: #define BUFFERDESC_PADDED_SIZE (SIZEOF_VOID_P == 8 ? 64 : 32) Hm, did you intentionally put a

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread David Steele
On 1/29/15 12:42 PM, Josh Berkus wrote: On 01/29/2015 09:11 AM, Bruce Momjian wrote: On Thu, Jan 29, 2015 at 12:09:58PM -0500, Andrew Dunstan wrote: Then step 2 should specify that it's for the master. Right. Josh is just listing all the steps --- the pg_upgrade docs already have that

Re: [HACKERS] Exposing the stats snapshot timestamp to SQL

2015-01-29 Thread Matt Kelly
Robert, I'll add it to the commitfest. Jim, I'm not sure I understand what you mean? This new function follows the same conventions as everything else in the file. TimestampTz is just a typedef for int64. Functions like pg_stat_get_buf_alloc follow the exact same pattern on the int64 fields of

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread Jim Nasby
On 1/29/15 7:02 PM, David Steele wrote: On 1/29/15 7:55 PM, Jim Nasby wrote: On 1/29/15 6:25 PM, David Steele wrote: Safe backups can be done without LSNs provided you are willing to trust your timestamps. Which AFAICT simply isn't safe to do at all... except maybe with the manifest stuff

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread Jim Nasby
On 1/29/15 5:53 PM, David Steele wrote: On 1/29/15 12:42 PM, Josh Berkus wrote: On 01/29/2015 09:11 AM, Bruce Momjian wrote: On Thu, Jan 29, 2015 at 12:09:58PM -0500, Andrew Dunstan wrote: Then step 2 should specify that it's for the master. Right. Josh is just listing all the steps --- the

Re: [HACKERS] Exposing the stats snapshot timestamp to SQL

2015-01-29 Thread Matt Kelly
This is now: https://commitfest.postgresql.org/4/128/ On Thu, Jan 29, 2015 at 7:01 PM, Matt Kelly mkell...@gmail.com wrote: Robert, I'll add it to the commitfest. Jim, I'm not sure I understand what you mean? This new function follows the same conventions as everything else in the file.

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread Jim Nasby
On 1/29/15 6:25 PM, David Steele wrote: Safe backups can be done without LSNs provided you are willing to trust your timestamps. Which AFAICT simply isn't safe to do at all... except maybe with the manifest stuff you've talked about? FWIW, I personally am very leery of relying on

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-29 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I have yet to understand what we fix by banning \u. How is different from any other four-digit hexadecimal number that's not a valid character in the current encoding? What does banning that one particular value do? BTW, as to the point

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread David Steele
On 1/29/15 11:34 AM, Bruce Momjian wrote: 3. Check that the replica is not very lagged. If it is, wait for traffic to die down and for it to catch up. I think I'd want a something a bit more specific here. When the primary shuts down it will kick out one last WAL. The filename should be

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread David Steele
On 1/29/15 7:55 PM, Jim Nasby wrote: On 1/29/15 6:25 PM, David Steele wrote: Safe backups can be done without LSNs provided you are willing to trust your timestamps. Which AFAICT simply isn't safe to do at all... except maybe with the manifest stuff you've talked about? Yes - that's what

Re: [HACKERS] Proposal: two new role attributes and/or capabilities?

2015-01-29 Thread Jim Nasby
On 1/29/15 4:02 PM, Robert Haas wrote: On Thu, Jan 29, 2015 at 4:09 PM, Jim Nasby jim.na...@bluetreble.com wrote: The difference between the autovacuum-run vacuum and the cron-run vacuum is that the one running out of cron will just keep holding the lock until it's actually able to truncate the

Re: [HACKERS] Proposal: two new role attributes and/or capabilities?

2015-01-29 Thread Tom Lane
Jim Nasby jim.na...@bluetreble.com writes: On 1/29/15 4:02 PM, Robert Haas wrote: I don't think this is true, and I don't think it's been true for a long time, if ever. The difference between a manual vacuum and autovacuum is that autovacuum commits suicide when it conflicts with somebody

Re: [HACKERS] Exposing the stats snapshot timestamp to SQL

2015-01-29 Thread Tom Lane
Matt Kelly mkell...@gmail.com writes: Jim, I'm not sure I understand what you mean? This new function follows the same conventions as everything else in the file. TimestampTz is just a typedef for int64. ... or double. Have you checked that the code behaves properly with

Re: [HACKERS] Possible typo in create_policy.sgml

2015-01-29 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Wed, Jan 28, 2015 at 10:45 PM, Stephen Frost sfr...@snowman.net wrote: I agree, especially after going back and re-reading this while fixing the issue mentioned earlier by Peter (which was an orthogonal complaint about the shadowing of WITH

Re: [HACKERS] Error check always bypassed in tablefunc.c

2015-01-29 Thread Michael Paquier
On Fri, Jan 30, 2015 at 10:32 AM, Tom Lane t...@sss.pgh.pa.us wrote: [blah] What I did about this was to leave the behavior alone in back branches, but insist on a type match in HEAD. I think we can reasonably tighten the type requirements in a new major release, but doing it in minor

Re: [HACKERS] Exposing the stats snapshot timestamp to SQL

2015-01-29 Thread Jim Nasby
On 1/28/15 11:18 PM, Matt Kelly wrote: In a previous thread Tom Lane said: (I'm also wondering if it'd make sense to expose the stats timestamp as a callable function, so that the case could be dealt with programmatically as well. But that's future-feature territory.)

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread David Steele
On 1/29/15 10:13 AM, Bruce Momjian wrote: Agreed. I have update the two mentions of rsync in our docs to clarify this. Thank you. The patch also has pg_upgrade doc improvements suggested by comments from Josh Berkus. It's very good to see this. Mentions of this rsync vulnerability are few

Re: [HACKERS] Parallel Seq Scan

2015-01-29 Thread Jim Nasby
On 1/28/15 7:27 PM, Stephen Frost wrote: * Jim Nasby (jim.na...@bluetreble.com) wrote: On 1/28/15 9:56 AM, Stephen Frost wrote: Such i/o systems do exist, but a single RAID5 group over spinning rust with a simple filter isn't going to cut it with a modern CPU- we're just too darn efficient

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread David Steele
On 1/29/15 7:07 PM, Jim Nasby wrote: Ultimately, there is no single best method. It depends a lot on your environment. I would prefer the official documents to contain very safe methods. How do we define safe though? Your method leaves you without a backup server until your base backup

Re: [HACKERS] Possible typo in create_policy.sgml

2015-01-29 Thread Robert Haas
On Thu, Jan 29, 2015 at 9:04 PM, Stephen Frost sfr...@snowman.net wrote: A policy grants the ability to SELECT, INSERT, UPDATE, or DELETE rows which match the relevant policy expression. Existing table rows are checked against the expression specified via USING, while new rows that would be

Re: [HACKERS] pg_check_dir comments and implementation mismatch

2015-01-29 Thread Michael Paquier
On Fri, Jan 30, 2015 at 2:49 AM, Tom Lane t...@sss.pgh.pa.us wrote: There is at least one other bug in that function now that I look at it: in event of a readdir() failure, it neglects to execute closedir(). Perhaps not too significant since all existing callers will just exit() anyway after a

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-29 Thread Tom Lane
Attached is a draft patch for this. It basically reverts commit 0ad1a816320a2b539a51628e2a0b1e83ff096b1d, adds a ban of \u if that would need to be converted to text (so it still works in the plain json type, so long as you don't do much processing), and adds some regression tests. I made

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2015-01-29 Thread Alvaro Herrera
Pavel Stehule wrote: should not be used a pessimist controlled locking instead? Patches welcome. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via pgsql-hackers mailing list

Re: [HACKERS] PATCH: Reducing lock strength of trigger and foreign key DDL

2015-01-29 Thread Michael Paquier
On Fri, Jan 23, 2015 at 8:55 AM, Andreas Karlsson andr...@proxel.se wrote: On 01/22/2015 10:31 PM, Andreas Karlsson wrote: I agree with this view, and am not sure myself that it is worth lowering the lock level of ALTER TRIGGER RENAME. I have attached a patch without the changes to ALTER

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-29 Thread Peter Geoghegan
On Thu, Jan 29, 2015 at 10:20 PM, Tom Lane t...@sss.pgh.pa.us wrote: I made the \u error be errcode(ERRCODE_INVALID_TEXT_REPRESENTATION) and errmsg(invalid input syntax for type json), by analogy to what's thrown for non-ASCII Unicode escapes in non-UTF8 encoding. I'm not terribly happy

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-29 Thread Tom Lane
Peter Geoghegan p...@heroku.com writes: I looked into it, and it turns out that MongoDB does not accept NUL in at least some contexts (for object keys). Apparently it wasn't always so. MongoDB previously had a security issue that was fixed by introducing this restriction. Their JSON-centric

Re: [HACKERS] Safe memory allocation functions

2015-01-29 Thread Michael Paquier
I wrote: Yes, this refactoring was good for testing actually... Oops, I have been too hasty when sending previous patch, there was a bug related to huge allocations. Patch correcting this bug is attached. Attached are as well two things I have used to test the new API: - A hack refactoring the

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-29 Thread Andrew Dunstan
On 01/29/2015 05:39 PM, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: I have yet to understand what we fix by banning \u. How is different from any other four-digit hexadecimal number that's not a valid character in the current encoding? What does banning that one

Re: [HACKERS] TABLESAMPLE patch

2015-01-29 Thread Jim Nasby
On 1/29/15 10:44 AM, Bruce Momjian wrote: On Thu, Jan 29, 2015 at 11:08:55AM -0500, Robert Haas wrote: On Wed, Jan 28, 2015 at 5:19 AM, Petr Jelinek p...@2ndquadrant.com wrote: Yes, that's my view too. I would generally be for that change also and it would be worth it if the code was used in

Re: [HACKERS] pg_upgrade and rsync

2015-01-29 Thread David Steele
On 1/29/15 8:09 PM, Jim Nasby wrote: On 1/29/15 7:02 PM, David Steele wrote: On 1/29/15 7:55 PM, Jim Nasby wrote: On 1/29/15 6:25 PM, David Steele wrote: Safe backups can be done without LSNs provided you are willing to trust your timestamps. Which AFAICT simply isn't safe to do at all...

Re: [HACKERS] Error check always bypassed in tablefunc.c

2015-01-29 Thread Tom Lane
Michael Paquier michael.paqu...@gmail.com writes: So, I have been poking at this code a bit more and as the values of the parameters are passed as-is to the SQL queries that connectby generates internally (this is as well mentioned in the documentation here:

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-29 Thread Tom Lane
Peter Geoghegan p...@heroku.com writes: On Thu, Jan 29, 2015 at 10:20 PM, Tom Lane t...@sss.pgh.pa.us wrote: I made the \u error be errcode(ERRCODE_INVALID_TEXT_REPRESENTATION) and errmsg(invalid input syntax for type json), by analogy to what's thrown for non-ASCII Unicode escapes in