Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Robert Haas
On Sat, Jan 24, 2015 at 10:04 PM, Bruce Momjian br...@momjian.us wrote: On Fri, Jan 23, 2015 at 02:34:36PM -0500, Stephen Frost wrote: You'd have to replace the existing data directory on the master to do that, which pg_upgrade was designed specifically to not do, in case things went

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-27 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 01/27/2015 01:40 PM, Tom Lane wrote: In particular, I would like to suggest that the current representation of \u is fundamentally broken and that we have to change it, not try to band-aid around it. This will mean an on-disk incompatibility

Re: [HACKERS] Release notes

2015-01-27 Thread Andres Freund
On 2015-01-27 10:09:12 -0800, Joshua D. Drake wrote: Where can one find the running copy of release notes for pending releases? In the source tree on the master once it exists. It doesn't yet for the next set of releases. Greetings, Andres Freund -- Andres Freund

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-27 Thread Andrew Dunstan
On 01/27/2015 02:28 PM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: On 01/27/2015 01:40 PM, Tom Lane wrote: In particular, I would like to suggest that the current representation of \u is fundamentally broken and that we have to change it, not try to band-aid around it.

[HACKERS] Release notes

2015-01-27 Thread Joshua D. Drake
Where can one find the running copy of release notes for pending releases? -- Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564 PostgreSQL Support, Training, Professional Services and Development High Availability, Oracle Conversion, @cmdpromptinc If we send our children to

Re: [HACKERS] Release notes

2015-01-27 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2015-01-27 10:09:12 -0800, Joshua D. Drake wrote: Where can one find the running copy of release notes for pending releases? In the source tree on the master once it exists. It doesn't yet for the next set of releases. The closest thing to a

Re: [HACKERS] Release notes

2015-01-27 Thread Joshua D. Drake
On 01/27/2015 10:17 AM, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2015-01-27 10:09:12 -0800, Joshua D. Drake wrote: Where can one find the running copy of release notes for pending releases? In the source tree on the master once it exists. It doesn't yet for the next

Re: [HACKERS] Re: Abbreviated keys for Numeric

2015-01-27 Thread Gavin Flower
On 28/01/15 06:29, Andrew Gierth wrote: Peter == Peter Geoghegan p...@heroku.com writes: Peter What I find particularly interesting about this patch is that it Peter makes sorting numerics significantly faster than even sorting Peter float8 values, Played some more with this. Testing on

Re: [HACKERS] proposal: plpgsql - Assert statement

2015-01-27 Thread Pavel Stehule
2015-01-26 22:34 GMT+01:00 Jim Nasby jim.na...@bluetreble.com: On 1/22/15 2:01 PM, Pavel Stehule wrote: * I would to simplify a behave of evaluating of message expression - probably I disallow NULL there. Well, the only thing I could see you doing there is throwing a

Re: [HACKERS] proposal: row_to_array function

2015-01-27 Thread Pavel Stehule
Hi 2015-01-27 11:41 GMT+01:00 Pavel Stehule pavel.steh...@gmail.com: 2015-01-26 21:44 GMT+01:00 Jim Nasby jim.na...@bluetreble.com: On 1/25/15 4:23 AM, Pavel Stehule wrote: I tested a concept iteration over array in format [key1, value1, key2, value2, .. ] - what is nice, it works for

Re: [HACKERS] basebackups during ALTER DATABASE ... SET TABLESPACE ... not safe?

2015-01-27 Thread Andres Freund
On 2015-01-27 07:16:27 -0500, Robert Haas wrote: On Mon, Jan 26, 2015 at 4:03 PM, Andres Freund and...@2ndquadrant.com wrote: I basically have two ideas to fix this. 1) Make do_pg_start_backup() acquire a SHARE lock on pg_database. That'll prevent it from starting while a movedb() is

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2015-01-27 Thread Andres Freund
On 2015-01-27 08:21:57 +0100, Andreas Karlsson wrote: On 01/23/2015 02:58 AM, Petr Jelinek wrote: On 23/01/15 00:40, Andreas Karlsson wrote: - Renamed some things from int12 to int128, there are still some places with int16 which I am not sure what to do with. I'd vote for renaming them to

Re: [HACKERS] [RFC] Incremental backup v3: incremental PoC

2015-01-27 Thread Giuseppe Broccolo
Hi Marco, On 16/01/15 16:55, Marco Nenciarini wrote: On 14/01/15 17:22, Gabriele Bartolini wrote: My opinion, Marco, is that for version 5 of this patch, you: 1) update the information on the wiki (it is outdated - I know you have been busy with LSN map optimisation) Done. 2)

Re: [HACKERS] [POC] FETCH limited by bytes.

2015-01-27 Thread Kyotaro HORIGUCHI
Thank you for the comment. The automatic way to determin the fetch_size looks become too much for the purpose. An example of non-automatic way is a new foreign table option like 'fetch_size' but this exposes the inside too much... Which do you think is preferable? Thu, 22 Jan 2015 11:17:52

Re: [HACKERS] Hot Standby WAL reply uses heavyweight session locks, but doesn't have enough infrastructure set up

2015-01-27 Thread Andres Freund
On 2015-01-27 16:23:53 +0900, Michael Paquier wrote: On Tue, Jan 27, 2015 at 6:24 AM, Andres Freund and...@2ndquadrant.com wrote: Unfortunately that Assert()s when there's a lock conflict because e.g. another backend is currently connecting. That's because ProcSleep() does a

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2015-01-27 Thread Andreas Karlsson
On 01/27/2015 09:05 AM, Andres Freund wrote: On 2015-01-27 08:21:57 +0100, Andreas Karlsson wrote: On 01/23/2015 02:58 AM, Petr Jelinek wrote: On 23/01/15 00:40, Andreas Karlsson wrote: - Renamed some things from int12 to int128, there are still some places with int16 which I am not sure what

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2015-01-27 Thread Abhijit Menon-Sen
At 2015-01-26 17:45:52 -0500, robertmh...@gmail.com wrote: Based on the recent emails, it appears there's been a shift of preference to having it be in-core […] Well, I'm not sure that anyone else here agreed with me on that Sure, an in-core AUDIT command would be great. Stephen has

Re: [HACKERS] Safe memory allocation functions

2015-01-27 Thread Michael Paquier
Alvaro Herrera wrote: So how about something like #define ALLOCFLAG_HUGE 0x01 #define ALLOCFLAG_NO_ERROR_ON_OOM 0x02 void * MemoryContextAllocFlags(MemoryContext context, Size size, int flags); The flag for huge allocations may be useful, but I don't actually see much

Re: [HACKERS] Safe memory allocation functions

2015-01-27 Thread Andres Freund
On 2015-01-27 17:27:53 +0900, Michael Paquier wrote: Alvaro Herrera wrote: So how about something like #define ALLOCFLAG_HUGE 0x01 #define ALLOCFLAG_NO_ERROR_ON_OOM 0x02 void * MemoryContextAllocFlags(MemoryContext context, Size size, int flags); The flag for

Re: [HACKERS] alter user/role CURRENT_USER

2015-01-27 Thread Kyotaro HORIGUCHI
Hello, thank you for the comment. Looking at this a bit, I'm not sure completely replacing RoleId with a node is the best idea; some of the users of that production in the grammar are okay with accepting only normal strings as names, and don't need all the CURRENT_USER etc stuff. You're

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-01-27 Thread Abhijit Menon-Sen
At 2015-01-26 18:47:29 -0600, jim.na...@bluetreble.com wrote: Anything happen with this? Nothing so far. -- Abhijit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: Partitioning: issues/ideas (Was: Re: [HACKERS] On partitioning)

2015-01-27 Thread Amit Langote
On 27-01-2015 AM 05:46, Jim Nasby wrote: On 1/25/15 7:42 PM, Amit Langote wrote: On 21-01-2015 PM 07:26, Amit Langote wrote: Ok, I will limit myself to focusing on following things at the moment: * Provide syntax in CREATE TABLE to declare partition key While working on this, I stumbled

Re: [HACKERS] Safe memory allocation functions

2015-01-27 Thread Michael Paquier
On Sat, Jan 17, 2015 at 11:06 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Jan 16, 2015 at 10:56 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: So how about something like #define ALLOCFLAG_HUGE 0x01 #define ALLOCFLAG_NO_ERROR_ON_OOM 0x02 void *

Re: [HACKERS] Parallel Seq Scan

2015-01-27 Thread Stephen Frost
Robert, all, * Robert Haas (robertmh...@gmail.com) wrote: There is a considerable amount of variation in the amount of time this takes to run based on how much of the relation is cached. Clearly, there's no way for the system to cache it all, but it can cache a significant portion, and that

Re: [HACKERS] Parallel Seq Scan

2015-01-27 Thread Robert Haas
On Thu, Jan 22, 2015 at 5:57 AM, Amit Kapila amit.kapil...@gmail.com wrote: Script used to test is attached (parallel_count.sh) Why does this use EXPLAIN ANALYZE instead of \timing ? IBM POWER-7 16 cores, 64 hardware threads RAM = 64GB Table Size - 120GB Used below statements to create

Re: [HACKERS] proposal: row_to_array function

2015-01-27 Thread Pavel Stehule
Example: postgres=# do $$ declare r record; declare k text; v text; begin for r in select * from foo loop foreach k,v in array row_to_array(r) loop raise notice 'k: %, v: %', k, v; end loop; end loop; end; $$; NOTICE: k: a, v: 2 NOTICE: k: b, v: NAZDAR NOTICE: k: c, v:

Re: [HACKERS] proposal: row_to_array function

2015-01-27 Thread Pavel Stehule
Hello here is a initial version of row_to_array function - transform any row to array in format proposed by Andrew. Regards Pavel 2015-01-27 19:58 GMT+01:00 Pavel Stehule pavel.steh...@gmail.com: Hi 2015-01-27 11:41 GMT+01:00 Pavel Stehule pavel.steh...@gmail.com: 2015-01-26 21:44

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-27 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 01/27/2015 02:28 PM, Tom Lane wrote: Well, we can either fix it now or suffer with a broken representation forever. I'm not wedded to the exact solution I described, but I think we'll regret it if we don't change the representation. The only

Re: [HACKERS] proposal: searching in array function - array_position

2015-01-27 Thread Jim Nasby
On 1/27/15 4:36 AM, Pavel Stehule wrote: 2015-01-26 23:29 GMT+01:00 Jim Nasby jim.na...@bluetreble.com mailto:jim.na...@bluetreble.com: On 1/26/15 4:17 PM, Pavel Stehule wrote: Any way to reduce the code duplication between the array and non-array versions? Maybe factor out

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2015-01-27 Thread Robert Haas
On Tue, Jan 27, 2015 at 5:55 PM, Jim Nasby jim.na...@bluetreble.com wrote: BTW, I know that at least earlier versions of EnterpriseDB's version of Postgres (circa 2007) had an auditing feature. I never dealt with any customers who were using it when I was there, but perhaps some other folks

Re: [HACKERS] proposal: plpgsql - Assert statement

2015-01-27 Thread Jim Nasby
On 1/27/15 1:30 PM, Pavel Stehule wrote: I don't see the separate warning as being helpful. I'd just do something like +(err_hint != NULL) ? errhint(%s, err_hint) : errhint(Message attached to failed assertion is null) )); done There should also

Re: [HACKERS] Shortcoming in CLOBBER_FREED_MEMORY coverage: disk buffer pointers

2015-01-27 Thread Tom Lane
Jim Nasby jim.na...@bluetreble.com writes: On 1/26/15 6:11 PM, Greg Stark wrote: Fwiw I think our experience is that bugs where buffers are unpinned get exposed pretty quickly in production. I suppose the same might not be true for rarely called codepaths or in cases where the buffers are

Re: [HACKERS] proposal: row_to_array function

2015-01-27 Thread Jim Nasby
On 1/27/15 2:26 PM, Pavel Stehule wrote: here is a initial version of row_to_array function - transform any row to array in format proposed by Andrew. Please start a new thread for this... does it depend on the key-value patch? -- Jim Nasby, Data Architect, Blue Treble Consulting Data in

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-27 Thread Tom Lane
Merlin Moncure mmonc...@gmail.com writes: On Tue, Jan 27, 2015 at 12:40 PM, Tom Lane t...@sss.pgh.pa.us wrote: In particular, I would like to suggest that the current representation of \u is fundamentally broken and that we have to change it, not try to band-aid around it. This will mean

Re: [HACKERS] proposal: row_to_array function

2015-01-27 Thread Jim Nasby
On 1/27/15 12:58 PM, Pavel Stehule wrote: postgres=# select array(select unnest('{{{1,2},{3,4}},{{5,6},{7,8}}}'::int[])); array --- {1,2,3,4,5,6,7,8} (1 row) so it generate pairs {1,2}{3,4},{5,6},{7,8} I fixed situation when array has not

Re: [HACKERS] Dereferenced pointers checked as NULL in btree_utils_var.c

2015-01-27 Thread Michael Paquier
On Wed, Jan 28, 2015 at 3:15 AM, Tom Lane t...@sss.pgh.pa.us wrote: So I'm fine with taking out both this documentation text and the dead null-pointer checks; but let's do that all in one patch not piecemeal. In any case, this is just cosmetic cleanup; there's no actual hazard here. Attached

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread David Steele
On 1/27/15 6:09 PM, Jim Nasby wrote: The whole remote_dir discussion made me think of something... would --link-dest be any help here? I'm pretty sure --link-dest would not be effective in this case. The problem exists on the source side and --link-dest only operates on the destination. -- -

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2015-01-27 Thread Robert Haas
On Tue, Jan 27, 2015 at 6:54 PM, Jim Nasby jim.na...@bluetreble.com wrote: On 1/27/15 5:06 PM, Robert Haas wrote: On Tue, Jan 27, 2015 at 5:55 PM, Jim Nasby jim.na...@bluetreble.com wrote: BTW, I know that at least earlier versions of EnterpriseDB's version of Postgres (circa 2007) had an

Re: [HACKERS] basebackups during ALTER DATABASE ... SET TABLESPACE ... not safe?

2015-01-27 Thread Robert Haas
On Tue, Jan 27, 2015 at 2:16 PM, Andres Freund and...@2ndquadrant.com wrote: That'd end up essentially being a re-emulation of locks. Don't find that all that pretty. But maybe we have to go there. The advantage of it is that it is simple. The only thing we're really giving up is the deadlock

Re: [HACKERS] [POC] FETCH limited by bytes.

2015-01-27 Thread Corey Huinker
Last year I was working on a patch to postgres_fdw where the fetch_size could be set at the table level and the server level. I was able to get the settings parsed and they would show up in pg_foreign_table and pg_foreign_servers. Unfortunately, I'm not very familiar with how foreign data

Re: [HACKERS] Additional role attributes superuser review

2015-01-27 Thread Adam Brightwell
All, I have attached and updated patch for review. Thanks, Adam -- Adam Brightwell - adam.brightw...@crunchydatasolutions.com Database Engineer - www.crunchydatasolutions.com diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml new file mode 100644 index 62305d2..fd4b9ab ***

Re: [HACKERS] Parallel Seq Scan

2015-01-27 Thread Robert Haas
On Tue, Jan 27, 2015 at 4:46 PM, Stephen Frost sfr...@snowman.net wrote: With 0 workers, first run took 883465.352 ms, and second run took 295050.106 ms. With 8 workers, first run took 340302.250 ms, and second run took 307767.758 ms. This is a confusing result, because you expect

Re: [HACKERS] basebackups during ALTER DATABASE ... SET TABLESPACE ... not safe?

2015-01-27 Thread Andres Freund
On 2015-01-27 19:24:24 -0500, Robert Haas wrote: On Tue, Jan 27, 2015 at 2:16 PM, Andres Freund and...@2ndquadrant.com wrote: That'd end up essentially being a re-emulation of locks. Don't find that all that pretty. But maybe we have to go there. The advantage of it is that it is simple.

Re: [HACKERS] Parallel Seq Scan

2015-01-27 Thread Robert Haas
On Tue, Jan 27, 2015 at 6:00 PM, Robert Haas robertmh...@gmail.com wrote: Now, when you did what I understand to be the same test on the same machine, you got times ranging from 9.1 seconds to 35.4 seconds. Clearly, there is some difference between our test setups. Moreover, I'm kind of

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Fri, Jan 23, 2015 at 1:48 PM, Andres Freund and...@2ndquadrant.com wrote: I don't understand why that'd be better than simply fixing (yes, that's imo the correct term) pg_upgrade to retain relfilenodes across the upgrade. Afaics there's no conflict

[HACKERS] GetLockConflicts() and thus recovery conflicts seem pretty broken

2015-01-27 Thread Andres Freund
Hi, While investigating other bugs I noticed that ResolveRecoveryConflictWithLock() wasn't really working. Turns out GetLockConflicts() violates it's API contract which says: * The result array is palloc'd and is terminated with an invalid VXID. Problem 1: We don't actually put the terminator

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Robert Haas
On Tue, Jan 27, 2015 at 9:36 AM, Stephen Frost sfr...@snowman.net wrote: The example listed works, but only when it's a local rsync: rsync --archive --hard-links --size-only old_dir new_dir remote_dir Perhaps a better example (or additional one) would be with a remote rsync, including

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Robert Haas
On Tue, Jan 27, 2015 at 9:50 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Fri, Jan 23, 2015 at 1:48 PM, Andres Freund and...@2ndquadrant.com wrote: I don't understand why that'd be better than simply fixing (yes, that's imo the correct term) pg_upgrade

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Sat, Jan 24, 2015 at 10:04 PM, Bruce Momjian br...@momjian.us wrote: On Fri, Jan 23, 2015 at 02:34:36PM -0500, Stephen Frost wrote: You'd have to replace the existing data directory on the master to do that, which pg_upgrade was designed

Re: [HACKERS] Parallel Seq Scan

2015-01-27 Thread David Fetter
On Tue, Jan 27, 2015 at 08:02:37AM +0100, Daniel Bausch wrote: Hi PG devs! Tom Lane t...@sss.pgh.pa.us writes: Wait for first IO, issue second IO request Compute Already have second IO request, issue third ... We'd be a lot less sensitive to IO latency. It would take about

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2015-01-27 Thread Jim Nasby
On 1/26/15 4:45 PM, Robert Haas wrote: On Mon, Jan 26, 2015 at 5:21 PM, Stephen Frost sfr...@snowman.net wrote: I don't disagree with you about any of that. I don't think you disagree with my comment either. What I'm not entirely clear on is how consensus could be reached. Leaving it dormant

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2015-01-27 Thread Jim Nasby
On 1/27/15 5:06 PM, Robert Haas wrote: On Tue, Jan 27, 2015 at 5:55 PM, Jim Nasby jim.na...@bluetreble.com wrote: BTW, I know that at least earlier versions of EnterpriseDB's version of Postgres (circa 2007) had an auditing feature. I never dealt with any customers who were using it when I was

Re: [HACKERS] Shortcoming in CLOBBER_FREED_MEMORY coverage: disk buffer pointers

2015-01-27 Thread Jim Nasby
On 1/27/15 5:16 PM, Tom Lane wrote: Jim Nasby jim.na...@bluetreble.com writes: On 1/26/15 6:11 PM, Greg Stark wrote: Fwiw I think our experience is that bugs where buffers are unpinned get exposed pretty quickly in production. I suppose the same might not be true for rarely called codepaths

Re: [HACKERS] Providing catalog view to pg_hba.conf file - Patch submission

2015-01-27 Thread Jim Nasby
On 1/27/15 1:04 AM, Haribabu Kommi wrote: On Mon, Jun 30, 2014 at 5:06 PM, Abhijit Menon-Sen a...@2ndquadrant.com wrote: I think having two columns would work. The columns could be called database and database_list and user and user_list respectively. The database column may contain one of

Re: [HACKERS] Parallel Seq Scan

2015-01-27 Thread Jim Nasby
On 1/26/15 11:11 PM, Amit Kapila wrote: On Tue, Jan 27, 2015 at 3:18 AM, Jim Nasby jim.na...@bluetreble.com mailto:jim.na...@bluetreble.com wrote: On 1/23/15 10:16 PM, Amit Kapila wrote: Further, if we want to just get the benefit of parallel I/O, then I think we can get that by

Re: [HACKERS] Parallel Seq Scan

2015-01-27 Thread Jim Nasby
On 1/27/15 3:46 PM, Stephen Frost wrote: With 0 workers, first run took 883465.352 ms, and second run took 295050.106 ms. With 8 workers, first run took 340302.250 ms, and second run took 307767.758 ms. This is a confusing result, because you expect parallelism to help more when the relation

Re: [HACKERS] Dereferenced pointers checked as NULL in btree_utils_var.c

2015-01-27 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: I think you are confusing NULL pointers with an SQL NULL. gistgetadjusted checks that it's not an SQL NULL (!oldisnull[i]), but it does not check if it's a NULL pointer (DatumGetPointer(oldentries[i].key) != NULL). The case we're worried

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-27 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 01/27/2015 12:23 PM, Tom Lane wrote: I think coding anything is premature until we decide how we're going to deal with the fundamental ambiguity. The input \\uabcd will be stored correctly as \uabcd, but this will in turn be rendered as \uabcd,

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-27 Thread Andrew Dunstan
On 01/27/2015 01:40 PM, Tom Lane wrote: In particular, I would like to suggest that the current representation of \u is fundamentally broken and that we have to change it, not try to band-aid around it. This will mean an on-disk incompatibility for jsonb data containing U+, but

Re: [HACKERS] Parallel Seq Scan

2015-01-27 Thread Robert Haas
On Fri, Jan 23, 2015 at 6:42 AM, Amit Kapila amit.kapil...@gmail.com wrote: Fixed-Chunks No. of workers/Time (ms) 0 2 4 8 16 24 32 Run-1 250536 266279 251263 234347 87930 50474 35474 Run-2 249587 230628 225648 193340 83036 35140 9100 Run-3 234963 220671 230002 256183 105382 62493 27903

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-01-27 Thread Jim Nasby
On 1/27/15 3:56 AM, Abhijit Menon-Sen wrote: At 2015-01-26 18:47:29 -0600, jim.na...@bluetreble.com wrote: Anything happen with this? Nothing so far. It would be best to get this into a commit fest so it's not lost. -- Jim Nasby, Data Architect, Blue Treble Consulting Data in Trouble? Get

Re: [HACKERS] Shortcoming in CLOBBER_FREED_MEMORY coverage: disk buffer pointers

2015-01-27 Thread Jim Nasby
On 1/26/15 6:11 PM, Greg Stark wrote: On Tue, Jan 27, 2015 at 12:03 AM, Jim Nasby jim.na...@bluetreble.com mailto:jim.na...@bluetreble.com wrote: But one backend can effectively pin a buffer more than once, no? If so, then ISTM there's some risk that code path A pins and forgets to

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Jim Nasby
On 1/27/15 9:29 AM, Stephen Frost wrote: My point is that Bruce's patch suggests looking for remote_dir in the rsync documentation, but no such term appears there. Ah, well, perhaps we could simply add a bit of clarification to this: for details on specifying optionremote_dir/ The whole

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-27 Thread Merlin Moncure
On Tue, Jan 27, 2015 at 12:40 PM, Tom Lane t...@sss.pgh.pa.us wrote: Andrew Dunstan and...@dunslane.net writes: On 01/27/2015 12:23 PM, Tom Lane wrote: I think coding anything is premature until we decide how we're going to deal with the fundamental ambiguity. The input \\uabcd will be

Re: [HACKERS] proposal: row_to_array function

2015-01-27 Thread Pavel Stehule
2015-01-26 21:44 GMT+01:00 Jim Nasby jim.na...@bluetreble.com: On 1/25/15 4:23 AM, Pavel Stehule wrote: I tested a concept iteration over array in format [key1, value1, key2, value2, .. ] - what is nice, it works for [[key1,value1],[key2, value2], ...] too It is only a few lines more to

Re: [HACKERS] proposal: searching in array function - array_position

2015-01-27 Thread Pavel Stehule
2015-01-26 23:29 GMT+01:00 Jim Nasby jim.na...@bluetreble.com: On 1/26/15 4:17 PM, Pavel Stehule wrote: Any way to reduce the code duplication between the array and non-array versions? Maybe factor out the operator caching code? I though about it - but there is different checks,

Re: [HACKERS] basebackups during ALTER DATABASE ... SET TABLESPACE ... not safe?

2015-01-27 Thread Robert Haas
On Mon, Jan 26, 2015 at 4:03 PM, Andres Freund and...@2ndquadrant.com wrote: I basically have two ideas to fix this. 1) Make do_pg_start_backup() acquire a SHARE lock on pg_database. That'll prevent it from starting while a movedb() is still in progress. Then additionally add

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Bruce Momjian
On Mon, Jan 26, 2015 at 05:41:59PM -0500, Stephen Frost wrote: I've thought about it a fair bit actually and I agree that there is some risk to using rsync for *incremental* base backups. That is, you have a setup where you loop with: pg_start_backup rsync - dest pg_stop_backup without

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread David Steele
On 1/27/15 9:51 PM, Bruce Momjian wrote: According to my empirical testing on Linux and OSX the answer is no: rsync does not use sub-second accuracy. This seems to be true even on file systems like ext4 that support millisecond mod times, at least it was true on Ubuntu 12.04 running ext4.

Re: [HACKERS] proposal: row_to_array function

2015-01-27 Thread Pavel Stehule
2015-01-28 6:49 GMT+01:00 Pavel Stehule pavel.steh...@gmail.com: Dne 28.1.2015 0:25 Jim Nasby jim.na...@bluetreble.com napsal(a): On 1/27/15 12:58 PM, Pavel Stehule wrote: postgres=# select array(select unnest('{{{1,2},{3,4}},{{5,6},{7,8}}}'::int[])); array

Re: [HACKERS] Parallel Seq Scan

2015-01-27 Thread Amit Kapila
On Wed, Jan 28, 2015 at 12:38 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 01/28/2015 04:16 AM, Robert Haas wrote: On Tue, Jan 27, 2015 at 6:00 PM, Robert Haas robertmh...@gmail.com wrote: Now, when you did what I understand to be the same test on the same machine, you got

Re: [HACKERS] proposal: searching in array function - array_position

2015-01-27 Thread Pavel Stehule
2015-01-28 0:01 GMT+01:00 Jim Nasby jim.na...@bluetreble.com: On 1/27/15 4:36 AM, Pavel Stehule wrote: 2015-01-26 23:29 GMT+01:00 Jim Nasby jim.na...@bluetreble.com mailto: jim.na...@bluetreble.com: On 1/26/15 4:17 PM, Pavel Stehule wrote: Any way to reduce the code

Re: [HACKERS] TABLESAMPLE patch

2015-01-27 Thread Kyotaro HORIGUCHI
Hi, I took a look on this and found nice. By the way, the parameter for REPEATABLE seems allowing to be a expression in ParseTableSample but the grammer rejects it. The following change seems enough. diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 4578b5e..8cf09d5

Re: [HACKERS] Providing catalog view to pg_hba.conf file - Patch submission

2015-01-27 Thread Haribabu Kommi
On Wed, Jan 28, 2015 at 9:47 AM, Jim Nasby jim.na...@bluetreble.com wrote: On 1/27/15 1:04 AM, Haribabu Kommi wrote: Here I attached the latest version of the patch. I will add this patch to the next commitfest. Apologies if this was covered, but why isn't the IP address an inet instead of

Re: [HACKERS] Hot Standby WAL reply uses heavyweight session locks, but doesn't have enough infrastructure set up

2015-01-27 Thread Michael Paquier
Andres Freund wrote: I think this isn't particularly pretty, but it seems to be working well enough, and changing it would be pretty invasive. So keeping in line with all that code seems to be easier. OK, I'm convinced with this part to remove the call of LockSharedObjectForSession that uses

Re: [HACKERS] Parallel Seq Scan

2015-01-27 Thread Heikki Linnakangas
On 01/28/2015 04:16 AM, Robert Haas wrote: On Tue, Jan 27, 2015 at 6:00 PM, Robert Haas robertmh...@gmail.com wrote: Now, when you did what I understand to be the same test on the same machine, you got times ranging from 9.1 seconds to 35.4 seconds. Clearly, there is some difference between our

Re: [HACKERS] proposal: plpgsql - Assert statement

2015-01-27 Thread Pavel Stehule
2015-01-28 0:13 GMT+01:00 Jim Nasby jim.na...@bluetreble.com: On 1/27/15 1:30 PM, Pavel Stehule wrote: I don't see the separate warning as being helpful. I'd just do something like +(err_hint != NULL) ? errhint(%s, err_hint) : errhint(Message

Re: [HACKERS] proposal: row_to_array function

2015-01-27 Thread Pavel Stehule
2015-01-28 0:16 GMT+01:00 Jim Nasby jim.na...@bluetreble.com: On 1/27/15 2:26 PM, Pavel Stehule wrote: here is a initial version of row_to_array function - transform any row to array in format proposed by Andrew. Please start a new thread for this... does it depend on the key-value patch?

Re: [HACKERS] proposal: row_to_array function

2015-01-27 Thread Pavel Stehule
Dne 28.1.2015 0:25 Jim Nasby jim.na...@bluetreble.com napsal(a): On 1/27/15 12:58 PM, Pavel Stehule wrote: postgres=# select array(select unnest('{{{1,2},{3,4}},{{5,6},{7,8}}}'::int[])); array --- {1,2,3,4,5,6,7,8} (1 row) so it

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-27 Thread Noah Misch
On Tue, Jan 27, 2015 at 03:56:22PM -0500, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: On 01/27/2015 02:28 PM, Tom Lane wrote: Well, we can either fix it now or suffer with a broken representation forever. I'm not wedded to the exact solution I described, but I think we'll

Re: [HACKERS] WIP: dynahash replacement for buffer table

2015-01-27 Thread Andres Freund
On 2015-01-27 10:36:35 -0500, Robert Haas wrote: On Mon, Jan 26, 2015 at 11:20 PM, Robert Haas robertmh...@gmail.com wrote: This developed a slight merge conflict. I've rebased the attached version, and I also took the step of getting rid of buf_table.c, as I think I proposed somewhere

Re: [HACKERS] Windows buildfarm animals are still not happy with abbreviated keys patch

2015-01-27 Thread Robert Haas
On Mon, Jan 26, 2015 at 9:52 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Tue, Jan 27, 2015 at 4:21 AM, Robert Haas robertmh...@gmail.com wrote: That's what I hope to find out. :-) Buildfarm seems happy now. I just gave a try to that on one of my small Windows VMs and compared the

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Robert Haas
On Fri, Jan 23, 2015 at 1:48 PM, Andres Freund and...@2ndquadrant.com wrote: On 2015-01-22 20:54:47 -0500, Stephen Frost wrote: * Bruce Momjian (br...@momjian.us) wrote: On Fri, Jan 23, 2015 at 01:19:33AM +0100, Andres Freund wrote: Or do you - as the text edited in your patch, but not the

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Jan 27, 2015 at 9:50 AM, Tom Lane t...@sss.pgh.pa.us wrote: That's certainly impossible for the system catalogs, which means you have to be able to deal with relfilenode discrepancies for them, which means that maintaining the same relfilenodes

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Andres Freund
On 2015-01-27 10:20:48 -0500, Robert Haas wrote: On Tue, Jan 27, 2015 at 9:50 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Fri, Jan 23, 2015 at 1:48 PM, Andres Freund and...@2ndquadrant.com wrote: I don't understand why that'd be better than simply

Re: [HACKERS] WIP: dynahash replacement for buffer table

2015-01-27 Thread Robert Haas
On Mon, Jan 26, 2015 at 11:20 PM, Robert Haas robertmh...@gmail.com wrote: This developed a slight merge conflict. I've rebased the attached version, and I also took the step of getting rid of buf_table.c, as I think I proposed somewhere upthread. This avoids the overhead of constructing a

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Jan 27, 2015 at 9:50 AM, Tom Lane t...@sss.pgh.pa.us wrote: That's certainly impossible for the system catalogs, which means you have to be able to deal with relfilenode discrepancies for them, which

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Tue, Jan 27, 2015 at 9:36 AM, Stephen Frost sfr...@snowman.net wrote: The example listed works, but only when it's a local rsync: rsync --archive --hard-links --size-only old_dir new_dir remote_dir Perhaps a better example (or additional

Re: [HACKERS] numeric access out of bounds

2015-01-27 Thread Tom Lane
I wrote: Andrew Gierth and...@tao11.riddles.org.uk writes: I can see two possible fixes: one to correct the assumptions in the macros, the other to check for NaN before calling init_var_from_num in numeric_send (all the other functions seem to do this check explicitly). Which would be

Re: [HACKERS] Re: Abbreviated keys for Numeric

2015-01-27 Thread Andrew Gierth
Peter == Peter Geoghegan p...@heroku.com writes: Peter What I find particularly interesting about this patch is that it Peter makes sorting numerics significantly faster than even sorting Peter float8 values, Played some more with this. Testing on some different gcc versions showed that the

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-27 Thread Andrew Dunstan
On 01/27/2015 12:24 AM, Noah Misch wrote: For the moment, maybe I could commit the fix for the non U+ case for escape_json, and we could think some more about detecting and warning about the problem strings. +1 for splitting development that way. Fixing the use of escape_json() is

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-27 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 01/27/2015 12:24 AM, Noah Misch wrote: +1 for splitting development that way. Fixing the use of escape_json() is objective, but the choices around the warning are more subtle. OK, so something like this patch? I'm mildly concerned that you and I

Re: [HACKERS] File based incremental backup v6

2015-01-27 Thread Marco Nenciarini
Hi, here it is another version of the file based incremental backup patch. Changelog from the previous one: * pg_basebackup --incremental option take the directory containing the base backup instead of the backup profile file * rename the backup_profile file at the same time of backup_label

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-27 Thread Andrew Dunstan
On 01/27/2015 12:23 PM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: On 01/27/2015 12:24 AM, Noah Misch wrote: +1 for splitting development that way. Fixing the use of escape_json() is objective, but the choices around the warning are more subtle. OK, so something like this

[HACKERS] File based Incremental backup v7

2015-01-27 Thread Marco Nenciarini
Il 27/01/15 10:25, Giuseppe Broccolo ha scritto: Hi Marco, On 16/01/15 16:55, Marco Nenciarini wrote: On 14/01/15 17:22, Gabriele Bartolini wrote: My opinion, Marco, is that for version 5 of this patch, you: 1) update the information on the wiki (it is outdated - I know you have been

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-01-27 Thread Abhijit Menon-Sen
At 2015-01-27 17:00:27 -0600, jim.na...@bluetreble.com wrote: It would be best to get this into a commit fest so it's not lost. It's there already. -- Abhijit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Bruce Momjian
On Tue, Jan 27, 2015 at 09:36:58AM -0500, Stephen Frost wrote: The example listed works, but only when it's a local rsync: rsync --archive --hard-links --size-only old_dir new_dir remote_dir Perhaps a better example (or additional one) would be with a remote rsync, including clarification

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread Bruce Momjian
On Tue, Jan 27, 2015 at 09:44:51PM -0500, David Steele wrote: On 1/27/15 9:32 PM, Bruce Momjian wrote Now, this isn't actually a problem for the first time that file is backed up- the issue is if that file isn't changed again. rsync won't re-copy it, but that change that rsync missed won't

Re: [HACKERS] pg_upgrade and rsync

2015-01-27 Thread David Steele
On 1/27/15 9:32 PM, Bruce Momjian wrote Now, this isn't actually a problem for the first time that file is backed up- the issue is if that file isn't changed again. rsync won't re-copy it, but that change that rsync missed won't be in the WAL history for the *second* backup that's done (only