Re: [HACKERS] PL/pgSQL, RAISE and error context

2015-01-26 Thread Marko Tiikkaja
On 1/26/15 1:14 PM, Pavel Stehule wrote: 2015-01-26 13:02 GMT+01:00 Marko Tiikkaja ma...@joh.to: I can see where it's a lot nicer not to have the context visible for people who only care about the contents of the message, but the way it's done in PL/PgSQL right now is just not good enough. On

Re: [HACKERS] PL/pgSQL, RAISE and error context

2015-01-26 Thread Marko Tiikkaja
On 1/26/15 1:44 PM, Pavel Stehule wrote: 2015-01-26 13:39 GMT+01:00 Marko Tiikkaja ma...@joh.to: On 1/26/15 1:14 PM, Pavel Stehule wrote: I afraid about some unexpected side effects of your proposal if somebody mix languages - these side effects should not be critical I have no idea what

Re: [HACKERS] fix typo in guc.c

2015-01-26 Thread Heikki Linnakangas
On 01/26/2015 02:56 PM, Sawada Masahiko wrote: Hi, Attached patch fixes the typo in guc.c. It's typo, I think. diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c index f6df077..f4f1965 100644 --- a/src/backend/utils/misc/guc.c +++ b/src/backend/utils/misc/guc.c @@ -3880,7

Re: [HACKERS] PL/pgSQL, RAISE and error context

2015-01-26 Thread Pavel Stehule
2015-01-26 13:39 GMT+01:00 Marko Tiikkaja ma...@joh.to: On 1/26/15 1:14 PM, Pavel Stehule wrote: 2015-01-26 13:02 GMT+01:00 Marko Tiikkaja ma...@joh.to: I can see where it's a lot nicer not to have the context visible for people who only care about the contents of the message, but the way

[HACKERS] fix typo in guc.c

2015-01-26 Thread Sawada Masahiko
Hi, Attached patch fixes the typo in guc.c. It's typo, I think. Regards, --- Sawada Masahiko fix_typo_guc_c.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-26 Thread Andrew Dunstan
On 01/23/2015 02:18 AM, Noah Misch wrote: On Wed, Jan 21, 2015 at 06:51:34PM -0500, Andrew Dunstan wrote: The following case has just been brought to my attention (look at the differing number of backslashes): andrew=# select jsonb '\\u'; jsonb -- \u

Re: [HACKERS] Unsafe coding in ReorderBufferCommit()

2015-01-26 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2015-01-23 16:47:30 -0500, Tom Lane wrote: There are at least two bugs in reorderbuffer.c's ReorderBufferCommit(): Thanks for fixing these! Unfortunately there's more - we'll currently do bad things if transaction commit fails. At the very

Re: [HACKERS] longjmp clobber warnings are utterly broken in modern gcc

2015-01-26 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2015-01-25 14:02:47 -0500, Tom Lane wrote: I've been looking for other instances of the problem Mark Wilding pointed out, about missing volatile markers on variables that are modified in PG_TRY blocks and then used in the PG_CATCH stanzas. There

Re: [HACKERS] longjmp clobber warnings are utterly broken in modern gcc

2015-01-26 Thread Robert Haas
On Sun, Jan 25, 2015 at 2:02 PM, Tom Lane t...@sss.pgh.pa.us wrote: This is scary as hell. I intend to go around and manually audit every single PG_TRY in the current source code, but that is obviously not a long-term solution. Anybody have an idea about how we might get trustworthy

Re: [HACKERS] longjmp clobber warnings are utterly broken in modern gcc

2015-01-26 Thread Andres Freund
On 2015-01-26 10:52:07 -0500, Robert Haas wrote: On Sun, Jan 25, 2015 at 2:02 PM, Tom Lane t...@sss.pgh.pa.us wrote: This is scary as hell. I intend to go around and manually audit every single PG_TRY in the current source code, but that is obviously not a long-term solution. Anybody have

Re: [HACKERS] PL/pgSQL, RAISE and error context

2015-01-26 Thread Tom Lane
Pavel Stehule pavel.steh...@gmail.com writes: 2015-01-26 14:02 GMT+01:00 Marko Tiikkaja ma...@joh.to: I am thinking, so solution /* if we are doing RAISE, don't report its location */ if (estate-err_text == raise_skip_msg) return; is too simple, and this part should be

Re: [HACKERS] PL/pgSQL, RAISE and error context

2015-01-26 Thread Pavel Stehule
2015-01-26 16:14 GMT+01:00 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: 2015-01-26 14:02 GMT+01:00 Marko Tiikkaja ma...@joh.to: I am thinking, so solution /* if we are doing RAISE, don't report its location */ if (estate-err_text == raise_skip_msg)

Re: [HACKERS] Additional role attributes superuser review

2015-01-26 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: On 2015-01-26 13:47:02 -0500, Stephen Frost wrote: Right. We already have a role attribute which allows pg_basebackup (replication). Also, with pg_basebackup / rolreplication, your role is able to read the entire data directory from the

Re: [HACKERS] Windows buildfarm animals are still not happy with abbreviated keys patch

2015-01-26 Thread Robert Haas
On Mon, Jan 26, 2015 at 2:10 PM, Peter Geoghegan p...@heroku.com wrote: On Mon, Jan 26, 2015 at 11:05 AM, Robert Haas robertmh...@gmail.com wrote: Now that these issues are fixed and the buildfarm is green again, I'm going to try re-enabling this optimization on Windows. My working theory is

Re: [HACKERS] fix typo in guc.c

2015-01-26 Thread Sawada Masahiko
On Mon, Jan 26, 2015 at 10:06 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 01/26/2015 02:56 PM, Sawada Masahiko wrote: Hi, Attached patch fixes the typo in guc.c. It's typo, I think. diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c index

Re: [HACKERS] PL/pgSQL, RAISE and error context

2015-01-26 Thread Pavel Stehule
2015-01-26 14:02 GMT+01:00 Marko Tiikkaja ma...@joh.to: On 1/26/15 1:44 PM, Pavel Stehule wrote: 2015-01-26 13:39 GMT+01:00 Marko Tiikkaja ma...@joh.to: On 1/26/15 1:14 PM, Pavel Stehule wrote: I afraid about some unexpected side effects of your proposal if somebody mix languages - these

Re: [HACKERS] longjmp clobber warnings are utterly broken in modern gcc

2015-01-26 Thread Robert Haas
On Mon, Jan 26, 2015 at 1:34 PM, Andres Freund and...@2ndquadrant.com wrote: I guess we'd need to tie it into PG_exception_stack levels, so it correctly handles nesting with sigsetjmp locations. In contrast to sigsetjmp() style handlers we can't rely on PG_CATCH cleaning up that state. I was

Re: [HACKERS] Additional role attributes superuser review

2015-01-26 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Mon, Jan 26, 2015 at 1:59 PM, Andres Freund and...@2ndquadrant.com wrote: On 2015-01-26 13:47:02 -0500, Stephen Frost wrote: Right. We already have a role attribute which allows pg_basebackup (replication). Also, with pg_basebackup /

Re: [HACKERS] New CF app deployment

2015-01-26 Thread Robert Haas
On Mon, Jan 26, 2015 at 3:13 PM, Peter Geoghegan p...@heroku.com wrote: On Sun, Jan 25, 2015 at 3:22 AM, Andrew Gierth and...@tao11.riddles.org.uk wrote: There's a fairly serious readability problem when someone has posted a patch as a subthread of some more general discussion. For example,

Re: [HACKERS] New CF app deployment

2015-01-26 Thread Magnus Hagander
On Mon, Jan 26, 2015 at 9:20 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, Jan 26, 2015 at 3:13 PM, Peter Geoghegan p...@heroku.com wrote: On Sun, Jan 25, 2015 at 3:22 AM, Andrew Gierth and...@tao11.riddles.org.uk wrote: There's a fairly serious readability problem when someone has

Re: [HACKERS] Additional role attributes superuser review

2015-01-26 Thread Andres Freund
On 2015-01-26 13:47:02 -0500, Stephen Frost wrote: * Robert Haas (robertmh...@gmail.com) wrote: On Wed, Jan 21, 2015 at 11:27 AM, Adam Brightwell adam.brightw...@crunchydatasolutions.com wrote: After re-reading through this thread is seems like EXCLUSIVEBACKUP (proposed by Magnus)

Re: [HACKERS] Additional role attributes superuser review

2015-01-26 Thread Andres Freund
On 2015-01-26 14:05:03 -0500, Stephen Frost wrote: This capability would be used in conjunction with the SAN snapshot capability, it's not intended to be a comparison to what SANs offer. Oh, on a reread that's now clear. Many of those actually allow hooks to be run when taking a snapshot,

Re: [HACKERS] Additional role attributes superuser review

2015-01-26 Thread Robert Haas
On Mon, Jan 26, 2015 at 1:59 PM, Andres Freund and...@2ndquadrant.com wrote: On 2015-01-26 13:47:02 -0500, Stephen Frost wrote: * Robert Haas (robertmh...@gmail.com) wrote: On Wed, Jan 21, 2015 at 11:27 AM, Adam Brightwell adam.brightw...@crunchydatasolutions.com wrote: After re-reading

Re: [HACKERS] Windows buildfarm animals are still not happy with abbreviated keys patch

2015-01-26 Thread Robert Haas
On Fri, Jan 23, 2015 at 12:34 PM, Robert Haas robertmh...@gmail.com wrote: In other words, even on systems that don't HAVE_LOCALE_T, we still have to support the default collation and the C collation, and they have to behave differently. There's no way to make that work using only strxfrm(),

Re: [HACKERS] Windows buildfarm animals are still not happy with abbreviated keys patch

2015-01-26 Thread Peter Geoghegan
On Mon, Jan 26, 2015 at 11:05 AM, Robert Haas robertmh...@gmail.com wrote: Now that these issues are fixed and the buildfarm is green again, I'm going to try re-enabling this optimization on Windows. My working theory is that disabling that categorically was a mis-diagnosis of the real

Re: [HACKERS] New CF app deployment

2015-01-26 Thread Peter Geoghegan
On Sun, Jan 25, 2015 at 3:22 AM, Andrew Gierth and...@tao11.riddles.org.uk wrote: There's a fairly serious readability problem when someone has posted a patch as a subthread of some more general discussion. For example, look at the adaptive ndistinct estimator patch: it's not obvious which

Re: [HACKERS] longjmp clobber warnings are utterly broken in modern gcc

2015-01-26 Thread Andres Freund
Aside from any reduction in the need for volatile, this might actually perform slightly better, because sigsetjmp() is a system call on some platforms. There are probably few cases where that actually matters, but the one in pq_getmessage(), for example, might not be entirely

Re: [HACKERS] Additional role attributes superuser review

2015-01-26 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Wed, Jan 21, 2015 at 11:27 AM, Adam Brightwell adam.brightw...@crunchydatasolutions.com wrote: After re-reading through this thread is seems like EXCLUSIVEBACKUP (proposed by Magnus) seemed to be a potentially acceptable alternative. So this

Re: [HACKERS] New CF app deployment

2015-01-26 Thread Robert Haas
On Mon, Jan 26, 2015 at 4:16 PM, Josh Berkus j...@agliodbs.com wrote: Well, if it's essentially unusable, then we've reached parity with the old app (yes, you deserved that). No, I didn't. What we had before I wrote that tool was a bunch of wiki pages you put together which were forever having

Re: [HACKERS] New CF app deployment

2015-01-26 Thread Josh Berkus
On 01/26/2015 01:29 PM, Robert Haas wrote: Furthermore, if it sucked so bad, why did it take anyone 5 years to get around to rewriting it? It took me less than a year to get around to replacing what you wrote. Because whoever replaced it knew they'd be facing a shitstorm of criticism? --

Re: [HACKERS] proposal: row_to_array function

2015-01-26 Thread Jim Nasby
On 1/25/15 4:23 AM, Pavel Stehule wrote: I tested a concept iteration over array in format [key1, value1, key2, value2, .. ] - what is nice, it works for [[key1,value1],[key2, value2], ...] too It is only a few lines more to current code, and this change doesn't break a compatibility. Do

Re: [HACKERS] New CF app deployment

2015-01-26 Thread Magnus Hagander
On Mon, Jan 26, 2015 at 9:46 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, Jan 26, 2015 at 3:24 PM, Magnus Hagander mag...@hagander.net wrote: Yes, and the agreement after that feedback was to try it out and then figure out what changes were needed? As about half the feedback said

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2015-01-26 Thread Jim Nasby
On 1/23/15 2:15 PM, Stephen Frost wrote: I happen to like the idea specifically because it would allow regular roles to change the auditing settings (no need to be a superuser or to be able to modify postgresql.conf/postgresql.auto.conf) Is there really a use case for non-superusers to be

Re: [HACKERS] New CF app deployment

2015-01-26 Thread Andres Freund
On 2015-01-26 13:32:51 -0800, Josh Berkus wrote: On 01/26/2015 01:29 PM, Robert Haas wrote: Furthermore, if it sucked so bad, why did it take anyone 5 years to get around to rewriting it? It took me less than a year to get around to replacing what you wrote. Because whoever replaced

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2015-01-26 Thread Robert Haas
On Fri, Jan 23, 2015 at 1:16 PM, Stephen Frost sfr...@snowman.net wrote: Well, I'm still of the view that there's little to lose by having this remain out-of-core for a release or three. We don't really all agree on what we want, and non-core code can evolve a lot faster than core code, so

Re: [HACKERS] New CF app deployment

2015-01-26 Thread Peter Geoghegan
On Mon, Jan 26, 2015 at 12:46 PM, Robert Haas robertmh...@gmail.com wrote: Well, I can't speak to anyone else's opinion, but I'm quite sure I raised the issue that we need a way to call out which messages in the thread are important, and I think that's pretty much what Peter is saying, too.

Re: [HACKERS] New CF app deployment

2015-01-26 Thread Robert Haas
On Mon, Jan 26, 2015 at 4:07 PM, Magnus Hagander mag...@hagander.net wrote: I assume what was referred to was that the old cf app would show the last 3 (I think it was) comments/patches/whatnot on a patch on the summary page (and then clickthrough for more details). Yep. -- Robert Haas

Re: [HACKERS] New CF app deployment

2015-01-26 Thread Josh Berkus
On 01/26/2015 12:46 PM, Robert Haas wrote: I find the new tool essentially unusable - having one link to the whole thread instead of individual links to just the important messages in that thread is a huge regression for me, as is the lack of the most recent activity on the summary page.

Re: [HACKERS] New CF app deployment

2015-01-26 Thread Robert Haas
On Mon, Jan 26, 2015 at 4:01 PM, Magnus Hagander mag...@hagander.net wrote: According to my mailbox, you didn't even respond on that thread. But it may well be that your email ended up on some other thread and therefor was not included when I went back and looked over all the responses I got on

[HACKERS] Hot Standby WAL reply uses heavyweight session locks, but doesn't have enough infrastructure set up

2015-01-26 Thread Andres Freund
Hi, dbase_redo does: if (InHotStandby) { /* * Lock database while we resolve conflicts to ensure that * InitPostgres() cannot fully re-execute concurrently. This *

[HACKERS] Re: Abbreviated keys for Numeric (was: Re: B-Tree support function number 3 (strxfrm() optimization))

2015-01-26 Thread Peter Geoghegan
On Mon, Jan 26, 2015 at 8:43 AM, Andrew Gierth and...@tao11.riddles.org.uk wrote: Another spinoff from the abbreviation discussion. Peter Geoghegan suggested on IRC that numeric would benefit from abbreviation, and indeed it does (in some cases by a factor of about 6-7x or more, because

Re: [HACKERS] Parallel Seq Scan

2015-01-26 Thread Jim Nasby
On 1/23/15 10:16 PM, Amit Kapila wrote: Further, if we want to just get the benefit of parallel I/O, then I think we can get that by parallelising partition scan where different table partitions reside on different disk partitions, however that is a matter of separate patch. I don't think we

Re: [HACKERS] proposal: searching in array function - array_position

2015-01-26 Thread Jim Nasby
On 1/24/15 2:48 AM, Pavel Stehule wrote: with array_offsets - returns a array of offsets + entryreturns a offset of first occurrence of some element in a array. It uses should be + entryreturns the offset of the first occurrence of some element in an array. It uses +

Re: [HACKERS] pg_upgrade and rsync

2015-01-26 Thread Jim Nasby
On 1/23/15 12:40 PM, Stephen Frost wrote: That said, the whole timestamp race condition in rsync gives me the heebie-jeebies. For normal workloads maybe it's not that big a deal, but when dealing with fixed-size data (ie: Postgres blocks)? Eww. The race condition is a problem for

Re: Partitioning: issues/ideas (Was: Re: [HACKERS] On partitioning)

2015-01-26 Thread Jim Nasby
On 1/25/15 7:42 PM, Amit Langote wrote: On 21-01-2015 PM 07:26, Amit Langote wrote: Ok, I will limit myself to focusing on following things at the moment: * Provide syntax in CREATE TABLE to declare partition key While working on this, I stumbled upon the question of how we deal with any

Re: [HACKERS] PL/pgSQL, RAISE and error context

2015-01-26 Thread Jim Nasby
On 1/26/15 9:46 AM, Pavel Stehule wrote: The design I thought had been agreed on was to add some new option to plpgsql's RAISE command which would cause suppression of all CONTEXT lines not just the most closely nested one. You could argue about whether the behavior needs to be

Re: [HACKERS] basebackups during ALTER DATABASE ... SET TABLESPACE ... not safe?

2015-01-26 Thread Andres Freund
Hi, On 2015-01-22 19:56:07 +0100, Andres Freund wrote: Hi, On 2015-01-20 16:28:19 +0100, Andres Freund wrote: I'm analyzing a problem in which a customer had a pg_basebackup (from standby) created 9.2 cluster that failed with WAL contains references to invalid pages. The failed record

Re: [HACKERS] New CF app deployment

2015-01-26 Thread Andres Freund
On 2015-01-26 12:54:04 -0800, Peter Geoghegan wrote: On Mon, Jan 26, 2015 at 12:46 PM, Robert Haas robertmh...@gmail.com wrote: Well, I can't speak to anyone else's opinion, but I'm quite sure I raised the issue that we need a way to call out which messages in the thread are important, and

Re: [HACKERS] New CF app deployment

2015-01-26 Thread Magnus Hagander
On Mon, Jan 26, 2015 at 10:05 PM, Andres Freund and...@2ndquadrant.com wrote: On 2015-01-26 12:54:04 -0800, Peter Geoghegan wrote: On Mon, Jan 26, 2015 at 12:46 PM, Robert Haas robertmh...@gmail.com wrote: Well, I can't speak to anyone else's opinion, but I'm quite sure I raised the

Re: [HACKERS] proposal: plpgsql - Assert statement

2015-01-26 Thread Pavel Stehule
2015-01-26 22:34 GMT+01:00 Jim Nasby jim.na...@bluetreble.com: On 1/22/15 2:01 PM, Pavel Stehule wrote: * I would to simplify a behave of evaluating of message expression - probably I disallow NULL there. Well, the only thing I could see you doing there is throwing a

Re: [HACKERS] New CF app deployment

2015-01-26 Thread Robert Haas
On Mon, Jan 26, 2015 at 4:32 PM, Josh Berkus j...@agliodbs.com wrote: On 01/26/2015 01:29 PM, Robert Haas wrote: Furthermore, if it sucked so bad, why did it take anyone 5 years to get around to rewriting it? It took me less than a year to get around to replacing what you wrote. Because

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

2015-01-26 Thread Robert Haas
On Mon, Jan 26, 2015 at 8:04 PM, Andres Freund and...@2ndquadrant.com wrote: On 2015-01-26 19:58:25 -0500, Bruce Momjian wrote: On Tue, Jan 27, 2015 at 01:43:41AM +0100, Andres Freund wrote: master + 32align.patch: -c max_connections=400 tps = 183791.872359 (high run vs. run variability)

Re: [HACKERS] ExplainModifyTarget doesn't work as expected

2015-01-26 Thread Etsuro Fujita
On 2015/01/27 9:15, Jim Nasby wrote: On 12/22/14 12:50 AM, Etsuro Fujita wrote: I think ExplainModifyTarget should show the parent of the inheritance tree in multi-target-table cases, as described there, but noticed that it doesn't always work like that. Here is an example. Anything ever

Re: [HACKERS] Providing catalog view to pg_hba.conf file - Patch submission

2015-01-26 Thread Haribabu Kommi
On Mon, Jun 30, 2014 at 5:06 PM, Abhijit Menon-Sen a...@2ndquadrant.com wrote: I think having two columns would work. The columns could be called database and database_list and user and user_list respectively. The database column may contain one of all, sameuser, samegroup, replication, but

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

2015-01-26 Thread Robert Haas
On Mon, Jan 26, 2015 at 9:08 PM, Robert Haas robertmh...@gmail.com wrote: Contrary opinions? Robert? I'm totally OK with further aligning just that one allocation. Of course, now that I think about it, aligning it probably works mostly because the size is almost exactly one cache line. If it

Re: [HACKERS] WIP: dynahash replacement for buffer table

2015-01-26 Thread Robert Haas
This developed a slight merge conflict. I've rebased the attached version, and I also took the step of getting rid of buf_table.c, as I think I proposed somewhere upthread. This avoids the overhead of constructing a BufferTag only to copy it into a BufferLookupEnt, plus some function calls and

Re: [HACKERS] Hot Standby WAL reply uses heavyweight session locks, but doesn't have enough infrastructure set up

2015-01-26 Thread Michael Paquier
On Tue, Jan 27, 2015 at 6:24 AM, Andres Freund and...@2ndquadrant.com wrote: Unfortunately that Assert()s when there's a lock conflict because e.g. another backend is currently connecting. That's because ProcSleep() does a enable_timeout_after(DEADLOCK_TIMEOUT, DeadlockTimeout) - and there's

Re: [HACKERS] basebackups during ALTER DATABASE ... SET TABLESPACE ... not safe?

2015-01-26 Thread Andres Freund
On 2015-01-26 22:03:03 +0100, Andres Freund wrote: Attached is a patch trying to this. Doesn't look too bad and lead me to discover missing recovery conflicts during a AD ST. But: It doesn't actually work on standbys, because lock.c prevents any stronger lock than RowExclusive from being

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2015-01-26 Thread Dilip kumar
On 23 January 2015 21:10, Alvaro Herrera Wrote, In case you're up for doing some more work later on, there are two ideas here: move the backend's TranslateSocketError to src/common, and try to merge pg_dump's select_loop function with the one in this new code. But that's for another patch

Re: [HACKERS] Windows buildfarm animals are still not happy with abbreviated keys patch

2015-01-26 Thread Michael Paquier
On Tue, Jan 27, 2015 at 4:21 AM, Robert Haas robertmh...@gmail.com wrote: That's what I hope to find out. :-) Buildfarm seems happy now. I just gave a try to that on one of my small Windows VMs and compared the performance with 9.4 for this simple test case when building with MSVC 2010: create

Re: [HACKERS] Parallel Seq Scan

2015-01-26 Thread Amit Kapila
On Tue, Jan 27, 2015 at 3:18 AM, Jim Nasby jim.na...@bluetreble.com wrote: On 1/23/15 10:16 PM, Amit Kapila wrote: Further, if we want to just get the benefit of parallel I/O, then I think we can get that by parallelising partition scan where different table partitions reside on different

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2015-01-26 Thread Dilip kumar
On 23 January 2015 23:55, Alvaro Herrera, -j1 is now the same as not specifying anything, and vacuum_one_database uses more common code in the parallel and not-parallel cases: the not- parallel case is just the parallel case with a single connection, so the setup and shutdown is mostly the

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2015-01-26 Thread Andreas Karlsson
On 01/23/2015 02:58 AM, Petr Jelinek wrote: On 23/01/15 00:40, Andreas Karlsson wrote: - Renamed some things from int12 to int128, there are still some places with int16 which I am not sure what to do with. I'd vote for renaming them to int128 too, there is enough C functions that user int16

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-26 Thread Noah Misch
On Mon, Jan 26, 2015 at 09:20:54AM -0500, Andrew Dunstan wrote: On 01/23/2015 02:18 AM, Noah Misch wrote: On Wed, Jan 21, 2015 at 06:51:34PM -0500, Andrew Dunstan wrote: We could probably fix this fairly easily for non- U+ cases by having jsonb_to_cstring use a different escape_json

Re: [HACKERS] Shortcoming in CLOBBER_FREED_MEMORY coverage: disk buffer pointers

2015-01-26 Thread Jim Nasby
On 1/26/15 4:51 PM, Tom Lane wrote: Jim Nasby jim.na...@bluetreble.com writes: On 1/24/15 3:31 PM, Tom Lane wrote: Another idea is to teach Valgrind that whenever a backend reduces its pin count on a shared buffer to zero, that buffer should become undefined memory. paranoia Shouldn't

Re: [HACKERS] ExplainModifyTarget doesn't work as expected

2015-01-26 Thread Jim Nasby
On 12/22/14 12:50 AM, Etsuro Fujita wrote: I think ExplainModifyTarget should show the parent of the inheritance tree in multi-target-table cases, as described there, but noticed that it doesn't always work like that. Here is an example. Anything ever happen with this? -- Jim Nasby, Data

Re: Suppressing elog.c context messages (was Re: [HACKERS] Wait free LW_SHARED acquisition)

2015-01-26 Thread Andres Freund
On 2015-01-26 18:30:13 -0600, Jim Nasby wrote: On 12/23/14 11:41 AM, Andres Freund wrote: I think it'd generally be useful to have something like errhidecontext() akin to errhidestatement() to avoid things like the above. Under this proposal, do you want to suppress the

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-01-26 Thread Jim Nasby
On 12/26/14 1:38 AM, Abhijit Menon-Sen wrote: At 2014-09-25 15:40:11 +0530,a...@2ndquadrant.com wrote: All right, then I'll post a version that addresses Amit's other points, adds a new file/function to pgstattuple, acquires content locks, and uses HeapTupleSatisfiesVacuum, hint-bit setting

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

2015-01-26 Thread Andres Freund
On 2015-01-26 19:58:25 -0500, Bruce Momjian wrote: On Tue, Jan 27, 2015 at 01:43:41AM +0100, Andres Freund wrote: master + 32align.patch: -c max_connections=400 tps = 183791.872359 (high run vs. run variability) -c max_connections=401 tps = 185494.98244 (high run vs. run variability)

Re: [HACKERS] Parallel Seq Scan

2015-01-26 Thread Daniel Bausch
Hi PG devs! Tom Lane t...@sss.pgh.pa.us writes: Wait for first IO, issue second IO request Compute Already have second IO request, issue third ... We'd be a lot less sensitive to IO latency. It would take about five minutes of coding to prove or disprove this: stick a PrefetchBuffer

[HACKERS] SSL renegotiation and other related woes

2015-01-26 Thread Andres Freund
Hi, When working on getting rid of ImmediateInterruptOK I wanted to verify that ssl still works correctly. Turned out it didn't. But neither did it in master. Turns out there's two major things we do wrong: 1) We ignore the rule that once called and returning SSL_ERROR_WANTS_(READ|WRITE)

Re: [HACKERS] SSL renegotiation and other related woes

2015-01-26 Thread Andres Freund
On 2015-01-26 11:14:05 +0100, Andres Freund wrote: My testcase for this is just to setup a server with a low ssl_renegotiation_limit, generate lots of WAL (wal.sql attached) and receive data via pg_receivexlog -n. Usually it'll error out quickly. ... Greetings, Andres Freund -- Andres

Re: [HACKERS] basebackups during ALTER DATABASE ... SET TABLESPACE ... not safe?

2015-01-26 Thread Andres Freund
On 2015-01-22 22:58:17 +0100, Andres Freund wrote: Because the way it currently works is a major crock. It's more luck than anything that it actually somewhat works. We normally rely on WAL to bring us into a consistent state. But around CREATE/MOVE/DROP DATABASE we've ignored that. Hah:

[HACKERS] pg_dump with both --serializable-deferrable and -j

2015-01-26 Thread Alexander Korotkov
Hackers, when pg_dump is run with both --serializable-deferrable and -j options to pg_dump, it returns errors: pg_dump: [archiver (db)] query failed: ERROR: a snapshot-importing transaction must not be READ ONLY DEFERRABLE pg_dump: [archiver (db)] query failed: ERROR: a snapshot-importing

Re: [HACKERS] Unsafe coding in ReorderBufferCommit()

2015-01-26 Thread Andres Freund
Hi Tom, On 2015-01-23 16:47:30 -0500, Tom Lane wrote: There are at least two bugs in reorderbuffer.c's ReorderBufferCommit(): Thanks for fixing these! Unfortunately there's more - we'll currently do bad things if transaction commit fails. At the very least the (sub-)transaction begin commands

Re: [HACKERS] longjmp clobber warnings are utterly broken in modern gcc

2015-01-26 Thread Andres Freund
Hi, On 2015-01-25 14:02:47 -0500, Tom Lane wrote: I've been looking for other instances of the problem Mark Wilding pointed out, about missing volatile markers on variables that are modified in PG_TRY blocks and then used in the PG_CATCH stanzas. There definitely are some. Current gcc

Re: [HACKERS] longjmp clobber warnings are utterly broken in modern gcc

2015-01-26 Thread Andres Freund
On 2015-01-25 15:40:10 -0500, Tom Lane wrote: Greg Stark st...@mit.edu writes: Perhaps Clang has a more useful warning? Clang, at least the version on my Mac, doesn't warn either with the settings we normally use, and it doesn't have -Wclobber at all. I tried turning on -Weverything, and

Re: [HACKERS] PL/pgSQL, RAISE and error context

2015-01-26 Thread Marko Tiikkaja
On 1/22/15 6:03 PM, Pavel Stehule wrote: 2015-01-22 12:37 GMT+01:00 Marko Tiikkaja ma...@joh.to: Or is that a stupid idea? I just think hacking libpq for something like this is a huge overkill. I don't think so only plpgsql solution is satisfactory idea. There are some mix plpgsql / plperl

Re: [HACKERS] PL/pgSQL, RAISE and error context

2015-01-26 Thread Pavel Stehule
2015-01-26 13:02 GMT+01:00 Marko Tiikkaja ma...@joh.to: On 1/22/15 6:03 PM, Pavel Stehule wrote: 2015-01-22 12:37 GMT+01:00 Marko Tiikkaja ma...@joh.to: Or is that a stupid idea? I just think hacking libpq for something like this is a huge overkill. I don't think so only plpgsql

Re: [HACKERS] proposal: searching in array function - array_position

2015-01-26 Thread Pavel Stehule
2015-01-26 23:01 GMT+01:00 Jim Nasby jim.na...@bluetreble.com: On 1/24/15 2:48 AM, Pavel Stehule wrote: with array_offsets - returns a array of offsets + entryreturns a offset of first occurrence of some element in a array. It uses should be + entryreturns the offset of

Re: [HACKERS] Shortcoming in CLOBBER_FREED_MEMORY coverage: disk buffer pointers

2015-01-26 Thread Jim Nasby
On 1/24/15 3:31 PM, Tom Lane wrote: Another idea is to teach Valgrind that whenever a backend reduces its pin count on a shared buffer to zero, that buffer should become undefined memory. paranoia Shouldn't this technically tie in with ResourceOwners? If a pointer takes the pin count from 1

Re: [HACKERS] Parallel Seq Scan

2015-01-26 Thread Tom Lane
Jim Nasby jim.na...@bluetreble.com writes: On 1/23/15 10:16 PM, Amit Kapila wrote: Further, if we want to just get the benefit of parallel I/O, then I think we can get that by parallelising partition scan where different table partitions reside on different disk partitions, however that is a

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2015-01-26 Thread Robert Haas
On Mon, Jan 26, 2015 at 5:21 PM, Stephen Frost sfr...@snowman.net wrote: I don't disagree with you about any of that. I don't think you disagree with my comment either. What I'm not entirely clear on is how consensus could be reached. Leaving it dormant for the better part of a year

Re: [HACKERS] New CF app deployment

2015-01-26 Thread Robert Haas
On Mon, Jan 26, 2015 at 5:36 PM, Josh Berkus j...@agliodbs.com wrote: In order to get a consensus on moving to a new app I had to explain what was wrong with the old app. Eventually I had to use strong language to do so, because nobody was paying attention otherwise. While Magnus's app isn't

Re: [HACKERS] pg_upgrade and rsync

2015-01-26 Thread David Steele
On 1/26/15 5:11 PM, Jim Nasby wrote: The race condition is a problem for pg_start/stop_backup and friends. In this instance, everything will be shut down when the rsync is running, so there isn't a timestamp race condition to worry about. Yeah, I'm more concerned about people that use rsync

Re: [HACKERS] Re: Abbreviated keys for Numeric

2015-01-26 Thread Peter Geoghegan
On Mon, Jan 26, 2015 at 3:12 PM, Andrew Gierth and...@tao11.riddles.org.uk wrote: Obvious overheads in float8 comparison include having to check for NaN, and the fact that DatumGetFloat8 on 64bit doesn't get inlined and forces a store/load to memory rather than just using a register. Looking at

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2015-01-26 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: On Fri, Jan 23, 2015 at 1:16 PM, Stephen Frost sfr...@snowman.net wrote: Well, I'm still of the view that there's little to lose by having this remain out-of-core for a release or three. We don't really all agree on what we want, and

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2015-01-26 Thread Stephen Frost
Jim, * Jim Nasby (jim.na...@bluetreble.com) wrote: When it comes to changing auditing settings, I think that needs to be very restrictive. Really, it should be more (or differently) restrictive than SU, so that you can effectively audit your superusers with minimal worries about superusers

Re: [HACKERS] proposal: searching in array function - array_position

2015-01-26 Thread Jim Nasby
On 1/26/15 4:17 PM, Pavel Stehule wrote: Any way to reduce the code duplication between the array and non-array versions? Maybe factor out the operator caching code? I though about it - but there is different checks, different result processing, different result type. I didn't find any

Re: [HACKERS] pg_upgrade and rsync

2015-01-26 Thread Stephen Frost
Jim, * Jim Nasby (jim.na...@bluetreble.com) wrote: On 1/23/15 12:40 PM, Stephen Frost wrote: That said, the whole timestamp race condition in rsync gives me the heebie-jeebies. For normal workloads maybe it's not that big a deal, but when dealing with fixed-size data (ie: Postgres blocks)?

Re: [HACKERS] New CF app deployment

2015-01-26 Thread Josh Berkus
Robert, Didn't stop me. And actually, I didn't face a shitstorm of criticism. The way I remember it, I got a pretty much unqualified positive reaction at the time. Including from me, because it was a huge improvement on what we had before. As the new app is. Only later, when you had

Re: [HACKERS] Shortcoming in CLOBBER_FREED_MEMORY coverage: disk buffer pointers

2015-01-26 Thread Tom Lane
Jim Nasby jim.na...@bluetreble.com writes: On 1/24/15 3:31 PM, Tom Lane wrote: Another idea is to teach Valgrind that whenever a backend reduces its pin count on a shared buffer to zero, that buffer should become undefined memory. paranoia Shouldn't this technically tie in with

Re: [HACKERS] Re: Abbreviated keys for Numeric

2015-01-26 Thread Andrew Gierth
Peter == Peter Geoghegan p...@heroku.com writes: Peter What I find particularly interesting about this patch is that it Peter makes sorting numerics significantly faster than even sorting Peter float8 values, I get a much smaller difference there than you do. Obvious overheads in float8

Re: [HACKERS] Re: Abbreviated keys for Numeric

2015-01-26 Thread Andres Freund
On 2015-01-26 15:35:44 -0800, Peter Geoghegan wrote: On Mon, Jan 26, 2015 at 3:12 PM, Andrew Gierth and...@tao11.riddles.org.uk wrote: Obvious overheads in float8 comparison include having to check for NaN, and the fact that DatumGetFloat8 on 64bit doesn't get inlined and forces a

Re: [HACKERS] pg_upgrade and rsync

2015-01-26 Thread Jim Nasby
On 1/26/15 5:08 PM, David Steele wrote: I've written tests to show the rsync vulnerability and another to show that this can affect a running database. However, to reproduce it reliably you need to force a checkpoint or have them happening pretty close together. Related to this and Stephen's

Re: [HACKERS] Shortcoming in CLOBBER_FREED_MEMORY coverage: disk buffer pointers

2015-01-26 Thread Greg Stark
On Tue, Jan 27, 2015 at 12:03 AM, Jim Nasby jim.na...@bluetreble.com wrote: But one backend can effectively pin a buffer more than once, no? If so, then ISTM there's some risk that code path A pins and forgets to unpin, but path B accidentally unpins for A. The danger is that there's a

Re: [HACKERS] Re: Abbreviated keys for Numeric

2015-01-26 Thread Petr Jelinek
On 27/01/15 00:51, Andres Freund wrote: On 2015-01-26 15:35:44 -0800, Peter Geoghegan wrote: On Mon, Jan 26, 2015 at 3:12 PM, Andrew Gierth and...@tao11.riddles.org.uk wrote: Obvious overheads in float8 comparison include having to check for NaN, and the fact that DatumGetFloat8 on 64bit

Re: Suppressing elog.c context messages (was Re: [HACKERS] Wait free LW_SHARED acquisition)

2015-01-26 Thread Jim Nasby
On 12/23/14 11:41 AM, Andres Freund wrote: I think it'd generally be useful to have something like errhidecontext() akin to errhidestatement() to avoid things like the above. Under this proposal, do you want to suppress the context/statement unconditionally or via some hook/variable,

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

2015-01-26 Thread Andres Freund
On 2015-01-05 21:43:04 -0500, Bruce Momjian wrote: On Fri, Jan 2, 2015 at 06:25:52PM +0100, Andres Freund wrote: I can't run tests right now... What exactly do you want to see with these tests? that's essentially what I've already benchmarked + some fprintfs? I want to test two patches

Re: [HACKERS] Proposal: two new role attributes and/or capabilities?

2015-01-26 Thread Jim Nasby
On 12/23/14 12:52 PM, Stephen Frost wrote: * José Luis Tallón (jltal...@adv-solutions.net) wrote: On 12/23/2014 05:29 PM, Stephen Frost wrote: The capabilities would be: * MAINTENANCE --- Ability to run VACUUM [ANALYZE | FREEZE] (but not VACUUM FULL), ANALYZE (including SET

  1   2   >