Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

2015-01-28 Thread Merlin Moncure
On Thu, Jan 22, 2015 at 3:50 PM, Merlin Moncure mmonc...@gmail.com wrote: I still haven't categorically ruled out pl/sh yet; that's something to keep in mind. Well, after bisection proved not to be fruitful, I replaced the pl/sh calls with dummy calls that approximated the same behavior and the

Re: [HACKERS] Parallel Seq Scan

2015-01-28 Thread Thom Brown
On 28 January 2015 at 14:03, Robert Haas robertmh...@gmail.com wrote: The problem here, as I see it, is that we're flying blind. If there's just one spindle, I think it's got to be right to read the relation sequentially. But if there are multiple spindles, it might not be, but it seems

Re: [HACKERS] Safe memory allocation functions

2015-01-28 Thread Michael Paquier
On Tue, Jan 27, 2015 at 5:34 PM, Andres Freund and...@2ndquadrant.com wrote: On 2015-01-27 17:27:53 +0900, Michael Paquier wrote: Alvaro Herrera wrote: So how about something like #define ALLOCFLAG_HUGE 0x01 #define ALLOCFLAG_NO_ERROR_ON_OOM 0x02 void *

Re: [HACKERS] TABLESAMPLE patch

2015-01-28 Thread Petr Jelinek
On 28/01/15 09:41, Kyotaro HORIGUCHI wrote: As an issue related to size esmation, I got a explain result as following, =# explain (analyze on, buffers on) select a from t1 tablesample system(10) where a 5; QUERY PLAN

Re: [HACKERS] pg_upgrade and rsync

2015-01-28 Thread Stephen Frost
* Jim Nasby (jim.na...@bluetreble.com) wrote: On 1/27/15 9:29 AM, Stephen Frost wrote: My point is that Bruce's patch suggests looking for remote_dir in the rsync documentation, but no such term appears there. Ah, well, perhaps we could simply add a bit of clarification to this: for details

Re: [HACKERS] Parallel Seq Scan

2015-01-28 Thread Robert Haas
On Wed, Jan 28, 2015 at 2:08 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: OTOH, spreading the I/O across multiple files is not a good thing, if you don't have a RAID setup like that. With a single spindle, you'll just induce more seeks. Perhaps the OS is smart enough to read in

Re: [HACKERS] pg_upgrade and rsync

2015-01-28 Thread Stephen Frost
Bruce, * Bruce Momjian (br...@momjian.us) wrote: On Tue, Jan 27, 2015 at 09:36:58AM -0500, Stephen Frost wrote: The example listed works, but only when it's a local rsync: rsync --archive --hard-links --size-only old_dir new_dir remote_dir Perhaps a better example (or additional one)

Re: [HACKERS] pg_upgrade and rsync

2015-01-28 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: Interesting problem, but doesn't rsync use sub-second accuracy? No. Simple test will show: touch xx/aa ; rsync -avv xx yy ; sleep 0.5 ; touch xx/aa ; rsync -avv xx yy Run that a few times and you'll see it report xx/aa is uptodate sometimes, depending

[HACKERS] ya comment typo

2015-01-28 Thread Erik Rijkers
'the the' in bufmgr.c--- src/backend/storage/buffer/bufmgr.c.orig 2015-01-28 09:13:39.681366670 +0100 +++ src/backend/storage/buffer/bufmgr.c 2015-01-28 09:14:14.225690845 +0100 @@ -138,7 +138,7 @@ static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); /* - * Ensure that the the

Re: [HACKERS] ya comment typo

2015-01-28 Thread Heikki Linnakangas
On 01/28/2015 10:16 AM, Erik Rijkers wrote: 'the the' in bufmgr.c Fixed, thanks. - Heikki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] TABLESAMPLE patch

2015-01-28 Thread Kyotaro HORIGUCHI
Hello, On 19/01/15 07:08, Amit Kapila wrote: On Sun, Jan 18, 2015 at 12:46 AM, Petr Jelinek p...@2ndquadrant.com mailto:p...@2ndquadrant.com wrote: No issues, but it seems we should check other paths where different handling could be required for tablesample scan. In set_rel_size(), it

Re: [HACKERS] Dereferenced pointers checked as NULL in btree_utils_var.c

2015-01-28 Thread Heikki Linnakangas
On 01/28/2015 02:47 AM, Michael Paquier wrote: On Wed, Jan 28, 2015 at 3:15 AM, Tom Lane t...@sss.pgh.pa.us wrote: So I'm fine with taking out both this documentation text and the dead null-pointer checks; but let's do that all in one patch not piecemeal. In any case, this is just cosmetic

Re: [HACKERS] WITH CHECK and Column-Level Privileges

2015-01-28 Thread Dean Rasheed
On 27 January 2015 at 22:45, Stephen Frost sfr...@snowman.net wrote: Here's the latest set with a few additional improvements (mostly comments but also a couple missed #include's and eliminating unnecessary whitespace changes). Unless there are issues with my testing tonight or concerns

Re: [HACKERS] EvalPlanQual behaves oddly for FDW queries involving system columns

2015-01-28 Thread Etsuro Fujita
On 2015/01/19 17:10, Etsuro Fujita wrote: Attached is an updated version of the patch. I'll add this to the next CF. Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

2015-01-28 Thread Andres Freund
On 2015-01-26 21:13:31 -0500, Robert Haas wrote: On Mon, Jan 26, 2015 at 9:08 PM, Robert Haas robertmh...@gmail.com wrote: Contrary opinions? Robert? I'm totally OK with further aligning just that one allocation. Of course, now that I think about it, aligning it probably works mostly

Re: [HACKERS] TABLESAMPLE patch

2015-01-28 Thread Petr Jelinek
On 28/01/15 08:23, Kyotaro HORIGUCHI wrote: Hi, I took a look on this and found nice. By the way, the parameter for REPEATABLE seems allowing to be a expression in ParseTableSample but the grammer rejects it. It wasn't my intention to support it, but you are correct, the code is generic

Re: [HACKERS] PQgetssl() and alternative SSL implementations

2015-01-28 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Stephen Frost sfr...@snowman.net writes: What bothers me about this is that it punts SSL work to the application and requires that they be coded to work with both OpenSSL and whatever else we implement (eg: GnuTLS) to do anything but the most simple

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-28 Thread Tom Lane
Noah Misch n...@leadboat.com writes: On Tue, Jan 27, 2015 at 03:56:22PM -0500, Tom Lane wrote: So at this point I propose that we reject \u when de-escaping JSON. I would have agreed on 2014-12-09, and this release is the last chance to make such a change. It is a bold wager that could

Re: [HACKERS] Sequence Access Method WIP

2015-01-28 Thread Heikki Linnakangas
On 01/23/2015 02:34 AM, Petr Jelinek wrote: On 22/01/15 17:02, Petr Jelinek wrote: The new version (the one that is not submitted yet) of gapless sequence is way more ugly and probably not best example either but does guarantee gaplessness (it stores the last value in it's own value table). So

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-28 Thread Andrew Dunstan
On 01/28/2015 12:50 AM, Noah Misch wrote: On Tue, Jan 27, 2015 at 03:56:22PM -0500, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: On 01/27/2015 02:28 PM, Tom Lane wrote: Well, we can either fix it now or suffer with a broken representation forever. I'm not wedded to the exact

Re: [HACKERS] PQgetssl() and alternative SSL implementations

2015-01-28 Thread Heikki Linnakangas
On 01/28/2015 06:58 PM, Stephen Frost wrote: Although I think OpenSSL SSL is a little bit duplicatively redundant. Why not just OpenSSL? I wondered also, but figured it was probably because it's OpenSSL's ssl structure, which then made sense. Right, that was the idea. I wanted it to include

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-28 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: It's not clear to me how we should represent a unicode null. i.e. given a json of '[foo\ubar]', I get that we'd store the element as 'foo\x00bar', but what is the result of (jsonb '[foo\ubar')-0 It's defined to be text so we can't

Re: [HACKERS] PATCH: decreasing memory needlessly consumed by array_agg

2015-01-28 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes: On Tue, Jan 20, 2015 at 6:44 AM, Tom Lane t...@sss.pgh.pa.us wrote: Uh, sorry, I've not been paying any attention to this thread for awhile. What's the remaining questions at issue? This patch is trying to improve the array_agg case where there are many

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-28 Thread Tom Lane
Jim Nasby jim.na...@bluetreble.com writes: While I sympathize with Noah's sentiments, the only thing that makes sense to me is that a JSON text field is treated the same way as we treat text. Right now, that means NUL is not allowed, period. If no one has bitched about this with text, is it

Re: [HACKERS] pg_dump with both --serializable-deferrable and -j

2015-01-28 Thread Andres Freund
On 2015-01-28 15:32:15 +, Kevin Grittner wrote: Andres Freund and...@2ndquadrant.com wrote: ISTM that the check is just overzelous and/or needs to be moved into ImportSnapshot(). There it then could be made to check if the exporting xact was also deferrable. That would be great if

Re: [HACKERS] Make hba available to client code

2015-01-28 Thread Andrew Dunstan
On 01/28/2015 04:26 PM, David Fetter wrote: On Wed, Jan 28, 2015 at 04:10:42PM -0500, Tom Lane wrote: David Fetter da...@fetter.org writes: While investigating another project, namely adding pg_hba.conf support to pgbouncer, I ran into a stumbling block others probably will, too: the hba code

Re: [HACKERS] Parallel Seq Scan

2015-01-28 Thread Jim Nasby
On 1/28/15 9:56 AM, Stephen Frost wrote: * Robert Haas (robertmh...@gmail.com) wrote: On Wed, Jan 28, 2015 at 10:40 AM, Tom Lane t...@sss.pgh.pa.us wrote: I thought the proposal to chunk on the basis of each worker processes one 1GB-sized segment should work all right. The kernel should see

Re: [HACKERS] pg_dump with both --serializable-deferrable and -j

2015-01-28 Thread Kevin Grittner
Kevin Grittner kgri...@ymail.com wrote: Having pg_dump use repeatable read transactions for the processes that import the snapshot would work fine, as long as they are reading a snapshot which was captured by a serializable read only deferrable transaction. It looks like the attached patch

[HACKERS] pg_dump issue - push useless statements REVOKE, GRANT

2015-01-28 Thread Pavel Stehule
Hi I have a dump with thousands objects.I found often pattern in dump, that has not any sense. These operations has zero sense, but it decrease a database restore. It is expected behave? REVOKE ALL ON TABLE zobjrozp FROM PUBLIC; REVOKE ALL ON TABLE zobjrozp FROM sitkhaso; GRANT ALL ON TABLE

Re: [HACKERS] Make hba available to client code

2015-01-28 Thread Tom Lane
David Fetter da...@fetter.org writes: While investigating another project, namely adding pg_hba.conf support to pgbouncer, I ran into a stumbling block others probably will, too: the hba code is backend-only, which means that if I were to do this as-is, I would be cooking a batch of very

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-28 Thread Jim Nasby
On 1/28/15 11:36 AM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: It's not clear to me how we should represent a unicode null. i.e. given a json of '[foo\ubar]', I get that we'd store the element as 'foo\x00bar', but what is the result of (jsonb '[foo\ubar')-0

Re: [HACKERS] Make hba available to client code

2015-01-28 Thread Tom Lane
David Fetter da...@fetter.org writes: On Wed, Jan 28, 2015 at 04:10:42PM -0500, Tom Lane wrote: How exactly would exporting those functions help anything client-side? Right now, pgbouncer, and aspirational things like it--other connection poolers, maybe distributed transaction managers,

Re: [HACKERS] pg_dump with both --serializable-deferrable and -j

2015-01-28 Thread Tom Lane
Kevin Grittner kgri...@ymail.com writes: Kevin Grittner kgri...@ymail.com wrote: Having pg_dump use repeatable read transactions for the processes that import the snapshot would work fine, as long as they are reading a snapshot which was captured by a serializable read only deferrable

Re: [HACKERS] pg_dump with both --serializable-deferrable and -j

2015-01-28 Thread Kevin Grittner
Andres Freund and...@2ndquadrant.com wrote: On 2015-01-28 15:32:15 +, Kevin Grittner wrote: Andres Freund and...@2ndquadrant.com wrote: ISTM that the check is just overzelous and/or needs to be moved into ImportSnapshot(). There it then could be made to check if the exporting xact was

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2015-01-28 Thread Peter Geoghegan
On Mon, Jan 26, 2015 at 11:21 PM, Andreas Karlsson andr...@proxel.se wrote: Do you also think the SQL functions should be named numeric_int128_sum, numeric_int128_avg, etc? Some quick review comments. These apply to int128-agg-v5.patch. * Why is there no declaration of the function

[HACKERS] Make hba available to client code

2015-01-28 Thread David Fetter
Folks, While investigating another project, namely adding pg_hba.conf support to pgbouncer, I ran into a stumbling block others probably will, too: the hba code is backend-only, which means that if I were to do this as-is, I would be cooking a batch of very unappetizing copypasta. I'm allergic

Re: [HACKERS] PATCH: decreasing memory needlessly consumed by array_agg

2015-01-28 Thread Tomas Vondra
On 28.1.2015 21:28, Tom Lane wrote: Jeff Davis pg...@j-davis.com writes: On Tue, Jan 20, 2015 at 6:44 AM, Tom Lane t...@sss.pgh.pa.us wrote: Uh, sorry, I've not been paying any attention to this thread for awhile. What's the remaining questions at issue? This patch is trying to improve the

Re: [HACKERS] Make hba available to client code

2015-01-28 Thread David Fetter
On Wed, Jan 28, 2015 at 04:10:42PM -0500, Tom Lane wrote: David Fetter da...@fetter.org writes: While investigating another project, namely adding pg_hba.conf support to pgbouncer, I ran into a stumbling block others probably will, too: the hba code is backend-only, which means that if I

Re: [HACKERS] File based Incremental backup v7

2015-01-28 Thread Giuseppe Broccolo
Hi Marco, 2015-01-27 19:04 GMT+01:00 Marco Nenciarini marco.nenciar...@2ndquadrant.it : I've done some test and it looks like that FSM nodes always have InvalidXLogRecPtr as LSN. Ive updated the patch to always include files if all their pages have LSN == InvalidXLogRecPtr Updated patch

Re: [HACKERS] Parallel Seq Scan

2015-01-28 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: The problem here, as I see it, is that we're flying blind. If there's just one spindle, I think it's got to be right to read the relation sequentially. But if there are multiple spindles, it might not be, but it seems hard to predict what we should

Re: [HACKERS] Parallel Seq Scan

2015-01-28 Thread Robert Haas
On Wed, Jan 28, 2015 at 10:40 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: The problem here, as I see it, is that we're flying blind. If there's just one spindle, I think it's got to be right to read the relation sequentially. But if there are multiple

Re: [HACKERS] Parallel Seq Scan

2015-01-28 Thread Amit Kapila
On Wed, Jan 28, 2015 at 7:46 AM, Robert Haas robertmh...@gmail.com wrote: All that aside, I still can't account for the numbers you are seeing. When I run with your patch and what I think is your test case, I get different (slower) numbers. And even if we've got 6 drives cranking along at

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

2015-01-28 Thread Robert Haas
On Wed, Jan 28, 2015 at 10:35 AM, Tom Lane t...@sss.pgh.pa.us wrote: Andres Freund and...@2ndquadrant.com writes: On 2015-01-26 21:13:31 -0500, Robert Haas wrote: So maybe we should also do something like what LWLocks do, and make a union between the actual structure and an appropriate array

Re: [HACKERS] pg_dump with both --serializable-deferrable and -j

2015-01-28 Thread Andres Freund
On 2015-01-28 14:54:15 +, Kevin Grittner wrote: Kevin Grittner kgri...@ymail.com wrote: Alexander Korotkov aekorot...@gmail.com wrote: Could we start snapshot-importing transaction with repeatable read isolation level? You can if you don't use the option which specifies that you

Re: [HACKERS] pg_dump with both --serializable-deferrable and -j

2015-01-28 Thread Kevin Grittner
Andres Freund and...@2ndquadrant.com wrote: Kevin Grittner kgri...@ymail.com wrote: Alexander Korotkov aekorot...@gmail.com wrote: Could we start snapshot-importing transaction with repeatable read isolation level? If you are talking about having pg_dump acquire a safe snapshot and have

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

2015-01-28 Thread Andres Freund
On 2015-01-28 10:35:28 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2015-01-26 21:13:31 -0500, Robert Haas wrote: So maybe we should also do something like what LWLocks do, and make a union between the actual structure and an appropriate array of padding bytes -

Re: [HACKERS] Parallel Seq Scan

2015-01-28 Thread Stephen Frost
* Stephen Frost (sfr...@snowman.net) wrote: Such i/o systems do exist, but a single RAID5 group over spinning rust with a simple filter isn't going to cut it with a modern CPU- we're just too darn efficient to end up i/o bound in that case. err, to *not* end up i/o bound. Thanks,

Re: [HACKERS] pg_dump with both --serializable-deferrable and -j

2015-01-28 Thread Kevin Grittner
Kevin Grittner kgri...@ymail.com wrote: Alexander Korotkov aekorot...@gmail.com wrote: Could we start snapshot-importing transaction with repeatable read isolation level? You can if you don't use the option which specifies that you want serializable behavior. Why specify

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

2015-01-28 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2015-01-26 21:13:31 -0500, Robert Haas wrote: So maybe we should also do something like what LWLocks do, and make a union between the actual structure and an appropriate array of padding bytes - say either 64 or 128 of them. Hm. That's a bit

Re: [HACKERS] pg_upgrade and rsync

2015-01-28 Thread Josh Berkus
Bruce, Stephen, etc.: So, I did a test trial of this and it seems like it didn't solve the issue of huge rsyncs. That is, the only reason to do this whole business via rsync, instead of doing a new basebackup of each replica, is to cut down on data transfer time by not resyncing the data from

Re: [HACKERS] PATCH: decreasing memory needlessly consumed by array_agg

2015-01-28 Thread Tomas Vondra
Hi, attached is v9 of the patch, modified along the lines of Tom's comments: 1) uses alen=64 for cases with private context, 8 otherwise 2) reverts removal of element_type from initArrayResultArr() When element_type=InvalidOid is passed to initArrayResultArr, it performs lookup using

Re: [HACKERS] pg_upgrade and rsync

2015-01-28 Thread Josh Berkus
On 01/28/2015 02:10 PM, Josh Berkus wrote: So 390MB were transferred out of a possible 474MB. That certainly seems like we're still transferring the majority of the data, even though I verified that the hard links are being sent as hard links. No? Looks like the majority of that was pg_xlog.

Re: [HACKERS] Providing catalog view to pg_hba.conf file - Patch submission

2015-01-28 Thread Jim Nasby
On 1/28/15 12:46 AM, Haribabu Kommi wrote: Also, what happens if someone reloads the config in the middle of running the SRF? hba entries are reloaded only in postmaster process, not in every backend. So there shouldn't be any problem with config file reload. Am i missing something? Ahh, good

Re: [HACKERS] compiler warnings in copy.c

2015-01-28 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: On 2015-01-28 15:05:11 -0500, Stephen Frost wrote: Also, you seem to have pushed these commits with a date more than two weeks in the past. Please don't do that! Oh, wow, sorry about that. I had expected a rebase to update the date.

Re: [HACKERS] Possible typo in create_policy.sgml

2015-01-28 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Fri, Jan 9, 2015 at 3:46 PM, Stephen Frost sfr...@snowman.net wrote: A policy permits SELECT, INSERT, UPDATE or DELETE commands to access rows in a table that has row level security enabled. Access to existing table rows is granted

Re: [HACKERS] Small bug on CREATE EXTENSION pgq...

2015-01-28 Thread Stephen Frost
* David Johnston (david.g.johns...@gmail.com) wrote: On Wednesday, January 28, 2015, Stephen Frost sfr...@snowman.net wrote: Ehh.. Shouldn't we try to take a bit more care that we reset things after a CREATE EXTENSION is run? Not really sure how much effort we want to put into it, but I

Re: [HACKERS] Small bug on CREATE EXTENSION pgq...

2015-01-28 Thread David Johnston
On Wednesday, January 28, 2015, Stephen Frost sfr...@snowman.net wrote: * David G Johnston (david.g.johns...@gmail.com javascript:;) wrote: Jerry Sievers-3 wrote Hackers; I noticed this trying to import a large pg_dump file with warnings supressed. It seems loading pgq sets

Re: [HACKERS] Possible typo in create_policy.sgml

2015-01-28 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Wed, Jan 7, 2015 at 3:06 PM, Stephen Frost sfr...@snowman.net wrote: If I'm following correctly, Peter's specifically talking about: [ USING ( replaceable class=parameterexpression/replaceable ) ] [ WITH CHECK ( replaceable

Re: [HACKERS] Possible typo in create_policy.sgml

2015-01-28 Thread Stephen Frost
Dean, * Dean Rasheed (dean.a.rash...@gmail.com) wrote: [There's also a typo further down -- filter out the records which are visible, should be not visible] I agree, that's not really worded quite right. I've reworded this along the lines of what you suggested (though not exactly- if you get

Re: [HACKERS] Small bug on CREATE EXTENSION pgq...

2015-01-28 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: * David Johnston (david.g.johns...@gmail.com) wrote: Fair enough but reset to what? I don't know the internal mechanics but if the session default is warning and a local change sets it to notice then an unconditional reset would not get us back to the

Re: [HACKERS] Parallel Seq Scan

2015-01-28 Thread Stephen Frost
Jim, * Jim Nasby (jim.na...@bluetreble.com) wrote: On 1/28/15 9:56 AM, Stephen Frost wrote: Such i/o systems do exist, but a single RAID5 group over spinning rust with a simple filter isn't going to cut it with a modern CPU- we're just too darn efficient to end up i/o bound in that case. A

Re: [HACKERS] Proposal: two new role attributes and/or capabilities?

2015-01-28 Thread Stephen Frost
Jim, * Jim Nasby (jim.na...@bluetreble.com) wrote: On 12/23/14 12:52 PM, Stephen Frost wrote: Autovacuum can certainly run vacuum/analyze on a few tables every 12 hours, so I'm not really following where you see autovacuum being unable to cope. I agree that there*are* such cases, but

[HACKERS] Small bug on CREATE EXTENSION pgq...

2015-01-28 Thread Jerry Sievers
Hackers; I noticed this trying to import a large pg_dump file with warnings supressed. It seems loading pgq sets client_min_messages to warning and leaves it this way which defeats an attempt to change the setting prior and have it stick. I tested with several other extensions in same DB and

Re: [HACKERS] Small bug on CREATE EXTENSION pgq...

2015-01-28 Thread David G Johnston
Jerry Sievers-3 wrote Hackers; I noticed this trying to import a large pg_dump file with warnings supressed. It seems loading pgq sets client_min_messages to warning and leaves it this way which defeats an attempt to change the setting prior and have it stick. I tested with several other

Re: [HACKERS] Possible typo in create_policy.sgml

2015-01-28 Thread Stephen Frost
Peter, * Peter Geoghegan (p...@heroku.com) wrote: I also don't see this behavior documented (this is from process_policies()): [...] But is that really the right place for it? Does it not equally well apply to FOR UPDATE policies, that can on their own have both barriers quals and WITH CHECK

Re: [HACKERS] Small bug on CREATE EXTENSION pgq...

2015-01-28 Thread Stephen Frost
* David G Johnston (david.g.johns...@gmail.com) wrote: Jerry Sievers-3 wrote Hackers; I noticed this trying to import a large pg_dump file with warnings supressed. It seems loading pgq sets client_min_messages to warning and leaves it this way which defeats an attempt to change the

Re: [HACKERS] Parallel Seq Scan

2015-01-28 Thread Robert Haas
On Wed, Jan 28, 2015 at 8:27 PM, Stephen Frost sfr...@snowman.net wrote: I feel like one of us is misunderstanding the numbers, which is probably in part because they're a bit piecemeal over email, but the seqscan speed in this case looks pretty close to dd performance for this particular

Re: [HACKERS] Missing updates at few places for row level security

2015-01-28 Thread Stephen Frost
Amit, * Amit Kapila (amit.kapil...@gmail.com) wrote: There is a new column added in pg_authid (rolbypassrls) and the updation for same is missed in below places: a. System catalog page for pg_authid http://www.postgresql.org/docs/devel/static/catalog-pg-authid.html b. Do we want to add

Re: [HACKERS] pg_upgrade and rsync

2015-01-28 Thread Josh Berkus
On 01/28/2015 02:28 PM, Josh Berkus wrote: On 01/28/2015 02:10 PM, Josh Berkus wrote: So 390MB were transferred out of a possible 474MB. That certainly seems like we're still transferring the majority of the data, even though I verified that the hard links are being sent as hard links. No?

Re: [HACKERS] pg_upgrade and rsync

2015-01-28 Thread Stephen Frost
* Josh Berkus (j...@agliodbs.com) wrote: On 01/28/2015 02:28 PM, Josh Berkus wrote: On 01/28/2015 02:10 PM, Josh Berkus wrote: So 390MB were transferred out of a possible 474MB. That certainly seems like we're still transferring the majority of the data, even though I verified that the

[HACKERS] Exposing the stats snapshot timestamp to SQL

2015-01-28 Thread Matt Kelly
In a previous thread Tom Lane said: (I'm also wondering if it'd make sense to expose the stats timestamp as a callable function, so that the case could be dealt with programmatically as well. But that's future-feature territory.)

Re: [HACKERS] pg_upgrade and rsync

2015-01-28 Thread Josh Berkus
So, for my 2c, I'm on the fence about it. On the one hand, I agree, it's a bit of a complex process to get right. On the other hand, it's far better if we put something out there along the lines of if you really want to, this is how to do it than having folks try to fumble through to find

Re: [HACKERS] Parallel Seq Scan

2015-01-28 Thread Daniel Bausch
Robert Haas robertmh...@gmail.com writes: On Wed, Jan 28, 2015 at 9:12 AM, Thom Brown t...@linux.com wrote: On 28 January 2015 at 14:03, Robert Haas robertmh...@gmail.com wrote: The problem here, as I see it, is that we're flying blind. If there's just one spindle, I think it's got to be

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-28 Thread Jeff Janes
On Wed, Jan 28, 2015 at 1:13 PM, Jim Nasby jim.na...@bluetreble.com wrote: My $0.01: While I sympathize with Noah's sentiments, the only thing that makes sense to me is that a JSON text field is treated the same way as we treat text. Right now, that means NUL is not allowed, period. If no

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2015-01-28 Thread Pavel Stehule
Hi I am testing this feature on relative complex schema (38619 tables in db) and I got deadlock [pavel@localhost bin]$ /usr/local/pgsql/bin/vacuumdb test2 -fz -j 4 vacuumdb: vacuuming database test2 vacuumdb: vacuuming of database test2 failed: ERROR: deadlock detected DETAIL: Process 24689

Re: [HACKERS] PQgetssl() and alternative SSL implementations

2015-01-28 Thread Robert Haas
On Wed, Jan 28, 2015 at 10:13 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Here's a patch to implement the above scheme. It adds four functions to libpq, to interrogate the SSL status: int PQsslInUse(const PGconn *conn) Returns true (1) if the connection uses SSL, false (0) if not.

Re: [HACKERS] Sequence Access Method WIP

2015-01-28 Thread Petr Jelinek
On 28/01/15 18:09, Heikki Linnakangas wrote: On 01/23/2015 02:34 AM, Petr Jelinek wrote: On 22/01/15 17:02, Petr Jelinek wrote: The new version (the one that is not submitted yet) of gapless sequence is way more ugly and probably not best example either but does guarantee gaplessness (it

Re: [HACKERS] PQgetssl() and alternative SSL implementations

2015-01-28 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: Right, that was the idea. I wanted it to include the word OpenSSL, to make it clear in the callers that it's specific to OpenSSL. And SSL, because that's the name of the struct. I agree it looks silly, though. One idea is to have two

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-28 Thread Tom Lane
David G Johnston david.g.johns...@gmail.com writes: Am I missing something or has there been no consideration in this forbid plan on whether users will be able to retrieve, even if partially incorrectly, any jsonb data that has already been stored? Data that's already been stored would look

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

2015-01-28 Thread Tom Lane
Merlin Moncure mmonc...@gmail.com writes: ...hm, I spoke to soon. So I deleted everything, and booted up a new instance 9.4 vanilla with asserts on and took no other action. Applying the script with no data activity fails an assertion every single time: TRAP: FailedAssertion(!(flags

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

2015-01-28 Thread Andres Freund
On 2015-01-28 13:38:51 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: I personally still think that a comment above sbufdesc's definition would be sufficient for now. But whatever. I'll enforce 64byte padding on 64bit platforms, and do nothing on 32bit platforms.

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-28 Thread David G Johnston
Tom Lane-2 wrote Andrew Dunstan lt; andrew@ gt; writes: On 01/27/2015 02:28 PM, Tom Lane wrote: Well, we can either fix it now or suffer with a broken representation forever. I'm not wedded to the exact solution I described, but I think we'll regret it if we don't change the

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

2015-01-28 Thread Andres Freund
On 2015-01-28 17:08:46 +0100, Andres Freund wrote: I just have no idea whether it'd be beneficial to use more space on 32bit to pad the individual entries. Since this mostly is beneficial on multi-socket, highly concurrent workloads, I doubt it really matter. I personally still think that a

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

2015-01-28 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: I personally still think that a comment above sbufdesc's definition would be sufficient for now. But whatever. I'll enforce 64byte padding on 64bit platforms, and do nothing on 32bit platforms. Patch doing that attached. Surely the sizeof() in

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

2015-01-28 Thread Merlin Moncure
On Wed, Jan 28, 2015 at 8:05 AM, Merlin Moncure mmonc...@gmail.com wrote: On Thu, Jan 22, 2015 at 3:50 PM, Merlin Moncure mmonc...@gmail.com wrote: I still haven't categorically ruled out pl/sh yet; that's something to keep in mind. Well, after bisection proved not to be fruitful, I replaced

Re: [HACKERS] pg_dump with both --serializable-deferrable and -j

2015-01-28 Thread Kevin Grittner
Alexander Korotkov aekorot...@gmail.com wrote: when pg_dump is run with both --serializable-deferrable and -j options to pg_dump, it returns errors: pg_dump: [archiver (db)] query failed: ERROR: a snapshot-importing transaction must not be READ ONLY DEFERRABLE pg_dump: [archiver (db)]

Re: [HACKERS] PQgetssl() and alternative SSL implementations

2015-01-28 Thread Heikki Linnakangas
On 08/20/2014 12:58 AM, Heikki Linnakangas wrote: On 08/19/2014 10:31 PM, Robert Haas wrote: On Tue, Aug 19, 2014 at 3:16 PM, Magnus Hagander mag...@hagander.net wrote: On Tue, Aug 19, 2014 at 9:09 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert's got a point though: there is always going to be

Re: [HACKERS] Parallel Seq Scan

2015-01-28 Thread Robert Haas
On Wed, Jan 28, 2015 at 9:12 AM, Thom Brown t...@linux.com wrote: On 28 January 2015 at 14:03, Robert Haas robertmh...@gmail.com wrote: The problem here, as I see it, is that we're flying blind. If there's just one spindle, I think it's got to be right to read the relation sequentially. But

Re: [HACKERS] hung backends stuck in spinlock heavy endless loop

2015-01-28 Thread Merlin Moncure
On Wed, Jan 28, 2015 at 12:47 PM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: ...hm, I spoke to soon. So I deleted everything, and booted up a new instance 9.4 vanilla with asserts on and took no other action. Applying the script with no data activity fails an

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

2015-01-28 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2015-01-28 13:38:51 -0500, Tom Lane wrote: #define BUFFERDESC_PADDED_SIZE (SIZEOF_VOID_P == 8 ? 64 : 32) Hm, did you intentionally put a 32in there or was that just the logical continuation of 64? Because there's no way it'll ever fit into

Re: [HACKERS] Providing catalog view to pg_hba.conf file - Patch submission

2015-01-28 Thread Fabrízio de Royes Mello
On Wed, Jan 28, 2015 at 4:46 AM, Haribabu Kommi kommi.harib...@gmail.com wrote: On Wed, Jan 28, 2015 at 9:47 AM, Jim Nasby jim.na...@bluetreble.com wrote: On 1/27/15 1:04 AM, Haribabu Kommi wrote: Here I attached the latest version of the patch. I will add this patch to the next

Re: [HACKERS] Providing catalog view to pg_hba.conf file - Patch submission

2015-01-28 Thread Tom Lane
=?UTF-8?Q?Fabr=C3=ADzio_de_Royes_Mello?= fabriziome...@gmail.com writes: But I'm thinking about this patch and would not be interesting to have a FDW to manipulate the hba file? Imagine if we are able to manipulate the HBA file using INSERT/UPDATE/DELETE. Since the HBA file is fundamentally

[HACKERS] compiler warnings in copy.c

2015-01-28 Thread Robert Haas
My compiler is unhappy with the latest changes to copy.c: gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -g -O2 -Wall -Werror -I../../../src/include

Re: [HACKERS] compiler warnings in copy.c

2015-01-28 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: My compiler is unhappy with the latest changes to copy.c: gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv

Re: [HACKERS] compiler warnings in copy.c

2015-01-28 Thread Andres Freund
On 2015-01-28 15:05:11 -0500, Stephen Frost wrote: Also, you seem to have pushed these commits with a date more than two weeks in the past. Please don't do that! Oh, wow, sorry about that. I had expected a rebase to update the date. It updates the committer, but not the author date. Use