Re: [HACKERS] [DESIGN] ParallelAppend

2015-07-27 Thread Amit Langote
KaiGai-san, On 2015-07-27 PM 11:07, Kouhei Kaigai wrote: Append -- Funnel -- PartialSeqScan on rel1 (num_workers = 4) -- Funnel -- PartialSeqScan on rel2 (num_workers = 8) -- SeqScan on rel3 shall be rewritten to Funnel -- PartialSeqScan on rel1

Re: [HACKERS] [DESIGN] ParallelAppend

2015-07-27 Thread Amit Kapila
On Sun, Jul 26, 2015 at 8:43 AM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: Hello, I'm recently working/investigating on ParallelAppend feature towards the next commit fest. Below is my design proposal. 1. Concept -- Its concept is quite simple anybody might consider more than

[HACKERS] 9.5a1 BUG FIX: pgbench negative latencies

2015-07-27 Thread Fabien COELHO
Under 9.5a1 pgbench -r negative latencies are reported on meta commands, probably as an oversight of 84f0ea3f. This patch ensures that now is reset on each loop inside doCustom. -- Fabien.diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c index 2c3e365..cce67e8 100644 ---

Re: [HACKERS] [DESIGN] ParallelAppend

2015-07-27 Thread Kouhei Kaigai
On Sun, Jul 26, 2015 at 8:43 AM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: Hello, I'm recently working/investigating on ParallelAppend feature towards the next commit fest. Below is my design proposal. 1. Concept -- Its concept is quite simple anybody might consider more

Re: [HACKERS] raw output from copy

2015-07-27 Thread Pavel Stehule
2015-07-27 10:41 GMT+02:00 Heikki Linnakangas hlinn...@iki.fi: On 07/27/2015 06:55 AM, Craig Ringer wrote: On 7 July 2015 at 14:32, Pavel Stehule pavel.steh...@gmail.com wrote: Hi previous patch was broken, and buggy Here is new version with fixed upload and more tests I routinely see

Re: [HACKERS] Proposal for CSN based snapshots

2015-07-27 Thread Alexander Korotkov
On Sat, Jul 25, 2015 at 11:39 AM, Simon Riggs si...@2ndquadrant.com wrote: On 24 July 2015 at 19:21, Robert Haas robertmh...@gmail.com wrote: On Fri, Jul 24, 2015 at 1:00 PM, Simon Riggs si...@2ndquadrant.com wrote: It depends on the exact design we use to get that. Certainly we do not

Re: [HACKERS] REVOKE [ADMIN OPTION FOR] ROLE

2015-07-27 Thread Egor Rogov
On Thu, Jul 23, 2015 at 8:30 AM, Egor Rogov e.ro...@postgrespro.ru wrote: So, the question: is it a documentation bug (as it seems to me), code bug, or I missed something? Your analysis looks right to me, but I don't know whether the code or the documentation should be changed. This claim

Re: [HACKERS] creating extension including dependencies

2015-07-27 Thread Michael Paquier
On Sun, Jul 26, 2015 at 1:01 AM, Petr Jelinek wrote: Yes that's what I meant by the change of checking order in the explanation above. I did that because I thought code would be more complicated otherwise, but apparently I was stupid... +In case the extension specifies schema in its

Re: [HACKERS] [DESIGN] ParallelAppend

2015-07-27 Thread Kouhei Kaigai
Hello, can I ask some questions? I suppose we can take this as the analog of ParalleSeqScan. I can see not so distinction between Append(ParalleSeqScan) and ParallelAppend(SeqScan). What difference is there between them? Append does not start to execute the second or later node until first

[HACKERS] Buildfarm TAP testing is useless as currently implemented

2015-07-27 Thread Tom Lane
I challenge anybody to figure out what happened here: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hornetdt=2015-07-27%2010%3A25%3A17 or here: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hamsterdt=2015-07-04%2016%3A00%3A23 or here:

Re: [HACKERS] Failing assertions in indxpath.c, placeholder.c and brin_minmax.c

2015-07-27 Thread Tom Lane
Andreas Seltenreich seltenre...@gmx.de writes: Tom Lane writes: I've fixed the first two of these --- thanks for the report! I let sqlsmith run during the night, and it did no longer trigger the first two. During roughly a million random queries it triggered the already mentioned brin one

Re: [HACKERS] Microvacuum for gist. Question about GISTPageOpaqueData flag

2015-07-27 Thread Teodor Sigaev
I need an advice, what would be better: - to add new flag like F_HAS_GARBAGE, - or to delete all mentions of F_TUPLES_DELETED and use it in gist microvacuum. According to commit message: commit 2effb72e682a7dbdc9a8a60a80c22ec1fa9d8079 Author: Heikki Linnakangas heikki.linnakan...@iki.fi Date:

[HACKERS] Microvacuum for gist. Question about GISTPageOpaqueData flag

2015-07-27 Thread Anastasia Lubennikova
Hi, I'm working on microvacuum for gist access method. Briefly microvacuum includes two steps: 1. When search tells us that the tuple is invisible to all transactions it is marked LP_DEAD and page is marked as has dead tuples, 2. Then, when insert touches full page which has dead tuples it calls

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-07-27 Thread Fabien COELHO
Attatched is the revised version of this patch. The first patch is not changed from before. The second is fixed a kind of bug. Ths third is the new one to allow backslash continuation for backslash commands. Ah, thanks:-) Would you consider adding the patch to the next commitfest? I may

Re: [HACKERS] All-zero page in GIN index causes assertion failure

2015-07-27 Thread Heikki Linnakangas
On 07/23/2015 07:43 PM, Jeff Janes wrote: On Mon, Jul 20, 2015 at 1:23 PM, Heikki Linnakangas hlinn...@iki.fi wrote: I came up with the attached, for GIN and SP-GiST. Instead of WAL-logging the page initialization in SP-GiST vacuum, I changed it so that it simply leaves the page as all-zeroes,

Re: [HACKERS] Buildfarm TAP testing is useless as currently implemented

2015-07-27 Thread Andrew Dunstan
On 07/27/2015 10:06 AM, Tom Lane wrote: I challenge anybody to figure out what happened here: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hornetdt=2015-07-27%2010%3A25%3A17 or here: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hamsterdt=2015-07-04%2016%3A00%3A23 or here:

Re: [HACKERS] Failing assertions in indxpath.c, placeholder.c and brin_minmax.c

2015-07-27 Thread Alvaro Herrera
Tom Lane wrote: Bottom line is that somebody failed to consider the possibility of a null comparison value reaching the BRIN index lookup machinery. The code stanza that's failing supposes that only IS NULL or IS NOT NULL tests could have SK_ISNULL set, but that's just wrong. Hm, okay, will

Re: [HACKERS] Restore-reliability mode

2015-07-27 Thread Alvaro Herrera
Noah Misch wrote: On Thu, Jul 23, 2015 at 04:53:49PM -0300, Alvaro Herrera wrote: Peter Geoghegan wrote: On Sat, Jun 6, 2015 at 12:58 PM, Noah Misch n...@leadboat.com wrote: - Call VALGRIND_MAKE_MEM_NOACCESS() on a shared buffer when its local pin count falls to zero.

Re: [HACKERS] Microvacuum for gist. Question about GISTPageOpaqueData flag

2015-07-27 Thread Anastasia Lubennikova
2015-07-27 20:05 GMT+04:00 Heikki Linnakangas hlinn...@iki.fi: On 07/27/2015 06:46 PM, Teodor Sigaev wrote: I need an advice, what would be better: - to add new flag like F_HAS_GARBAGE, - or to delete all mentions of F_TUPLES_DELETED and use it in gist microvacuum. According to commit

Re: [HACKERS] Buildfarm failure from overly noisy warning message

2015-07-27 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: Kevin Grittner kgri...@ymail.com writes: I think a LOG entry when an autovacuum process is actually canceled has value just in case it is happening on a particular table so frequently that the table starts to bloat. I see no reason to log anything if there

Re: [HACKERS] Reduce ProcArrayLock contention

2015-07-27 Thread Robert Haas
On Sat, Jul 25, 2015 at 12:42 AM, Amit Kapila amit.kapil...@gmail.com wrote: I thought that internal API will automatically take care of it, example for msvc it uses _InterlockedCompareExchange64 which if doesn't work on 32-bit systems or is not defined, then we have to use 32-bit version, but

Re: [HACKERS] Buildfarm TAP testing is useless as currently implemented

2015-07-27 Thread Heikki Linnakangas
On 07/27/2015 05:06 PM, Tom Lane wrote: I challenge anybody to figure out what happened here: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hornetdt=2015-07-27%2010%3A25%3A17 or here: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hamsterdt=2015-07-04%2016%3A00%3A23 or here:

Re: [HACKERS] All-zero page in GIN index causes assertion failure

2015-07-27 Thread Amit Langote
On Tue, Jul 28, 2015 at 12:21 AM, Heikki Linnakangas hlinn...@iki.fi wrote: Thanks, I've pushed this, as well a fix to similar failure from B-tree vacuum that Amit Langote reported in the other thread. Thanks Heikki and sorry I didn't notice this new thread. Regards, Amit -- Sent via

Re: [HACKERS] optimizing vacuum truncation scans

2015-07-27 Thread Robert Haas
On Sat, Jul 25, 2015 at 4:46 AM, Simon Riggs si...@2ndquadrant.com wrote: On 22 July 2015 at 14:59, Robert Haas robertmh...@gmail.com wrote: Urgh. So if we do this, that forever precludes having HOT pruning set the all-visible bit. What is the reason why we don't do that already? Surely its

Re: [HACKERS] Microvacuum for gist. Question about GISTPageOpaqueData flag

2015-07-27 Thread Heikki Linnakangas
On 07/27/2015 06:46 PM, Teodor Sigaev wrote: I need an advice, what would be better: - to add new flag like F_HAS_GARBAGE, - or to delete all mentions of F_TUPLES_DELETED and use it in gist microvacuum. According to commit message: commit 2effb72e682a7dbdc9a8a60a80c22ec1fa9d8079 Author: Heikki

Re: [HACKERS] Several memory leaks for pg_rewind caused by missing PQclear calls

2015-07-27 Thread Heikki Linnakangas
On 07/23/2015 05:08 PM, Michael Paquier wrote: Hi all, After a run of valgrind on pg_rewind, I found a couple of code paths missing some PQclear calls after running a query. Attached is a patch to fix all those leaks. Fixed, thanks. - Heikki -- Sent via pgsql-hackers mailing list

Re: [HACKERS] A little RLS oversight?

2015-07-27 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/27/2015 10:03 AM, Joe Conway wrote: On 07/26/2015 07:59 AM, Joe Conway wrote: On 07/26/2015 07:19 AM, Dean Rasheed wrote: Attached is an updated patch (still needs some docs for the functions). Thanks for that. I'll add the docs.

Re: [HACKERS] A little RLS oversight?

2015-07-27 Thread Joe Conway
On 07/26/2015 07:59 AM, Joe Conway wrote: On 07/26/2015 07:19 AM, Dean Rasheed wrote: Attached is an updated patch (still needs some docs for the functions). Thanks for that. I'll add the docs. Documentation added. Also added comment to check_enable_rls about passing InvalidOid versus

Re: [HACKERS] Reduce ProcArrayLock contention

2015-07-27 Thread Pavan Deolasee
On Sat, Jul 25, 2015 at 10:12 AM, Amit Kapila amit.kapil...@gmail.com wrote: Numbers look impressive and definitely shows that the idea is worth pursuing. I tried patch on my laptop. Unfortunately, at least for 4 and 8 clients, I did not see any improvement. I can't help in this

[HACKERS] Alpha 2 wrapping August 3

2015-07-27 Thread Josh Berkus
... so please get those fixes/overhauls in in the next week. Thanks! -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] proposal: multiple psql option -c

2015-07-27 Thread Robert Haas
On Mon, Jul 27, 2015 at 2:37 PM, Pavel Stehule pavel.steh...@gmail.com wrote: 2015-07-27 20:32 GMT+02:00 Robert Haas robertmh...@gmail.com: On Sat, Jul 25, 2015 at 5:27 AM, Pavel Stehule pavel.steh...@gmail.com wrote: It will be nice side effect, but my primary problem was a impossibility

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2015-07-27 Thread Robert Haas
On Mon, Jul 27, 2015 at 2:43 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Robert Haas wrote: On Mon, Jul 27, 2015 at 2:32 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I think this is already possible, is it not? You just have to look for an identically-identified pg_locks

Re: [HACKERS] proposal: multiple psql option -c

2015-07-27 Thread Robert Haas
On Sat, Jul 25, 2015 at 5:27 AM, Pavel Stehule pavel.steh...@gmail.com wrote: It will be nice side effect, but my primary problem was a impossibility to combine VACUUM and any other statement to one simple psql call. Seems like you can do that easily enough: [rhaas pgsql]$ (echo 'SELECT 1;';

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2015-07-27 Thread Robert Haas
On Mon, Jul 27, 2015 at 2:32 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: David Rowley wrote: I've not looked into the feasibility of it, but if it were also possible to have a waiting_for column which would store the process ID of the process that's holding a lock that this process is

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2015-07-27 Thread Alvaro Herrera
Robert Haas wrote: On Mon, Jul 27, 2015 at 2:32 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I think this is already possible, is it not? You just have to look for an identically-identified pg_locks entry with granted=true. That gives you a PID and vxid/xid. You can self-join

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2015-07-27 Thread Alvaro Herrera
David Rowley wrote: I've not looked into the feasibility of it, but if it were also possible to have a waiting_for column which would store the process ID of the process that's holding a lock that this process is waiting on, then it would be possible for some smart guy to write some code

Re: [HACKERS] REVOKE [ADMIN OPTION FOR] ROLE

2015-07-27 Thread Stephen Frost
Egor, * Egor Rogov (e.ro...@postgrespro.ru) wrote: On Thu, Jul 23, 2015 at 8:30 AM, Egor Rogov e.ro...@postgrespro.ru wrote: So, the question: is it a documentation bug (as it seems to me), code bug, or I missed something? Your analysis looks right to me, but I don't know whether the code or

Re: [HACKERS] spgist recovery assertion failure

2015-07-27 Thread Heikki Linnakangas
On 07/27/2015 04:24 PM, Michael Paquier wrote: On Mon, Jul 27, 2015 at 2:33 PM, Piotr Stefaniak postg...@piotr-stefaniak.me wrote: On 07/27/2015 07:19 AM, Michael Paquier wrote: On Mon, Jul 27, 2015 at 2:00 PM, Noah Misch n...@leadboat.com wrote: When I caused a crash during the

Re: [HACKERS] more RLS oversights

2015-07-27 Thread Alvaro Herrera
Joe Conway wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/03/2015 10:03 AM, Noah Misch wrote: (2) CreatePolicy() and AlterPolicy() omit to create a pg_shdepend entry for each role in the TO clause. Test case: Please see the attached patch. Note that I used

Re: [HACKERS] proposal: multiple psql option -c

2015-07-27 Thread Andrew Dunstan
On 07/27/2015 02:53 PM, Pavel Stehule wrote: I am trying to run parallel execution psql -At -c select datname from pg_database postgres | xargs -n 1 -P 3 psql -c select current_database() I don't think it's going to be a hugely important feature, but I don't see a problem with

Re: [HACKERS] proposal: multiple psql option -c

2015-07-27 Thread Pavel Stehule
2015-07-27 20:32 GMT+02:00 Robert Haas robertmh...@gmail.com: On Sat, Jul 25, 2015 at 5:27 AM, Pavel Stehule pavel.steh...@gmail.com wrote: It will be nice side effect, but my primary problem was a impossibility to combine VACUUM and any other statement to one simple psql call. Seems

Re: [HACKERS] anole: assorted stability problems

2015-07-27 Thread Robert Haas
On Sun, Jul 26, 2015 at 11:36 AM, Andres Freund and...@anarazel.de wrote: On 2015-07-07 13:25:24 +0200, Andres Freund wrote: So, it's starting to look good. Not exactly allowing for a lot of confidence yet, but still: http://buildfarm.postgresql.org/cgi-bin/show_history.pl?nm=anolebr=HEAD

Re: [HACKERS] proposal: multiple psql option -c

2015-07-27 Thread Pavel Stehule
2015-07-27 20:47 GMT+02:00 Robert Haas robertmh...@gmail.com: On Mon, Jul 27, 2015 at 2:37 PM, Pavel Stehule pavel.steh...@gmail.com wrote: 2015-07-27 20:32 GMT+02:00 Robert Haas robertmh...@gmail.com: On Sat, Jul 25, 2015 at 5:27 AM, Pavel Stehule pavel.steh...@gmail.com wrote: It

[HACKERS] WIP: Make timestamptz_out less slow.

2015-07-27 Thread Andres Freund
Hi, I recently once more noticed that timestamptz_out is really, really slow. To quantify that, I created a bunch of similar sized tables: CREATE TABLE tbl_timestamp AS SELECT NOW() FROM generate_series(1, 10) a, generate_series(1, 100) b; CREATE TABLE tbl_int8 AS SELECT 1::bigint FROM

Re: [HACKERS] A little RLS oversight?

2015-07-27 Thread Dean Rasheed
On 27 July 2015 at 21:58, Stephen Frost sfr...@snowman.net wrote: Dean, * Dean Rasheed (dean.a.rash...@gmail.com) wrote: On 27 July 2015 at 18:13, Joe Conway m...@joeconway.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/27/2015 10:03 AM, Joe Conway wrote: On

[HACKERS] Optimization idea: merging multiple EXISTS(...) with constraint-based join removal

2015-07-27 Thread Frédéric TERRAZZONI
I've frequently encountered queries where the WHERE clause contains a lot of EXISTS(...) subqueries linked by logical ANDs. They are typically generated queries, and the EXISTS(...) fragments are composable pieces of SQL code acting as filters. Moreover, all those filters ends up being very

Re: [HACKERS] WIP: Make timestamptz_out less slow.

2015-07-27 Thread Andres Freund
On 2015-07-27 17:31:41 -0400, Tom Lane wrote: Andres Freund and...@anarazel.de writes: So nearly all the time is spent somewhere inside the sprintf calls. Not nice. What happens if you force use of port/snprintf.c instead of glibc's version? Good question. Even worse. 15900.014 ms.

Re: [HACKERS] A little RLS oversight?

2015-07-27 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/27/2015 01:25 PM, Dean Rasheed wrote: Looks good to me, except I'm not sure about those latest changes because I don't understand the reasoning behind the logic in check_enable_rls() when row_security is set to OFF. I would expect that

Re: [HACKERS] proposal: multiple psql option -c

2015-07-27 Thread Robert Haas
On Mon, Jul 27, 2015 at 2:53 PM, Pavel Stehule pavel.steh...@gmail.com wrote: I am trying to run parallel execution psql -At -c select datname from pg_database postgres | xargs -n 1 -P 3 psql -c select current_database() Put this in a shell script called run-psql: #!/bin/bash test $# = 0

Re: [HACKERS] optimizing vacuum truncation scans

2015-07-27 Thread Simon Riggs
On 22 July 2015 at 17:11, Jeff Janes jeff.ja...@gmail.com wrote: On Wed, Jul 22, 2015 at 6:59 AM, Robert Haas robertmh...@gmail.com wrote: On Mon, Jun 29, 2015 at 1:54 AM, Jeff Janes jeff.ja...@gmail.com wrote: Attached is a patch that implements the vm scan for truncation. It introduces

Re: [HACKERS] Autonomous Transaction is back

2015-07-27 Thread Joel Jacobson
On Fri, Jul 24, 2015 at 9:39 PM, Merlin Moncure mmonc...@gmail.com wrote: On Thu, Jul 23, 2015 at 1:49 PM, Josh Berkus j...@agliodbs.com wrote: Batch Jobs: large data-manipulation tasks which need to be broken up into segments, with each segment committing separately. Example: updating 1

Re: [HACKERS] A little RLS oversight?

2015-07-27 Thread Dean Rasheed
On 27 July 2015 at 18:13, Joe Conway m...@joeconway.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/27/2015 10:03 AM, Joe Conway wrote: On 07/26/2015 07:59 AM, Joe Conway wrote: On 07/26/2015 07:19 AM, Dean Rasheed wrote: Attached is an updated patch (still needs some docs

Re: [HACKERS] A little RLS oversight?

2015-07-27 Thread Stephen Frost
Dean, * Dean Rasheed (dean.a.rash...@gmail.com) wrote: On 27 July 2015 at 18:13, Joe Conway m...@joeconway.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/27/2015 10:03 AM, Joe Conway wrote: On 07/26/2015 07:59 AM, Joe Conway wrote: On 07/26/2015 07:19 AM, Dean Rasheed

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2015-07-27 Thread Jim Nasby
On 7/27/15 1:46 PM, Robert Haas wrote: On Mon, Jul 27, 2015 at 2:43 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Robert Haas wrote: On Mon, Jul 27, 2015 at 2:32 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I think this is already possible, is it not? You just have to look for

Re: [HACKERS] Autonomous Transaction is back

2015-07-27 Thread Josh Berkus
On 07/27/2015 02:41 PM, Joel Jacobson wrote: However, we should also add a way for the caller to protect against an Autonomous Transaction in a function called by the caller. Imagine if you're the author of function X() and within X() make use of some other function Y() which has been written

Re: [HACKERS] copy.c handling for RLS is insecure

2015-07-27 Thread Stephen Frost
All, * Stephen Frost (sfr...@snowman.net) wrote: * Andres Freund (and...@anarazel.de) wrote: On 2015-07-09 01:28:28 -0400, Noah Misch wrote: - Keep the OID check, shouldn't hurt to have it What benefit is left? A bit of defense in depth. We execute user defined code in COPY

Re: [HACKERS] proposal: multiple psql option -c

2015-07-27 Thread Jim Nasby
On 7/27/15 2:57 PM, Andrew Dunstan wrote: psql -At -c select datname from pg_database postgres | xargs -n 1 -P 3 psql -c select current_database() I don't think it's going to be a hugely important feature, but I don't see a problem with creating a new option (-C seems fine) which would

Re: [HACKERS] WIP: Make timestamptz_out less slow.

2015-07-27 Thread Tom Lane
Andres Freund and...@anarazel.de writes: So nearly all the time is spent somewhere inside the sprintf calls. Not nice. What happens if you force use of port/snprintf.c instead of glibc's version? regards, tom lane -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Autonomous Transaction is back

2015-07-27 Thread Josh Berkus
On 07/27/2015 02:47 AM, Rajeev rastogi wrote: Why have any fixed maximum? Since we are planning to have nested autonomous transaction, so it is required to have limit on this so that resources can be controlled. Is there a particular reason why this limit wouldn't just be max_stack_depth? --

Re: [HACKERS] WIP: Make timestamptz_out less slow.

2015-07-27 Thread Tom Lane
Andres Freund and...@anarazel.de writes: On 2015-07-27 17:31:41 -0400, Tom Lane wrote: What happens if you force use of port/snprintf.c instead of glibc's version? Even worse. 15900.014 ms. Interesting. So as a separate optimization problem, we might consider try to put snprintf.c at least

Re: [HACKERS] multivariate statistics / patch v7

2015-07-27 Thread Tomas Vondra
Hello Horiguchi-san, On 07/27/2015 09:04 AM, Kyotaro HORIGUCHI wrote: Hello, At Sat, 25 Jul 2015 23:09:31 +0200, Tomas Vondra tomas.von...@2ndquadrant.com wrote in 55b3fb0b.7000...@2ndquadrant.com Hi, On 07/16/2015 01:51 PM, Kyotaro HORIGUCHI wrote: Hi, I'd like to show you the modified

Re: [HACKERS] Autonomous Transaction is back

2015-07-27 Thread Merlin Moncure
On Mon, Jul 27, 2015 at 4:41 PM, Joel Jacobson j...@trustly.com wrote: On Fri, Jul 24, 2015 at 9:39 PM, Merlin Moncure mmonc...@gmail.com wrote: On Thu, Jul 23, 2015 at 1:49 PM, Josh Berkus j...@agliodbs.com wrote: Batch Jobs: large data-manipulation tasks which need to be broken up into

Re: [HACKERS] A little RLS oversight?

2015-07-27 Thread Dean Rasheed
On 27 July 2015 at 22:36, Joe Conway m...@joeconway.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/27/2015 01:25 PM, Dean Rasheed wrote: Looks good to me, except I'm not sure about those latest changes because I don't understand the reasoning behind the logic in

Re: [HACKERS] Optimization idea: merging multiple EXISTS(...) with constraint-based join removal

2015-07-27 Thread David Rowley
On 28 July 2015 at 09:37, Frédéric TERRAZZONI frederic.terrazz...@gmail.com wrote: SELECT * FROM t1 WHERE EXISTS( SELECT 1 FROM t2, t3, t4 WHERE t2.id = t1.t2_id AND t3.id = t2.t3_id AND t4.id = t3.t4_id AND t4.val = 'XYZ' ) AND

Re: [HACKERS] Autonomous Transaction is back

2015-07-27 Thread Jim Nasby
On 7/27/15 5:56 PM, Josh Berkus wrote: Can you explain what use case you have where simply telling the staff if you use ATX without clearing it, you'll be fired is not sufficient? Possibly there's something we failed to account for in the unconference discussion. That there's no way to

Re: [HACKERS] Autonomous Transaction is back

2015-07-27 Thread Joel Jacobson
On Mon, Jul 27, 2015 at 11:49 PM, Josh Berkus j...@agliodbs.com wrote: Ah, you're missing how commits in ATX are expected to work. Let me illustrate: X ( Data write A1 call Y( Start ATX Data write B1 Commit ATX ) Data write A2 Exception ) In

Re: [HACKERS] A little RLS oversight?

2015-07-27 Thread Stephen Frost
On Monday, July 27, 2015, Dean Rasheed dean.a.rash...@gmail.com wrote: On 27 July 2015 at 22:36, Joe Conway m...@joeconway.com javascript:; wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/27/2015 01:25 PM, Dean Rasheed wrote: Looks good to me, except I'm not sure about

Re: [HACKERS] Autonomous Transaction is back

2015-07-27 Thread Josh Berkus
On 07/27/2015 03:12 PM, Joel Jacobson wrote: On Mon, Jul 27, 2015 at 11:49 PM, Josh Berkus j...@agliodbs.com mailto:j...@agliodbs.com wrote: Ah, you're missing how commits in ATX are expected to work. Let me illustrate: X ( Data write A1 call Y(

Re: [HACKERS] optimizing vacuum truncation scans

2015-07-27 Thread Jim Nasby
On 7/27/15 10:39 AM, Robert Haas wrote: But that's not enough: we also need to know that any tuple that survived the prune operation (that is, it wasn't redirected or marked unused) has a new-enough xmin, which isn't tracked anywhere. Wouldn't that be old-enough xmin? heap_prune_chain() is

Re: [HACKERS] WIP: Make timestamptz_out less slow.

2015-07-27 Thread David Rowley
On 28 July 2015 at 09:17, Andres Freund and...@anarazel.de wrote: Hi, I recently once more noticed that timestamptz_out is really, really slow. To quantify that, I created a bunch of similar sized tables: CREATE TABLE tbl_timestamp AS SELECT NOW() FROM generate_series(1, 10) a,

Re: [HACKERS] Autonomous Transaction is back

2015-07-27 Thread Jim Nasby
On 7/27/15 5:12 PM, Joel Jacobson wrote: Right now, when writing a function, if you raise an exception, you can be sure all writes you have made will be rollbacked, but your caller function might caught the exception and decide to carry on and commit work made before your function was called,

Re: [HACKERS] Planner debug views

2015-07-27 Thread Qingqing Zhou
On Thu, Jul 23, 2015 at 4:11 PM, Tatsuo Ishii is...@postgresql.org wrote: Sounds like a great feature! Thanks! Attached is a draft patch implementing the idea. To play with it, you shall create the follow two foreign tables: CREATE EXTENSION file_fdw; CREATE SERVER pglog FOREIGN DATA WRAPPER

Re: [HACKERS] Sharing aggregate states between different aggregate functions

2015-07-27 Thread David Rowley
On 27 July 2015 at 20:11, Heikki Linnakangas hlinn...@iki.fi wrote: On 07/27/2015 08:34 AM, David Rowley wrote: - * agg_input_types, agg_state_type, agg_result_type identify the input, - * transition, and result types of the aggregate. These should all be - * resolved to actual types (ie,

Re: [HACKERS] Buildfarm TAP testing is useless as currently implemented

2015-07-27 Thread Michael Paquier
On Tue, Jul 28, 2015 at 1:15 AM, Andrew Dunstan and...@dunslane.net wrote: Well, it does create a lot of files that we don't pick up. An example list is show below, and I am attaching their contents in a single gzipped attachment. However, these are in the wrong location. This was a vpath build

Re: [HACKERS] A little RLS oversight?

2015-07-27 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/27/2015 03:05 PM, Stephen Frost wrote: AFK at the moment, but my thinking was that we should avoid having the error message change based on what a GUC is set to. I agree that there should be comments which explain that. I changed back to

Re: [HACKERS] Autonomous Transaction is back

2015-07-27 Thread Jim Nasby
On 7/27/15 6:40 PM, Jim Nasby wrote: On 7/27/15 5:12 PM, Joel Jacobson wrote: Right now, when writing a function, if you raise an exception, you can be sure all writes you have made will be rollbacked, but your caller function might caught the exception and decide to carry on and commit work

Re: [HACKERS] Several memory leaks for pg_rewind caused by missing PQclear calls

2015-07-27 Thread Michael Paquier
On Tue, Jul 28, 2015 at 2:45 AM, Heikki Linnakangas hlinn...@iki.fi wrote: On 07/23/2015 05:08 PM, Michael Paquier wrote: Hi all, After a run of valgrind on pg_rewind, I found a couple of code paths missing some PQclear calls after running a query. Attached is a patch to fix all those

Re: [HACKERS] more RLS oversights

2015-07-27 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/27/2015 01:13 PM, Alvaro Herrera wrote: Hmm, these are not ACL objects, so conceptually it seems cleaner to use a different symbol for this. I think the catalog state and the error messages would be a bit confusing otherwise. Ok -- done

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2015-07-27 Thread Ildus Kurbangaliev
Hello. In the attached patch I've made a refactoring for tranches. The prefix for them was extended, and I've did a split of LWLockAssign to two functions (one with tranche and second for user defined LWLocks). -- Ildus Kurbangaliev Postgres Professional: http://www.postgrespro.com The

Re: [HACKERS] Feature - Index support on an lquery field (from the ltree module)

2015-07-27 Thread Heikki Linnakangas
On 07/28/2015 06:40 AM, David Belle wrote: I have a requirement for a project that I am working on and was hoping to gain some discussion on the idea of implementing an index type for lquery fields (see ltree’s http://www.postgresql.org/docs/9.4/static/ltree.html

Re: [HACKERS] [DESIGN] ParallelAppend

2015-07-27 Thread Kouhei Kaigai
-Original Message- From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Kouhei Kaigai Sent: Monday, July 27, 2015 11:07 PM To: Amit Kapila Cc: pgsql-hackers@postgresql.org; Robert Haas; Kyotaro HORIGUCHI Subject: Re: [HACKERS] [DESIGN]

Re: [HACKERS] MultiXact member wraparound protections are now enabled

2015-07-27 Thread Noah Misch
On Mon, Jul 27, 2015 at 07:59:40AM +0100, Simon Riggs wrote: On 26 July 2015 at 20:15, Noah Misch n...@leadboat.com wrote: On Fri, Jul 24, 2015 at 09:14:09PM -0400, Peter Eisentraut wrote: On 7/22/15 4:45 PM, Robert Haas wrote: But it seemed to me that this could be rather confusing. I

Re: [HACKERS] CustomScan and readfuncs.c

2015-07-27 Thread Kouhei Kaigai
2. Reproduce method table on background worker -- The method field of CustomPath/Scan/ScanState is expected to be a reference to a static structure. Thus, copyObject() does not copy the entire table, but only pointers. However, we have no way to

Re: [HACKERS] proposal: multiple psql option -c

2015-07-27 Thread Pavel Stehule
2015-07-27 21:57 GMT+02:00 Andrew Dunstan and...@dunslane.net: On 07/27/2015 02:53 PM, Pavel Stehule wrote: I am trying to run parallel execution psql -At -c select datname from pg_database postgres | xargs -n 1 -P 3 psql -c select current_database() I don't think it's going to be

Re: [HACKERS] more RLS oversights

2015-07-27 Thread Joe Conway
On 07/03/2015 10:03 AM, Noah Misch wrote: +static void +dumpPolicy(Archive *fout, PolicyInfo *polinfo) ... +if (polinfo-polqual != NULL) +appendPQExpBuffer(query, USING %s, polinfo-polqual); (3) The USING clause needs parentheses; a dump+reload failed like so: Also needed

Re: [HACKERS] Failing assertions in indxpath.c, placeholder.c and brin_minmax.c

2015-07-27 Thread Tom Lane
Andreas Seltenreich seltenre...@gmx.de writes: I let sqlsmith run during the night, and it did no longer trigger the first two. During roughly a million random queries it triggered the already mentioned brin one 10 times, but there was also one instance of this new one in the log: TRAP:

[HACKERS] Feature - Index support on an lquery field (from the ltree module)

2015-07-27 Thread David Belle
Hi Forgive me if this is not the right place to discuss this. I am new to the postgresql community. I have a requirement for a project that I am working on and was hoping to gain some discussion on the idea of implementing an index type for lquery fields (see ltree’s

Re: [HACKERS] Autonomous Transaction is back

2015-07-27 Thread Pavel Stehule
2015-07-27 23:59 GMT+02:00 Merlin Moncure mmonc...@gmail.com: On Mon, Jul 27, 2015 at 4:41 PM, Joel Jacobson j...@trustly.com wrote: On Fri, Jul 24, 2015 at 9:39 PM, Merlin Moncure mmonc...@gmail.com wrote: On Thu, Jul 23, 2015 at 1:49 PM, Josh Berkus j...@agliodbs.com wrote: Batch

Re: [HACKERS] proposal: multiple psql option -c

2015-07-27 Thread Pavel Stehule
2015-07-28 5:24 GMT+02:00 Pavel Stehule pavel.steh...@gmail.com: 2015-07-27 21:57 GMT+02:00 Andrew Dunstan and...@dunslane.net: On 07/27/2015 02:53 PM, Pavel Stehule wrote: I am trying to run parallel execution psql -At -c select datname from pg_database postgres | xargs -n 1 -P 3

Re: [HACKERS] [DESIGN] ParallelAppend

2015-07-27 Thread Amit Kapila
On Tue, Jul 28, 2015 at 7:59 AM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: -Original Message- From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Kouhei Kaigai Sent: Monday, July 27, 2015 11:07 PM To: Amit Kapila Is there a

Re: [HACKERS] Failing assertions in indxpath.c, placeholder.c and brin_minmax.c

2015-07-27 Thread Andreas Seltenreich
Tom Lane writes: Andreas Seltenreich seltenre...@gmx.de writes: when running my random query generator contraption[1] against the regression database of 9.5 or master, it occasionally triggers one of the following three assertions. I've fixed the first two of these --- thanks for the

Re: [HACKERS] Sharing aggregate states between different aggregate functions

2015-07-27 Thread Haribabu Kommi
On Thu, Jul 9, 2015 at 7:44 PM, David Rowley david.row...@2ndquadrant.com wrote: On 15 June 2015 at 12:05, David Rowley david.row...@2ndquadrant.com wrote: This basically allows an aggregate's state to be shared between other aggregate functions when both aggregate's transition functions (and

Re: [HACKERS] MultiXact member wraparound protections are now enabled

2015-07-27 Thread Simon Riggs
On 26 July 2015 at 20:15, Noah Misch n...@leadboat.com wrote: On Fri, Jul 24, 2015 at 09:14:09PM -0400, Peter Eisentraut wrote: On 7/22/15 4:45 PM, Robert Haas wrote: But it seemed to me that this could be rather confusing. I thought it would be better to be explicit about whether the

Re: [HACKERS] Sharing aggregate states between different aggregate functions

2015-07-27 Thread David Rowley
On 27 July 2015 at 18:15, Haribabu Kommi kommi.harib...@gmail.com wrote: On Thu, Jul 9, 2015 at 7:44 PM, David Rowley david.row...@2ndquadrant.com wrote: On 15 June 2015 at 12:05, David Rowley david.row...@2ndquadrant.com wrote: This basically allows an aggregate's state to be shared

Re: [HACKERS] multivariate statistics / patch v7

2015-07-27 Thread Kyotaro HORIGUCHI
Hello, At Sat, 25 Jul 2015 23:09:31 +0200, Tomas Vondra tomas.von...@2ndquadrant.com wrote in 55b3fb0b.7000...@2ndquadrant.com Hi, On 07/16/2015 01:51 PM, Kyotaro HORIGUCHI wrote: Hi, I'd like to show you the modified constitution of multivariate statistics application logic. Please find

Re: [HACKERS] We need to support ForeignRecheck for late row locking, don't we?

2015-07-27 Thread Etsuro Fujita
KaiGai-san, On 2015/07/24 23:51, Kouhei Kaigai wrote: On 2015/07/22 19:10, Etsuro Fujita wrote: While working on the issue Foreign join pushdown vs EvalPlanQual, I happened to notice odd behaviors of late row locking in FDWs. I think the reason for that is because we don't check pushed-down

Re: [HACKERS] Sharing aggregate states between different aggregate functions

2015-07-27 Thread Heikki Linnakangas
On 07/27/2015 08:34 AM, David Rowley wrote: - * agg_input_types, agg_state_type, agg_result_type identify the input, - * transition, and result types of the aggregate. These should all be - * resolved to actual types (ie, none should ever be ANYELEMENT etc). + * agg_input_types identifies the

Re: [HACKERS] pg_dump -Fd and compression level

2015-07-27 Thread Marc Mamin
As per attached patch. Comments? It seems that the first test on the compression in pg_backup_tar.c is now obsolete. It didn't make much sense anyway. 211 if (AH-compression 0 || AH-compression 9) 212 AH-compression = Z_DEFAULT_COMPRESSION; 213

Re: [HACKERS] Sharing aggregate states between different aggregate functions

2015-07-27 Thread David Rowley
On 27 July 2015 at 20:11, Heikki Linnakangas hlinn...@iki.fi wrote: On 07/27/2015 08:34 AM, David Rowley wrote: In this function I also wasn't quite sure if it was with comparing both non-NULL INITCOND's here. I believe my code comments may slightly contradict what the code actually does,

Re: [HACKERS] We need to support ForeignRecheck for late row locking, don't we?

2015-07-27 Thread Kouhei Kaigai
On 2015/07/24 23:51, Kouhei Kaigai wrote: On 2015/07/22 19:10, Etsuro Fujita wrote: While working on the issue Foreign join pushdown vs EvalPlanQual, I happened to notice odd behaviors of late row locking in FDWs. I think the reason for that is because we don't check pushed-down quals

  1   2   >