Re: [HACKERS] Supporting huge pages on Windows

2017-04-04 Thread Andres Freund
On 2017-04-05 04:25:41 +, Tsunakawa, Takayuki wrote: > From: Craig Ringer [mailto:craig.rin...@2ndquadrant.com] > > On 5 April 2017 at 10:37, Tsunakawa, Takayuki > > wrote: > > > > OTOH, I tried again to leave the DISABLE_MAX_PRIVILEGE as is and add Lock > > Pages in Memory, using the attache

Re: [HACKERS] [pgsql-www] Small issue in online devel documentation build

2017-04-04 Thread Noah Misch
On Thu, Mar 23, 2017 at 11:21:39PM -0400, Peter Eisentraut wrote: > I think the fix belongs into the web site CSS, so there is nothing to > commit into PostgreSQL here. I will close the commit fest entry, but I > have added a section to the open items list so we keep track of it. > (https://wiki.p

[HACKERS] Re: extended stats not friendly towards ANALYZE with subset of columns

2017-04-04 Thread Noah Misch
On Tue, Mar 28, 2017 at 10:30:03PM +1300, David Rowley wrote: > I'm just reviewing Tomas' code for the dependencies part of the stats > when I saw something that looked a bit unusual. > > I tested with: > > CREATE TABLE ab1 (a INTEGER, b INTEGER); > ALTER TABLE ab1 ALTER a SET STATISTICS 0; > INS

Re: [HACKERS] [COMMITTERS] pgsql: Sync pg_dump and pg_dumpall output

2017-04-04 Thread Noah Misch
On Fri, Mar 24, 2017 at 12:26:33PM +0900, Michael Paquier wrote: > On Thu, Mar 23, 2017 at 1:10 AM, Stephen Frost wrote: > > * Andrew Dunstan (andrew.duns...@2ndquadrant.com) wrote: > >> On 03/22/2017 11:39 AM, Stephen Frost wrote: > >> > * Andrew Dunstan (and...@dunslane.net) wrote: > >> >> Sync

[HACKERS] Re: Query fails when SRFs are part of FROM clause (Commit id: 69f4b9c85f)

2017-04-04 Thread Noah Misch
On Fri, Mar 10, 2017 at 11:49:46AM -0800, Andres Freund wrote: > On 2017-03-09 13:34:22 -0500, Robert Haas wrote: > > On Mon, Jan 30, 2017 at 6:54 PM, Tom Lane wrote: > > > Andres Freund writes: > > >> Wonder if we there's an argument to be made for implementing this > > >> roughly similarly to s

Re: [HACKERS] Quorum commit for multiple synchronous replication.

2017-04-04 Thread Noah Misch
On Mon, Dec 19, 2016 at 09:49:58PM +0900, Fujii Masao wrote: > Regarding this feature, there are some loose ends. We should work on > and complete them until the release. > > (1) > Which synchronous replication method, priority or quorum, should be > chosen when neither FIRST nor ANY is specified

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-04 Thread Tatsuo Ishii
> From: pgsql-hackers-ow...@postgresql.org >> [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Tatsuo Ishii >> I have done tests using pgproto. One thing I noticed a strange behavior. >> Below is an output of pgproto. The test first set the timeout to 3 seconds, >> and parse/bind for "SELEC

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-04-04 Thread Ashutosh Bapat
On Wed, Apr 5, 2017 at 8:39 AM, Robert Haas wrote: > On Tue, Apr 4, 2017 at 10:22 AM, Ashutosh Bapat > wrote: >> Yes, I agree. For an inner join, the partition key types need to "shrink" >> and for outer join they need to be "widened". I don't know if there is a way >> to know "wider" or "shorter

Re: [HACKERS] multivariate statistics (v25)

2017-04-04 Thread Sven R. Kunze
Thanks Tomas and David for hacking on this patch. On 04.04.2017 20:19, Tomas Vondra wrote: I'm not sure we still need the min_group_size, when evaluating dependencies. It was meant to deal with 'noisy' data, but I think it after switching to the 'degree' it might actually be a bad idea. Consi

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-04 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Tatsuo Ishii > I have done tests using pgproto. One thing I noticed a strange behavior. > Below is an output of pgproto. The test first set the timeout to 3 seconds, > and parse/bind for "SELECT pg_s

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-04 Thread Tatsuo Ishii
>> What do you think? I've not really tested this with the extended protocol, >> so I'd appreciate if you could rerun your test from the older thread. > > The patch looks good and cleaner. It looks like the code works as expected. > As before, I ran one INSERT statement with PgJDBC, with gdb's

Re: [HACKERS] Adding support for Default partition in partitioning

2017-04-04 Thread Rushabh Lathia
On Wed, Apr 5, 2017 at 10:59 AM, Amit Langote wrote: > On 2017/04/05 6:22, Keith Fiske wrote: > > On Tue, Apr 4, 2017 at 9:30 AM, Rahila Syed wrote: > >> Please find attached an updated patch. > >> Following has been accomplished in this update: > >> > >> 1. A new partition can be added after def

Re: [HACKERS] BRIN cost estimate

2017-04-04 Thread Emre Hasegeli
> Interested to hear comments on this. I don't have chance to test it right now, but I am sure it would be an improvement over what we have right now. There is no single correct equation with so many unknowns we have. > *indexTotalCost += (numTuples * *indexSelectivity) * (cpu_index_tuple_cos

Re: [HACKERS] Adding support for Default partition in partitioning

2017-04-04 Thread Amit Langote
On 2017/04/05 6:22, Keith Fiske wrote: > On Tue, Apr 4, 2017 at 9:30 AM, Rahila Syed wrote: >> Please find attached an updated patch. >> Following has been accomplished in this update: >> >> 1. A new partition can be added after default partition if there are no >> conflicting rows in default parti

Re: [HACKERS] partitioned tables and contrib/sepgsql

2017-04-04 Thread Tom Lane
Peter Eisentraut writes: > On 4/5/17 00:58, Tom Lane wrote: >> Another issue is whether you won't get compiler complaints about >> redefinition of the "true" and "false" macros. But those would >> likely only be warnings, not flat-out errors. > The complaint about bool is also just a warning. R

Re: [HACKERS] partitioned tables and contrib/sepgsql

2017-04-04 Thread Peter Eisentraut
On 4/5/17 00:58, Tom Lane wrote: > Another issue is whether you won't get compiler complaints about > redefinition of the "true" and "false" macros. But those would > likely only be warnings, not flat-out errors. The complaint about bool is also just a warning. -- Peter Eisentraut

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-04 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of 'Andres Freund' > Attached. I did not like that the previous patch had the timeout handling > duplicated in the individual functions, I instead centralized it into > start_xact_command(). Given tha

Re: [HACKERS] partitioned tables and contrib/sepgsql

2017-04-04 Thread Tom Lane
Robert Haas writes: > On Tue, Apr 4, 2017 at 6:56 PM, Joe Conway wrote: >> Any objections? > I'm guessing Tom's going to have a strong feeling about whether 0001a > is the right way to address the stdbool issue, I will? [ looks ... ] Yup, you're right. I doubt that works at all, TBH. What I

Re: [HACKERS] Compiler warning in costsize.c

2017-04-04 Thread Michael Paquier
On Wed, Apr 5, 2017 at 2:54 AM, Tom Lane wrote: > Michael Paquier writes: >> In builds where USE_ASSERT_CHECKING is not enabled, costsize.c can >> generate warnings. Here is for example with MSVC: >> src/backend/optimizer/path/costsize.c(4520): warning C4101: 'rte' : >> unreferenced local vari

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-04 Thread Tom Lane
Andres Freund writes: > On 2017-04-05 10:05:19 +0900, Tatsuo Ishii wrote: >> What's your point of the question? What kind of problem do you expect >> if the timeout starts only once at the first parse meesage out of >> bunch of parse messages? > It's perfectly valid to send a lot of Parse message

Re: [HACKERS] BRIN cost estimate

2017-04-04 Thread David Rowley
On 3 April 2017 at 03:05, Emre Hasegeli wrote: > Unfortunately, I am on vacation for two weeks without my computer. I can > post another version after 18th. I know we are under time pressure for > release. I wouldn't mind if you or Alvaro would change it anyway you like. I've made some changes

Re: [HACKERS] Supporting huge pages on Windows

2017-04-04 Thread Tsunakawa, Takayuki
From: Craig Ringer [mailto:craig.rin...@2ndquadrant.com] > On 5 April 2017 at 10:37, Tsunakawa, Takayuki > wrote: > > OTOH, I tried again to leave the DISABLE_MAX_PRIVILEGE as is and add Lock > Pages in Memory, using the attached pg_ctl.c. Please see > EnableLockPagesPrivilege() and its call sit

Re: [HACKERS] Implementation of SASLprep for SCRAM-SHA-256

2017-04-04 Thread Michael Paquier
fore On Wed, Apr 5, 2017 at 7:05 AM, Heikki Linnakangas wrote: > I will continue tomorrow, but I wanted to report on what I've done so far. > Attached is a new patch version, quite heavily modified. Notable changes so > far: Great, thanks! > * Use Unicode codepoints, rather than UTF-8 bytes pac

Re: [HACKERS] Re: PATCH: pageinspect / add page_checksum and bt_page_items(bytea)

2017-04-04 Thread Peter Eisentraut
On 4/4/17 12:55, Ashutosh Sharma wrote: > As I am not seeing any response from Tomas for last 2-3 days and since > the commit-fest is coming towards end, I have planned to work on the > review comments that I had given few days back and submit the updated > patch. PFA new version of patch that take

Re: [HACKERS] increasing the default WAL segment size

2017-04-04 Thread Peter Eisentraut
On 4/4/17 22:47, Amit Kapila wrote: >> Committed first part to allow internal representation change (only). >> >> No commitment yet to increasing wal-segsize in the way this patch has it. >> > > What part of patch you don't like and do you have any suggestions to > improve the same? I think there

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-04-04 Thread Pavan Deolasee
On Wed, Apr 5, 2017 at 8:42 AM, Robert Haas wrote: > On Tue, Apr 4, 2017 at 10:21 PM, Pavan Deolasee > wrote: > > On Thu, Mar 30, 2017 at 7:55 PM, Robert Haas > wrote: > >> but > >> try to access the TOAST table would be fatal; that probably would have > >> deadlock hazards among other problem

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-04-04 Thread Robert Haas
On Tue, Apr 4, 2017 at 10:21 PM, Pavan Deolasee wrote: > On Thu, Mar 30, 2017 at 7:55 PM, Robert Haas wrote: >> but >> try to access the TOAST table would be fatal; that probably would have >> deadlock hazards among other problems. > > Hmm. I think you're right. We could make a copy of the heap

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-04-04 Thread Robert Haas
On Tue, Apr 4, 2017 at 10:22 AM, Ashutosh Bapat wrote: > Yes, I agree. For an inner join, the partition key types need to "shrink" > and for outer join they need to be "widened". I don't know if there is a way > to know "wider" or "shorter" of two given types. We might have to implement > a method

Re: [HACKERS] Supporting huge pages on Windows

2017-04-04 Thread Craig Ringer
On 5 April 2017 at 10:37, Tsunakawa, Takayuki wrote: > Good point! And I said earlier in this thread, I think managing privileges > (adding/revoking privileges from the user account) is the DBA's or sysadmin's > duty, and PG's removing all privileges feels overkill. I think it's a sensible al

Re: [HACKERS] multivariate statistics (v25)

2017-04-04 Thread Kyotaro HORIGUCHI
At Tue, 4 Apr 2017 20:19:39 +0200, Tomas Vondra wrote in <56f40b20-c464-fad2-ff39-06b668fac...@2ndquadrant.com> > On 04/04/2017 09:55 AM, David Rowley wrote: > > On 1 April 2017 at 04:25, David Rowley > > wrote: > >> I've attached an updated patch. > > > > I've made another pass at this and ende

Re: [HACKERS] identity columns

2017-04-04 Thread Vitaly Burovoy
On 4/3/17, Peter Eisentraut wrote: > On 3/30/17 22:57, Vitaly Burovoy wrote: >> Why do you still want to leave "ADD IF NOT EXISTS" instead of using >> "SET IF NOT EXISTS"? >> If someone wants to follow the standard he can simply not to use "IF >> NOT EXISTS" at all. Without it error should be rais

Re: [HACKERS] increasing the default WAL segment size

2017-04-04 Thread Amit Kapila
On Wed, Apr 5, 2017 at 3:36 AM, Simon Riggs wrote: > On 27 March 2017 at 15:36, Beena Emerson wrote: > >> 02-increase-max-wal-segsize.patch - Increases the wal-segsize and changes >> the internal representation of max_wal_size and min_wal_size to mb. > > Committed first part to allow internal rep

Re: [HACKERS] Faster methods for getting SPI results (460% improvement)

2017-04-04 Thread Craig Ringer
On 5 April 2017 at 08:23, Craig Ringer wrote: > On 5 April 2017 at 08:00, Craig Ringer wrote: > >> Taking a look at this now. > > Rebased to current master with conflicts and whitespace errors fixed. > Review pending. This patch fails to update the documentation at all. https://www.postgresql.o

Re: [HACKERS] Supporting huge pages on Windows

2017-04-04 Thread Tsunakawa, Takayuki
From: Craig Ringer [mailto:craig.rin...@2ndquadrant.com] > TBH, anyone who cares about security and runs Win7 or Win2k8 or newer should > be using virtual service accounts and managed service accounts. > > https://technet.microsoft.com/en-us/library/dd548356 > > > Those are more like Unix servic

Re: [HACKERS] Parallel Append implementation

2017-04-04 Thread Ashutosh Bapat
On Wed, Apr 5, 2017 at 1:43 AM, Andres Freund wrote: > On 2017-04-04 08:01:32 -0400, Robert Haas wrote: > > On Tue, Apr 4, 2017 at 12:47 AM, Andres Freund > wrote: > > > I don't think the parallel seqscan is comparable in complexity with the > > > parallel append case. Each worker there does th

Re: [HACKERS] Rewriting the test of pg_upgrade as a TAP test

2017-04-04 Thread Michael Paquier
On Tue, Apr 4, 2017 at 10:52 PM, Peter Eisentraut wrote: > On 4/3/17 11:32, Andres Freund wrote: >> That doesn't strike as particularly future proof. We intentionally >> leave objects behind pg_regress runs, but that only works if we actually >> run them... > > I generally agree with the sentimen

Re: [HACKERS] Rewriting the test of pg_upgrade as a TAP test

2017-04-04 Thread Michael Paquier
On Tue, Apr 4, 2017 at 9:30 PM, Stephen Frost wrote: > * Michael Paquier (michael.paqu...@gmail.com) wrote: >> On Tue, Apr 4, 2017 at 7:38 AM, Stephen Frost wrote: >> The patch presented here does lower the coverage we have now. > > I assume (perhaps mistakenly) that this statement was based on a

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-04-04 Thread Pavan Deolasee
On Thu, Mar 30, 2017 at 7:55 PM, Robert Haas wrote: > but > try to access the TOAST table would be fatal; that probably would have > deadlock hazards among other problems. Hmm. I think you're right. We could make a copy of the heap tuple, drop the lock and then access TOAST to handle that. Wou

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-04 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Tatsuo Ishii > Hmm. IMO, that could happen even with the current statement timeout > implementation as well. > > Or we could start/stop the timeout in exec_execute_message() only. This > could avoid

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-04 Thread Tatsuo Ishii
> On 2017-04-04 16:56:26 -0700, 'Andres Freund' wrote: >> On 2017-04-04 23:52:28 +, Tsunakawa, Takayuki wrote: >> > From: Andres Freund [mailto:and...@anarazel.de] >> > > Looks to me like npgsql doesn't do that either. None of libpq, pgjdbs >> > > and >> > > npgsql doing it seems like some ev

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-04 Thread 'Andres Freund'
On 2017-04-04 16:56:26 -0700, 'Andres Freund' wrote: > On 2017-04-04 23:52:28 +, Tsunakawa, Takayuki wrote: > > From: Andres Freund [mailto:and...@anarazel.de] > > > Looks to me like npgsql doesn't do that either. None of libpq, pgjdbs and > > > npgsql doing it seems like some evidence that it

Re: [HACKERS] partitioned tables and contrib/sepgsql

2017-04-04 Thread Robert Haas
On Tue, Apr 4, 2017 at 6:56 PM, Joe Conway wrote: > On 04/04/2017 10:02 AM, Joe Conway wrote: >> On 04/04/2017 09:55 AM, Mike Palmiotto wrote: >>> After some discussion off-list, I've rebased and udpated the patches. >>> Please see attached for further review. >> >> Thanks -- will have another loo

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-04 Thread Tatsuo Ishii
>> What's your point of the question? What kind of problem do you expect >> if the timeout starts only once at the first parse meesage out of >> bunch of parse messages? > > It's perfectly valid to send a lot of Parse messages without > interspersed Sync or Bind/Execute message. There'll be one t

Re: [HACKERS] ANALYZE command progress checker

2017-04-04 Thread Masahiko Sawada
On Wed, Apr 5, 2017 at 1:49 AM, Robert Haas wrote: > On Tue, Apr 4, 2017 at 4:57 AM, Amit Langote > wrote: >> Hmm, you're right. It could be counted with a separate variable >> initialized to 0 and incremented every time we decide to add a row to the >> final set of sampled rows, although differ

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-04 Thread Andres Freund
On 2017-04-05 10:05:19 +0900, Tatsuo Ishii wrote: > > Hm. I started to edit it, but I'm halfway coming back to my previous > > view that this isn't necessarily ready. > > > > If a client were to to prepare a large number of prepared statements > > (i.e. a lot of parse messages), this'll only start

[HACKERS] Outdated comments around HandleFunctionRequest

2017-04-04 Thread Andres Freund
Hi, PostgresMain() has the following blurb for fastpath functions: /* * Note: we may at this point be inside an aborted * transaction. We can't throw error for that until we've

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-04 Thread Tatsuo Ishii
> Hm. I started to edit it, but I'm halfway coming back to my previous > view that this isn't necessarily ready. > > If a client were to to prepare a large number of prepared statements > (i.e. a lot of parse messages), this'll only start the timeout once, at > the first statement sent. It's not

Re: [HACKERS] pgbench - allow to store select results into variables

2017-04-04 Thread Tatsuo Ishii
>> Please find attached a v8 which hopefully fixes these two issues. > Looks good to me, marking as ready for committer. I have looked into this a little bit. It seems the new feature \gset doesn't work with tables having none ascii column names: $ src/bin/pgbench/pgbench -t 1 -f /tmp/f test sta

Re: [HACKERS] [PATCH] Reduce src/test/recovery verbosity

2017-04-04 Thread Michael Paquier
On Wed, Apr 5, 2017 at 12:39 AM, Stephen Frost wrote: > Committed, with those additions. Thanks for the commit. The final result is nice. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/p

Re: [HACKERS] Faster methods for getting SPI results (460% improvement)

2017-04-04 Thread Craig Ringer
On 5 April 2017 at 08:00, Craig Ringer wrote: > Taking a look at this now. Rebased to current master with conflicts and whitespace errors fixed. Review pending. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services From 71b616

Re: [HACKERS] Speedup twophase transactions

2017-04-04 Thread Michael Paquier
On Tue, Mar 28, 2017 at 3:10 PM, Michael Paquier wrote: > OK, done. I have just noticed that Simon has marked himself as a > committer of this patch 24 hours ago. For the archive's sake, this has been committed as 728bd991. Thanks Simon! -- Michael -- Sent via pgsql-hackers mailing list (pgsq

Re: [HACKERS] delta relations in AFTER triggers

2017-04-04 Thread Thomas Munro
On Wed, Apr 5, 2017 at 11:49 AM, Kevin Grittner wrote: > Worked on the docs some more and then pushed it. > > Nice job cutting the number of *.[ch] lines by 30 while adding support for > the other three core PLs. :-) Great. Thanks. I wonder if there is some way we can automatically include cod

Re: [HACKERS] Faster methods for getting SPI results (460% improvement)

2017-04-04 Thread Craig Ringer
On 6 March 2017 at 05:09, Jim Nasby wrote: > On 2/28/17 9:42 PM, Jim Nasby wrote: >>> >>> >>> I'll post a plpython patch that doesn't add the output format control. >> >> >> I've attached the results of that. Unfortunately the speed improvement >> is only 27% at this point (with 999 tuples). P

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-04 Thread 'Andres Freund'
On 2017-04-04 23:52:28 +, Tsunakawa, Takayuki wrote: > From: Andres Freund [mailto:and...@anarazel.de] > > Looks to me like npgsql doesn't do that either. None of libpq, pgjdbs and > > npgsql doing it seems like some evidence that it's ok. > > And psqlODBC now uses always libpq. > > Now time

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-04 Thread Tsunakawa, Takayuki
From: Andres Freund [mailto:and...@anarazel.de] > Looks to me like npgsql doesn't do that either. None of libpq, pgjdbs and > npgsql doing it seems like some evidence that it's ok. And psqlODBC now uses always libpq. Now time for final decision? Regards Takayuki Tsunakawa -- Sent via pgsql

Re: [HACKERS] delta relations in AFTER triggers

2017-04-04 Thread Kevin Grittner
On Mon, Apr 3, 2017 at 7:16 PM, Thomas Munro wrote: > On Tue, Apr 4, 2017 at 3:41 AM, Kevin Grittner wrote: >> On Mon, Apr 3, 2017 at 8:59 AM, Tom Lane wrote: >>> Thomas Munro writes: Or perhaps the code to inject trigger data transition tables into SPI (a near identical code block th

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-04 Thread Andres Freund
On 2017-04-04 16:38:53 -0700, Andres Freund wrote: > On 2017-04-04 16:10:32 +0900, Tatsuo Ishii wrote: > > >> If what Tatsuo-san said to Tom is correct (i.e. each Parse/Bind/Execute > > >> starts and stops the timer), then it's a concern and the patch should > > >> not be ready for committer. Ho

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-04 Thread Andres Freund
On 2017-04-05 08:34:43 +0900, Tatsuo Ishii wrote: > Andres, > > >> I think the code needs a few clarifying comments around this, but > >> otherwise seems good. Not restarting the timeout in those cases > >> obviously isn't entirely "perfect"/"correct", but a tradeoff - the > >> comments should no

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-04 Thread Andres Freund
On 2017-04-04 16:10:32 +0900, Tatsuo Ishii wrote: > >> If what Tatsuo-san said to Tom is correct (i.e. each Parse/Bind/Execute > >> starts and stops the timer), then it's a concern and the patch should not > >> be ready for committer. However, the current patch is not like that -- it > >> seems

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-04 Thread Tatsuo Ishii
Andres, >> I think the code needs a few clarifying comments around this, but >> otherwise seems good. Not restarting the timeout in those cases >> obviously isn't entirely "perfect"/"correct", but a tradeoff - the >> comments should note that. >> >> Tatsuo-san, do you want to change those, and p

Re: [HACKERS] partitioned tables and contrib/sepgsql

2017-04-04 Thread Joe Conway
On 04/04/2017 10:02 AM, Joe Conway wrote: > On 04/04/2017 09:55 AM, Mike Palmiotto wrote: >> After some discussion off-list, I've rebased and udpated the patches. >> Please see attached for further review. > > Thanks -- will have another look and test on a machine with selinux > setup. Robert, did

Re: [HACKERS] increasing the default WAL segment size

2017-04-04 Thread Simon Riggs
On 27 March 2017 at 15:36, Beena Emerson wrote: > 02-increase-max-wal-segsize.patch - Increases the wal-segsize and changes > the internal representation of max_wal_size and min_wal_size to mb. Committed first part to allow internal representation change (only). No commitment yet to increasing

Re: [HACKERS] Implementation of SASLprep for SCRAM-SHA-256

2017-04-04 Thread Heikki Linnakangas
On 04/04/2017 01:52 PM, Heikki Linnakangas wrote: On 03/31/2017 10:10 AM, Michael Paquier wrote: On Wed, Mar 8, 2017 at 10:39 PM, Robert Haas wrote: On Tue, Mar 7, 2017 at 10:01 PM, Michael Paquier wrote: I kinda hope Heikki is going to step up to the plate here, because I think he understand

Re: [HACKERS] bug in SlabAlloc / VALGRIND_MAKE_MEM_DEFINED

2017-04-04 Thread Andres Freund
On 2017-04-04 23:23:30 +0200, Tomas Vondra wrote: > On 04/04/2017 10:42 PM, Tomas Vondra wrote: > > Hi, > > > > Andres nagged to me about valgrind runs taking much longer since > > 9fab40ad introduced the SlabContext into reorderbuffer.c. And by > > "longer" I mean hours instead of minutes. > > >

Re: [HACKERS] bug in SlabAlloc / VALGRIND_MAKE_MEM_DEFINED

2017-04-04 Thread Tomas Vondra
On 04/04/2017 10:42 PM, Tomas Vondra wrote: Hi, Andres nagged to me about valgrind runs taking much longer since 9fab40ad introduced the SlabContext into reorderbuffer.c. And by "longer" I mean hours instead of minutes. After a bit of investigation I stumbled on this line in slab.c: VALGRIND

Re: [HACKERS] Adding support for Default partition in partitioning

2017-04-04 Thread Keith Fiske
On Tue, Apr 4, 2017 at 9:30 AM, Rahila Syed wrote: > Hello, > > Please find attached an updated patch. > Following has been accomplished in this update: > > 1. A new partition can be added after default partition if there are no > conflicting rows in default partition. > 2. Solved the crash repor

Re: [HACKERS] PATCH: recursive json_populate_record()

2017-04-04 Thread Andrew Dunstan
On 04/03/2017 05:17 PM, Andres Freund wrote: > On 2017-03-21 14:31:08 -0400, Andrew Dunstan wrote: >> >> On 03/21/2017 01:37 PM, David Steele wrote: >>> On 3/16/17 11:54 AM, David Steele wrote: On 2/1/17 12:53 AM, Michael Paquier wrote: > On Thu, Jan 26, 2017 at 6:49 AM, Tom Lane wrote:

Re: [HACKERS] psql - add special variable to reflect the last query status

2017-04-04 Thread Pavel Stehule
2017-04-04 22:05 GMT+02:00 Fabien COELHO : > > After some discussions about what could be useful since psql scripts now > accepts tests, this patch sets a few variables which can be used by psql > after a "front door" (i.e. actually typed by the user) query: > > - RESULT_STATUS: the status of the

Re: [HACKERS] tuplesort_gettuple_common() and *should_free argument

2017-04-04 Thread Peter Geoghegan
On Tue, Apr 4, 2017 at 1:32 PM, Andres Freund wrote: > s/Avoid/Allow to avoid/ WFM. >> + * >> + * Callers cannot rely on memory for tuple in returned slot remaining valid >> + * past any subsequent manipulation of the sorter, such as another fetch of >> + * input from sorter. (The sorter may re

[HACKERS] bug in SlabAlloc / VALGRIND_MAKE_MEM_DEFINED

2017-04-04 Thread Tomas Vondra
Hi, Andres nagged to me about valgrind runs taking much longer since 9fab40ad introduced the SlabContext into reorderbuffer.c. And by "longer" I mean hours instead of minutes. After a bit of investigation I stumbled on this line in slab.c: VALGRIND_MAKE_MEM_DEFINED(chunk, SlabChunkGetPoint

Re: [HACKERS] tuplesort_gettuple_common() and *should_free argument

2017-04-04 Thread Andres Freund
On 2017-03-13 18:14:07 -0700, Peter Geoghegan wrote: > From 5351b5db257cb39832647d9117465c0217e6268b Mon Sep 17 00:00:00 2001 > From: Peter Geoghegan > Date: Thu, 13 Oct 2016 10:54:31 -0700 > Subject: [PATCH 1/2] Avoid copying within tuplesort_gettupleslot(). s/Avoid/Allow to avoid/ > Add a "cop

Re: [HACKERS] Logical decoding on standby

2017-04-04 Thread Andres Freund
On 2017-04-04 22:32:40 +0800, Craig Ringer wrote: > I'm much happier with this. I'm still fixing some issues in the tests > for 03 and tidying them up, but 03 should allow 01 and 02 to be > reviewed in their proper context now. To me this very clearly is too late for v10, and now should be moved t

Re: [HACKERS] Parallel Append implementation

2017-04-04 Thread Andres Freund
On 2017-04-04 08:01:32 -0400, Robert Haas wrote: > On Tue, Apr 4, 2017 at 12:47 AM, Andres Freund wrote: > > I don't think the parallel seqscan is comparable in complexity with the > > parallel append case. Each worker there does the same kind of work, and > > if one of them is behind, it'll just

[HACKERS] psql - add special variable to reflect the last query status

2017-04-04 Thread Fabien COELHO
After some discussions about what could be useful since psql scripts now accepts tests, this patch sets a few variables which can be used by psql after a "front door" (i.e. actually typed by the user) query: - RESULT_STATUS: the status of the query - ERROR: whether the query failed - ERROR

Re: [HACKERS] Re: PATCH: pageinspect / add page_checksum and bt_page_items(bytea)

2017-04-04 Thread David Steele
On 4/4/17 12:55 PM, Ashutosh Sharma wrote: > > As I am not seeing any response from Tomas for last 2-3 days and since > the commit-fest is coming towards end, I have planned to work on the > review comments that I had given few days back and submit the updated > patch. PFA new version of patch tha

Re: [HACKERS] WIP: Covering + unique indexes.

2017-04-04 Thread Peter Geoghegan
On Tue, Apr 4, 2017 at 3:07 AM, Anastasia Lubennikova wrote: >> * I think that we should store this (the number of attributes), and >> use it directly when comparing, per my remarks to Tom over on that >> other thread. We should also use the free bit within >> IndexTupleData.t_info, to indicate th

Re: [HACKERS] Re: PATCH: pageinspect / add page_checksum and bt_page_items(bytea)

2017-04-04 Thread Tomas Vondra
Thanks. I planned to look into this today, but you've been faster ;-) regards Tomas On 04/04/2017 06:55 PM, Ashutosh Sharma wrote: Hi, As I am not seeing any response from Tomas for last 2-3 days and since the commit-fest is coming towards end, I have planned to work on the review comments tha

Re: [HACKERS] multivariate statistics (v25)

2017-04-04 Thread Tomas Vondra
On 04/04/2017 09:55 AM, David Rowley wrote: On 1 April 2017 at 04:25, David Rowley wrote: I've attached an updated patch. I've made another pass at this and ended up removing the tryextstats variable. We now only try to use extended statistics when clauselist_selectivity() is given a valid Re

Re: [HACKERS] logical replication launcher crash on buildfarm

2017-04-04 Thread Petr Jelinek
So this is what I came up with on plane. Generalized the loading functionality into load_library_function which that can load either known postgres functions or call load_external_function. I am not quite sure if fmgr.c is best place to put it, but I didn't want to include stuff from executor in b

Re: [HACKERS] Compiler warning in costsize.c

2017-04-04 Thread Tom Lane
Michael Paquier writes: > In builds where USE_ASSERT_CHECKING is not enabled, costsize.c can > generate warnings. Here is for example with MSVC: > src/backend/optimizer/path/costsize.c(4520): warning C4101: 'rte' : > unreferenced local variable [C:\Users\ioltas\git\postgres\postgres.vcxproj] >

Re: [HACKERS] strange parallel query behavior after OOM crashes

2017-04-04 Thread Tomas Vondra
On 04/04/2017 06:52 PM, Robert Haas wrote: On Mon, Apr 3, 2017 at 6:08 AM, Kuntal Ghosh wrote: On Fri, Mar 31, 2017 at 6:50 PM, Robert Haas wrote: On Thu, Mar 30, 2017 at 4:35 PM, Kuntal Ghosh wrote: 2. the server restarts automatically, initialize BackgroundWorkerData->parallel_register_co

Re: [HACKERS] logical decoding of two-phase transactions

2017-04-04 Thread Andres Freund
On 2017-04-04 13:06:13 +0300, Stas Kelvich wrote: > That is just argument against Andres concern that prepared transaction > is able to deadlock with decoding process — at least no such cases in > regression tests. There's few longer / adverse xacts, that doesn't say much. > And that concern is

Re: [HACKERS] identity columns

2017-04-04 Thread Tom Lane
Peter Eisentraut writes: > On 4/3/17 14:19, Andres Freund wrote: > + *op->resvalue = > Int64GetDatum(nextval_internal(op->d.nextvalueexpr.seqid, false)); >> Is it guaranteed that the caller expects an int64? I saw that >> nextvalueexpr's have a typeid field. > It expects on

Re: [HACKERS] logical decoding of two-phase transactions

2017-04-04 Thread Masahiko Sawada
On Tue, Apr 4, 2017 at 7:06 PM, Stas Kelvich wrote: > >> On 4 Apr 2017, at 04:23, Masahiko Sawada wrote: >> >> >> I reviewed this patch but when I tried to build contrib/test_decoding >> I got the following error. >> > > Thanks! > > Yes, seems that 18ce3a4a changed ProcessUtility_hook signature.

Re: [HACKERS] Instead of DROP function use UPDATE pg_proc in an upgrade extension script

2017-04-04 Thread Pavel Stehule
2017-04-04 15:40 GMT+02:00 Vicky Vergara : > Thanks, > > > > It is not safe due views - that are saved in post analyze form. > > > What is post analyze form? any link that you can give me to read about it? > The Query pipe line is: parsing, analyze, optimalization, execution when you change a AP

Re: [HACKERS] pg_partman 3.0.0 - real-world usage of native partitioning and a case for native default

2017-04-04 Thread Keith Fiske
On Mon, Apr 3, 2017 at 11:33 PM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > > >>> Thankfully since native partitioning still uses inheritance internally for the most part, pg_partman works pretty well without nearly as much change as I thought I would need.

Re: [HACKERS] partitioned tables and contrib/sepgsql

2017-04-04 Thread Joe Conway
On 04/04/2017 09:55 AM, Mike Palmiotto wrote: > On Tue, Apr 4, 2017 at 10:19 AM, Joe Conway wrote: >> On 04/04/2017 06:45 AM, Robert Haas wrote: >>> On Mon, Apr 3, 2017 at 12:02 PM, Joe Conway wrote: > 0002 looks extremely straightforward, but I wonder if we could get one > of the people

Re: [HACKERS] partitioned tables and contrib/sepgsql

2017-04-04 Thread Mike Palmiotto
On Tue, Apr 4, 2017 at 10:19 AM, Joe Conway wrote: > On 04/04/2017 06:45 AM, Robert Haas wrote: >> On Mon, Apr 3, 2017 at 12:02 PM, Joe Conway wrote: 0002 looks extremely straightforward, but I wonder if we could get one of the people on this list who knows about sepgsql to have a look?

Re: [HACKERS] Re: PATCH: pageinspect / add page_checksum and bt_page_items(bytea)

2017-04-04 Thread Ashutosh Sharma
Hi, As I am not seeing any response from Tomas for last 2-3 days and since the commit-fest is coming towards end, I have planned to work on the review comments that I had given few days back and submit the updated patch. PFA new version of patch that takes care of all the review comments given by

Re: [HACKERS] strange parallel query behavior after OOM crashes

2017-04-04 Thread Robert Haas
On Mon, Apr 3, 2017 at 6:08 AM, Kuntal Ghosh wrote: > On Fri, Mar 31, 2017 at 6:50 PM, Robert Haas wrote: >> On Thu, Mar 30, 2017 at 4:35 PM, Kuntal Ghosh >> wrote: >>> 2. the server restarts automatically, initialize >>> BackgroundWorkerData->parallel_register_count and >>> BackgroundWorkerData

Re: [HACKERS] sequence data type

2017-04-04 Thread Peter Eisentraut
On 3/30/17 22:47, Vitaly Burovoy wrote: > It seemed not very hard to fix it. > Please find attached patch to be applied on top of your one. > > I've added more tests to cover different cases of changing bounds when > data type is changed. Committed all that. Thanks! -- Peter Eisentraut

Re: [HACKERS] ANALYZE command progress checker

2017-04-04 Thread Robert Haas
On Tue, Apr 4, 2017 at 4:57 AM, Amit Langote wrote: > Hmm, you're right. It could be counted with a separate variable > initialized to 0 and incremented every time we decide to add a row to the > final set of sampled rows, although different implementations of > AcquireSampleRowsFunc have differe

Re: [HACKERS] Instead of DROP function use UPDATE pg_proc in an upgrade extension script

2017-04-04 Thread Robert Haas
On Tue, Apr 4, 2017 at 9:07 AM, Vicky Vergara wrote: > you answered so fast that I know I am stepping into dangerous grounds. > > But I would like to know more about your experience. > > Any links that you can give me to read about the code and/or issues > regarding the ip4r experience? I can't c

Re: [HACKERS] PATCH: Make pg_stop_backup() archive wait optional

2017-04-04 Thread David Steele
On 4/4/17 11:42 AM, Stephen Frost wrote: > * David Steele (da...@pgmasters.net) wrote: >> On 3/22/17 4:42 PM, Peter Eisentraut wrote: >>> On 3/22/17 15:14, Stephen Frost wrote: > -SELECT * FROM pg_stop_backup(false); > +SELECT * FROM pg_stop_backup(false [, true ]); > > I think that

Re: [HACKERS] Proposal: Local indexes for partitioned table

2017-04-04 Thread Maksim Milyutin
On 01.03.2017 13:53, Maksim Milyutin wrote: Hi hackers! As I've understood from thread [1] the main issue of creating local indexes for partitions is supporting REINDEX and DROP INDEX operations on parent partitioned tables. Furthermore Robert Haas mentioned the problem of creating index on key

Re: [HACKERS] Making clausesel.c Smarter

2017-04-04 Thread Claudio Freire
On Tue, Apr 4, 2017 at 8:21 AM, David Rowley wrote: > On 4 April 2017 at 13:35, Claudio Freire wrote: >> On Mon, Apr 3, 2017 at 9:19 PM, Claudio Freire >> wrote: >>> On Mon, Apr 3, 2017 at 8:52 PM, David Rowley >>> wrote: > One last observation: > > +/* > + * An

Re: [HACKERS] Making clausesel.c Smarter

2017-04-04 Thread Claudio Freire
On Tue, Apr 4, 2017 at 8:12 AM, David Rowley wrote: > Result Comparison > > Master median tps Patch median tps comparison > Test 1 6993.7 6714.3 104.16% > Test 2 7053.1 6921.6 101.90% > Test 3 5137.2 4954.2 103.69% > Test 4 27.1 19.4 139.72% > Test 5 54.1 51.4 105.28% > Test 6 9328.1 9478.2 98.42%

Re: [HACKERS] postgres_fdw: support parameterized foreign joins

2017-04-04 Thread Arthur Zakirov
On 23.03.2017 15:45, Etsuro Fujita wrote: Done. Also, I added regression tests and revised code and comments a bit. (As for create_foreignscan_path(), I haven't done anything about that yet.) Please find attached a new version created on top of [1]. Thank you! I didn't notice that it is ne

Re: [HACKERS] PATCH: Make pg_stop_backup() archive wait optional

2017-04-04 Thread Stephen Frost
* David Steele (da...@pgmasters.net) wrote: > On 3/22/17 4:42 PM, Peter Eisentraut wrote: > >On 3/22/17 15:14, Stephen Frost wrote: > >>>-SELECT * FROM pg_stop_backup(false); > >>>+SELECT * FROM pg_stop_backup(false [, true ]); > >>> > >>>I think that it's better to get rid of "[" and "]" from the

Re: [HACKERS] [PATCH] Reduce src/test/recovery verbosity

2017-04-04 Thread Stephen Frost
Greetings, * Craig Ringer (cr...@2ndquadrant.com) wrote: > On 31 March 2017 at 04:29, Stephen Frost wrote: > > > Unless people wish to object, I'll use Michael's patch to remove > > --verbose from the top level tomorrow. > > Sounds good. > > Maybe add > > To get more detailed output from test

  1   2   >