Re: [HACKERS] Error while copying a large file in pg_rewind

2017-07-21 Thread Michael Paquier
On Fri, Jul 21, 2017 at 8:22 AM, Michael Paquier wrote: > On Thu, Jul 20, 2017 at 9:31 PM, Robert Haas wrote: >> Still, it can't be worse than the status quo, where instead of int64 >> we're using int and int32, so maybe we ought to back-patch it as-is >> for now and look at any further cleanup t

Re: [HACKERS] More race conditions in logical replication

2017-07-21 Thread Alvaro Herrera
I'm back at looking into this again, after a rather exhausting week. I think I have a better grasp of what is going on in this code now, and it appears to me that we should change the locking so that active_pid is protected by ReplicationSlotControlLock instead of the slot's spinlock; but I haven'

Re: [HACKERS] [WIP] [B-Tree] Keep indexes sorted by heap physical location

2017-07-21 Thread Peter Geoghegan
On Wed, Aug 17, 2016 at 7:54 PM, Claudio Freire wrote: > The attached patch tries to maintain the initial status of B-Tree > indexes, which are created with equal-key runs in physical order, > during the whole life of the B-Tree, and make key-tid pairs > efficiently searchable in the process. I d

Re: [HACKERS] segfault in HEAD when too many nested functions call

2017-07-21 Thread Tom Lane
Andres Freund writes: > On 2017-07-18 16:53:43 -0400, Tom Lane wrote: >> BTW, I don't see why you really need to mess with anything except >> ExecProcNode? Surely the other cases such as MultiExecProcNode are >> not called often enough to justify changing them away from the >> switch technology.

Re: [HACKERS] More optimization effort?

2017-07-21 Thread Greg Stark
On 21 July 2017 at 20:00, Tom Lane wrote: >> I have, however, decided not to volunteer to be the one who works on >> that project. > > Me either. Any one of these things would require a *lot* of work in > order to have a coherent feature that provided useful behavior across > a bunch of differen

Re: [HACKERS] pg_stop_backup(wait_for_archive := true) on standby server

2017-07-21 Thread Stephen Frost
Masahiko, all, * Masahiko Sawada (sawada.m...@gmail.com) wrote: > On Tue, Jul 18, 2017 at 1:28 PM, Stephen Frost wrote: > > Masahiko, Michael, > > > > * Masahiko Sawada (sawada.m...@gmail.com) wrote: > >> > This is beginning to shape. > >> > >> Sorry, I missed lots of typo in the last patch. All

Re: [HACKERS] Locale dependency in new postgres_fdw test

2017-07-21 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> We could stabilize this test result by forcing lc_messages = C in >> the foreign server options. However, that would lose regression >> coverage of situations where the remote locale is different, which >> doesn't seem like a terribly good thing. > I d

Re: [HACKERS] Locale dependency in new postgres_fdw test

2017-07-21 Thread Alvaro Herrera
Tom Lane wrote: > We could stabilize this test result by forcing lc_messages = C in > the foreign server options. However, that would lose regression > coverage of situations where the remote locale is different, which > doesn't seem like a terribly good thing. I don't understand what is the ben

Re: [HACKERS] segfault in HEAD when too many nested functions call

2017-07-21 Thread Julien Rouhaud
On 21/07/2017 13:40, Andres Freund wrote: > Attached is a > patch that converts just ExecProcNode. The big change in comparison to > the earlier patch is that the assignment of the callback is now done in > the respective ExecInit* routines. As a consequence the ExecProcNode > callbacks now are sta

Re: [HACKERS] Increase Vacuum ring buffer.

2017-07-21 Thread Claudio Freire
On Fri, Jul 21, 2017 at 2:41 PM, Sokolov Yura wrote: > > My friend noticed, that I didn't said why I bother with autovacuum. > Our customers suffers from table bloating. I've made synthetic > bloating test, and started experiments with modifying micro- and > auto-vacuum. My first attempts were to

[HACKERS] Buildfarm failure and dubious coding in predicate.c

2017-07-21 Thread Tom Lane
Buildfarm member culicidae just showed a transient failure in the 9.4 branch: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=culicidae&dt=2017-07-21%2017%3A49%3A37 It's an assert trap, for which the buildfarm helpfully captured a stack trace: #0 __GI_raise (sig=sig@entry=6) at ../sysde

Re: [HACKERS] Syncing sql extension versions with shared library versions

2017-07-21 Thread Andrew Dunstan
On 07/21/2017 04:17 PM, Mat Arye wrote: > Hi All, > > I am developing the TimescaleDB extension for postgres > (https://github.com/timescale/timescaledb) and have some questions > about versioning. First of all, I have to say that the versioning > system on the sql side is wonderful. It's really

[HACKERS] Syncing sql extension versions with shared library versions

2017-07-21 Thread Mat Arye
Hi All, I am developing the TimescaleDB extension for postgres ( https://github.com/timescale/timescaledb) and have some questions about versioning. First of all, I have to say that the versioning system on the sql side is wonderful. It's really simple to write migrations etc. However when thinki

[HACKERS] possible effective_io_concurrency performance regression

2017-07-21 Thread Joshua D. Drake
-hackers, While updating my Postgres performance curriculum I was doing some testing with effective_io_concurrency and I may have found a regression. I am aware that the parameter only works under certain conditions. However, what I appear to have found is that if it is set to anything but 0,

Re: [HACKERS] [PATCH] A hook for session start

2017-07-21 Thread Fabrízio de Royes Mello
On Fri, Jul 21, 2017 at 12:19 PM, Fabrízio de Royes Mello < fabriziome...@gmail.com> wrote: > > > On Fri, Jul 21, 2017 at 10:58 AM, Yugo Nagata wrote: > > > > On Fri, 21 Jul 2017 10:31:57 -0300 > > Fabrízio de Royes Mello wrote: > > > > > On Fri, Jul 21, 2017 at 9:35 AM, Yugo Nagata wrote: > > >

Re: [HACKERS] More optimization effort?

2017-07-21 Thread Tom Lane
Robert Haas writes: > Another very similar (but possibly easier) case is: > select * from pgbench_accounts where aid = 1.0; > This will use a sequential scan rather than an index scan, because the > query optimizer doesn't know that the only integer for which =(int4, > numeric) will return true

Re: [HACKERS] More optimization effort?

2017-07-21 Thread Robert Haas
On Fri, Jul 21, 2017 at 10:33 AM, Tom Lane wrote: > But the bigger picture is that doing something that helps to any > useful extent would require a really substantial amount of new, > datatype- and operator-specific knowledge that doesn't exist in the > system today. And as Craig noted, applying

Re: [HACKERS] Locale dependency in new postgres_fdw test

2017-07-21 Thread Tom Lane
I wrote: > I had not realized (or forgot) that postgres_fdw allows the remote > end to run in whatever lc_messages locale is default for the remote > installation. It's a bit surprising that none of the existing test > cases expose any remote-side messages directly, but evidently not. > We could

Re: [HACKERS] Increase Vacuum ring buffer.

2017-07-21 Thread Sokolov Yura
On 2017-07-21 19:32, Robert Haas wrote: On Fri, Jul 21, 2017 at 4:19 AM, Sokolov Yura wrote: Probably with increased ring buffer there is no need in raising vacuum_cost_limit. Will you admit it? No, I definitely won't admit that. With default settings autovacuum won't write more than ~2.3MB

[HACKERS] Locale dependency in new postgres_fdw test

2017-07-21 Thread Tom Lane
So 8bf58c0d9 immediately blew up in the buildfarm, with eg this on jaguarundi: *** *** 130,136 ALTER SERVER loopback OPTIONS (SET dbname 'no such database'); SELECT c3, c4 FROM ft1 ORDER BY c3, c1 LIMIT 1; -- should fail ERROR: could not connect to server "loopback" ! DETA

Re: [HACKERS] Multi column range partition table

2017-07-21 Thread Dean Rasheed
On 17 July 2017 at 17:37, Dean Rasheed wrote: > On 17 July 2017 at 16:34, Robert Haas wrote: >> Do you want to own this open item, then? >> > OK. > > I need to give the patch another read-through, and then I'll aim to > push it sometime in the next few days. > Committed. Regards, Dean -- Sen

Re: [HACKERS] [COMMITTERS] pgsql: Add a Gather executor node.

2017-07-21 Thread Tom Lane
Alvaro Herrera writes: > Robert Haas wrote: >> I think those top-of-file lists are just annoying, and would prefer to >> rip them out entirely rather than spend time maintaining them. > +1 To the extent that they're just lists of function names, +1. Some of them have some documentation in them,

Re: [HACKERS] [PATCH] A hook for session start

2017-07-21 Thread Jim Mlodgenski
> > > Can a user do anything remotely interesting or useful without hitting > either > > ExecutorStart_hook or ProcessUtility_hook? They can parse queries I guess > > but you could just set your hook up in the parser instead. If you hook > the > > parser all they can do is open an idle session and

Re: [HACKERS] autovacuum can't keep up, bloat just continues to rise

2017-07-21 Thread Joshua D. Drake
On 07/20/2017 08:58 PM, Joshua D. Drake wrote: On 07/19/2017 07:57 PM, Tom Lane wrote: Peter Geoghegan writes: Test 1: 55G/srv/main TPS:955 Test 2: 112G/srv/main TPS:531 (Not sure what happened here, long checkpoint?) Test 3: 109G/srv/main TPS:868 Test 4: 143G TPS:

Re: [HACKERS] PgFDW connection invalidation by ALTER SERVER/ALTER USER MAPPING

2017-07-21 Thread Tom Lane
Ashutosh Bapat writes: > On Fri, Jul 21, 2017 at 10:55 AM, Kyotaro HORIGUCHI > wrote: >> The attached patch differs only in this point. > +1. The patch looks good to me. Pushed with a couple additional changes: we'd all missed that the header comment for GetConnection was obsoleted by this chan

Re: [HACKERS] [PATCH] A hook for session start

2017-07-21 Thread Robert Haas
On Fri, Jul 21, 2017 at 11:10 AM, Craig Ringer wrote: > Don't we have that timestamp already? > > What practical use cases are there for acting post-auth but that can't wait > until the user tries to do something? Have, yes; record, no. > Can a user do anything remotely interesting or useful wit

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2017-07-21 Thread Marina Polyakova
Hello again! Here is the third version of the patch for pgbench thanks to Fabien Coelho comments. As in the previous one, transactions with serialization and deadlock failures are rolled back and retried until they end successfully or their number of tries reaches maximum. Differences from t

Re: [HACKERS] Increase Vacuum ring buffer.

2017-07-21 Thread Robert Haas
On Fri, Jul 21, 2017 at 4:19 AM, Sokolov Yura wrote: > You are one of leadership. I know it is not your job to test every tiny > change a school boy proposed. But here is a lot of people, who waits for > your word. Instead of cooling rush and closing discussions, you may just > say: "please, someo

Re: [HACKERS] [COMMITTERS] pgsql: Add a Gather executor node.

2017-07-21 Thread Andres Freund
On July 21, 2017 5:34:32 PM GMT+02:00, Alvaro Herrera wrote: >Robert Haas wrote: > >> I think those top-of-file lists are just annoying, and would prefer >to >> rip them out entirely rather than spend time maintaining them. > >+1 I'm quite sympathetic to that view. But I think it's either them

Re: [HACKERS] Increase Vacuum ring buffer.

2017-07-21 Thread Sokolov Yura
On 2017-07-20 22:51, Tom Lane wrote: Robert Haas writes: I think that's a valid point. There are also other concerns here - e.g. whether instead of adopting the patch as proposed we ought to (a) use some smaller size, or (b) keep the size as-is but reduce the maximum fraction of shared_buffers

Re: [HACKERS] [COMMITTERS] pgsql: Add a Gather executor node.

2017-07-21 Thread Alvaro Herrera
Robert Haas wrote: > I think those top-of-file lists are just annoying, and would prefer to > rip them out entirely rather than spend time maintaining them. +1 -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -

Re: [HACKERS] [PATCH] A hook for session start

2017-07-21 Thread Fabrízio de Royes Mello
On Fri, Jul 21, 2017 at 10:58 AM, Yugo Nagata wrote: > > On Fri, 21 Jul 2017 10:31:57 -0300 > Fabrízio de Royes Mello wrote: > > > On Fri, Jul 21, 2017 at 9:35 AM, Yugo Nagata wrote: > > > > > > On Fri, 21 Jul 2017 09:53:19 +0800 > > > Craig Ringer wrote: > > > > > > > On 21 July 2017 at 08:42,

Re: [HACKERS] [PATCH] A hook for session start

2017-07-21 Thread Craig Ringer
On 21 Jul. 2017 21:58, "Yugo Nagata" wrote: On Fri, 21 Jul 2017 10:31:57 -0300 Fabrízio de Royes Mello wrote: > On Fri, Jul 21, 2017 at 9:35 AM, Yugo Nagata wrote: > > > > On Fri, 21 Jul 2017 09:53:19 +0800 > > Craig Ringer wrote: > > > > > On 21 July 2017 at 08:42, Robert Haas wrote: > > >

Re: [HACKERS] [GENERAL] huge RAM use in multi-command ALTER of table heirarchy

2017-07-21 Thread Greg Stark
On 20 July 2017 at 14:19, Tom Lane wrote: > Greg Stark writes: > >> Would it be useful to keep in one of the memory checking assertion builds? > > Why? Code that expects to continue accessing a relcache entry's tupdesc > after closing the relcache entry is broken, independently of whether it's >

Re: [HACKERS] WIP Patch: Precalculate stable functions, infrastructure v1

2017-07-21 Thread Marina Polyakova
Like for the previous patches it seems that there is no obvious performance degradation too on regular queries (according to pgbench). pgbench probably isn't a very good test for this sort of thing - it only issues very short-running queries where the cost of evaluating expressions is a relati

Re: [HACKERS] More optimization effort?

2017-07-21 Thread Tom Lane
Tatsuo Ishii writes: > Currently following query does not use an index: > test=# explain select * from pgbench_accounts where aid*100 < 1; > While following one does use the index. > test=# explain select * from pgbench_accounts where aid < 1/100; > Is it worth to make our optimizer a lit

Re: [HACKERS] [PATCH] Make ExplainBeginGroup()/ExplainEndGroup() public.

2017-07-21 Thread Hadi Moshayedi
Title should have been "Make ExplainOpenGroup()/ExplainCloseGroup() public.". Sorry for the misspell.

Re: [HACKERS] autovacuum can't keep up, bloat just continues to rise

2017-07-21 Thread Joshua D. Drake
On 07/20/2017 11:54 PM, Sokolov Yura wrote: On 2017-07-21 06:58, Joshua D. Drake wrote: On 07/19/2017 07:57 PM, Tom Lane wrote: Peter Geoghegan writes: -- Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc PostgreSQL Centered full stack support, consulting and developm

[HACKERS] [PATCH] Make ExplainBeginGroup()/ExplainEndGroup() public.

2017-07-21 Thread Hadi Moshayedi
Hello, The attached patch moves declarations of ExplainBeginGroup()/ExplainEndGroup() from explain.c to explain.h. This can be useful for extensions that need explain groups in their custom-scan explain output. For example, Citus uses groups in its custom explain outputs [1]. But it achieves it

Re: [HACKERS] [PATCH] A hook for session start

2017-07-21 Thread Yugo Nagata
On Fri, 21 Jul 2017 10:31:57 -0300 Fabrízio de Royes Mello wrote: > On Fri, Jul 21, 2017 at 9:35 AM, Yugo Nagata wrote: > > > > On Fri, 21 Jul 2017 09:53:19 +0800 > > Craig Ringer wrote: > > > > > On 21 July 2017 at 08:42, Robert Haas wrote: > > > > > > > On Thu, Jul 20, 2017 at 8:27 AM, Fabrí

Re: [HACKERS] [COMMITTERS] pgsql: Add a Gather executor node.

2017-07-21 Thread Robert Haas
On Fri, Jul 21, 2017 at 7:06 AM, Andres Freund wrote: > On 2015-09-30 23:29:30 +, Robert Haas wrote: >> Add a Gather executor node. >> ... >> src/backend/executor/execProcnode.c | 46 > > I just noticed that this added a new execProcnode dispatch routine, but > didn't add that

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2017-07-21 Thread Alexander Korotkov
On Wed, Jul 19, 2017 at 11:08 PM, Alvaro Herrera wrote: > I'm not entirely sure what's the best way to deal with the polymorphic > problem, but on the other hand as Robert says downthread maybe we > shouldn't be solving it at this stage anyway. So let's step back a bit, > get a patch that works

Re: [HACKERS] [PATCH] Pageinspect - add functions on GIN and GiST indexes from gevel

2017-07-21 Thread Alexander Korotkov
Hi, Alexey! On Fri, Jul 21, 2017 at 3:05 PM, Alexey Chernyshov < a.chernys...@postgrespro.ru> wrote: > the following patch transfers functionality from gevel module > (http://www.sai.msu.su/~megera/wiki/Gevel) which provides functions for > analyzing GIN and GiST indexes to pageinspect. Gevel was

Re: [HACKERS] [PATCH] A hook for session start

2017-07-21 Thread Fabrízio de Royes Mello
On Fri, Jul 21, 2017 at 9:35 AM, Yugo Nagata wrote: > > On Fri, 21 Jul 2017 09:53:19 +0800 > Craig Ringer wrote: > > > On 21 July 2017 at 08:42, Robert Haas wrote: > > > > > On Thu, Jul 20, 2017 at 8:27 AM, Fabrízio de Royes Mello > > > wrote: > > > > I'm not sure your real needs but doesn't it

Re: [HACKERS] Adding -E switch to pg_dumpall

2017-07-21 Thread Michael Paquier
On Thu, Jul 20, 2017 at 11:17 AM, Fabien COELHO wrote: > Ok for me. I switched the status to "Ready for committers". Thanks for the review, Fabien. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/

Re: [HACKERS] [PATCH] Make sure all statistics is sent after a few DML are performed

2017-07-21 Thread Yugo Nagata
On Fri, 21 Jul 2017 04:58:47 -0700 Andres Freund wrote: > Hi, > > (please don't top-reply on this list) > > On 2017-07-19 14:04:39 +0900, Yugo Nagata wrote: > > On Tue, 18 Jul 2017 10:10:49 -0400 > > Tom Lane wrote: > > > > Thank you for your comments. I understand the problem of my proposal

Re: [HACKERS] [PATCH] A hook for session start

2017-07-21 Thread Yugo Nagata
On Fri, 21 Jul 2017 09:53:19 +0800 Craig Ringer wrote: > On 21 July 2017 at 08:42, Robert Haas wrote: > > > On Thu, Jul 20, 2017 at 8:27 AM, Fabrízio de Royes Mello > > wrote: > > > I'm not sure your real needs but doesn't it material for improve Event > > > Triggers??? > > > > I've thought ab

Re: [HACKERS] xlogfilename

2017-07-21 Thread Yugo Nagata
On Fri, 21 Jul 2017 10:11:05 +0800 Craig Ringer wrote: > On 20 July 2017 at 21:33, Yugo Nagata wrote: > > > On Thu, 20 Jul 2017 11:02:25 +0200 > > Michael Paquier wrote: > > > > > On Thu, Jul 20, 2017 at 10:58 AM, DEV_OPS wrote: > > > > I think you may reference to function: pg_xlogfile_name

Re: [HACKERS] [PATCH] Make sure all statistics is sent after a few DML are performed

2017-07-21 Thread Andres Freund
Hi, (please don't top-reply on this list) On 2017-07-19 14:04:39 +0900, Yugo Nagata wrote: > On Tue, 18 Jul 2017 10:10:49 -0400 > Tom Lane wrote: > > Thank you for your comments. I understand the problem of my proposal > patch. Does that mean you're trying to rewrite it in the way that was sug

Re: [HACKERS] [WIP] Zipfian distribution in pgbench

2017-07-21 Thread Alik Khilazhev
Hello! I realized that I was sending emails as HTML and latest patch is not visible in the archive now. That’s why I am attaching it again. I am sorry for that. pgbench-zipf-05v.patch Description: Binary data — Thanks and Regards, Alik Khilazhev Postgres Professional: http://www.postgrespro.

Re: [HACKERS] segfault in HEAD when too many nested functions call

2017-07-21 Thread Andres Freund
On 2017-07-18 16:53:43 -0400, Tom Lane wrote: > Andres Freund writes: > > ... If we were to go this route we'd have to probably move > > the callback assignment into the ExecInit* routines, and possibly > > replace the switch in ExecInitNode() with another callback, assigned in > > make_*, and im

Re: [HACKERS] Small improvement to compactify_tuples

2017-07-21 Thread Sokolov Yura
On 2017-05-17 17:46, Sokolov Yura wrote: Alvaro Herrera писал 2017-05-15 20:13: As I understand, these patches are logically separate, so putting them together in a single file isn't such a great idea. If you don't edit the patches further, then you're all set because we already have the previo

Re: [HACKERS] Mishandling of WCO constraints in direct foreign table modification

2017-07-21 Thread Etsuro Fujita
On 2017/07/21 17:18, Kyotaro HORIGUCHI wrote: At Fri, 21 Jul 2017 12:00:03 +0900, Etsuro Fujita wrote in <15aa9936-9bd8-c9e3-7ca1-394861073...@lab.ntt.co.jp> Attached is the second version which updated docs in postgres-fdw.sgml as well. !no local joins for the query, no row-level local

Re: [HACKERS] Bug in ExecModifyTable function and trigger issues for foreign tables

2017-07-21 Thread Etsuro Fujita
On 2017/07/20 11:21, Etsuro Fujita wrote: On 2017/07/19 23:36, Tom Lane wrote: Please put the responsibility of doing const-expression simplification in these cases somewhere closer to where the problem is being created. It would be reasonable that it's the FDW's responsibility to do that con

Re: [HACKERS] [WIP] Zipfian distribution in pgbench

2017-07-21 Thread Alik Khilazhev
Hmmm. On second thought, maybe one or the other is enough, either restrict the parameter to values where the approximation is good, or put out a clear documentation about when the approximation is not very good, but it may be still useful even if not precise.So I would be in favor of expanding the

Re: [HACKERS] Increase Vacuum ring buffer.

2017-07-21 Thread Sokolov Yura
On 2017-07-20 20:59, Robert Haas wrote: If you want something changed, it's your job to do that testing. I've been testing for two weeks before I wrote to pgsql-hackers. And I wrote some highlevel results in first letter. I haven't noticed transactions slowdown from increased vacuum ring buffe

Re: [HACKERS] Mishandling of WCO constraints in direct foreign table modification

2017-07-21 Thread Kyotaro HORIGUCHI
At Fri, 21 Jul 2017 12:00:03 +0900, Etsuro Fujita wrote in <15aa9936-9bd8-c9e3-7ca1-394861073...@lab.ntt.co.jp> > On 2017/07/21 3:24, Robert Haas wrote: > > I think that's reasonable. This should be committed and back-patched > > to 9.6, right? > > Yeah, because direct modify was introduced in

Re: [HACKERS] PgFDW connection invalidation by ALTER SERVER/ALTER USER MAPPING

2017-07-21 Thread Ashutosh Bapat
On Fri, Jul 21, 2017 at 10:55 AM, Kyotaro HORIGUCHI wrote: > At Thu, 20 Jul 2017 18:15:42 -0400, Tom Lane wrote in > <18927.1500588...@sss.pgh.pa.us> >> This seems like overkill. We can test it reasonably easily within the >> existing framework, as shown in the attached patch. I'm also fairly

Re: [HACKERS] [TRAP: FailedAssertion] causing server to crash

2017-07-21 Thread Kyotaro HORIGUCHI
At Fri, 21 Jul 2017 11:39:38 +0530, Neha Sharma wrote in > Here is the back trace from the core dump attached. > > (gdb) bt > #0 0x7f4a71424495 in raise () from /lib64/libc.so.6 > #1 0x7f4a71425c75 in abort () from /lib64/libc.so.6 > #2 0x009dc18a in ExceptionalCondition (con