Re: Problem while setting the fpw with SIGHUP

2018-04-13 Thread Kyotaro HORIGUCHI
At Fri, 13 Apr 2018 13:47:51 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20180413.134751.76149471.horiguchi.kyot...@lab.ntt.co.jp> > At Fri, 13 Apr 2018 08:31:02 +0530, Amit Kapila > wrote in > > On Fri, Apr 13, 2018 at 6:59 AM, Michael Paquier > > wrote: > > > On Thu, Apr 12,

Re: Problem while setting the fpw with SIGHUP

2018-04-13 Thread Kyotaro HORIGUCHI
Sorry, the patch attached to the previous main is slightly old. The attached is the correct one. # They differ only in some phrase in a comment. At Fri, 13 Apr 2018 17:28:40 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20180413.172840.228724367.horiguchi.kyot...@lab.ntt.co.jp>

Re: wal_consistency_checking reports an inconsistency on master branch

2018-04-13 Thread amul sul
On Fri, Apr 13, 2018 at 9:06 AM, Andres Freund wrote: > On 2018-04-13 12:29:21 +0900, Amit Langote wrote: >> On 2018/04/13 7:36, Peter Geoghegan wrote: >> > On Thu, Apr 12, 2018 at 11:47 AM, Peter Geoghegan wrote: >> >> In short, it looks like the tests added to update.sql by commit >> >> 2f17844

Re: Boolean partitions syntax

2018-04-13 Thread Amit Langote
Horiguchi-san, Thanks for the latest patch. On 2018/04/12 13:12, Kyotaro HORIGUCHI wrote: > Thank you for verification and the revised patch. The format is > fine and the fix is correct but I noticed that I forgot to remove > plural S's from error messages. The attached is the version with > the

Re: Built-in connection pooling

2018-04-13 Thread Konstantin Knizhnik
On 06.04.2018 20:03, Konstantin Knizhnik wrote: On 06.04.2018 20:00, Konstantin Knizhnik wrote: Attached please find new version of the patch with  several bug fixes + support of more than one session pools associated with different ports. Now it is possible to make postmaster listen severa

Re: wal_consistency_checking reports an inconsistency on master branch

2018-04-13 Thread Michael Paquier
On Fri, Apr 13, 2018 at 02:15:35PM +0530, amul sul wrote: > I have looked into this and found that the issue is in heap_xlog_delete -- we > have missed to set the correct offset number from the target_tid when > XLH_DELETE_IS_PARTITION_MOVE flag is set. Oh, this looks good to me. So when a row wa

Re: wal_consistency_checking reports an inconsistency on master branch

2018-04-13 Thread Amit Langote
On 2018/04/13 19:08, Michael Paquier wrote: > On Fri, Apr 13, 2018 at 02:15:35PM +0530, amul sul wrote: >> I have looked into this and found that the issue is in heap_xlog_delete -- we >> have missed to set the correct offset number from the target_tid when >> XLH_DELETE_IS_PARTITION_MOVE flag is s

Re: crash with sql language partition support function

2018-04-13 Thread Amit Langote
On 2018/04/13 3:10, Alvaro Herrera wrote: > Alvaro Herrera wrote: > >> I'm dealing with this now -- will push shortly. The sane thing to do is >> backpatch my previous memcxt fixes, since your patch introduces a >> problem that we discussed with that other patch, namely that you would >> leak the

Re: crash with sql language partition support function

2018-04-13 Thread Alvaro Herrera
Amit Langote wrote: > On 2018/04/13 6:58, Alvaro Herrera wrote: > > After going over your patch, I think you went slightly overboard here. > > Or maybe not, but this patch is so large that it's hard to form an > > opinion about it. > > It's mostly code movement, but there are some other changes a

Re: pgsql: New files for MERGE

2018-04-13 Thread David Steele
On 4/11/18 5:28 PM, Simon Riggs wrote: > On 11 April 2018 at 21:00, Simon Riggs wrote: > >> "Commence primary ignition." > > Revert patch attached > > Mostly Pavan, minor cleanup by me I have moved this entry to the next CF in Waiting for Author state. Regards, -- -David da...@pgmasters.net

Re: WIP: Covering + unique indexes.

2018-04-13 Thread Andrey Borodin
Hi! > 12 апр. 2018 г., в 21:21, Teodor Sigaev написал(а): I was adapting tests for GiST covering index and found out that REINDEX test is somewhat not a REINDEX test... I propose following micropatch. Best regards, Andrey Borodin. fix-reindex-test.diff Description: Binary data

Re: Covering GiST indexes

2018-04-13 Thread Alexander Korotkov
On Thu, Apr 12, 2018 at 11:20 PM, Darafei "Komяpa" Praliaskouski < m...@komzpa.net> wrote: > Another thing that could be done for PostGIS geometries is just another >> opclass which >> stores geometries "as is" in leafs. As I know, geometries contain MBRs >> inside their >> own, so there is no ne

Re: vacuum_cost_limit doc description patch

2018-04-13 Thread Martín Marqués
El 11/04/18 a las 02:04, David Rowley escribió: > On 11 April 2018 at 09:13, Martín Marqués wrote: >> This is a patch to add some further description, plus the upper and >> lower limits it has. > > Hi, > > + for vacuum_cost_delay. The parameter can take a value > between 1 and 1. >

Re: Covering GiST indexes

2018-04-13 Thread Andrey Borodin
> 12 апр. 2018 г., в 17:03, Teodor Sigaev написал(а): > > Interesting work. I don't have a time now to learn deep your patch, so, add > it to next commitfest, pls. Thanks! Sure, I'll add it. > First of all I'd like to see more tests in patch, not only CREATE INDEX. Here's V2, with basic set

Re: Postgres stucks in deadlock detection

2018-04-13 Thread Konstantin Knizhnik
Updated patch is attached. On 05.04.2018 11:03, Konstantin Knizhnik wrote: Hi, Thank for your feedback. On 04.04.2018 21:15, Andres Freund wrote: Hi, On 2018-04-04 11:54:14 +0300, Konstantin Knizhnik wrote: Several times we and our customers are suffered from the problem that Postgres got st

Re: Instability in partition_prune test?

2018-04-13 Thread Alvaro Herrera
David Rowley wrote: > On 13 April 2018 at 14:41, David Rowley wrote: > > I'll just need to go think about how we can make the test stable now. > > Thomas and I discussed this a bit off-list. > > The attached basically adds: > > set max_parallel_workers = 0; > > before the Parallel Append tests

Re: Instability in partition_prune test?

2018-04-13 Thread Tom Lane
David Rowley writes: > The attached basically adds: > set max_parallel_workers = 0; It seems quite silly to be asking for a parallel plan and then insisting it not run in parallel. Maybe the right solution is to strip out the loop_count from what's printed. We've already done that sort of thing

Re: Instability in partition_prune test?

2018-04-13 Thread Alvaro Herrera
Tom Lane wrote: > David Rowley writes: > > The attached basically adds: > > set max_parallel_workers = 0; > > It seems quite silly to be asking for a parallel plan and then insisting > it not run in parallel. The idea is to use the parallel append code, but run it in the leader. Now that you me

Re: [HACKERS] path toward faster partition pruning

2018-04-13 Thread Alvaro Herrera
Robert Haas wrote: > On Wed, Apr 11, 2018 at 8:35 AM, Alvaro Herrera > wrote: > > Here's an idea. Why don't we move the function/opclass creation lines > > to insert.sql, without the DROPs, and use the same functions/opclasses > > in the three tests insert.sql, alter_table.sql, hash_part.sql and

Re: Postgres stucks in deadlock detection

2018-04-13 Thread Evgeniy Shishkin
> On Apr 5, 2018, at 11:03, Konstantin Knizhnik > wrote: > > From the other side, the fact that any of this > > a) increasing deadlock timeout > b) avoid concurrent execution of deadlock check > c) perform first stage of deadlock check under shared lock > > have very noticeable effect on the

Re: Postgres stucks in deadlock detection

2018-04-13 Thread Andres Freund
Hi, On 2018-04-13 16:43:09 +0300, Konstantin Knizhnik wrote: > Updated patch is attached. > + /* > + * Ensure that only one backend is checking for deadlock. > + * Otherwise under high load cascade of deadlock timeout expirations > can cause stuck of Postgres. > + */ > + if

Re: [PATCH] pg_hba.conf : new auth option : clientcert=verify-full

2018-04-13 Thread Julian Markwort
On Tue, 2018-04-10 at 18:35 +0200, Magnus Hagander wrote: > As Peter mentionde, there are in src/test/ssl. I forgot about those, > but yes, it would be useful to have that. I've added three tests: - verify-full specified, CN and username match -- should connect ok - verify-full specified, CN and us

Re: Overcoming SELECT ... FOR UPDATE permission restrictions

2018-04-13 Thread Tom Lane
Alexander Lakhin writes: > Can you please explain, is this a bug or intended behaviour? I'd say it's a bug. The permissions restriction should apply even with the intermediate view. After some rooting around, it seems like this can be blamed on wrong order-of-operations in ApplyRetrieveRule().

Re: Instability in partition_prune test?

2018-04-13 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> It seems quite silly to be asking for a parallel plan and then insisting >> it not run in parallel. > Now that you mention it, this probably decreases coverage for the > choose_next_subplan_for_worker function. Yeah, loss of executor code coverage was

Re: Built-in connection pooling

2018-04-13 Thread Nikolay Samokhvalov
On Fri, Apr 13, 2018 at 2:59 AM, Konstantin Knizhnik < k.knizh...@postgrespro.ru> wrote: > > Development in built-in connection pooling will be continued in > https://github.com/postgrespro/postgresql.builtin_pool.git > I am not going to send new patches to hackers mailing list any more. > Why?

Re: Postgres stucks in deadlock detection

2018-04-13 Thread Konstantin Knizhnik
On 13.04.2018 18:41, Andres Freund wrote: Hi, On 2018-04-13 16:43:09 +0300, Konstantin Knizhnik wrote: Updated patch is attached. + /* +* Ensure that only one backend is checking for deadlock. +* Otherwise under high load cascade of deadlock timeout expirations can caus

Re: Commit 4dba331cb3 broke ATTACH PARTITION behaviour.

2018-04-13 Thread Robert Haas
On Thu, Apr 12, 2018 at 3:59 PM, Alvaro Herrera wrote: > Robert Haas wrote: >> Please revert the part of this commit that changed the lock level. > > Done. Thanks. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Native partitioning tablespace inheritance

2018-04-13 Thread Robert Haas
On Thu, Apr 12, 2018 at 3:55 PM, Keith Fiske wrote: > Your last example is why I proposed taking the TABLESPACE defined on the > parent and applying it to the children. Then all the children have one > defined and nothing breaks as long as all tablespaces are properly defined > as part of the rest

Re: [HACKERS] Runtime Partition Pruning

2018-04-13 Thread Robert Haas
On Thu, Apr 12, 2018 at 6:01 PM, David Rowley wrote: > On 13 April 2018 at 04:57, Robert Haas wrote: >> BTW, looking at ExecSetupPartitionPruneState: >> >> /* >> * Create a sub memory context which we'll use when making calls to >> the >> * query planner's function to d

Re: Problem while setting the fpw with SIGHUP

2018-04-13 Thread Robert Haas
On Thu, Apr 12, 2018 at 9:29 PM, Michael Paquier wrote: > Still does it matter when the change is effective? I don't really care deeply about when the change takes effect, but I do care about whether the time when the system *says* the change took effect is the same as when it *actually* took eff

Re: [HACKERS] path toward faster partition pruning

2018-04-13 Thread Robert Haas
On Fri, Apr 13, 2018 at 10:50 AM, Alvaro Herrera wrote: > Robert Haas wrote: >> On Wed, Apr 11, 2018 at 8:35 AM, Alvaro Herrera >> wrote: >> > Here's an idea. Why don't we move the function/opclass creation lines >> > to insert.sql, without the DROPs, and use the same functions/opclasses >> > i

Re: Interesting paper: Contention-Aware Lock Scheduling

2018-04-13 Thread Konstantin Knizhnik
On 31.01.2018 22:48, Thomas Munro wrote: Hi hackers, I saw this today: http://www.vldb.org/pvldb/vol11/p648-tian.pdf It describes the "LDSF" (largest-dependency-set-first) lock scheduling algorithm and related work, as an alternative to the FIFO scheduling used by PostgreSQL and most other RD

Re: Native partitioning tablespace inheritance

2018-04-13 Thread Evgeniy Shishkin
> On Apr 13, 2018, at 20:01, Robert Haas wrote: > > Well, let me put it this way. Someone who assumes that partitioning > works like inheritance except where we've explicitly made it work > differently will be correct. Someone who assumes something else will > be incorrect. I'm not saying th

Re: MinIndexTupleSize seems slightly wrong

2018-04-13 Thread Tom Lane
Kyotaro HORIGUCHI writes: > At Thu, 12 Apr 2018 17:26:33 -0700, Peter Geoghegan wrote in > >> However, that at least seems questionable to me. See _bt_pgaddtup() >> for a simple example of this -- "minus infinity" items on internal >> pages are sized sizeof(IndexTupleData). > MaxIndexTuplesPer

Re: Interesting paper: Contention-Aware Lock Scheduling

2018-04-13 Thread Garym
LDFS does show improvements for certain workloads, however it sacrifices temporal order and may interfere with historical analytics. If applications can tolerate ambiguous order of processing, it shows good gains. Sent from my iPad > On Apr 13, 2018, at 11:14 AM, Konstantin Knizhnik > wrote:

Re: Interesting paper: Contention-Aware Lock Scheduling

2018-04-13 Thread Evgeniy Shishkin
> On Apr 13, 2018, at 20:46, Garym wrote: > > LDFS does show improvements for certain workloads, however it sacrifices > temporal order and may interfere with historical analytics. If applications > can tolerate ambiguous order of processing, it shows good gains. AFAIK, we don't guarantee o

Re: Interesting paper: Contention-Aware Lock Scheduling

2018-04-13 Thread Garym
I did testing on 9.6 and 10. Outside of slaves at distance, it does demonstrate consistent OOA operation whether intentional/enforced or not. :) Sent from my iPad > On Apr 13, 2018, at 11:50 AM, Evgeniy Shishkin wrote: > > > >> On Apr 13, 2018, at 20:46, Garym wrote: >> >> LDFS does show

Re: crash with sql language partition support function

2018-04-13 Thread Alvaro Herrera
I think this is a good improvement. On top of that, I propose a new file partitioning/partdefs.h with the following approximate contents. This reduces cross-inclusion of headers to the minimum. I'm dealing with the fallout from this now, will post a complete patch shortly. /*---

Re: Postgres stucks in deadlock detection

2018-04-13 Thread Andres Freund
Hi, On 2018-04-13 19:13:07 +0300, Konstantin Knizhnik wrote: > On 13.04.2018 18:41, Andres Freund wrote: > > On 2018-04-13 16:43:09 +0300, Konstantin Knizhnik wrote: > > > Updated patch is attached. > > > + /* > > > + * Ensure that only one backend is checking for deadlock. > > > + * Otherwise u

Re: crash with sql language partition support function

2018-04-13 Thread Andres Freund
Hi, On 2018-04-13 15:08:30 -0300, Alvaro Herrera wrote: > I think this is a good improvement. On top of that, I propose a new > file partitioning/partdefs.h with the following approximate contents. > This reduces cross-inclusion of headers to the minimum. I'm dealing > with the fallout from this

Re: partitioning code reorganization

2018-04-13 Thread Alvaro Herrera
Further thought: compute_hash_value and satisfies_hash_partition both belong in partbounds.c. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Native partitioning tablespace inheritance

2018-04-13 Thread Michael Paquier
On Fri, Apr 13, 2018 at 08:23:22PM +0300, Evgeniy Shishkin wrote: >> On Apr 13, 2018, at 20:01, Robert Haas wrote: >> >> Well, let me put it this way. Someone who assumes that partitioning >> works like inheritance except where we've explicitly made it work >> differently will be correct. Someo

Proposal: Adding json logging

2018-04-13 Thread David Arnold
*Hello,* I'm new here. I'm David and would describe myself as an ambitious newbie, so please take my suggestion with a grain of salt. *Use case:* I find it difficult to properly parse postgres logs into some kind of log aggregator (I use fluent bit). My two standard option are standard and csvlog

Re: Proposal: Adding json logging

2018-04-13 Thread Michael Paquier
On Sat, Apr 14, 2018 at 12:00:16AM +, David Arnold wrote: > I'm new here. I'm David and would describe myself as an ambitious newbie, > so please take my suggestion with a grain of salt. Welcome here. > I asked Michael Paquier about his solution: > https://github.com/michaelpq/pg_plugins/tre