Re: [HACKERS] pg_background contrib module proposal

2016-12-12 Thread amul sul
On Mon, Dec 12, 2016 at 10:47 PM, Andrew Borodin wrote: > Hi! > Thanks a lot for your review. > Just in case you'd like to include sleepsort as a test, here it is > wrapped as a regression test(see attachment). But it has serious > downside: it runs no less than 5 seconds.

Re: [HACKERS] Declarative partitioning - another take

2016-12-12 Thread Amit Langote
Hi, On 2016/12/13 2:45, Dmitry Ivanov wrote: > Huh, this code is broken as well. We have to ignore partitions that don't > have any subpartitions. Patch is attached below (v2). Good catch and thanks a lot for the patch! I have revised it a bit and added some explanatory comments to that

Re: [HACKERS] Parallel bitmap heap scan

2016-12-12 Thread Dilip Kumar
On Sat, Dec 10, 2016 at 5:36 PM, Amit Kapila wrote: > Few assorted comments: Thanks for the review > > 1. > + else if (needWait) > + { > + /* Add ourself to wait queue */ > + ConditionVariablePrepareToSleep(>cv); > + queuedSelf = true; > + } > > With the committed

[HACKERS] Proposal: add error message in backend/catalog/index.c

2016-12-12 Thread Ioseph Kim
Hi, I propose to append an error message when index name and table name are same. example: postgres@postgres=# create table t (a int not null, constraint t primary key (a)); ERROR: relation "t" already exists End users will confusing pretty, because if users meet this message, users

Re: [HACKERS] Radix tree for character conversion

2016-12-12 Thread Kyotaro HORIGUCHI
Hello, I looked on this closer. The attached is the revised version of this patch. At Mon, 05 Dec 2016 19:29:54 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20161205.192954.12189.horiguchi.kyot...@lab.ntt.co.jp> > Apart from the aboves, I have

Re: [HACKERS] Logical Replication WIP

2016-12-12 Thread Petr Jelinek
On 13/12/16 03:26, Petr Jelinek wrote: > On 13/12/16 02:41, Andres Freund wrote: >> On 2016-12-10 08:48:55 +0100, Petr Jelinek wrote: >> >>> +static List * >>> +OpenTableList(List *tables) >>> +{ >>> + List *relids = NIL; >>> + List *rels = NIL; >>> + ListCell *lc; >>> + >>> +

Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol

2016-12-12 Thread Michael Paquier
On Tue, Dec 13, 2016 at 10:43 AM, Michael Paquier wrote: > On Mon, Dec 12, 2016 at 11:39 PM, Heikki Linnakangas wrote: >> A few couple more things that caught my eye while hacking on this: Looking at what we have now, in the branch... >> * Use

[HACKERS] Fixing matching of boolean index columns to sort ordering

2016-12-12 Thread Tom Lane
The attached patch addresses the complaint raised in https://www.postgresql.org/message-id/CAHt_Luuao4gd6De61GryK=2ff-mtghzjqffdjz02usdvqym...@mail.gmail.com namely, that if you have an index on, say, integer columns i and j, then the planner will figure out that it can use an indexscan with no

Re: [HACKERS] Re: [sqlsmith] FailedAssertion("!(XLogCtl->Insert.exclusiveBackup)", File: "xlog.c", Line: 10200)

2016-12-12 Thread Fujii Masao
On Tue, Nov 8, 2016 at 11:18 AM, Kyotaro HORIGUCHI wrote: > Hello, > > At Sat, 5 Nov 2016 21:18:42 +0900, Michael Paquier > wrote in >

Re: [HACKERS] Quorum commit for multiple synchronous replication.

2016-12-12 Thread Amit Kapila
On Mon, Dec 12, 2016 at 9:54 PM, Masahiko Sawada wrote: > On Mon, Dec 12, 2016 at 9:52 PM, Fujii Masao wrote: >> On Mon, Dec 12, 2016 at 9:31 PM, Masahiko Sawada >> wrote: >>> On Sat, Dec 10, 2016 at 5:17 PM, Amit Kapila

Re: [HACKERS] background sessions

2016-12-12 Thread Petr Jelinek
On 12/12/16 16:29, Robert Haas wrote: > On Mon, Dec 12, 2016 at 10:02 AM, Craig Ringer > wrote: >> On 12 Dec. 2016 21:55, "Robert Haas" wrote: >> On Sun, Dec 11, 2016 at 5:38 AM, Andrew Borodin >> wrote: >>> 1. As far as

Re: [HACKERS] WIP: Faster Expression Processing and Tuple Deforming (including JIT)

2016-12-12 Thread CK Tan
On Mon, Dec 12, 2016 at 6:14 PM, Andres Freund wrote: > > > For Q1 I think the bigger win is JITing the transition function > invocation in advance_aggregates/transition_function - that's IIRC where > the biggest bottleneck lies. > Yeah, we bundle the agg core into our expr

Re: [HACKERS] Logical Replication WIP

2016-12-12 Thread Petr Jelinek
On 13/12/16 02:41, Andres Freund wrote: > On 2016-12-10 08:48:55 +0100, Petr Jelinek wrote: > >> diff --git a/src/backend/catalog/pg_publication.c >> b/src/backend/catalog/pg_publication.c >> new file mode 100644 >> index 000..e3560b7 >> --- /dev/null >> +++

Re: [HACKERS] Hash Indexes

2016-12-12 Thread Amit Kapila
On Tue, Dec 13, 2016 at 2:51 AM, Robert Haas wrote: > On Sun, Dec 11, 2016 at 1:24 AM, Amit Kapila wrote: >> >> With above fixes, the test ran successfully for more than a day. > > Instead of doing this: > > +_hash_chgbufaccess(rel, bucket_buf,

Re: [HACKERS] WIP: Faster Expression Processing and Tuple Deforming (including JIT)

2016-12-12 Thread Andres Freund
Hi, On 2016-12-12 18:11:13 -0800, CK Tan wrote: > Andres, > > dev (no jiting): > > Time: 30343.532 ms > > > dev (jiting): > > SET jit_tuple_deforming = on; > > SET jit_expressions = true; > > > > Time: 24439.803 ms > > FYI, ~20% improvement for TPCH Q1 is consistent with what we find when we >

Re: [HACKERS] WIP: Faster Expression Processing and Tuple Deforming (including JIT)

2016-12-12 Thread CK Tan
Andres, > dev (no jiting): > Time: 30343.532 ms > dev (jiting): > SET jit_tuple_deforming = on; > SET jit_expressions = true; > > Time: 24439.803 ms FYI, ~20% improvement for TPCH Q1 is consistent with what we find when we only jit expression. Cheers, -cktan

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2016-12-12 Thread Craig Ringer
On 13 December 2016 at 09:13, Michael Paquier wrote: > On Tue, Dec 13, 2016 at 10:05 AM, Craig Ringer wrote: >> We should probably expose a proc_type or something, with types: >> >> * client_backend >> * bgworker >> * walsender >> * autovacuum >>

Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol

2016-12-12 Thread Michael Paquier
On Mon, Dec 12, 2016 at 11:39 PM, Heikki Linnakangas wrote: > A few couple more things that caught my eye while hacking on this: > > 1. We don't use SASLPrep to scrub username's and passwords. That's by > choice, for usernames, because historically in PostgreSQL usernames can be

Re: [HACKERS] Logical Replication WIP

2016-12-12 Thread Andres Freund
On 2016-12-10 08:48:55 +0100, Petr Jelinek wrote: > diff --git a/src/backend/catalog/pg_publication.c > b/src/backend/catalog/pg_publication.c > new file mode 100644 > index 000..e3560b7 > --- /dev/null > +++ b/src/backend/catalog/pg_publication.c > + > +Datum

Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol

2016-12-12 Thread Craig Ringer
On 12 December 2016 at 22:39, Heikki Linnakangas wrote: > * Throw an error if an "authorization identity" is given. ATM, we just > ignore it, but seems better to reject the attempt than do something that > might not be what the client expects. Yeah. That might be an opportunity

Re: [HACKERS] PATCH: two slab-like memory allocators

2016-12-12 Thread Petr Jelinek
On 13/12/16 01:45, Tomas Vondra wrote: > On 12/12/2016 11:39 PM, Tomas Vondra wrote: >> On 12/12/2016 05:05 AM, Petr Jelinek wrote: >>> >>> I'd be happy with this patch now (as in committer ready) except that it >>> does have some merge conflicts after the recent commits, so rebase is >>> needed.

Re: [HACKERS] background sessions

2016-12-12 Thread Craig Ringer
On 12 December 2016 at 23:29, Robert Haas wrote: > On Mon, Dec 12, 2016 at 10:02 AM, Craig Ringer > wrote: >> On 12 Dec. 2016 21:55, "Robert Haas" wrote: >> On Sun, Dec 11, 2016 at 5:38 AM, Andrew Borodin

Re: [HACKERS] jsonb problematic operators

2016-12-12 Thread Michael Paquier
On Mon, Dec 12, 2016 at 10:22 PM, Greg Stark wrote: > On 12 December 2016 at 04:59, Craig Ringer wrote: >> I didn't realise Pg's use of ? was that old, so thanks. That makes >> offering alternatives much less appealing. > > One option might be for Postgres

Re: [HACKERS] pg_background contrib module proposal

2016-12-12 Thread Craig Ringer
On 13 December 2016 at 01:17, Andrew Borodin wrote: > 6. Cancelation: a way to signal to background query that it's time to > quit gracefully. That at least should be fuss-free. SIGTERM it, and make sure the worker does CHECK_FOR_INTERRUPTS() in regularly-hit places and

Re: [HACKERS] Declarative partitioning - another take

2016-12-12 Thread Amit Langote
On 2016/12/12 23:14, Peter Eisentraut wrote: > On 12/7/16 1:20 PM, Robert Haas wrote: >> I've committed 0001 - 0006 with that correction and a few other >> adjustments. There's plenty of room for improvement here, and almost >> certainly some straight-up bugs too, but I think we're at a point >>

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2016-12-12 Thread Michael Paquier
On Tue, Dec 13, 2016 at 10:05 AM, Craig Ringer wrote: > We should probably expose a proc_type or something, with types: > > * client_backend > * bgworker > * walsender > * autovacuum > * checkpointer > * bgwriter A text field is adapted then, more than a single character.

Re: [HACKERS] Proposal : Parallel Merge Join

2016-12-12 Thread Dilip Kumar
On Tue, Dec 13, 2016 at 12:11 AM, Robert Haas wrote: > > Hmm, so it seems my initial guess that we didn't need to bother > generating such paths was wrong. Oops. > > This patch is hard to read because it is reorganizing a bunch of code > as well as adding new

Re: [HACKERS] PATCH: recursive json_populate_record()

2016-12-12 Thread Michael Paquier
On Tue, Dec 13, 2016 at 9:38 AM, Nikita Glukhov wrote: > It also fixes the following errors/inconsistencies caused by lost quoting of > string json values: > > [master]=# select * from json_to_record('{"js": "a"}') as rec(js json); > ERROR: invalid input syntax for type

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2016-12-12 Thread Craig Ringer
On 13 December 2016 at 01:45, Kevin Grittner wrote: > On Mon, Dec 12, 2016 at 10:33 AM, Andres Freund wrote: >> On 2016-12-12 13:26:32 -0300, Alvaro Herrera wrote: >>> Robert Haas wrote: > 1. Show all processes that have a PGPROC in pg_stat_activity, >

Re: [HACKERS] Logical Replication WIP

2016-12-12 Thread Petr Jelinek
On 13/12/16 01:33, Andres Freund wrote: > HJi, > > On 2016-12-12 09:18:48 -0500, Peter Eisentraut wrote: >> On 12/8/16 4:10 PM, Petr Jelinek wrote: >>> On 08/12/16 20:16, Peter Eisentraut wrote: On 12/6/16 11:58 AM, Peter Eisentraut wrote: > On 12/5/16 6:24 PM, Petr Jelinek wrote: >>

Re: [HACKERS] PATCH: two slab-like memory allocators

2016-12-12 Thread Tomas Vondra
On 12/12/2016 11:39 PM, Tomas Vondra wrote: On 12/12/2016 05:05 AM, Petr Jelinek wrote: I'd be happy with this patch now (as in committer ready) except that it does have some merge conflicts after the recent commits, so rebase is needed. Attached is a rebased version of the patch, resolving

[HACKERS] PATCH: recursive json_populate_record()

2016-12-12 Thread Nikita Glukhov
Hi. The first attached patch implements recursive processing of nested objects and arrays in json[b]_populate_record[set](), json[b]_to_record[set](). See regression tests for examples. It also fixes the following errors/inconsistencies caused by lost quoting of string json values:

Re: [HACKERS] Logical Replication WIP

2016-12-12 Thread Andres Freund
HJi, On 2016-12-12 09:18:48 -0500, Peter Eisentraut wrote: > On 12/8/16 4:10 PM, Petr Jelinek wrote: > > On 08/12/16 20:16, Peter Eisentraut wrote: > >> On 12/6/16 11:58 AM, Peter Eisentraut wrote: > >>> On 12/5/16 6:24 PM, Petr Jelinek wrote: > I think that the removal of changes to

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2016-12-12 Thread Michael Paquier
On Tue, Dec 13, 2016 at 1:45 AM, Robert Haas wrote: > And now I'm noticing that Michael Paquier previously started a thread > on this problem which I failed to note before starting this one: > >

Re: [HACKERS] snapbuild woes

2016-12-12 Thread Petr Jelinek
On 12/12/16 23:33, Andres Freund wrote: > On 2016-12-12 23:27:30 +0100, Petr Jelinek wrote: >> On 12/12/16 22:42, Andres Freund wrote: >>> Hi, >>> >>> On 2016-12-10 23:10:19 +0100, Petr Jelinek wrote: Hi, First one is outright bug, which has to do with how we track running

Re: [HACKERS] [OSSTEST PATCH 0/1] PostgreSQL db: Retry on constraint violation

2016-12-12 Thread Thomas Munro
On Tue, Dec 13, 2016 at 10:47 AM, Kevin Grittner wrote: > On Mon, Dec 12, 2016 at 1:06 PM, Kevin Grittner wrote: >> On Mon, Dec 12, 2016 at 12:32 PM, Kevin Grittner wrote: >> >>> As you can see, this generated a serialization failure. >>

Re: [HACKERS] postgres_fdw bug in 9.6

2016-12-12 Thread Tom Lane
Jeff Janes writes: > I have a test case where I made the fdw connect back to itself, and > stripped out all the objects that I could and still reproduce the case. It > is large, 21MB compressed, 163MB uncompressed, so I am linking it here: >

Re: [HACKERS] PATCH: two slab-like memory allocators

2016-12-12 Thread Tomas Vondra
On 12/12/2016 05:05 AM, Petr Jelinek wrote: I'd be happy with this patch now (as in committer ready) except that it does have some merge conflicts after the recent commits, so rebase is needed. Attached is a rebased version of the patch, resolving the Makefile merge conflicts. regards --

Re: [HACKERS] snapbuild woes

2016-12-12 Thread Andres Freund
On 2016-12-12 23:27:30 +0100, Petr Jelinek wrote: > On 12/12/16 22:42, Andres Freund wrote: > > Hi, > > > > On 2016-12-10 23:10:19 +0100, Petr Jelinek wrote: > >> Hi, > >> First one is outright bug, which has to do with how we track running > >> transactions. What snapbuild basically does while

Re: [HACKERS] snapbuild woes

2016-12-12 Thread Petr Jelinek
On 12/12/16 22:42, Andres Freund wrote: > Hi, > > On 2016-12-10 23:10:19 +0100, Petr Jelinek wrote: >> Hi, >> First one is outright bug, which has to do with how we track running >> transactions. What snapbuild basically does while doing initial snapshot >> is read the xl_running_xacts record,

Re: [HACKERS] [OSSTEST PATCH 0/1] PostgreSQL db: Retry on constraint violation

2016-12-12 Thread Kevin Grittner
On Mon, Dec 12, 2016 at 1:06 PM, Kevin Grittner wrote: > On Mon, Dec 12, 2016 at 12:32 PM, Kevin Grittner wrote: > >> As you can see, this generated a serialization failure. > > That was on 9.6. On earlier versions it does indeed allow the > transaction on

Re: [HACKERS] snapbuild woes

2016-12-12 Thread Andres Freund
Hi, On 2016-12-10 23:10:19 +0100, Petr Jelinek wrote: > Hi, > First one is outright bug, which has to do with how we track running > transactions. What snapbuild basically does while doing initial snapshot > is read the xl_running_xacts record, store the list of running txes and > then wait until

Re: [HACKERS] tuplesort_gettuple_common() and *should_free argument

2016-12-12 Thread Peter Geoghegan
On Mon, Dec 12, 2016 at 12:59 PM, Robert Haas wrote: > Committed 0001. Thanks. I should have already specifically pointed out that the original discussion on what became 0002-* is here: postgr.es/m/7256.1476711...@sss.pgh.pa.us As I said already, the general idea seems

Re: [HACKERS] Hash Indexes

2016-12-12 Thread Robert Haas
On Sun, Dec 11, 2016 at 1:24 AM, Amit Kapila wrote: > The reason for this and the similar error in vacuum was that in one of > the corner cases after freeing the overflow page and updating the link > for the previous bucket, we were not marking the buffer as dirty. So, >

Re: [HACKERS] jacana hung after failing to acquire random number

2016-12-12 Thread Andrew Dunstan
On 12/12/2016 09:02 AM, Heikki Linnakangas wrote: On 12/12/2016 03:40 PM, Andrew Dunstan wrote: Or should we at least get pg_regress to try to shut down the postmaster if it can't connect after 120 seconds? I think that makes a lot of sense, independently of this random stuff. I will

Re: [HACKERS] tuplesort_gettuple_common() and *should_free argument

2016-12-12 Thread Robert Haas
On Mon, Dec 12, 2016 at 2:30 PM, Peter Geoghegan wrote: > On Mon, Dec 12, 2016 at 9:31 AM, Robert Haas wrote: >> I think this patch might have a bug. In the existing code, >> tuplesort_gettupleslot sets should_free = true if it isn't already >> just

Re: [HACKERS] Nested Wait Events?

2016-12-12 Thread Robert Haas
On Mon, Dec 12, 2016 at 2:42 PM, Simon Riggs wrote: > There's too many "I"s in that para. I've not presented this as a > defect, nor is there any reason to believe this post is aimed at you > personally. Well, actually, there is. You said in your original post that

Re: [HACKERS] [COMMITTERS] pgsql: Add support for temporary replication slots

2016-12-12 Thread Tom Lane
Fujii Masao writes: > On Mon, Dec 12, 2016 at 11:16 PM, Peter Eisentraut wrote: >> Add support for temporary replication slots > Doesn't this need catversion bump? Yes, absolutely, because of the ABI break for the affected functions. Pushed.

Re: [HACKERS] Nested Wait Events?

2016-12-12 Thread Simon Riggs
On 12 December 2016 at 18:05, Robert Haas wrote: > On Mon, Dec 12, 2016 at 12:16 PM, Simon Riggs wrote: >> On 12 December 2016 at 16:52, Robert Haas wrote: >>> On Mon, Dec 12, 2016 at 11:33 AM, Simon Riggs

Re: [HACKERS] tuplesort_gettuple_common() and *should_free argument

2016-12-12 Thread Peter Geoghegan
On Mon, Dec 12, 2016 at 9:31 AM, Robert Haas wrote: > I think this patch might have a bug. In the existing code, > tuplesort_gettupleslot sets should_free = true if it isn't already > just before calling ExecStoreMinimalTuple((MinimalTuple) stup.tuple, > slot,

Re: [HACKERS] [OSSTEST PATCH 0/1] PostgreSQL db: Retry on constraint violation

2016-12-12 Thread Kevin Grittner
On Mon, Dec 12, 2016 at 12:32 PM, Kevin Grittner wrote: > As you can see, this generated a serialization failure. That was on 9.6. On earlier versions it does indeed allow the transaction on connection 2 to commit, yielding a non-serializable result. This makes a pretty

Re: [HACKERS] Typo in doc/src/sgml/catalogs.sgml

2016-12-12 Thread Robert Haas
On Sun, Dec 11, 2016 at 5:13 AM, Joel Jacobson wrote: > Hi, > > I found a minor typo at > https://www.postgresql.org/docs/9.6/static/catalog-pg-am.html. > > pg_catalog.pg_am. amhandler is of type "oid" according to the > documentation, but it's actually of type "regproc" in

Re: [HACKERS] Proposal : Parallel Merge Join

2016-12-12 Thread Robert Haas
On Sat, Dec 10, 2016 at 7:59 AM, Dilip Kumar wrote: > I would like to propose a patch for parallelizing merge join path. > This idea is derived by analyzing TPCH results. > > I have done this analysis along with my colleagues Rafia sabih and Amit > kaplia. > > Currently we

Re: [HACKERS] [OSSTEST PATCH 0/1] PostgreSQL db: Retry on constraint violation

2016-12-12 Thread Kevin Grittner
On Mon, Dec 12, 2016 at 8:45 AM, Ian Jackson wrote: > AIUI the documented behavour is that "every set of successful > transactions is serialisable". Well, in context that is referring to serializable transactions. No such guarantee is provided for other isolation

Re: [HACKERS] Nested Wait Events?

2016-12-12 Thread Robert Haas
On Mon, Dec 12, 2016 at 12:16 PM, Simon Riggs wrote: > On 12 December 2016 at 16:52, Robert Haas wrote: >> On Mon, Dec 12, 2016 at 11:33 AM, Simon Riggs wrote: >>> Last week I noticed that the Wait Event/Locks system doesn't

Re: [HACKERS] Declarative partitioning - another take

2016-12-12 Thread Dmitry Ivanov
Huh, this code is broken as well. We have to ignore partitions that don't have any subpartitions. Patch is attached below (v2). -- Dmitry Ivanov Postgres Professional: http://www.postgrespro.com Russian Postgres Companydiff --git a/src/backend/catalog/partition.c

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2016-12-12 Thread Kevin Grittner
On Mon, Dec 12, 2016 at 10:33 AM, Andres Freund wrote: > On 2016-12-12 13:26:32 -0300, Alvaro Herrera wrote: >> Robert Haas wrote: >>> 1. Show all processes that have a PGPROC in pg_stat_activity, >>> 2. Add a second view, say pg_stat_system_activity, >> I vote 1. > > +1

Re: [HACKERS] tuplesort_gettuple_common() and *should_free argument

2016-12-12 Thread Robert Haas
On Fri, Dec 9, 2016 at 5:59 PM, Peter Geoghegan wrote: > Attached patch 0001-* removes all should_free arguments. To reiterate, > this is purely a refactoring patch. I think this patch might have a bug. In the existing code, tuplesort_gettupleslot sets should_free = true if it

Re: [HACKERS] pg_background contrib module proposal

2016-12-12 Thread Andrew Borodin
Hi! Just in case you'd like to include sleepsort as a test, here it is wrapped as a regression test(see attachment). But it has serious downside: it runs no less than 5 seconds. Also I'll list here every parallelism feature I managed to imagine. It is not to say that I suggest having some of

Re: [HACKERS] Declarative partitioning - another take

2016-12-12 Thread Dmitry Ivanov
Hi guys, Looks like I've just encountered a bug. Please excuse me for the messy email, I don't have much time at the moment. Here's the test case: create table test(val int) partition by range (val); create table test_1 partition of test for values from (1) to (1000) partition by

Re: [HACKERS] Nested Wait Events?

2016-12-12 Thread Simon Riggs
On 12 December 2016 at 16:52, Robert Haas wrote: > On Mon, Dec 12, 2016 at 11:33 AM, Simon Riggs wrote: >> Last week I noticed that the Wait Event/Locks system doesn't correctly >> describe waits for tuple locks because in some cases that happens in

Re: [HACKERS] new table partitioning breaks \d table to older versions

2016-12-12 Thread Robert Haas
On Fri, Dec 9, 2016 at 1:18 PM, Fabrízio de Royes Mello wrote: > On Fri, Dec 9, 2016 at 3:59 PM, Jeff Janes wrote: >> Since: >> commit f0e44751d7175fa3394da2c8f85e3ceb3cdbfe63 >> Author: Robert Haas >> Date: Wed Dec 7

Re: [HACKERS] Nested Wait Events?

2016-12-12 Thread Robert Haas
On Mon, Dec 12, 2016 at 11:33 AM, Simon Riggs wrote: > Last week I noticed that the Wait Event/Locks system doesn't correctly > describe waits for tuple locks because in some cases that happens in > two stages. Well, I replied to that email to say that I didn't agree with

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2016-12-12 Thread Robert Haas
On Mon, Dec 12, 2016 at 11:19 AM, Robert Haas wrote: > So, one of the problems in this patch as committed is that for any > process that doesn't show up in pg_stat_activity, there's no way to > see the wait event information. That sucks. I think there are > basically two

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2016-12-12 Thread David G. Johnston
On Mon, Dec 12, 2016 at 9:19 AM, Robert Haas wrote: > So, one of the problems in this patch as committed is that for any > process that doesn't show up in pg_stat_activity, there's no way to > see the wait event information. That sucks. I think there are > basically two

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2016-12-12 Thread Andres Freund
On 2016-12-12 13:26:32 -0300, Alvaro Herrera wrote: > Robert Haas wrote: > > > So, one of the problems in this patch as committed is that for any > > process that doesn't show up in pg_stat_activity, there's no way to > > see the wait event information. That sucks. I think there are > >

[HACKERS] Nested Wait Events?

2016-12-12 Thread Simon Riggs
Last week I noticed that the Wait Event/Locks system doesn't correctly describe waits for tuple locks because in some cases that happens in two stages. Now I notice that the Wait Event system doesn't handle waiting for recovery conflicts at all, though it does access ProcArrayLock multiple times.

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2016-12-12 Thread Tom Lane
Alvaro Herrera writes: > Robert Haas wrote: >> So, one of the problems in this patch as committed is that for any >> process that doesn't show up in pg_stat_activity, there's no way to >> see the wait event information. That sucks. I think there are >> basically two

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2016-12-12 Thread Alvaro Herrera
Robert Haas wrote: > So, one of the problems in this patch as committed is that for any > process that doesn't show up in pg_stat_activity, there's no way to > see the wait event information. That sucks. I think there are > basically two ways to fix this: > > 1. Show all processes that have a

Re: [HACKERS] Quorum commit for multiple synchronous replication.

2016-12-12 Thread Masahiko Sawada
On Mon, Dec 12, 2016 at 9:52 PM, Fujii Masao wrote: > On Mon, Dec 12, 2016 at 9:31 PM, Masahiko Sawada > wrote: >> On Sat, Dec 10, 2016 at 5:17 PM, Amit Kapila wrote: >>> On Thu, Dec 8, 2016 at 3:02 PM, Masahiko Sawada

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2016-12-12 Thread Robert Haas
On Tue, Oct 4, 2016 at 11:59 AM, Robert Haas wrote: > On Mon, Oct 3, 2016 at 8:43 PM, Michael Paquier > wrote: >> The rest looks good to me. Thanks for the feedback and the time! > > Thanks for the fixes. I committed this with an additional

Re: [HACKERS] jacana hung after failing to acquire random number

2016-12-12 Thread Tom Lane
Heikki Linnakangas writes: > On 12/12/2016 03:40 PM, Andrew Dunstan wrote: >> Should one or more of these errors be fatal? Or should we at least get >> pg_regress to try to shut down the postmaster if it can't connect after >> 120 seconds? > Making it fatal, i.e. bringing down

Re: [HACKERS] background sessions

2016-12-12 Thread Robert Haas
On Mon, Dec 12, 2016 at 10:02 AM, Craig Ringer wrote: > On 12 Dec. 2016 21:55, "Robert Haas" wrote: > On Sun, Dec 11, 2016 at 5:38 AM, Andrew Borodin > wrote: >> 1. As far as I can see, we connot use COPY FROM STDIN in

Re: [HACKERS] Declarative partitioning - another take

2016-12-12 Thread Tomas Vondra
On 12/12/2016 07:37 AM, Amit Langote wrote: Hi Tomas, On 2016/12/12 10:02, Tomas Vondra wrote: 2) I'm wondering whether having 'table' in the catalog name (and also in the new relkind) is too limiting. I assume we'll have partitioned indexes one day, for example - do we expect to use the

Re: [HACKERS] background sessions

2016-12-12 Thread Craig Ringer
On 12 Dec. 2016 21:55, "Robert Haas" wrote: On Sun, Dec 11, 2016 at 5:38 AM, Andrew Borodin wrote: > 1. As far as I can see, we connot use COPY FROM STDIN in bg session? > Since one of purposes is to orchestrate transactions, may be that > would be

Re: [HACKERS] jsonb problematic operators

2016-12-12 Thread Craig Ringer
On 12 Dec. 2016 22:22, "Merlin Moncure" wrote: If we really wanted to fix this, maybe the right way to think about the problem is a highly reduced character set and a pre-processor or an extension. I'm pretty OK with expecting client drivers not to be stupid and offer

Re: [HACKERS] [OSSTEST PATCH 0/1] PostgreSQL db: Retry on constraint violation

2016-12-12 Thread Ian Jackson
Kevin Grittner writes ("Re: [HACKERS] [OSSTEST PATCH 0/1] PostgreSQL db: Retry on constraint violation"): > If I recall correctly, the constraints for which there can be > errors appearing due to concurrent transactions are primary key, > unique, and foreign key constraints. I don't remember

Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol

2016-12-12 Thread Heikki Linnakangas
A few couple more things that caught my eye while hacking on this: 1. We don't use SASLPrep to scrub username's and passwords. That's by choice, for usernames, because historically in PostgreSQL usernames can be stored in any encoding, but SASLPrep assumes UTF-8. We dodge that by passing an

Re: [HACKERS] [COMMITTERS] pgsql: Add support for temporary replication slots

2016-12-12 Thread Fujii Masao
On Mon, Dec 12, 2016 at 11:16 PM, Peter Eisentraut wrote: > Add support for temporary replication slots > > This allows creating temporary replication slots that are removed > automatically at the end of the session or on error. > > From: Petr Jelinek

Re: [HACKERS] jsonb problematic operators

2016-12-12 Thread Merlin Moncure
On Sun, Dec 11, 2016 at 10:59 PM, Craig Ringer wrote: > PgJDBC allows you to write ??, which is ugly, but tolerable, since the > JDBC spec doesn't have an escape syntax for it. This is the core problem; *JDBC* is busted. SQL reserves words but not punctuation marks so any

Re: [HACKERS] Logical Replication WIP

2016-12-12 Thread Peter Eisentraut
On 12/8/16 4:10 PM, Petr Jelinek wrote: > On 08/12/16 20:16, Peter Eisentraut wrote: >> On 12/6/16 11:58 AM, Peter Eisentraut wrote: >>> On 12/5/16 6:24 PM, Petr Jelinek wrote: I think that the removal of changes to ReplicationSlotAcquire() that you did will result in making it

Re: [HACKERS] Declarative partitioning - another take

2016-12-12 Thread Peter Eisentraut
On 12/7/16 1:20 PM, Robert Haas wrote: > I've committed 0001 - 0006 with that correction and a few other > adjustments. There's plenty of room for improvement here, and almost > certainly some straight-up bugs too, but I think we're at a point > where it will be easier and less error-prone to

Re: [HACKERS] jacana hung after failing to acquire random number

2016-12-12 Thread Heikki Linnakangas
On 12/12/2016 03:40 PM, Andrew Dunstan wrote: Or should we at least get pg_regress to try to shut down the postmaster if it can't connect after 120 seconds? I think that makes a lot of sense, independently of this random stuff. - Heikki -- Sent via pgsql-hackers mailing list

Re: [HACKERS] jacana hung after failing to acquire random number

2016-12-12 Thread Heikki Linnakangas
On 12/12/2016 03:40 PM, Andrew Dunstan wrote: On 12/12/2016 02:32 AM, Heikki Linnakangas wrote: On 12/12/2016 05:58 AM, Michael Paquier wrote: On Sun, Dec 11, 2016 at 9:06 AM, Andrew Dunstan wrote: jascana (mingw, 64 bit compiler, no openssl) is currently hung on

Re: [HACKERS] Fix for segfault in plpython's exception handling

2016-12-12 Thread Rafa de la Torre
For the record: I tested the patch by Tom Lane in our setup (python 2.7.3-0ubuntu3.8) and works like a charm. https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=9cda81f0056ca488dbd6cded64db1238aed816b2 Also in 9.5 and 9.6 series. My request in commitfest queue can be closed.

Re: [HACKERS] background sessions

2016-12-12 Thread Robert Haas
On Sun, Dec 11, 2016 at 5:38 AM, Andrew Borodin wrote: > 1. As far as I can see, we connot use COPY FROM STDIN in bg session? > Since one of purposes is to orchestrate transactions, may be that > would be valuable. A background worker has no client connection, so what would

[HACKERS] [PATCH] Fix for documentation of timestamp type

2016-12-12 Thread Aleksander Alekseev
Hello. Currently doc/src/sgml/datatype.sgml states: ``` When timestamp values are stored as eight-byte integers (currently the default), microsecond precision is available over the full range of values. When timestamp values are stored as double precision floating-point numbers

Re: [HACKERS] jacana hung after failing to acquire random number

2016-12-12 Thread Andrew Dunstan
On 12/12/2016 02:32 AM, Heikki Linnakangas wrote: On 12/12/2016 05:58 AM, Michael Paquier wrote: On Sun, Dec 11, 2016 at 9:06 AM, Andrew Dunstan wrote: jascana (mingw, 64 bit compiler, no openssl) is currently hung on "make check". After starting the autovacuum

Re: [HACKERS] jsonb problematic operators

2016-12-12 Thread Greg Stark
On 12 December 2016 at 04:59, Craig Ringer wrote: > I didn't realise Pg's use of ? was that old, so thanks. That makes > offering alternatives much less appealing. One option might be for Postgres to define duplicate operator names using ¿ or something else. I think ¿ is a

Re: [HACKERS] pgcrypto compilation error due to stack-allocated EVP_CIPHER_CTX

2016-12-12 Thread Michael Paquier
On Mon, Dec 12, 2016 at 6:17 PM, Heikki Linnakangas wrote: > Removed that, did some further cosmetic changes, and pushed. I renamed a > bunch variables and structs, so that they are more consistent with the > similar digest stuff. That definitely makes sense this way, thanks for

Re: [HACKERS] Quorum commit for multiple synchronous replication.

2016-12-12 Thread Fujii Masao
On Mon, Dec 12, 2016 at 9:31 PM, Masahiko Sawada wrote: > On Sat, Dec 10, 2016 at 5:17 PM, Amit Kapila wrote: >> On Thu, Dec 8, 2016 at 3:02 PM, Masahiko Sawada >> wrote: >>> On Thu, Dec 8, 2016 at 4:39 PM, Michael Paquier

Re: [HACKERS] Quorum commit for multiple synchronous replication.

2016-12-12 Thread Masahiko Sawada
On Sat, Dec 10, 2016 at 5:17 PM, Amit Kapila wrote: > On Thu, Dec 8, 2016 at 3:02 PM, Masahiko Sawada wrote: >> On Thu, Dec 8, 2016 at 4:39 PM, Michael Paquier >> wrote: >>> On Thu, Dec 8, 2016 at 9:07 AM, Robert Haas

Re: [HACKERS] multivariate statistics (v19)

2016-12-12 Thread Amit Langote
Hi Tomas, On 2016/10/30 4:23, Tomas Vondra wrote: > Hi, > > Attached is v20 of the multivariate statistics patch series, doing mostly > the changes outlined in the preceding e-mail from October 11. > > The patch series currently has these parts: > > * 0001 : (FIX) teach pull_varno about

Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol

2016-12-12 Thread Heikki Linnakangas
On 12/09/2016 01:10 PM, Michael Paquier wrote: On Fri, Dec 09, 2016 at 11:51:45AM +0200, Heikki Linnakangas wrote: On 12/09/2016 05:58 AM, Michael Paquier wrote: One thing is: when do we look up at pg_authid? After receiving the first message from client or before beginning the exchange? As

Re: [HACKERS] Patch to implement pg_current_logfile() function

2016-12-12 Thread Gilles Darold
Le 11/12/2016 à 04:38, Karl O. Pinc a écrit : > On Sat, 10 Dec 2016 19:41:21 -0600 > "Karl O. Pinc" wrote: > >> On Fri, 9 Dec 2016 23:36:12 -0600 >> "Karl O. Pinc" wrote: >> >>> Instead I propose (code I have not actually executed): >>> ... >>> char

Re: [HACKERS] pgcrypto compilation error due to stack-allocated EVP_CIPHER_CTX

2016-12-12 Thread Heikki Linnakangas
On 12/12/2016 07:18 AM, Michael Paquier wrote: On Fri, Dec 9, 2016 at 10:22 AM, Michael Paquier wrote: Thanks for looking at the patch. Looking forward to hearing more! Here is an updated patch based on which reviews should be done. I have fixed the issue you have

Re: [HACKERS] Fix for segfault in plpython's exception handling

2016-12-12 Thread Rafa de la Torre
Thank you! Glad to see that the report was useful. On Fri, Dec 9, 2016 at 9:33 PM, Tom Lane wrote: > I wrote: > > Rafa de la Torre writes: > >> exc = PyErr_NewException(exception_map[i].name, base, > dict); > >> +Py_INCREF(exc); >