Re: [HACKERS] Parallel Append implementation

2017-11-21 Thread Amit Khandekar
On 21 November 2017 at 12:44, Rafia Sabih wrote: > On Mon, Nov 13, 2017 at 12:54 PM, Amit Khandekar > wrote: >> Thanks a lot Robert for the patch. I will have a look. Quickly tried >> to test some aggregate queries with a partitioned pgbench_accounts >> table, and it is crashing. Will get back w

Re: Failed to delete old ReorderBuffer spilled files

2017-11-21 Thread Masahiko Sawada
On Tue, Nov 21, 2017 at 3:48 PM, Masahiko Sawada wrote: > On Mon, Nov 20, 2017 at 7:35 PM, atorikoshi > wrote: >> Hi, >> >> I put many queries into one transaction and made ReorderBuffer spill >> data to disk, and sent SIGKILL to postgres before the end of the >> transaction. >> >> After starting

Re: [HACKERS] CLUSTER command progress monitor

2017-11-21 Thread Antonin Houska
Tom Lane wrote: > Antonin Houska writes: > > Robert Haas wrote: > >> These two phases overlap, though. I believe progress reporting for > >> sorts is really hard. > > > Whatever complexity is hidden in the sort, cost_sort() should have taken it > > into consideration when called via plan_clust

Re: Anybody care about having the verbose form of the tzdata files?

2017-11-21 Thread Daniel Gustafsson
> On 20 Nov 2017, at 21:38, Tom Lane wrote: > So I'm wondering if we should replace src/timezone/data/* with > tzdata.zi, in the name of reducing the size of our tarballs. +1 > Anybody here actually care about reading the zone data files? I doubt there is anyone who cares about that who isn’t

Re: [HACKERS] Assertion failure when the non-exclusive pg_stop_backup aborted.

2017-11-21 Thread Masahiko Sawada
On Tue, Nov 21, 2017 at 10:01 AM, Michael Paquier wrote: > On Tue, Nov 21, 2017 at 9:37 AM, Masahiko Sawada > wrote: >> On Tue, Nov 21, 2017 at 8:03 AM, Michael Paquier >> wrote: >>> You could just add "as this allows to keep backup counters kept in >>> shared memory consistent with the state o

Re: [HACKERS] [PROPOSAL] Temporal query processing with range types

2017-11-21 Thread Peter Moser
2017-11-14 18:42 GMT+01:00 Tom Lane : > You might consider putting the rewriting into, um, the rewriter. > It could be a separate pass after view expansion, if direct integration > with the existing behavior seems unduly spaghetti-ish.  Or do it in > an early phase of planning as he suggested.  The

Re: default range partition and constraint exclusion

2017-11-21 Thread Amit Langote
On 2017/11/18 8:28, Robert Haas wrote: > On Fri, Nov 17, 2017 at 12:57 AM, Amit Langote > wrote: >> While working on the patch for partition pruning for declarative >> partitioned tables, I noticed that default range partition will fail to be >> included in a plan in certain cases due to pruning b

Re: [HACKERS] path toward faster partition pruning

2017-11-21 Thread Kyotaro HORIGUCHI
Thank you and sorry for the confused comments. At Mon, 13 Nov 2017 18:46:28 +0900, Amit Langote wrote in <8460a6c3-68c5-b78a-7d18-d253180f2...@lab.ntt.co.jp> > Horiguchi-san, > > Thanks for taking a look. Replying to all your emails here. > > In 0003, > > > > +match_clauses_to_partkey(RelO

Re: [HACKERS] path toward faster partition pruning

2017-11-21 Thread Rajkumar Raghuwanshi
On Fri, Nov 17, 2017 at 3:31 PM, Amit Langote wrote: > > Support for hash partitioning and tests for the same. Also, since > update/delete on partitioned tables still depend on constraint exclusion > for pruning, fix things such that get_relation_constraints includes > partition constraints in i

Re: [HACKERS] pg_basebackup --progress output for batch execution

2017-11-21 Thread Martín Marqués
El 21/11/17 a las 04:56, Arthur Zakirov escribió: > On Mon, Nov 20, 2017 at 04:45:48PM -0300, Martín Marqués wrote: >> New version of patch, without the --batch-mode option and using isatty() >> > > Great! > >> +fprintf(stderr, "waiting for checkpoint"); >> +if (isatty(fil

Re: [HACKERS] pg_basebackup --progress output for batch execution

2017-11-21 Thread Arthur Zakirov
On Tue, Nov 21, 2017 at 07:16:46AM -0300, Martín Marqués wrote: > > Ups! Attached the corrected version.:) > Thank you for the new version. The patch applies via 'patch' command without warnings and errors, tests passed. Marked the patch as "Ready for Commiter". -- Arthur Zakirov Postgres Pr

Re: [HACKERS] pg_basebackup --progress output for batch execution

2017-11-21 Thread Martín Marqués
Arthur, Thank you very much for reviewing the patch and for your valuable input (you made me read the Microsoft Visual C specs ;)) Regards, 2017-11-21 8:11 GMT-03:00 Arthur Zakirov : > On Tue, Nov 21, 2017 at 07:16:46AM -0300, Martín Marqués wrote: >> >> Ups! Attached the corrected version.:) >>

Re: Failed to delete old ReorderBuffer spilled files

2017-11-21 Thread atorikoshi
Thanks for reviewing! On 2017/11/21 18:12, Masahiko Sawada wrote: On Tue, Nov 21, 2017 at 3:48 PM, Masahiko Sawada wrote: On Mon, Nov 20, 2017 at 7:35 PM, atorikoshi wrote: Hi, I put many queries into one transaction and made ReorderBuffer spill data to disk, and sent SIGKILL to postgres b

Re: Failed to delete old ReorderBuffer spilled files

2017-11-21 Thread Kyotaro HORIGUCHI
Hello, At Tue, 21 Nov 2017 20:27:25 +0900, atorikoshi wrote in > Thanks for reviewing! > > > On 2017/11/21 18:12, Masahiko Sawada wrote: > > On Tue, Nov 21, 2017 at 3:48 PM, Masahiko Sawada > > wrote: > >> On Mon, Nov 20, 2017 at 7:35 PM, atorikoshi > >> wrote: > >>> Hi, > >>> > >>> I put

Re: [HACKERS] Parallel Append implementation

2017-11-21 Thread amul sul
On Tue, Nov 21, 2017 at 2:22 PM, Amit Khandekar wrote: > On 21 November 2017 at 12:44, Rafia Sabih > wrote: >> On Mon, Nov 13, 2017 at 12:54 PM, Amit Khandekar >> wrote: >>> Thanks a lot Robert for the patch. I will have a look. Quickly tried >>> to test some aggregate queries with a partition

[PATCH] using arc4random for strong randomness matters.

2017-11-21 Thread David CARLIER
Hello, This is my first small personal contribution. Motivation : - Using fail-safe, file descriptor free solution on *BSD and Darwin system - Somehow avoiding at the moment FreeBSD as it still uses RC4 (seemingly updated to Chacha20 for FreeBSD 12.0 and eventually backported later on). - For imp

Re: [HACKERS] Partition-wise aggregation/grouping

2017-11-21 Thread Robert Haas
On Fri, Nov 17, 2017 at 7:24 AM, Ashutosh Bapat wrote: > + * this value should be multiplied with cpu_tuple_cost wherever applicable. > + */ > +#define DEFAULT_APPEND_COST_FACTOR 0.5 > > I am wondering whether we should just define > #define APPEND_TUPLE_COST (cpu_tuple_cost * 0.5) > and use this

Re: feature request: consume asynchronous notification via a function

2017-11-21 Thread Robert Haas
On Fri, Nov 17, 2017 at 9:49 AM, Merlin Moncure wrote: > Currently the only way that I know of to consume async notifications > via SQL (as opposed to a client application) is via dblink_get_notify. > This method isn't very good; it requires some extra support coding, > eats a connection and a bac

Re: IndexTupleDSize macro seems redundant

2017-11-21 Thread Amit Kapila
On Mon, Nov 20, 2017 at 9:01 PM, Ildar Musin wrote: > Hi all, > > While I was looking through the indexes code I got confused by couple of > macros - IndexTupleSize() and IndexTupleDSize() - which seem to do the same > thing with only difference that the first one takes pointer as an argument > wh

Re: [HACKERS] GnuTLS support

2017-11-21 Thread Andreas Karlsson
On 11/20/2017 02:56 AM, Michael Paquier wrote: Sorry about that. Something more specific needs to happen here as well for channel binding support with SCRAM. CheckSCRAMAuth() now assumes that the channel binding mechanism SCRAM-SHA-256-PLUS can be published to the client if SSL is used, because O

Re: Inlining functions with "expensive" parameters

2017-11-21 Thread Robert Haas
On Thu, Nov 16, 2017 at 2:51 PM, Andres Freund wrote: > Right, but it doesn't sound that hard to introduce. Basically there'd need to > be a WithParamValue node, that first evaluates parameters and then executes > the child expression. I'm thinking of doing this hierarchically so there's > les

Re: Inlining functions with "expensive" parameters

2017-11-21 Thread Tom Lane
Robert Haas writes: > On Thu, Nov 16, 2017 at 2:51 PM, Andres Freund wrote: >> Right, but it doesn't sound that hard to introduce. Basically there'd >> need to be a WithParamValue node, that first evaluates parameters and >> then executes the child expression. > I don't quite follow the need fo

Re: Inlining functions with "expensive" parameters

2017-11-21 Thread Tom Lane
I wrote: > Robert Haas writes: >> I don't quite follow the need for this. I mean, if we just stick a >> Param reference in there and create a corresponding InitPlan, the >> Param will be evaluated on demand, right? Is the point of the new >> node to make sure that the Param gets re-evaluated whe

RE: Re: User defined data types in Logical Replication

2017-11-21 Thread Huong Dangminh
Sorry for not replying sooner. > > Attached draft patch fixed this issue, at least on my environment. > > It works good for me. > > > Please review it. > > I will review it soon. There is one more case that user-defined data type is not supported in Logical Replication. That is when remote da

Re: Treating work_mem as a shared resource (Was: Parallel Hash take II)

2017-11-21 Thread Robert Haas
On Fri, Nov 17, 2017 at 9:22 PM, Peter Geoghegan wrote: > I think that it's reasonable for us to make it a goal of the executor > to have operations that have a smooth cost function, in order to > manage the risk of misestimation well, and to make it a goal to have > operations that are otherwise

Does XMLSERIALIZE output xmlattributes in a stable order?

2017-11-21 Thread Chapman Flack
Suppose I have a query that generates some XML content, and I want to do this on a periodic schedule and check the resulting XML into a version control system. To avoid spurious diffs, I know I can control the order of child elements generated by xmlagg by slipping an ORDER BY into the aggregate e

Re: feature request: consume asynchronous notification via a function

2017-11-21 Thread Merlin Moncure
On Tue, Nov 21, 2017 at 7:59 AM, Robert Haas wrote: > On Fri, Nov 17, 2017 at 9:49 AM, Merlin Moncure wrote: >> Currently the only way that I know of to consume async notifications >> via SQL (as opposed to a client application) is via dblink_get_notify. >> This method isn't very good; it require

Re: Inlining functions with "expensive" parameters

2017-11-21 Thread Andres Freund
On 2017-11-21 09:59:00 -0500, Robert Haas wrote: > On Thu, Nov 16, 2017 at 2:51 PM, Andres Freund wrote: > > Right, but it doesn't sound that hard to introduce. Basically there'd need > > to be a WithParamValue node, that first evaluates parameters and then > > executes the child expression. I'

Re: Inlining functions with "expensive" parameters

2017-11-21 Thread Andres Freund
Hi, On 2017-11-21 10:16:58 -0500, Tom Lane wrote: > I'm also wondering about folding CaseTestExpr and CoerceToDomainValue > into the same mechanism. It's not very hard to see those cases as > being the same as a function-based lambda. Yea, that'd be good. The current mechanisms is uh, historical

Re: Does XMLSERIALIZE output xmlattributes in a stable order?

2017-11-21 Thread Tom Lane
Chapman Flack writes: > Suppose I have a query that generates some XML content, and I want > to do this on a periodic schedule and check the resulting XML into > a version control system. > ... > But then, there are the attributes of elements. Order of attributes > is not significant in XML, and i

Re: View with duplicate GROUP BY entries

2017-11-21 Thread Tom Lane
Ashutosh Bapat writes: > While reviewing patch for similar problem in postgres_fdw [1], I > noticed that we don't use positional notation while creating the view. > This might introduced anomalies when GROUP BY entries are > non-immutable. Yeah, we probably ought to make more of an effort to rege

Re: to_typemod(type_name) information function

2017-11-21 Thread Sophie Herold
Hi, the following patch allows to retrieve the typemod. Without this patch, it does not seem to be possible to generate the first column. SELECT format_type(to_regtype(t), pg_to_typemod(t)), format_type(to_regtype(t), NULL) FROM (VALUES ('INTERVAL SECOND (5)'), ('Varchar(17)'

Re: to_typemod(type_name) information function

2017-11-21 Thread Stephen Frost
Greeting, Sophie! * Sophie Herold (sophi...@hemio.de) wrote: > I did not find any advice on how to choose a new OID for pg_proc. (Haven't looked at the patch itself yet really, but wanted to answer this.) The main thing is to not duplicate the OID, which you can avoid by calling 'unused_oids' in

Re: [HACKERS] Custom compression methods

2017-11-21 Thread Tomas Vondra
Hi, On 11/21/2017 03:47 PM, Ildus Kurbangaliev wrote: > On Mon, 20 Nov 2017 00:04:53 +0100 > Tomas Vondra wrote: > > ... > >> 6) I'm rather confused by AttributeCompression vs. >> ColumnCompression. I mean, attribute==column, right? Of course, one >> is for data from parser, the other one is for

Re: View with duplicate GROUP BY entries

2017-11-21 Thread Robert Haas
On Tue, Nov 21, 2017 at 12:05 PM, Tom Lane wrote: > Ashutosh Bapat writes: >> While reviewing patch for similar problem in postgres_fdw [1], I >> noticed that we don't use positional notation while creating the view. >> This might introduced anomalies when GROUP BY entries are >> non-immutable. >

Re: feature request: consume asynchronous notification via a function

2017-11-21 Thread Robert Haas
On Tue, Nov 21, 2017 at 11:32 AM, Merlin Moncure wrote: >> I think that wouldn't work very well, because I think we must have a >> snapshot open in order to run pg_get_notifications(), and that means >> we're holding back the system-wide xmin. > > I am very much looking at the new stored procedure

Re: [COMMITTERS] pgsql: Add hash partitioning.

2017-11-21 Thread Robert Haas
On Mon, Nov 20, 2017 at 7:46 AM, amul sul wrote: > Thanks for fixing this function. Patch looks good to me, except column number > in the following errors message should to be 2. > > 354 +SELECT satisfies_hash_partition('mchash'::regclass, 2, 1, > NULL::int, NULL::int); > 355 +ERROR: column 1 of

Re: View with duplicate GROUP BY entries

2017-11-21 Thread Tom Lane
Robert Haas writes: > On Tue, Nov 21, 2017 at 12:05 PM, Tom Lane wrote: >> Yeah, we probably ought to make more of an effort to regenerate the >> original query wording. I do not think that forcing positional notation >> is a suitable answer in this case, because it would result in converting >>

Re: View with duplicate GROUP BY entries

2017-11-21 Thread Robert Haas
On Tue, Nov 21, 2017 at 1:11 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Nov 21, 2017 at 12:05 PM, Tom Lane wrote: >>> Yeah, we probably ought to make more of an effort to regenerate the >>> original query wording. I do not think that forcing positional notation >>> is a suitable answe

Re: feature request: consume asynchronous notification via a function

2017-11-21 Thread Tom Lane
Robert Haas writes: > On Tue, Nov 21, 2017 at 11:32 AM, Merlin Moncure wrote: >> I am very much looking at the new stored procedure functionality and >> imaging a loop like this: >> >> LOOP >> FOR r IN SELECT * FROM pg_get_notifications(30) >> LOOP >> PERFORM do_stuff(r); >> END LOOP;

Re: Does XMLSERIALIZE output xmlattributes in a stable order?

2017-11-21 Thread Chapman Flack
On 11/21/2017 11:49 AM, Tom Lane wrote: > AFAICS, XMLSERIALIZE in our current implementation boils down to > being a binary-compatible coercion from XML (which is stored as > a string) to text. So the interesting question here is where are > you getting the XML values from? The stability of the

Re: Logical Replication and triggers

2017-11-21 Thread Robert Haas
On Sat, Nov 18, 2017 at 7:30 PM, Craig Ringer wrote: > On 15 November 2017 at 21:12, Thomas Rosenstein > wrote: >> I would like somebody to consider Petr Jelineks patch for worker.c from >> here >> (https://www.postgresql.org/message-id/619c557d-93e6-1833-1692-b010b176ff77%402ndquadrant.com) >> >

Re: [HACKERS] SQL procedures

2017-11-21 Thread Andrew Dunstan
On 11/20/2017 04:25 PM, I wrote: > I've been through this fairly closely, and I think it's pretty much > committable. The only big item that stands out for me is the issue of > OUT parameters. > > While returning multiple result sets will be a useful feature, it's also > very common in my experie

Re: [HACKERS] Re: protocol version negotiation (Re: Libpq PGRES_COPY_BOTH - version compatibility)

2017-11-21 Thread Robert Haas
On Sun, Nov 19, 2017 at 7:49 PM, Badrul Chowdhury wrote: >>> I spent a little more time looking at this patch today. I think that the >>> patch >>> should actually send NegotiateProtocolVersion when *either* the requested >>> version is differs from the latest one we support *or* an unsupported

Re: [HACKERS] Parallel Append implementation

2017-11-21 Thread Robert Haas
On Tue, Nov 21, 2017 at 6:57 AM, amul sul wrote: > By doing following change on the v19 patch does the fix for me: > > --- a/src/backend/executor/nodeAppend.c > +++ b/src/backend/executor/nodeAppend.c > @@ -489,11 +489,9 @@ choose_next_subplan_for_worker(AppendState *node) > } > > /* Pick

Re: feature request: consume asynchronous notification via a function

2017-11-21 Thread Merlin Moncure
On Tue, Nov 21, 2017 at 12:20 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Nov 21, 2017 at 11:32 AM, Merlin Moncure wrote: >>> I am very much looking at the new stored procedure functionality and >>> imaging a loop like this: >>> >>> LOOP >>> FOR r IN SELECT * FROM pg_get_notifications

Re: [HACKERS] pgbench regression test failure

2017-11-21 Thread Robert Haas
On Mon, Nov 20, 2017 at 1:40 PM, Tom Lane wrote: > I dunno, it just looks odd to me that when we've set up a test case in > which every one of the transactions is guaranteed to exceed the latency > limit, that it doesn't say that they all did. I don't particularly buy > your assumption that the p

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2017-11-21 Thread Robert Haas
On Mon, Nov 20, 2017 at 5:19 PM, Masahiko Sawada wrote: > Attached updated version patch. I've moved only relation extension > locks out of heavy-weight lock as per discussion so far. > > I've done a write-heavy benchmark on my laptop; loading 24kB data to > one table using COPY by 1 client, for 1

Re: [HACKERS] Custom compression methods

2017-11-21 Thread Ildus K
On Tue, 21 Nov 2017 18:47:49 +0100 Tomas Vondra wrote: > > I propose to use either > >CompressionMethodOptions (and CompressionMethodRoutine) > > or > >CompressionOptions (and CompressionRoutine) Sounds good, thanks. > > OK. But then I don't understand why tsvector.c does things

Re: [HACKERS] pgbench regression test failure

2017-11-21 Thread Tom Lane
Robert Haas writes: > On Mon, Nov 20, 2017 at 1:40 PM, Tom Lane wrote: >> I dunno, it just looks odd to me that when we've set up a test case in >> which every one of the transactions is guaranteed to exceed the latency >> limit, that it doesn't say that they all did. I don't particularly buy >>

Re: Logical Replication and triggers

2017-11-21 Thread Simon Riggs
On 21 November 2017 at 13:27, Robert Haas wrote: > On Sat, Nov 18, 2017 at 7:30 PM, Craig Ringer wrote: >> On 15 November 2017 at 21:12, Thomas Rosenstein >> wrote: >>> I would like somebody to consider Petr Jelineks patch for worker.c from >>> here >>> (https://www.postgresql.org/message-id/619

Re: Logical Replication and triggers

2017-11-21 Thread Robert Haas
On Tue, Nov 21, 2017 at 3:29 PM, Simon Riggs wrote: >> You realize we're talking about a bug fix, right? And for a feature >> that was developed and committed by your colleagues? > > Craig is asking Thomas to confirm the proposed bug fix works. How is > this not normal? That's not exactly how I

Re: [HACKERS] pgbench regression test failure

2017-11-21 Thread Robert Haas
On Tue, Nov 21, 2017 at 3:28 PM, Tom Lane wrote: > Seems like a good idea, but the way you've written it is inconsistent > with the "n/m" notation used just above. I'd suggest > > ... latency limit: 33 (33/33, 100.000 %) > > or just > > ... latency limit: 33/33 (100.000 %) Oh, yeah. That last o

Re: [HACKERS] Custom compression methods

2017-11-21 Thread Tomas Vondra
On 11/21/2017 09:28 PM, Ildus K wrote: >> Hmmm, it still doesn't work for me. See this: >> >> test=# create extension pg_lz4 ; >> CREATE EXTENSION >> test=# create table t_lz4 (v text compressed lz4); >> CREATE TABLE >> test=# create table t_pglz (v text); >> CREATE TABLE >

Re: Combine function returning NULL unhandled?

2017-11-21 Thread Robert Haas
On Mon, Nov 20, 2017 at 10:36 PM, Andres Freund wrote: > The plain transition case contains: > if (pergroupstate->transValueIsNull) > { > /* > * Don't call a strict function with NULL inputs. > Note it is >

Re: [HACKERS] CLUSTER command progress monitor

2017-11-21 Thread Robert Haas
On Mon, Nov 20, 2017 at 12:25 PM, Tom Lane wrote: > Antonin Houska writes: >> Robert Haas wrote: >>> These two phases overlap, though. I believe progress reporting for >>> sorts is really hard. > >> Whatever complexity is hidden in the sort, cost_sort() should have taken it >> into consideration

Re: [HACKERS] CLUSTER command progress monitor

2017-11-21 Thread Robert Haas
On Mon, Nov 20, 2017 at 12:05 PM, Antonin Houska wrote: > Robert Haas wrote: >> On Wed, Aug 30, 2017 at 10:12 PM, Tatsuro Yamada >> wrote: >> > 1. scanning heap >> > 2. sort tuples >> >> These two phases overlap, though. I believe progress reporting for >> sorts is really hard. In the simpl

Re: [HACKERS] [PATCH] Incremental sort

2017-11-21 Thread Thomas Munro
On Tue, Nov 21, 2017 at 1:00 PM, Alexander Korotkov wrote: > On Mon, Nov 20, 2017 at 12:24 AM, Thomas Munro > wrote: >> Is there some reason not to use ApplySortComparator for this? I think >> you're missing out on lower-overhead comparators, and in any case it's >> probably good code reuse, no?

Re: feature request: consume asynchronous notification via a function

2017-11-21 Thread Merlin Moncure
On Tue, Nov 21, 2017 at 2:16 PM, Merlin Moncure wrote: > On Tue, Nov 21, 2017 at 12:20 PM, Tom Lane wrote: >> Robert Haas writes: >>> On Tue, Nov 21, 2017 at 11:32 AM, Merlin Moncure wrote: I am very much looking at the new stored procedure functionality and imaging a loop like this:

Re: [HACKERS] Replication status in logical replication

2017-11-21 Thread Masahiko Sawada
On Tue, Nov 14, 2017 at 6:46 AM, Thomas Munro wrote: > On Tue, Sep 26, 2017 at 3:45 PM, Masahiko Sawada > wrote: >> On Tue, Sep 26, 2017 at 10:36 AM, Vaishnavi Prabakaran >> wrote: >>> On Wed, Sep 13, 2017 at 9:59 AM, Daniel Gustafsson wrote: I’m not entirely sure why this was flagged as

Re: [HACKERS] CLUSTER command progress monitor

2017-11-21 Thread Peter Geoghegan
On Mon, Oct 2, 2017 at 6:04 AM, Robert Haas wrote: > Progress reporting on sorts seems like a tricky problem to me, as I > said before. In most cases, a sort is going to involve an initial > stage where it reads all the input tuples and writes out quicksorted > runs, and then a merge phase where

Re: Logical Replication and triggers

2017-11-21 Thread Thomas Rosenstein
On Tue, Nov 21, 2017 at 3:29 PM, Simon Riggs wrote: You realize we're talking about a bug fix, right? And for a feature that was developed and committed by your colleagues? Craig is asking Thomas to confirm the proposed bug fix works. How is this not normal? That's not exactly how I read Cr

Re: [HACKERS] pgbench regression test failure

2017-11-21 Thread Tom Lane
Robert Haas writes: > On Tue, Nov 21, 2017 at 3:28 PM, Tom Lane wrote: >> or just >> ... latency limit: 33/33 (100.000 %) > Oh, yeah. That last one sounds good; no reason to print the same > value more than once. Sold; I'll go make it so. regards, tom lane

Re: [HACKERS] CLUSTER command progress monitor

2017-11-21 Thread Peter Geoghegan
On Tue, Nov 21, 2017 at 12:55 PM, Robert Haas wrote: > I agree. > > I have been of the opinion all along that progress monitoring needs to > report facts, not theories. The number of tuples read thus far is a > fact, and is fine to report for whatever value it may have to someone. That makes a l

Re: Logical Replication and triggers

2017-11-21 Thread Simon Riggs
On 21 November 2017 at 16:13, Thomas Rosenstein wrote: > To weigh in here, I actually find it's a big hurdle > > I'm a postgres user and not a postgres dev, so I definitly have the feeling > I'm not qualified to answer if this really does what it's intended todo. > Further more not beeing in the

Re: default range partition and constraint exclusion

2017-11-21 Thread Robert Haas
On Tue, Nov 21, 2017 at 4:36 AM, Amit Langote wrote: >>> The attached will make the constraint to look like: >> >> Uh, if the constraint exclusion logic we're using is drawing false >> conclusions, we need to fix it so it doesn't, not change the >> constraint so that the wrong logic gives the righ

Re: Logical Replication and triggers

2017-11-21 Thread Robert Haas
On Tue, Nov 21, 2017 at 4:28 PM, Simon Riggs wrote: > I would have acted on this myself a few days back if I thought the > patch was correct, but I see multiple command counter increments > there, so suspect an alternate fix is correct. Oh, well, I'm glad you said something. I was actually think

Re: [HACKERS] GnuTLS support

2017-11-21 Thread Michael Paquier
On Tue, Nov 21, 2017 at 11:45 PM, Andreas Karlsson wrote: > There is a function called gnutls_session_channel_binding() which seems to > do something very similar to SSL_get*_finished() which has been in GnuTLS > since 2.12. > > https://www.gnutls.org/manual/gnutls.html#Channel-Bindings Nice. Thi

Re: Treating work_mem as a shared resource (Was: Parallel Hash take II)

2017-11-21 Thread Peter Geoghegan
On Tue, Nov 21, 2017 at 7:29 AM, Robert Haas wrote: > Hash joins are a place where we could have a smoother cost function > than we do. Yes, it definitely is. > When we run out of memory, instead of switching from > (say) a single batch to two batches, switch to 64 batches, but > initially keep

Re: [HACKERS] More stats about skipped vacuums

2017-11-21 Thread Michael Paquier
On Tue, Nov 21, 2017 at 4:09 PM, Kyotaro HORIGUCHI wrote: > By the way I'm uneasy that the 'last_vacuum_index_scans' (and > vacuum_fail_count in 0002 and others in 0003, 0004) is mentioning > both VACUUM command and autovacuum, while last_vacuum and > vacuum_count is mentioning only the command. S

Re: [HACKERS] Avoiding OOM in a hash join with many duplicate inner keys

2017-11-21 Thread Thomas Munro
On Wed, Mar 8, 2017 at 1:29 PM, Tom Lane wrote: > Thomas Munro writes: >> I have been wondering about a couple of different worst case execution >> strategies that would be better than throwing our hands up and >> potentially exploding memory once we detect that further partitioning >> is not goi

Re: Failed to delete old ReorderBuffer spilled files

2017-11-21 Thread Kyotaro HORIGUCHI
At Tue, 21 Nov 2017 20:53:04 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20171121.205304.90315453.horiguchi.kyot...@lab.ntt.co.jp> > At Tue, 21 Nov 2017 20:27:25 +0900, atorikoshi > wrote in > > > Thanks for reviewing! > > > > > > On 2017/11/21 18:12, Masahiko Sawada wrote: >

Re: Anybody care about having the verbose form of the tzdata files?

2017-11-21 Thread Michael Paquier
On Tue, Nov 21, 2017 at 6:28 PM, Daniel Gustafsson wrote: >> On 20 Nov 2017, at 21:38, Tom Lane wrote: >> Anybody here actually care about reading the zone data files? > > I doubt there is anyone who cares about that who isn’t already consuming the > upstream data. Perhaps I do. If this set of f

Re: Logical Replication and triggers

2017-11-21 Thread Michael Paquier
On Wed, Nov 22, 2017 at 6:35 AM, Robert Haas wrote: > On Tue, Nov 21, 2017 at 4:28 PM, Simon Riggs wrote: >> I would have acted on this myself a few days back if I thought the >> patch was correct, but I see multiple command counter increments >> there, so suspect an alternate fix is correct. > >

Re: Anybody care about having the verbose form of the tzdata files?

2017-11-21 Thread Tom Lane
Michael Paquier writes: > On Tue, Nov 21, 2017 at 6:28 PM, Daniel Gustafsson wrote: > On 20 Nov 2017, at 21:38, Tom Lane wrote: >>> Anybody here actually care about reading the zone data files? >> I doubt there is anyone who cares about that who isn’t already consuming the >> upstream data. >

Re: [HACKERS] Issues with logical replication

2017-11-21 Thread Michael Paquier
On Fri, Nov 17, 2017 at 5:12 AM, Robert Haas wrote: > On Thu, Nov 16, 2017 at 2:41 PM, Andres Freund wrote: >>> To me, it seems like SnapBuildWaitSnapshot() is fundamentally >>> misdesigned >> >> Maybe I'm confused, but why is it fundamentally misdesigned? It's not >> such an absurd idea to wait

Re: Anybody care about having the verbose form of the tzdata files?

2017-11-21 Thread Michael Paquier
On Wed, Nov 22, 2017 at 9:34 AM, Tom Lane wrote: > Michael Paquier writes: >> On Tue, Nov 21, 2017 at 6:28 PM, Daniel Gustafsson wrote: >> On 20 Nov 2017, at 21:38, Tom Lane wrote: Anybody here actually care about reading the zone data files? >> Perhaps I do. If this set of files gets remo

Re: [HACKERS] CLUSTER command progress monitor

2017-11-21 Thread Michael Paquier
On Wed, Nov 22, 2017 at 5:55 AM, Robert Haas wrote: > I have been of the opinion all along that progress monitoring needs to > report facts, not theories. The number of tuples read thus far is a > fact, and is fine to report for whatever value it may have to someone. > The number of tuples that w

Re: [HACKERS] CLUSTER command progress monitor

2017-11-21 Thread Thomas Munro
On Wed, Nov 22, 2017 at 1:53 PM, Michael Paquier wrote: > On Wed, Nov 22, 2017 at 5:55 AM, Robert Haas wrote: >> I have been of the opinion all along that progress monitoring needs to >> report facts, not theories. The number of tuples read thus far is a >> fact, and is fine to report for whatev

Re: Failed to delete old ReorderBuffer spilled files

2017-11-21 Thread Masahiko Sawada
On Wed, Nov 22, 2017 at 9:03 AM, Kyotaro HORIGUCHI wrote: > At Tue, 21 Nov 2017 20:53:04 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI > wrote in > <20171121.205304.90315453.horiguchi.kyot...@lab.ntt.co.jp> >> At Tue, 21 Nov 2017 20:27:25 +0900, atorikoshi >> wrote in >> >> > Thanks for re

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-21 Thread Michael Paquier
On Tue, Nov 21, 2017 at 4:18 AM, Andres Freund wrote: > On 2017-11-13 19:03:41 -0800, Andres Freund wrote: >> On 2017-11-03 07:53:30 -0700, Andres Freund wrote: >> > Here's that patch. I've stared at this some, and Robert did too. Robert >> > mentioned that the commit message might need some poli

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2017-11-21 Thread Masahiko Sawada
On Wed, Nov 22, 2017 at 5:25 AM, Robert Haas wrote: > On Mon, Nov 20, 2017 at 5:19 PM, Masahiko Sawada > wrote: >> Attached updated version patch. I've moved only relation extension >> locks out of heavy-weight lock as per discussion so far. >> >> I've done a write-heavy benchmark on my laptop;

Re: Failed to delete old ReorderBuffer spilled files

2017-11-21 Thread Kyotaro HORIGUCHI
Hi, At Wed, 22 Nov 2017 10:10:27 +0900, Masahiko Sawada wrote in > >> Using last changing LSN might work but I'm afraid that that fails > >> to remove the last snap file if the crash happens at the very > >> start of a segment. > > I think it works even in the case because the we can compute t

Re: Failed to delete old ReorderBuffer spilled files

2017-11-21 Thread Craig Ringer
On 20 November 2017 at 18:35, atorikoshi wrote: > Hi, > > I put many queries into one transaction and made ReorderBuffer spill > data to disk, and sent SIGKILL to postgres before the end of the > transaction. > > After starting up postgres again, I observed the files spilled to > data wasn't dele

Re: Logical Replication and triggers

2017-11-21 Thread Craig Ringer
On 22 November 2017 at 02:27, Robert Haas wrote: > On Sat, Nov 18, 2017 at 7:30 PM, Craig Ringer > wrote: > > On 15 November 2017 at 21:12, Thomas Rosenstein > > wrote: > >> I would like somebody to consider Petr Jelineks patch for worker.c from > >> here > >> (https://www.postgresql.org/messag

Re: Logical Replication and triggers

2017-11-21 Thread Craig Ringer
On 22 November 2017 at 05:35, Robert Haas wrote: > On Tue, Nov 21, 2017 at 4:28 PM, Simon Riggs > wrote: > > I would have acted on this myself a few days back if I thought the > > patch was correct, but I see multiple command counter increments > > there, so suspect an alternate fix is correct.

generic-msvc.h(91): error C2664

2017-11-21 Thread Vicky Vergara
Hello all I am trying to compile pgRouting on appveyor CI when compiling with pg >= 9.5 I am getting the following error: C:\Program Files\PostgreSQL\9.5\include\server\port/atomics/generic-msvc.h(91): error C2664: '__int64 _InterlockedCompareExchange64(volatile __int64 *,__int64,__int64)'

Re: Failed to delete old ReorderBuffer spilled files

2017-11-21 Thread Michael Paquier
On Wed, Nov 22, 2017 at 11:49 AM, Craig Ringer wrote: > On 20 November 2017 at 18:35, atorikoshi > wrote: >> I put many queries into one transaction and made ReorderBuffer spill >> data to disk, and sent SIGKILL to postgres before the end of the >> transaction. >> >> After starting up postgres ag

Re: [HACKERS] More stats about skipped vacuums

2017-11-21 Thread Kyotaro HORIGUCHI
Hello, At Wed, 22 Nov 2017 08:20:22 +0900, Michael Paquier wrote in > On Tue, Nov 21, 2017 at 4:09 PM, Kyotaro HORIGUCHI > wrote: > > By the way I'm uneasy that the 'last_vacuum_index_scans' (and > > vacuum_fail_count in 0002 and others in 0003, 0004) is mentioning > > both VACUUM command and

Re: [HACKERS] More stats about skipped vacuums

2017-11-21 Thread Michael Paquier
On Wed, Nov 22, 2017 at 1:08 PM, Kyotaro HORIGUCHI wrote: > At Wed, 22 Nov 2017 08:20:22 +0900, Michael Paquier > wrote in > >> On Tue, Nov 21, 2017 at 4:09 PM, Kyotaro HORIGUCHI >> wrote: >> > By the way I'm uneasy that the 'last_vacuum_index_scans' (and >> > vacuum_fail_count in 0002 and ot

Re: Failed to delete old ReorderBuffer spilled files

2017-11-21 Thread Kyotaro HORIGUCHI
At Wed, 22 Nov 2017 12:57:34 +0900, Michael Paquier wrote in > On Wed, Nov 22, 2017 at 11:49 AM, Craig Ringer wrote: > > On 20 November 2017 at 18:35, atorikoshi > > wrote: > >> I put many queries into one transaction and made ReorderBuffer spill > >> data to disk, and sent SIGKILL to postgres

Re: [HACKERS] Issues with logical replication

2017-11-21 Thread Craig Ringer
On 4 October 2017 at 07:35, Petr Jelinek wrote: > On 02/10/17 18:59, Petr Jelinek wrote: > >> > >> Now fix the trigger function: > >> CREATE OR REPLACE FUNCTION replication_trigger_proc() RETURNS TRIGGER > AS $$ > >> BEGIN > >> RETURN NEW; > >> END $$ LANGUAGE plpgsql; > >> > >> And manually pe

Re: [HACKERS] Support to COMMENT ON DATABASE CURRENT_DATABASE

2017-11-21 Thread Jing Wang
Hi Nathan, Thanks for review comments. Enclosed please find the patch which has been updated according to your suggestion. The CURRENT_DATABASE can be used as following SQL statements and people can find information from sgml files: 1. COMMENT ON DATABASE CURRENT_DATABASE is ... 2. ALTER

With commit 4e5fe9ad19, range partition missing handling for the NULL partition key

2017-11-21 Thread Rushabh Lathia
Consider the below test: CREATE TABLE range_tab(a int, b int) PARTITION BY RANGE(a); CREATE TABLE range_tab_p1 PARTITION OF range_tab FOR VALUES FROM (minvalue) TO (10); CREATE TABLE range_tab_p2 PARTITION OF range_tab FOR VALUES FROM (10) TO (20); CREATE TABLE range_tab_p3 PARTITION OF range_tab

Re: [HACKERS] Issues with logical replication

2017-11-21 Thread Simon Riggs
On 4 October 2017 at 10:35, Petr Jelinek wrote: > On 02/10/17 18:59, Petr Jelinek wrote: >>> >>> Now fix the trigger function: >>> CREATE OR REPLACE FUNCTION replication_trigger_proc() RETURNS TRIGGER AS $$ >>> BEGIN >>> RETURN NEW; >>> END $$ LANGUAGE plpgsql; >>> >>> And manually perform at ma

Re: Logical Replication and triggers

2017-11-21 Thread Simon Riggs
On 22 November 2017 at 08:35, Robert Haas wrote: > On Tue, Nov 21, 2017 at 4:28 PM, Simon Riggs wrote: >> I would have acted on this myself a few days back if I thought the >> patch was correct, but I see multiple command counter increments >> there, so suspect an alternate fix is correct. > > Oh

Re: Failed to delete old ReorderBuffer spilled files

2017-11-21 Thread Craig Ringer
On 22 November 2017 at 12:15, Kyotaro HORIGUCHI < horiguchi.kyot...@lab.ntt.co.jp> wrote: > At Wed, 22 Nov 2017 12:57:34 +0900, Michael Paquier < > michael.paqu...@gmail.com> wrote in tzwhp2oeervv7r...@mail.gmail.com> > > On Wed, Nov 22, 2017 at 11:49 AM, Craig Ringer > wrote: > > > On 20 Novemb

Re: With commit 4e5fe9ad19, range partition missing handling for the NULL partition key

2017-11-21 Thread Amit Langote
Hi Rushabh, On 2017/11/22 13:45, Rushabh Lathia wrote: > Consider the below test: > > CREATE TABLE range_tab(a int, b int) PARTITION BY RANGE(a); > CREATE TABLE range_tab_p1 PARTITION OF range_tab FOR VALUES FROM (minvalue) > TO (10); > CREATE TABLE range_tab_p2 PARTITION OF range_tab FOR VALUES

Re: [HACKERS] [WIP] Zipfian distribution in pgbench

2017-11-21 Thread Fabien COELHO
Note that the patch may interact with other patches which add functions to pgbench, so might need a rebase depending on the order in which the patch are applied. Attached a minor rebase after 16827d4424. -- Fabien.diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml ind

Re: Logical Replication and triggers

2017-11-21 Thread Thomas Rosenstein
On 22 November 2017 at 08:35, Robert Haas wrote: On Tue, Nov 21, 2017 at 4:28 PM, Simon Riggs wrote: I would have acted on this myself a few days back if I thought the patch was correct, but I see multiple command counter increments there, so suspect an alternate fix is correct. Oh, well, I'

  1   2   >