Re: plan cache overhead on plpgsql expression

2020-03-26 Thread Tom Lane
I wrote: > I had a thought about a possibly-cleaner way to do this. We could invent > a resowner function, say ResourceOwnerReleaseAllPlanCacheRefs, that > explicitly releases all plancache pins it knows about. So plpgsql > would not call the regular ResourceOwnerRelease entry point at all, >

Re: [PATCH] Fix CommitTransactionCommand() to CallXactCallbacks() in TBLOCK_ABORT_END

2020-03-26 Thread Gilles Darold
Le 25/03/2020 à 03:49, Dave Sharpe a écrit : > > Hi pghackers, > > This is my first time posting here ...  Gilles Darold and I are > developing a new FDW which is based on the contrib/postgres_fdw. The > postgres_fdw logic uses a RegisterXactCallback function to send local > transactions remote.

Re: Patch: to pass query string to pg_plan_query()

2020-03-26 Thread Julien Rouhaud
On Thu, Mar 26, 2020 at 11:44:44AM -0400, Tom Lane wrote: > Fujii Masao writes: > > Does anyone object to this patch? I'm thinking to commit it separetely > > at first before committing the planning_counter_in_pg_stat_statements > > patch. > > I took a quick look through

Re: proposal \gcsv

2020-03-26 Thread Vik Fearing
On 3/24/20 3:02 AM, Pavel Stehule wrote: > Hi > > rebase Thank you, Pavel. I have now had time to review it, and it looks good to me except for two issues. The first is, even though I suggested gf, I think it should actually be gfmt. There may be something else in the future that starts with

Re: plan cache overhead on plpgsql expression

2020-03-26 Thread Tom Lane
Amit Langote writes: > One thing -- I don't get the division between > CachedPlanAllowsSimpleValidityCheck() and CachedPlanIsSimplyValid(). > Maybe I am missing something, but could there not be just one > function, possibly using whether expr_simple_expr is set or not to > skip or do, resp., the

Re: backup manifests

2020-03-26 Thread Mark Dilger
> On Mar 26, 2020, at 9:34 AM, Stephen Frost wrote: > > I'm not actually argueing about which hash functions we should support, > but rather what the default is and if crc32c, specifically, is actually > a reasonable choice. Just because it's fast and we already had an > implementation of it

Re: plan cache overhead on plpgsql expression

2020-03-26 Thread Andres Freund
Hi, On 2020-03-26 14:37:59 -0400, Tom Lane wrote: > I wrote: > > I had a thought about a possibly-cleaner way to do this. We could invent > > a resowner function, say ResourceOwnerReleaseAllPlanCacheRefs, that > > explicitly releases all plancache pins it knows about. So plpgsql > > would not

Re: Berserk Autovacuum (let's save next Mandrill)

2020-03-26 Thread Andres Freund
Hi, On 2020-03-26 10:12:39 +0100, Laurenz Albe wrote: > On Wed, 2020-03-25 at 23:19 +0300, Alexander Korotkov wrote: > I am reluctant to introduce new semantics like a reloption value of -2 > to disable a feature in this patch right before feature freeze. > > I believe there are enough options

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-03-26 Thread Alexey Kondratov
On 2020-03-26 02:40, Justin Pryzby wrote: On Thu, Mar 12, 2020 at 08:08:46PM +0300, Alexey Kondratov wrote: On 09.03.2020 23:04, Justin Pryzby wrote: On Sat, Feb 29, 2020 at 08:53:04AM -0600, Justin Pryzby wrote: On Sat, Feb 29, 2020 at 03:35:27PM +0300, Alexey Kondratov wrote: tests for

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-03-26 Thread Justin Pryzby
> I included your new solution regarding this part from 0004 into 0001. It > seems that at least a tip of the problem was in that we tried to change > tablespace to pg_default being already there. Right, causing it to try to drop that filenode twice. > +++ b/doc/src/sgml/ref/cluster.sgml > +

Re: Columns correlation and adaptive query optimization

2020-03-26 Thread Konstantin Knizhnik
On 25.03.2020 20:04, Rafia Sabih wrote: Also, there is no description about any of the functions here, wouldn’t hurt having some more comments there. Attached please find new version of the patch with more comments and descriptions added. -- Konstantin Knizhnik Postgres Professional:

Re: backup manifests

2020-03-26 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Mar 25, 2020 at 4:54 PM Stephen Frost wrote: > > > That's a fair argument, but I think the other relevant principle is > > > that we try to give people useful defaults for things. I think that > > > checksums are a sufficiently

Re: backup manifests

2020-03-26 Thread Robert Haas
On Thu, Mar 26, 2020 at 12:34 PM Stephen Frost wrote: > I do agree with excluding things like md5 and others that aren't good > options. I wasn't saying we should necessarily exclude crc32c either.. > but rather saying that it shouldn't be the default. > > Here's another way to look at it- where

Re: Patch: to pass query string to pg_plan_query()

2020-03-26 Thread Tom Lane
Fujii Masao writes: > Does anyone object to this patch? I'm thinking to commit it separetely > at first before committing the planning_counter_in_pg_stat_statements > patch. I took a quick look through v9-0001-Pass-query-string-to-the-planner.patch and it's fine by me. It also matches up with

Re: proposal \gcsv

2020-03-26 Thread Erik Rijkers
On 2020-03-26 18:49, Pavel Stehule wrote: Hi [psql-gfmt.patch] This seems useful and works well; I haven't found any errors. Well done. However, I have a suggestion that is perhaps slightly outside of this patch but functionally so close that maybe we can discuss it here. When you try to

Re: backup manifests

2020-03-26 Thread Robert Haas
On Wed, Mar 25, 2020 at 4:54 PM Stephen Frost wrote: > > That's a fair argument, but I think the other relevant principle is > > that we try to give people useful defaults for things. I think that > > checksums are a sufficiently useful thing that having the default be > > not to do it doesn't

Re: Make mesage at end-of-recovery less scary.

2020-03-26 Thread Robert Haas
On Wed, Mar 25, 2020 at 8:53 AM Peter Eisentraut wrote: > HINT: This is to be expected if this is the end of the WAL. Otherwise, > it could indicate corruption. First, I agree that this general issue is a problem, because it's come up for me in quite a number of customer situations. Either

Tab completion for \gx

2020-03-26 Thread Vik Fearing
While reviewing the patch for \gf, I noticed that \gx does not have tab completion for its optional filename. Trivial patch attached. I would also suggest this be backpatched. -- Vik Fearing diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c index ae35fa4aa9..7252b6c4e6

Re: proposal \gcsv

2020-03-26 Thread Pavel Stehule
čt 26. 3. 2020 v 18:55 odesílatel Vik Fearing napsal: > On 3/26/20 10:49 AM, Pavel Stehule wrote: > > Hi > > > > čt 26. 3. 2020 v 17:45 odesílatel Vik Fearing > > napsal: > > > >> After some opinions on the first issue and fixing the second, I think > >> this is good to be committed. > >> > > >

Re: plan cache overhead on plpgsql expression

2020-03-26 Thread Tom Lane
Andres Freund writes: > On 2020-03-26 14:37:59 -0400, Tom Lane wrote: >> + * This function, together with CachedPlanIsSimplyValid, provides a fast >> path >> + * for revalidating "simple" generic plans. The core requirement to be >> simple >> + * is that the plan must not require taking any

Re: proposal \gcsv

2020-03-26 Thread Vik Fearing
On 3/26/20 10:49 AM, Pavel Stehule wrote: > Hi > > čt 26. 3. 2020 v 17:45 odesílatel Vik Fearing > napsal: > >> After some opinions on the first issue and fixing the second, I think >> this is good to be committed. >> > > Thank you for review This patch now looks good to me. Marking as Ready

Re: A bug when use get_bit() function for a long bytea string

2020-03-26 Thread Ashutosh Bapat
On Thu, 26 Mar 2020 at 19:39, Tom Lane wrote: > Ashutosh Bapat writes: > > On Wed, 18 Mar 2020 at 08:18, movead...@highgo.ca > > wrote: > >> if we change return type of all those functions to int64, we won't need > >> this cast. > >> I change the 'encode' function, it needs an int64 return

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-03-26 Thread Alexey Kondratov
On 2020-03-26 21:01, Justin Pryzby wrote: @@ -262,7 +280,7 @@ cluster(ClusterStmt *stmt, bool isTopLevel) * and error messages should refer to the operation as VACUUM not CLUSTER. */ void -cluster_rel(Oid tableOid, Oid indexOid, int options) +cluster_rel(Oid tableOid, Oid indexOid, Oid

Re: pgbench - add \aset to store results of a combined query

2020-03-26 Thread Fabien COELHO
Bonjour Michaël, [...] Still sounds strange to me to invent a new variable to this structure if it is possible to track the exact same thing with an existing part of a Command, or it would make sense to split Command into two different structures with an extra structure used after the

Re: Include sequence relation support in logical replication

2020-03-26 Thread Cary Huang
Hi Andres thanks for your reply and your patch review. Please see my comments below >On 2020-03-24 16:19:21 -0700, Cary Huang wrote: >> I have shared a patch that allows sequence relation to be supported in >> logical replication via the decoding plugin ( test_decoding for >> example

Re: error context for vacuum to include block number

2020-03-26 Thread Justin Pryzby
On Thu, Mar 26, 2020 at 10:04:57AM -0500, Justin Pryzby wrote: > Does that address your comment ? I hope so. > > I'm not sure why "free_oldindname" is necessary. Since we initialize > > vacrelstats->indname with NULL and revert the callback arguments at > > the end of functions that needs update

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2020-03-26 Thread Michael Paquier
On Thu, Mar 26, 2020 at 06:56:36PM -0300, Alvaro Herrera wrote: > Uh, is XLOGDIR the only reason to include xlog_internal.h? Maybe it > would be easier to have a routine in xlog.c that returns the path? > There's a few functions in xlog.c that could use it, as well. Yep, that's all. We could

Re: backup manifests

2020-03-26 Thread Stephen Frost
Greetings, * Mark Dilger (mark.dil...@enterprisedb.com) wrote: > > On Mar 26, 2020, at 9:34 AM, Stephen Frost wrote: > > I'm not actually argueing about which hash functions we should support, > > but rather what the default is and if crc32c, specifically, is actually > > a reasonable choice.

Re: backup manifests

2020-03-26 Thread Mark Dilger
> On Mar 26, 2020, at 12:37 PM, Stephen Frost wrote: > > Greetings, > > * Mark Dilger (mark.dil...@enterprisedb.com) wrote: >>> On Mar 26, 2020, at 9:34 AM, Stephen Frost wrote: >>> I'm not actually argueing about which hash functions we should support, >>> but rather what the default is

Re: backup manifests

2020-03-26 Thread David Steele
On 3/26/20 11:37 AM, Robert Haas wrote: On Wed, Mar 25, 2020 at 4:54 PM Stephen Frost wrot > This is where I feel like I'm trying to make decisions in a vacuum. If we had a few more people weighing in on the thread on this point, I'd be happy to go with whatever the consensus was. If most

Re: Berserk Autovacuum (let's save next Mandrill)

2020-03-26 Thread David Rowley
On Fri, 27 Mar 2020 at 07:51, Andres Freund wrote: > > Hi, > > On 2020-03-26 10:12:39 +0100, Laurenz Albe wrote: > > On Wed, 2020-03-25 at 23:19 +0300, Alexander Korotkov wrote: > > I am reluctant to introduce new semantics like a reloption value of -2 > > to disable a feature in this patch right

Re: proposal \gcsv

2020-03-26 Thread Pavel Stehule
Hi čt 26. 3. 2020 v 17:45 odesílatel Vik Fearing napsal: > On 3/24/20 3:02 AM, Pavel Stehule wrote: > > Hi > > > > rebase > > Thank you, Pavel. > > I have now had time to review it, and it looks good to me except for two > issues. > > The first is, even though I suggested gf, I think it should

Re: backup manifests

2020-03-26 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Thu, Mar 26, 2020 at 12:34 PM Stephen Frost wrote: > > I do agree with excluding things like md5 and others that aren't good > > options. I wasn't saying we should necessarily exclude crc32c either.. > > but rather saying that it

Re: Ltree syntax improvement

2020-03-26 Thread Nikita Glukhov
On 25.03.2020 2:08, Tom Lane wrote: Nikita Glukhov writes: Attached new version of the patch. I spent a little bit of time looking through this, and have a few comments: * You have a lot of places where tests for particular ASCII characters are done like this: if ((charlen == 1) &&

Re: error context for vacuum to include block number

2020-03-26 Thread Alvaro Herrera
On 2020-Mar-26, Justin Pryzby wrote: > On Thu, Mar 26, 2020 at 10:04:57AM -0500, Justin Pryzby wrote: > > And I think you're right: we only save state when the calling function has a > > indname=NULL, so we never "put back" a non-NULL indname. We go from having > > a > > indname=NULL at

Re: error context for vacuum to include block number

2020-03-26 Thread Amit Kapila
On Thu, Mar 26, 2020 at 12:03 PM Amit Kapila wrote: > > On Thu, Mar 26, 2020 at 10:11 AM Justin Pryzby wrote: > > > > Seems fine. Rather than saying "different phases" I, would say: > > "The index vacuum and heap vacuum phases may be called multiple times in the > > middle of the heap scan

Re: error context for vacuum to include block number

2020-03-26 Thread Amit Kapila
On Thu, Mar 26, 2020 at 10:11 AM Justin Pryzby wrote: > > Seems fine. Rather than saying "different phases" I, would say: > "The index vacuum and heap vacuum phases may be called multiple times in the > middle of the heap scan phase." > Okay, I have slightly adjusted the wording as per your

Re: [DOC] Document concurrent index builds waiting on each other

2020-03-26 Thread Michael Paquier
On Wed, Mar 25, 2020 at 05:12:48PM -0300, Alvaro Herrera wrote: > Hmm, that sounds more promising. Haven't looked at that myself in details. But as I doubt that this would be backpatched, wouldn't it be better to document the issue for now? -- Michael signature.asc Description: PGP signature

Re: color by default

2020-03-26 Thread Michael Paquier
On Mon, Mar 23, 2020 at 09:32:08AM +0100, Peter Eisentraut wrote: > Attached is the documentation patch reworked. Thanks! > Should we delete all the repetitive mentions on the man pages? I am not sure that deleting all the mentions would be a good idea, as we'd lose track of which tool supports

Re: allow online change primary_conninfo

2020-03-26 Thread Michael Paquier
On Wed, Mar 25, 2020 at 09:08:12PM -0300, Alvaro Herrera wrote: > ... as in the attached version. Patch 0001 is one that has already been discussed previously (thanks for keeping the extra comments about GUCs and WAL receivers). That looks fine to me. > Sergei included LOG messages to indicate

Re: Include sequence relation support in logical replication

2020-03-26 Thread Michael Paquier
On Wed, Mar 25, 2020 at 12:27:28PM -0700, Andres Freund wrote: > On 2020-03-24 16:19:21 -0700, Cary Huang wrote: >> For the replication to make sense, the patch actually disables the WAL >> update at every 32 nextval() calls, so every call to nextval() will >> emit a WAL update for proper

Re: allow online change primary_conninfo

2020-03-26 Thread Michael Paquier
On Thu, Mar 26, 2020 at 09:39:17PM -0300, Alvaro Herrera wrote: > Now, would anyone be too upset if I push these in this other order? I > realized that the reason the tests broke after Sergei's patch is that > recovery/t/001_stream_rep.pl's get_slot_xmins() is broken for temp > walreceiver slots,

RE: Conflict handling for COPY FROM

2020-03-26 Thread asaba.takan...@fujitsu.com
Hello Surafel, From: Surafel Temesgen >An error that can be surly handled without transaction rollback can >be included in error handling but i will like to proceed without binary file >errors handling for the time being Thank you. Also it seems that you apply Alexey's comment. So I'll mark

Re: Race condition in SyncRepGetSyncStandbysPriority

2020-03-26 Thread Fujii Masao
On 2020/03/27 10:26, Tom Lane wrote: Twice in the past month [1][2], buildfarm member hoverfly has managed to reach the "unreachable" Assert(false) at the end of SyncRepGetSyncStandbysPriority. When I search the past discussions, I found that Noah Misch reported the same issue.

Re: backup manifests

2020-03-26 Thread Andres Freund
Hi, On 2020-03-26 15:37:11 -0400, Stephen Frost wrote: > The argument is that adding checksums takes more time. I can understand > that argument, though I don't really agree with it. Certainly a few > percent really shouldn't be that big of an issue, and in many cases even > a sha256 hash isn't

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-03-26 Thread Justin Pryzby
> Another issue is this: > > +VACUUM ( FULL [, ...] ) [ TABLESPACE > class="parameter">new_tablespace ] [ > class="parameter">table_and_columns [, ...] ] > As you mentioned in your v1 patch, in the other cases, "tablespace > [tablespace]" is added at the end of the command rather than in the

Re: error context for vacuum to include block number

2020-03-26 Thread Amit Kapila
On Fri, Mar 27, 2020 at 3:47 AM Justin Pryzby wrote: > > > > Hm, I was just wondering what happens if an error happens *during* > > update_vacuum_error_cbarg(). It seems like if we set > > errcbarg->phase=VACUUM_INDEX before setting errcbarg->indname=indname, then > > an > > error would cause a

Re: backup manifests

2020-03-26 Thread Andres Freund
Hi, On 2020-03-26 14:02:29 -0400, Robert Haas wrote: > On Thu, Mar 26, 2020 at 12:34 PM Stephen Frost wrote: > > Why was crc32c > > picked for that purpose? > > Because it was discovered that 64-bit CRC was too slow, per commit > 21fda22ec46deb7734f793ef4d7fa6c226b4c78e. Well, a 32bit crc, not

Re: error context for vacuum to include block number

2020-03-26 Thread Justin Pryzby
On Thu, Mar 26, 2020 at 07:49:51PM -0300, Alvaro Herrera wrote: > > > ... So once we've "reverted back", 1) the pointer is null; and, 2) > > > the callback function doesn't access it for the previous/reverted > > > phase anyway. > > BTW I'm pretty sure this "revert back" phrasing is not good

Re: [HACKERS] make async slave to wait for lsn to be replayed

2020-03-26 Thread Kartyshov Ivan
Anna, thank you for your review. On 2020-03-25 21:10, Anna Akenteva wrote: On 2020-03-21 14:16, Kartyshov Ivan wrote: and event is: LSN value [options] TIMESTAMP value I would maybe remove WAIT FOR TIMESTAMP. As Robert Haas has pointed out, it seems a lot like

Re: Some problems of recovery conflict wait events

2020-03-26 Thread Fujii Masao
On 2020/03/26 14:33, Masahiko Sawada wrote: On Tue, 24 Mar 2020 at 17:04, Fujii Masao wrote: On 2020/03/05 20:16, Fujii Masao wrote: On 2020/03/05 16:58, Masahiko Sawada wrote: On Wed, 4 Mar 2020 at 15:21, Fujii Masao wrote: On 2020/03/04 14:31, Masahiko Sawada wrote: On Wed, 4

Re: Memory-Bounded Hash Aggregation

2020-03-26 Thread Tomas Vondra
On Thu, Mar 26, 2020 at 05:56:56PM +0800, Richard Guo wrote: Hello, When calculating the disk costs of hash aggregation that spills to disk, there is something wrong with how we determine depth: depth = ceil( log(nbatches - 1) / log(num_partitions) ); If nbatches is some number

Re: error context for vacuum to include block number

2020-03-26 Thread Justin Pryzby
On Fri, Mar 27, 2020 at 09:49:29AM +0530, Amit Kapila wrote: > On Fri, Mar 27, 2020 at 3:47 AM Justin Pryzby wrote: > > > > > Hm, I was just wondering what happens if an error happens *during* > > > update_vacuum_error_cbarg(). It seems like if we set > > > errcbarg->phase=VACUUM_INDEX before

Re: SLRU statistics

2020-03-26 Thread Tomas Vondra
Hi, Attached is v3 of the patch with one big change and various small ones. The main change is that it gets rid of the SlruType enum and the new field in SlruCtlData. Instead, the patch now uses the name passed to SimpleLruInit (which is then stored as LWLock tranche name). The counters are

Re: error context for vacuum to include block number

2020-03-26 Thread Alvaro Herrera
On 2020-Mar-26, Justin Pryzby wrote: > On Thu, Mar 26, 2020 at 07:49:51PM -0300, Alvaro Herrera wrote: > > BTW I'm pretty sure this "revert back" phrasing is not good English -- > > you should just use "revert". Maybe get some native speaker's opinion > > on it. > > I'm a native speaker;

Re: allow online change primary_conninfo

2020-03-26 Thread Alvaro Herrera
Now, would anyone be too upset if I push these in this other order? I realized that the reason the tests broke after Sergei's patch is that recovery/t/001_stream_rep.pl's get_slot_xmins() is broken for temp walreceiver slots, since it's using the non-temp name it tries to give to the slot, rather

Race condition in SyncRepGetSyncStandbysPriority

2020-03-26 Thread Tom Lane
Twice in the past month [1][2], buildfarm member hoverfly has managed to reach the "unreachable" Assert(false) at the end of SyncRepGetSyncStandbysPriority. What seems likely to me, after quickly eyeballing the code, is that hoverfly is hitting the blatantly-obvious race condition in that

Re: backup manifests

2020-03-26 Thread Andres Freund
Hi, On 2020-03-26 11:37:48 -0400, Robert Haas wrote: > I mean, you're just repeating the same argument here, and it's just > not valid. Regardless of the file size, the chances of a false > checksum match are literally less than one in a billion. There is > every reason to believe that users will

Re: backup manifests

2020-03-26 Thread Stephen Frost
Greetings, * Mark Dilger (mark.dil...@enterprisedb.com) wrote: > > On Mar 26, 2020, at 12:37 PM, Stephen Frost wrote: > > * Mark Dilger (mark.dil...@enterprisedb.com) wrote: > >>> On Mar 26, 2020, at 9:34 AM, Stephen Frost wrote: > >>> I'm not actually argueing about which hash functions we

Re: plan cache overhead on plpgsql expression

2020-03-26 Thread Tom Lane
Andres Freund writes: > On 2020-03-26 14:37:59 -0400, Tom Lane wrote: >> Testing that reminded me of the other regression test failure I'd seen >> when I first tried to do it: select_parallel.sql shows a WARNING about >> a plancache leak in a parallel worker process. When I looked into the >>

Re: NOT IN subquery optimization

2020-03-26 Thread Li, Zheng
>BTW, so far as I can see, the only reason you're bothering with the whole thing is to compare the size of the subquery output with work_mem, because that's all that subplan_is_hashable does. I wonder whether that consideration is even still necessary in the wake of 1f39bce02. If

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2020-03-26 Thread Alexey Kondratov
On 2020-03-26 10:34, Michael Paquier wrote: On Tue, Mar 24, 2020 at 12:22:16PM +0900, Michael Paquier wrote: Thanks Alvaro and Alexander. 0001 has been applied as of e09ad07. Now for 0002, let's see about it later. Attached is a rebased version, with no actual changes. I was looking at this

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2020-03-26 Thread Alvaro Herrera
On 2020-Mar-26, Michael Paquier wrote: > I was looking at this patch again today and I am rather fine with the > existing semantics. Still I don't like much to name the frontend-side > routine FrontendRestoreArchivedFile() and use a different name than > the backend counterpart because we have

Re: A rather hackish POC for alternative implementation of WITH TIES

2020-03-26 Thread Surafel Temesgen
On Wed, Jan 22, 2020 at 3:35 PM Andrew Gierth wrote: > > "Alvaro" == Alvaro Herrera writes: > > > I was largely holding off on doing further work hoping for some > discussion of which way we should go. If you think my approach is worth > pursuing (I haven't seriously tested the performance,

Re: Resolving the python 2 -> python 3 mess

2020-03-26 Thread Peter Eisentraut
On 2020-03-26 06:46, Marco Atzeri wrote: Am 17.02.2020 um 17:49 schrieb Tom Lane: We've had multiple previous discussions of $SUBJECT (eg [1][2]), without any resolution of what to do exactly. Thinking about this some more, I had an idea that I don't think has been discussed. To wit: 1. On

Re: Memory-Bounded Hash Aggregation

2020-03-26 Thread Richard Guo
Hello, When calculating the disk costs of hash aggregation that spills to disk, there is something wrong with how we determine depth: >depth = ceil( log(nbatches - 1) / log(num_partitions) ); If nbatches is some number between 1.0 and 2.0, we would have a negative depth. As a

Re: Berserk Autovacuum (let's save next Mandrill)

2020-03-26 Thread Laurenz Albe
On Wed, 2020-03-25 at 23:19 +0300, Alexander Korotkov wrote: > On Wed, Mar 25, 2020 at 10:26 PM Andres Freund wrote: > > On 2020-03-25 11:05:21 -0500, Justin Pryzby wrote: > > > Since we talked about how scale_factor can be used to effectively disable > > > this > > > new feature, I thought that

Re: allow online change primary_conninfo

2020-03-26 Thread Sergei Kornilov
Hello Thank you! You were ahead of me. I wanted to double-check my changes this morning before posting. > Sergei included LOG messages to indicate which setting has been changed. > I put these in "#if 0" for now, but I'm thinking to remove these > altogether; No objections. > Not sure if we

Re: Improve handling of parameter differences in physical replication

2020-03-26 Thread Masahiko Sawada
On Thu, 12 Mar 2020 at 04:34, Peter Eisentraut wrote: > > Here is an updated patch that incorporates some of the suggestions. In > particular, some of the warning messages have been rephrased to more > accurate (but also less specific), the warning message at recovery pause > repeats every 1

Re: Collation versions on Windows (help wanted, apply within)

2020-03-26 Thread Juan José Santamaría Flecha
On Wed, Mar 25, 2020 at 4:18 AM Thomas Munro wrote: > > Thanks! Pushed. > Great! > From the things we learned in this thread, I think there is an open > item for someone to write a patch to call EnumSystemLocalesEx() and > populate the initial set of collations, where we use "locale -a" on >

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2020-03-26 Thread Michael Paquier
On Tue, Mar 24, 2020 at 12:22:16PM +0900, Michael Paquier wrote: > Thanks Alvaro and Alexander. 0001 has been applied as of e09ad07. > Now for 0002, let's see about it later. Attached is a rebased > version, with no actual changes. I was looking at this patch again today and I am rather fine

Re: Conflict handling for COPY FROM

2020-03-26 Thread Surafel Temesgen
Hi Takanori Asaba, > > > Although it is a small point, it may be better like this: > +70005 27 36 46 56 -> 70005 27 37 47 57 > > done > I want to discuss about copy from binary file. > It seems that this patch tries to avoid the error that number of field is >

Re: Negative cost is seen for plan node

2020-03-26 Thread Richard Guo
On Tue, Mar 24, 2020 at 12:01 PM Kyotaro Horiguchi wrote: > At Mon, 23 Mar 2020 21:13:48 +0800, Richard Guo > wrote in > > Hi all, > > > > With the following statements on latest master (c81bd3b9), I find > > negative cost for plan nodes. > > > > create table a (i int, j int); > > insert into a

Re: plan cache overhead on plpgsql expression

2020-03-26 Thread Amit Langote
On Thu, Mar 26, 2020 at 4:44 AM Tom Lane wrote: > > Pavel Stehule writes: > > I'll mark this patch as ready for commiters. > > Thanks for reviewing! Amit, do you have any thoughts on this? Thanks for picking this up. Test cases added by your patch really shows why the plancache and the

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-26 Thread Fujii Masao
On 2020/03/25 22:45, Julien Rouhaud wrote: On Wed, Mar 25, 2020 at 10:09:37PM +0900, Fujii Masao wrote: On 2020/03/21 4:30, Julien Rouhaud wrote: On Fri, Mar 20, 2020 at 05:09:05PM +0300, Sergei Kornilov wrote: Hello Yet another is missed in docs: total_time Oh good catch! I rechecked

Re: error context for vacuum to include block number

2020-03-26 Thread Masahiko Sawada
On Thu, 26 Mar 2020 at 15:34, Amit Kapila wrote: > > On Thu, Mar 26, 2020 at 12:03 PM Amit Kapila wrote: > > > > On Thu, Mar 26, 2020 at 10:11 AM Justin Pryzby wrote: > > > > > > Seems fine. Rather than saying "different phases" I, would say: > > > "The index vacuum and heap vacuum phases may

Re: potential stuck lock in SaveSlotToPath()

2020-03-26 Thread Peter Eisentraut
On 2020-03-25 17:56, Robert Haas wrote: On Wed, Mar 25, 2020 at 6:13 AM Peter Eisentraut wrote: Any concerns about applying and backpatching the patch I posted? Not from me. The talk about reorganizing this code doesn't seem very concrete at the moment and would probably not be backpatch

Re: Patch: to pass query string to pg_plan_query()

2020-03-26 Thread Fujii Masao
On 2020/03/10 6:31, legrand legrand wrote: Hello, This is a call for committers, reviewers and users, regarding "planning counters in pg_stat_statements" patch [1] but not only. Does anyone object to this patch? I'm thinking to commit it separetely at first before committing the

Re: A bug when use get_bit() function for a long bytea string

2020-03-26 Thread Tom Lane
Ashutosh Bapat writes: > On Wed, 18 Mar 2020 at 08:18, movead...@highgo.ca > wrote: >> if we change return type of all those functions to int64, we won't need >> this cast. >> I change the 'encode' function, it needs an int64 return type, but keep >> other >> functions in 'pg_encoding', because

Re: pg_checksums in backend/storage/page/README

2020-03-26 Thread Magnus Hagander
On Thu, Mar 26, 2020 at 3:02 PM Daniel Gustafsson wrote: > > I noticed in passing that backend/storage/page/README hadn't gotten the memo > about pg_checksums. The attached tiny diff adds a small mention of it. That seems obvious enough. Pushed, thanks! -- Magnus Hagander Me:

Re: pg_checksums in backend/storage/page/README

2020-03-26 Thread Julien Rouhaud
On Thu, Mar 26, 2020 at 03:02:42PM +0100, Daniel Gustafsson wrote: > I noticed in passing that backend/storage/page/README hadn't gotten the memo > about pg_checksums. The attached tiny diff adds a small mention of it. Good catch! LGTM

Re: adding partitioned tables to publications

2020-03-26 Thread Amit Langote
On Wed, Mar 25, 2020 at 9:29 PM Peter Eisentraut wrote: > On 2020-03-23 06:02, Amit Langote wrote: > > Okay, added some tests. > > > > Attached updated patches. > > I have committed the worker.c refactoring patch. > > "Add subscription support to replicate into partitioned tables" still > has

Re: Resolving the python 2 -> python 3 mess

2020-03-26 Thread Tom Lane
Marco Atzeri writes: > Am 17.02.2020 um 17:49 schrieb Tom Lane: >> 1. On platforms where Python 2.x is still supported, recommend that >> packagers continue to build both plpython2 and plpython3, same as now. > there is some documentation on how to build both ? > The INSTALL gives no hint. It's

Re: replay pause vs. standby promotion

2020-03-26 Thread Sergei Kornilov
Hello > If we don't ignore walreceiver and does try to connect to the master, > a promotion and recovery cannot end forever since new WAL data can > be streamed. You think this behavior is more consistent? We have no simple point to stop replay. Well, except for "immediately" - just one easy

Re: Patch: to pass query string to pg_plan_query()

2020-03-26 Thread Julien Rouhaud
On Thu, Mar 26, 2020 at 10:54:35PM +0900, Fujii Masao wrote: > > On 2020/03/10 6:31, legrand legrand wrote: > > Hello, > > > > This is a call for committers, reviewers and users, > > regarding "planning counters in pg_stat_statements" > > patch [1] but not only. > > Does anyone object to this

Re: Columns correlation and adaptive query optimization

2020-03-26 Thread Konstantin Knizhnik
Thank you very much for review. On 25.03.2020 20:04, Rafia Sabih wrote: +static void +AddMultiColumnStatisticsForNode(PlanState *planstate, ExplainState *es); + This doesn't look like the right place for it, you might want to declare it with other functions in the starting of the file. Also,

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-03-26 Thread Julien Rouhaud
On Thu, Mar 26, 2020 at 08:08:35PM +0900, Fujii Masao wrote: > > On 2020/03/25 22:45, Julien Rouhaud wrote: > > On Wed, Mar 25, 2020 at 10:09:37PM +0900, Fujii Masao wrote: > > > + /* > > > + * We can't process the query if no query_text is provided, as > > > pgss_store > > > + * needs it. We

Re: A bug when use get_bit() function for a long bytea string

2020-03-26 Thread Ashutosh Bapat
On Wed, 18 Mar 2020 at 08:18, movead...@highgo.ca wrote: > > Hello thanks for the detailed review, > > >I think the second argument indicates the bit position, which would be > max bytea length * 8. If max bytea length covers whole int32, the second > argument >needs to be wider i.e. int64. >

Re: plan cache overhead on plpgsql expression

2020-03-26 Thread Tom Lane
Andres Freund writes: > On 2020-03-25 17:51:50 -0400, Tom Lane wrote: >> Andres Freund writes: >>> Hm, any chance that the multiple resowner calls here could show up in a >>> profile? Probably not? >> Doubt it. On the other hand, as the code stands it's certain that the >> resowner contains

pg_checksums in backend/storage/page/README

2020-03-26 Thread Daniel Gustafsson
I noticed in passing that backend/storage/page/README hadn't gotten the memo about pg_checksums. The attached tiny diff adds a small mention of it. cheers ./daniel cksum_page_readme.diff Description: Binary data

Re: replay pause vs. standby promotion

2020-03-26 Thread Fujii Masao
On 2020/03/25 19:42, Sergei Kornilov wrote: Hi  Could we add a few words in func.sgml to clarify the behavior? Especially for users from my example above. Something like:  If a promotion is triggered while recovery is paused, the paused state ends, replay of any WAL immediately

Re: error context for vacuum to include block number

2020-03-26 Thread Justin Pryzby
On Thu, Mar 26, 2020 at 08:56:54PM +0900, Masahiko Sawada wrote: > 1. > @@ -1844,9 +1914,15 @@ lazy_vacuum_page(Relation onerel, BlockNumber > blkno, Buffer buffer, > int uncnt = 0; > TransactionId visibility_cutoff_xid; > boolall_frozen; > + LVRelStats olderrcbarg;