Re: Catalog invalidations vs catalog scans vs ScanPgRelation()

2020-02-28 Thread Andres Freund
Hi, On 2020-02-28 21:24:59 -0800, Andres Freund wrote: > Turns out that I am to blame for that. All the way back in 9.4. For > logical decoding I needed to make ScanPgRelation() use a specific type > of snapshot during one corner case of logical decoding. For reasons lost > to time, I didn't

proposal \gcsv

2020-02-28 Thread Pavel Stehule
Hi I would to enhance \g command about variant \gcsv proposed command has same behave like \g, only the result will be every time in csv format. It can helps with writing psql macros wrapping \g command. Options, notes? Regards Pavel

Catalog invalidations vs catalog scans vs ScanPgRelation()

2020-02-28 Thread Andres Freund
Hi, While self reviewing a patch I'm about to send I changed the assertion in index_getnext_tid from Assert(TransactionIdIsValid(RecentGlobalXmin)) to instead test (via an indirection) Assert(TransactionIdIsValid(MyProc->xmin)) Without ->xmin being set, it's not safe to do scans. And

Re: Making psql error out on output failures

2020-02-28 Thread David Zhang
Hi Alvaro, Thanks for your review, now the new patch with the error message in PG style is attached. On 2020-02-28 8:03 a.m., Alvaro Herrera wrote: On 2020-Feb-18, David Zhang wrote: 3. I think a better way to resolve this issue will still be the solution with an extra %m, which can make

Re: Some problems of recovery conflict wait events

2020-02-28 Thread Masahiko Sawada
On Wed, 26 Feb 2020 at 16:19, Masahiko Sawada wrote: > > On Tue, 18 Feb 2020 at 17:58, Masahiko Sawada > wrote: > > > > Hi all, > > > > When recovery conflicts happen on the streaming replication standby, > > the wait event of startup process is null when > > max_standby_streaming_delay = 0 (to

Re: ALTER tbl rewrite loses CLUSTER ON index

2020-02-28 Thread Justin Pryzby
On Fri, Feb 28, 2020 at 06:26:04PM -0500, Tom Lane wrote: > Justin Pryzby writes: > > I think the attached is 80% complete (I didn't touch pg_dump). > > One objection to this change would be that all relations (including indices) > > end up with relclustered fields, and pg_index already has a

Re: [PATCH] Opclass parameters

2020-02-28 Thread Nikita Glukhov
Attached new version of the patches. On 12.09.2019 3:16, Tomas Vondra wrote: On Wed, Sep 11, 2019 at 01:44:28AM +0300, Nikita Glukhov wrote: On 11.09.2019 1:03, Tomas Vondra wrote: On Tue, Sep 10, 2019 at 04:30:41AM +0300, Nikita Glukhov wrote: 2. New AM method amattoptions().  

Re: Portal->commandTag as an enum

2020-02-28 Thread Tom Lane
Mark Dilger writes: >> On Feb 28, 2020, at 3:05 PM, Tom Lane wrote: >> Is there a way to drop that logic altogether by making the tagname string >> be "INSERT 0" for the INSERT case? Or would the zero bleed into other >> places where we don't want it? > In general, I don't think we want to

Re: Allowing ALTER TYPE to change storage strategy

2020-02-28 Thread Tom Lane
Tomas Vondra writes: > I think we might check if there are any attributes with the given data > type, and allow the change if there are none. That would still allow the > change when the type is used only for things like function parameters > etc. But we'd also have to check for domains

Re: Allowing ALTER TYPE to change storage strategy

2020-02-28 Thread Tomas Vondra
On Fri, Feb 28, 2020 at 01:59:49PM -0500, Tom Lane wrote: Tomas Vondra writes: My understanding is that pg_type.typstorage essentially specifies two things: (a) default storage strategy for the attributes with this type, and (b) whether the type implementation is prepared to handle TOAST-ed

Re: Portal->commandTag as an enum

2020-02-28 Thread Mark Dilger
> On Feb 28, 2020, at 3:05 PM, Tom Lane wrote: > > Alvaro Herrera writes: >> I just realized that we could rename command_tag_display_last_oid() to >> something like command_tag_print_a_useless_zero_for_historical_reasons() >> and nothing would be lost. > > Is there a way to drop that

Re: Assert failure due to "drop schema pg_temp_3 cascade" for temporary tables and \d+ is not showing any info after drooping temp table schema

2020-02-28 Thread Tom Lane
Michael Paquier writes: > On Fri, Feb 28, 2020 at 01:45:29PM -0500, Tom Lane wrote: >> After poking around, I see there aren't any other callers. But I think >> that the cause of this bug is clearly failure to think carefully about >> the different cases that isTempNamespaceInUse is recognizing,

Re: BUG #15858: could not stat file - over 4GB

2020-02-28 Thread Tom Lane
=?UTF-8?Q?Juan_Jos=C3=A9_Santamar=C3=ADa_Flecha?= writes: > The latest version of this patch could benefit from an update. Please find > attached a new version. The cfbot thinks this doesn't compile on Windows [1]. Looks like perhaps a missing-#include problem?

Re: Binary support for pgoutput plugin

2020-02-28 Thread Tom Lane
Dave Cramer writes: > Rebased against head The cfbot's failing to apply this [1]. It looks like the reason is only that you included a catversion bump in what you submitted. Protocol is to *not* do that in submitted patches, but rely on the committer to add it at the last minute --- otherwise

Re: ALTER tbl rewrite loses CLUSTER ON index (consider moving indisclustered to pg_class)

2020-02-28 Thread Tom Lane
Justin Pryzby writes: > I think the attached is 80% complete (I didn't touch pg_dump). > One objection to this change would be that all relations (including indices) > end up with relclustered fields, and pg_index already has a number of bools, > so > it's not like this one bool is wasting a

Re: Portal->commandTag as an enum

2020-02-28 Thread Alvaro Herrera
On 2020-Feb-28, Tom Lane wrote: > Alvaro Herrera writes: > > I just realized that we could rename command_tag_display_last_oid() to > > something like command_tag_print_a_useless_zero_for_historical_reasons() > > and nothing would be lost. > > Is there a way to drop that logic altogether by

Re: Assert failure due to "drop schema pg_temp_3 cascade" for temporary tables and \d+ is not showing any info after drooping temp table schema

2020-02-28 Thread Michael Paquier
On Fri, Feb 28, 2020 at 01:45:29PM -0500, Tom Lane wrote: > After poking around, I see there aren't any other callers. But I think > that the cause of this bug is clearly failure to think carefully about > the different cases that isTempNamespaceInUse is recognizing, so that > the right way to

Re: d25ea01275 and partitionwise join

2020-02-28 Thread Tom Lane
Amit Langote writes: > On Wed, Nov 6, 2019 at 2:00 AM Tom Lane wrote: >> Just to leave a breadcrumb in this thread --- the planner failure >> induced by d25ea01275 has been fixed in 529ebb20a. The difficulty >> with multiway full joins that Amit started this thread with remains >> open, but I

Re: SLRU statistics

2020-02-28 Thread Alvaro Herrera
On 2020-Jan-21, Tomas Vondra wrote: > On Tue, Jan 21, 2020 at 05:09:33PM +0900, Masahiko Sawada wrote: > > I've not tested the performance impact but perhaps we might want to > > disable these counter by default and controlled by a GUC. And similar > > to buffer statistics it might be better to

Re: Portal->commandTag as an enum

2020-02-28 Thread Tom Lane
Alvaro Herrera writes: > I just realized that we could rename command_tag_display_last_oid() to > something like command_tag_print_a_useless_zero_for_historical_reasons() > and nothing would be lost. Is there a way to drop that logic altogether by making the tagname string be "INSERT 0" for the

Re: Portal->commandTag as an enum

2020-02-28 Thread Alvaro Herrera
I just realized that we could rename command_tag_display_last_oid() to something like command_tag_print_a_useless_zero_for_historical_reasons() and nothing would be lost. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training &

Re: Portal->commandTag as an enum

2020-02-28 Thread Alvaro Herrera
On 2020-Feb-28, Alvaro Herrera wrote: > On 2020-Feb-21, John Naylor wrote: > > > Thinking about this some more, would it be possible to treat these > > like we do parser/kwlist.h? Something like this: > > > > commandtag_list.h: > > PG_COMMANDTAG(ALTER_ACCESS_METHOD, "ALTER ACCESS METHOD", true,

Re: Less-silly selectivity for JSONB matching operators

2020-02-28 Thread Tom Lane
I wrote: > This patch is not complete, because I didn't look at changing > the contrib modules, and grep says at least some of them are using > contsel for non-geometric data types. But I thought I'd put it up > for discussion at this stage. Hearing nothing, I went ahead and hacked on the

Re: HAVE_WORKING_LINK still needed?

2020-02-28 Thread Alvaro Herrera
On 2020-Feb-28, Tom Lane wrote: > Also +1 for s/durable_link_or_rename/durable_link/. Actually, it's not *that* either, because what the function does is link followed by unlink. So it's more a variation of durable_rename with slightly different semantics -- the difference is what happens if a

Re: HAVE_WORKING_LINK still needed?

2020-02-28 Thread Alvaro Herrera
On 2020-Feb-28, Peter Eisentraut wrote: > @@ -788,7 +788,6 @@ durable_link_or_rename(const char *oldfile, const char > *newfile, int elevel) > if (fsync_fname_ext(oldfile, false, false, elevel) != 0) > return -1; > > -#ifdef HAVE_WORKING_LINK > if (link(oldfile,

Re: Improve handling of parameter differences in physical replication

2020-02-28 Thread Alvaro Herrera
On 2020-Feb-27, Peter Eisentraut wrote: > So this patch relaxes this a bit. Upon receipt of > XLOG_PARAMETER_CHANGE, we still check the settings but only issue a > warning and set a global flag if there is a problem. Then when we > actually hit the resource issue and the flag was set, we issue

Re: more ALTER .. DEPENDS ON EXTENSION fixes

2020-02-28 Thread Alvaro Herrera
On 2020-Feb-28, ahsan hadi wrote: > Tested the pg_dump patch for dumping "ALTER .. DEPENDS ON EXTENSION" in case > of indexes, functions, triggers etc. The "ALTER .. DEPENDS ON EXTENSION" is > included in the dump. However in some case not sure why "ALTER > INDEX.DEPENDS ON EXTENSION" is

Re: Making psql error out on output failures

2020-02-28 Thread Alvaro Herrera
On 2020-Feb-18, David Zhang wrote: > 3. I think a better way to resolve this issue will still be the solution > with an extra %m, which can make the error message much more informative to > the end users. Yes, agreed. However, we use a style like this: pg_log_error("could not

Re: Created feature for to_date() conversion using patterns 'YYYY-WW', 'YYYY-WW-D', 'YYYY-MM-W' and 'YYYY-MM-W-D'

2020-02-28 Thread Alvaro Herrera
I just noticed that this patch has been classified under "bug fixes", but per Tom's comments, this is not a bug fix -- it seems we would need a new format code to implement some different week numbering mechanism. That seems a new feature, not a bug fix. Therefore I propose to move this in

Re: Portal->commandTag as an enum

2020-02-28 Thread Alvaro Herrera
On 2020-Feb-21, John Naylor wrote: > Thinking about this some more, would it be possible to treat these > like we do parser/kwlist.h? Something like this: > > commandtag_list.h: > PG_COMMANDTAG(ALTER_ACCESS_METHOD, "ALTER ACCESS METHOD", true, false, > false, false) > ... I liked this idea, so

Re: Libpq support to connect to standby server as priority

2020-02-28 Thread Alvaro Herrera
MauMau, Greg, is any of you submitting a new patch for this? -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [PATCH] Comments related to "Take fewer snapshots" and "Revert patch for taking fewer snapshots"

2020-02-28 Thread Alvaro Herrera
On 2020-Feb-10, Michail Nikolaev wrote: > I think it is good idea to add few comments to code related to the > topic in order to safe time for a next guy. Applied, thanks. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training &

Re: Add LogicalTapeSetExtend() to logtape.c

2020-02-28 Thread Jeff Davis
On Fri, 2020-02-28 at 14:16 +0800, Adam Lee wrote: > I noticed another difference, I was using palloc0(), which could be > one of the > reason, but not sure. I changed the palloc0()'s in your code to plain palloc(), and it didn't make any perceptible difference. Still slower than the version I

Re: Allowing ALTER TYPE to change storage strategy

2020-02-28 Thread Tom Lane
Tomas Vondra writes: > My understanding is that pg_type.typstorage essentially specifies two > things: (a) default storage strategy for the attributes with this type, > and (b) whether the type implementation is prepared to handle TOAST-ed > values or not. And pg_attribute.attstorage has to

Re: Assert failure due to "drop schema pg_temp_3 cascade" for temporary tables and \d+ is not showing any info after drooping temp table schema

2020-02-28 Thread Tom Lane
I wrote: > Also, I notice that isTempNamespaceInUse is already detecting the case > where the namespace doesn't exist or isn't really a temp namespace. > I wonder whether it'd be better to teach that to return an indicator about > the namespace not being what you think it is. That would force us

Re: Assert failure due to "drop schema pg_temp_3 cascade" for temporary tables and \d+ is not showing any info after drooping temp table schema

2020-02-28 Thread Tom Lane
Michael Paquier writes: > And back on that one, I still like better the solution as of the > attached which skips any relations with their namespace gone missing > as 246a6c87's intention was only to allow orphaned temp relations to > be dropped by autovacuum when a backend slot is connected, but

Re: HAVE_WORKING_LINK still needed?

2020-02-28 Thread Tom Lane
Peter Eisentraut writes: > I came across the HAVE_WORKING_LINK define in pg_config_manual.h. > AFAICT, hard links are supported on Windows and Cygwin in the OS > versions that we support, and pg_upgrade already contains the required > shim. It seems to me we could normalize and simplify that,

Re: HAVE_WORKING_LINK still needed?

2020-02-28 Thread Juan José Santamaría Flecha
On Fri, Feb 28, 2020 at 2:15 PM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > I came across the HAVE_WORKING_LINK define in pg_config_manual.h. > AFAICT, hard links are supported on Windows and Cygwin in the OS > versions that we support, and pg_upgrade already contains the

Re: Minor issues in .pgpass

2020-02-28 Thread Hamid Akhtar
The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: not tested Spec compliant: not tested Documentation:not tested First of all, this seems like fixing a valid issue, albeit, the probability

Re: Trying to pull up EXPR SubLinks

2020-02-28 Thread Tom Lane
Richard Guo writes: > On Fri, Feb 28, 2020 at 3:02 PM Andy Fan wrote: >> Glad to see this. I think the hard part is this transform is not *always* >> good. for example foo.a only has 1 rows, but bar has a lot of rows, if >> so the original would be the better one. > Yes exactly. TBH I'm not

Re: proposal: schema variables

2020-02-28 Thread Pavel Stehule
CONSTANT); > After some more thinking and because in other patch I support syntax CREATE TRANSACTION VARIABLE ... I change my opinion and implemented support for syntax CREATE IMMUTABLE VARIABLE for define constants. See attached patch Regards Pavel > > ? > > Regards > > Pavel > > > schema-variables-20200228.patch.gz Description: application/gzip

Re: Use compiler intrinsics for bit ops in hash

2020-02-28 Thread David Fetter
On Thu, Feb 27, 2020 at 02:41:49PM +0800, John Naylor wrote: > On Thu, Feb 27, 2020 at 1:56 PM David Fetter wrote: > > [v6 set] > > Hi David, > > In 0002, the pg_bitutils functions have a test (input > 0), and the > new callers ceil_log2_* and next_power_of_2_* have asserts. That seems >

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-02-28 Thread legrand legrand
Hi Julien, >> But I would have prefered this new feature to work the same way with or >> without track_planning activated ;o( > Definitely, but fixing the issue in pgss (ignoring planner calls when > we don't have a query text) means that pgss won't give an exhaustive > view of activity anymore,

Re: Resume vacuum and autovacuum from interruption and cancellation

2020-02-28 Thread Masahiko Sawada
On Tue, 5 Nov 2019 at 15:57, Masahiko Sawada wrote: > > On Sat, 2 Nov 2019 at 02:10, Robert Haas wrote: > > > > On Thu, Aug 8, 2019 at 9:42 AM Rafia Sabih > > wrote: > > > Sounds like an interesting idea, but does it really help? Because if > > > vacuum was interrupted previously, wouldn't it

HAVE_WORKING_LINK still needed?

2020-02-28 Thread Peter Eisentraut
I came across the HAVE_WORKING_LINK define in pg_config_manual.h. AFAICT, hard links are supported on Windows and Cygwin in the OS versions that we support, and pg_upgrade already contains the required shim. It seems to me we could normalize and simplify that, as in the attached patches.

Re: Add PostgreSQL home page to --help output

2020-02-28 Thread Peter Eisentraut
On 2020-02-20 12:09, Daniel Gustafsson wrote: On 20 Feb 2020, at 10:53, Daniel Gustafsson wrote: On 20 Feb 2020, at 10:15, Peter Eisentraut wrote: On 2020-02-13 14:24, Greg Stark wrote: Sounds like a fine idea. But personally I would prefer it without the <> around the it, just a url on

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2020-02-28 Thread Alexey Kondratov
On 2020-02-28 09:43, Michael Paquier wrote: On Thu, Feb 27, 2020 at 06:29:34PM +0300, Alexey Kondratov wrote: On 2020-02-27 16:41, Alexey Kondratov wrote: > > New version of the patch is attached. Thanks again for your review. > Last patch (v18) got a conflict with one of today commits

Re: Trying to pull up EXPR SubLinks

2020-02-28 Thread Richard Guo
On Fri, Feb 28, 2020 at 3:02 PM Andy Fan wrote: > > > On Fri, Feb 28, 2020 at 2:35 PM Richard Guo > wrote: > >> Hi All, >> >> Currently we will not consider EXPR_SUBLINK when pulling up sublinks and >> this would cause performance issues for some queries with the form of: >> 'a > (SELECT agg(b)

Re: Implementing Incremental View Maintenance

2020-02-28 Thread legrand legrand
>> thank you for patch v14, that fix problems inherited from temporary tables. >> it seems that this ASSERT problem with pgss patch is still present ;o( >> > > Sorry but we are busy on fixing and improving IVM patches. I think fixing > the assertion failure needs non trivial changes to other part

Re: BUG #15858: could not stat file - over 4GB

2020-02-28 Thread Juan José Santamaría Flecha
On Wed, Feb 5, 2020 at 12:47 PM Emil Iggland wrote: > The following review has been posted through the commitfest application: > make installcheck-world: not tested > Implements feature: tested, passed > Spec compliant: not tested > Documentation:not tested > The

Re: base backup client as auxiliary backend process

2020-02-28 Thread Peter Eisentraut
I have set this patch to "returned with feedback" in the upcoming commit fest, because I will not be able to finish it. Unsurprisingly, the sequencing of startup actions in postmaster.c is extremely tricky and needs more thinking. All the rest worked pretty well, I thought. -- Peter

Re: truncating timestamps on arbitrary intervals

2020-02-28 Thread John Naylor
On Wed, Feb 26, 2020 at 11:36 PM Tom Lane wrote: > > John Naylor writes: > > [ v3-datetrunc_interval.patch ] > > A few thoughts: > > * In general, binning involves both an origin and a stride. When > working with plain numbers it's almost always OK to set the origin > to zero, but it's less

Re: Make mesage at end-of-recovery less scary.

2020-02-28 Thread Kyotaro Horiguchi
Thank you for the comments. At Fri, 28 Feb 2020 16:33:18 +0900, Michael Paquier wrote in > On Fri, Feb 28, 2020 at 04:01:00PM +0900, Kyotaro Horiguchi wrote: > > Hello, this is a followup thread of [1]. > > > > # I didn't noticed that the thread didn't cover -hackers.. > > > > When recovery

Re: Ought to use heap_multi_insert() for pg_attribute/depend insertions?

2020-02-28 Thread Michael Paquier
On Tue, Feb 25, 2020 at 10:44:40PM +0100, Daniel Gustafsson wrote: > In doing that I realized that there is another hunk in this patch for fixing > up > logical decoding multi-insert support, which is independent of the patch in > question here so I split it off. It's another issue which cause

RE: [Proposal] Add accumulated statistics for wait event

2020-02-28 Thread imai.yoshik...@fujitsu.com
On Wed, Feb 26, 2020 at 1:39 AM, Kyotaro Horiguchi wrote: > Hello. I had a brief look on this and have some comments on this. Hi, Horiguchi-san. Thank you for looking at this! > It uses its own hash implement. Aside from the appropriateness of > having another implement of existing tool, in

Some improvements to numeric sqrt() and ln()

2020-02-28 Thread Dean Rasheed
Attached is a WIP patch to improve the performance of numeric sqrt() and ln(), which also makes a couple of related improvements to div_var_fast(), all of which have knock-on benefits for other numeric functions. The actual impact varies greatly depending on the inputs, but the overall effect is

Asynchronous Append on postgres_fdw nodes.

2020-02-28 Thread Kyotaro Horiguchi
Hello, this is a follow-on of [1] and [2]. Currently the executor visits execution nodes one-by-one. Considering sharding, Append on multiple postgres_fdw nodes can work simultaneously and that can largely shorten the respons of the whole query. For example, aggregations that can be pushed-down

Re: ALTER INDEX fails on partitioned index

2020-02-28 Thread Michael Paquier
On Thu, Feb 27, 2020 at 05:25:13PM -0600, Justin Pryzby wrote: > /* > - * Option parser for partitioned tables > - */ > -bytea * > -partitioned_table_reloptions(Datum reloptions, bool validate) > -{ > - /* > - * There are no options for partitioned tables yet, but this is able to > do >

Re: Improve handling of parameter differences in physical replication

2020-02-28 Thread Michael Paquier
On Fri, Feb 28, 2020 at 08:49:08AM +0100, Peter Eisentraut wrote: > Perhaps it might be better to track the combined MaxBackends instead, > however. Not sure about that. I think that we should keep them separated, as that's more useful for debugging and more verbose for error reporting. (Worth