Re: intermittent failures in Cygwin from select_parallel tests

2021-06-21 Thread Thomas Munro
On Tue, Jun 22, 2021 at 5:21 PM Noah Misch wrote: > On Thu, Aug 03, 2017 at 10:45:50AM -0400, Robert Haas wrote: > > On Wed, Aug 2, 2017 at 11:47 PM, Noah Misch wrote: > > > postmaster algorithms rely on the PG_SETMASK() calls preventing that. > > > Without > > > such protection, duplicate

Re: [HACKERS] logical decoding of two-phase transactions

2021-06-21 Thread Greg Nancarrow
On Mon, Jun 21, 2021 at 4:37 PM Peter Smith wrote: > > Please find attached the latest patch set v88* > Some minor comments: (1) v88-0002 doc/src/sgml/logicaldecoding.sgml "examples shows" is not correct. I think there is only ONE example being referred to. BEFORE: +The following

Re: Toast compression method options

2021-06-21 Thread Dilip Kumar
On Fri, Jun 18, 2021 at 12:13 PM Michael Paquier wrote: > > On Thu, May 06, 2021 at 07:23:48PM +0530, Dilip Kumar wrote: > > I have fixed some comments offlist reported by Justin. Apart from > > that, I have also improved documentation and test case. Stil it has a > > lot of cleanup to be done

Re: Detecting File Damage & Inconsistencies

2021-06-21 Thread Craig Ringer
On Tue, 22 Jun 2021 at 00:24, Simon Riggs wrote: > On Thu, Mar 18, 2021 at 6:20 AM Craig Ringer > wrote: > > > > On Mon, 15 Mar 2021 at 21:01, David Steele wrote: > >> > >> On 11/18/20 5:23 AM, Simon Riggs wrote: > >> > On Wed, 18 Nov 2020 at 06:42, Craig Ringer > >> > wrote: > >> >> > >> >>

Re: Doc chapter for Hash Indexes

2021-06-21 Thread Amit Kapila
On Tue, Jun 22, 2021 at 4:25 AM Justin Pryzby wrote: > > On Mon, Jun 21, 2021 at 02:08:12PM +0100, Simon Riggs wrote: > > New chapter for Hash Indexes, designed to help users understand how > > they work and when to use them. > > > > Mostly newly written, but a few paras lifted from README when

subscription/t/010_truncate.pl failure on desmoxytes in REL_13_STABLE

2021-06-21 Thread Thomas Munro
Hi, While scanning for assertion failures on the build farm that don't appear to have been discussed, I found this[1] in 010_truncate_publisher.log on the 13 branch: TRAP: FailedAssertion("tupdesc->tdrefcount <= 0", File:

Re: fdatasync performance problem with large number of DB files

2021-06-21 Thread Thomas Munro
On Fri, Jun 18, 2021 at 1:11 PM Justin Pryzby wrote: > Thomas, could you comment on this ? Sorry, I missed that. It is initially a confusing proposal, but after trying it out (that is: making recovery_init_sync_method PGC_SIGHUP and testing a scenario where you want to make the next crash use

Re: fdatasync performance problem with large number of DB files

2021-06-21 Thread Michael Paquier
On Fri, Jun 18, 2021 at 06:18:59PM +0900, Fujii Masao wrote: > On 2021/06/04 23:39, Justin Pryzby wrote: >> You said switching to SIGHUP "would have zero effect"; but, actually it >> allows >> an admin who's DB took a long time in recovery/startup to change the >> parameter >> without shutting

Re: Added schema level support for publication.

2021-06-21 Thread Bharath Rupireddy
On Tue, Jun 22, 2021 at 9:45 AM vignesh C wrote: > I have removed the skip table patches to keep the focus on the main > patch, once this patch gets into committable shape, I will focus on > the skip table patch. IMO it's a good idea to start a new thread for the "skip table" feature so that we

Re: SSL/TLS instead of SSL in docs

2021-06-21 Thread Michael Paquier
On Mon, Jun 21, 2021 at 01:23:56PM +0200, Daniel Gustafsson wrote: > On 18 Jun 2021, at 07:37, Michael Paquier wrote: >> It looks inconsistent to me to point to the libpq documentation to get >> the details about SNI. Wouldn't is be better to have an item in the >> glossary that refers to the

Re: Different compression methods for FPI

2021-06-21 Thread Michael Paquier
On Mon, Jun 21, 2021 at 10:13:58PM -0500, Justin Pryzby wrote: > @Michael: I assume that if you merge this patch, you'd set your animals to use > wal_compression=lz4, and then they would fail the recovery tests. Yes, I'd like to do that on my animal dangomushi. > So the patches that you say are

RE: Fix for segfault in logical replication on master

2021-06-21 Thread osumi.takami...@fujitsu.com
On Tuesday, June 22, 2021 11:08 AM Japin Li wrote: > On Mon, 21 Jun 2021 at 19:06, Amit Kapila wrote: > > On Mon, Jun 21, 2021 at 4:09 PM Japin Li wrote: > >> > >> On Mon, 21 Jun 2021 at 17:54, Amit Kapila > wrote: > >> > On Mon, Jun 21, 2021 at 2:06 PM Japin Li wrote: > >> >> > >> >> On Mon,

Re: Different compression methods for FPI

2021-06-21 Thread Justin Pryzby
On Tue, May 25, 2021 at 12:05:19PM +0530, Dilip Kumar wrote: > +++ b/src/test/recovery/t/011_crash_recovery.pl > @@ -14,7 +14,7 @@ use Config; > plan tests => 3; > > my $node = get_new_node('primary'); > -$node->init(allows_streaming => 1); > +$node->init(); > $node->start; > > How this

Re: Two patches to speed up pg_rewind.

2021-06-21 Thread Paul Guo
On Thu, Jun 17, 2021 at 3:19 PM Michael Paquier wrote: > > On Wed, Jun 02, 2021 at 05:02:10PM +0900, Michael Paquier wrote: > > On Wed, Jun 02, 2021 at 06:20:30PM +1200, Thomas Munro wrote: > > > The main thing I noticed was that Linux < 5.3 can fail with EXDEV if > > > you cross a filesystem

Re: Maintaining a list of pgindent commits for "git blame" to ignore

2021-06-21 Thread Bruce Momjian
On Mon, Jun 21, 2021 at 07:43:59PM -0700, Peter Geoghegan wrote: > On Mon, Jun 21, 2021 at 5:06 PM Tom Lane wrote: > > Agreed. I think I'd previously suggested something under src/tools, > > but we might as well do like others are doing; especially since > > we have .gitattributes and the like

Re: Optionally automatically disable logical replication subscriptions on error

2021-06-21 Thread Mark Dilger
> On Jun 21, 2021, at 5:57 PM, Peter Smith wrote: > > * Is the goal to prevent some *unattended* SUBSCRIPTION from going bad > at some point in future and then going into a relaunch loop for > days/weeks and causing 1000's of errors without the user noticing. In > that case, this patch seems

Re: Use simplehash.h instead of dynahash in SMgr

2021-06-21 Thread Thomas Munro
On Tue, Jun 22, 2021 at 1:55 PM David Rowley wrote: > On Tue, 22 Jun 2021 at 03:43, Tom Lane wrote: > > I kind of wonder if we really need four different hash table > > implementations (this being the third "generic" one, plus hash join > > has its own, and I may have forgotten others). Should

Re: Optionally automatically disable logical replication subscriptions on error

2021-06-21 Thread Amit Kapila
On Tue, Jun 22, 2021 at 6:27 AM Peter Smith wrote: > > #3. There is another suggestion to use the Stats Collector to hold the > error message [Amit-2]. For me, this felt like blurring too much the > distinction between "stats tracking/metrics" and "logs". ERROR logs > must be flushed, whereas for

Re: Maintaining a list of pgindent commits for "git blame" to ignore

2021-06-21 Thread Peter Geoghegan
On Mon, Jun 21, 2021 at 5:06 PM Tom Lane wrote: > Agreed. I think I'd previously suggested something under src/tools, > but we might as well do like others are doing; especially since > we have .gitattributes and the like there already. Great. Attached is a patch file that puts it all

Re: Optionally automatically disable logical replication subscriptions on error

2021-06-21 Thread Masahiko Sawada
On Mon, Jun 21, 2021 at 7:48 PM Amit Kapila wrote: > > On Mon, Jun 21, 2021 at 11:19 AM Amit Kapila wrote: > > > > On Mon, Jun 21, 2021 at 10:55 AM Mark Dilger > > wrote: > > > > > I don't mind if you want to store more information, and maybe that needs > > > to be stored somewhere else. Do

Re: row filtering for logical replication

2021-06-21 Thread Peter Smith
On Fri, Jun 18, 2021 at 9:40 PM Amit Kapila wrote: > [...] > I have rebased the patch so that you can try it out. The main thing I > have done is to remove changes in worker.c and created a specialized > function to create estate for pgoutput.c as I don't think we need what > is done in worker.c.

Re: Optionally automatically disable logical replication subscriptions on error

2021-06-21 Thread Mark Dilger
> On Jun 21, 2021, at 5:57 PM, Peter Smith wrote: > > * Is the goal mainly to help automated (TAP) testing? Absolutely, that was my original motivation. But I don't think that is the primary reason the patch would be accepted. There is a cost to having the logical replication workers

Re: RFC: Logging plan of the running query

2021-06-21 Thread torikoshia
On 2021-06-16 20:36, torikoshia wrote: other background or parallel worker. As far as I looked around, there seems no easy ways to do so. If we were to invent a new mechanism just for addressing the above comment, I would rather choose to not do that as it seems like an overkill. We can

Re: Optionally automatically disable logical replication subscriptions on error

2021-06-21 Thread Mark Dilger
> On Jun 21, 2021, at 5:57 PM, Peter Smith wrote: > > #5. Document to refer to the logs. All ERROR details are already in > the logs, and this seems to me the intuitive place to look for them. My original motivation came from writing TAP tests to check that the permissions systems would

Re: Fix for segfault in logical replication on master

2021-06-21 Thread Japin Li
On Mon, 21 Jun 2021 at 19:06, Amit Kapila wrote: > On Mon, Jun 21, 2021 at 4:09 PM Japin Li wrote: >> >> On Mon, 21 Jun 2021 at 17:54, Amit Kapila wrote: >> > On Mon, Jun 21, 2021 at 2:06 PM Japin Li wrote: >> >> >> >> On Mon, 21 Jun 2021 at 16:22, Amit Kapila wrote: >> >> > On Mon, Jun 21,

Re: Use simplehash.h instead of dynahash in SMgr

2021-06-21 Thread David Rowley
On Tue, 22 Jun 2021 at 03:43, Tom Lane wrote: > I kind of wonder if we really need four different hash table > implementations (this being the third "generic" one, plus hash join > has its own, and I may have forgotten others). Should we instead > think about revising simplehash to gain the

Re: Use simplehash.h instead of dynahash in SMgr

2021-06-21 Thread David Rowley
On Tue, 22 Jun 2021 at 02:53, Robert Haas wrote: > At the risk of kibitzing the least-important detail of this proposal, > I'm not very happy with the names of our hash implementations. > simplehash is not especially simple, and dynahash is not particularly > dynamic, especially now that the main

Re: Added schema level support for publication.

2021-06-21 Thread Ajin Cherian
On Mon, Jun 21, 2021 at 3:16 PM vignesh C wrote: > I felt this is ok as we specify the keycount to be 1, so only the > key[0] will be used. Thoughts? > ScanKeyInit([0], > Anum_pg_class_relkind, > BTEqualStrategyNumber, F_CHAREQ, > CharGetDatum(RELKIND_PARTITIONED_TABLE)); > > scan =

Re: Different compression methods for FPI

2021-06-21 Thread Michael Paquier
On Mon, Jun 21, 2021 at 07:19:27PM -0500, Justin Pryzby wrote: > The two similar, existing messages are: > > +#define NO_LZ4_SUPPORT() \ > + ereport(ERROR, \ > + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), \ > +errmsg("unsupported LZ4 compression

Re: Optionally automatically disable logical replication subscriptions on error

2021-06-21 Thread Peter Smith
Much of the discussion above seems to be related to where to store the error information and how much information is needed to be useful. As a summary, the 5 alternatives I have seen mentioned are: #1. Store some simple message in the pg_subscription ("I wasn't trying to capture everything, just

Re: disfavoring unparameterized nested loops

2021-06-21 Thread Peter Geoghegan
On Mon, Jun 21, 2021 at 4:51 PM Bruce Momjian wrote: > There were a lot of interesting ideas in this thread and I want to > analyze some of them. First, there is the common assumption (not > stated) that over-estimating by 5% and underestimating by 5% cause the > same harm, which is clearly

Re: pgbench logging broken by time logic changes

2021-06-21 Thread Michael Paquier
On Sun, Jun 20, 2021 at 10:15:55AM -0400, Andrew Dunstan wrote: > If this were core server code threatening data integrity I would be > inclined to be more strict, but after all pg_bench is a utility program, > and I think we can allow a little more latitude. +1. Let's be flexible here. It

Re: PXGS vs TAP tests

2021-06-21 Thread Michael Paquier
On Sun, Jun 20, 2021 at 01:24:04PM -0400, Andrew Dunstan wrote: > Tests pass with the attached patch, which puts the setting in the > Makefile for the recovery tests. The script itself doesn't need any > changing. +REGRESS_SHLIB=$(abs_top_builddir)/src/test/regress/regress$(DLSUFFIX) +export

Re: Different compression methods for FPI

2021-06-21 Thread Justin Pryzby
On Tue, Jun 22, 2021 at 09:11:26AM +0900, Michael Paquier wrote: > On Sun, Jun 20, 2021 at 11:15:08PM +0500, Andrey Borodin wrote: > > I have some small questions. > > > > 1. > > + report_invalid_record(record, "image at %X/%X > > compressed with %s not supported, block %d", >

Re: Different compression methods for FPI

2021-06-21 Thread Michael Paquier
On Sun, Jun 20, 2021 at 11:15:08PM +0500, Andrey Borodin wrote: > I have some small questions. > > 1. > + report_invalid_record(record, "image at %X/%X > compressed with %s not supported, block %d", > + (uint32) >

Re: Maintaining a list of pgindent commits for "git blame" to ignore

2021-06-21 Thread Tom Lane
Peter Geoghegan writes: > The convention seems to be that it is located in the top-level > directory. ISTM that we should follow that convention, since following > the convention is good, and does not in itself force anybody to ignore > any of the listed commits. Any thoughts on that? Agreed. I

Re: disfavoring unparameterized nested loops

2021-06-21 Thread Bruce Momjian
On Mon, Jun 21, 2021 at 12:52:39PM -0400, Tom Lane wrote: > You're throwing around a lot of pejorative adjectives there without > having bothered to quantify any of them. This sounds less like a sound > argument to me than like a witch trial. > > Reflecting for a bit on the ancient principle that

Re: Maintaining a list of pgindent commits for "git blame" to ignore

2021-06-21 Thread Peter Geoghegan
On Mon, Jun 21, 2021 at 8:34 AM Tom Lane wrote: > It's possible that some of these touch few enough lines that they > are not worth listing; I did not check the commit delta sizes. Commits that touch very few lines weren't included originally, just because it didn't seem necessary. Even still,

Re: Doc chapter for Hash Indexes

2021-06-21 Thread Justin Pryzby
On Mon, Jun 21, 2021 at 02:08:12PM +0100, Simon Riggs wrote: > New chapter for Hash Indexes, designed to help users understand how > they work and when to use them. > > Mostly newly written, but a few paras lifted from README when they were > helpful. + + PostgreSQL includes an implementation

Re: disfavoring unparameterized nested loops

2021-06-21 Thread Peter Geoghegan
On Mon, Jun 21, 2021 at 1:52 PM Tom Lane wrote: > You're ignoring the fact that the plan shape we generate now is in fact > *optimal*, and easily proven to be so, in some very common cases. As I've said I don't reject the idea that there is room for disagreement on the specifics. For example

Re: disfavoring unparameterized nested loops

2021-06-21 Thread Tom Lane
Peter Geoghegan writes: > On Mon, Jun 21, 2021 at 10:14 AM Robert Haas wrote: >> The other thing is - the risk of a particular path doesn't matter in >> an absolute sense, only a relative one. In the particular case I'm on >> about here, we *know* there's a less-risky alternative. > Exactly!

Re: disfavoring unparameterized nested loops

2021-06-21 Thread Peter Geoghegan
On Mon, Jun 21, 2021 at 10:14 AM Robert Haas wrote: > Hmm, maybe I need to see an example of the sort of plan shape that you > have in mind. To me it feels like a comparison on a unique key ought > to use a *parameterized* nested loop. And it's also not clear to me > why a nested loop is actually

Re: disfavoring unparameterized nested loops

2021-06-21 Thread Tom Lane
Robert Haas writes: > On Mon, Jun 21, 2021 at 1:38 PM Tom Lane wrote: >> NestLoop Join >> Join Filter: t1.x op t2.y >> -> Index Scan on t1_pkey >>Index Cond: t1.id = constant >> -> Seq Scan on t2 > Hmm, yeah, I guess that's possible. How much do you think this loses > as compared

Re: disfavoring unparameterized nested loops

2021-06-21 Thread Robert Haas
On Mon, Jun 21, 2021 at 1:38 PM Tom Lane wrote: > > Hmm, maybe I need to see an example of the sort of plan shape that you > > have in mind. To me it feels like a comparison on a unique key ought > > to use a *parameterized* nested loop. > > The unique-key comparison would be involved in the

Re: disfavoring unparameterized nested loops

2021-06-21 Thread Tom Lane
I wrote: > ... As an example, > select * from t1, t2 where t1.id = constant and t1.x op t2.y; > where I'm not assuming much about the properties of "op". > This could be amenable to a plan like > NestLoop Join > Join Filter: t1.x op t2.y > -> Index Scan on t1_pkey >

Re: disfavoring unparameterized nested loops

2021-06-21 Thread Tom Lane
Robert Haas writes: > On Mon, Jun 21, 2021 at 11:55 AM Tom Lane wrote: >> There are certainly cases where the optimizer can prove (in principle; >> it doesn't do so today) that a plan node will produce at most one row. >> They're hardly uncommon either: an equality comparison on a unique >> key,

Re: disfavoring unparameterized nested loops

2021-06-21 Thread Robert Haas
On Mon, Jun 21, 2021 at 1:11 PM Peter Geoghegan wrote: > On Mon, Jun 21, 2021 at 9:52 AM Tom Lane wrote: > > > I'm not so sure that it is. The point isn't the risk, even if it could > > > be calculated. The point is the downsides of being wrong are huge and > > > pretty much unbounded, whereas

Re: Out-of-bounds (src/backend/utils/misc/queryjumble.c)

2021-06-21 Thread Tom Lane
Ranier Vilela writes: > Per Coverity. > 3 out-of-bounds at function AppendJumble. > They have the face, smell and color of typo. > And we usually increment the character count after a memcpy. > Coverity no longer complained after the patch. > Thoughts? This patch is incorrect on its face, as

Re: disfavoring unparameterized nested loops

2021-06-21 Thread Robert Haas
On Mon, Jun 21, 2021 at 11:55 AM Tom Lane wrote: > There are certainly cases where the optimizer can prove (in principle; > it doesn't do so today) that a plan node will produce at most one row. > They're hardly uncommon either: an equality comparison on a unique > key, or a subquery with a

Re: disfavoring unparameterized nested loops

2021-06-21 Thread Peter Geoghegan
On Mon, Jun 21, 2021 at 9:52 AM Tom Lane wrote: > > I'm not so sure that it is. The point isn't the risk, even if it could > > be calculated. The point is the downsides of being wrong are huge and > > pretty much unbounded, whereas the benefits of being right are tiny > > and bounded. It almost

Out-of-bounds (src/backend/utils/misc/queryjumble.c)

2021-06-21 Thread Ranier Vilela
Hi, Per Coverity. 3 out-of-bounds at function AppendJumble. They have the face, smell and color of typo. And we usually increment the character count after a memcpy. Coverity no longer complained after the patch. Thoughts? regards, Ranier Vilela fix_out_of_bounds_queryjumble.patch

Re: disfavoring unparameterized nested loops

2021-06-21 Thread Tom Lane
Tomas Vondra writes: > I've been thinking about this a bit recently and searching for papers > talking about this, and but it's not clear to me how to calculate the > risk (and propagate it through the plan) without making the whole cost > evaluation way more complicated / expensive :-( Yeah,

Re: disfavoring unparameterized nested loops

2021-06-21 Thread Tom Lane
Peter Geoghegan writes: > On Mon, Jun 21, 2021 at 8:55 AM Tom Lane wrote: >> I'd be a lot happier if this proposal were couched around some sort >> of estimate of the risk of the outer side producing more than the >> expected number of rows. The arguments so far seem like fairly lame >>

Re: Discarding DISCARD ALL

2021-06-21 Thread Simon Riggs
On Wed, Jan 20, 2021 at 3:53 PM James Coleman wrote: > > On Wed, Jan 20, 2021 at 9:58 AM Simon Riggs wrote: > > > > On Wed, 20 Jan 2021 at 14:21, James Coleman wrote: > > > > > An alternative approach that occurred to me while typing this reply: a > > > setting in Postgres that would disallow

Re: Add version macro to libpq-fe.h

2021-06-21 Thread Andrew Dunstan
On 6/21/21 12:34 PM, Tom Lane wrote: > Daniel Gustafsson writes: >> On 21 Jun 2021, at 17:27, Robert Haas wrote: >>> What will prevent us from forgetting to do something about this again, >>> a year from now? >> An entry in a release checklist could perhaps be an idea? > Yeah, I was wondering

Re: disfavoring unparameterized nested loops

2021-06-21 Thread Tomas Vondra
On 6/21/21 5:55 PM, Tom Lane wrote: Peter Geoghegan writes: The heuristic that has the optimizer flat out avoids an unparameterized nested loop join is justified by the belief that that's fundamentally reckless. Even though we all agree on that much, I don't know when it stops being

Re: Add version macro to libpq-fe.h

2021-06-21 Thread Tom Lane
Daniel Gustafsson writes: > On 21 Jun 2021, at 17:27, Robert Haas wrote: >> What will prevent us from forgetting to do something about this again, >> a year from now? > An entry in a release checklist could perhaps be an idea? Yeah, I was wondering if adding an entry to RELEASE_CHANGES would

Re: disfavoring unparameterized nested loops

2021-06-21 Thread Peter Geoghegan
On Mon, Jun 21, 2021 at 8:55 AM Tom Lane wrote: > There are certainly cases where the optimizer can prove (in principle; > it doesn't do so today) that a plan node will produce at most one row. > They're hardly uncommon either: an equality comparison on a unique > key, or a subquery with a simple

Re: Detecting File Damage & Inconsistencies

2021-06-21 Thread Simon Riggs
On Thu, Mar 18, 2021 at 6:20 AM Craig Ringer wrote: > > On Mon, 15 Mar 2021 at 21:01, David Steele wrote: >> >> On 11/18/20 5:23 AM, Simon Riggs wrote: >> > On Wed, 18 Nov 2020 at 06:42, Craig Ringer >> > wrote: >> >> >> >> On Fri, Nov 13, 2020 at 7:24 PM Simon Riggs wrote: >> >>> >> >>> >>

Re: Add version macro to libpq-fe.h

2021-06-21 Thread Daniel Gustafsson
> On 21 Jun 2021, at 17:27, Robert Haas wrote: > > On Sat, Jun 19, 2021 at 11:45 AM Tom Lane wrote: >> Hearing no further comments, done that way. > > What will prevent us from forgetting to do something about this again, > a year from now? An entry in a release checklist could perhaps be an

Re: disfavoring unparameterized nested loops

2021-06-21 Thread Tom Lane
Peter Geoghegan writes: > The heuristic that has the optimizer flat out avoids an > unparameterized nested loop join is justified by the belief that > that's fundamentally reckless. Even though we all agree on that much, > I don't know when it stops being reckless and starts being "too risky >

Re: Use simplehash.h instead of dynahash in SMgr

2021-06-21 Thread Tom Lane
Robert Haas writes: > On Mon, Jun 21, 2021 at 10:15 AM David Rowley wrote: >> I've come up with a new hash table implementation that I've called >> generichash. > At the risk of kibitzing the least-important detail of this proposal, > I'm not very happy with the names of our hash

Re: Add version macro to libpq-fe.h

2021-06-21 Thread Tom Lane
Robert Haas writes: > What will prevent us from forgetting to do something about this again, > a year from now? As long as we notice it before 15.0, we can fix it retroactively, as we just did for 14. For that matter, fixing before 15.1 or so would likely be Good Enough. But realistically, how

Re: disfavoring unparameterized nested loops

2021-06-21 Thread Peter Geoghegan
On Mon, Jun 21, 2021 at 7:45 AM Robert Haas wrote: > On Mon, Jun 21, 2021 at 6:41 AM David Rowley wrote: > > For example, in an unparameterized Nested Loop that estimates the > > outer Path to have 1 row will cost an entire additional inner cost if > > there are 2 rows. With Hash Join the cost

Re: Maintaining a list of pgindent commits for "git blame" to ignore

2021-06-21 Thread Tom Lane
Peter Geoghegan writes: > What do you think of the attached? I prefer the ISO date style myself, > so I went with that. I grepped the git logs for "indent" and found a bunch more commits that look like they should be included: db6e2b4c5 d84213909 1e9c85809 f04d4ac91 9ef2dbefc 651902deb

Re: Add version macro to libpq-fe.h

2021-06-21 Thread Robert Haas
On Sat, Jun 19, 2021 at 11:45 AM Tom Lane wrote: > Hearing no further comments, done that way. What will prevent us from forgetting to do something about this again, a year from now? -- Robert Haas EDB: http://www.enterprisedb.com

Re: [bug?] Missed parallel safety checks, and wrong parallel safety

2021-06-21 Thread Robert Haas
On Mon, Jun 21, 2021 at 12:56 AM Amit Kapila wrote: > I thought if we scan a system catalog using DirtySnapshot, then we > should be able to find such a relation. But, if the system catalog is > updated after our scan then surely we won't be able to see it and in > that case, we won't be able to

Re: Use simplehash.h instead of dynahash in SMgr

2021-06-21 Thread Robert Haas
On Mon, Jun 21, 2021 at 10:15 AM David Rowley wrote: > I've come up with a new hash table implementation that I've called > generichash. At the risk of kibitzing the least-important detail of this proposal, I'm not very happy with the names of our hash implementations. simplehash is not

Re: disfavoring unparameterized nested loops

2021-06-21 Thread Robert Haas
On Mon, Jun 21, 2021 at 6:41 AM David Rowley wrote: > For example, in an unparameterized Nested Loop that estimates the > outer Path to have 1 row will cost an entire additional inner cost if > there are 2 rows. With Hash Join the cost is just an additional > hashtable lookup, which is dead

Re: Use extended statistics to estimate (Var op Var) clauses

2021-06-21 Thread Dean Rasheed
On Sun, 13 Jun 2021 at 21:28, Tomas Vondra wrote: > > Here is a slightly updated version of the patch > > The main issue I ran into > is the special case clauselist_selectivity, which does > > if (list_length(clauses) == 1) > return clause_selectivity_ext(...); > > which applies to

Re: PG 14 release notes, first draft

2021-06-21 Thread Bruce Momjian
On Mon, Jun 21, 2021 at 02:47:16PM +0900, Tatsuo Ishii wrote: > > I got the parse error after applying the patch: > > > > release-14.sgml:3562: parser error : Input is not proper UTF-8, > > indicate encoding ! > > Bytes: 0xE9 0x20 0x53 0x61 > > (Juan Jos Santamara Flecha) > >

Re: Use simplehash.h instead of dynahash in SMgr

2021-06-21 Thread David Rowley
I'd been thinking of this patch again. When testing with simplehash, I found that the width of the hash bucket type was fairly critical for getting good performance from simplehash.h. With simplehash.h I didn't manage to narrow this any more than 16 bytes. I needed to store the 32-bit hash value

Re: disfavoring unparameterized nested loops

2021-06-21 Thread Kenneth Marshall
> > > > Most of the time when I see that happen it's down to either the > > selectivity of some correlated base-quals being multiplied down to a > > number low enough that we clamp the estimate to be 1 row. The other > > case is similar, but with join quals. > > If an estimate is lower than 1,

Fix pkg-config file for static linking

2021-06-21 Thread Filip Gospodinov
Since https://github.com/postgres/postgres/commit/ea53100d5 (or Postgres 12.0) the shipped pkg-config file is broken for statically linking libpq because libpgcommon and libpgport are missing. This patch adds those two missing private dependencies. diff --git a/src/interfaces/libpq/Makefile

Re: Character expansion with ICU collations

2021-06-21 Thread Finnerty, Jim
I have a proposal for how to support tailoring rules in ICU collations: The ucol_openRules() function is an alternative to the ucol_open() function that PostgreSQL calls today, but it takes the collation strength as one if its parameters so the locale string would need to be parsed before

Doc chapter for Hash Indexes

2021-06-21 Thread Simon Riggs
New chapter for Hash Indexes, designed to help users understand how they work and when to use them. Mostly newly written, but a few paras lifted from README when they were helpful. -- Simon Riggshttp://www.EnterpriseDB.com/ doc_hash_index.v1.patch Description: Binary data

Using indexUnchanged with nbtree

2021-06-21 Thread Simon Riggs
Seems like we can skip the uniqueness check if indexUnchanged, which will speed up non-HOT UPDATEs on tables with B-Trees. Passes tests. Comments? -- Simon Riggshttp://www.EnterpriseDB.com/ skip_nonHOT_btree_unique_check.v1.patch Description: Binary data

Re: pg14b1 stuck in lazy_scan_prune/heap_page_prune of pg_statistic

2021-06-21 Thread Andres Freund
Hi, On 2021-06-16 12:12:23 -0700, Andres Freund wrote: > Could you share your testcase? I've been working on a series of patches > to address this (I'll share in a bit), and I've run quite a few tests, > and didn't hit any infinite loops. Sorry for not yet doing that. Unfortunately I have an

Re: Add index OID macro argument to DECLARE_INDEX

2021-06-21 Thread John Naylor
On Mon, Jun 21, 2021 at 3:23 AM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > > > This patch changes places like this > > DECLARE_UNIQUE_INDEX_PKEY(pg_aggregate_fnoid_index, 2650, on > pg_aggregate using btree(aggfnoid oid_ops)); > #define AggregateFnoidIndexId 2650 > > to this >

Re: Remove useless int64 range checks on BIGINT sequence MINVALUE/MAXVALUE values

2021-06-21 Thread Greg Nancarrow
On Mon, Jun 21, 2021 at 8:32 PM David Rowley wrote: > > It might be worth putting in a comment to mention that the check is > not needed. Just in case someone looks again one day and thinks the > checks are missing. > > Probably best to put this in the July commitfest so it does not get missed.

Re: disfavoring unparameterized nested loops

2021-06-21 Thread John Naylor
On Tue, Jun 15, 2021 at 8:00 PM David Rowley wrote: > In my experience, the most common reason that the planner chooses > non-parameterized nested loops wrongly is when there's row > underestimation that says there's just going to be 1 row returned by > some set of joins. The problem often

Re: SSL/TLS instead of SSL in docs

2021-06-21 Thread Daniel Gustafsson
> On 18 Jun 2021, at 07:37, Michael Paquier wrote: > > On Tue, Jun 15, 2021 at 03:59:18PM +0200, Daniel Gustafsson wrote: >> While in there I added IMO missing items to the glossary and acronyms >> sections >> as well as fixed up markup around OpenSSL. >> >> This only deals with docs, but if

Is the testing a bit too light on GROUP BY DISTINCT?

2021-06-21 Thread David Rowley
In [1], Yaoguang reported an Assert failure in expand_grouping_sets. Since beta2 deadline is looming, I pushed a quick fix for that. As mentioned over on bugs, only 1 test triggers that code and because the List of IntLists always had an empty list as the first element due to the code just above

Re: Doc patch for Logical Replication Message Formats (PG14)

2021-06-21 Thread Brar Piening
Amit Kapila schrieb: On Mon, Jun 21, 2021 at 4:13 PM Brar Piening wrote: Amit Kapila wrote: After looking at the docs once again I have another minor amendment (new patch attached). +The value of the column, eiter in binary or in text format. Typo. /eiter/either Fixed -

RE: [bug?] Missed parallel safety checks, and wrong parallel safety

2021-06-21 Thread houzj.f...@fujitsu.com
On Wednesday, June 16, 2021 11:27 AM houzj.f...@fujitsu.com wrote: > On Tuesday, June 15, 2021 10:01 PM Robert Haas wrote: > > Just to be clear here, I don't think it really matters what we *want* > > to do. I don't think it's reasonably *possible* to check all the > > partitions, because we

Re: Doc patch for Logical Replication Message Formats (PG14)

2021-06-21 Thread Amit Kapila
On Mon, Jun 21, 2021 at 4:13 PM Brar Piening wrote: > > Amit Kapila wrote: > > > After looking at the docs once again I have another minor amendment (new > patch attached). > +The value of the column, eiter in binary or in text format. Typo. /eiter/either -- With Regards, Amit

Re: Fix for segfault in logical replication on master

2021-06-21 Thread Amit Kapila
On Mon, Jun 21, 2021 at 4:09 PM Japin Li wrote: > > On Mon, 21 Jun 2021 at 17:54, Amit Kapila wrote: > > On Mon, Jun 21, 2021 at 2:06 PM Japin Li wrote: > >> > >> On Mon, 21 Jun 2021 at 16:22, Amit Kapila wrote: > >> > On Mon, Jun 21, 2021 at 1:30 PM Japin Li wrote: > >> >> > >> >> On Sat, 19

Re: Optionally automatically disable logical replication subscriptions on error

2021-06-21 Thread Amit Kapila
On Mon, Jun 21, 2021 at 11:19 AM Amit Kapila wrote: > > On Mon, Jun 21, 2021 at 10:55 AM Mark Dilger > wrote: > > > I don't mind if you want to store more information, and maybe that needs > > to be stored somewhere else. Do you believe pg_subscription_rel is a > > suitable location? > > >

RE: Refactor ECPGconnect and allow IPv6 connection

2021-06-21 Thread kuroda.hay...@fujitsu.com
Dear Michael, Thank you for replying! > it does not strike me as a great idea to have a duplicate > logic doing the parsing of URIs, even if libpq accepts multiple > hosts/ports as an option. Yeah, I agree your argument that duplicated parse function should be removed. ECPG parses connection

Re: Doc patch for Logical Replication Message Formats (PG14)

2021-06-21 Thread Brar Piening
Amit Kapila wrote: On Mon, Jun 21, 2021 at 12:26 PM Brar Piening wrote: Hello Hackers, while amending Npgsql to account for the Logical Streaming Replication Protocol changes in PostgreSQL 14 I stumbled upon two documentation inaccuracies in the Logical Replication Message Formats

Re: disfavoring unparameterized nested loops

2021-06-21 Thread David Rowley
On Wed, 16 Jun 2021 at 15:08, Peter Geoghegan wrote: > It seems important to distinguish between risk and uncertainty -- > they're rather different things. The short version is that you can > model risk but you cannot model uncertainty. This seems like a problem > of uncertainty to me. You might

Re: Fix for segfault in logical replication on master

2021-06-21 Thread Japin Li
On Mon, 21 Jun 2021 at 17:54, Amit Kapila wrote: > On Mon, Jun 21, 2021 at 2:06 PM Japin Li wrote: >> >> On Mon, 21 Jun 2021 at 16:22, Amit Kapila wrote: >> > On Mon, Jun 21, 2021 at 1:30 PM Japin Li wrote: >> >> >> >> On Sat, 19 Jun 2021 at 17:18, Amit Kapila wrote: >> >> > On Fri, Jun 18,

Re: Remove useless int64 range checks on BIGINT sequence MINVALUE/MAXVALUE values

2021-06-21 Thread David Rowley
On Mon, 21 Jun 2021 at 22:10, Greg Nancarrow wrote: > Sequence MINVALUE/MAXVALUE values are read into "int64" variables and > then range-checked according to the sequence data-type. > However, for a BIGINT sequence, checking whether these are < > PG_INT64_MIN or > PG_INT64_MAX always evaluates to

Re: seawasp failing, maybe in glibc allocator

2021-06-21 Thread Andres Freund
Hi, On 2021-06-20 19:56:56 -0400, Tom Lane wrote: > Thomas Munro writes: > > Looking at their release schedule on https://llvm.org/, I see we have > > a gamble to make. They currently plan to cut RC1 at the end of July, > > and to release in late September (every second LLVM major release > >

Remove useless int64 range checks on BIGINT sequence MINVALUE/MAXVALUE values

2021-06-21 Thread Greg Nancarrow
Hi, Sequence MINVALUE/MAXVALUE values are read into "int64" variables and then range-checked according to the sequence data-type. However, for a BIGINT sequence, checking whether these are < PG_INT64_MIN or > PG_INT64_MAX always evaluates to false, as an int64 can't hold such values. I've

Re: Fix for segfault in logical replication on master

2021-06-21 Thread Amit Kapila
On Mon, Jun 21, 2021 at 2:06 PM Japin Li wrote: > > On Mon, 21 Jun 2021 at 16:22, Amit Kapila wrote: > > On Mon, Jun 21, 2021 at 1:30 PM Japin Li wrote: > >> > >> On Sat, 19 Jun 2021 at 17:18, Amit Kapila wrote: > >> > On Fri, Jun 18, 2021 at 9:18 AM Amit Kapila > >> > wrote: > >> > >> Or we

Re: [Patch] Rename PQtraceSetFlags to PQsetTraceFlags for bookindex.html

2021-06-21 Thread Noah Misch
On Mon, Jun 21, 2021 at 02:36:19AM +, zhangj...@fujitsu.com wrote: > Here is a patch. Pushed.

Re: Doc patch for Logical Replication Message Formats (PG14)

2021-06-21 Thread Amit Kapila
On Mon, Jun 21, 2021 at 12:26 PM Brar Piening wrote: > > Hello Hackers, > while amending Npgsql to account for the Logical Streaming Replication > Protocol changes in PostgreSQL 14 I stumbled upon two documentation > inaccuracies in the Logical Replication Message Formats documentation >

Re: Pipeline mode and PQpipelineSync()

2021-06-21 Thread Boris Kolpackov
Alvaro Herrera writes: > I think I should rephrase this to say that PQpipelineSync() is needed > where the user needs the server to start executing commands; and > separately indicate that it is possible (but not promised) that the > server would start executing commands ahead of time because

Re: Fix for segfault in logical replication on master

2021-06-21 Thread Japin Li
On Mon, 21 Jun 2021 at 16:22, Amit Kapila wrote: > On Mon, Jun 21, 2021 at 1:30 PM Japin Li wrote: >> >> On Sat, 19 Jun 2021 at 17:18, Amit Kapila wrote: >> > On Fri, Jun 18, 2021 at 9:18 AM Amit Kapila >> > wrote: >> >> Or we can free the memory owned by indexoidlist after check whether it

  1   2   >