Re: Speed up pg_checksums in cases where checksum already set

On Wed, Jun 02, 2021 at 05:09:36PM -0400, Greg Sabino Mullane wrote: > Newer version attach that adds a small documentation tweak as well. - enabling checksums, every file in the cluster is rewritten in-place. + enabling checksums, every file in the cluster with a changed checksum is +

Re: Unresolved repliaction hang and stop problem.

At Thu, 17 Jun 2021 12:56:42 -0400, Alvaro Herrera wrote in > On 2021-Jun-17, Kyotaro Horiguchi wrote: > > > I don't see a call to hash_*seq*_search there. Instead, I see one in > > ReorderBufferCheckMemoryLimit(). > > Doh, of course -- I misread. > > ReorderBufferCheckMemoryLimit is new in

Re: SSL/TLS instead of SSL in docs

On Tue, Jun 15, 2021 at 03:59:18PM +0200, Daniel Gustafsson wrote: > While in there I added IMO missing items to the glossary and acronyms sections > as well as fixed up markup around OpenSSL. > > This only deals with docs, but if this is deemed interesting then userfacing > messages in the code

Re: [HACKERS] logical decoding of two-phase transactions

On Fri, Jun 18, 2021 at 7:43 AM Peter Smith wrote: > > On Thu, Jun 17, 2021 at 6:22 PM Greg Nancarrow wrote: > > > > For example, in the v86-0001 patch: > > > > +/* > > + * Handle PREPARE message. > > + */ > > +static void > > +apply_handle_prepare(StringInfo s) > > +{ > > +

Re: [HACKERS] logical decoding of two-phase transactions

On Thu, Jun 17, 2021 at 7:40 PM Ajin Cherian wrote: > > On Wed, Jun 16, 2021 at 9:08 AM Peter Smith wrote: > > > > On Fri, Jun 11, 2021 at 6:34 PM Peter Smith wrote: > > > > > KNOWN ISSUES: This v85 patch was built and tested using yesterday's > > > master, but due to lots of recent activity in

Re: snapshot too old issues, first around wraparound and then more.

On Wed, Apr 15, 2020 at 2:21 PM Thomas Munro wrote: > On Mon, Apr 13, 2020 at 2:58 PM Thomas Munro wrote: > > On Fri, Apr 3, 2020 at 2:22 PM Peter Geoghegan wrote: > > > I'm thinking of a version of "snapshot too old" that amounts to a > > > statement timeout that gets applied for xmin horizon

Re: Optionally automatically disable logical replication subscriptions on error

On Fri, Jun 18, 2021 at 1:48 AM Mark Dilger wrote: > > Hackers, > > Logical replication apply workers for a subscription can easily get stuck in > an infinite loop of attempting to apply a change, triggering an error (such > as a constraint violation), exiting with an error written to the

Re: [PoC] Federated Authn/z with OAUTHBEARER

On Tue, Jun 08, 2021 at 04:37:46PM +, Jacob Champion wrote: > 1. Prep > > 0001 decouples the SASL code from the SCRAM implementation. > 0002 makes it possible to use common/jsonapi from the frontend. > 0003 lets the json_errdetail() result be freed, to avoid leaks. > > The first three

Re: pgbench logging broken by time logic changes

On Wed, Jun 16, 2021 at 03:13:30PM -0400, Andrew Dunstan wrote: > On 6/16/21 2:59 PM, Fabien COELHO wrote: > > The key feedback for me is the usual one: what is not tested does not > > work. Wow:-) > > Agreed. > > > I'm unhappy because I already added tap tests for time-sensitive > > features

Re: Decoding speculative insert with toast leaks memory

On Thu, Jun 17, 2021 at 2:55 PM Amit Kapila wrote: > > Your patch looks good to me as well. I would like to retain the > comment as it is from master for now. I'll do some testing and push it > tomorrow unless there are additional comments. > Pushed! -- With Regards, Amit Kapila.

Re: snapshot too old issues, first around wraparound and then more.

On Wed, Jun 16, 2021 at 12:00:57PM -0400, Tom Lane wrote: > Greg Stark writes: > > I think Andres's point earlier is the one that stands out the most for me: > > > > > I still think that's the most reasonable course. I actually like the > > > feature, but I don't think a better implementation of

Re: Fix for segfault in logical replication on master

On Thu, Jun 17, 2021 at 9:26 PM Mark Dilger wrote: > > > On Jun 17, 2021, at 6:40 AM, Amit Kapila wrote: > > > > I think such a problem won't happen because we are using historic > > snapshots in this context. We rely on that in a similar way in > > reorderbuffer.c, see ReorderBufferProcessTXN.

Re: Decoding of two-phase xacts missing from CREATE_REPLICATION_SLOT command

On Tue, Jun 15, 2021 at 5:34 PM Ajin Cherian wrote: Since we've decided to not commit this for PG-14, I've added these patches along with the larger patch-set for subscriber side 2pc in thread [1] [1] -

RE: Transactions involving multiple postgres foreign servers, take 2

From: Robert Haas > On Tue, Jun 15, 2021 at 5:51 AM tsunakawa.ta...@fujitsu.com > wrote: > > Postgres can do that, but other implementations can not necessaily do it, > > I'm > afraid. But before that, the FDW interface documentation doesn't describe > anything about how to handle

RE: locking [user] catalog tables vs 2pc vs logical rep

On Thursday, June 17, 2021 10:34 PM Simon Riggs wrote: > On Thu, Jun 17, 2021 at 12:57 PM Amit Kapila > wrote: > > On Thu, Jun 17, 2021 at 4:27 PM Amit Kapila > wrote: > > > > > > On Thu, Jun 17, 2021 at 8:41 AM osumi.takami...@fujitsu.com > > > wrote: > > > > > > Pushed! > > > > >

Re: Teaching users how they can get the most out of HOT in Postgres 14

On Thu, Jun 17, 2021 at 5:55 AM Justin Pryzby wrote: > (Various sgml typos) Fixed in the v3 I just posted. > + removed until index cleanup is completed. This option has no > + effect for tables that do not have an index and is ignored if > + the FULL option is used. > > I'd say

Re: Teaching users how they can get the most out of HOT in Postgres 14

On Thu, Jun 17, 2021 at 2:14 AM Masahiko Sawada wrote: > Thank you for updating the patch! Here are comments on v2 patch: Thanks for the review! Attached is v3, which has all the changes that you suggested (plus the doc stuff from Justin). I also renamed the "default" VacOptTernaryValue

Re: Centralizing protective copying of utility statements

Michael Paquier writes: > The DECLARE CURSOR case in ExplainOneUtility() does a copy of a Query. > Perhaps a comment should be added to explain why a copy is still > required? I did add a comment about that in the v2 patch --- the issue is the call path for EXPLAIN EXECUTE.

Re: [HACKERS] logical decoding of two-phase transactions

On Thu, Jun 17, 2021 at 6:22 PM Greg Nancarrow wrote: > > On Wed, Jun 16, 2021 at 9:08 AM Peter Smith wrote: > > > > > > Please find attached the latest patch set v86* > > > > A couple of comments: > > (1) I think one of my suggested changes was missed (or was that > intentional?): > > BEFORE:

Re: Avoid call MaintainOldSnapshotTimeMapping, if old_snapshot_threshold is disabled.

Em qui., 17 de jun. de 2021 às 22:08, Andres Freund escreveu: > Hi, > > On 2021-06-17 21:27:15 -0300, Ranier Vilela wrote: > > While another long thread discusses the situation of > old_snapshot_threshold, > > I believe we can improve procarray.c by avoiding calling > >

Re: Outdated replication protocol error?

On Wed, 2021-06-16 at 16:17 -0700, Andres Freund wrote: > I think we should explicitly compute the current timeline before > using > ThisTimelineID. E.g. in StartReplication() call a new version of > GetFlushRecPtr() that also returns the current timeline id. I think all we need to do is follow

Re: fdatasync performance problem with large number of DB files

Thomas, could you comment on this ? On Sat, May 29, 2021 at 02:23:21PM -0500, Justin Pryzby wrote: > On Tue, May 25, 2021 at 07:13:59PM -0500, Justin Pryzby wrote: > > On Sat, Mar 20, 2021 at 12:16:27PM +1300, Thomas Munro wrote: > > > > > + { > > > > > +

Re: Avoid call MaintainOldSnapshotTimeMapping, if old_snapshot_threshold is disabled.

Hi, On 2021-06-17 21:27:15 -0300, Ranier Vilela wrote: > While another long thread discusses the situation of old_snapshot_threshold, > I believe we can improve procarray.c by avoiding calling > MaintainOldSnapshotTimeMapping (src/backend/utils/time/snapmgr.c). > > There's a very explicit

Re: Centralizing protective copying of utility statements

On Thu, Jun 17, 2021 at 02:08:34PM -0700, Andres Freund wrote: > Unfortunately github is annoying to search through - it doesn't seem to > have any logic to prevent duplicates across repositories :(. Which means > there's dozens of copies of postgres code included... I agree with the position of

Re: pgbench logging broken by time logic changes

On Fri, Jun 18, 2021 at 12:49:42AM +1200, Thomas Munro wrote: > I'm on the fence TBH, I can see that it's really small things and it > seems we have the patches, but it's late, late enough that > benchmarking gurus are showing up to benchmark with it for real, and > it's not great to be getting in

Avoid call MaintainOldSnapshotTimeMapping, if old_snapshot_threshold is disabled.

Hi, While another long thread discusses the situation of old_snapshot_threshold, I believe we can improve procarray.c by avoiding calling MaintainOldSnapshotTimeMapping (src/backend/utils/time/snapmgr.c). There's a very explicit comment there, which says (line 1866): "Never call this function

Re: Replication protocol doc fix

On Thu, 2021-06-17 at 12:42 -0400, Robert Haas wrote: > On a casual read-through this seems pretty reasonable, but it > essentially documents that libpq is doing the wrong thing by sending > Sync unconditionally. As I say above, I disagree with that from a > philosophical perspective. Then again,

Re: Patch for bug #17056 fast default on non-plain table

Andrew Dunstan writes: > On 6/17/21 1:45 PM, Tom Lane wrote: >> OK. One other point is that in HEAD, you only need the hunk that >> prevents atthasmissing from becoming incorrectly set. > Good point. Should I replace the relcache.c changes in HEAD with an > Assert? Or just skip them altogether?

Re: Patch for bug #17056 fast default on non-plain table

On 6/17/21 1:45 PM, Tom Lane wrote: > Andrew Dunstan writes: >> revised patch attached. > OK. One other point is that in HEAD, you only need the hunk that > prevents atthasmissing from becoming incorrectly set. The hacks > to cope with it being already wrong are only needed in the back >

Re: Centralizing protective copying of utility statements

I wrote: > (In any case, if someone does get excited about this, they > could rearrange things to push the copyObject calls into the > individual arms of the switch in ProcessUtility. Personally > though I doubt it could be worth the code bloat.) It occurred to me to try making the copying code

Re: Centralizing protective copying of utility statements

Hi, On 2021-06-17 16:50:57 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2021-06-17 15:53:22 -0400, Tom Lane wrote: > >> Uh, nobody ever promised that server-internal APIs are frozen as of beta1; > >> that would be a horrid crimp on our ability to fix bugs during beta. > > > Sure,

Re: pgbench logging broken by time logic changes

Hello Thomas, I prepared a draft revert patch for discussion, just in case it comes in handy. This reverts "pgbench: Improve time logic.", but "pgbench: Synchronize client threads." remains (slightly rearranged). I had a quick look. I had forgotten that this patch also fixed the

Re: Centralizing protective copying of utility statements

Andres Freund writes: > On 2021-06-17 15:53:22 -0400, Tom Lane wrote: >> Uh, nobody ever promised that server-internal APIs are frozen as of beta1; >> that would be a horrid crimp on our ability to fix bugs during beta. > Sure, there's no promise. But I still think it's worth taking the amount >

Re: Centralizing protective copying of utility statements

Andres Freund writes: > On 2021-06-16 21:39:49 -0400, Tom Lane wrote: >> Although this adds some overhead in the form of copying of >> utility node trees that won't actually mutate during execution, >> I think that won't be too bad because those trees tend to be >> small and hence cheap to copy.

Re: pgbench logging broken by time logic changes

Hello Greg, I think the only thing you and I disagree on is that you see a "first issue in a corner case" where I see a process failure that is absolutely vital for me to improve. Hmmm. I agree that improvements are needed, but for me there is simply a few missing (removed) tap tests which

Optionally automatically disable logical replication subscriptions on error

Hackers, Logical replication apply workers for a subscription can easily get stuck in an infinite loop of attempting to apply a change, triggering an error (such as a constraint violation), exiting with an error written to the subscription worker log, and restarting. As things currently

Re: Centralizing protective copying of utility statements

Hi, On 2021-06-17 15:53:22 -0400, Tom Lane wrote: > Andres Freund writes: > > Phew. Do we really want to break a quite significant number of > > extensions this long after feature freeze? Since we already need to find > > a backpatchable way to deal with the issue it seems like deferring the > >

Re: Add version macro to libpq-fe.h

Robert Haas writes: > On Thu, Jun 17, 2021 at 2:34 PM Tom Lane wrote: >> ... Just because the >> version-number approach offloads work from us doesn't make it a good >> idea, because the work doesn't vanish; it will be dumped in the laps >> of packagers and end users. > What work? Including an

Re: Centralizing protective copying of utility statements

Andres Freund writes: > Phew. Do we really want to break a quite significant number of > extensions this long after feature freeze? Since we already need to find > a backpatchable way to deal with the issue it seems like deferring the > API change to 15 might be prudent? Uh, nobody ever promised

Re: Add version macro to libpq-fe.h

On Thu, Jun 17, 2021 at 2:34 PM Tom Lane wrote: > Interesting, but then you have to explain why this is the first time > that somebody has asked for a version number in libpq-fe.h. Maybe > all those previous additions were indeed minor enough that the > problem didn't come up. (Another likely

Re: Add version macro to libpq-fe.h

On Thu, Jun 17, 2021 at 08:15:42PM +0100, Dagfinn Ilmari Mannsåker wrote: > Tom Lane writes: > > Robert Haas writes: > >> I just went and looked at how exports.txt has evolved over the years. > >> Since PostgreSQL 8.1, every release except for 9.4 and 11 added at > >> least one new function to

Re: unnesting multirange data types

On Thu, Jun 17, 2021 at 7:54 PM Alexander Korotkov wrote: > On Wed, Jun 16, 2021 at 3:44 PM Alexander Korotkov > wrote: > > On Tue, Jun 15, 2021 at 8:18 PM Tom Lane wrote: > > > Alexander Korotkov writes: > > > > I did run "check-world", but it passed for me. Probably the same > > > > reason

Re: Centralizing protective copying of utility statements

Hi, On 2021-06-17 13:03:29 -0400, Tom Lane wrote: > Here's a v2 that does it like that. In this formulation, we're > basically hoisting the responsibility for doing copyObject up into > ProcessUtility from its direct children, which seems like a clearer > way of thinking about what has to

Re: Centralizing protective copying of utility statements

Hi, On 2021-06-16 21:39:49 -0400, Tom Lane wrote: > Although this adds some overhead in the form of copying of > utility node trees that won't actually mutate during execution, > I think that won't be too bad because those trees tend to be > small and hence cheap to copy. The statements that can

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

On Thu, Jun 17, 2021 at 2:48 PM Andres Freund wrote: > Or do you mean that looking at the filesystem at all is bypassing shared > buffers? This is what I mean. I think we will end up in a better spot if we can avoid doing that without creating too much ugliness elsewhere. -- Robert Haas EDB:

Re: Add version macro to libpq-fe.h

Tom Lane writes: > Robert Haas writes: >> I just went and looked at how exports.txt has evolved over the years. >> Since PostgreSQL 8.1, every release except for 9.4 and 11 added at >> least one new function to libpq. That means in 14 releases we've done >> something that might break someone's

Re: Add version macro to libpq-fe.h

Hi, On 2021-06-17 14:41:40 -0400, Tom Lane wrote: > Andres Freund writes: > > I'm not sure I understand why you think that exposing the version number > > for libpq is such a bad idea? > > I think it'd be reasonable to add a few more carefully chosen macros to > > pg_config_ext.h. > > The

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

Hi, On 2021-06-17 14:22:52 -0400, Robert Haas wrote: > On Thu, Jun 17, 2021 at 2:17 PM Andres Freund wrote: > > Adding a hacky special case implementation for cross-database relation > > accesses that violates all kinds of assumptions (like holding a lock on > > a relation when accessing it /

Re: Add version macro to libpq-fe.h

Andres Freund writes: > I'm not sure I understand why you think that exposing the version number > for libpq is such a bad idea? > I think it'd be reasonable to add a few more carefully chosen macros to > pg_config_ext.h. The primary problem I've got with that is the risk of confusion between

Re: Add version macro to libpq-fe.h

Robert Haas writes: > I just went and looked at how exports.txt has evolved over the years. > Since PostgreSQL 8.1, every release except for 9.4 and 11 added at > least one new function to libpq. That means in 14 releases we've done > something that might break someone's compile 12 times. Now

Re: Add version macro to libpq-fe.h

Hi, On 2021-06-17 13:16:17 -0400, Tom Lane wrote: > > Then again, why would pg_config.h be absent? > > Likely because somebody decided it was a server-side include rather > than an application-side include. Which is the right call - pg_config.h can't easily be included in applications that

Re: Add version macro to libpq-fe.h

On Thu, Jun 17, 2021 at 1:16 PM Tom Lane wrote: > We don't really add major new APIs to libpq very often, so I don't > find that too compelling. I do find it compelling that user code > shouldn't embed knowledge about "feature X arrived in version Y". I just went and looked at how exports.txt

Re: Add version macro to libpq-fe.h

> On 17 Jun 2021, at 19:16, Tom Lane wrote: > A more critical point is that if pg_config is present, it'll likely > contain the server version, which might not have a lot to do with the > libpq version. Debian's already shipping things in a way that decouples > those, and I gather Red Hat is

Re: Centralizing protective copying of utility statements

On Thu, Jun 17, 2021 at 01:03:29PM -0400, Tom Lane wrote: > > Here's a v2 that does it like that. In this formulation, we're > basically hoisting the responsibility for doing copyObject up into > ProcessUtility from its direct children, which seems like a clearer > way of thinking about what has

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

On Thu, Jun 17, 2021 at 5:20 AM Heikki Linnakangas wrote: > You only need relpersistence if you want to use the buffer cache, right? > I think that's a good argument for not using it. I think the root of the problem with this feature is that it doesn't go through shared_buffers, so in my

Re: Patch for bug #17056 fast default on non-plain table

Andrew Dunstan writes: > revised patch attached. OK. One other point is that in HEAD, you only need the hunk that prevents atthasmissing from becoming incorrectly set. The hacks to cope with it being already wrong are only needed in the back branches. Since we already forced initdb for beta2,

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

On Wed, Jun 16, 2021 at 6:13 PM Andres Freund wrote: > I don't think the main issue is the speed of checkpointing itself? The reaoson > to maintain the old paths is that the "new approach" is bloating WAL volume, > no? Right now cloning a 1TB database costs a few hundred bytes of WAL and > about

Re: Add version macro to libpq-fe.h

Robert Haas writes: > On Thu, Jun 17, 2021 at 9:34 AM Tom Lane wrote: >> I think putting a version number as such in there is a truly >> horrid idea. However, I could get behind adding a boolean flag >> that says specifically whether the pipeline feature exists. > I realize that this kind of

Re: Centralizing protective copying of utility statements

I wrote: > What seems like a much safer answer is to make the API change noisy. > That is, move the responsibility for actually calling copyObject > into ProcessUtility itself, and add a bool parameter saying whether > that needs to be done. That forces all callers to consider the > issue, and

Re: Add version macro to libpq-fe.h

On Thu, Jun 17, 2021 at 9:34 AM Tom Lane wrote: > I think putting a version number as such in there is a truly > horrid idea. However, I could get behind adding a boolean flag > that says specifically whether the pipeline feature exists. > Then you'd do something like > > #ifdef

Re: Unresolved repliaction hang and stop problem.

On 2021-Jun-17, Kyotaro Horiguchi wrote: > I don't see a call to hash_*seq*_search there. Instead, I see one in > ReorderBufferCheckMemoryLimit(). Doh, of course -- I misread. ReorderBufferCheckMemoryLimit is new in pg13 (cec2edfa7859) so now at least we have a reason why this workload

Re: unnesting multirange data types

On Wed, Jun 16, 2021 at 3:44 PM Alexander Korotkov wrote: > On Tue, Jun 15, 2021 at 8:18 PM Tom Lane wrote: > > Alexander Korotkov writes: > > > I did run "check-world", but it passed for me. Probably the same > > > reason it passed for some buildfarm animals... > > > > It looks to me like the

Re: Replication protocol doc fix

On Wed, Jun 16, 2021 at 5:15 PM Jeff Davis wrote: > * A normal command, where you know that you've sent everything that you > will send. In this case, the client needs to send the Sync message in > order to get the ReadyForQuery message. > > * A command that initiates CopyIn/CopyBoth, where you

Re: Fix for segfault in logical replication on master

> On Jun 17, 2021, at 6:40 AM, Amit Kapila wrote: > > I think such a problem won't happen because we are using historic > snapshots in this context. We rely on that in a similar way in > reorderbuffer.c, see ReorderBufferProcessTXN. I think you are right, but that's the part I have trouble

Re: Patch for bug #17056 fast default on non-plain table

On 6/17/21 11:13 AM, Andrew Dunstan wrote: > On 6/17/21 11:05 AM, Tom Lane wrote: >> Andrew Dunstan writes: >>> Here's a patch I propose to apply to fix this bug (See >>>

Re: pgbench logging broken by time logic changes

Is there an identified issue beyond the concrete example Greg gave of the timestamps? AFAICS, there is a patch which fixes all known issues linked to pgbench logging. Whether there exists other issues is possible, but the "broken" area was quite specific. There are also some TAP tests on

Re: snapshot too old issues, first around wraparound and then more.

Greetings, * Peter Geoghegan (p...@bowt.ie) wrote: > On Wed, Jun 16, 2021 at 12:06 PM Andres Freund wrote: > > > I would think that it wouldn't really matter inside VACUUM -- it would > > > only really need to be either an opportunistic pruning or an > > > opportunistic index deletion thing --

Re: Centralizing protective copying of utility statements

I wrote: > What I found is that there are only two places that call > ProcessUtility with a statement that might be coming from the plan > cache. _SPI_execute_plan is always doing so, so it can just > unconditionally copy the statement. The other one is > PortalRunUtility, which can examine the

Re: pgbench logging broken by time logic changes

On Wed, Jun 16, 2021 at 2:59 PM Fabien COELHO wrote: > I'm unhappy because I already added tap tests for time-sensitive features > (-T and others, maybe logging aggregates, cannot remember), which have > been removed because they could fail under some circonstances (eg very > very very very slow

Re: Iterating on IndexTuple attributes and nbtree page-level dynamic prefix truncation

On Fri, 23 Apr 2021 at 12:45, Matthias van de Meent wrote: > > On Sat, 17 Apr 2021 at 01:05, Peter Geoghegan wrote: > > > It would be convenient if the first patch could be treated > > as an independent thing. > > Patch 0002 was the reason for writing 0001, and uses the performance >

Re: Patch for bug #17056 fast default on non-plain table

On 6/17/21 11:05 AM, Tom Lane wrote: > Andrew Dunstan writes: >> Here's a patch I propose to apply to fix this bug (See >> ) > If I'm reading the code correctly, your

Re: Patch for bug #17056 fast default on non-plain table

Andrew Dunstan writes: > Here's a patch I propose to apply to fix this bug (See > ) If I'm reading the code correctly, your change in RelationBuildTupleDesc is

Patch for bug #17056 fast default on non-plain table

Here's a patch I propose to apply to fix this bug (See ) cheers andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com >From

Re: [bug?] Missed parallel safety checks, and wrong parallel safety

On Thu, Jun 17, 2021 at 4:54 AM Amit Kapila wrote: > I have responded about heavy-weight locking stuff in my next email [1] > and why I think the approach I mentioned will work. I don't deny that > I might be missing something here. > > [1] - >

Re: Update maintenance_work_mem/autovacuum_work_mem to reflect the 1GB limitation with VACUUM

The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: not tested Spec compliant: not tested Documentation:not tested Latest patch looks fine to me, to be clear. The new status of this patch

Re: pgbench logging broken by time logic changes

On 6/17/21 8:49 AM, Thomas Munro wrote: > On Thu, Jun 17, 2021 at 7:24 PM Fabien COELHO wrote: >>> Seems like it should be straightforward to fix, though, with fixes >>> already proposed (though I haven't studied them yet, will do). >> I think that fixing logging is simple enough, thus a revert

Re: Fix for segfault in logical replication on master

On Thu, Jun 17, 2021 at 6:50 PM Mark Dilger wrote: > > > On Jun 17, 2021, at 3:39 AM, osumi.takami...@fujitsu.com wrote: > > > > For the 1st check, isn't it better to use RelationIsValid() ? > > Yes, you are right. > > > Additionally, In what kind of actual scenario, did you think that > > we

Re: locking [user] catalog tables vs 2pc vs logical rep

On Thu, Jun 17, 2021 at 12:57 PM Amit Kapila wrote: > > On Thu, Jun 17, 2021 at 4:27 PM Amit Kapila wrote: > > > > On Thu, Jun 17, 2021 at 8:41 AM osumi.takami...@fujitsu.com > > wrote: > > > > Pushed! > > > [Responding to Simon's comments] > > > If LOCK and TRUNCATE is advised against on all

Re: Add version macro to libpq-fe.h

Boris Kolpackov writes: > I am making use of the new pipeline mode added to libpq in > PostgreSQL 14. At the same time I would still like to support > older libpq versions by not providing the extended functionality > that depends on this mode. Good point. > The natural way to achieve this in

Re: Fix for segfault in logical replication on master

> On Jun 17, 2021, at 3:39 AM, osumi.takami...@fujitsu.com wrote: > > For the 1st check, isn't it better to use RelationIsValid() ? Yes, you are right. > Additionally, In what kind of actual scenario, did you think that > we come to the part to "log a complaint" ? The way that

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

On Thu, Jun 17, 2021 at 2:50 PM Heikki Linnakangas wrote: > > On 17/06/2021 08:45, Dilip Kumar wrote: > > Another problem with directly scanning the directory is, how we > > are supposed to get the "relpersistence" which is stored in pg_class > > tuple right? > > You only need relpersistence if

Re: Issue with some calls to GetMultiXactIdMembers()

On 16/06/2021 13:22, Greg Nancarrow wrote: Hi, There's a couple of calls to GetMultiXactIdMembers() in heapam.c which subsequently pfree() the returned "members" pointer (pass-by-reference parameter) if it's non-NULL. However, there's an error return within GetMultiXactIdMembers() that returns

Re: Teaching users how they can get the most out of HOT in Postgres 14

+ AUTO. With OFF index + cleanup is disabled, with ON it is enabled, OFF comma + bypassing index cleanup in cases where there is more than zero + dead tuples. *are* more than zero Or (preferably): "cases when there are no dead tuples at all" + If INDEX_CLEANUP is set

Re: when the startup process doesn't (logging startup delays)

+ * Codes of the operations performed during startup process + */ +typedef enum StartupProcessOp +{ + SYNCFS_IN_PROGRESS, + FSYNC_IN_PROGRESS, + RECOVERY_IN_PROGRESS, + RESET_UNLOGGED_REL_IN_PROGRESS, + DUMMY, + SYNCFS_END, + FSYNC_END, +

Re: pgbench logging broken by time logic changes

On Thu, Jun 17, 2021 at 7:24 PM Fabien COELHO wrote: > > Seems like it should be straightforward to fix, though, with fixes > > already proposed (though I haven't studied them yet, will do). > > I think that fixing logging is simple enough, thus a revert is not > necessary. I prepared a draft

Re: locking [user] catalog tables vs 2pc vs logical rep

On Thu, Jun 17, 2021 at 4:27 PM Amit Kapila wrote: > > On Thu, Jun 17, 2021 at 8:41 AM osumi.takami...@fujitsu.com > wrote: > > Pushed! > [Responding to Simon's comments] > If LOCK and TRUNCATE is advised against on all user catalog tables, why would > CLUSTER only apply to pg_class? Surely

Re: pgbench logging broken by time logic changes

I found you forgot to fix printProgressReport(). Indeed. Also, according to the document, interval_start in Aggregated Logging seems to be printed in seconds instead of ms. Indeed. I'm unsure about what we should really want there, but for a beta bug fix I agree that it must simply

Re: Unresolved repliaction hang and stop problem.

On Thu, Jun 17, 2021 at 7:28 AM Kyotaro Horiguchi wrote: > > At Wed, 16 Jun 2021 18:28:28 -0400, Alvaro Herrera > wrote in > > On 2021-Jun-16, Ha Ka wrote: > > # Children Self Command Shared Object Symbol > > # . > >

Re: Fix for segfault in logical replication on master

On Thu, Jun 17, 2021 at 4:09 PM osumi.takami...@fujitsu.com wrote: > > On Thursday, June 17, 2021 2:43 PM Mark Dilger > wrote: > > > On Jun 16, 2021, at 10:19 PM, osumi.takami...@fujitsu.com wrote: > > > > > > I started to analyze your report and > > > will reply after my idea to your

Re: PoC: Using Count-Min Sketch for join cardinality estimation

On 6/17/21 2:23 AM, Tomas Vondra wrote: > On 6/17/21 1:31 AM, John Naylor wrote: >> On Wed, Jun 16, 2021 at 12:23 PM Tomas Vondra >> mailto:tomas.von...@enterprisedb.com>> >> wrote: >> >> ... >> >> + * depth 8 and width 128 is sufficient for relative error ~1.5% with a >> + * probability of

Re: when the startup process doesn't

> Since you agreed that SUSET was wrong, and PGC_POSTMASTER doesn't allow > changing the value without restart, doesn't it follow that SIGHUP is what's > wanted ? Yes. I have done the changes in the attached patch. Apart from this, I have done a few other changes to the patch. The changes

Re: locking [user] catalog tables vs 2pc vs logical rep

On Thu, Jun 17, 2021 at 8:41 AM osumi.takami...@fujitsu.com wrote: > > On Wednesday, June 16, 2021 7:21 PM Amit Kapila > wrote: > > On Mon, Jun 14, 2021 at 5:33 PM osumi.takami...@fujitsu.com > > wrote: > > > > > > On Friday, June 11, 2021 2:13 PM vignesh C > > wrote: > > > > > > Attached

RE: Fix for segfault in logical replication on master

On Thursday, June 17, 2021 2:43 PM Mark Dilger wrote: > > On Jun 16, 2021, at 10:19 PM, osumi.takami...@fujitsu.com wrote: > > > > I started to analyze your report and > > will reply after my idea to your modification is settled. > > Thank you. I'll share my first analysis. > In commit

Re: pgbench bug candidate: negative "initial connection time"

Second, currently the *only* function to change the client state is advanceConnectionState, so it can be checked there and any bug is only there. We had issues before when several functions where doing updates, and it was a mess to understand what was going on. I really want that it stays that

Re: Less compiler errors in pg_crc32c_sse42_choose.c

On Tue, Jun 15, 2021 at 03:04:11PM +0200, Nat! wrote: Please don't send emails in html only. Those includes are protected by some #ifdef which shouldn't be set unless configure detects that that they're usable. Do you have a different behavior?

Less compiler errors in pg_crc32c_sse42_choose.c

I am just quoting the whole file here for simplicity, as its smallhttps://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/port/pg_crc32c_sse42_choose.c;h=0608e02011f7f5d8dbba7673a5ab4ba071d017a0;hb=e4f9737fac77a5cb03a84d1f4038d300ffd28afd.In line #43 the compiler errors out, if there is

Re: Decoding speculative insert with toast leaks memory

On Thu, Jun 17, 2021 at 1:35 PM Amit Langote wrote: > > Hi Dilip, > > On Thu, Jun 17, 2021 at 4:45 PM Dilip Kumar wrote: > > On Thu, Jun 17, 2021 at 12:52 PM Amit Langote > > wrote: > > > > > Oh I missed that the problem report is for the PG13 branch. > > > > > > How about the attached patch

Re: Skipping logical replication transactions on subscriber side

On Thu, Jun 17, 2021 at 3:24 PM Masahiko Sawada wrote: > > > Now, if this function is used by super > > users then we can probably trust that they provide the XIDs that we > > can trust to be skipped but OTOH making a restriction to allow these > > functions to be used by superusers might

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

On 17/06/2021 08:45, Dilip Kumar wrote: Another problem with directly scanning the directory is, how we are supposed to get the "relpersistence" which is stored in pg_class tuple right? You only need relpersistence if you want to use the buffer cache, right? I think that's a good argument for

Re: pgbench bug candidate: negative "initial connection time"

Hello Fabien, On Thu, 17 Jun 2021 10:37:05 +0200 (CEST) Fabien COELHO wrote: > > ). Is this acceptable for you? > > I disagree on two counts: > > First, thread[0] should not appear. > > Second, currently the *only* function to change the client state is > advanceConnectionState, so it can

  1   2   >