Re: failures in t/031_recovery_conflict.pl on CI

2022-04-08 Thread Andres Freund
Hi, On 2022-04-08 21:55:15 -0700, Andres Freund wrote: > on CI [1] the new t/031_recovery_conflict.pl is failing occasionally. Which is > interesting, because I ran it there dozens if not hundreds of times before > commit, with - I think - only cosmetic changes. Scratch that part - I found an

failures in t/031_recovery_conflict.pl on CI

2022-04-08 Thread Andres Freund
Hi, on CI [1] the new t/031_recovery_conflict.pl is failing occasionally. Which is interesting, because I ran it there dozens if not hundreds of times before commit, with - I think - only cosmetic changes. I've reproduced it in a private branch, with more logging. And the results are sure

Re: Add parameter jit_warn_above_fraction

2022-04-08 Thread Julien Rouhaud
Hi, On Fri, Apr 08, 2022 at 09:39:18AM -0400, Stephen Frost wrote: > > * Magnus Hagander (mag...@hagander.net) wrote: > > The addition to pg_stat_statements I pushed a short while ago would help > > with that. But I think having a warning like this would also be useful. As > > a stop-gap measure,

Re: Expose JIT counters/timing in pg_stat_statements

2022-04-08 Thread Julien Rouhaud
Hi, On Sat, Apr 09, 2022 at 01:51:21AM +, Shinoda, Noriyoshi (PN Japan FSIP) wrote: > Hi, > thank you for the great features. > > The attached small patch changes the data type in the document. > The following columns are actually double precision but bigint in the docs. >

Commitfest wrapup

2022-04-08 Thread Greg Stark
These remaining CF entries look like they're bugs that are maybe Open Issues for release? * fix spinlock contention in LogwrtResult * Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work * Add checkpoint and redo LSN to LogCheckpointEnd log message *

cfbot requests

2022-04-08 Thread Justin Pryzby
I mentioned most/all of these ideas for cfbot at some point. I'm writing them now so other people know about them and they're in once place. - Keep the original patch series and commit messages, rather than squishing them into a single commit with cfbot's own commit messages. Maybe append an

Re: PostgreSQL commitfest: 2022-09-01

2022-04-08 Thread Justin Pryzby
On Fri, Apr 08, 2022 at 12:55:04PM +, webmas...@postgresql.org wrote: > The following patches that you follow, directly or indirectly, > have received updates in the PostgreSQL commitfest app: > > > Function to log backtrace of postgres processes > https://commitfest.postgresql.org/38/2863/

RE: Expose JIT counters/timing in pg_stat_statements

2022-04-08 Thread Shinoda, Noriyoshi (PN Japan FSIP)
Hi, thank you for the great features. The attached small patch changes the data type in the document. The following columns are actually double precision but bigint in the docs. jit_generation_time jit_inlining_time jit_optimization_time jit_emission_count Regards, Noriyoshi Shinoda From:

Re: avoid multiple hard links to same WAL file after a crash

2022-04-08 Thread Justin Pryzby
On Fri, Apr 08, 2022 at 09:00:36PM -0400, Robert Haas wrote: > > I think there might be another problem. The man page for rename() seems to > > indicate that overwriting an existing file also introduces a window where > > the old and new path are hard links to the same file. This isn't a problem

Re: avoid multiple hard links to same WAL file after a crash

2022-04-08 Thread Robert Haas
On Fri, Apr 8, 2022 at 12:53 PM Nathan Bossart wrote: > On Fri, Apr 08, 2022 at 10:38:03AM -0400, Robert Haas wrote: > > I see that durable_rename_excl() has the following comment: "Similar > > to durable_rename(), except that this routine tries (but does not > > guarantee) not to overwrite the

Re: pgsql: Add TAP test for archive_cleanup_command and recovery_end_comman

2022-04-08 Thread Andres Freund
Hi, On 2022-04-08 17:55:51 -0400, Tom Lane wrote: > I tried adjusting the patch so it does guarantee that (as attached), > and in two out of two tries it got past the archive_cleanup_command > failure but then hung up waiting for standby2 to promote. Adding $node_standby->safe_psql('postgres',

Re: Mingw task for Cirrus CI

2022-04-08 Thread Justin Pryzby
On Thu, Apr 07, 2022 at 10:10:21AM -0700, Andres Freund wrote: > Hi, > > On 2022-04-06 11:03:37 -0400, Andrew Dunstan wrote: > > On 3/30/22 20:26, Andres Freund wrote: > > > Could you try using dash to invoke configure here, and whether it makes > > > configure faster? > > I got weird failures

Re: pgsql: Add TAP test for archive_cleanup_command and recovery_end_comman

2022-04-08 Thread Andres Freund
Hi, On 2022-04-08 17:55:51 -0400, Tom Lane wrote: > After seeing skink's results, I tried running that test under valgrind > here, and it fails just like that every time. skink's history allows > us to bound the failure introduction between 79b716cfb7 and > d7ab2a9a3c, which I think makes it

Re: Mingw task for Cirrus CI

2022-04-08 Thread Andres Freund
Hi, On 2022-04-08 17:04:34 +0200, Alvaro Herrera wrote: > > Since dash won't help us to get the build time down sufficiently, and the > > tests don't pass without a separate build tree, I looked at what makes > > config/prep_buildtree so slow. > > Maybe we can replace prep_buildtree with a Perl

Re: Lowering the ever-growing heap->pd_lower

2022-04-08 Thread Peter Geoghegan
On Fri, Apr 8, 2022 at 1:29 PM Robert Haas wrote: > Hmm. I wonder if we could teach the system to figure out which of > those things is happening. In the case that I'm worried about, when > we're considering growing the line pointer array, either the line > pointers will be dead or the line

Re: pgsql: Add TAP test for archive_cleanup_command and recovery_end_comman

2022-04-08 Thread Tom Lane
Andres Freund writes: > On 2022-04-07 13:57:45 -0400, Tom Lane wrote: >> Yeah, with only one instance it could just be cosmic rays or something. >> However, assuming it is real, I guess I wonder why we don't say >> CHECKPOINT_FORCE in standby mode too. > I guess it might partially be that

Re: Lowering the ever-growing heap->pd_lower

2022-04-08 Thread Peter Geoghegan
On Fri, Apr 8, 2022 at 2:06 PM Andres Freund wrote: > It's not hard to hit scenarios where pages are effectively unusable, because > they have close to 291 dead items, without autovacuum triggering (or > autovacuum just taking a while). I think that this is mostly a problem with HOT-updates, and

Re: MERGE bug report

2022-04-08 Thread Alvaro Herrera
On 2022-Apr-06, Richard Guo wrote: > That's right. The varattno is set to zero for whole-row Var. And in this > case these whole-row Vars are not included in the targetlist. > > Attached is an attempt for the fix. Wow, this is very interesting. I was surprised that this patch was necessary at

Re: Lowering the ever-growing heap->pd_lower

2022-04-08 Thread Peter Geoghegan
On Fri, Apr 8, 2022 at 2:18 PM Andres Freund wrote: > It's 4 bytes per line pointer, right? Yeah, it's 4 bytes in Postgres. Most other DB systems only need 2 bytes, which is implemented in exactly the way that you're imagining. -- Peter Geoghegan

Re: Lowering the ever-growing heap->pd_lower

2022-04-08 Thread Andres Freund
Hi, On 2022-04-08 15:04:37 -0400, Robert Haas wrote: > I meant wasting space in the page. I think that's a real concern. > Imagine you allowed 1000 line pointers per page. Each one consumes 2 > bytes. It's 4 bytes per line pointer, right? struct ItemIdData { unsigned int

Re: Lowering the ever-growing heap->pd_lower

2022-04-08 Thread Andres Freund
Hi, On 2022-04-08 09:17:40 -0400, Robert Haas wrote: > I agree that the value of 291 is pretty much accidental, but it also > seems fairly generous to me. The bigger you make it, the more space > you can waste. I must have missed (or failed to understand) previous > discussions about why raising

Re: Pre-allocating WAL files

2022-04-08 Thread Nathan Bossart
On Thu, Mar 17, 2022 at 04:12:12PM -0700, Nathan Bossart wrote: > It seems unlikely that this will be committed for v15, so I've adjusted the > commitfest entry to v16 and moved it to the next commitfest. rebased -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From

Re: Lowering the ever-growing heap->pd_lower

2022-04-08 Thread Robert Haas
On Fri, Apr 8, 2022 at 3:31 PM Peter Geoghegan wrote: > What if we miss the opportunity to systematically keep successor > versions of a given logical row on the same heap page over time, due > only to the current low MaxHeapLinePointersPerPage limit of 291? If we > had only been able to "absorb"

Re: Frontend error logging style

2022-04-08 Thread Tom Lane
I wrote: > One other loose end is bothering me: I stuck with logging.h's > original choice to put "if (likely())" or "if (unlikely())" > conditionals into the macros, but I rather suspect that that's > just a waste. I think we should put a centralized level check > into logging.c, and get rid of

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2022-04-08 Thread Andrei Zubkov
Hi, I've rebased this patch so that it can be applied after 57d6aea00fc. v14 attached -- regards, Andrei From 6c541f3001d952e72e5d865fde09de3fb4f36d10 Mon Sep 17 00:00:00 2001 From: Andrei Zubkov Date: Fri, 8 Apr 2022 23:12:55 +0300 Subject: [PATCH] pg_stat_statements: Track statement entry

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-04-08 Thread Nathan Bossart
On Wed, Mar 30, 2022 at 09:21:30AM -0700, Nathan Bossart wrote: > Here is an updated patch set. rebased -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From a92ccf47c9c334eea5b8d07b8fcab7031181c37e Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Tue, 15 Feb 2022 09:40:53

Re: avoid multiple hard links to same WAL file after a crash

2022-04-08 Thread Nathan Bossart
On Fri, Apr 08, 2022 at 10:38:03AM -0400, Robert Haas wrote: > I'd actually be in favor of nuking durable_rename_excl() from orbit > and putting the file-exists tests in the callers. Otherwise, someone > might assume that it actually has the semantics that its name > suggests, which could be

Re: Lowering the ever-growing heap->pd_lower

2022-04-08 Thread Peter Geoghegan
On Fri, Apr 8, 2022 at 12:04 PM Robert Haas wrote: > I meant wasting space in the page. I think that's a real concern. > Imagine you allowed 1000 line pointers per page. Each one consumes 2 > bytes. So now you could have ~25% of each page in the table storing > dead line pointers. That sounds

Re: Frontend error logging style

2022-04-08 Thread Tom Lane
Daniel Gustafsson writes: > On 30 Mar 2022, at 00:38, Tom Lane wrote: >> Feel free to work on a followup editing patch though. > Thats my plan, once this lands I'll rebase the comments on top of your work > and > we can have a separate discussion around them then. The main patch is pushed

Re: Lowering the ever-growing heap->pd_lower

2022-04-08 Thread Robert Haas
On Fri, Apr 8, 2022 at 12:57 PM Peter Geoghegan wrote: > What do you mean about wasting space? Wasting space on the stack? I > can't imagine you meant wasting space on the page, since being able to > accomodate more items on each heap page seems like it would be > strictly better, barring any

Re: Lowering the ever-growing heap->pd_lower

2022-04-08 Thread Peter Geoghegan
On Fri, Apr 8, 2022 at 9:44 AM Peter Geoghegan wrote: > On Fri, Apr 8, 2022 at 4:38 AM Matthias van de Meent > wrote: > > Yeah, I think we should definately support more line pointers on a > > heap page, but abusing MaxHeapTuplesPerPage for that is misleading: > > the current value is the

Re: trigger example for plsample

2022-04-08 Thread Mark Wong
On Thu, Apr 07, 2022 at 06:29:53PM -0400, Tom Lane wrote: > Chapman Flack writes: > > v4 looks good to me. > > Pushed with very minor editorialization. Mainly, I undid the > decision to stop printing the function source text, on the > grounds that (1) it falsified the comment immediately above,

Re: avoid multiple hard links to same WAL file after a crash

2022-04-08 Thread Nathan Bossart
On Fri, Apr 08, 2022 at 09:53:12AM -0700, Nathan Bossart wrote: > On Fri, Apr 08, 2022 at 10:38:03AM -0400, Robert Haas wrote: >> I'd actually be in favor of nuking durable_rename_excl() from orbit >> and putting the file-exists tests in the callers. Otherwise, someone >> might assume that it

Re: Defer selection of asynchronous subplans until the executor initialization stage

2022-04-08 Thread Etsuro Fujita
Hi, On Fri, Apr 8, 2022 at 9:43 PM Justin Pryzby wrote: > This patch seems to be causing the planner to crash. > Here's a query reduced from sqlsmith. > > | explain SELECT 1 FROM information_schema.constraint_column_usage WHERE 1 <= > pg_trigger_depth(); > > Program terminated with signal

Re: Lowering the ever-growing heap->pd_lower

2022-04-08 Thread Peter Geoghegan
On Fri, Apr 8, 2022 at 6:17 AM Robert Haas wrote: > I agree that the value of 291 is pretty much accidental, but it also > seems fairly generous to me. The bigger you make it, the more space > you can waste. I must have missed (or failed to understand) previous > discussions about why raising it

Re: avoid multiple hard links to same WAL file after a crash

2022-04-08 Thread Nathan Bossart
On Fri, Apr 08, 2022 at 10:38:03AM -0400, Robert Haas wrote: > I see that durable_rename_excl() has the following comment: "Similar > to durable_rename(), except that this routine tries (but does not > guarantee) not to overwrite the target file." If those are the desired > semantics, we could

Re: How to simulate sync/async standbys being closer/farther (network distance) to primary in core postgres?

2022-04-08 Thread SATYANARAYANA NARLAPURAM
On Fri, Apr 8, 2022 at 6:44 AM Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > On Wed, Apr 6, 2022 at 4:30 PM Ashutosh Bapat > wrote: > > > > On Tue, Apr 5, 2022 at 9:23 PM Bharath Rupireddy > > wrote: > > > > > > Hi, > > > > > > I'm thinking if there's a way in core

Re: Lowering the ever-growing heap->pd_lower

2022-04-08 Thread Peter Geoghegan
On Fri, Apr 8, 2022 at 4:38 AM Matthias van de Meent wrote: > Yeah, I think we should definately support more line pointers on a > heap page, but abusing MaxHeapTuplesPerPage for that is misleading: > the current value is the physical limit for heap tuples, as we have at > most 1 heap tuple per

Re: pgsql: Add TAP test for archive_cleanup_command and recovery_end_comman

2022-04-08 Thread Tom Lane
Andres Freund writes: > On 2022-04-07 13:57:45 -0400, Tom Lane wrote: >> Yeah, with only one instance it could just be cosmic rays or something. Not cosmic rays: skink has shown the same symptom three times running. Looks like maybe the archive_cleanup_command itself is doing something it

Re: Defer selection of asynchronous subplans until the executor initialization stage

2022-04-08 Thread Zhihong Yu
On Fri, Apr 8, 2022 at 5:43 AM Justin Pryzby wrote: > On Wed, Apr 06, 2022 at 03:58:29PM +0900, Etsuro Fujita wrote: > > I have committed the patch after modifying it as such. (I think we > > can improve these later, if necessary.) > > This patch seems to be causing the planner to crash. >

Re: Size of pg_rewrite (Was: Report checkpoint progress with pg_stat_progress_checkpoint)

2022-04-08 Thread Andres Freund
Hi, On April 8, 2022 7:52:07 AM PDT, Matthias van de Meent wrote: >On Sat, 19 Mar 2022 at 01:15, Andres Freund wrote: >> pg_rewrite without pg_stat_progress_checkpoint: 745472, with: 753664 >> >> pg_rewrite is the second biggest relation in an empty database already... > >Yeah, that's not

Re: remove more archiving overhead

2022-04-08 Thread Nathan Bossart
On Fri, Apr 08, 2022 at 10:20:27AM -0400, Robert Haas wrote: > On Thu, Apr 7, 2022 at 6:23 PM Nathan Bossart > wrote: >> On Thu, Feb 24, 2022 at 09:55:53AM -0800, Nathan Bossart wrote: >> > Yes. I found that a crash at an unfortunate moment can produce multiple >> > links to the same file in

Re: Temporary file access API

2022-04-08 Thread Robert Haas
On Fri, Apr 8, 2022 at 4:54 AM Antonin Houska wrote: > Do you think that the use of a system call is a problem itself (e.g. because > the code looks less simple if read/write is used somewhere and fread/fwrite > elsewhere; of course of read/write is mandatory in special cases like WAL, > heap

Re: shared-memory based stats collector

2022-04-08 Thread Andres Freund
Hi, On April 8, 2022 4:49:48 AM PDT, Ranier Vilela wrote: >Hi, > >Per Coverity. > >pgstat_reset_entry does not check if lock it was really blocked. >I think if shared_stat_reset_contents is called without lock, >is it an issue not? I don't think so - the nowait parameter is say to false, so

Re: Atomic rename feature for Windows.

2022-04-08 Thread Robert Haas
On Fri, Apr 8, 2022 at 11:45 AM Greg Stark wrote: > But that's useful for some things and not for others. Like, it's > useful to be sure we don't have odd dependencies on timing quirks of > the specific machines that are currently common, or depend on gcc/llvm > compiler behaviour that isn't

Re: API stability [was: pgsql: Fix possible recovery trouble if TRUNCATE overlaps a checkpoint.]

2022-04-08 Thread Robert Haas
On Fri, Apr 8, 2022 at 4:47 AM Markus Wanner wrote: > I agree with Michael, it would be nice to not duplicate the code, but > use a common underlying method. A modified patch is attached. I don't think this is better, but I don't think it's worth arguing about, either, so I'll do it this way if

Re: Atomic rename feature for Windows.

2022-04-08 Thread Greg Stark
On Fri, 8 Apr 2022 at 11:30, Robert Haas wrote: > > On Fri, Apr 8, 2022 at 10:12 AM Greg Stark wrote: > > On Thu, 9 Dec 2021 at 23:36, Tom Lane wrote: > > > I'm not for dropping support for some platform just because it's old. > > > > I guess I'll have to spin up the Vax again :) > > This is a

Re: Commitfest Closed

2022-04-08 Thread Greg Stark
I moved to next CF almost all the Needs Review and Waiting on Author patches. The remaining ones are either: 1) Bug fixes, Documentation, or testing patches that we may want to make Open Issues 2) Patches that look like we may want to mark Rejected or Returned with Feedback and start a new

Re: Kerberos delegation support in libpq and postgres_fdw

2022-04-08 Thread Robert Haas
On Fri, Apr 8, 2022 at 11:29 AM Stephen Frost wrote: > > + allow_cred_delegation > > > > First, I again recommend not choosing words at random to abbreviate. > > "delegate_credentials" would be shorter and clearer. Second, I think > > we need to decide whether we envision just having one

Re: Size of pg_rewrite

2022-04-08 Thread Matthias van de Meent
On Fri, 8 Apr 2022 at 17:20, Dagfinn Ilmari Mannsåker wrote: > > Matthias van de Meent writes: > > > But, as text literal concatenations don't seem to get constant folded > > before storing them in the rules table, this rewrite of the views > > would result in long lines in the system_views.sql

Re: Atomic rename feature for Windows.

2022-04-08 Thread Robert Haas
On Fri, Apr 8, 2022 at 10:12 AM Greg Stark wrote: > On Thu, 9 Dec 2021 at 23:36, Tom Lane wrote: > > I'm not for dropping support for some platform just because it's old. > > I guess I'll have to spin up the Vax again :) This is a pretty good summary of what's wrong with our current deprecation

Re: Kerberos delegation support in libpq and postgres_fdw

2022-04-08 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Fri, Apr 8, 2022 at 8:21 AM Stephen Frost wrote: > > Added an explicit 'environment' option to allow for, basically, existing > > behavior, where we don't mess with the environment variable at all, > > though I kept the default as

Re: Size of pg_rewrite

2022-04-08 Thread Dagfinn Ilmari Mannsåker
Matthias van de Meent writes: > But, as text literal concatenations don't seem to get constant folded > before storing them in the rules table, this rewrite of the views > would result in long lines in the system_views.sql file, or we'd have > to deal with the additional overhead of the append

Re: Mingw task for Cirrus CI

2022-04-08 Thread Melih Mutlu
> On windows that makes prep_buildtree go from 42.4s to 5.8s for me. > I applied Andres's faster prep build tree changes and triggered some cirrus runs Without these changes, preparing build tree was taking around 42.3s (sometimes even more) [1] It seems like with these changes it drops to

Re: Commitfest Closed

2022-04-08 Thread Julien Rouhaud
On Fri, Apr 08, 2022 at 08:09:16AM -0700, Peter Geoghegan wrote: > On Fri, Apr 8, 2022 at 5:58 AM Alvaro Herrera wrote: > > Thanks for herding through the CF! > > +1 +1!

Re: [Proposal] vacuumdb --schema only

2022-04-08 Thread Gilles Darold
Le 08/04/2022 à 02:46, Justin Pryzby a écrit : On Wed, Apr 06, 2022 at 07:43:42PM +0200, Gilles Darold wrote: Thanks for the review, all these changes are available in new version v6 of the patch and attached here. This is failing in CI (except on macos, which is strangely passing).

Re: [COMMITTERS] pgsql: Allow time delayed standbys and recovery

2022-04-08 Thread Robert Haas
On Fri, Apr 8, 2022 at 10:45 AM Thom Brown wrote: > Thanks. This doesn't include my self-correction: > > s/kept on standby/kept on the standby/ Here is v2, endeavoring to rectify that oversight. -- Robert Haas EDB: http://www.enterprisedb.com

Re: Commitfest Closed

2022-04-08 Thread Peter Geoghegan
On Fri, Apr 8, 2022 at 5:58 AM Alvaro Herrera wrote: > Thanks for herding through the CF! +1 -- Peter Geoghegan

Re: Mark all GUC variable as PGDLLIMPORT

2022-04-08 Thread Julien Rouhaud
On Fri, Apr 08, 2022 at 03:04:18PM +0200, Magnus Hagander wrote: > On Fri, Apr 8, 2022 at 2:42 PM Robert Haas wrote: > > > On Wed, Apr 6, 2022 at 7:56 PM Michael Paquier > > wrote: > > > On Wed, Apr 06, 2022 at 12:57:29AM +0700, John Naylor wrote: > > > > For these two patches, I'd say a day or

Re: Mingw task for Cirrus CI

2022-04-08 Thread Alvaro Herrera
On 2022-Apr-07, Andres Freund wrote: > Since dash won't help us to get the build time down sufficiently, and the > tests don't pass without a separate build tree, I looked at what makes > config/prep_buildtree so slow. Maybe we can replace prep_buildtree with a Perl script. Surely that should

Re: Kerberos delegation support in libpq and postgres_fdw

2022-04-08 Thread Robert Haas
On Fri, Apr 8, 2022 at 8:21 AM Stephen Frost wrote: > Added an explicit 'environment' option to allow for, basically, existing > behavior, where we don't mess with the environment variable at all, > though I kept the default as MEMORY since I don't think it's really > typical that folks actually

Re: Mingw task for Cirrus CI

2022-04-08 Thread Melih Mutlu
Hi Andrew, You should set MSYSTEM=UCRT64 in the environment section. Given that, > there should be no need to specify a --host= setting for configure. > It's set to UCRT64 in the docker image side [1]. I didn't know --host isn't necessary on UCRT64 environment. I'll remove it then. [1]

Size of pg_rewrite (Was: Report checkpoint progress with pg_stat_progress_checkpoint)

2022-04-08 Thread Matthias van de Meent
On Sat, 19 Mar 2022 at 01:15, Andres Freund wrote: > pg_rewrite without pg_stat_progress_checkpoint: 745472, with: 753664 > > pg_rewrite is the second biggest relation in an empty database already... Yeah, that's not great. Thanks for nerd-sniping me into looking into how views and pg_rewrite

Re: [COMMITTERS] pgsql: Allow time delayed standbys and recovery

2022-04-08 Thread Thom Brown
On Fri, 8 Apr 2022, 14:36 Robert Haas, wrote: > On Wed, Apr 6, 2022 at 8:15 AM Robert Haas wrote: > > On Tue, Apr 5, 2022 at 8:43 PM Thom Brown wrote: > > > I share your discomfort with the wording. How about: > > > > > > WAL records must be kept on standby until they are ready to be applied.

Re: avoid multiple hard links to same WAL file after a crash

2022-04-08 Thread Robert Haas
On Thu, Apr 7, 2022 at 2:30 PM Nathan Bossart wrote: > Presently, WAL recycling uses durable_rename_excl(), which notes that a > crash at an unfortunate moment can result in two links to the same file. > My testing [1] demonstrated that it was possible to end up with two links > to the same file

Re: REINDEX blocks virtually any queries but some prepared queries.

2022-04-08 Thread Frédéric Yhuel
On 4/8/22 02:22, Michael Paquier wrote: On Thu, Apr 07, 2022 at 05:29:36PM +0200, Guillaume Lelarge a écrit : Le jeu. 7 avr. 2022 à 15:44, Frédéric Yhuel a écrit : On 4/7/22 14:40, Justin Pryzby wrote: Thank you Justin! I applied your fixes in the v2 patch (attached). v2 patch sounds

Re: remove more archiving overhead

2022-04-08 Thread Robert Haas
On Thu, Apr 7, 2022 at 6:23 PM Nathan Bossart wrote: > On Thu, Feb 24, 2022 at 09:55:53AM -0800, Nathan Bossart wrote: > > Yes. I found that a crash at an unfortunate moment can produce multiple > > links to the same file in pg_wal, which seemed bad independent of archival. > > By fixing that

GSOC: New and improved website for pgjdbc (JDBC) (2022)

2022-04-08 Thread S.R Keshav
postgreSql_ New and improved website for pgjdbc (JDBC) (2022).pdf Description: Adobe PDF document

Re: Atomic rename feature for Windows.

2022-04-08 Thread Greg Stark
On Thu, 9 Dec 2021 at 23:36, Tom Lane wrote: > > I'm not for dropping support for some platform just because it's old. I guess I'll have to spin up the Vax again :)

Re: why pg_walfile_name() cannot be executed during recovery?

2022-04-08 Thread Robert Haas
On Fri, Apr 8, 2022 at 9:31 AM Bharath Rupireddy wrote: > Fundamental question - should the pg_walfile_{name, name_offset} check > whether the file with the computed WAL file name exists on the server > right now or ever existed earlier? Right now, they don't do that, see > [1]. I don't think

Re: Can we automatically add elapsed times to tap test log?

2022-04-08 Thread Andrew Dunstan
On 4/7/22 19:55, Andrew Dunstan wrote: > On 4/7/22 17:58, Andres Freund wrote: >> Hi, >> >> On 2022-04-07 17:45:09 -0400, Tom Lane wrote: >>> Andres Freund writes: On 2022-04-07 17:21:09 -0400, Tom Lane wrote: > I too think that the elapsed time is useful. I'm less convinced > that

Re: GSoC: New and improved website for pgjdbc (JDBC)

2022-04-08 Thread Dave Cramer
Joseph On Thu, 7 Apr 2022 at 17:49, Joseph Ho wrote: > Hello, > > I am Joseph Ho, a senior at Dr Norman Bethune Collegiate Institute > interested in going into computer science. I am interested in working to > create and improve the website for pgjdbc during GSoC 2022. > > I am wondering how

Re: [COMMITTERS] pgsql: Allow time delayed standbys and recovery

2022-04-08 Thread Magnus Hagander
On Fri, Apr 8, 2022 at 3:36 PM Robert Haas wrote: > On Wed, Apr 6, 2022 at 8:15 AM Robert Haas wrote: > > On Tue, Apr 5, 2022 at 8:43 PM Thom Brown wrote: > > > I share your discomfort with the wording. How about: > > > > > > WAL records must be kept on standby until they are ready to be

Re: How to simulate sync/async standbys being closer/farther (network distance) to primary in core postgres?

2022-04-08 Thread Bharath Rupireddy
On Wed, Apr 6, 2022 at 4:30 PM Ashutosh Bapat wrote: > > On Tue, Apr 5, 2022 at 9:23 PM Bharath Rupireddy > wrote: > > > > Hi, > > > > I'm thinking if there's a way in core postgres to achieve $subject. In > > reality, the sync/async standbys can either be closer/farther (which > > means

Re: Add parameter jit_warn_above_fraction

2022-04-08 Thread Stephen Frost
Greetings, * Magnus Hagander (mag...@hagander.net) wrote: > On Fri, Apr 8, 2022 at 2:19 PM David Rowley wrote: > > On Fri, 8 Apr 2022 at 23:27, Magnus Hagander wrote: > > > On Wed, Mar 30, 2022 at 3:04 PM Magnus Hagander > > wrote: > > >> > > >> On Tue, Mar 29, 2022 at 10:06 PM David Rowley >

Re: [COMMITTERS] pgsql: Allow time delayed standbys and recovery

2022-04-08 Thread Robert Haas
On Wed, Apr 6, 2022 at 8:15 AM Robert Haas wrote: > On Tue, Apr 5, 2022 at 8:43 PM Thom Brown wrote: > > I share your discomfort with the wording. How about: > > > > WAL records must be kept on standby until they are ready to be applied. > > Therefore, longer delays will result in a greater

Re: Support for grabbing multiple consecutive values with nextval()

2022-04-08 Thread Greg Stark
On Sun, 27 Feb 2022 at 07:09, Jille Timmermans wrote: > > Hi, > > First time PostgreSQL contributor here :) I wish I had noticed this patch during the CF. It seems like a nice self-contained feature that could have been easily reviewed and committed and it's always good to see first-time

Re: why pg_walfile_name() cannot be executed during recovery?

2022-04-08 Thread Bharath Rupireddy
On Thu, Apr 7, 2022 at 9:07 PM Robert Haas wrote: > > On Thu, Apr 7, 2022 at 9:32 AM Bharath Rupireddy > wrote: > > I spent some time today to allow pg_walfile_{name, name_offset} run in > > recovery. Timeline ID is computed while in recovery as follows - WAL > > receiver's last received and

Re: Lowering the ever-growing heap->pd_lower

2022-04-08 Thread Robert Haas
On Thu, Apr 7, 2022 at 7:01 PM Andres Freund wrote: > On 2022-04-04 19:24:22 -0700, Peter Geoghegan wrote: > > We should definitely increase MaxHeapTuplesPerPage before too long, > > for a variety of reasons that I have talked about in the past. Its > > current value is 291 on all mainstream

Re: Support logical replication of DDLs

2022-04-08 Thread Robert Haas
On Fri, Apr 8, 2022 at 7:34 AM Alvaro Herrera wrote: > > For runtime conditions, one of the things you have mentioned in that > > thread is to add schema name in the statement at the required places > > which this patch deals with in a different way by explicitly sending > > it along with the DDL

Re: Extract epoch from Interval weird behavior

2022-04-08 Thread Tom Lane
Peter Eisentraut writes: > We really wanted to avoid doing calculations in numeric as much as > possible. So we should figure out a different way to write this. The > attached patch works for me. It's a bit ugly since it hardcodes some > factors. Maybe we can rephrase it a bit more

Re: Mark all GUC variable as PGDLLIMPORT

2022-04-08 Thread Magnus Hagander
On Fri, Apr 8, 2022 at 2:42 PM Robert Haas wrote: > On Wed, Apr 6, 2022 at 7:56 PM Michael Paquier > wrote: > > On Wed, Apr 06, 2022 at 12:57:29AM +0700, John Naylor wrote: > > > For these two patches, I'd say a day or two after feature freeze is a > > > reasonable goal. > > > > Yeah. For

Re: Commitfest Closed

2022-04-08 Thread Alvaro Herrera
On 2022-Apr-08, Greg Stark wrote: > Thanks to all the reviewers and committers who put a lot of work in, > especially in the last two weeks. I especially want to thank Andres > who showed me how to use the cfbot to check on patch statuses and did > a lot of work doing that until I was up to

Commitfest Closed

2022-04-08 Thread Greg Stark
It has reached 2022-03-08 Anywhere on Earth[*] so I believe that means Postgres 15 Feature Freeze is in effect (modulo a couple patches that were held until the end of the commitfest to make merging easier). I've marked the commitfest closed and will be moving any patches that didn't receive

Re: Defer selection of asynchronous subplans until the executor initialization stage

2022-04-08 Thread Justin Pryzby
On Wed, Apr 06, 2022 at 03:58:29PM +0900, Etsuro Fujita wrote: > I have committed the patch after modifying it as such. (I think we > can improve these later, if necessary.) This patch seems to be causing the planner to crash. Here's a query reduced from sqlsmith. | explain SELECT 1 FROM

Re: Mark all GUC variable as PGDLLIMPORT

2022-04-08 Thread Robert Haas
On Wed, Apr 6, 2022 at 7:56 PM Michael Paquier wrote: > On Wed, Apr 06, 2022 at 12:57:29AM +0700, John Naylor wrote: > > For these two patches, I'd say a day or two after feature freeze is a > > reasonable goal. > > Yeah. For patches as invasive as the PGDLLIMPORT business and the > frontend

Re: Add parameter jit_warn_above_fraction

2022-04-08 Thread Magnus Hagander
On Fri, Apr 8, 2022 at 2:19 PM David Rowley wrote: > On Fri, 8 Apr 2022 at 23:27, Magnus Hagander wrote: > > > > > > > > On Wed, Mar 30, 2022 at 3:04 PM Magnus Hagander > wrote: > >> > >> On Tue, Mar 29, 2022 at 10:06 PM David Rowley > wrote: > >>> > >>> If we go with this patch, the problem

Re: Kerberos delegation support in libpq and postgres_fdw

2022-04-08 Thread Stephen Frost
Greetings, * Stephen Frost (sfr...@snowman.net) wrote: > The new krb_user_ccache is a lot closer to 'global', though it's > specifically for user-authenticated backends (allowing the postmaster > and other things like replication connections to use whatever the > credential cache is set to by the

Re: Add parameter jit_warn_above_fraction

2022-04-08 Thread David Rowley
On Fri, 8 Apr 2022 at 23:27, Magnus Hagander wrote: > > > > On Wed, Mar 30, 2022 at 3:04 PM Magnus Hagander wrote: >> >> On Tue, Mar 29, 2022 at 10:06 PM David Rowley wrote: >>> >>> If we go with this patch, the problem I see here is that the amount >>> of work the JIT compiler must do for a

Re: SQL/JSON: functions

2022-04-08 Thread Andrew Dunstan
On 4/8/22 08:02, Justin Pryzby wrote: > On Thu, Mar 31, 2022 at 04:25:58PM -0400, Andrew Dunstan wrote: >> No code chunks left, only a documentation patch which should land > Documentation review for a6baa4bad. > >> Construct a JSON the provided strings: > a JSON what ? > *from* the provided

Re: SQL/JSON: functions

2022-04-08 Thread Justin Pryzby
On Thu, Mar 31, 2022 at 04:25:58PM -0400, Andrew Dunstan wrote: > No code chunks left, only a documentation patch which should land Documentation review for a6baa4bad. > Construct a JSON the provided strings: a JSON what ? *from* the provided strings ? > Construct a JSON from the provided

Re: PROXY protocol support

2022-04-08 Thread Magnus Hagander
On Sat, Apr 2, 2022 at 12:17 AM wilfried roset wrote: > Hi, > > I've been able to test the patch. Here is a recap of the experimentation. > > # Setup > > All tests have been done witch 3 VMs (PostgreSQL, HAproxy, psql client) on > Debian 11 communicating over private network. > * PostgreSQL have

Re: shared-memory based stats collector

2022-04-08 Thread Ranier Vilela
Hi, Per Coverity. pgstat_reset_entry does not check if lock it was really blocked. I think if shared_stat_reset_contents is called without lock, is it an issue not? regards, Ranier Vilela 0001-avoid-reset-stats-without-lock.patch Description: Binary data

Re: Expose JIT counters/timing in pg_stat_statements

2022-04-08 Thread Magnus Hagander
On Tue, Mar 8, 2022 at 4:08 AM Julien Rouhaud wrote: > On Mon, Mar 07, 2022 at 01:40:34PM +0100, Magnus Hagander wrote: > > > > I wonder if there might be an interesting middle ground, or if that is > > making it too much. That is, we could have an > > Option 3: > > jit_count > > total_jit_time

Re: generic plans and "initial" pruning

2022-04-08 Thread Amit Langote
Hi David, On Fri, Apr 8, 2022 at 8:16 PM David Rowley wrote: > On Fri, 8 Apr 2022 at 17:49, Amit Langote wrote: > > Attached updated patch with these changes. > Thanks for making the changes. I started looking over this patch but > really feel like it needs quite a few more iterations of what

Re: Extract epoch from Interval weird behavior

2022-04-08 Thread Peter Eisentraut
On 24.02.22 03:35, Joseph Koshakow wrote: However when executing EXTRACT we first truncate DAYS_PER_YEAR to an integer, and then multiply it by the total years in the Interval /* this always fits into int64 */ secs_from_day_month = ((int64) DAYS_PER_YEAR * (interval->month / MONTHS_PER_YEAR) +

Re: Lowering the ever-growing heap->pd_lower

2022-04-08 Thread Matthias van de Meent
On Fri, 8 Apr 2022 at 01:01, Andres Freund wrote: > > Hi, > > On 2022-04-04 19:24:22 -0700, Peter Geoghegan wrote: > > We should definitely increase MaxHeapTuplesPerPage before too long, > > for a variety of reasons that I have talked about in the past. Its > > current value is 291 on all

Re: Support logical replication of DDLs

2022-04-08 Thread Alvaro Herrera
On 2022-Apr-08, Amit Kapila wrote: > On Thu, Mar 17, 2022 at 3:36 AM Alvaro Herrera > wrote: > > > > Did you see some old code I wrote towards this goal? > > https://www.postgresql.org/message-id/20150215044814.gl3...@alvh.no-ip.org > > The intention was that DDL would produce some JSON blob

Re: Last day of commitfest

2022-04-08 Thread Magnus Hagander
On Thu, Apr 7, 2022 at 3:59 AM Julien Rouhaud wrote: > > > * JIT counters in pg_stat_statements (Magnus Hagander) > > Feedback from Dmitry Dolgov and Julien Rouhaud > > Note that the code looks good and no one disagreed with the proposed > fields. > > The only remaining problem is a copy/pasto

Re: Add parameter jit_warn_above_fraction

2022-04-08 Thread Stephen Frost
Greetings, On Fri, Apr 8, 2022 at 07:27 Magnus Hagander wrote: > On Wed, Mar 30, 2022 at 3:04 PM Magnus Hagander > wrote: > >> On Tue, Mar 29, 2022 at 10:06 PM David Rowley >> wrote: >> >>> On Wed, 30 Mar 2022 at 02:38, Robert Haas wrote: >>> > I think WARNING is fine. After all, the

  1   2   >