Re: Clang optimiser vs preproc.c

2023-12-15 Thread Thomas Munro
On Sat, Dec 16, 2023 at 4:19 PM Tom Lane wrote: > We actually noticed this or a closely-related problem before [1] > and briefly discussed the possibility of rearranging the generated > code to make it less indigestible to clang. But there was no concrete > idea about what to do specifically,

Re: Statistics Import and Export

2023-12-15 Thread Corey Huinker
On Fri, Dec 15, 2023 at 3:36 AM Andrei Lepikhov wrote: > On 13/12/2023 17:26, Corey Huinker wrote:> 4. I don't yet have a > complete vision for how these tools will be used > > by pg_upgrade and pg_dump/restore, the places where these will provide > > the biggest win for users. > > Some issues

Re: Clang optimiser vs preproc.c

2023-12-15 Thread Tom Lane
Thomas Munro writes: > On Sat, Dec 16, 2023 at 3:44 PM Tom Lane wrote: >> Thomas Munro writes: >>> FYI, it looks like there is a big jump in CPU time to compile preproc.c at >>> -O2: >>> clang15: ~16s >>> clang16: ~211s >>> clang17: ~233s >> What are the numbers for gram.c? > clang15: ~3.8s

Re: Clang optimiser vs preproc.c

2023-12-15 Thread Thomas Munro
On Sat, Dec 16, 2023 at 3:44 PM Tom Lane wrote: > Thomas Munro writes: > > FYI, it looks like there is a big jump in CPU time to compile preproc.c at > > -O2: > > > clang15: ~16s > > clang16: ~211s > > clang17: ~233s > > What are the numbers for gram.c? clang15: ~3.8s clang16: ~3.2s clang17:

Re: Clang optimiser vs preproc.c

2023-12-15 Thread Tom Lane
Thomas Munro writes: > FYI, it looks like there is a big jump in CPU time to compile preproc.c at > -O2: > clang15: ~16s > clang16: ~211s > clang17: ~233s What are the numbers for gram.c? regards, tom lane

Clang optimiser vs preproc.c

2023-12-15 Thread Thomas Munro
Hi, FYI, it looks like there is a big jump in CPU time to compile preproc.c at -O2: clang15: ~16s clang16: ~211s clang17: ~233s First noticed on FreeBSD (where the system cc is now clang16), but reproduced also on Debian (via apt.llvm.org packages).

Re: encoding affects ICU regex character classification

2023-12-15 Thread Thomas Munro
On Sat, Dec 16, 2023 at 1:48 PM Jeremy Schneider wrote: > On 12/14/23 7:12 AM, Jeff Davis wrote: > > The concern over unassigned code points is misplaced. The application > > may be aware of newly-assigned code points, and there's no way they > > will be mapped correctly in Postgres if the

Re: Transaction timeout

2023-12-15 Thread Japin Li
On Fri, 15 Dec 2023 at 17:51, Andrey M. Borodin wrote: >> On 8 Dec 2023, at 15:29, Japin Li wrote: >> >> Thanks for updating the patch. LGTM. > > PFA v9. Changes: > 1. Added tests for idle_in_transaction_timeout > 2. Suppress statement_timeout if it’s shorter than transaction_timeout > +

Re: encoding affects ICU regex character classification

2023-12-15 Thread Jeremy Schneider
On 12/14/23 7:12 AM, Jeff Davis wrote: > The concern over unassigned code points is misplaced. The application > may be aware of newly-assigned code points, and there's no way they > will be mapped correctly in Postgres if the provider is not aware of > those code points. The user can either

Re: Built-in CTYPE provider

2023-12-15 Thread Jeremy Schneider
On 12/13/23 5:28 AM, Jeff Davis wrote: > On Tue, 2023-12-12 at 13:14 -0800, Jeremy Schneider wrote: >> My biggest concern is around maintenance. Every year Unicode is >> assigning new characters to existing code points, and those existing >> code points can of course already be stored in old

Re: Revisiting {CREATE INDEX, REINDEX} CONCURRENTLY improvements

2023-12-15 Thread Matthias van de Meent
On Fri, 15 Dec 2023, 20:07 Michail Nikolaev, wrote: > Hello, hackers! > > I think about revisiting (1) ({CREATE INDEX, REINDEX} CONCURRENTLY > improvements) in some lighter way. > > Yes, a serious bug was (2) caused by this optimization and now it reverted. > > But what about a more safe idea in

Revisiting {CREATE INDEX, REINDEX} CONCURRENTLY improvements

2023-12-15 Thread Michail Nikolaev
Hello, hackers! I think about revisiting (1) ({CREATE INDEX, REINDEX} CONCURRENTLY improvements) in some lighter way. Yes, a serious bug was (2) caused by this optimization and now it reverted. But what about a more safe idea in that direction: 1) add new horizon which ignores PROC_IN_SAFE_IC

Re: POC: Extension for adding distributed tracing - pg_tracing

2023-12-15 Thread Nikita Malakhov
Hi! Overall solution looks good for me except SQL Commenter - query instrumentation with SQL comments is often not possible on production systems. Instead the very often requested feature is to enable tracing for a given single query ID, or several (very limited number of) queries by IDs. It

Re: LargeObjectRelationId vs LargeObjectMetadataRelationId, redux

2023-12-15 Thread Tom Lane
I wrote: > This is an absolutely horrid mess, of course. The big problem > is that at this point I have exactly zero confidence that there > are not other places with the same bug; and it's not apparent > how to find them. I took a look at every reference to LargeObjectRelationId and

Re: Clean up find_typedefs and add support for Mac

2023-12-15 Thread Andrew Dunstan
On 2023-12-15 Fr 11:06, Tom Lane wrote: Andrew Dunstan writes: Here's more or less what I had in mind. Do we really need the dependency on an install tree? Can't we just find the executables (or .o files for Darwin) in the build tree? Seems like that would simplify usage, and reduce the

Re: Improve eviction algorithm in ReorderBuffer

2023-12-15 Thread Euler Taveira
On Fri, Dec 15, 2023, at 9:11 AM, Masahiko Sawada wrote: > > I assume you mean to add ReorderBufferTXN entries to the binaryheap > and then build it by comparing their sizes (i.e. txn->size). But > ReorderBufferTXN entries are removed and deallocated once the > transaction finished. How can we

Re: Clean up find_typedefs and add support for Mac

2023-12-15 Thread Tristan Partin
On Fri Dec 15, 2023 at 10:06 AM CST, Tom Lane wrote: Andrew Dunstan writes: > Here's more or less what I had in mind. Do we really need the dependency on an install tree? Can't we just find the executables (or .o files for Darwin) in the build tree? Seems like that would simplify usage, and

Re: Clean up find_typedefs and add support for Mac

2023-12-15 Thread Tristan Partin
if ($using_osx) { # On OS X, we need to examine the .o files # exclude ecpg/test, which pgindent does too my $obj_wanted = sub { /^.*\.o\z/s && !($File::Find::name =~

Re: Clean up find_typedefs and add support for Mac

2023-12-15 Thread Tom Lane
Andrew Dunstan writes: > Here's more or less what I had in mind. Do we really need the dependency on an install tree? Can't we just find the executables (or .o files for Darwin) in the build tree? Seems like that would simplify usage, and reduce the possibility for version-skew errors.

Re: Add --check option to pgindent

2023-12-15 Thread Tristan Partin
On Fri Dec 15, 2023 at 8:18 AM CST, Jelte Fennema-Nio wrote: This part of the first patch seems incorrect, i.e. same condition in the if as elsif - if ($silent_diff) + if ($check) + { + print show_diff($source, $source_filename); + exit 2; + } +

Re: [PATCH] Add --syntax to postgres for SQL syntax checking

2023-12-15 Thread Josef Šimánek
pá 15. 12. 2023 v 16:32 odesílatel David G. Johnston napsal: > > On Fri, Dec 15, 2023 at 8:20 AM Josef Šimánek wrote: >> >> (parser is not available >> in public APIs of postgres_fe.h or libpq). > > > What about building "libpg" that does expose and exports some public APIs for > the parser?

Re: [PATCH] Add --syntax to postgres for SQL syntax checking

2023-12-15 Thread David G. Johnston
On Fri, Dec 15, 2023 at 8:20 AM Josef Šimánek wrote: > (parser is not available > in public APIs of postgres_fe.h or libpq). > What about building "libpg" that does expose and exports some public APIs for the parser? We can include a reference CLI implementation for basic usage of the

Re: [PATCH] Add --syntax to postgres for SQL syntax checking

2023-12-15 Thread Josef Šimánek
pá 15. 12. 2023 v 16:16 odesílatel David G. Johnston napsal: > > On Fri, Dec 15, 2023 at 8:05 AM Josef Šimánek wrote: >> >> pá 15. 12. 2023 v 15:50 odesílatel Tom Lane napsal: >> > >> > Laurenz Albe writes: >> > > On Fri, 2023-12-15 at 13:21 +0100, Josef Šimánek wrote: >> > >> Inspired by

Re: [PATCH] Add --syntax to postgres for SQL syntax checking

2023-12-15 Thread David G. Johnston
On Fri, Dec 15, 2023 at 8:05 AM Josef Šimánek wrote: > pá 15. 12. 2023 v 15:50 odesílatel Tom Lane napsal: > > > > Laurenz Albe writes: > > > On Fri, 2023-12-15 at 13:21 +0100, Josef Šimánek wrote: > > >> Inspired by Simon Riggs' keynote talk at PGCounf.eu 2023 sharing list > > >> of ideas for

Re: [PATCH] Add --syntax to postgres for SQL syntax checking

2023-12-15 Thread Josef Šimánek
pá 15. 12. 2023 v 15:50 odesílatel Tom Lane napsal: > > Laurenz Albe writes: > > On Fri, 2023-12-15 at 13:21 +0100, Josef Šimánek wrote: > >> Inspired by Simon Riggs' keynote talk at PGCounf.eu 2023 sharing list > >> of ideas for PostgreSQL > >>

Re: [PATCH] Add --syntax to postgres for SQL syntax checking

2023-12-15 Thread Tom Lane
Laurenz Albe writes: > On Fri, 2023-12-15 at 13:21 +0100, Josef Šimánek wrote: >> Inspired by Simon Riggs' keynote talk at PGCounf.eu 2023 sharing list >> of ideas for PostgreSQL >> (https://riggs.business/blog/f/postgresql-todo-2023), I have crafted a >> quick patch to do SQL syntax validation.

Re: [meson] expose buildtype debug/optimization info to pg_config

2023-12-15 Thread Andres Freund
Hi, On 2023-12-14 17:24:58 +0800, Junwang Zhao wrote: > On Thu, Dec 14, 2023 at 4:50 PM Peter Eisentraut wrote: > > > > On 12.12.23 11:40, Junwang Zhao wrote: > > > build system using configure set VAL_CFLAGS with debug and > > > optimization flags, so pg_config will show these infos. Some > > >

Re: Add --check option to pgindent

2023-12-15 Thread Jelte Fennema-Nio
This part of the first patch seems incorrect, i.e. same condition in the if as elsif - if ($silent_diff) + if ($check) + { + print show_diff($source, $source_filename); + exit 2; + } + elsif ($check) { exit 2; } On

Re: optimize atomic exchanges

2023-12-15 Thread Andres Freund
Hi, On 2023-12-04 15:08:57 -0600, Nathan Bossart wrote: > On Mon, Dec 04, 2023 at 12:18:05PM -0600, Nathan Bossart wrote: > > Barring objections or additional feedback, I think I'm inclined to press > > forward with this one and commit it in the next week or two. I'm currently > > planning to

Re: Clean up find_typedefs and add support for Mac

2023-12-15 Thread Andrew Dunstan
On 2023-12-14 Th 10:36, Tristan Partin wrote: On Thu Dec 14, 2023 at 9:16 AM CST, Andrew Dunstan wrote: On 2023-12-13 We 15:59, Tristan Partin wrote: > On Wed Dec 13, 2023 at 2:35 PM CST, Andrew Dunstan wrote: >> >> On 2023-12-12 Tu 18:02, Tom Lane wrote: >> > "Tristan Partin" writes: >> >>

Re: "pgoutput" options missing on documentation

2023-12-15 Thread Emre Hasegeli
> I saw that the original "publication_names" error was using > errcode(ERRCODE_INVALID_PARAMETER_VALUE), but TBH since there is no > option given at all I felt ERRCODE_SYNTAX_ERROR might be more > appropriate errcode for those 2 mandatory option errors. It makes sense to me. Changed. > 2. > >

Re: [PATCH] Add --syntax to postgres for SQL syntax checking

2023-12-15 Thread Pavel Stehule
Dne pá 15. 12. 2023 14:14 uživatel Josef Šimánek napsal: > > > Dne pá 15. 12. 2023 14:09 uživatel Laurenz Albe > napsal: > >> On Fri, 2023-12-15 at 13:21 +0100, Josef Šimánek wrote: >> > Inspired by Simon Riggs' keynote talk at PGCounf.eu 2023 sharing list >> > of ideas for PostgreSQL >> >

Re: [PATCH] Add --syntax to postgres for SQL syntax checking

2023-12-15 Thread Josef Šimánek
Dne pá 15. 12. 2023 14:09 uživatel Laurenz Albe napsal: > On Fri, 2023-12-15 at 13:21 +0100, Josef Šimánek wrote: > > Inspired by Simon Riggs' keynote talk at PGCounf.eu 2023 sharing list > > of ideas for PostgreSQL > > (https://riggs.business/blog/f/postgresql-todo-2023), I have crafted a > >

Re: pgBufferUsage.blk_{read|write}_time are zero although there are pgBufferUsage.local_blks_{read|written}

2023-12-15 Thread hubert depesz lubaczewski
> > Well, if it's possible to deduce what is the meaning in given line, > > I can add the logic to do the deduction to parser. > > Also, I want to say that I appreciate being looped in the discussion. > I lost sight of this thread, so my apologies for the delay. The patch > to fix the description

Re: [PATCH] Add --syntax to postgres for SQL syntax checking

2023-12-15 Thread Laurenz Albe
On Fri, 2023-12-15 at 13:21 +0100, Josef Šimánek wrote: > Inspired by Simon Riggs' keynote talk at PGCounf.eu 2023 sharing list > of ideas for PostgreSQL > (https://riggs.business/blog/f/postgresql-todo-2023), I have crafted a > quick patch to do SQL syntax validation. > > What do you think? I

Re: Synchronizing slots from primary to standby

2023-12-15 Thread Nisha Moond
Review for v47 patch - (1) When we try to create a subscription on standby using a synced slot that is in 'r' sync_state, the subscription will be created at the subscriber, and on standby, two actions will take place - (i) As copy_data is true by default, it will switch the failover state

[PATCH] Add --syntax to postgres for SQL syntax checking

2023-12-15 Thread Josef Šimánek
Hello! Inspired by Simon Riggs' keynote talk at PGCounf.eu 2023 sharing list of ideas for PostgreSQL (https://riggs.business/blog/f/postgresql-todo-2023), I have crafted a quick patch to do SQL syntax validation. It is also heavily inspired by the "ruby -c" command, useful to check syntax of

Re: Improve eviction algorithm in ReorderBuffer

2023-12-15 Thread Masahiko Sawada
On Fri, Dec 15, 2023 at 2:59 PM Masahiko Sawada wrote: > > On Fri, Dec 15, 2023 at 12:37 PM Amit Kapila wrote: > > > > On Wed, Dec 13, 2023 at 6:01 AM Masahiko Sawada > > wrote: > > > > > > On Tue, Dec 12, 2023 at 1:33 PM Dilip Kumar wrote: > > > > > > > > On Tue, Dec 12, 2023 at 9:01 AM

Re: Improve eviction algorithm in ReorderBuffer

2023-12-15 Thread Masahiko Sawada
On Fri, Dec 15, 2023 at 7:10 PM Alvaro Herrera wrote: > > On 2023-Dec-12, Masahiko Sawada wrote: > > > To deal with this problem, I initially thought of the idea (a) > > mentioned in the comment; use a binary heap to maintain the > > transactions sorted by the amount of changes or the size. But

Re: trying again to get incremental backup

2023-12-15 Thread Peter Eisentraut
A separate bikeshedding topic: The GUC "summarize_wal", could that be "wal_something" instead? (wal_summarize? wal_summarizer?) It would be nice if these settings names group together a bit, both with existing wal_* ones and also with the new ones you are adding (wal_summary_keep_time).

Re: trying again to get incremental backup

2023-12-15 Thread Peter Eisentraut
I have a couple of quick fixes here. The first fixes up some things in nls.mk related to a file move. The second is some cleanup because some function you are using has been removed in the meantime; you probably found that yourself while rebasing. The pg_walsummary patch doesn't have a

Re: trying again to get incremental backup

2023-12-15 Thread Jakub Wartak
Hi Robert, On Wed, Dec 13, 2023 at 2:16 PM Robert Haas wrote: > > > > > not even in case of an intervening > > > timeline switch. So, all of the errors in this function are warning > > > you that you've done something that you really should not have done. > > > In this particular case, you've

Re: Improve eviction algorithm in ReorderBuffer

2023-12-15 Thread Alvaro Herrera
On 2023-Dec-12, Masahiko Sawada wrote: > To deal with this problem, I initially thought of the idea (a) > mentioned in the comment; use a binary heap to maintain the > transactions sorted by the amount of changes or the size. But it seems > not a good idea to try maintaining all transactions by

Re: Transaction timeout

2023-12-15 Thread Andrey M. Borodin
> On 8 Dec 2023, at 15:29, Japin Li wrote: > > Thanks for updating the patch. LGTM. PFA v9. Changes: 1. Added tests for idle_in_transaction_timeout 2. Suppress statement_timeout if it’s shorter than transaction_timeout Consider changing status of the commitfest entry if you think it’s ready

Re: Add 64-bit XIDs into PostgreSQL 15

2023-12-15 Thread wenhui qiu
Hi Pavel Borisov Many thanks Best whish Pavel Borisov 于2023年12月15日周五 17:13写道: > Hi, Wenhui! > > On Fri, 15 Dec 2023 at 05:52, wenhui qiu wrote: > >> Hi Maxim Orlov >> Good news,xid64 has achieved a successful first phase,I tried to >> change the path status

Re: Add 64-bit XIDs into PostgreSQL 15

2023-12-15 Thread Pavel Borisov
Hi, Wenhui! On Fri, 15 Dec 2023 at 05:52, wenhui qiu wrote: > Hi Maxim Orlov > Good news,xid64 has achieved a successful first phase,I tried to > change the path status (https://commitfest.postgresql.org/43/3594/) ,But > it seems incorrect > > Maxim Orlov 于2023年12月13日周三 20:26写道: > >> Hi!

Re: planner chooses incremental but not the best one

2023-12-15 Thread Richard Guo
On Thu, Dec 14, 2023 at 6:02 PM Richard Guo wrote: > It seems that we need to improve estimate of distinct values in > estimate_num_groups() when taking the selectivity of restrictions into > account. > > In 84f9a35e3 we changed to a new formula to perform such estimation. > But that does not

Re: remaining sql/json patches

2023-12-15 Thread jian he
hi. since InitJsonItemCoercions cannot return NULL. per transformJsonFuncExpr, jsexpr->item_coercions not null imply jsexpr->result_coercion not null. so I did the attached refactoring. now every ExecInitJsonExprCoercion function call followed with: scratch->opcode = EEOP_JUMP;

Re: Statistics Import and Export

2023-12-15 Thread Andrei Lepikhov
On 13/12/2023 17:26, Corey Huinker wrote:> 4. I don't yet have a complete vision for how these tools will be used by pg_upgrade and pg_dump/restore, the places where these will provide the biggest win for users. Some issues here with docs: func.sgml:28465: parser error : Opening and ending

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-12-15 Thread Masahiko Sawada
On Fri, Dec 15, 2023 at 10:30 AM John Naylor wrote: > > On Thu, Dec 14, 2023 at 7:22 AM Masahiko Sawada wrote: > > In v45, 0001 - 0006 are from earlier versions but I've merged previous > > updates. So the radix tree now has RT_SET() and RT_FIND() but not > > RT_GET() and RT_SEARCH(). 0007 and

Re: Is a clearer memory lifespan for outerTuple and innerTuple useful?

2023-12-15 Thread Andy Fan
Andy Fan writes: > ..., I attached the 2 MemoryContext in > JoinState rather than MergeJoinState, which is for the "shared detoast > value"[0] more or less. > After thinking more, if it is designed for "shared detoast value" patch (happens on ExecInterpExpr stage), the inner_tuple_memory