Re: autovectorize page checksum code included elsewhere

2023-11-24 Thread John Naylor
On Thu, Nov 23, 2023 at 1:49 AM Nathan Bossart wrote: > > On Wed, Nov 22, 2023 at 02:54:13PM +0200, Ants Aasma wrote: > > For reference, executing the page checksum 10M times on a AMD 3900X CPU: > > > > clang-14 -O2 4.292s (17.8 GiB/s) > > clang-14 -O2 -msse4.12.859s (26.7

Re: autovectorize page checksum code included elsewhere

2023-11-24 Thread John Naylor
On Thu, Nov 23, 2023 at 11:51 PM Nathan Bossart wrote: > > On Thu, Nov 23, 2023 at 05:50:48PM +0700, John Naylor wrote: > > On Thu, Nov 23, 2023 at 1:49 AM Nathan Bossart > > wrote: > >> One half-formed idea I have is to introduce some sort of ./configure flag > >> that enables all the newer

Re: [HACKERS] patch: change magic constants to DEFINE value for readability.

2023-11-24 Thread Bruce Momjian
On Sat, May 23, 2015 at 11:40:36AM +0900, CharSyam wrote: > in src/backend/utils/misc/tzparser.c > > It uses 60 * 60 to represent SECS_PER_HOUR. > > and It is already define in other files. > > so I think using SECS_PER_HOUR is more clear for readability. > > and I attached patch.(it just

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2023-11-24 Thread Julien Rouhaud
Hi, On Sat, Nov 25, 2023 at 02:45:07AM +0200, Alexander Korotkov wrote: > > I've reviewed this patch. I think this is the feature of high demand. > New columns (stats_since and minmax_stats_since) to the > pg_stat_statements view, enhancing the granularity and precision of > performance

Re: [HACKERS] GIN pageinspect functions

2023-11-24 Thread Bruce Momjian
On Mon, Aug 10, 2015 at 09:14:48AM -0700, Jeff Janes wrote: > When I call gin_leafpage_items on a {leaf} page, I get the ERROR: > > ERROR:  input page is not a compressed GIN data leaf page > DETAIL:  Flags 0002, expected 0083 > > I'm don't know why it won't work on an uncompressed leaf page (or

Re: pg_upgrade and logical replication

2023-11-24 Thread vignesh C
On Mon, 20 Nov 2023 at 05:27, Michael Paquier wrote: > > On Sun, Nov 19, 2023 at 06:56:05AM +0530, vignesh C wrote: > > On Sun, 19 Nov 2023 at 06:52, vignesh C wrote: > >> On Fri, 10 Nov 2023 at 19:26, vignesh C wrote: > >>> I will analyze more on this and post the analysis in the subsequent

Re: POC, WIP: OR-clause support for indexes

2023-11-24 Thread Alexander Korotkov
On Fri, Nov 24, 2023 at 7:05 AM Alena Rybakina wrote: > On 23.11.2023 12:23, Andrei Lepikhov wrote: > > I think the usage of nodeToString for the generation of clause hash is > > too expensive and buggy. > > Also, in the code, you didn't resolve hash collisions. So, I've > > rewritten the patch a

Re: [PATCH] Add CHECK_FOR_INTERRUPTS in scram_SaltedPassword loop.

2023-11-24 Thread Michael Paquier
On Thu, Nov 23, 2023 at 11:19:51AM +0300, Aleksander Alekseev wrote: >>> I don't think it would be useful to limit this at an arbitrary point, >>> iteration >>> count can be set per password and if someone wants a specific password to be >>> super-hard to brute force then why should we limit

Re: POC, WIP: OR-clause support for indexes

2023-11-24 Thread Alexander Korotkov
Hi! On Mon, Nov 13, 2023 at 9:48 PM a.rybakina wrote: > These days I was porting a patch for converting or expressions to ANY to > the choose_bitmap_and function. Unfortunately, it is not possible to > transfer the conversion there, since expressions are processed one by > one, as far as I saw.

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2023-11-24 Thread Alexander Korotkov
Hi! On Fri, Nov 17, 2023 at 10:40 AM Andrei Zubkov wrote: > > A little fix in "level_tracking" tests after merge. I've reviewed this patch. I think this is the feature of high demand. New columns (stats_since and minmax_stats_since) to the pg_stat_statements view, enhancing the granularity and

Re: pg_walfile_name_offset can return inconsistent values

2023-11-24 Thread Bruce Momjian
On Mon, Nov 13, 2023 at 02:12:12PM -0500, Bruce Momjian wrote: > On Mon, Nov 13, 2023 at 09:49:53AM -0800, Andres Freund wrote: > > Hi, > > > > On 2023-11-13 12:14:57 -0500, Bruce Momjian wrote: > > > +SELECT * > > > +FROM (values ('0/16ff'), ('0/1700'), ('0/1701')) as t(lsn), > > > +

Re: Relation bulk write facility

2023-11-24 Thread Heikki Linnakangas
On 19/11/2023 02:04, Andres Freund wrote: On 2023-11-17 11:37:21 +0100, Heikki Linnakangas wrote: The new facility makes it easier to optimize bulk loading, as the logic for buffering, WAL-logging, and syncing the relation only needs to be implemented once. It's also less error-prone: We have

Re: pg_stats and range statistics

2023-11-24 Thread Alexander Korotkov
Hi! On Wed, Sep 6, 2023 at 6:18 PM jian he wrote: > +lower(ARRAY[numrange(1.1,2.2),numrange(3.3,4.4)]) > should be > + > ranges_lower(ARRAY[numrange(1.1,2.2),numrange(3.3,4.4)]) > > +upper(ARRAY[numrange(1.1,2.2),numrange(3.3,4.4)]) > should be > + >

Re: remove deprecated @@@ operator ?

2023-11-24 Thread Bruce Momjian
On Thu, Nov 9, 2023 at 06:40:55PM -0500, Bruce Momjian wrote: > On Sun, Oct 21, 2018 at 04:24:16PM -0400, Tom Lane wrote: > > Oleg Bartunov writes: > > > The commit 9b5c8d45f62bd3d243a40cc84deb93893f2f5122 is now 10+ years > > > old, may be we could remove deprecated @@@ operator ? > > > > Is

Re: Lifetime of commit timestamps

2023-11-24 Thread Bruce Momjian
On Fri, Nov 17, 2023 at 04:36:44PM -0500, Bruce Momjian wrote: > On Fri, Nov 17, 2023 at 01:20:46PM -0800, Andres Freund wrote: > > On 2023-11-17 15:39:14 -0300, Euler Taveira wrote: > > > > I think the connection between freezing and removal of commit timestamps is > > a > > lot less direct

Re: Improving the comments in pqsignal()

2023-11-24 Thread Thomas Munro
On Fri, Nov 24, 2023 at 8:55 PM Heikki Linnakangas wrote: > On 24/11/2023 00:33, Thomas Munro wrote: > > This is program 10.12 from Advanced Programming in the UNIX > > Environment, with minor changes. > In the copy I found online (3rd edition), it's "Figure 10.18", not > "program 10.12". > >

Re: Questions regarding Index AMs and natural ordering

2023-11-24 Thread Peter Geoghegan
On Fri, Nov 24, 2023 at 10:58 AM Tom Lane wrote: > Peter Geoghegan writes: > > I suppose that amcanorder=true cannot mean that, since we have the > > SAOP path key thing (at least for now). > > As things stand, amcanorder definitely means that the index always > returns ordered data, since the

Re: mxid_age() and age(xid) appear undocumented

2023-11-24 Thread Bruce Momjian
On Fri, Nov 17, 2023 at 01:39:46PM -0500, Bruce Momjian wrote: > On Mon, Nov 13, 2023 at 05:32:24PM -0800, Andres Freund wrote: > > Hi, > > > > On 2023-11-13 17:00:43 -0800, Peter Geoghegan wrote: > > > On Mon, Nov 13, 2023 at 4:43 PM Bruce Momjian wrote: > > > > I looked into this and all the

Re: Questions regarding Index AMs and natural ordering

2023-11-24 Thread Tom Lane
Peter Geoghegan writes: > On Fri, Nov 24, 2023 at 8:44 AM Matthias van de Meent > wrote: >> Yes, the part where btree opclasses determine a type's ordering is >> clear. But what I'm looking for is "how do I, as an index AM >> implementation, get the signal that I need to return column-ordered >>

Re: Questions regarding Index AMs and natural ordering

2023-11-24 Thread Peter Geoghegan
On Fri, Nov 24, 2023 at 8:44 AM Matthias van de Meent wrote: > Also, was that a confirmation that amcanorder is a requirement for the > AM to return data in index order (unless amrescan's orderbys is not > null), or just a comment on the reason for the name of 'amcanorder' > being unclear? It

Re: [HACKERS] pg_upgrade vs vacuum_cost_delay

2023-11-24 Thread Bruce Momjian
On Fri, Nov 24, 2023 at 06:20:28PM +0100, Magnus Hagander wrote: > On Fri, Nov 24, 2023 at 5:34 PM Bruce Momjian wrote: > > On Fri, Nov 24, 2023 at 01:10:01PM +0100, Michael Banck wrote: > > > You're right, I somehow only saw your mail after I had already sent > > > mine. > > > > > > To make up

Re: [HACKERS] pg_upgrade vs vacuum_cost_delay

2023-11-24 Thread Magnus Hagander
On Fri, Nov 24, 2023 at 5:34 PM Bruce Momjian wrote: > > On Fri, Nov 24, 2023 at 01:10:01PM +0100, Michael Banck wrote: > > Hi, > > > > On Fri, Nov 24, 2023 at 12:17:56PM +0100, Magnus Hagander wrote: > > > On Fri, Nov 24, 2023 at 11:21 AM Michael Banck wrote: > > > > On Wed, Nov 22, 2023 at

Re: Questions regarding Index AMs and natural ordering

2023-11-24 Thread Matthias van de Meent
On Thu, 23 Nov 2023 at 19:52, Peter Geoghegan wrote: > > On Thu, Nov 23, 2023 at 9:16 AM Matthias van de Meent > wrote: > > For example, btree ignores any ordering scan keys that it is given in > > btrescan, which seems fine for btree because the ordering of a btree > > is static (and no other

Re: [HACKERS] pg_upgrade vs vacuum_cost_delay

2023-11-24 Thread Bruce Momjian
On Fri, Nov 24, 2023 at 01:10:01PM +0100, Michael Banck wrote: > Hi, > > On Fri, Nov 24, 2023 at 12:17:56PM +0100, Magnus Hagander wrote: > > On Fri, Nov 24, 2023 at 11:21 AM Michael Banck wrote: > > > On Wed, Nov 22, 2023 at 11:23:34PM -0500, Bruce Momjian wrote: > > > > + Non-zero values

Re: pg_upgrade and logical replication

2023-11-24 Thread vignesh C
On Fri, 24 Nov 2023 at 07:00, Peter Smith wrote: > > I have only trivial review comments for patch v18-0001 > > == > src/bin/pg_upgrade/check.c > > 1. check_new_cluster_subscription_configuration > > + /* > + * A slot not created yet refers to the 'i' (initialize) state, while > + * 'r'

Re: [HACKERS] psql casts aspersions on server reliability

2023-11-24 Thread Bruce Momjian
On Fri, Nov 24, 2023 at 04:06:22AM +0100, Laurenz Albe wrote: > On Thu, 2023-11-23 at 11:12 -0500, Bruce Momjian wrote: > > On Wed, Nov 22, 2023 at 10:25:14PM -0500, Bruce Momjian wrote: > > > Yes, you are correct. Here is a patch that implements the FATAL test, > > > though I am not sure I have

Re: Table AM Interface Enhancements

2023-11-24 Thread Mark Dilger
> On Nov 23, 2023, at 4:42 AM, Alexander Korotkov wrote: > 0006-Generalize-table-AM-API-for-INSERT-.-ON-CONFLICT-v1.patch > > Provides a new table AM API method to encapsulate the whole INSERT ... > ON CONFLICT ... algorithm rather than just implementation of > speculative tokens. I

Re: Stack overflow issue

2023-11-24 Thread Heikki Linnakangas
On 21/06/2023 16:45, Egor Chindyaskin wrote: Hello! In continuation of the topic I would like to suggest solution. This patch adds several checks to the vulnerable functions above. I looked at this last patch. The depth checks are clearly better than segfaulting, but I think we can also avoid

Re: [PATCH] pg_convert improvement

2023-11-24 Thread Yurii Rashkovskii
Hi Bertrand, On Fri, Nov 24, 2023 at 6:26 AM Drouvot, Bertrand < bertranddrouvot...@gmail.com> wrote: > > The patch is pretty straightforward, I just have one remark: > > + /* if no actual conversion happened, return the original string */ > + /* (we are checking pointers to strings

Re: [PATCH] pg_convert improvement

2023-11-24 Thread Drouvot, Bertrand
Hi, On 11/24/23 3:05 PM, Yurii Rashkovskii wrote: Hi, I propose a patch that ensures `pg_convert` doesn't allocate and copy data when no conversion is done. It is an unnecessary overhead, especially when such conversions are done frequently and for large values. +1 for the patch, I think

Re: Adding facility for injection points (or probe points?) for more advanced tests

2023-11-24 Thread Michael Paquier
On Fri, Nov 24, 2023 at 04:37:58PM +0530, Ashutosh Bapat wrote: > Interesting idea. For that the callback needs to know the injection > point name. At least we should pass that to the callback. It's trivial > thing to do. This is what's done from the beginning, as well as of 0001 in the v5

[PATCH] pg_convert improvement

2023-11-24 Thread Yurii Rashkovskii
Hi, I propose a patch that ensures `pg_convert` doesn't allocate and copy data when no conversion is done. It is an unnecessary overhead, especially when such conversions are done frequently and for large values. I've tried measuring the performance impact, and the patched version has a small

Re: GUC names in messages

2023-11-24 Thread Michael Paquier
On Fri, Nov 24, 2023 at 10:53:40AM +0100, Alvaro Herrera wrote: > I think we could leave these improvements for a second round. They > don't need to hold back the improvement we already have. Of course, no problem here to do things one step at a time. -- Michael signature.asc Description: PGP

Re: Assert failure on 'list_member_ptr(rel->joininfo, restrictinfo)'

2023-11-24 Thread Alexander Korotkov
On Thu, Nov 23, 2023 at 4:33 AM Richard Guo wrote: > > On Sun, Nov 19, 2023 at 9:17 AM Alexander Korotkov > wrote: >> >> It's here. New REALLOCATE_BITMAPSETS forces bitmapset reallocation on >> each modification. > > > +1 to the idea of introducing a reallocation mode to Bitmapset. > >> >> I

Re: pgoutput incorrectly replaces missing values with NULL since PostgreSQL 15

2023-11-24 Thread Nikhil Benesch
Thank you both for reviewing. The updated patch set LGTM. Nikhil On Fri, Nov 24, 2023 at 7:21 AM Zhijie Hou (Fujitsu) wrote: > > On Friday, November 24, 2023 7:47 PM Amit Kapila > wrote: > > > > On Thu, Nov 23, 2023 at 2:33 PM Amit Kapila > > wrote: > > > > > > On Thu, Nov 23, 2023 at 1:10 

Re: Improve tab completion for ALTER DEFAULT PRIVILEGE and ALTER TABLE

2023-11-24 Thread Shubham Khanna
n Fri, Nov 24, 2023 at 6:33 PM vignesh C wrote: > > Hi, > > Improved tab completion for "ALTER DEFAULT PRIVILEGE" and "ALTER TABLE": > 1) GRANT, REVOKE and FOR USER keyword was not displayed in tab > completion of alter default privileges like the below statement: > ALTER DEFAULT PRIVILEGES GRANT

Re: How to accurately determine when a relation should use local buffers?

2023-11-24 Thread Aleksander Alekseev
Hi, > There are some changes in ReadBuffer logic if to compare with pg15. To define > which buffers to use, ReadBuffer used SmgrIsTemp function in pg15. The > decision was based on backend id of the relation. In pg16 the decision is > based on relpersistence attribute, that caused some

Re: remaining sql/json patches

2023-11-24 Thread Alvaro Herrera
Some quick grepping gave me this table, YYLAST YYNTOKENS YYNNTS YYNRULES YYNSTATES YYMAXUTOK REL9_1_STABLE 69680429 546 22184179666 REL9_2_STABLE 73834432 546 22614301669 REL9_3_STABLE 77969

RE: pgoutput incorrectly replaces missing values with NULL since PostgreSQL 15

2023-11-24 Thread Zhijie Hou (Fujitsu)
On Friday, November 24, 2023 7:47 PM Amit Kapila wrote: > > On Thu, Nov 23, 2023 at 2:33 PM Amit Kapila > wrote: > > > > On Thu, Nov 23, 2023 at 1:10 PM Nikhil Benesch > wrote: > > > > > > While working on Materialize's streaming logical replication from > > > Postgres [0], my colleagues Sean

Re: [HACKERS] pg_upgrade vs vacuum_cost_delay

2023-11-24 Thread Michael Banck
Hi, On Fri, Nov 24, 2023 at 12:17:56PM +0100, Magnus Hagander wrote: > On Fri, Nov 24, 2023 at 11:21 AM Michael Banck wrote: > > On Wed, Nov 22, 2023 at 11:23:34PM -0500, Bruce Momjian wrote: > > > + Non-zero values of > > > + vacuum_cost_delay will delay statistics > > > generation. >

Re: pgoutput incorrectly replaces missing values with NULL since PostgreSQL 15

2023-11-24 Thread Amit Kapila
On Thu, Nov 23, 2023 at 2:33 PM Amit Kapila wrote: > > On Thu, Nov 23, 2023 at 1:10 PM Nikhil Benesch > wrote: > > > > While working on Materialize's streaming logical replication from Postgres > > [0], > > my colleagues Sean Loiselle and Petros Angelatos (CC'd) discovered today > > what > >

Re: Patch: Global Unique Index

2023-11-24 Thread Nikita Malakhov
Hi! Please advise on the status of this patch set - are there any improvements? Is there any work going on? Thanks! -- Regards, Nikita Malakhov Postgres Professional The Russian Postgres Company https://postgrespro.ru/

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-11-24 Thread Shlok Kyal
Hi, > I tried to reproduce the issue and was able to reproduce it with > scripts shared by Tomas. > I tried testing it from PG17 to PG 11. This issue is reproducible for > each version. > > Next I would try to test with the patch in the thread shared by Amit. I have created the v1 patch to

Re: [HACKERS] pg_upgrade vs vacuum_cost_delay

2023-11-24 Thread Magnus Hagander
On Fri, Nov 24, 2023 at 11:21 AM Michael Banck wrote: > > Hi, > > On Wed, Nov 22, 2023 at 11:23:34PM -0500, Bruce Momjian wrote: > > + Non-zero values of > > + vacuum_cost_delay will delay statistics generation. > > Now I wonder wheter vacuumdb maybe should have an option to explicitly >

Re: Adding facility for injection points (or probe points?) for more advanced tests

2023-11-24 Thread Ashutosh Bapat
On Fri, Nov 24, 2023 at 7:26 AM Michael Paquier wrote: > If you wish to use a combination of N points with a sleep callback and > different sleep times, one can just register a second shmem area in > the extension holding the callbacks that links the point names with > the sleep time to use. >

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2023-11-24 Thread Ashutosh Bapat
On Fri, Nov 24, 2023 at 3:56 PM Alena Rybakina wrote: > > On 24.11.2023 13:20, Alena Rybakina wrote: > > Hi! Thank you for your work on the subject, I think it's a really useful > feature. > > I've reviewed your patch and I have a few questions. > > First of all, have you thought about creating

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2023-11-24 Thread Alena Rybakina
On 24.11.2023 13:20, Alena Rybakina wrote: Hi! Thank you for your work on the subject, I think it's a really useful feature. I've reviewed your patch and I have a few questions. First of all, have you thought about creating a gun parameter to display memory scheduling information? I agree

Re: [HACKERS] pg_upgrade vs vacuum_cost_delay

2023-11-24 Thread Michael Banck
Hi, On Wed, Nov 22, 2023 at 11:23:34PM -0500, Bruce Momjian wrote: > + Non-zero values of > + vacuum_cost_delay will delay statistics generation. Now I wonder wheter vacuumdb maybe should have an option to explicitly force vacuum_cost_delay to 0 (I don't think it has?)? Michael

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2023-11-24 Thread Alena Rybakina
Hi! Thank you for your work on the subject, I think it's a really useful feature. I've reviewed your patch and I have a few questions. First of all, have you thought about creating a gun parameter to display memory scheduling information? I agree that this is an important feature, but I

Re: pipe_read_line for reading arbitrary strings

2023-11-24 Thread Daniel Gustafsson
> On 22 Nov 2023, at 13:47, Alvaro Herrera wrote: > > On 2023-Mar-07, Daniel Gustafsson wrote: > >> The attached POC diff replace fgets() with pg_get_line(), which may not be an >> Ok way to cross the streams (it's clearly not a great fit), but as a POC it >> provided a neater interface for

Re: GUC names in messages

2023-11-24 Thread Alvaro Herrera
On 2023-Nov-24, Michael Paquier wrote: > On Thu, Nov 23, 2023 at 06:27:04PM +1100, Peter Smith wrote: > > There may be some changes I've missed, but hopefully, this is a nudge > > in the right direction. > > Thanks for spending some time on that. +1 > > +In messages containing

Re: Synchronizing slots from primary to standby

2023-11-24 Thread Drouvot, Bertrand
Hi, On 11/24/23 10:45 AM, Amit Kapila wrote: On Fri, Nov 24, 2023 at 1:53 PM Drouvot, Bertrand wrote: On 11/24/23 4:35 AM, Zhijie Hou (Fujitsu) wrote: On Thursday, November 23, 2023 11:45 PM Drouvot, Bertrand wrote: IOW, the logical slot's LSN can still be advanced after the walreceiver

Re: Synchronizing slots from primary to standby

2023-11-24 Thread Amit Kapila
On Fri, Nov 24, 2023 at 1:53 PM Drouvot, Bertrand wrote: > > On 11/24/23 4:35 AM, Zhijie Hou (Fujitsu) wrote: > > On Thursday, November 23, 2023 11:45 PM Drouvot, Bertrand > > wrote: > > > > IOW, the logical slot's LSN can still be advanced after the > > walreceiver shutdown if it was far

Re: remaining sql/json patches

2023-11-24 Thread Amit Langote
On Thu, Nov 23, 2023 at 4:38 AM Andres Freund wrote: > On 2023-11-21 12:52:35 +0900, Amit Langote wrote: > > version gram.o text bytes %change gram.c bytes %change > > > > 9.6 534010 -2108984 - > > 10 582554 9.09 2258313 7.08

Re: Synchronizing slots from primary to standby

2023-11-24 Thread Drouvot, Bertrand
Hi, On 11/23/23 11:45 AM, Amit Kapila wrote: On Wed, Nov 22, 2023 at 10:02 AM Zhijie Hou (Fujitsu) wrote: Or we could just document that it is user's responsibility to match the failover property in case it changes the slot_name. Personally, I think we should document this behavior

Re: remaining sql/json patches

2023-11-24 Thread jian he
hi. + /* + * Set information for RETURNING type's input function used by + * ExecEvalJsonExprCoercion(). + */ "ExecEvalJsonExprCoercion" comment is wrong? + /* + * Step to jump to the EEOP_JSONEXPR_FINISH step skipping over item + * coercion steps that will be added below, if any. + */

Re: Synchronizing slots from primary to standby

2023-11-24 Thread Drouvot, Bertrand
Hi, On 11/24/23 4:35 AM, Zhijie Hou (Fujitsu) wrote: On Thursday, November 23, 2023 11:45 PM Drouvot, Bertrand wrote: IOW, the logical slot's LSN can still be advanced after the walreceiver shutdown if it was far bebind the physical slot's LSN. oh yeah right, it would need much more

Re: Random pg_upgrade test failure on drongo

2023-11-24 Thread Alexander Lakhin
Hello Kuroda-san, 23.11.2023 15:15, Hayato Kuroda (Fujitsu) wrote: I agree with your analysis and would like to propose a PoC fix (see attached). With this patch applied, 20 iterations succeeded for me. Thanks, here are comments. I'm quite not sure for the windows, so I may say something