Re: Automatically sizing the IO worker pool

2025-05-27 Thread Dmitry Dolgov
On Mon, May 26, 2025, 8:01 AM Thomas Munro wrote: > But ... I'm not even sure if we can say that our > I/O arrivals have a Poisson distribution, since they are not all > independent. > Yeah, a good point, one have to be careful with assumptions about distribution -- from what I've read many proc

Re: Automatically sizing the IO worker pool

2025-05-24 Thread Dmitry Dolgov
> On Sun, Apr 13, 2025 at 04:59:54AM GMT, Thomas Munro wrote: > It's hard to know how to set io_workers=3. If it's too small, > io_method=worker's small submission queue overflows and it silently > falls back to synchronous IO. If it's too high, it generates a lot of > pointless wakeups and sched

Re: I/O worker and ConfigReload

2025-05-24 Thread Dmitry Dolgov
> On Sun, May 25, 2025 at 02:15:12AM GMT, Thomas Munro wrote: > On Sun, May 25, 2025 at 1:56 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > I've been rebasing the patch for online resizing of shared memory, and > > noticed something strange about IoWorkerMain: alt

I/O worker and ConfigReload

2025-05-24 Thread Dmitry Dolgov
Hi, I've been rebasing the patch for online resizing of shared memory, and noticed something strange about IoWorkerMain: although it sets the handler SignalHandlerForConfigReload, it doesn't look like it acts upon ConfigReloadPending. From what I see it happens because it only does CHECK_FOR_INTER

Re: queryId constant squashing does not support prepared statements

2025-05-23 Thread Dmitry Dolgov
> On Fri, May 23, 2025 at 09:05:54AM GMT, Sami Imseih wrote: > > > On Thu, May 22, 2025 at 10:23:31PM GMT, Sami Imseih wrote: > > > > This does not get squashed: > > > > Q: select where 2 in (1, 4) and > > > > 1 in (5, cast(7 as int), 6, (cast(8 as int)), 9, 10, (cast(8 as > > > > text))::int); >

Re: queryId constant squashing does not support prepared statements

2025-05-23 Thread Dmitry Dolgov
> On Thu, May 22, 2025 at 10:23:31PM GMT, Sami Imseih wrote: > > This does not get squashed: > > Q: select where 2 in (1, 4) and > > 1 in (5, cast(7 as int), 6, (cast(8 as int)), 9, 10, (cast(8 as > > text))::int); > > R: select where $1 in ($2 /*, ... */) and > > $3 in ($4, cast($5 as int), $6, (

Re: queryId constant squashing does not support prepared statements

2025-05-22 Thread Dmitry Dolgov
> On Wed, May 21, 2025 at 08:22:19PM GMT, Sami Imseih wrote: > > > > At the same time AFAICT there isn't much more code paths > > > > to worry about in case of a LocationExpr as a node > > > > > > I can imagine there are others like value expressions, > > > row expressions, json array expressions,

Re: queryId constant squashing does not support prepared statements

2025-05-21 Thread Dmitry Dolgov
> On Tue, May 20, 2025 at 04:50:12PM GMT, Sami Imseih wrote: > > At the same time AFAICT there isn't much more code paths > > to worry about in case of a LocationExpr as a node > > I can imagine there are others like value expressions, > row expressions, json array expressions, etc. that we may > w

Re: queryId constant squashing does not support prepared statements

2025-05-20 Thread Dmitry Dolgov
> On Tue, May 20, 2025 at 06:30:25AM GMT, Michael Paquier wrote: > On Mon, May 12, 2025 at 06:40:43PM -0400, Sami Imseih wrote: > > Also, LocationExpr is not really an expression node, but a wrapper to > > an expression node, so I think it's wrong to define it as a Node and be > > required to add t

Re: queryId constant squashing does not support prepared statements

2025-05-20 Thread Dmitry Dolgov
> On Tue, May 20, 2025 at 11:03:52AM GMT, Dmitry Dolgov wrote: > > By the way, the new test cases for ARRAY lists are sent in the last > > patch of the series posted on this thread: > > https://www.postgresql.org/message-id/7zbzwk4btnxoo4o3xbtzefoqvht54cszjj4bol22fm

Re: queryId constant squashing does not support prepared statements

2025-05-13 Thread Dmitry Dolgov
> On Mon, May 12, 2025 at 06:40:43PM GMT, Sami Imseih wrote: > > The static variables was only part of the concern, another part was > > using A_Expr to carry this information, which will have impact on lots > > of unrelated code. > > What would be the problem if A_Expr carries an extra pointer to

Re: queryId constant squashing does not support prepared statements

2025-05-12 Thread Dmitry Dolgov
> On Fri, May 09, 2025 at 12:47:19PM GMT, Sami Imseih wrote: > So, I think we can create a new parse node ( parsenode.h ) that will only be > used in parsing (and gram.c only ) to track the start/end locations > and List and > based on this node we can create A_ArrayExpr and A_Expr with the List >

Re: queryId constant squashing does not support prepared statements

2025-05-12 Thread Dmitry Dolgov
> On Fri, May 09, 2025 at 10:12:24AM GMT, Dmitry Dolgov wrote: > Agree, I'll try to extend number of test cases here as a separate patch. Here is the extended version, where start/end is replaced by location/length, array_expr is handled as well, and more ARRAY cases are a

Re: queryId constant squashing does not support prepared statements

2025-05-09 Thread Dmitry Dolgov
> On Fri, May 09, 2025 at 08:47:58AM GMT, Michael Paquier wrote: > SELECT query, calls FROM pg_stat_statements ORDER BY query COLLATE "C"; > - query| calls > -+--- > - SELECT ARRAY[$1 /*, ... */]

Re: queryId constant squashing does not support prepared statements

2025-05-09 Thread Dmitry Dolgov
> On Fri, May 09, 2025 at 02:35:33PM GMT, Michael Paquier wrote: > On Fri, May 09, 2025 at 11:05:43AM +0800, Junwang Zhao wrote: > > Why not a location and a length, it should be more natural, it > > seems we use this convention in some existing nodes, like > > RawStmt, InsertStmt etc. > > These ar

Re: queryId constant squashing does not support prepared statements

2025-05-08 Thread Dmitry Dolgov
> On Thu, May 08, 2025 at 02:22:00PM GMT, Michael Paquier wrote: > On Wed, May 07, 2025 at 10:41:22AM +0200, Dmitry Dolgov wrote: > > Ah, I see what you mean. I think the idea is fine, it will simplify > > certain things as well as address the issue. But I'm afraid adding &g

Re: queryId constant squashing does not support prepared statements

2025-05-07 Thread Dmitry Dolgov
> On Tue, May 06, 2025 at 03:01:32PM GMT, Sami Imseih wrote: > > > Without properly accounting for the boundaries of the list of > > > expressions, i.e., > > > the start and end positions of '(' and ')' or '[' and ']' and normalizing > > > the > > > expressions in between, it will be very difficu

Re: queryId constant squashing does not support prepared statements

2025-05-06 Thread Dmitry Dolgov
> On Tue, May 06, 2025 at 01:32:48PM GMT, Sami Imseih wrote: > > I also agree with Alvaro that this discussion doesn't justify a > > revert. If the pre-v18 behavior wasn't chiseled on stone tablets, > > the new behavior isn't either. We can improve it some more later. > > As I was looking further

Re: queryId constant squashing does not support prepared statements

2025-05-06 Thread Dmitry Dolgov
> On Tue, May 06, 2025 at 11:50:07PM GMT, Junwang Zhao wrote: > Would it make sense to rename `RecordConstLocation` to something like > `RecordExpressionLocation` instead? Yeah, naming is hard. RecordExpressionLocation is somehow more vague, but I see what you mean, maybe something along these lin

Re: Changing shared_buffers without restart

2025-05-06 Thread Dmitry Dolgov
> On Tue, May 06, 2025 at 04:23:07AM GMT, Jack Ng wrote: > Thanks Dmitry. Right, the coordination mechanism in v4-0006 works as expected > in various tests (sorry, I misunderstood some details initially). Great, thanks for checking. > I also want to report a couple of minor issues found during t

Re: queryId constant squashing does not support prepared statements

2025-05-04 Thread Dmitry Dolgov
> On Thu, May 01, 2025 at 09:55:47PM GMT, Dmitry Dolgov wrote: > > On Thu, May 01, 2025 at 09:29:13AM GMT, Michael Paquier wrote: > > > > I agree that the current solution we have in the tree feels incomplete > > because we are not taking into account the most common

Re: queryId constant squashing does not support prepared statements

2025-05-02 Thread Dmitry Dolgov
> On Fri, May 02, 2025 at 04:18:37PM GMT, Michael Paquier wrote: > On Fri, May 02, 2025 at 09:13:39AM +0200, Dmitry Dolgov wrote: > > Squashing constants was ment to be a first step towards doing the same > > for other types of queries (params, rte_values), reverting it to > &

Re: queryId constant squashing does not support prepared statements

2025-05-02 Thread Dmitry Dolgov
> On Fri, May 02, 2025 at 07:10:19AM GMT, Michael Paquier wrote: > > I am really leaning towards that we should revert this feature as the > > limitation we have now with parameters is a rather large one and I think > > we need to go back and address this issue. > > I am wondering if this would not

Re: queryId constant squashing does not support prepared statements

2025-05-01 Thread Dmitry Dolgov
> On Thu, May 01, 2025 at 09:29:13AM GMT, Michael Paquier wrote: > > I agree that the current solution we have in the tree feels incomplete > because we are not taking into account the most common cases that > users would care about. Now, allowing PARAM_EXTERN means that we > allow any expression

Re: Changing shared_buffers without restart

2025-04-21 Thread Dmitry Dolgov
> On Thu, Apr 17, 2025 at 07:05:36PM GMT, Ni Ku wrote: > I also have a related question about how ftruncate() is used in the patch. > In my testing I also see that when using ftruncate to shrink a shared > segment, the memory is freed immediately after the call, even if other > processes still have

Re: Changing shared_buffers without restart

2025-04-21 Thread Dmitry Dolgov
> On Fri, Apr 18, 2025 at 10:06:23AM GMT, Konstantin Knizhnik wrote: > The only drawback is that we are loosing content of shared buffers in case > of resize. It may be sadly, but not looks like there is no better > alternative. No, why would we loose the content? If we do mremap, it will leave th

Re: Changing shared_buffers without restart

2025-04-21 Thread Dmitry Dolgov
> On Fri, Apr 18, 2025 at 09:17:21PM GMT, Thomas Munro wrote: > I was imagining that you might map some maximum possible size at the > beginning to reserve the address space permanently, and then adjust > the virtual memory object's size with ftruncate as required to provide > backing. Doesn't tha

Re: Changing shared_buffers without restart

2025-04-18 Thread Dmitry Dolgov
> On Thu, Apr 17, 2025 at 02:21:07PM GMT, Konstantin Knizhnik wrote: > > 1. Performance of Postgres CLOCK page eviction algorithm depends on number > of shared buffers. My first native attempt just to mark unused buffers as > invalid cause significant degrade of performance Thanks for sharing! Ri

Re: Changing shared_buffers without restart

2025-04-18 Thread Dmitry Dolgov
> On Thu, Apr 17, 2025 at 03:22:28PM GMT, Ashutosh Bapat wrote: > > In an offlist chat Thomas Munro mentioned that just ftruncate() would > be enough to resize the shared memory without touching address maps > using mmap and munmap(). > > ftruncate man page seems to concur with him > >If th

Re: Changing shared_buffers without restart

2025-04-14 Thread Dmitry Dolgov
> On Mon, Apr 14, 2025 at 10:40:28AM GMT, Ashutosh Bapat wrote: > > However, when we put back the patches to shrink buffers, we will evict > the extra buffers, and shrink - if all the processes haven't > participated in the barrier by then, some of them may try to access > those buffers - re-instal

Re: Proposal: Adding compression of temporary files

2025-04-13 Thread Dmitry Dolgov
> On Fri, Mar 28, 2025 at 09:23:13AM GMT, Filip Janus wrote: > > +else > > +if (nbytesOriginal - file->pos != 0) > > +/* curOffset must be corrected also if compression is > > + * enabled, nbytes was changed by compression but we > > + * have to use t

Re: Changing shared_buffers without restart

2025-04-11 Thread Dmitry Dolgov
> On Fri, Apr 11, 2025 at 08:04:39PM GMT, Ashutosh Bapat wrote: > On Mon, Apr 7, 2025 at 2:13 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > > Yes, you're right, plain dynamic Barrier does not ensure all available > > processes will be synchronized.

Re: Changing shared_buffers without restart

2025-04-09 Thread Dmitry Dolgov
> On Wed, Apr 09, 2025 at 01:20:16PM GMT, Ashutosh Bapat wrote: > ../../coderoot/pg/src/include/storage/s_lock.h:93:2: error: #error > "s_lock.h may not be included from frontend code" > > How about this? Why is that happening? The same -- as you can see it comes from compiling pg_numa.c, which as

Re: Changing shared_buffers without restart

2025-04-09 Thread Dmitry Dolgov
> On Wed, Apr 09, 2025 at 11:12:18AM GMT, Ashutosh Bapat wrote: > On Mon, Apr 7, 2025 at 2:13 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > > In the new v4 version > > of the patch the first option is implemented. > > > > The patches don't a

Re: Changing shared_buffers without restart

2025-04-05 Thread Dmitry Dolgov
> On Thu, Mar 20, 2025 at 04:55:47PM GMT, Ni Ku wrote: > > I ran some simple tests (outside of PG) on linux kernel v6.1, which has > this commit that added some hugepage support to mremap ( > https://patchwork.kernel.org/project/linux-mm/patch/20211013195825.3058275-1-almasrym...@google.com/ > ). >

Re: System views for versions reporting

2025-04-01 Thread Dmitry Dolgov
> On Sun, Mar 23, 2025 at 06:21:33PM GMT, Tom Lane wrote: > > FWIW, I think the 0004 patch is about to be mostly obsoleted by > Andrei's proposal at [1]. To the extent that it's not obsoleted, > I question whether it's something we want at all, given the ground > rule that unprivileged users are n

Re: Squash constant lists in query jumbling by default

2025-03-25 Thread Dmitry Dolgov
On Tue, Mar 25, 2025, 7:40 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > >>> On Tue, Mar 25, 2025, 6:28 PM Álvaro Herrera > wrote: > > On 2025-Mar-25, Tom Lane wrote: > > > If this GUC sticks around, it should be at least PGC_SUSET (on > > > the a

Re: Squash constant lists in query jumbling by default

2025-03-25 Thread Dmitry Dolgov
> > >> On Tue, Mar 25, 2025, 6:28 PM Álvaro Herrera wrote: On 2025-Mar-25, Tom Lane wrote: > If this GUC sticks around, it should be at least PGC_SUSET (on > the analogy of compute_query_id) to make it harder to break > pg_stat_statements that way. I have no problem making it superuser-only,

Re: Changing shared_buffers without restart

2025-03-21 Thread Dmitry Dolgov
> On Fri, Mar 21, 2025 at 04:48:30PM GMT, Ni Ku wrote: > Thanks for your insights and confirmation, Dmitry. > Right, I think the anonymous fd approach would work to keep the memory > contents intact in between munmap and mmap with the new size, so bufferpool > expansion would work. > But it seems s

Re: pg_stat_statements and "IN" conditions

2025-03-19 Thread Dmitry Dolgov
> On Tue, Mar 18, 2025 at 02:54:18PM GMT, Sami Imseih wrote: > I want to mention that the current patch does not deal > with external parameters ( prepared statements ) [0] [1]. This could be a > follow-up, as it may need some further discussion. It is important to > address this case, IMO. Sure,

Re: pg_stat_statements and "IN" conditions

2025-03-19 Thread Dmitry Dolgov
> On Tue, Mar 18, 2025 at 07:33:25PM GMT, Álvaro Herrera wrote: > > By the way, I'm still open to adding the 'powers' mechanism that was > discussed earlier and other simple additions on top of what's there now, > if you have some spare energy to spend on this. (For full disclosure: > the "powers"

Re: pg_stat_statements and "IN" conditions

2025-03-18 Thread Dmitry Dolgov
> On Mon, Mar 17, 2025 at 08:14:16PM GMT, Álvaro Herrera wrote: > On 2025-Mar-17, Álvaro Herrera wrote: > > > You can see my patch on top of yours here: > > https://github.com/alvherre/postgres/commits/query_id_squash_values/ > > and the CI run here: > > https://cirrus-ci.com/build/5660053472018432

Re: pg_stat_statements and "IN" conditions

2025-03-17 Thread Dmitry Dolgov
> On Mon, Mar 17, 2025 at 12:07:44PM GMT, vignesh C wrote: > > I noticed that the feedback from Sami at [1] has not yet been > addressed, I have changed the status to Waiting on Author, kindly > address them and update the status to Needs review. > [1] - > https://www.postgresql.org/message-id/CAA

Re: pg_stat_statements and "IN" conditions

2025-03-03 Thread Dmitry Dolgov
> On Mon, Mar 03, 2025 at 12:56:24PM GMT, Álvaro Herrera wrote: > In the meantime, here's v28 which is Dmitry's v27 plus pgindent. No > other changes. Dmitry, were you planning to submit a new version? Nope, there was nothing I wanted to add so far.

Re: Changing shared_buffers without restart

2025-02-27 Thread Dmitry Dolgov
> On Tue, Feb 25, 2025 at 10:52:05AM GMT, Dmitry Dolgov wrote: > > On Fri, Oct 18, 2024 at 09:21:19PM GMT, Dmitry Dolgov wrote: > > TL;DR A PoC for changing shared_buffers without PostgreSQL restart, via > > changing shared memory mapping layout. Any feedback is appreciated.

Re: Changing shared_buffers without restart

2025-02-25 Thread Dmitry Dolgov
> On Fri, Oct 18, 2024 at 09:21:19PM GMT, Dmitry Dolgov wrote: > TL;DR A PoC for changing shared_buffers without PostgreSQL restart, via > changing shared memory mapping layout. Any feedback is appreciated. Hi, Here is a new version of the patch, which contains a proposal about how to c

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2025-02-21 Thread Dmitry Dolgov
> On Mon, Nov 25, 2024 at 11:20:05AM GMT, Ashutosh Bapat wrote: > > I've tried to reproduce some subset of those results, in case if I would > > be able to notice anything useful. Strangely enough, I wasn't able to > > get much boost in planning time e.g. with 4 first patches, 100 > > partitions an

Re: pg_stat_statements and "IN" conditions

2025-02-18 Thread Dmitry Dolgov
> On Mon, Feb 17, 2025 at 01:50:00PM GMT, Sami Imseih wrote: > > This test was to catch a crash that was happening in older version of > > the patch, so it doesn't have to verify the actual pgss entry. > > It seems odd to keep this test because of crash behavior experienced > in a previous version

Re: pg_stat_statements and "IN" conditions

2025-02-17 Thread Dmitry Dolgov
> On Mon, Feb 17, 2025 at 09:51:32AM GMT, Sami Imseih wrote: > > This should do it. The last patch for today, > > I looked at v27 today and have a few comments. > > 1/ It looks like the CTE test is missing a check for results. This test was to catch a crash that was happening in older version of t

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Dmitry Dolgov
> On Fri, Feb 14, 2025 at 09:06:24AM GMT, Sami Imseih wrote: > I think it will be sad to not include this very common case from > the start, because it is going to be one of the most common > cases. > > Wouldn't doing something like this inside IsMergeableConst > """ > if (!IsA(arg, Const) && !IsA(

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Dmitry Dolgov
> On Fri, Feb 14, 2025 at 11:12:25PM GMT, Julien Rouhaud wrote: > > > There seems to be an off-by-1 error in parameter numbering when merging > > > them. > > > > There are indeed three constants, but the second is not visible in the > > query text. Maybe makes sense to adjust the number in this ca

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Dmitry Dolgov
> On Fri, Feb 14, 2025 at 10:39:45PM GMT, Julien Rouhaud wrote: > There seems to be an off-by-1 error in parameter numbering when merging them. There are indeed three constants, but the second is not visible in the query text. Maybe makes sense to adjust the number in this case, let me try. > Not

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Dmitry Dolgov
> On Fri, Feb 14, 2025 at 05:26:19AM GMT, Sami Imseih wrote: > > > On Thu, Feb 13, 2025 at 05:08:45PM GMT, Sami Imseih wrote: > > > Constants passed as parameters to a prepared statement will not be > > > handled as expected. I did not not test explicit PREPARE/EXECUTE > > > statement, > > > but I

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Dmitry Dolgov
> On Fri, Feb 14, 2025 at 05:57:01PM GMT, Julien Rouhaud wrote: > On Fri, Feb 14, 2025 at 10:36:48AM +0100, Álvaro Herrera wrote: > > On 2025-Feb-14, Julien Rouhaud wrote: > > > > > Since the merging is a yes/no option (I think there used to be some > > > discussions > > > about having a threshold

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Dmitry Dolgov
> On Thu, Feb 13, 2025 at 05:08:45PM GMT, Sami Imseih wrote: > Constants passed as parameters to a prepared statement will not be > handled as expected. I did not not test explicit PREPARE/EXECUTE statement, > but I assume it will have the same issue. This is the same question of supporting variou

Re: pg_stat_statements and "IN" conditions

2025-02-13 Thread Dmitry Dolgov
> On Thu, Feb 13, 2025 at 01:47:01PM GMT, Álvaro Herrera wrote: > Also, how wed are you to > "query_id_merge_values" as a name? It's not in any way obvious that > this is about values in arrays. How about query_id_squash_arrays? Or > are you thinking in having values in other types of structures

Re: pg_stat_statements and "IN" conditions

2025-02-13 Thread Dmitry Dolgov
> On Wed, Feb 12, 2025 at 08:48:03PM GMT, Dmitry Dolgov wrote: > > On Wed, Feb 12, 2025 at 07:39:39PM GMT, Álvaro Herrera wrote: > > The nastiness level of this seems quite low, compared to what happens to > > this other example if we didn't handle these easy cases: >

Re: pg_stat_statements and "IN" conditions

2025-02-12 Thread Dmitry Dolgov
> On Wed, Feb 12, 2025 at 07:39:39PM GMT, Álvaro Herrera wrote: > The nastiness level of this seems quite low, compared to what happens to > this other example if we didn't handle these easy cases: > > create table t (a float); > select i from t where i in (1, 2); > select i from t where i in (1, '

Re: pg_stat_statements and "IN" conditions

2025-02-12 Thread Dmitry Dolgov
> On Tue, Feb 11, 2025 at 08:00:27PM GMT, Dmitry Dolgov wrote: > > Hmm, what about doing something much simpler, such as testing whether > > there's just a CoerceViaIO/RelabelType around a Const or a one-parameter > > function call of an immutable boostrap-OID fu

Re: pg_stat_statements and "IN" conditions

2025-02-11 Thread Dmitry Dolgov
> On Tue, Feb 11, 2025 at 07:51:43PM GMT, Álvaro Herrera wrote: > On 2025-Feb-11, Dmitry Dolgov wrote: > > > > On Tue, Feb 11, 2025 at 10:49:59AM GMT, Sami Imseih wrote: > > > I have only looked at 0001, but I am wondering why > > > query_id_const_merge is a

Re: pg_stat_statements and "IN" conditions

2025-02-11 Thread Dmitry Dolgov
> On Tue, Feb 11, 2025 at 10:49:59AM GMT, Sami Imseih wrote: > I have only looked at 0001, but I am wondering why > query_id_const_merge is a pg_stat_statements GUC > rather than a core GUC? It was moved from being a core GUC into a pg_stat_statements GUC on the request from the reviewers. Communi

Re: System views for versions reporting

2025-01-25 Thread Dmitry Dolgov
> On Thu, Jan 02, 2025 at 10:36:48AM GMT, jian he wrote: > hi. > https://cirrus-ci.com/github/postgresql-cfbot/postgresql/cf%2F5318 > shows lots of failures, but it doesn't seem to tell you about doc build > failure. Thanks for checking this out. Here is the updated version with applied changes.

Re: Fwd: Re: proposal: schema variables

2025-01-17 Thread Dmitry Dolgov
> On Fri, Jan 17, 2025 at 11:01:41AM GMT, Bruce Momjian wrote: > On Fri, Jan 17, 2025 at 04:55:07PM +0100, Pavel Stehule wrote: > > pá 17. 1. 2025 v 16:35 odesílatel Bruce Momjian napsal: > > > > So this feature would be like global GUC variables, with permission > > control? > > > > + typ

Re: pg_stat_statements and "IN" conditions

2024-12-03 Thread Dmitry Dolgov
> On Thu, Nov 28, 2024 at 08:36:47PM GMT, Kirill Reshke wrote: > > Hi! Can you please send a rebased version of this? Sure, here it is. >From 2de1af6489d46449b2884a9194515cd1090d5e8c Mon Sep 17 00:00:00 2001 From: Dmitrii Dolgov <9erthali...@gmail.com> Date: Tue, 3 Dec 2024 14:55:45 +0100 Subject:

Re: Changing shared_buffers without restart

2024-12-02 Thread Dmitry Dolgov
> On Fri, Nov 29, 2024 at 05:47:27PM GMT, Dmitry Dolgov wrote: > > On Fri, Nov 29, 2024 at 01:56:30AM GMT, Matthias van de Meent wrote: > > > > I mean, we can do the following to get a nice contiguous empty address > > space no other mmap(NULL)s will get put into: > &

Re: Consider the number of columns in the sort cost model

2024-12-02 Thread Dmitry Dolgov
Hi folks, Just wanted to mention, that looks like some CFBot test are failing, something around level_tracking in pgss.

Re: Partition-wise join with whole row vars

2024-12-02 Thread Dmitry Dolgov
> On Tue, Oct 08, 2024 at 09:24:15AM GMT, Alexander Pyhalov wrote: > > Attaching rebased patches. Just to let you know, looks like CFBot tests are red again, but this time there are some unexpected differences in some test query plan.

Re: CRC32C Parallel Computation Optimization on ARM

2024-12-01 Thread Dmitry Dolgov
> On Mon, Dec 04, 2023 at 10:18:09PM -0600, Nathan Bossart wrote: > > Thanks for the new patch. I am hoping to spend much more time on this in > the near future... Hi, The patch looks interesting, having around 8% improvement on that sounds attractive. Nathan, do you plan to come back to it and

Re: Changing shared_buffers without restart

2024-11-29 Thread Dmitry Dolgov
> On Fri, Nov 29, 2024 at 01:56:30AM GMT, Matthias van de Meent wrote: > > I mean, we can do the following to get a nice contiguous empty address > space no other mmap(NULL)s will get put into: > > /* reserve size bytes of memory */ > base = mmap(NULL, size, PROT_NONE, ...flags, ...); >

Re: Changing shared_buffers without restart

2024-11-28 Thread Dmitry Dolgov
> On Thu, Nov 28, 2024 at 12:18:54PM GMT, Robert Haas wrote: > > All that having been said, what does concern me a bit is our ability > to predict what Linux will do well enough to keep what we're doing > safe; and also whether the Linux behavior might abruptly change in the > future. Users would b

Re: Changing shared_buffers without restart

2024-11-28 Thread Dmitry Dolgov
> On Wed, Nov 27, 2024 at 04:05:47PM GMT, Robert Haas wrote: > On Wed, Nov 27, 2024 at 3:48 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > My understanding is that clashing of mappings (either at creation time > > or when resizing) could happen only withing the proces

Re: Changing shared_buffers without restart

2024-11-27 Thread Dmitry Dolgov
> On Wed, Nov 27, 2024 at 10:20:27AM GMT, Robert Haas wrote: > > > > > > code, but I'm not sure exactly which points are safe. If we have no > > > code anywhere that assumes the address of an unpinned buffer can't > > > change before we pin it, then I guess the check for pins is the only > > > thin

Re: [PoC] Reducing planning time when tables have many partitions

2024-11-27 Thread Dmitry Dolgov
> On Tue, Oct 15, 2024 at 12:20:04PM GMT, Yuya Watari wrote: > > The previous patches do not apply to the current master, so I have > attached the rebased version. Thanks for keeping it up to date. > v25-0001 > This patch is one of the main parts of my optimization. Traditionally, > EquivalenceCl

Re: Changing shared_buffers without restart

2024-11-26 Thread Dmitry Dolgov
> On Mon, Nov 25, 2024 at 02:33:48PM GMT, Robert Haas wrote: > > I think the idea of having multiple shared memory segments is > interesting and makes sense, but I would prefer to see them called > "segments" rather than "slots" just as do we do for DSMs. The name > "slot" is somewhat overused, and

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2024-11-24 Thread Dmitry Dolgov
> On Thu, Oct 10, 2024 at 05:36:10PM GMT, Ashutosh Bapat wrote: > > 3. With just patch 0001 applied, planning time usually shows > degradation (column Q and X in planning time sheets) with or without > PWJ enabled. I first thought that it might be because of the increased > size of PlannerInfo. We

Re: Extract numeric filed in JSONB more effectively

2024-11-22 Thread Dmitry Dolgov
> On Mon, Nov 18, 2024 at 08:23:52AM GMT, Andy Fan wrote: > > >> > I imagined you'd the patch should create a SupportRequestSimplify > >> > support function for jsonb_numeric() that checks if the input > >> > expression is an OpExpr with funcid of jsonb_object_field(). All you > >> > do then is di

Re: An inefficient query caused by unnecessary PlaceHolderVar

2024-11-21 Thread Dmitry Dolgov
> On Fri, Jun 21, 2024 at 10:35:30AM GMT, Richard Guo wrote: > On Mon, Jan 15, 2024 at 1:50 PM Richard Guo wrote: > > Updated this patch over 29f114b6ff, which indicates that we should apply > > the same rules for PHVs. > > Here is a new rebase of this patch, with some tweaks to comments. I've >

Re: ALTER TABLE uses a bistate but not for toast tables

2024-11-20 Thread Dmitry Dolgov
> On Wed, Nov 20, 2024 at 06:43:58AM -0600, Justin Pryzby wrote: > > > Thanks for rebasing. To help with review, could you also describe > > current status of the patch? I have to admit, currently the commit > > message doesn't tell much, and looks more like notes for the future you. > > The patch

Re: proposal: schema variables

2024-11-20 Thread Dmitry Dolgov
> On Tue, Nov 19, 2024 at 08:14:01PM +0100, Pavel Stehule wrote: > Hi > > I wrote POC of VARIABLE(varname) syntax support Thanks, the results look good. I'm still opposed the idea of having a warning, and think it has to be an error -- but it's my subjective opinion. Lets see if there are more vot

Re: Changing shared_buffers without restart

2024-11-19 Thread Dmitry Dolgov
> On Tue, Nov 19, 2024 at 01:57:00PM GMT, Peter Eisentraut wrote: > On 18.10.24 21:21, Dmitry Dolgov wrote: > > v1-0001-Allow-to-use-multiple-shared-memory-mappings.patch > > > > Preparation, introduces the possibility to work with many shmem mappings. To > > make it

Re: ALTER TABLE uses a bistate but not for toast tables

2024-11-19 Thread Dmitry Dolgov
> On Mon, Jul 15, 2024 at 03:43:24PM GMT, Justin Pryzby wrote: > @cfbot: rebased Hey Justin, Thanks for rebasing. To help with review, could you also describe current status of the patch? I have to admit, currently the commit message doesn't tell much, and looks more like notes for the future you

Re: Improving btree performance through specializing by key shape, take 2

2024-11-16 Thread Dmitry Dolgov
> On Mon, Mar 04, 2024 at 09:39:37PM GMT, Matthias van de Meent wrote: > Performance > > > I haven't retested the results separately yet, but I assume the > performance results of [2] hold mostly true in comparing 0002 vs 0007. > I will do a performance (re-)evaluation of only this patch i

Re: proposal: schema variables

2024-11-16 Thread Dmitry Dolgov
> On Sat, Nov 16, 2024 at 07:10:31AM GMT, Pavel Stehule wrote: Sorry, got distracted. Let me try to answer step by step. > > As far as I recall, last time this topic was discussed in hackers, two > > options were proposed: the one with VARIABLE(name), what you mention > > here; and another one wi

Re: Extract numeric filed in JSONB more effectively

2024-11-15 Thread Dmitry Dolgov
> On Thu, Sep 12, 2024 at 03:03:18AM GMT, Andy Fan wrote: > > > I imagined you'd the patch should create a SupportRequestSimplify > > support function for jsonb_numeric() that checks if the input > > expression is an OpExpr with funcid of jsonb_object_field(). All you > > do then is ditch the cast

Re: btree: implement dynamic prefix truncation (was: Improving btree performance through specializing by key shape, take 2)

2024-11-13 Thread Dmitry Dolgov
> On Tue, Aug 13, 2024 at 02:39:10PM GMT, Peter Geoghegan wrote: > On Tue, Aug 6, 2024 at 5:42 PM Matthias van de Meent > wrote: > > Attached is version 16 now. > > I ran this with my old land registry benchmark, used for validating > the space utilization impact of nbtree deduplication (among oth

Re: proposal: schema variables

2024-11-13 Thread Dmitry Dolgov
> On Sun, Nov 10, 2024 at 06:51:40PM GMT, Pavel Stehule wrote: > ne 10. 11. 2024 v 17:19 odesílatel Pavel Stehule > napsal: > I thought a lot of time about better solutions for identifier collisions > and I really don't think so there is some consistent user friendly syntax. > Personally I think t

Re: proposal: schema variables

2024-11-10 Thread Dmitry Dolgov
> On Sun, Nov 10, 2024 at 05:19:09PM GMT, Pavel Stehule wrote: > ne 10. 11. 2024 v 16:24 odesílatel Dmitry Dolgov <9erthali...@gmail.com> > napsal: > > > Hi folks, > > > > Thanks for continuing this work. As a side note, I would like to mention > > how st

Re: proposal: schema variables

2024-11-10 Thread Dmitry Dolgov
Hi folks, Thanks for continuing this work. As a side note, I would like to mention how strange the situation in this CF item is. If I understand correctly, there are two hackers threads discussing the same patch, with recent patches posted in both of them. This is obviously confusing, e.g. the mai

Re: Changing shared_buffers without restart

2024-11-08 Thread Dmitry Dolgov
> On Wed, Nov 06, 2024 at 07:10:06PM GMT, Vladlen Popolitov wrote: > Hi > > I tried to apply patches, but failed. I suppose the problem with CRLF in the > end of lines in the patch files. At least, after manual change of v1-0001 and > v1-0002 from CRLF to LF patches applied, but it was not helped

Re: Changing shared_buffers without restart

2024-11-08 Thread Dmitry Dolgov
> On Thu, Nov 07, 2024 at 02:05:52PM GMT, Thomas Munro wrote: > On Sat, Oct 19, 2024 at 8:21 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > Currently it > > supports only an increase of shared_buffers. > > Just BTW in case it is interesting, Palak and I experim

Re: Identify huge pages accessibility using madvise

2024-11-08 Thread Dmitry Dolgov
> On Thu, Sep 26, 2024 at 08:46:17AM GMT, Dmitry Dolgov wrote: > > On Thu, Sep 26, 2024 at 07:57:12AM GMT, Gabriele Bartolini wrote: > > Hi Dmitry, > > > > I've been attempting to replicate this issue directly in Kubernetes, but I > > haven't been succes

Re: Changing shared_buffers without restart

2024-11-01 Thread Dmitry Dolgov
> On Fri, Oct 18, 2024 at 09:21:19PM GMT, Dmitry Dolgov wrote: > > TL;DR A PoC for changing shared_buffers without PostgreSQL restart, via > changing shared memory mapping layout. Any feedback is appreciated. It was pointed out to me, that earlier this year there was a useful disc

Re: System views for versions reporting

2024-10-19 Thread Dmitry Dolgov
> On Mon, Oct 07, 2024 at 11:26:41AM GMT, Dmitry Dolgov wrote: > > On Sun, Oct 06, 2024 at 12:01:29PM GMT, Joe Conway wrote: > > I'm not sure why ICU is "Compile Time" rather than "Run Time" when it is not > > statically linked. > > It

Changing shared_buffers without restart

2024-10-18 Thread Dmitry Dolgov
TL;DR A PoC for changing shared_buffers without PostgreSQL restart, via changing shared memory mapping layout. Any feedback is appreciated. Hi, Being able to change PostgreSQL configuration on the fly is an important property for performance tuning, since it reduces the feedback time and invasive

Re: System views for versions reporting

2024-10-07 Thread Dmitry Dolgov
> On Sun, Oct 06, 2024 at 12:01:29PM GMT, Joe Conway wrote: > On 10/6/24 11:36, Dmitry Dolgov wrote: > > Hi, > > > > Based on the feedback in [1], here is my attempt at implementing system > > views for versions reporting. It adds pg_system_versions for showin

System views for versions reporting

2024-10-06 Thread Dmitry Dolgov
Hi, Based on the feedback in [1], here is my attempt at implementing system views for versions reporting. It adds pg_system_versions for showing things like core version, compiler, LLVM, etc, and pg_system_libraries for showing linked shared objects. I think everyone has ageed that the first was a

Re: Identify huge pages accessibility using madvise

2024-09-25 Thread Dmitry Dolgov
> On Thu, Sep 26, 2024 at 07:57:12AM GMT, Gabriele Bartolini wrote: > Hi Dmitry, > > I've been attempting to replicate this issue directly in Kubernetes, but I > haven't been successful so far. I've been using EKS nodes, and it seems > that they all run cgroup v2 now. Do you have anything that coul

Re: Add llvm version into the version string

2024-09-24 Thread Dmitry Dolgov
> On Mon, Sep 23, 2024 at 02:45:18PM GMT, Tom Lane wrote: > Maybe another idea could be a new system view? > > => select * from pg_system_version; > property | value > > core version | 18.1 > architecture | x86_64-pc-linux-gnu > word

Re: Add llvm version into the version string

2024-09-23 Thread Dmitry Dolgov
> On Sun, Sep 22, 2024 at 01:15:54PM GMT, Dmitry Dolgov wrote: > > On Sat, Sep 21, 2024 at 05:25:30PM GMT, Tom Lane wrote: > > > > Is there a way to get the llvm library's version at run time? If so > > we could consider building a new function to return

Re: Add llvm version into the version string

2024-09-22 Thread Dmitry Dolgov
> On Sat, Sep 21, 2024 at 05:25:30PM GMT, Tom Lane wrote: > > Is there a way to get the llvm library's version at run time? If so > we could consider building a new function to return that. Yes, there is a C api LLVMGetVersion to get the major, minor and patch numbers. The jit provider could be e

  1   2   3   4   5   6   7   8   >