Re: Failures in constraints regression test, "read only 0 of 8192 bytes"

2024-03-08 Thread Ronan Dunklau
> root cause, or whether the commit maybe just made some pre-existing > issue easier to hit. Also, I haven't followed the discussion on the > pgsql-bugs thread [1], maybe there are some interesting findings. > If that happens only on HEAD and not on 16, and doesn't involve WAL replay, th

Re: Provide a pg_truncate_freespacemap function

2024-03-06 Thread Ronan Dunklau
function). Thank you for the review. Here is an updated patch for both of those. Best regards, -- Ronan>From 812061b47443ce1052f65ba2867223f9db2cfa18 Mon Sep 17 00:00:00 2001 From: Ronan Dunklau Date: Fri, 1 Mar 2024 08:57:49 +0100 Subject: [PATCH] Provide a pg_t

Provide a pg_truncate_freespacemap function

2024-03-04 Thread Ronan Dunklau
-- Ronan Dunklau >From b3e28e4542311094ee7177b39cc9cdf3672d1b55 Mon Sep 17 00:00:00 2001 From: Ronan Dunklau Date: Fri, 1 Mar 2024 08:57:49 +0100 Subject: [PATCH] Provide a pg_truncate_freespacemap function. Similar to the pg_truncate_visibilitymap function, this one allows to avoid downtime to

Re: Failures in constraints regression test, "read only 0 of 8192 bytes"

2024-03-04 Thread Ronan Dunklau
FSM corruption issue I'm facing: https://www.postgresql.org/message-id/ 1925490.taCxCBeP46%40aivenlaptop So if you still have the data, it would be nice to see if you indeed have a corrupted FSM, and if you have indications when it happened. Best regards, -- Ronan Dunklau

Re: scalability bottlenecks with (many) partitions (and more)

2024-01-29 Thread Ronan Dunklau
Le lundi 29 janvier 2024, 15:59:04 CET Tomas Vondra a écrit : > I'm not sure work_mem is a good parameter to drive this. It doesn't say > how much memory we expect the backend to use - it's a per-operation > limit, so it doesn't work particularly well with partitioning (e.g. with > 100 partitions,

Re: scalability bottlenecks with (many) partitions (and more)

2024-01-29 Thread Ronan Dunklau
Le lundi 29 janvier 2024, 13:17:07 CET Tomas Vondra a écrit : > > Did you try running an strace on the process ? That may give you some > > hindsights into what malloc is doing. A more sophisticated approach would > > be using stap and plugging it into the malloc probes, for example > >

Re: scalability bottlenecks with (many) partitions (and more)

2024-01-29 Thread Ronan Dunklau
the sbrk pointer up and down repeatedly. The automatic trade off between the mmap and trim thresholds is supposed to prevent that, but the way it is incremented means you can end in a bad place depending on your particular allocation patttern. Best regards, -- Ronan Dunklau

Re: LLVM 16 (opaque pointers)

2023-10-16 Thread Ronan Dunklau
Le vendredi 13 octobre 2023, 22:32:13 CEST Thomas Munro a écrit : > On Wed, Oct 11, 2023 at 10:31 PM Ronan Dunklau wrote: > > Le mercredi 11 octobre 2023, 10:59:50 CEST Thomas Munro a écrit : > > > The back-patch to 12 was a little trickier than anticipated, but after &g

Re: LLVM 16 (opaque pointers)

2023-10-11 Thread Ronan Dunklau
e anyone has comments on those. For PG13 and PG12, it looks like the ExecEvalBoolSubroutineTemplate is not used anywhere, as ExecEvalBoolSubroutine was introduced in PG14 if I'm not mistaken. Best regards, -- Ronan Dunklau

Re: LLVM 16 (opaque pointers)

2023-08-10 Thread Ronan Dunklau
out it made me indeed question my sanity. It's quite easy for PG 15, 14, 13. PG 12 is nothing insurmontable either, but PG 11 is a bit hairier most notably due to to the change in fcinfo args representation. But I guess that's also a topic for another day :-) Best regards, -- Ronan Dunklau

Re: Add GUC to tune glibc's malloc implementation.

2023-06-27 Thread Ronan Dunklau
if I understand your different suggestions, we should: - use mmap ourselves for what we deem to be "one-off" allocations, to make sure that memory is not hanging around after we don't use - keep some pool allocated which will not be freed in between queries, but reused for the next time we need it. Thank you for looking at this problem. Regards, -- Ronan Dunklau

Re: Add GUC to tune glibc's malloc implementation.

2023-06-27 Thread Ronan Dunklau
Le mardi 27 juin 2023, 08:35:28 CEST Ronan Dunklau a écrit : > I re-attached the simple script I used. I've run this script with different > values for glibc_malloc_max_trim_threshold. I forgot to add that it was using default parametrers except for work_mem, set

Re: Add GUC to tune glibc's malloc implementation.

2023-06-27 Thread Ronan Dunklau
Le lundi 26 juin 2023, 23:03:48 CEST Andres Freund a écrit : > Hi, > > On 2023-06-26 08:38:35 +0200, Ronan Dunklau wrote: > > I hope what I'm trying to achieve is clearer that way. Maybe this patch is > > not the best way to go about this, but since the memory allocator &

Re: Add GUC to tune glibc's malloc implementation.

2023-06-26 Thread Ronan Dunklau
Le vendredi 23 juin 2023, 22:55:51 CEST Peter Eisentraut a écrit : > On 22.06.23 15:35, Ronan Dunklau wrote: > > The thing is, by default, those parameters are adjusted dynamically by the > > glibc itself. It starts with quite small thresholds, and raises them when > > t

Re: Add GUC to tune glibc's malloc implementation.

2023-06-22 Thread Ronan Dunklau
_mem and adapt glibc behaviour accordingly, then get back to being conservative with memory until the next such transaction. Regards, -- Ronan Dunklau

Add GUC to tune glibc's malloc implementation.

2023-06-22 Thread Ronan Dunklau
value) Best regards, -- Ronan Dunklau >From 3686e660446facfb2d64683286176887914cd9fd Mon Sep 17 00:00:00 2001 From: Ronan Dunklau Date: Tue, 20 Jun 2023 16:17:32 +0200 Subject: [PATCH v1] Add options to tune malloc. Add a new GUC glibc_malloc_max_trim_threshold which is used only when being

Re: Use generation context to speed up tuplesorts

2023-06-19 Thread Ronan Dunklau
evaluate the possible benefits on the current ASet and Generation contexts usecases. I'll make sure to use a new thread to post my findings. Thank you for remembering ! Best regards, -- Ronan Dunklau

Re: Exclusion constraints on partitioned tables

2023-03-20 Thread Ronan Dunklau
d patch with that change (also rebased). Thanks ! This looks fine to me like this. > > I also included a more specific error message. If we find a matching > column in the index but with the wrong operator, we should say so, and > not say there is no matching column. > I agree tha

Re: Allow ordered partition scans in more cases

2023-02-22 Thread Ronan Dunklau
d ? It seems to underestimate the memory amount needed. I'm not sure it makes a real difference in real use cases though. Regards, -- Ronan Dunklau

SQLFunctionCache and generic plans

2023-02-07 Thread Ronan Dunklau
? Best regards, -- Ronan Dunklau

Re: Exclusion constraints on partitioned tables

2023-01-24 Thread Ronan Dunklau
Le vendredi 16 décembre 2022, 06:11:49 CET Paul Jungwirth a écrit : > On 12/15/22 16:12, Tom Lane wrote: > >> This patch also requires the matching constraint columns to use equality > >> comparisons (`(foo WITH =)`), so it is really equivalent to the existing > >> b-tree rule. > > > > That's not

Re: Ordering behavior for aggregates

2022-12-13 Thread Ronan Dunklau
ount for. I think the ability to mark certain aggregates as being able to completely ignore the ordering because they produce exactly the same results is still a useful optimization. -- Ronan Dunklau

Re: Ordering behavior for aggregates

2022-12-13 Thread Ronan Dunklau
If the ordering doesn't > make sense, we should forbid it. It is allowed as of now, and so it would be a compatibility issue for queries existing in the wild. Ignoring it is just an optimization, just how we optimize away some joins entirely. -- Ronan Dunklau

Re: Fix gin index cost estimation

2022-12-06 Thread Ronan Dunklau
Le vendredi 2 décembre 2022, 13:58:27 CET Ronan Dunklau a écrit : > Le vendredi 2 décembre 2022, 12:33:33 CET Alexander Korotkov a écrit : > > Hi, Ronan! > > Thank you for your patch. Couple of quick questions. > > 1) What magic number 50.0 stands for? I think we

Re: Fix gin index cost estimation

2022-12-02 Thread Ronan Dunklau
hould this > be dependent on number of search entries? Good point, multiplying it by the number of search entries would do the trick. Thank you for looking at this ! Regards, -- Ronan Dunklau

Re: Fix gin index cost estimation

2022-12-02 Thread Ronan Dunklau
f charging the quals to each tuple, charge them per entry only. We still charge cpu_index_tuple_cost per tuple though. With those changes, no need to tweak the magic number formula estimating the number of pages. Maybe we can come up with something better for estimating

Re: Asynchronous execution support for Custom Scan

2022-12-01 Thread Ronan Dunklau
> IIUC, we already can use ctid in the where clause on the latest > PostgreSQL, can't we? Oh, sorry, I missed the TidRangeScan. My apologies for the noise. Best regards, -- Ronan Dunklau

Re: Asynchronous execution support for Custom Scan

2022-11-22 Thread Ronan Dunklau
looking for null values. If you want to do this, it's much faster to update rows in a range of block, performing a first series of batch updating all such block ranges, and then finally update the ones we missed transactionally (inserted in a block we already processed while in the middle of the batch, or in new blocks resulting from a relation extension). Best regards, -- Ronan Dunklau

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2022-11-08 Thread Ronan Dunklau
g this overcharged just made us miss a new optimization without changing existing plans. With ordered aggregates, the balance is a bit trickier and we are at risk of either regressing on aggregate plans, or more common ordered ones. -- Ronan Dunklau

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2022-11-07 Thread Ronan Dunklau
remental sort was a bit too pessimistic. -- Ronan Dunklau

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2022-11-07 Thread Ronan Dunklau
But with this patch, we may actually replace a "hidden" incremental sort which was done in the agg codepath by a full sort. Best regards, -- Ronan Dunklau

Re: Fix gin index cost estimation

2022-10-12 Thread Ronan Dunklau
g about it I'll proceed with what we agree on. -- Ronan Dunklau

Re: A potential memory leak on Merge Join when Sort node is not below Materialize node

2022-09-29 Thread Ronan Dunklau
Le jeudi 29 septembre 2022, 16:10:03 CEST Tom Lane a écrit : > Ronan Dunklau writes: > >> Yeah, I think the same rules around scope apply as > >> tuplesort_gettupleslot() with copy==false. We could do it by adding a > >> copy flag to the existing fu

Re: A potential memory leak on Merge Join when Sort node is not below Materialize node

2022-09-29 Thread Ronan Dunklau
but it seems to then only be used by heapam_index_validate_scan and percentile_cont_multi_final_common. This is from my investigation looking for references to the specialized version in the DWARF debug information. Regards, -- Ronan Dunklau

Re: Fix gin index cost estimation

2022-09-19 Thread Ronan Dunklau
Le vendredi 16 septembre 2022, 22:04:59 CEST Tom Lane a écrit : > Ronan Dunklau writes: > > The attached does that and is much simpler. I only took into account > > entryPagesFetched, not sure if we should also charge something for data pages. > > I think this is wrong, bec

Re: Fix gin index cost estimation

2022-09-15 Thread Ronan Dunklau
Le lundi 12 septembre 2022, 16:41:16 CEST Ronan Dunklau a écrit : > But I realised that another approach might be better suited: since we want to > charge a cpu cost for every page visited, actually basing that on the already > estimated entryPagesFetched and dataPagesFetched would

Re: Fix gin index cost estimation

2022-09-12 Thread Ronan Dunklau
you. Either run the > comments into single paragraphs, or if you really want them to be two > paras then leave an empty comment line between. Another formatting > nitpick is that you seem to have added a number of unnecessary blank > lines. Thanks, noted. I'll submit a new patch soon, as soon as i've resolved some of the problems I have when accounting for scalararrayops. Best regards, -- Ronan Dunklau

Fix gin index cost estimation

2022-08-03 Thread Ronan Dunklau
#0c2498c6a85e31a589b3e9a6a3616c52 -- Ronan Dunklau>From 0aa1ff24e58234d759c07f4eeec163a82244be25 Mon Sep 17 00:00:00 2001 From: Ronan Dunklau Date: Wed, 6 Jul 2022 17:29:01 +0200 Subject: [PATCH v1] Fix gin costing. GIN index scans were not taking any descent CPU-based cost into account. That m

Re: Support for grabbing multiple consecutive values with nextval()

2022-07-13 Thread Ronan Dunklau
em and assigning them to the individual records before sending them via COPY FROM would solve that for them. -- Ronan Dunklau

Re: Support for grabbing multiple consecutive values with nextval()

2022-07-08 Thread Ronan Dunklau
the knowledge of the sequence details themselves, they can always wrap it in an aggregate: SELECT min(newvals), max(newvals) FROM nextvals() as newvals Regards, -- Ronan Dunklau

Re: Removing unneeded self joins

2022-07-04 Thread Ronan Dunklau
Le jeudi 30 juin 2022, 16:11:51 CEST Andrey Lepikhov a écrit : > On 19/5/2022 16:47, Ronan Dunklau wrote: > > I'll take a look at that one. > > New version of the patch, rebased on current master: > 1. pgindent over the patch have passed. > 2. number of changed files

Re: Removing unneeded self joins

2022-05-19 Thread Ronan Dunklau
Le jeudi 19 mai 2022, 12:48:18 CEST Andrey Lepikhov a écrit : > On 5/17/22 19:14, Ronan Dunklau wrote: > > Le vendredi 13 mai 2022, 07:07:47 CEST Andrey Lepikhov a écrit : > >> New version of the feature. > >> Here a minor bug with RowMarks is fixed. A degenerat

Re: Limiting memory allocation

2022-05-18 Thread Ronan Dunklau
have better knowledge of the memory allocations pattern than what malloc empirically discovers: in particular, we could lower work_mem, adjust the threshold and maybe even call malloc_trim ourselves when work_mem is lowered, to reduce the padding we may keep. > > So what do you propose PostgreSQL's response to high memory pressure to be? > > > Regards, Jan -- Ronan Dunklau

Re: Removing unneeded self joins

2022-05-17 Thread Ronan Dunklau
we don't have any additional base restrict info is also confusing, as well as the degenerate_case name. I'll update if I think of more interesting things to add. -- Ronan Dunklau

Re: ORDER BY pushdowns seem broken in postgres_fdw

2022-03-31 Thread Ronan Dunklau
le. No objections on my end. -- Ronan Dunklau

Re: Proposal: More structured logging

2022-01-27 Thread Ronan Dunklau
Le jeudi 27 janvier 2022, 08:15:01 CET Michael Paquier a écrit : > On Tue, Jan 18, 2022 at 06:46:03AM +0100, Ronan Dunklau wrote: > > Hum, there was a missing import in csvlog.c from the fix above. Sorry > > about > > that. > > + > > > You are also f

Re: Use generation context to speed up tuplesorts

2022-01-25 Thread Ronan Dunklau
Le jeudi 20 janvier 2022, 08:36:46 CET Ronan Dunklau a écrit : > Le jeudi 20 janvier 2022, 02:31:15 CET David Rowley a écrit : > > On Tue, 18 Jan 2022 at 19:45, Julien Rouhaud wrote: > > > I'm also a bit confused about which patch(es) should actually be > > > reviewe

Re: Use generation context to speed up tuplesorts

2022-01-19 Thread Ronan Dunklau
eration allocator itself from the side effect of avoiding glibc's pattern of releasing memory back to the kernel too quickly. Julien, sorry for the confusion of adding that discussion and those patches to the same thread, but I don't really see a way of dissociating those two aspects. -- Ronan Dunklau

Re: Proposal: More structured logging

2022-01-17 Thread Ronan Dunklau
Le lundi 17 janvier 2022, 09:18:04 CET Ronan Dunklau a écrit : > Le samedi 15 janvier 2022, 07:09:59 CET Julien Rouhaud a écrit : > > Hi, > > > > On Tue, Jan 11, 2022 at 11:05:26AM +0100, Ronan Dunklau wrote: > > > Done, and I added anoher commit per your

Re: Proposal: More structured logging

2022-01-17 Thread Ronan Dunklau
Le samedi 15 janvier 2022, 07:09:59 CET Julien Rouhaud a écrit : > Hi, > > On Tue, Jan 11, 2022 at 11:05:26AM +0100, Ronan Dunklau wrote: > > Done, and I added anoher commit per your suggestion to add this comment. > > The cfbot reports that the patchset doesn't app

Re: Proposal: More structured logging

2022-01-11 Thread Ronan Dunklau
ould also update file-fdw.sgml - which I only think of since we forgot > in to update it before e568ed0eb and 0830d21f5. config.sgml should have a > comment as a reminder to do that. Done, and I added anoher commit per your suggestion to add this comment. Thank you for this review.

Matching domains-over-enums to anyenum types

2022-01-10 Thread Ronan Dunklau
in doing the same for domains over enums. What do you think ? -- Ronan Dunklau

Re: Use generation context to speed up tuplesorts

2022-01-07 Thread Ronan Dunklau
Le vendredi 7 janvier 2022, 13:03:28 CET Tomas Vondra a écrit : > On 1/7/22 12:03, Ronan Dunklau wrote: > > Le vendredi 31 décembre 2021, 22:26:37 CET David Rowley a écrit : > >> I've attached some benchmark results that I took recently. The > >> spreadsheet conta

Re: Use generation context to speed up tuplesorts

2021-12-17 Thread Ronan Dunklau
t's allocated. Or do you want a benchmark with an even bigger initial block size ? With the growing blocks patch with a large initial size ? I can run either, I just want to understand what is interesting to you. One thing that would be interesting would be to trace the total amount of memory allocated in the different cases. This is something I will need to do anyway when I propose that patch; Best regards, -- Ronan Dunklau

Re: Use generation context to speed up tuplesorts

2021-12-17 Thread Ronan Dunklau
Le vendredi 17 décembre 2021, 09:08:06 CET Ronan Dunklau a écrit : > It is my understanding that malloc will try to compact memory by moving it > around. So the memory should be actually be released to the kernel at some > point. In the meantime, malloc can reuse it for our next invocati

Re: Use generation context to speed up tuplesorts

2021-12-17 Thread Ronan Dunklau
Le jeudi 16 décembre 2021, 18:00:56 CET Tomas Vondra a écrit : > On 12/16/21 17:03, Ronan Dunklau wrote: > > Le jeudi 16 décembre 2021, 11:56:15 CET Ronan Dunklau a écrit : > >> I will follow up with a benchmark of the test sorting a table with a > >> width > &

Re: Use generation context to speed up tuplesorts

2021-12-16 Thread Ronan Dunklau
Le jeudi 16 décembre 2021, 11:56:15 CET Ronan Dunklau a écrit : > I will follow up with a benchmark of the test sorting a table with a width > varying from 1 to 32 columns. > So please find attached another benchmark for that case. The 3 different patchsets tested are: - master

Re: Use generation context to speed up tuplesorts

2021-12-16 Thread Ronan Dunklau
Le mercredi 8 décembre 2021, 22:07:12 CET Tomas Vondra a écrit : > On 12/8/21 16:51, Ronan Dunklau wrote: > > Le jeudi 9 septembre 2021, 15:37:59 CET Tomas Vondra a écrit : > >> And now comes the funny part - if I run it in the same backend as the > >> > >

Re: Use generation context to speed up tuplesorts

2021-12-08 Thread Ronan Dunklau
can improve here. I'll try to see if I can understand more precisely what is happening. -- Ronan Dunklau

Re: Non-superuser subscription owners

2021-12-07 Thread Ronan Dunklau
tions to non-superuser owners. > 3) Allow the creation of subscriptions by non-superusers who are members of > some as yet to be created predefined role, say "pg_create_subscriptions" This roadmap seems sensible. -- Ronan Dunklau

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2021-11-04 Thread Ronan Dunklau
Le jeudi 22 juillet 2021, 10:42:49 CET Ronan Dunklau a écrit : > Le jeudi 22 juillet 2021, 09:38:50 CEST David Rowley a écrit : > > On Thu, 22 Jul 2021 at 02:01, Ronan Dunklau wrote: > > > I tested the 0001 patch against both HEAD and my proposed bugfix for

Re: pg_receivewal starting position

2021-10-29 Thread Ronan Dunklau
Le vendredi 29 octobre 2021, 04:27:51 CEST Michael Paquier a écrit : > On Thu, Oct 28, 2021 at 03:55:12PM +0200, Ronan Dunklau wrote: > > Interesting ideas, thanks. For the record, the time drops from ~4.5s to 3s > > on average on my machine. > > I think i

Re: pg_receivewal starting position

2021-10-28 Thread Ronan Dunklau
Le jeudi 28 octobre 2021, 14:31:36 CEST Michael Paquier a écrit : > On Wed, Oct 27, 2021 at 10:11:00AM +0200, Ronan Dunklau wrote: > > Sorry I sent an intermediary version of the patch, here is the correct > > one. > > While looking at this patch, I have figured out

Re: pg_receivewal starting position

2021-10-27 Thread Ronan Dunklau
Le mercredi 27 octobre 2021, 10:00:40 CEST Ronan Dunklau a écrit : > Le mercredi 27 octobre 2021, 04:17:28 CEST Kyotaro Horiguchi a écrit : > > +my @walfiles = glob "$slot_dir/*"; > > > > This is not used. > > Sorry, fixed in attached version. > > &

Re: pg_receivewal starting position

2021-10-27 Thread Ronan Dunklau
s not incremented after we fetched the end lsn, and thus we would wait for quite a while. I fixed that too. Thank you for the review ! -- Ronan Dunklau>From 04747199b7be896a62021e4f064b2342234427d5 Mon Sep 17 00:00:00 2001 From: Ronan Dunklau Date: Tue, 26 Oct 2021 10:54:12 +0200 Subject: [PATCH v1

Re: pg_receivewal starting position

2021-10-26 Thread Ronan Dunklau
Le mardi 26 octobre 2021, 08:27:47 CEST Ronan Dunklau a écrit : > Yes, I will try to simplify the logic of the patch I sent last week. I'll > keep you posted here soon. I was able to simplify it quite a bit, by using only one standby for both test scenarios. This test case verify that

Re: pg_receivewal starting position

2021-10-26 Thread Ronan Dunklau
d a new patch for this part? Yes, I will try to simplify the logic of the patch I sent last week. I'll keep you posted here soon. -- Ronan Dunklau

Re: pg_receivewal starting position

2021-10-25 Thread Ronan Dunklau
Le lundi 25 octobre 2021, 10:10:13 CEST Michael Paquier a écrit : > On Mon, Oct 25, 2021 at 09:50:01AM +0200, Ronan Dunklau wrote: > > Le lundi 25 octobre 2021, 09:40:10 CEST Michael Paquier a écrit : > >> On Mon, Oct 25, 2021 at 09:15:32AM +0200, Ronan Dunklau wrote: > &

Re: pg_receivewal starting position

2021-10-25 Thread Ronan Dunklau
Le lundi 25 octobre 2021, 09:40:10 CEST Michael Paquier a écrit : > On Mon, Oct 25, 2021 at 09:15:32AM +0200, Ronan Dunklau wrote: > > Does it make sense though ? The NULL slot_name case handling is pretty > > straight forward has it will be handled by string formatting, but

Re: Non-superuser subscription owners

2021-10-25 Thread Ronan Dunklau
tch looks good to me, and provides both better security (by closing the "dropping superuser role" loophole) and usefule features. -- Ronan Dunklau

Re: pg_receivewal starting position

2021-10-25 Thread Ronan Dunklau
you all for your work on this patch. -- Ronan Dunklau

Re: pg_receivewal starting position

2021-10-25 Thread Ronan Dunklau
Le lundi 25 octobre 2021, 08:51:23 CEST Michael Paquier a écrit : > On Thu, Oct 21, 2021 at 10:36:42AM +0200, Ronan Dunklau wrote: > > Done. I haven't touched the timeline switch test patch for now, but I > > still > > include it here for completeness. > > As 0001 an

Re: pg_receivewal starting position

2021-10-21 Thread Ronan Dunklau
Le jeudi 21 octobre 2021, 09:21:44 CEST Michael Paquier a écrit : > On Thu, Oct 21, 2021 at 08:29:54AM +0200, Ronan Dunklau wrote: > > Ok, do you want me to propose a different patch for previous versions ? > > That's not necessary. Thanks for proposing. > > >

Re: pg_receivewal starting position

2021-10-21 Thread Ronan Dunklau
Le jeudi 21 octobre 2021, 07:35:08 CEST Michael Paquier a écrit : > On Wed, Oct 20, 2021 at 02:58:26PM +0200, Ronan Dunklau wrote: > > After sending the previous patch suite, I figured it would be worthwhile > > to > > also have tests covering timeline switches, which w

Re: pg_receivewal starting position

2021-10-20 Thread Ronan Dunklau
hose tests, covering both "resume from last know archive" and "resume from the replication slots position" cases. -- Ronan Dunklau>From 5a47f17a17594cc171f744ce383ba820d44b6446 Mon Sep 17 00:00:00 2001 From: Michael Paquier Date: Thu, 2 Sep 2021 16:25:25 +0900 Subject: [PATCH

Re: pg_receivewal starting position

2021-10-20 Thread Ronan Dunklau
Le mercredi 20 octobre 2021, 07:13:15 CEST Michael Paquier a écrit : > On Tue, Oct 19, 2021 at 05:32:55PM +0200, Ronan Dunklau wrote: > > Following recommendations, I stripped most of the features from the patch. > > For now we support only physical replication slots, and only prov

Re: pg_receivewal starting position

2021-10-19 Thread Ronan Dunklau
testrictions can always be lifted in the > > future. > > And marked as RwF as this was three weeks ago. Please feel free to > register a new entry if this is being worked on. > -- > Michael -- Ronan Dunklau>From 4147665664164eb597fdcc86637ec9c497c36660 Mon Sep 17 00:00:00 2001 Fr

Re: Delegating superuser tasks to new security roles (Was: Granting control of SUSET gucs to non-superusers)

2021-10-19 Thread Ronan Dunklau
w that we don't allow to run ALTER SUBSCRIPTION REFRESH in a function anymore, there is no way to delegate this to a non superuser (using a security definer function). Since it doesn't involve the rest of the patchset, patches 16, 17 and 18 could be separated in another thread / patchset. -- Ronan Dunklau

Re: Patch to avoid orphaned dependencies

2021-09-17 Thread Ronan Dunklau
't predicate locks on the pg_depend index pages corresponding to the dropped object / referenced objects work as a different approach ? I'm not familiar enough with them so maybe there is some fundamental misunderstanding on my end. -- Ronan Dunklau

Re: Proposal: More structured logging

2021-09-09 Thread Ronan Dunklau
Le mercredi 8 septembre 2021, 11:51:31 CEST Peter Eisentraut a écrit : > On 01.09.21 10:00, Ronan Dunklau wrote: > > In-core it would open up the possibility to split log messages into > > different fields, for example the different statistics reported in the > > logs by VAC

Re: ORDER BY pushdowns seem broken in postgres_fdw

2021-09-06 Thread Ronan Dunklau
Le lundi 6 septembre 2021, 11:25:39 CEST Zhihong Yu a écrit : > On Mon, Sep 6, 2021 at 1:17 AM Ronan Dunklau wrote: > > Le vendredi 3 septembre 2021, 22:54:25 CEST David Zhang a écrit : > > > The following review has been posted through the commitfest application: > > &

Re: ORDER BY pushdowns seem broken in postgres_fdw

2021-09-06 Thread Ronan Dunklau
l running make installcheck-world. In any case, I just checked and running make installcheck-world doesn't produce any error. Since HEAD had moved a bit since the last version, I rebased the patch, resulting in the attached v7. Best regards, -- Ronan Dunklau >From ccfb0a3d7dae42f56c699aa

Re: pg_receivewal starting position

2021-09-06 Thread Ronan Dunklau
Le vendredi 3 septembre 2021 17:49:34 CEST, vous avez écrit : > On Fri, Sep 3, 2021 at 3:28 PM Ronan Dunklau wrote: > > There is still the concern raised by Bharath about being able to select > > the > > way to fetch the replication slot information for the user, and what

Re: pg_receivewal starting position

2021-09-06 Thread Ronan Dunklau
Le lundi 6 septembre 2021, 06:22:30 CEST Michael Paquier a écrit : > On Fri, Sep 03, 2021 at 11:58:27AM +0200, Ronan Dunklau wrote: > > 0002 for pg_receivewal tried to simplify the logic of information to > > return, by using a dedicated struct for this. This accounts for Bahrat

Re: pg_receivewal starting position

2021-09-03 Thread Ronan Dunklau
Le jeudi 2 septembre 2021, 10:37:20 CEST Michael Paquier a écrit : > On Thu, Sep 02, 2021 at 10:08:26AM +0200, Ronan Dunklau wrote: > > I could see the use for sending active_pid for use within pg_basebackup: > > at > > least we could fail early if the slot is already in

Re: improve pg_receivewal code

2021-09-02 Thread Ronan Dunklau
+ /* +* No valid data can be found in the existing output directory. +* Get start LSN position and current timeline ID from the server. + */ And here it is not formatted properly: +static char *server_sysid = NULL; > > Regards, > Bharath Rupireddy. -- Ronan Dunklau

Re: pg_receivewal starting position

2021-09-02 Thread Ronan Dunklau
to to get the semantics of the command to what looks > like the right shape. Thanks, I was about to send a new patchset with basically the same thing. It would be nice to know we work on the same thing concurrently in the future to avoid duplicate efforts. I'll rebase and send the updated version for patches 0002 and 0003 of my original proposal once we reach an agreement over the behaviour / options of pg_receivewal. Also considering the number of different fields to be filled by the GetSlotInformation function, my local branch groups them into a dedicated struct which is more convenient than having X possibly null arguments. -- Ronan Dunklau

Re: pg_receivewal starting position

2021-09-02 Thread Ronan Dunklau
Le jeudi 2 septembre 2021, 08:42:22 CEST Bharath Rupireddy a écrit : > On Thu, Sep 2, 2021 at 11:15 AM Kyotaro Horiguchi > > wrote: > > At Wed, 1 Sep 2021 10:30:05 +0530, Bharath Rupireddy > > wrote in> > > > On Mon, Aug 30, 2021 at 3:26 PM Ronan Dunklau wro

Re: Proposal: More structured logging

2021-09-01 Thread Ronan Dunklau
Le mercredi 1 septembre 2021, 09:36:50 CEST Peter Eisentraut a écrit : > On 13.08.21 15:23, Ronan Dunklau wrote: > > The logging system already captures a lot of information in the ErrorData. > > But at present there is no way for a log message authors to include more > > m

Re: pg_receivewal starting position

2021-08-31 Thread Ronan Dunklau
Le mardi 31 août 2021, 13:17:22 CEST Bharath Rupireddy a écrit : > On Mon, Aug 30, 2021 at 3:26 PM Ronan Dunklau wrote: > > Thank you for this review ! Please see the other side of the thread where > > I > > answered Michael Paquier with a new patchset, which includes some

Re: pg_receivewal starting position

2021-08-30 Thread Ronan Dunklau
Le samedi 28 août 2021, 14:10:25 CEST Bharath Rupireddy a écrit : > On Thu, Aug 26, 2021 at 5:45 PM Ronan Dunklau wrote: > > order to fail early if the replication slot doesn't exist, so please find > > attached v2 for that. > > Thanks for the patches. Here are some co

Re: pg_receivewal starting position

2021-08-30 Thread Ronan Dunklau
Le vendredi 27 août 2021, 05:44:32 CEST Michael Paquier a écrit : > On Thu, Aug 26, 2021 at 02:14:27PM +0200, Ronan Dunklau wrote: > > Following the discussion at [1], I refactored the implementation into > > streamutil and added a third patch making use of it in pg_basebackup >

Re: pg_receivewal starting position

2021-08-26 Thread Ronan Dunklau
Le jeudi 29 juillet 2021, 11:09:40 CEST Ronan Dunklau a écrit : > Patch 0001 adds the new READ_REPLICATION_SLOT command. > It returns for a given slot the type, restart_lsn, flush_lsn, > restart_lsn_timeline and flush_lsn_timeline. > The timelines are determined by reading the curr

Re: Proposal: More structured logging

2021-08-23 Thread Ronan Dunklau
Le vendredi 20 août 2021, 11:31:21 CEST Ronan Dunklau a écrit : > Le jeudi 19 août 2021, 15:04:30 CEST Alvaro Herrera a écrit : > > On 2021-Aug-13, Ronan Dunklau wrote: > > > ereport(NOTICE, > > > > > > (errmsg("My log message")), > > > (er

Re: Proposal: More structured logging

2021-08-20 Thread Ronan Dunklau
about > Fluentbit and similar tools. Michael, your jsonlog module already fullfills this need. Is it something that should be merged into our tree ? -- Ronan Dunklau

Re: Proposal: More structured logging

2021-08-20 Thread Ronan Dunklau
Le jeudi 19 août 2021, 15:04:30 CEST Alvaro Herrera a écrit : > On 2021-Aug-13, Ronan Dunklau wrote: > > ereport(NOTICE, > > > > (errmsg("My log message")), > > (errtag("EMITTER", "MYEXTENSION")), > > (errtag("MSG-ID&quo

Proposal: More structured logging

2021-08-13 Thread Ronan Dunklau
ving syslog implemented by journald itself. Would there be any interest in having native journald support as a logging_destination ? Best regards, -- Ronan Dunklau>From 270ffc5ed2fbe5f5076bddee14c5fb3555b87e4f Mon Sep 17 00:00:00 2001 From: Ronan Dunklau Date: Fri, 13 Aug 2021 15:03:18 +0200 S

Re: Minimal logical decoding on standbys

2021-08-02 Thread Ronan Dunklau
that you drop an additional object on the standby, which doesn't exist and that you don't necessarily have any knowledge of on the primary. As such, I thought it would be better to be explicit about it to warn users of that possible case. Regards, -- Ronan Dunklau

Re: Minimal logical decoding on standbys

2021-08-02 Thread Ronan Dunklau
n patch 0004, small typo in the test file: +## +# Test standby promotion and logical decoding bheavior +# after the standby gets promoted. +## Thank you for working on this ! Regards, -- Ronan Dunklau

Re: pg_receivewal starting position

2021-07-29 Thread Ronan Dunklau
line switch. It would probably be good to add that, both for the case where we determine the previous timeline from the archived segments and when it comes from the new command. What do you think ? Regards, -- Ronan Dunklau>From 37d9545c05b9e36aafac751f9dc549e75798413c Mon Sep 17 00:00:00

  1   2   >