Re: First draft of PG 17 release notes

2024-05-22 Thread torikoshia
On 2024-05-09 13:03, Bruce Momjian wrote: I have committed the first draft of the PG 17 release notes; you can see the results here: https://momjian.us/pgsql_docs/release-17.html It will be improved until the final release. The item count is 188, which is similar to recent releases:

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-04-17 Thread torikoshia
On 2024-04-16 13:16, Masahiko Sawada wrote: On Tue, Apr 2, 2024 at 7:34 PM torikoshia wrote: On 2024-04-01 11:31, Masahiko Sawada wrote: > On Fri, Mar 29, 2024 at 11:54 AM torikoshia > wrote: >> >> On 2024-03-28 21:54, Masahiko Sawada wrote: >> > On Thu, Mar 28,

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-04-02 Thread torikoshia
On 2024-04-01 11:31, Masahiko Sawada wrote: On Fri, Mar 29, 2024 at 11:54 AM torikoshia wrote: On 2024-03-28 21:54, Masahiko Sawada wrote: > On Thu, Mar 28, 2024 at 9:38 PM torikoshia > wrote: >> >> On 2024-03-28 10:20, Masahiko Sawada wrote: >> > Hi, >> &g

Re: Add new error_action COPY ON_ERROR "log"

2024-03-29 Thread torikoshia
On 2024-03-28 21:36, torikoshia wrote: On 2024-03-28 17:27, Bharath Rupireddy wrote: On Thu, Mar 28, 2024 at 1:43 PM torikoshia wrote: Attached patch fixes the doc, May I know which patch you are referring to? And, what do you mean by "fixes the doc"? Ugh, I replied to the wr

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-03-28 Thread torikoshia
On 2024-03-28 21:54, Masahiko Sawada wrote: On Thu, Mar 28, 2024 at 9:38 PM torikoshia wrote: On 2024-03-28 10:20, Masahiko Sawada wrote: > Hi, > > On Thu, Jan 18, 2024 at 5:33 PM Masahiko Sawada > wrote: >> >> On Thu, Jan 18, 2024 at 4:59 PM Alexander Korotkov >

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-03-28 Thread torikoshia
On 2024-03-28 10:20, Masahiko Sawada wrote: Hi, On Thu, Jan 18, 2024 at 5:33 PM Masahiko Sawada wrote: On Thu, Jan 18, 2024 at 4:59 PM Alexander Korotkov wrote: > > On Thu, Jan 18, 2024 at 4:16 AM torikoshia wrote: > > On 2024-01-18 10:10, jian he wrote: > > > On T

Re: Add new error_action COPY ON_ERROR "log"

2024-03-28 Thread torikoshia
On 2024-03-28 17:27, Bharath Rupireddy wrote: On Thu, Mar 28, 2024 at 1:43 PM torikoshia wrote: Attached patch fixes the doc, May I know which patch you are referring to? And, what do you mean by "fixes the doc"? Ugh, I replied to the wrong thread. Sorry for making you confused

Re: Add new error_action COPY ON_ERROR "log"

2024-03-28 Thread torikoshia
On 2024-03-26 17:16, Masahiko Sawada wrote: On Tue, Mar 26, 2024 at 3:04 PM Bharath Rupireddy wrote: On Tue, Mar 26, 2024 at 9:56 AM Masahiko Sawada wrote: > > > > errmsg("data type incompatibility at line %llu for column %s: \"%s\"", > > > > I guess it would be better to make the log

Re: RFC: Logging plan of the running query

2024-03-17 Thread torikoshia
On 2024-03-14 04:33, Robert Haas wrote: Thanks for your review! On Wed, Mar 13, 2024 at 1:28 AM torikoshia wrote: - I saw no way to find the next node to be executed from the planstate tree, so the patch wraps all the ExecProcNode of the planstate tree at CHECK_FOR_INTERRUPTS(). I don't

Re: RFC: Logging plan of the running query

2024-03-12 Thread torikoshia
On Fri, Feb 16, 2024 at 11:42 PM torikoshia wrote: I'm not so sure about the implementation now, i.e. finding the next node to be executed from the planstate tree, but I'm going to try this approach. Attached a patch which takes this approach. - I saw no way to find the next node

Re: Add new error_action COPY ON_ERROR "log"

2024-03-06 Thread torikoshia
On 2024-03-07 13:00, Michael Paquier wrote: On Wed, Mar 06, 2024 at 07:32:28PM +0530, Bharath Rupireddy wrote: Please see the attached v4 patch. If it looks good, I can pull LOG_VERBOSITY changes out into 0001 and with 0002 containing the detailed messages for discarded rows. The approach

Re: Add new error_action COPY ON_ERROR "log"

2024-02-26 Thread torikoshia
On 2024-02-20 17:22, torikoshia wrote: On 2024-02-17 15:00, Bharath Rupireddy wrote: On Fri, Feb 16, 2024 at 8:17 PM torikoshia wrote: I may be wrong since I seldom do data loading tasks, but I greed with you. I also a little concerned about the case where there are many malformed data

Re: RFC: Logging plan of the running query

2024-02-26 Thread torikoshia
On 2024-02-24 00:23, Robert Haas wrote: On Fri, Feb 23, 2024 at 7:50 PM Julien Rouhaud wrote: On Fri, Feb 23, 2024 at 10:22:32AM +0530, Robert Haas wrote: > On Thu, Feb 22, 2024 at 6:25 AM James Coleman wrote: > > This is potentially a bit of a wild idea, but I wonder if having some > > kind

Re: Add new error_action COPY ON_ERROR "log"

2024-02-20 Thread torikoshia
On 2024-02-17 15:00, Bharath Rupireddy wrote: On Fri, Feb 16, 2024 at 8:17 PM torikoshia wrote: I may be wrong since I seldom do data loading tasks, but I greed with you. I also a little concerned about the case where there are many malformed data and it causes lots of messages

Re: Add new error_action COPY ON_ERROR "log"

2024-02-16 Thread torikoshia
On 2024-02-16 17:15, Bharath Rupireddy wrote: On Wed, Feb 14, 2024 at 5:04 PM torikoshia wrote: [] let the users know what line numbers are > containing the errors that COPY ignored something like [1] with a > simple change like [2]. Agreed. Unlike my patch, it hides the

Re: RFC: Logging plan of the running query

2024-02-16 Thread torikoshia
On Thu, Feb 15, 2024 at 6:12 PM Robert Haas wrote: Hi, I've just been catching up on this thread. + if (MyProc->heldLocks) + { + ereport(LOG_SERVER_ONLY, + errmsg("ignored request for logging query plan due to lock conflicts"), + errdetail("You can try again in a moment.")); + return; + }

Re: Add new error_action COPY ON_ERROR "log"

2024-02-14 Thread torikoshia
On 2024-02-12 15:46, Bharath Rupireddy wrote: Thanks for your comments. On Mon, Jan 29, 2024 at 8:41 AM torikoshia wrote: On 2024-01-27 00:43, David G. Johnston wrote: >> I'd like to have a new option "log", which skips soft errors and >> logs >> informati

Re: RFC: Logging plan of the running query

2024-02-13 Thread torikoshia
On 2024-02-13 11:30, torikoshia wrote: I'll update the patch including other points such as removing ensuring no page lock code. Updated the patch. Using injection point support we should be able to add tests for testing pg_log_query_plan behaviour when there are page locks held or when

Re: RFC: Logging plan of the running query

2024-02-12 Thread torikoshia
On 2024-02-12 09:00, jian he wrote: Thanks for you comments. On Mon, Jan 29, 2024 at 9:02 PM torikoshia wrote: Hi, Updated the patch to fix typos and move ProcessLogQueryPlanInterruptActive from errfinish() to AbortTransaction. + + + + pg_log_query_plan

Re: RFC: Logging plan of the running query

2024-02-12 Thread torikoshia
On 2024-02-07 19:14, torikoshia wrote: On 2024-02-07 13:58, Ashutosh Bapat wrote: The prologue refers to a very populated lock hash table. I think that will happen if thousands of tables are queried in a single query OR a query runs on a partitioned table with thousands of partitions. May

Re: RFC: Logging plan of the running query

2024-02-07 Thread torikoshia
On 2024-02-07 13:58, Ashutosh Bapat wrote: On Wed, Feb 7, 2024 at 9:38 AM torikoshia wrote: Hi Ashutosh, On 2024-02-06 19:51, Ashutosh Bapat wrote: > Thanks for the summary. It is helpful. I think patch is also getting > better. > > I have a few questions and suggestions Tha

Re: RFC: Logging plan of the running query

2024-02-06 Thread torikoshia
Hi Ashutosh, On 2024-02-06 19:51, Ashutosh Bapat wrote: Thanks for the summary. It is helpful. I think patch is also getting better. I have a few questions and suggestions Thanks for your comments. 1. Prologue of GetLockMethodLocalHash() mentions * NOTE: When there are many entries in

Re: Change COPY ... ON_ERROR ignore to ON_ERROR ignore_row

2024-02-04 Thread torikoshia
Hi, On 2024-02-03 15:22, jian he wrote: The idea of on_error is to tolerate errors, I think. if a column has a not null constraint, let it cannot be used with (on_error 'null') + /* +* we can specify on_error 'null', but it can only apply to columns +* don't have not

Re: Add new COPY option REJECT_LIMIT

2024-02-01 Thread torikoshia
On 2024-01-27 00:20, David G. Johnston wrote: Thanks for your comments! On Fri, Jan 26, 2024 at 2:49 AM torikoshia wrote: Hi, 9e2d870 enabled the COPY command to skip soft error, and I think we can add another option which specifies the maximum tolerable number of soft errors. I remember

Re: Small fix on COPY ON_ERROR document

2024-02-01 Thread torikoshia
On 2024-02-01 15:16, Yugo NAGATA wrote: On Mon, 29 Jan 2024 15:47:25 +0900 Yugo NAGATA wrote: On Sun, 28 Jan 2024 19:14:58 -0700 "David G. Johnston" wrote: > > Also, I think "invalid input syntax" is a bit ambiguous. For example, > > COPY FROM raises an error when the number of input column

Re: RFC: Logging plan of the running query

2024-01-29 Thread torikoshia
Hi, Updated the patch to fix typos and move ProcessLogQueryPlanInterruptActive from errfinish() to AbortTransaction. BTW since the thread is getting long, I list the some points of the discussion so far: # Safety concern ## Catalog access inside CFI - it seems safe if the CFI call is

Re: Add new error_action COPY ON_ERROR "log"

2024-01-28 Thread torikoshia
ied them to NOTICE in accordance with the following summary message: NOTICE: x row was skipped due to data type incompatibility On 2024-01-27 00:43, David G. Johnston wrote: On Thu, Jan 25, 2024 at 9:42 AM torikoshia wrote: Hi, As described in 9e2d870119, COPY ON_EEOR is expected to have

Re: Small fix on COPY ON_ERROR document

2024-01-28 Thread torikoshia
On 2024-01-27 00:04, David G. Johnston wrote: On Fri, Jan 26, 2024 at 2:30 AM Yugo NAGATA wrote: On Fri, 26 Jan 2024 00:00:57 -0700 "David G. Johnston" wrote: I will need to make this tweak and probably a couple others to my own suggestions in 12 hours or so. And here is my v2.

Add new COPY option REJECT_LIMIT

2024-01-26 Thread torikoshia
Hi, 9e2d870 enabled the COPY command to skip soft error, and I think we can add another option which specifies the maximum tolerable number of soft errors. I remember this was discussed in [1], and feel it would be useful when loading 'dirty' data but there is a limit to how dirty it can

Add new error_action COPY ON_ERROR "log"

2024-01-25 Thread torikoshia
Hi, As described in 9e2d870119, COPY ON_EEOR is expected to have more "error_action". (Note that option name was changed by b725b7eec) I'd like to have a new option "log", which skips soft errors and logs information that should have resulted in errors to PostgreSQL log. I think this

Re: Add tuples_skipped to pg_stat_progress_copy

2024-01-24 Thread torikoshia
On 2024-01-24 17:05, Masahiko Sawada wrote: On Tue, Jan 23, 2024 at 1:02 AM torikoshia wrote: On 2024-01-17 14:47, Masahiko Sawada wrote: > On Wed, Jan 17, 2024 at 2:22 PM torikoshia > wrote: >> >> Hi, >> >> 132de9968840c introduced SAVE_ERROR_TO option

Re: Add tuples_skipped to pg_stat_progress_copy

2024-01-22 Thread torikoshia
On 2024-01-17 14:47, Masahiko Sawada wrote: On Wed, Jan 17, 2024 at 2:22 PM torikoshia wrote: Hi, 132de9968840c introduced SAVE_ERROR_TO option to COPY and enabled to skip malformed data, but there is no way to watch the number of skipped rows during COPY. Attached patch adds

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-01-19 Thread torikoshia
On 2024-01-19 22:27, Alexander Korotkov wrote: Hi! On Fri, Jan 19, 2024 at 2:37 PM torikoshia wrote: Thanks for making the patch! The patch is pushed! The proposed changes are incorporated excluding this. > - /* If SAVE_ERROR_TO is specified, skip r

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-01-19 Thread torikoshia
On 2024-01-18 23:59, jian he wrote: Hi. patch refactored based on "on_error {stop|ignore}" doc changes: --- a/doc/src/sgml/ref/copy.sgml +++ b/doc/src/sgml/ref/copy.sgml @@ -43,7 +43,7 @@ COPY { table_name [ ( column_name [, ...] ) | * } FORCE_NOT_NULL { ( column_name [, ...] ) | * }

Re: Parent/child context relation in pg_get_backend_memory_contexts()

2024-01-19 Thread torikoshia
On 2024-01-16 18:41, Melih Mutlu wrote: Hi, Thanks for reviewing. torikoshia , 10 Oca 2024 Çar, 09:37 tarihinde şunu yazdı: + + + context_id int4 + + + Current context id. Note that the context id is a temporary id and may + change in each

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-01-18 Thread torikoshia
On 2024-01-18 16:59, Alexander Korotkov wrote: On Thu, Jan 18, 2024 at 4:16 AM torikoshia wrote: On 2024-01-18 10:10, jian he wrote: > On Thu, Jan 18, 2024 at 8:57 AM Masahiko Sawada > wrote: >> On Thu, Jan 18, 2024 at 6:38 AM Tom Lane wrote: >> > Kyotaro-san's suggestio

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-01-17 Thread torikoshia
On 2024-01-18 10:10, jian he wrote: On Thu, Jan 18, 2024 at 8:57 AM Masahiko Sawada wrote: On Thu, Jan 18, 2024 at 6:38 AM Tom Lane wrote: > > Alexander Korotkov writes: > > On Wed, Jan 17, 2024 at 9:49 AM Kyotaro Horiguchi > > wrote: > >> On the other hand, SAVE_ERROR_TO takes 'error' or

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-01-16 Thread torikoshia
Hi, Thanks for applying! + errmsg_plural("%zd row were skipped due to data type incompatibility", Sorry, I just noticed it, but 'were' should be 'was' here? BTW I'm thinking we should add a column to pg_stat_progress_copy that counts soft errors. I'll suggest

Add tuples_skipped to pg_stat_progress_copy

2024-01-16 Thread torikoshia
Hi, 132de9968840c introduced SAVE_ERROR_TO option to COPY and enabled to skip malformed data, but there is no way to watch the number of skipped rows during COPY. Attached patch adds tuples_skipped to pg_stat_progress_copy, which counts the number of skipped tuples because source data is

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-01-15 Thread torikoshia
On 2024-01-16 00:17, Alexander Korotkov wrote: On Mon, Jan 15, 2024 at 8:44 AM Masahiko Sawada wrote: On Mon, Jan 15, 2024 at 8:21 AM Alexander Korotkov wrote: > > On Sun, Jan 14, 2024 at 10:35 PM Masahiko Sawada wrote: > > Thank you for updating the patch. Here are two comments: > > > >

Re: doc: add LITERAL tag to RETURNING

2024-01-14 Thread torikoshia
On 2024-01-12 20:56, Alvaro Herrera wrote: On 2024-Jan-12, Ashutosh Bapat wrote: On Fri, Jan 12, 2024 at 11:27 AM torikoshia wrote: > > RETURNING is usually tagged with appropriate tags, such as , > but not in the 'query' section of COPY. The patch looks good. Good catc

doc: add LITERAL tag to RETURNING

2024-01-11 Thread torikoshia
Hi, RETURNING is usually tagged with appropriate tags, such as , but not in the 'query' section of COPY. https://www.postgresql.org/docs/devel/sql-copy.html Would it be better to put here as well? -- Regards, -- Atsushi Torikoshi NTT DATA Group CorporationFrom

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-01-11 Thread torikoshia
ature. +1. I'm also going to make patch for "logging errors", since this functionality is isolated from v7 patch. Seems promising. I'll look at the patch. Thanks a lot! Sorry to attach v2 if you already reviewed v1.. On 2024-01-11 12:13, jian he wrote: On Tue, Jan 9, 2024 at 10:36 P

Re: Parent/child context relation in pg_get_backend_memory_contexts()

2024-01-09 Thread torikoshia
On 2024-01-03 20:40, Melih Mutlu wrote: Hi, Thanks for reviewing. Please find the updated patch attached. torikoshia , 4 Ara 2023 Pzt, 07:43 tarihinde şunu yazdı: I reviewed v3 patch and here are some minor comments: + + + path int4 Should 'int4' be 'int4[]'? Other

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-01-09 Thread torikoshia
On Tue, Dec 19, 2023 at 10:14 AM Masahiko Sawada wrote: If we want only such a feature we need to implement it together (the patch could be split, though). But if some parts of the feature are useful for users as well, I'd recommend implementing it incrementally. That way, the patches can get

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-12-17 Thread torikoshia
Hi, save the error metadata to system catalogs would be more expensive, please see below explanation. I have no knowledge of publications. but i feel there is a feature request: publication FOR ALL TABLES exclude regex_pattern. Anyway, that would be another topic. I think saving error

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-12-17 Thread torikoshia
On 2023-12-15 05:48, Masahiko Sawada wrote: Thanks for joining this discussion! I've read this thread and the latest patch. IIUC with SAVE_ERROR option, COPY FROM creates an error table for the target table and writes error information there. While I agree that the final shape of this feature

Re: RFC: Logging plan of the running query

2023-12-10 Thread torikoshia
On 2023-12-07 08:33, Rafael Thofehrn Castro wrote: Hello hackers, Last Saturday I submitted a patch to the pgsql-hackers list with the title "Proposal: In-flight explain logging" with a patch proposing a feature very similar to the one being worked on in this thread. I should have done a better

Re: Separate memory contexts for relcache and catcache

2023-12-03 Thread torikoshia
Hi, I also think this change would be helpful. I imagine you're working on the Andres's comments and you already notice this, but v1 patch cannot be applied to HEAD. For the convenience of other reviewers, I marked it 'Waiting on Author'. -- Regards, -- Atsushi Torikoshi NTT DATA Group

Re: Parent/child context relation in pg_get_backend_memory_contexts()

2023-12-03 Thread torikoshia
Thanks for working on this improvement! On 2023-10-23 21:02, Melih Mutlu wrote: Hi, Thanks for reviewing. Attached the updated patch v3. I reviewed v3 patch and here are some minor comments: + + role="column_definition"> + path int4 Should 'int4' be 'int4[]'? Other

Re: Add new option 'all' to pg_stat_reset_shared()

2023-11-16 Thread torikoshia
On 2023-11-16 16:48, Michael Paquier wrote: On Wed, Nov 15, 2023 at 04:25:14PM +0900, Michael Paquier wrote: Other than that, it looks OK. Tweaked the queries of this one slightly, and applied. So I think that we are now good for this thread. Thanks, all! Thanks for the modification and

Re: Add new option 'all' to pg_stat_reset_shared()

2023-11-14 Thread torikoshia
On 2023-11-15 09:47, Michael Paquier wrote: On Tue, Nov 14, 2023 at 10:02:32PM +0900, torikoshia wrote: Attached patch. You have forgotten to update the errhint at the end of pg_stat_reset_shared(), where "slru" needs to be listed :) Oops, attached v2 patch. BTW

Re: Add new option 'all' to pg_stat_reset_shared()

2023-11-14 Thread torikoshia
On 2023-11-13 13:15, torikoshia wrote: On 2023-11-12 16:46, Michael Paquier wrote: On Fri, Nov 10, 2023 at 01:15:50PM +0900, Michael Paquier wrote: The comments added could be better grammatically, but basically LGTM. I'll take care of that if there are no objections. The documentation also

Re: Add new option 'all' to pg_stat_reset_shared()

2023-11-12 Thread torikoshia
://www.postgresql.org/message-id/CALj2ACW4Fqc_m%2BOaavrOMEivZ5aBa24pVKvoXRTmuFECsNBfAg%40mail.gmail.com On 2023-11-12 16:54, Michael Paquier wrote: On Fri, Nov 10, 2023 at 08:32:34PM +0900, torikoshia wrote: On 2023-11-10 13:18, Andres Freund wrote: I see no reason to not include slrus. We should never have added

Re: Add new option 'all' to pg_stat_reset_shared()

2023-11-10 Thread torikoshia
On 2023-11-10 13:18, Andres Freund wrote: Hi, On November 8, 2023 11:28:08 PM PST, Michael Paquier wrote: On Thu, Nov 09, 2023 at 01:50:34PM +0900, torikoshia wrote: PGSTAT_KIND_SLRU cannot be reset by pg_stat_reset_shared(), so I feel uncomfortable to delete it all together. It might

Re: Add new option 'all' to pg_stat_reset_shared()

2023-11-09 Thread torikoshia
On 2023-11-09 16:28, Michael Paquier wrote: Thanks for your review. Attached v2 patch. On Thu, Nov 09, 2023 at 01:50:34PM +0900, torikoshia wrote: PGSTAT_KIND_SLRU cannot be reset by pg_stat_reset_shared(), so I feel uncomfortable to delete it all together. It might be better after

Re: RFC: Logging plan of the running query

2023-11-09 Thread torikoshia
On 2023-11-09 16:11, Ashutosh Bapat wrote: On Thu, Nov 9, 2023 at 12:03 PM torikoshia wrote: >> >> 1. When a backend is running nested queries, we will see the plan of >> the innermost query. That query may not be the actual culprit if the >> user query is running slowl

Re: RFC: Logging plan of the running query

2023-11-08 Thread torikoshia
On 2023-11-06 15:32, Ashutosh Bapat wrote: Thanks for the suggestion and example. On Fri, Nov 3, 2023 at 7:31 PM Ashutosh Bapat wrote: I have following questions related to the functionality. (Please point me to the relevant discussion if this has been already discussed.) 1. When a backend

Re: pg_rewind WAL segments deletion pitfall

2023-11-08 Thread torikoshia
On 2023-11-06 23:58, Alexander Kukushkin wrote: Hi Torikoshia, On Thu, 2 Nov 2023 at 04:24, torikoshia wrote: +extern void preserve_file(char *filepath); Is this necessary? This function was defined in older version patch, but no longer seems to exist. +# We use "perl -e 'e

Re: Add new option 'all' to pg_stat_reset_shared()

2023-11-08 Thread torikoshia
On Thu, Nov 09, 2023 at 10:10:39AM +0900, torikoshia wrote: I'll attach the patch. Attached. On Mon, Nov 6, 2023 at 5:30 PM Bharath Rupireddy 3. I think the new reset all stats function must also consider resetting all SLRU stats, no? /* stats for fixed-numbered objects

Re: Add new option 'all' to pg_stat_reset_shared()

2023-11-08 Thread torikoshia
On 2023-11-09 08:58, Michael Paquier wrote: On Wed, Nov 08, 2023 at 02:15:24PM +0530, Bharath Rupireddy wrote: On Wed, Nov 8, 2023 at 9:43 AM Andres Freund wrote: It's not like oids are a precious resource. It's a more confusing API to have to have to specify a NULL as an argument than not

Re: Add new option 'all' to pg_stat_reset_shared()

2023-11-05 Thread torikoshia
Thanks all for the comments! On Fri, Nov 3, 2023 at 5:17 AM Matthias van de Meent wrote: Knowing that your metrics have a shared starting point can be quite valuable, as it allows you to do some math that would otherwise be much less accurate when working with stats over a short amount of

Re: pg_rewind WAL segments deletion pitfall

2023-11-01 Thread torikoshia
On 2023-10-31 00:26, Alexander Kukushkin wrote: Hi, On Wed, 18 Oct 2023 at 08:50, torikoshia wrote: I have very minor questions on the regression tests mainly regarding the consistency with other tests for pg_rewind: Please find attached a new version of the patch. It addresses all your

Re: Add new option 'all' to pg_stat_reset_shared()

2023-10-31 Thread torikoshia
On Mon, Oct 30, 2023 at 5:46 PM Bharath Rupireddy wrote: Thanks for the comments! Isn't calling pg_stat_reset_shared() for all stats types helping you out? Is there any problem with it? Can you be more specific about the use-case? Yes, calling pg_stat_reset_shared() for all stats types can

Add new option 'all' to pg_stat_reset_shared()

2023-10-30 Thread torikoshia
Hi, After 96f052613f3, we have below 6 types of parameter for pg_stat_reset_shared(). "archiver", "bgwriter", "checkpointer", "io", "recovery_prefetch", "wal" How about adding a new option 'all' to delete all targets above? I imagine there are cases where people want to initialize all

Re: RFC: Logging plan of the running query

2023-10-27 Thread torikoshia
On 2023-10-27 16:06, Étienne BERSAC wrote: Hi Torikoshia, If so, we once tried to implement such function for getting memory contexts. However, this attempt didn't succeed because it could lead dead lock situation[1]. Thanks for the pointer. Why not use client log message to allow client

Re: RFC: Logging plan of the running query

2023-10-26 Thread torikoshia
On 2023-10-25 12:40, Ashutosh Bapat wrote: On Wed, Oct 18, 2023 at 10:04 PM James Coleman wrote: While that decision as regards auto_explain has long since been made (and there would be work to undo it), I don't believe that we should repeat that choice here. I'm -10 on moving this into

Re: RFC: Logging plan of the running query

2023-10-24 Thread torikoshia
On 2023-10-24 16:12, Étienne BERSAC wrote: Hi, Yeah, and when we have a situation where we want to run pg_log_query_plan(), we can run it in any environment as long as it is bundled with the core. Is it possible to get the plan of a backend programmatically without access to the logs?

Re: pg_rewind WAL segments deletion pitfall

2023-10-18 Thread torikoshia
Thanks for the patch. I tested the v6 patch using the test script attached on [1], old primary has succeeded to become new standby. I have very minor questions on the regression tests mainly regarding the consistency with other tests for pg_rewind: +setup_cluster; +create_standby;

Re: RFC: Logging plan of the running query

2023-10-18 Thread torikoshia
On 2023-10-16 18:46, Ashutosh Bapat wrote: On Thu, Oct 12, 2023 at 6:51 PM torikoshia wrote: On 2023-10-11 16:22, Ashutosh Bapat wrote: > > Considering the similarity with auto_explain I wondered whether this > function should be part of auto_explain contrib module itself? I

Re: RFC: Logging plan of the running query

2023-10-12 Thread torikoshia
On 2023-10-11 16:22, Ashutosh Bapat wrote: Like many others I think this feature is useful to debug a long running query. Sorry for jumping late into this. I have a few of high level comments Thanks for your comments! There is a lot of similarity between what this feature does and what

Re: RFC: Logging plan of the running query

2023-10-10 Thread torikoshia
On 2023-10-04 03:00, James Coleman wrote: and I think what we need to do is explicitly disallow running this code any time we are inside of lock acquisition code. Updated patch to check if any locks have already been acquired by examining MyProc->heldLocks. I'm not sure this change can

Re: RFC: Logging plan of the running query

2023-10-06 Thread torikoshia
On 2023-10-04 03:00, James Coleman wrote: On Thu, Sep 7, 2023 at 2:09 AM torikoshia wrote: On 2023-09-06 11:17, James Coleman wrote: >> > I've never been able to reproduce it (haven't tested the new version, >> > but v28 at least) on my M1 Mac; where I've reproduc

Re: Make --help output fit within 80 columns per line

2023-10-06 Thread torikoshia
On 2023-09-25 15:27, torikoshia wrote: Ugh, regression tests failed and it appears to be due to reasons related to meson. I'm going to investigate it. ISTM On 2023-10-06 19:49, Peter Eisentraut wrote: On 25.09.23 08:27, torikoshia wrote: So in summary, I think 80 is a decent soft limit

Re: RFC: Logging plan of the running query

2023-09-27 Thread torikoshia
On 2023-09-25 18:49, Andrey Lepikhov wrote: On 25/9/2023 14:21, torikoshia wrote: On 2023-09-20 14:39, Lepikhov Andrei wrote: Hmm, as a test, I made sure to call ProcessLogQueryPlanInterrupt() on all CFI using v28-0002-Testing-attempt-logging-plan-on-ever-CFI-call.patch, and then ran

Re: Make --help output fit within 80 columns per line

2023-09-25 Thread torikoshia
On 2023-09-25 15:27, torikoshia wrote: On Tue, Sep 19, 2023 at 3:23 AM Greg Sabino Mullane wrote: Thanks for your investigation! On Fri, Sep 15, 2023 at 11:11 AM torikoshia wrote: I do not intend to adhere to this rule(my terminals are usually bigger than 80 chars per line), but wouldn't

Re: RFC: Logging plan of the running query

2023-09-25 Thread torikoshia
On 2023-09-20 14:39, Lepikhov Andrei wrote: Thanks for your reply. On Tue, Sep 19, 2023, at 8:39 PM, torikoshia wrote: On 2023-09-15 15:21, Lepikhov Andrei wrote: On Thu, Sep 7, 2023, at 1:09 PM, torikoshia wrote: I have explored this patch and, by and large, agree with the code. But some

Re: Make --help output fit within 80 columns per line

2023-09-25 Thread torikoshia
On Tue, Sep 19, 2023 at 3:23 AM Greg Sabino Mullane wrote: Thanks for your investigation! On Fri, Sep 15, 2023 at 11:11 AM torikoshia wrote: I do not intend to adhere to this rule(my terminals are usually bigger than 80 chars per line), but wouldn't it be a not bad direction to use 80

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-09-19 Thread torikoshia
On 2023-09-15 19:02, Damir Belyalov wrote: Since v5 patch failed applying anymore, updated the patch. Thank you for updating the patch . I made a little review on it where corrected some formatting. Thanks for your review and update! I don't have objections the modification of the codes and

Re: RFC: Logging plan of the running query

2023-09-19 Thread torikoshia
On 2023-09-15 15:21, Lepikhov Andrei wrote: On Thu, Sep 7, 2023, at 1:09 PM, torikoshia wrote: On 2023-09-06 11:17, James Coleman wrote: It seems that we can know what queries were running from the stack traces you shared. As described above, I suspect a lock which was acquired prior

Re: Make --help output fit within 80 columns per line

2023-09-15 Thread torikoshia
On 2023-09-12 15:27, Peter Eisentraut wrote: Also, it would be very useful if the TAP test function could print out the violating lines if a test fails. (Similar to how is() and like() print the failing values.) Attached patch for this. Below are the the outputs when test failed: ``` $ cd

Re: Make --help output fit within 80 columns per line

2023-09-13 Thread torikoshia
On 2023-09-12 15:27, Peter Eisentraut wrote: I like this work a lot. It's good to give developers easy to verify guidance about formatting the --help messages. However, I think instead of just adding a bunch of line breaks to satisfy the test, we should really try to make the lines shorter by

Re: RFC: Logging plan of the running query

2023-09-07 Thread torikoshia
On 2023-09-06 11:17, James Coleman wrote: > I've never been able to reproduce it (haven't tested the new version, > but v28 at least) on my M1 Mac; where I've reproduced it is on Debian > (first buster and now bullseye). > > I'm attaching several stacktraces in the hope that they provide some >

Re: RFC: Logging plan of the running query

2023-09-05 Thread torikoshia
On 2023-08-28 22:47, James Coleman wrote: On Mon, Aug 28, 2023 at 3:01 AM torikoshia wrote: On 2023-08-26 21:03, James Coleman wrote: > On Fri, Aug 25, 2023 at 7:43 AM James Coleman wrote: >> >> On Thu, Aug 17, 2023 at 10:02 AM torikoshia >> wrote: >> > &

Re: Make --help output fit within 80 columns per line

2023-08-31 Thread torikoshia
8-22 22:57, torikoshia wrote: On 2023-08-21 13:08, Masahiro Ikeda wrote: (2) Is there any reason that only src/bin commands are targeted? I found that we also need to fix vacuumlo with the above test. I think it's better to fix it because it's a contrib module. $ vacuumlo --help | while IFS=''

Re: pg_rewind WAL segments deletion pitfall

2023-08-29 Thread torikoshia
On 2023-08-24 09:45, Kyotaro Horiguchi wrote: At Wed, 23 Aug 2023 13:44:52 +0200, Alexander Kukushkin wrote in On Tue, 22 Aug 2023 at 07:32, Michael Paquier wrote: > I don't like much this patch. While it takes correctly advantage of > the backward record read logic from

Re: RFC: Logging plan of the running query

2023-08-28 Thread torikoshia
On 2023-08-26 21:03, James Coleman wrote: On Fri, Aug 25, 2023 at 7:43 AM James Coleman wrote: On Thu, Aug 17, 2023 at 10:02 AM torikoshia wrote: > > On 2023-06-16 01:34, James Coleman wrote: > > Attached is v28 > > which sets ProcessLogQueryPlanInterruptActive to

Re: pg_rewind WAL segments deletion pitfall

2023-08-23 Thread torikoshia
On 2023-08-22 14:32, Michael Paquier wrote: Thanks for your review! On Fri, Aug 18, 2023 at 03:40:57PM +0900, torikoshia wrote: Thanks for the patch, I've marked this as ready-for-committer. BTW, this issue can be considered a bug, right? I think it would be appropriate to provide backpatch

Re: Make --help output fit within 80 columns per line

2023-08-22 Thread torikoshia
On 2023-08-21 13:08, Masahiro Ikeda wrote: Thanks for your review! (1) Why don't you add test for the purpose? It could be overkill... I though the following function is the best place. diff --git a/src/test/perl/PostgreSQL/Test/Utils.pm b/src/test/perl/PostgreSQL/Test/Utils.pm index

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-08-21 Thread torikoshia
Since v5 patch failed applying anymore, updated the patch. On 2023-03-23 02:50, Andres Freund wrote: I suggest adding a few more tests: - COPY with a datatype error that can't be handled as a soft error I didn't know proper way to test this, but I've found data type widget's input function

Re: pg_rewind WAL segments deletion pitfall

2023-08-18 Thread torikoshia
to provide backpatch. On 2023-06-29 18:42, torikoshia wrote: On 2023-06-29 10:25, Kyotaro Horiguchi wrote: Thanks for the comment! At Wed, 28 Jun 2023 22:28:13 +0900, torikoshia wrote in On 2022-09-29 17:18, Polina Bungina wrote: > I agree with your suggestions, so here is the updated vers

Re: RFC: Logging plan of the running query

2023-08-17 Thread torikoshia
On 2023-06-16 01:34, James Coleman wrote: Attached is v28 which sets ProcessLogQueryPlanInterruptActive to false in errfinish when necessary. Once built with those two patches I'm simply running `make check`. With v28-0001 and v28-0002 patch, I confirmed backend processes consume huge amount

Re: Allow pg_archivecleanup to remove backup history files

2023-07-19 Thread torikoshia
On 2023-07-19 13:58, Michael Paquier wrote: On Fri, Jun 30, 2023 at 03:48:43PM +0900, Michael Paquier wrote: I have begun cleaning up my board, and applied 0001 for the moment. And a few weeks later.. I have come around this thread and applied 0002 and 0003. The flow of 0002 was

Make --help output fit within 80 columns per line

2023-07-04 Thread torikoshia
Hi, As discussed in [1], outputs of --help for some commands fits into 80 columns per line, while others do not. Since it seems preferable to have consistent line break policy and some people use 80-column terminal, wouldn't it be better to make all commands in 80 columns per line?

Re: pg_rewind WAL segments deletion pitfall

2023-06-29 Thread torikoshia
On 2023-06-29 10:25, Kyotaro Horiguchi wrote: Thanks for the comment! At Wed, 28 Jun 2023 22:28:13 +0900, torikoshia wrote in On 2022-09-29 17:18, Polina Bungina wrote: > I agree with your suggestions, so here is the updated version of > patch. Hope I haven't missed anything. >

Re: pg_rewind WAL segments deletion pitfall

2023-06-28 Thread torikoshia
On 2022-09-29 17:18, Polina Bungina wrote: I agree with your suggestions, so here is the updated version of patch. Hope I haven't missed anything. Regards, Polina Bungina Thanks for working on this! It seems like we are also facing the same issue. I tested the v3 patch under our condition,

Re: RFC: Logging plan of the running query

2023-06-27 Thread torikoshia
On 2023-06-16 01:34, James Coleman wrote: On Thu, Jun 15, 2023 at 9:00 AM torikoshia wrote: On 2023-06-15 01:48, James Coleman wrote: > On Tue, Jun 13, 2023 at 11:53 AM James Coleman > wrote: >> >> ... >> I'm going to re-run tests with my patch version + resetti

Re: Allow pg_archivecleanup to remove backup history files

2023-06-23 Thread torikoshia
On 2023-06-22 16:47, Kyotaro Horiguchi wrote: Thanks for your review! 0002: +* Check file name. +* +* We skip files which are not WAL file or partial WAL file. There's no need to spend this many lines describing this, and it's suggested to

Re: Allow pg_archivecleanup to remove backup history files

2023-06-21 Thread torikoshia
On 2023-06-21 11:59, Kyotaro Horiguchi wrote: At Tue, 20 Jun 2023 22:27:36 +0900, torikoshia wrote in On 2023-06-19 14:37, Michael Paquier wrote: > On Mon, Jun 19, 2023 at 11:24:29AM +0900, torikoshia wrote: >> Thanks, now I understand what you meant. > If I may ask, why is the

Re: Allow pg_archivecleanup to remove backup history files

2023-06-20 Thread torikoshia
On 2023-06-19 14:37, Michael Paquier wrote: On Mon, Jun 19, 2023 at 11:24:29AM +0900, torikoshia wrote: Thanks, now I understand what you meant. If I may ask, why is the refactoring of 0003 done after the feature in 0002? Shouldn't the order be reversed? That would make for a cleaner git

  1   2   3   >