Print logical WAL message content

Hi, Right now pg_waldump just prints whether the message is transactional or not and its size. That doesn't help much to understand the message itself. If it prints the contents of a logical WAL message, it helps debugging logical replication related problems. Prefix is a null-terminated ASCII

Re: display offset along with block number in vacuum errors

On Mon, Aug 17, 2020 at 11:38 AM Masahiko Sawada wrote: > > On Sat, 15 Aug 2020 at 12:19, Amit Kapila wrote: > > > > The reason why I have not included heap_page_prune related change in > > the patch is that I don't want to sprinkle this in every possible > > function (code path) called via

Re: doc examples for pghandler

On Mon, Aug 17, 2020 at 04:30:07PM -0700, Mark Wong wrote: > I've attached a small word diff to suggest a few different words to use > in the README, if that sounds better? Sounds good to me. So applied with those changes. It is really tempting to add an example of validator (one simple thing

Re: Terminate the idle sessions

On Aug 18, 2020, at 9:19 AM, Kyotaro Horiguchi mailto:horikyota@gmail.com>> wrote: The same already happens for idle_in_transaction_session_timeout and we can use "ALTER ROLE/DATABASE SET" to dislable or loosen them, it's a bit cumbersome, though. I don't think we should (at least

Re: Terminate the idle sessions

Hello. At Mon, 17 Aug 2020 19:28:10 +0530, Bharath Rupireddy wrote in > On Fri, Aug 14, 2020 at 1:32 PM Li Japin wrote: > > > > On Aug 14, 2020, at 2:15 PM, Bharath Rupireddy < > bharath.rupireddyforpostg...@gmail.com> wrote: > > > > I think, since the idle_session_timeout is by default

Re: EDB builds Postgres 13 with an obsolete ICU version

On Tue, Aug 18, 2020 at 09:44:35AM +0900, Michael Paquier wrote: > On Mon, Aug 17, 2020 at 02:23:57PM -0400, Bruce Momjian wrote: > > Also, is it the ICU library version we should be tracking for reindex, > > or each _collation_ version? If the later, do we store the collation > > version for

Re: EDB builds Postgres 13 with an obsolete ICU version

On Mon, Aug 17, 2020 at 02:23:57PM -0400, Bruce Momjian wrote: > Also, is it the ICU library version we should be tracking for reindex, > or each _collation_ version? If the later, do we store the collation > version for each index? You need to store the collation version(s) for each index.

Re: use pg_get_functiondef() in pg_dump

> > I'm sure there's a lot of folks who'd like to see more of the logic we > have in pg_dump for building objects from the catalog available to more > tools through libpgcommon- psql being one of the absolute first > use-cases for exactly that (there's certainly no shortage of people > who've

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

On 2020-Aug-14, Ibrar Ahmed wrote: > The table used for the test contains three columns (integer, text, > varchar). > The total number of rows is 1000 in total. > > Unpatched (Master: 92c12e46d5f1e25fc85608a6d6a19b8f5ea02600) > COPY: 9069.432 ms vacuum; 2567.961ms > COPY: 9004.533 ms vacuum:

Re: doc examples for pghandler

On Fri, Aug 14, 2020 at 02:25:52PM +0900, Michael Paquier wrote: > On Tue, Aug 11, 2020 at 01:01:10PM -0700, Mark Wong wrote: > > Ah, right. For the moment I've added some empty conditionals for > > trigger and event trigger handling. > > > > I've created a new entry in the commitfest app. [1]

Re: Improving connection scalability: GetSnapshotData()

On 2020-Aug-16, Peter Geoghegan wrote: > On Sun, Aug 16, 2020 at 2:11 PM Andres Freund wrote: > > For the first, one issue is that there's no obviously good candidate for > > an uninitialized xid. We could use something like FrozenTransactionId, > > which may never be in the procarray. But it's

Re: run pgindent on a regular basis / scripted manner

Andres Freund writes: > On 2020-08-15 13:44:34 -0400, Tom Lane wrote: >> Andres Freund writes: >>> One thing is that some here are actively against manually adding entries >>> to typedefs.list. >> I've been of the opinion that it's pointless to do so under the current >> regime where (a) only a

Re: [BUG] Error in BRIN summarization

On 2020-Aug-15, Tom Lane wrote: > hyrax's latest report suggests that this patch has issues under > CLOBBER_CACHE_ALWAYS: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hyrax=2020-08-13%2005%3A09%3A58 > > Hard to tell whether there's an actual bug there or just test instability, >

Re: Optimising compactify_tuples()

On Mon, Aug 17, 2020 at 4:01 AM Thomas Munro wrote: > While writing this email, I checked the archives and discovered that a > couple of other people have complained about this hot spot before and > proposed faster sorts already[2][3], and then there was a wide ranging > discussion of various

Re: EDB builds Postgres 13 with an obsolete ICU version

On Mon, Aug 17, 2020 at 04:55:13PM +0100, Dave Page wrote: > That was more if the installer actually handles the whole chain. It > clearly > doesn't today (since it doesn't support upgrades), I agree this might > definitely be overkill. But then also I don't really see the problem

Re: Corruption during WAL replay

Hi, On 2020-08-17 14:05:37 +0300, Heikki Linnakangas wrote: > On 14/04/2020 22:04, Teja Mupparti wrote: > > Thanks Kyotaro and Masahiko for the feedback. I think there is a > > consensus on the critical-section around truncate, > > +1 I'm inclined to think that we should do that independent of

Re: run pgindent on a regular basis / scripted manner

Hi, On 2020-08-15 13:44:34 -0400, Tom Lane wrote: > Andres Freund writes: > > One thing is that some here are actively against manually adding entries > > to typedefs.list. > > I've been of the opinion that it's pointless to do so under the current > regime where (a) only a few people do that

Re: run pgindent on a regular basis / scripted manner

Bruce Momjian writes: > On Sat, Aug 15, 2020 at 01:44:34PM -0400, Tom Lane wrote: >> Well, we can certainly do a tree-wide re-indent anytime we're ready. >> I doubt it would be very painful right now, with so little new work >> since the last run. > Uh, I thought Tom was saying we need to

Re: run pgindent on a regular basis / scripted manner

Hi, On 2020-08-17 13:54:15 -0400, Bruce Momjian wrote: > On Sat, Aug 15, 2020 at 01:44:34PM -0400, Tom Lane wrote: > > Andres Freund writes: > > > Without a properly indented baseline that's hard to do, because it'll > > > cause damage all over. So I don't think we easily can start just there -

Re: run pgindent on a regular basis / scripted manner

On Sat, Aug 15, 2020 at 01:44:34PM -0400, Tom Lane wrote: > Andres Freund writes: > > Without a properly indented baseline that's hard to do, because it'll > > cause damage all over. So I don't think we easily can start just there - > > we'd first need to re-indent everything. > > Well, we can

Re: Include access method in listTables output

On Sat, Aug 1, 2020 at 8:12 AM vignesh C wrote: > > > > > > +-- access method column should not be displayed for sequences > > > +\ds+ > > > > > > - List of relations > > > > > > > > > - Schema | Name | Type | Owner | Persistence | Size | Description > > >

Re: Add information to rm_redo_error_callback()

On 2020-Aug-17, Drouvot, Bertrand wrote: > Having this "pg_waldump" kind of format in this place > (rm_redo_error_callback()) ensures that we'll always see the same piece of > information during rm_redo. > > I think it's good to guarantee that we'll always see the same piece of > information

RE: Is it useful to record whether plans are generic or custom?

I thought it might be preferable to make a GUC to enable or disable this feature, but changing the hash key makes it harder. >> >>> What happens if the server was running with this option enabled and then >>> restarted with the option disabled? Firstly two entries for the same

Re: [PATCH] Covering SPGiST index

With a little bugfix вт, 11 авг. 2020 г. в 22:50, Pavel Borisov : > > > вт, 11 авг. 2020 г. в 12:11, Pavel Borisov : > >> I added changes in documentation into the patch. >> >> >> -- >> Best regards, >> Pavel Borisov >> >> Postgres Professional: http://postgrespro.com >>

Re: EDB builds Postgres 13 with an obsolete ICU version

On Mon, Aug 17, 2020 at 4:14 PM Magnus Hagander wrote: > > > On Mon, Aug 17, 2020 at 1:44 PM Dave Page wrote: > >> >> >> On Mon, Aug 17, 2020 at 11:19 AM Magnus Hagander >> wrote: >> >>> >>> >>> On Fri, Aug 14, 2020 at 3:00 PM Bruce Momjian wrote: >>> On Tue, Aug 11, 2020 at 02:58:30PM

Re: EDB builds Postgres 13 with an obsolete ICU version

On Mon, Aug 17, 2020 at 1:44 PM Dave Page wrote: > > > On Mon, Aug 17, 2020 at 11:19 AM Magnus Hagander > wrote: > >> >> >> On Fri, Aug 14, 2020 at 3:00 PM Bruce Momjian wrote: >> >>> On Tue, Aug 11, 2020 at 02:58:30PM +0200, Magnus Hagander wrote: >>> > On Tue, Aug 4, 2020 at 11:42 AM Dave

Re: One-off failure in "cluster" test

Thomas Munro writes: > Ahh, I see what's happening. You don't need a concurrent process > scanning *your* table for scan order to be nondeterministic. The > preceding CLUSTER command can leave the start block anywhere if its > call to ss_report_location() fails to acquire SyncScanLock >

Re: Newline after --progress report

On 17/08/2020 16:59, Tom Lane wrote: Heikki Linnakangas writes: Good point. Pushed a patch along those lines. Uh ... you patched v12 but not v13? Darn, I forgot it exists. Also, I'd recommend that you NOT do this: + fprintf(stderr, (!finished && isatty(fileno(stderr))) ? "\r" : "\n");

Re: Parallel bitmap index scan

On Mon, 17 Aug 2020 at 7:42 PM, Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > On Sun, Jul 26, 2020 at 6:43 PM Dilip Kumar wrote: > > > > > > I would like to propose a patch for enabling the parallelism for the > > > bitmap index scan path. > > > > > > Background: > > >

Re: Parallel bitmap index scan

On Sun, Jul 26, 2020 at 6:43 PM Dilip Kumar wrote: > > I would like to propose a patch for enabling the parallelism for the > bitmap index scan path. > > Background: > Currently, we support only a parallel bitmap heap scan path. Therein, > the underlying bitmap index scan is done by a single

Re: Improve planner cost estimations for alternative subplans

On Mon, Jun 22, 2020 at 9:39 AM Tom Lane wrote: > I wrote: > > Nope. The entire reason why we have that kluge is that we don't know > > until much later how many times we expect to execute the subplan. > > AlternativeSubPlan allows the decision which subplan form to use to be > > postponed till

Re: jsonb, collection & postgres_fdw

On Fri, Aug 14, 2020 at 12:46 PM Konstantin Knizhnik < k.knizh...@postgrespro.ru> wrote: > > Right now postgres_fdw treat as shippable only builtin functions or > functions from extensions explicitly specified as shippable extensions > in parameters of this FDW server. So I do no see a problem

Re: Is it useful to record whether plans are generic or custom?

On 2020/07/30 16:34, legrand legrand wrote: >> Main purpose is to decide (1) the user interface and (2) the way to get the plan type from pg_stat_statements. (1) the user interface I added a new boolean column 'generic_plan' to both pg_stat_statements view and the member of the hash key of

Re: Newline after --progress report

Heikki Linnakangas writes: > Good point. Pushed a patch along those lines. Uh ... you patched v12 but not v13? Also, I'd recommend that you NOT do this: + fprintf(stderr, (!finished && isatty(fileno(stderr))) ? "\r" : "\n"); as it breaks printf format verification in many/most compilers.

Re: Terminate the idle sessions

On Fri, Aug 14, 2020 at 1:32 PM Li Japin wrote: > > On Aug 14, 2020, at 2:15 PM, Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > > I think, since the idle_session_timeout is by default disabled, we > have no problem. My thought is what if a user enables the >

Re: Add header support to text format and matching feature

Thanks for your comments, Please find my thoughts inline. > In my tests it works fine except for one crash that I can reproduce > on a fresh build and default configuration with: > > $ cat >file.txt > i > 1 > > $ psql > postgres=# create table x(i int); > CREATE TABLE > postgres=# \copy x(i) from

Re: track_planning causing performance regression

On 2020/08/17 18:34, Hamid Akhtar wrote: On Mon, Aug 17, 2020 at 2:21 PM Fujii Masao mailto:masao.fu...@oss.nttdata.com>> wrote: On 2020/07/31 21:40, Hamid Akhtar wrote: > > > On Mon, Jul 6, 2020 at 10:29 AM Fujii Masao

Re: Creating a function for exposing memory usage of backend process

On 2020/08/17 21:14, Fujii Masao wrote: On 2020/08/11 15:24, torikoshia wrote: On 2020-08-08 10:44, Michael Paquier wrote: On Fri, Jul 31, 2020 at 03:23:52PM -0400, Robert Haas wrote: On Fri, Jul 31, 2020 at 4:25 AM torikoshia wrote: And as Fujii-san told me in person, exposing memory

Re: Creating a function for exposing memory usage of backend process

On 2020/08/11 15:24, torikoshia wrote: On 2020-08-08 10:44, Michael Paquier wrote: On Fri, Jul 31, 2020 at 03:23:52PM -0400, Robert Haas wrote: On Fri, Jul 31, 2020 at 4:25 AM torikoshia wrote: And as Fujii-san told me in person, exposing memory address seems not preferable considering

Re: EDB builds Postgres 13 with an obsolete ICU version

On Mon, Aug 17, 2020 at 11:19 AM Magnus Hagander wrote: > > > On Fri, Aug 14, 2020 at 3:00 PM Bruce Momjian wrote: > >> On Tue, Aug 11, 2020 at 02:58:30PM +0200, Magnus Hagander wrote: >> > On Tue, Aug 4, 2020 at 11:42 AM Dave Page wrote: >> > That would require fairly large changes to the

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

On Mon, Aug 17, 2020 at 2:19 PM Hamid Akhtar wrote: > Unfortunately the latest patch doesn't apply cleanly on the master branch. > Can you please share an updated one. Please see the attached patch rebased with master ( a28d731a1187e8d9d8c2b6319375fcbf0a8debd5) -- Ibrar Ahmed

Re: Corruption during WAL replay

On 14/04/2020 22:04, Teja Mupparti wrote: Thanks Kyotaro and Masahiko for the feedback. I think there is a consensus on the critical-section around truncate, +1 but I just want to emphasize the need for reversing the order of the dropping the buffers and the truncation.  Repro details

Optimising compactify_tuples()

Hi, With [1] applied so that you can get crash recovery to be CPU bound with a pgbench workload, we spend an awful lot of time in qsort(), called from compactify_tuples(). I tried replacing that with a specialised sort, and I got my test crash recovery time from ~55.5s down to ~49.5s quite

Re: EDB builds Postgres 13 with an obsolete ICU version

On Fri, Aug 14, 2020 at 3:00 PM Bruce Momjian wrote: > On Tue, Aug 11, 2020 at 02:58:30PM +0200, Magnus Hagander wrote: > > On Tue, Aug 4, 2020 at 11:42 AM Dave Page wrote: > > That would require fairly large changes to the installer to allow it > to > > login to the database server

Re: track_planning causing performance regression

On Mon, Aug 17, 2020 at 2:21 PM Fujii Masao wrote: > > > On 2020/07/31 21:40, Hamid Akhtar wrote: > > > > > > On Mon, Jul 6, 2020 at 10:29 AM Fujii Masao > wrote: > > > > > > > > On 2020/07/04 12:22, Pavel

Re: track_planning causing performance regression

On 2020/07/31 21:40, Hamid Akhtar wrote: On Mon, Jul 6, 2020 at 10:29 AM Fujii Masao mailto:masao.fu...@oss.nttdata.com>> wrote: On 2020/07/04 12:22, Pavel Stehule wrote: > > > pá 3. 7. 2020 v 13:02 odesílatel Fujii Masao

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

Unfortunately the latest patch doesn't apply cleanly on the master branch. Can you please share an updated one.

Re: proposal - reference to plpgsql_check from plpgsql doc

po 17. 8. 2020 v 10:37 odesílatel Magnus Hagander napsal: > > > On Mon, Aug 17, 2020 at 8:47 AM Pavel Stehule > wrote: > >> Hi >> >> plpgsql_check extension is almost complete now. This extension is >> available on all environments and for all supported Postgres releases. It >> is probably too

Re: proposal - reference to plpgsql_check from plpgsql doc

On Mon, Aug 17, 2020 at 8:47 AM Pavel Stehule wrote: > Hi > > plpgsql_check extension is almost complete now. This extension is > available on all environments and for all supported Postgres releases. It > is probably too big to be part of contrib, but I think so it can be > referenced in >

Making the function range_union_internal available to other extensions

Dear all In MobilityDB https://github.com/MobilityDB/MobilityDB we use extensively the range types. Is there any possibility to make the function range_union_internal available to use by other extensions ? Otherwise we need to copy/paste it verbatim. For example lines 114-153 in

Re: Commit/abort WAL records with dropped rels missing XLR_SPECIAL_REL_UPDATE

On 17/08/2020 10:00, Michael Paquier wrote: On Sat, Aug 15, 2020 at 11:05:43AM +0530, Amit Kapila wrote: On Fri, Aug 14, 2020 at 2:17 PM Heikki Linnakangas wrote: It's always been like that, but I am not going backport, for fear of breaking existing applications. If a program reads the WAL,

Re: Newline after --progress report

On 14/08/2020 16:51, Tom Lane wrote: Heikki Linnakangas writes: Attached is a patch to fix this, as well as a similar issue in pg_checksums. pg_basebackup and pgbench also print progres reports like this, but they seem correct to me. I wonder whether it'd be better to push the responsibility

Re: Commit/abort WAL records with dropped rels missing XLR_SPECIAL_REL_UPDATE

On Sat, Aug 15, 2020 at 11:05:43AM +0530, Amit Kapila wrote: > On Fri, Aug 14, 2020 at 2:17 PM Heikki Linnakangas wrote: >> It's always been like that, but I am not going backport, for fear of >> breaking existing applications. If a program reads the WAL, and would >> actually need to do

Re: More tests with USING INDEX replident and dropped indexes

On Wed, Jun 03, 2020 at 12:08:56PM -0300, Euler Taveira wrote: > Consistency is a good goal. Why don't we clear the relreplident from the > relation while dropping the index? relation_mark_replica_identity() already > does that but do other things too. Let's move the first code block from >

proposal - reference to plpgsql_check from plpgsql doc

Hi plpgsql_check extension is almost complete now. This extension is available on all environments and for all supported Postgres releases. It is probably too big to be part of contrib, but I think so it can be referenced in https://www.postgresql.org/docs/current/plpgsql-development-tips.html

proposal: enhancing plpgsql debug API - returns text value of variable content

Hi I am working on tracing support to plpgsql_check https://github.com/okbob/plpgsql_check I would like to print content of variables - and now, I have to go some deeper than I would like. I need to separate between scalar, row, and record variables. PLpgSQL has code for it - but it is private.

Re: Autovacuum on partitioned table (autoanalyze)

I'm sorry for the late reply. > This version seems to fail under Werror which is used in the Travis builds: > > autovacuum.c: In function ‘relation_needs_vacanalyze’: > autovacuum.c:3117:59: error: ‘reltuples’ may be used uninitialized in this > function [-Werror=maybe-uninitialized] >

Re: display offset along with block number in vacuum errors

On Sat, 15 Aug 2020 at 12:19, Amit Kapila wrote: > > On Fri, Aug 14, 2020 at 4:06 PM Amit Kapila wrote: > > > > On Mon, Aug 10, 2020 at 10:24 AM Masahiko Sawada > > wrote: > > > > > > It's true that heap_page_is_all_visible() is called from only > > > lazy_vacuum_page() but I'm concerned it

Re: recovering from "found xmin ... from before relfrozenxid ..."

Hello Masahiko-san, Thanks for the review. Please check the comments inline below: On Fri, Aug 14, 2020 at 10:07 AM Masahiko Sawada wrote: > Thank you for updating the patch! Here are my comments on v5 patch: > > --- a/contrib/Makefile > +++ b/contrib/Makefile > @@ -35,6 +35,7 @@ SUBDIRS = \ >