Re: [PATCH] Identify LWLocks in tracepoints

2021-03-10 Thread Peter Eisentraut
On 10.03.21 06:38, Craig Ringer wrote: On Wed, 3 Mar 2021 at 20:50, David Steele > wrote: On 1/22/21 6:02 AM, Peter Eisentraut wrote: This patch set no longer applies: http://cfbot.cputube.org/patch_32_2927.log

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-10 Thread Amit Langote
On Wed, Mar 10, 2021 at 6:18 PM Amit Kapila wrote: > On Mon, Mar 8, 2021 at 7:19 PM Amit Langote wrote: > > I just read through v25 and didn't find anything to complain about. > > Thanks a lot, pushed now! Amit L., your inputs are valuable for this work. Glad I could be of help. Really

Re: POC: Cleaning up orphaned files using undo logs

2021-03-10 Thread Antonin Houska
tsunakawa.ta...@fujitsu.com wrote: > I'm crawling like a snail to read the patch set. Below are my first set of > review comments, which are all minor. Thanks. I will check your comments when I'll be preparing the next version of the patch. -- Antonin Houska Web:

Re: Queries for PostgreSQL DDL convert

2021-03-10 Thread Julien Rouhaud
On Wed, Mar 10, 2021 at 04:09:49PM +0800, 杨逸存 wrote: > Dear hacker: > I am a student from Nanjing University. I have some troubles > about DDL statement convertion. I have several MySQL DDL statements > fromMySQL dump command. Now I wanna convert the statements' grammar so > that they can be

Re: [PATCH] pg_permissions

2021-03-10 Thread Joel Jacobson
New version attached. Changes: * Added documentation in catalogs.sgml * Dropped "objsubid" from pg_ownerships since columns have no owner, only tables Do we prefer "pg_permissions" or "pg_privileges"? I can see "privileges" occur 2325 times in the sources, while "permissions" occur only 1097

Re: Freeze the inserted tuples during CTAS?

2021-03-10 Thread Masahiko Sawada
On Wed, Mar 10, 2021 at 3:57 PM Paul Guo wrote: > > > On Mar 3, 2021, at 1:35 PM, Masahiko Sawada wrote: > >> On Sun, Feb 21, 2021 at 4:46 PM Paul Guo wrote: > >> Attached is the v2 version that fixes a test failure due to plan change > >> (bitmap index scan -> index only scan). > > > I think

RE: Parallel INSERT (INTO ... SELECT ...)

2021-03-10 Thread tsunakawa.ta...@fujitsu.com
From: Amit Kapila > Now, coming back to Hou-San's patch to introduce a GUC and reloption > for this feature, I think both of those make sense to me because when > the feature is enabled via GUC, one might want to disable it for > partitioned tables? Do we agree on that part or someone thinks >

Re: PostmasterIsAlive() in recovery (non-USE_POST_MASTER_DEATH_SIGNAL builds)

2021-03-10 Thread Michael Paquier
On Thu, Mar 11, 2021 at 04:37:39PM +1300, Thomas Munro wrote: > Michael, when you said "That's pretty hack-ish, still efficient" in > reference to this code: > >> - if (IsUnderPostmaster && !PostmasterIsAlive()) >> + if (IsUnderPostmaster && >> +#ifndef USE_POSTMASTER_DEATH_SIGNAL >> +

Re: shared-memory based stats collector

2021-03-10 Thread Fujii Masao
On 2021/03/11 13:42, Kyotaro Horiguchi wrote: At Wed, 10 Mar 2021 19:21:00 -0800, Andres Freund wrote in Hi, Two minor nits: Thanks for the comments! On 2021-03-10 21:47:51 +0900, Fujii Masao wrote: +/* Shared memory area for archiver process */ +typedef struct PgArchData +{ +

Re: Huge memory consumption on partitioned table with FKs

2021-03-10 Thread Tatsuro Yamada
On 2021/03/11 9:39, Amit Langote wrote: On Thu, Mar 11, 2021 at 4:25 AM Tom Lane wrote: Amit Langote writes: On Wed, Mar 10, 2021 at 8:37 AM Tom Lane wrote: Hmm. So, the key point is that the values coming from the partitioned child table are injected into the test query as parameters,

Re: shared-memory based stats collector

2021-03-10 Thread Kyotaro Horiguchi
At Wed, 10 Mar 2021 19:21:00 -0800, Andres Freund wrote in > Hi, > > Two minor nits: Thanks for the comments! > On 2021-03-10 21:47:51 +0900, Fujii Masao wrote: > > +/* Shared memory area for archiver process */ > > +typedef struct PgArchData > > +{ > > + Latch *latch;

Re: [HACKERS] Custom compression methods

2021-03-10 Thread Dilip Kumar
On Thu, Mar 11, 2021 at 8:50 AM Justin Pryzby wrote: > > Looking at v23-0002-alter-table-set-compression, ATRewriteTable() was calling > CompareCompressionMethodAndDecompress(). While changing the compression method user might be just interested to compress the future tuple with the new

Re: libpq debug log

2021-03-10 Thread Kyotaro Horiguchi
At Thu, 11 Mar 2021 04:12:57 +, "tsunakawa.ta...@fujitsu.com" wrote in > From: Kyotaro Horiguchi > > Right. So something like this? > > > > unsigned char p; > > > > p = buf + *cursor; > > result = (uint32) (*p << 24) + (*(p + 1)) << 16 + ...); > > Yes, that would work (if p is a

Re: shared-memory based stats collector

2021-03-10 Thread Andres Freund
Hi, On 2021-03-10 17:51:37 +0900, Kyotaro Horiguchi wrote: > From ed2fb2fca47fccbf9af1538688aab8334cf6470c Mon Sep 17 00:00:00 2001 > From: Kyotaro Horiguchi > Date: Fri, 13 Mar 2020 16:58:03 +0900 > Subject: [PATCH v52 1/7] sequential scan for dshash > > Dshash did not allow scan the all

RE: libpq debug log

2021-03-10 Thread tsunakawa.ta...@fujitsu.com
From: Kyotaro Horiguchi > Right. So something like this? > > unsigned char p; > > p = buf + *cursor; > result = (uint32) (*p << 24) + (*(p + 1)) << 16 + ...); Yes, that would work (if p is a pointer), but I think memcpy() is enough like pqGetInt() does. Regards Takayuki Tsunakawa

Re: [PATCH] Provide more information to filter_prepare

2021-03-10 Thread Amit Kapila
On Wed, Mar 10, 2021 at 4:26 PM Markus Wanner wrote: > > On 10.03.21 11:18, Amit Kapila wrote: > > On Tue, Mar 9, 2021 at 2:14 PM Markus Wanner > > wrote: > >> currently, only the gid is passed on to the filter_prepare callback. > >> While we probably should not pass a full ReorderBufferTXN (as

Re: PostmasterIsAlive() in recovery (non-USE_POST_MASTER_DEATH_SIGNAL builds)

2021-03-10 Thread Thomas Munro
On Tue, Mar 2, 2021 at 2:10 PM Thomas Munro wrote: > ... One question I haven't > got to the bottom of: is it a problem for the startup process that CVs > use CHECK_FOR_INTERRUPTS()? This was a red herring. The startup process already reaches CFI() via various paths, as I figured out pretty

Re: shared-memory based stats collector

2021-03-10 Thread Andres Freund
Hi, Two minor nits: On 2021-03-10 21:47:51 +0900, Fujii Masao wrote: > +/* Shared memory area for archiver process */ > +typedef struct PgArchData > +{ > + Latch *latch; /* latch to wake the archiver > up */ > + slock_t mutex; /* locks

Re: [HACKERS] Custom compression methods

2021-03-10 Thread Justin Pryzby
On Thu, Mar 11, 2021 at 08:17:46AM +0530, Dilip Kumar wrote: > On Thu, Mar 11, 2021 at 2:21 AM Robert Haas wrote: > > > > On Wed, Mar 10, 2021 at 6:52 AM Dilip Kumar wrote: > > > The pending comment is providing a way to rewrite a table and > > > re-compress the data with the current compression

Re: libpq debug log

2021-03-10 Thread Kyotaro Horiguchi
At Thu, 11 Mar 2021 03:01:16 +, "tsunakawa.ta...@fujitsu.com" wrote in > From: Kyotaro Horiguchi > > The output functions copy message bytes into local variable but the > > same effect can be obtained by just casing via typed pointer type. > > > > uint32 tmp4; > > .. > > memcpy(,

Re: Replace buffer I/O locks with condition variables (reviving an old patch)

2021-03-10 Thread Julien Rouhaud
On Thu, Mar 11, 2021 at 03:54:06PM +1300, Thomas Munro wrote: > On Thu, Mar 11, 2021 at 3:27 PM Julien Rouhaud wrote: > > - /* > > -* It would be nice to include the I/O locks in the BufferDesc, but that > > -* would increase the size of a BufferDesc to more than one cache line, > > -

Re: libpq debug log

2021-03-10 Thread Kyotaro Horiguchi
At Thu, 11 Mar 2021 01:51:55 +, "tsunakawa.ta...@fujitsu.com" wrote in > Alvaro-san, Tom-san, Horiguchi-san, > > From: Kyotaro Horiguchi > > +1 for the thanks for the quick work. I have some random comments > > after a quick look on it. > > Thank you very much for giving many comments.

Re: libpq debug log

2021-03-10 Thread Kyotaro Horiguchi
At Wed, 10 Mar 2021 20:38:20 -0500, Tom Lane wrote in > Kyotaro Horiguchi writes: > > Maybe I'm missing something, but the above doesn't seem working. I > > didn't see such message when making a SSL connection. > > As far as that goes, I don't see any point in trying to trace >

RE: libpq debug log

2021-03-10 Thread tsunakawa.ta...@fujitsu.com
From: Kyotaro Horiguchi > The output functions copy message bytes into local variable but the > same effect can be obtained by just casing via typed pointer type. > > uint32 tmp4; > .. > memcpy(, buf + *cursor, 4); > result = (int) pg_ntoh32(tmp4); > > can be written as > > result =

Re: Replace buffer I/O locks with condition variables (reviving an old patch)

2021-03-10 Thread Thomas Munro
On Thu, Mar 11, 2021 at 3:27 PM Julien Rouhaud wrote: > - /* > -* It would be nice to include the I/O locks in the BufferDesc, but that > -* would increase the size of a BufferDesc to more than one cache line, > -* and benchmarking has shown that keeping every BufferDesc aligned on

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-03-10 Thread Fujii Masao
On 2021/03/11 9:38, Masahiro Ikeda wrote: On 2021-03-10 17:08, Fujii Masao wrote: On 2021/03/10 14:11, Masahiro Ikeda wrote: On 2021-03-09 17:51, Fujii Masao wrote: On 2021/03/05 8:38, Masahiro Ikeda wrote: On 2021-03-05 01:02, Fujii Masao wrote: On 2021/03/04 16:14, Masahiro Ikeda wrote:

Re: [HACKERS] Custom compression methods

2021-03-10 Thread Dilip Kumar
On Thu, Mar 11, 2021 at 2:21 AM Robert Haas wrote: > > On Wed, Mar 10, 2021 at 6:52 AM Dilip Kumar wrote: > > The pending comment is providing a way to rewrite a table and > > re-compress the data with the current compression method. > > I spent some time poking at this yesterday and ran

RE: Parallel INSERT (INTO ... SELECT ...)

2021-03-10 Thread houzj.f...@fujitsu.com
> > 2. Should we keep the default value of GUC to on or off? It is > > currently off. I am fine keeping it off for this release and we can > > always turn it on in the later releases if required. Having said that, > > I see the value in keeping it on because in many cases Insert ... > > Select

Re: [HACKERS] Custom compression methods

2021-03-10 Thread Justin Pryzby
This includes a patch to use pkgconfig, in an attempt to build on mac, which currently fails like: https://cirrus-ci.com/task/5993712963551232?command=build#L126 checking for LZ4_compress in -llz4... no configure: error: library 'lz4' is required for LZ4 support -- Justin >From

Re: Replace buffer I/O locks with condition variables (reviving an old patch)

2021-03-10 Thread Julien Rouhaud
On Thu, Mar 11, 2021 at 10:48:40AM +1300, Thomas Munro wrote: > On Tue, Mar 9, 2021 at 6:24 PM Julien Rouhaud wrote: > > > > +1 for adding the cv into BufferDesc. That brings the struct size to > > exactly > > 64 bytes on x86 64 bits architecture. This won't add any extra overhead to > >

Re: Removing vacuum_cleanup_index_scale_factor

2021-03-10 Thread Masahiko Sawada
On Thu, Mar 11, 2021 at 10:12 AM Peter Geoghegan wrote: > > On Tue, Mar 9, 2021 at 7:42 PM Peter Geoghegan wrote: > > My current plan is to commit everything within the next day or two. > > This includes backpatching to Postgres 13 only. > > Pushed, thanks. Great! Thank you! Regards, --

RE: libpq debug log

2021-03-10 Thread tsunakawa.ta...@fujitsu.com
Alvaro-san, Tom-san, Horiguchi-san, From: Kyotaro Horiguchi > +1 for the thanks for the quick work. I have some random comments > after a quick look on it. Thank you very much for giving many comments. And We're sorry to have caused you trouble. I told Iwata-san yesterday to modify the

Re: libpq debug log

2021-03-10 Thread Tom Lane
Kyotaro Horiguchi writes: > Maybe I'm missing something, but the above doesn't seem working. I > didn't see such message when making a SSL connection. As far as that goes, I don't see any point in trying to trace connection-related messages, because there is no way to enable tracing on a

Re: 64-bit XIDs in deleted nbtree pages

2021-03-10 Thread Peter Geoghegan
On Mon, Mar 1, 2021 at 7:25 PM Peter Geoghegan wrote: > Attached is v8 of the patch series, which has new patches. No real > changes compared to v7 for the first patch, though. Here is another bitrot-fix-only revision, v9. Just the recycling patch again. I'll commit this when we get your patch

Re: fdatasync performance problem with large number of DB files

2021-03-10 Thread Thomas Munro
On Thu, Mar 11, 2021 at 2:25 PM Tom Lane wrote: > Trolling the net, I found a newer-looking version of the man page, > and behold it says > >In mainline kernel versions prior to 5.8, syncfs() will fail only >when passed a bad file descriptor (EBADF). Since Linux 5.8, >

Re: fdatasync performance problem with large number of DB files

2021-03-10 Thread Tom Lane
Thomas Munro writes: > On Thu, Mar 11, 2021 at 1:16 PM Tom Lane wrote: >> I'm a little skeptical about the "simple" part. At minimum, you'd >> have to syncfs() each tablespace, since we have no easy way to tell >> which of them are on different filesystems. (Although, if we're >> presuming

Re: fdatasync performance problem with large number of DB files

2021-03-10 Thread Thomas Munro
On Thu, Mar 11, 2021 at 2:00 PM Fujii Masao wrote: > On 2021/03/11 8:30, Thomas Munro wrote: > > I've run into a couple of users who have just commented that recursive > > fsync() code out! > > BTW, we can skip that recursive fsync() by disabling fsync GUC even without > commenting out the code?

Re: libpq debug log

2021-03-10 Thread Kyotaro Horiguchi
At Wed, 10 Mar 2021 10:31:27 -0300, "'alvhe...@alvh.no-ip.org'" wrote in > On 2021-Mar-10, iwata@fujitsu.com wrote: > > > Hi all, > > > > Following all reviewer's advice, I have created a new patch. > > > > In this patch, I add only two tracing entry points; I call > >

Re: Removing vacuum_cleanup_index_scale_factor

2021-03-10 Thread Peter Geoghegan
On Tue, Mar 9, 2021 at 7:42 PM Peter Geoghegan wrote: > My current plan is to commit everything within the next day or two. > This includes backpatching to Postgres 13 only. Pushed, thanks. -- Peter Geoghegan

Re: Huge memory consumption on partitioned table with FKs

2021-03-10 Thread Keisuke Kuroda
Hi hackers, > > > > I did some cosmetic fooling with this (mostly, rewriting the comments > > YA time) and pushed it. > > Perfect. Thanks for your time on this. > Thank you for your help! I'm glad to solve it. -- Keisuke Kuroda NTT Software Innovation Center keisuke.kuroda.3...@gmail.com

RE: Enhance traceability of wal_level changes for backup management

2021-03-10 Thread osumi.takami...@fujitsu.com
On Tuesday, March 9, 2021 11:13 PM David Steele > On 3/7/21 9:45 PM, osumi.takami...@fujitsu.com wrote: > > On Sun, Mar 7, 2021 3:48 AM Peter Eisentraut > wrote: > >> On 28.01.21 01:44, osumi.takami...@fujitsu.com wrote: > (1) writing the time or LSN in the control file to indicate >

Re: Boundary value check in lazy_tid_reaped()

2021-03-10 Thread Masahiko Sawada
On Wed, Mar 10, 2021 at 11:53 PM Peter Eisentraut wrote: > > On 10.03.21 02:29, Masahiko Sawada wrote: > >> There is no noticeable effect of inlining lazy_tid_reaped(). So it > >> would be better to not do that. > > > > Attached the patch that doesn't inline lazy_tid_reaped(). > > committed

Re: fdatasync performance problem with large number of DB files

2021-03-10 Thread Fujii Masao
On 2021/03/11 8:30, Thomas Munro wrote: On Thu, Mar 11, 2021 at 11:38 AM Thomas Munro wrote: On Thu, Mar 11, 2021 at 11:01 AM Michael Brown wrote: * is there a knob missing we can configure? * can we get an opt-in knob to use a single sync() call instead of a recursive fsync()? * would

Re: pgbench - add pseudo-random permutation function

2021-03-10 Thread Bruce Momjian
On Mon, Mar 8, 2021 at 11:50:43AM +0100, Fabien COELHO wrote: > > > > What are your current thoughts? > > > > Thanks for prodding. I still think it's a useful feature. However I > > don't think I'll have to time to get it done on the current commitfest. > > I suggest to let it sit in the

Re: shared-memory based stats collector

2021-03-10 Thread Kyotaro Horiguchi
At Wed, 10 Mar 2021 21:47:51 +0900, Fujii Masao wrote in > Attached is the updated version of the 0003 patch. Barring any > objection, > I will commit this patch. > > > -#include "storage/latch.h" > -#include "storage/proc.h" > > I removed these because they are no longer necessary. Mmm.

Re: Improve join selectivity estimation using extended statistics

2021-03-10 Thread Tomas Vondra
Hi Konstantin, Thanks for working on this! Using extended statistics to improve join cardinality estimates was definitely on my radar, and this patch seems like a good start. I had two basic ideas about how we might improve join estimates: (a) use per-table extended statistics to estimate join

Re: [HACKERS] Custom compression methods

2021-03-10 Thread Justin Pryzby
On Wed, Mar 10, 2021 at 03:50:48PM -0500, Robert Haas wrote: > On Wed, Mar 10, 2021 at 6:52 AM Dilip Kumar wrote: > > The pending comment is providing a way to rewrite a table and > > re-compress the data with the current compression method. > > I spent some time poking at this yesterday and ran

Re: Huge memory consumption on partitioned table with FKs

2021-03-10 Thread Amit Langote
On Thu, Mar 11, 2021 at 4:25 AM Tom Lane wrote: > Amit Langote writes: > > On Wed, Mar 10, 2021 at 8:37 AM Tom Lane wrote: > >> Hmm. So, the key point is that the values coming from the partitioned > >> child table are injected into the test query as parameters, not as > >> column references,

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-03-10 Thread Masahiro Ikeda
On 2021-03-10 17:08, Fujii Masao wrote: On 2021/03/10 14:11, Masahiro Ikeda wrote: On 2021-03-09 17:51, Fujii Masao wrote: On 2021/03/05 8:38, Masahiro Ikeda wrote: On 2021-03-05 01:02, Fujii Masao wrote: On 2021/03/04 16:14, Masahiro Ikeda wrote: On 2021-03-03 20:27, Masahiro Ikeda wrote:

Re: fdatasync performance problem with large number of DB files

2021-03-10 Thread Thomas Munro
On Thu, Mar 11, 2021 at 1:16 PM Tom Lane wrote: > Thomas Munro writes: > > Thinking about this some more, if you were to propose a patch like > > that syncfs() one but make it a configurable option, I'd personally be > > in favour of trying to squeeze it into v14. Others might object on > >

Re: fdatasync performance problem with large number of DB files

2021-03-10 Thread Tom Lane
Thomas Munro writes: > Thinking about this some more, if you were to propose a patch like > that syncfs() one but make it a configurable option, I'd personally be > in favour of trying to squeeze it into v14. Others might object on > commitfest procedural grounds, I dunno, but I think this is a

Re: fdatasync performance problem with large number of DB files

2021-03-10 Thread Thomas Munro
On Thu, Mar 11, 2021 at 11:38 AM Thomas Munro wrote: > On Thu, Mar 11, 2021 at 11:01 AM Michael Brown > wrote: > > * is there a knob missing we can configure? > > * can we get an opt-in knob to use a single sync() call instead of a > > recursive fsync()? > > * would you be open to merging a

Re: libpq debug log

2021-03-10 Thread Tom Lane
Alvaro Herrera writes: > On 2021-Mar-10, Tom Lane wrote: >> After studying this further, I think we should apply the attached >> patch to remove that responsibility from pqParseInput3's subroutines. >> This will allow a single trace call near the bottom of pqParseInput3 >> to handle all cases

Re: PROXY protocol support

2021-03-10 Thread Jacob Champion
On Tue, 2021-03-09 at 11:25 +0100, Magnus Hagander wrote: > I've also added some trivial tests (man that took an ungodly amount of > fighting perl -- it's clearly been a long time since I used perl > properly). Yeah. The tests I'm writing for this and NSS have been the same way; it's a real

Re: Columns correlation and adaptive query optimization

2021-03-10 Thread Tomas Vondra
On 3/10/21 3:00 AM, Tomas Vondra wrote: > Hello Konstantin, > > > Sorry for not responding to this thread earlier. I definitely agree the > features proposed here are very interesting and useful, and I appreciate > you kept rebasing the patch. > > I think the patch improving join estimates can

Re: Speeding up GIST index creation for tsvectors

2021-03-10 Thread John Naylor
I wrote: > I'll post a patch soon that builds on that, so you can see what I mean. I've attached where I was imagining this heading, as a text file to avoid distracting the cfbot. Here are the numbers I got with your test on the attached, as well as your 0001, on x86-64 Clang 10, default siglen:

Re: fdatasync performance problem with large number of DB files

2021-03-10 Thread Thomas Munro
On Thu, Mar 11, 2021 at 11:01 AM Michael Brown wrote: > * pg_basebackup receives a streaming backup (via [2] fsync_dir_recurse > or fsync_pgdata) unless --no-sync is specified > * postgres starts up unclean (via [3] SyncDataDirectory) > > We run multiple postgres clusters and some of those

Re: automatic analyze: readahead - add "IO read time" log message

2021-03-10 Thread Tomas Vondra
On 3/8/21 8:42 PM, Stephen Frost wrote: > Greetings, > > * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: >> On 2/10/21 11:10 PM, Stephen Frost wrote: >>> * Heikki Linnakangas (hlinn...@iki.fi) wrote: On 05/02/2021 23:22, Stephen Frost wrote: > Unless there's anything else on this,

Re: libpq debug log

2021-03-10 Thread Alvaro Herrera
On 2021-Mar-10, Tom Lane wrote: > After studying this further, I think we should apply the attached > patch to remove that responsibility from pqParseInput3's subroutines. > This will allow a single trace call near the bottom of pqParseInput3 > to handle all cases that that function processes.

Re: GiST comment improvement

2021-03-10 Thread Bruce Momjian
On Tue, Mar 2, 2021 at 11:40:21AM -0500, Bruce Momjian wrote: > In talking to Teodor this week, I have written the attached C comment > patch which improves our description of GiST's NSN and GistBuildLSN > values. Patch applied. -- Bruce Momjian https://momjian.us EDB

fdatasync performance problem with large number of DB files

2021-03-10 Thread Michael Brown
I initially posted this on the pgsql-general mailing list [5] but was advised to also post this to the -hackers list as it deals with internals. We've encountered a production performance problem with pg13 related to how it fsyncs the whole data directory in certain scenarios, related to what

Re: Replace buffer I/O locks with condition variables (reviving an old patch)

2021-03-10 Thread Thomas Munro
On Tue, Mar 9, 2021 at 6:24 PM Julien Rouhaud wrote: > > The old I/O lock array was the only user of struct > > LWLockMinimallyPadded, added in commit 6150a1b08a9, and it seems kinda > > strange to leave it in the tree with no user. Of course it's remotely > > possible there are extensions using

Re: libpq debug log

2021-03-10 Thread Tom Lane
I wrote: > Or we could rethink the logic. It's not quite clear to me, after > all this time, why getRowDescriptions() et al are allowed to > move inStart themselves rather than letting the main loop in > pqParseInput3 do it. It might well be an artifact of having not > rewritten the v2 logic too

Re: libpq debug log

2021-03-10 Thread 'alvhe...@alvh.no-ip.org'
On 2021-Mar-10, Tom Lane wrote: > Or we could rethink the logic. It's not quite clear to me, after > all this time, why getRowDescriptions() et al are allowed to > move inStart themselves rather than letting the main loop in > pqParseInput3 do it. It might well be an artifact of having not >

Re: libpq debug log

2021-03-10 Thread 'alvhe...@alvh.no-ip.org'
On 2021-Mar-10, Tom Lane wrote: > "'alvhe...@alvh.no-ip.org'" writes: > > After staring at it a couple of times, I think that the places in > > pqParseInput3() where there's a comment "... moves inStart itself" and > > then "continue;" should have a call to pqTraceOutputMsg(), since AFAIU > >

Re: [HACKERS] Custom compression methods

2021-03-10 Thread Robert Haas
On Wed, Mar 10, 2021 at 6:52 AM Dilip Kumar wrote: > The pending comment is providing a way to rewrite a table and > re-compress the data with the current compression method. I spent some time poking at this yesterday and ran couldn't figure out what was going on here. There are two places where

Re: pg_amcheck contrib application

2021-03-10 Thread Robert Haas
On Wed, Mar 10, 2021 at 11:10 AM Mark Dilger wrote: > Once again, I think you are right and have removed the objectionable > behavior, but > > The --startblock and --endblock options make the most sense when the user is > only checking one table, like > > pg_amcheck --startblock=17

Re: libpq debug log

2021-03-10 Thread Tom Lane
"'alvhe...@alvh.no-ip.org'" writes: > After staring at it a couple of times, I think that the places in > pqParseInput3() where there's a comment "... moves inStart itself" and > then "continue;" should have a call to pqTraceOutputMsg(), since AFAIU > those are the places where the message in

Re: Occasional tablespace.sql failures in check-world -jnn

2021-03-10 Thread Andres Freund
Hi, On 2021-03-10 15:40:38 +0900, Michael Paquier wrote: > On Mon, Mar 08, 2021 at 11:53:57AM +0100, Peter Eisentraut wrote: > > On 09.12.20 08:55, Michael Paquier wrote: > >> ... Because we may still introduce this problem again if some new > >> stuff uses src/test/pg_regress in a way similar

Re: libpq debug log

2021-03-10 Thread Tom Lane
"'alvhe...@alvh.no-ip.org'" writes: > After staring at it a couple of times, I think that the places in > pqParseInput3() where there's a comment "... moves inStart itself" and > then "continue;" should have a call to pqTraceOutputMsg(), since AFAIU > those are the places where the message in

Re: libpq debug log

2021-03-10 Thread 'alvhe...@alvh.no-ip.org'
On 2021-Mar-10, 'alvhe...@alvh.no-ip.org' wrote: > I also found that DataRow messages are not being printed. This seems to > fix that in the normal case and singlerow, at least in pipeline mode. > Didn't verify the non-pipeline mode. Hm, and RowDescription ('T') messages are also not being

Re: Huge memory consumption on partitioned table with FKs

2021-03-10 Thread Tom Lane
Amit Langote writes: > On Wed, Mar 10, 2021 at 8:37 AM Tom Lane wrote: >> Hmm. So, the key point is that the values coming from the partitioned >> child table are injected into the test query as parameters, not as >> column references, thus it doesn't matter *to the test query* what >> numbers

Re: Speeding up GIST index creation for tsvectors

2021-03-10 Thread John Naylor
On Mon, Mar 8, 2021 at 8:43 AM Amit Khandekar wrote: > > On Wed, 3 Mar 2021 at 23:32, John Naylor wrote: > > 0001: > > > > + /* > > + * We can process 64-bit chunks only if both are mis-aligned by the same > > + * number of bytes. > > + */ > > + if (b_aligned - b == a_aligned - a) > > > > The

Re: get rid of tags in the docs?

2021-03-10 Thread Tom Lane
John Naylor writes: > On Thu, Feb 4, 2021 at 11:31 AM Tom Lane wrote: >> +1, nobody italicizes those in normal usage. > Now that protocol v2 is gone, here's a patch to remove those tags. Pushed. regards, tom lane

Re: Consider parallel for lateral subqueries with limit

2021-03-10 Thread David Steele
On 12/7/20 6:45 PM, James Coleman wrote: On Sun, Dec 6, 2020 at 7:34 PM Brian Davis wrote: Played around with this a bit, here's a non-correlated subquery that gets us to that if statement While I haven't actually tracked down to guarantee this is handled elsewhere, a thought experiment --

Re: Add header support to text format and matching feature

2021-03-10 Thread David Steele
On 12/7/20 6:40 PM, Rémi Lapeyre wrote: Hi, here’s a rebased version of the patch. Michael, since the issue of duplicated options has been fixed do either of these patches look like they are ready for commit? Regards, -- -David da...@pgmasters.net

Re: cryptohash: missing locking functions for OpenSSL <= 1.0.2?

2021-03-10 Thread Jacob Champion
On Wed, 2021-03-10 at 17:21 +0900, Michael Paquier wrote: > Do you mean something like the attached? Yes! Patch LGTM. --Jacob

Re: WIP: System Versioned Temporal Table

2021-03-10 Thread Vik Fearing
On 3/10/21 5:49 PM, Surafel Temesgen wrote: > hi Ibrar, > thank you for rebasing > > On Mon, Mar 8, 2021 at 9:34 AM Ibrar Ahmed wrote: > >> >>> Since the get_row_start_time_col_name() and get_row_end_time_col_name() >>> are similar, IMO we can pass a flag to get StartTime/EndTime column name,

Re: Evaluate expression at planning time for two more cases

2021-03-10 Thread Surafel Temesgen
Hi Ibrar, On Mon, Mar 8, 2021 at 8:13 AM Ibrar Ahmed wrote: > > It was a minor change therefore I rebased the patch, please take a look. > It is perfect thank you regards Surafel

Re: WIP: System Versioned Temporal Table

2021-03-10 Thread Surafel Temesgen
hi Ibrar, thank you for rebasing On Mon, Mar 8, 2021 at 9:34 AM Ibrar Ahmed wrote: > >> Since the get_row_start_time_col_name() and get_row_end_time_col_name() >> are similar, IMO we can pass a flag to get StartTime/EndTime column name, >> thought? >> >> For me your option is better. i will

Re: WIP: document the hook system

2021-03-10 Thread David Fetter
On Wed, Mar 10, 2021 at 09:38:39AM -0500, David Steele wrote: > On 3/9/21 12:20 PM, Bruce Momjian wrote: > > On Sat, Mar 6, 2021 at 08:32:43PM -0500, Tom Lane wrote: > > > I think that the best you should hope for here is that people are > > > willing to add a short, not-too-detailed para to a

Re: [PATCH] Covering SPGiST index

2021-03-10 Thread David Steele
On 12/4/20 12:31 PM, Pavel Borisov wrote: The cfbot's still unhappy --- looks like you omitted a file from the patch? You are right, thank you. Corrected this. PFA v13 Tom, do the changes as enumerated in [1] look like they are going in the right direction? Regards, -- -David

Re: Huge memory consumption on partitioned table with FKs

2021-03-10 Thread Amit Langote
On Wed, Mar 10, 2021 at 8:37 AM Tom Lane wrote: > Amit Langote writes: > > On Fri, Mar 5, 2021 at 6:00 AM Tom Lane wrote: > >> This claim seems false on its face: > >>> All child constraints of a given foreign key constraint can use the > >>> same RI query and the resulting plan, that is, no

Re: Boundary value check in lazy_tid_reaped()

2021-03-10 Thread Peter Eisentraut
On 10.03.21 02:29, Masahiko Sawada wrote: There is no noticeable effect of inlining lazy_tid_reaped(). So it would be better to not do that. Attached the patch that doesn't inline lazy_tid_reaped(). committed

Re: WIP: document the hook system

2021-03-10 Thread David Steele
On 3/9/21 12:20 PM, Bruce Momjian wrote: On Sat, Mar 6, 2021 at 08:32:43PM -0500, Tom Lane wrote: I think that the best you should hope for here is that people are willing to add a short, not-too-detailed para to a markup-free plain-text README file that lists all the hooks. As soon as it

Re: libpq debug log

2021-03-10 Thread 'alvhe...@alvh.no-ip.org'
I also found that DataRow messages are not being printed. This seems to fix that in the normal case and singlerow, at least in pipeline mode. Didn't verify the non-pipeline mode. -- Álvaro Herrera39°49'30"S 73°17'W "Nunca se desea ardientemente lo que solo se desea

Re: [PATCH] SET search_path += octopus

2021-03-10 Thread David Steele
On 12/1/20 9:49 AM, Abhijit Menon-Sen wrote: Therefore, for lack of any plausible way to proceed, I do not plan to work on this feature any more. It sounds like this CF entry should be closed. I'll do that on March 12 unless somebody beats me to it. Regards, -- -David da...@pgmasters.net

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-10 Thread Dilip Kumar
On Wed, Mar 10, 2021 at 2:48 PM Amit Kapila wrote: > > On Mon, Mar 8, 2021 at 7:19 PM Amit Langote wrote: > > > > Hi Amit > > > > On Mon, Mar 8, 2021 at 10:18 PM Amit Kapila wrote: > > > On Mon, Mar 8, 2021 at 3:54 PM Greg Nancarrow wrote: > > > > I've attached an updated set of patches with

Re: Lowering the ever-growing heap->pd_lower

2021-03-10 Thread Matthias van de Meent
On Tue, 9 Mar 2021 at 22:35, Tom Lane wrote: > > Matthias van de Meent writes: > > The only two existing mechanisms that I could find (in the access/heap > > directory) that possibly could fail on shrunken line pointer arrays; > > being xlog recovery (I do not have enough knowledge on recovery

Re: proposal: unescape_text function

2021-03-10 Thread David Steele
On 12/2/20 1:30 PM, Pavel Stehule wrote: st 2. 12. 2020 v 11:37 odesílatel Pavel Stehule st 2. 12. 2020 v 9:23 odesílatel Peter Eisentraut Heh.  The fact that there is a table of two dozen possible representations kind of proves my point that we should be deliberate

Re: get rid of tags in the docs?

2021-03-10 Thread John Naylor
On Thu, Feb 4, 2021 at 11:31 AM Tom Lane wrote: > > John Naylor writes: > > While looking at the proposed removal of the v2 protocol, I noticed that we > > italicize some, but not all, instances of 'per se', 'pro forma', and 'ad > > hoc'. I'd say these are widespread enough in formal registers

Re: libpq debug log

2021-03-10 Thread 'alvhe...@alvh.no-ip.org'
On 2021-Mar-10, iwata@fujitsu.com wrote: > Hi all, > > Following all reviewer's advice, I have created a new patch. > > In this patch, I add only two tracing entry points; I call > pqTraceOutputMsg(PGconn conn, int msgCursor, PGCommSource commsource) in > pqParseInput3 () and pqPutMsgEnd

Re: error_severity of brin work item

2021-03-10 Thread David Steele
On 12/1/20 5:25 PM, Justin Pryzby wrote: On Tue, Dec 01, 2020 at 03:57:24PM -0300, Alvaro Herrera wrote: Another idea is if perform_work_item() were responsible for discarding relations which disappear. Currently it does this, which is racy since it holds no lock. That has the property that

Re: Change JOIN tutorial to focus more on explicit joins

2021-03-10 Thread David Steele
On 12/1/20 3:38 AM, Jürgen Purtz wrote: On 30.11.20 21:25, David G. Johnston wrote: Sorry, I managed to overlook the most recent patch. I admitted my use of parentheses was incorrect and I don't see anyone else defending them.  Please remove them. Minor typos: "the database compare" ->

Re: shared-memory based stats collector

2021-03-10 Thread Fujii Masao
On 2021/03/10 17:51, Kyotaro Horiguchi wrote: At Wed, 10 Mar 2021 15:20:43 +0900, Fujii Masao wrote in On 2021/03/10 12:10, Kyotaro Horiguchi wrote: Agreed. The code moved to the original place and added the crash handling code. And I added a phrase to the comment. +* Was

Re: Allow batched insert during cross-partition updates

2021-03-10 Thread Georgios
‐‐‐ Original Message ‐‐‐ On Wednesday, March 10, 2021 1:23 PM, Georgios wrote: > > > Hi, > > ‐‐‐ Original Message ‐‐‐ > On Thursday, February 18, 2021 10:54 AM, Amit Langote amitlangot...@gmail.com > wrote: > > > On Thu, Feb 18, 2021 at 6:52 PM Amit Langote

Re: Allow batched insert during cross-partition updates

2021-03-10 Thread Georgios
Hi, ‐‐‐ Original Message ‐‐‐ On Thursday, February 18, 2021 10:54 AM, Amit Langote wrote: > On Thu, Feb 18, 2021 at 6:52 PM Amit Langote amitlangot...@gmail.com wrote: > > > Based on the discussion at: > >

Re: Logical Replication - improve error message while adding tables to the publication in check_publication_add_relation

2021-03-10 Thread Euler Taveira
On Wed, Mar 10, 2021, at 2:14 AM, Bharath Rupireddy wrote: > While providing thoughts on [1], I observed that the error messages > that are emitted while adding foreign, temporary and unlogged tables > can be improved a bit from the existing [2] to [3]. For instance, the > existing message when

RE: Parallel Inserts in CREATE TABLE AS

2021-03-10 Thread houzj.f...@fujitsu.com
> Seems like v22 patch was failing in cfbot for one of the unstable test cases. > Attaching v23 patch set with modification in 0003 and 0004 patches. No > changes to 0001 and 0002 patches. Hopefully cfbot will be happy with v23. > > Please consider v23 for further review. Hi, I was looking into

Re: Logical Replication - improve error message while adding tables to the publication in check_publication_add_relation

2021-03-10 Thread Bharath Rupireddy
On Wed, Mar 10, 2021 at 1:27 PM Jeevan Ladhe wrote: > > On Wed, Mar 10, 2021 at 10:44 AM Bharath Rupireddy > wrote: >> >> Hi, >> >> While providing thoughts on [1], I observed that the error messages >> that are emitted while adding foreign, temporary and unlogged tables >> can be improved a

  1   2   >