Re: Failure with 004_logrotate in prairiedog

2021-07-17 Thread Tom Lane
Michael Paquier writes: > prairiedog has failed in a way that seems a bit obscure to me: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=prairiedog&dt=2021-07-18%2000%3A23%3A29 > ... > We could use some eval blocks in this area, but a non-atomic rename() > would cause problems in more ar

Re: Added documentation for cascade and restrict option of drop statistics

2021-07-17 Thread Michael Paquier
On Fri, Jul 16, 2021 at 08:12:56PM +0530, vignesh C wrote: > Cascade and restrict options are supported for drop statistics syntax: > drop statistics stat1 cascade; > drop statistics stat2 restrict; > > The documentation for this was missing, attached a patch which > includes the documentation for

Failure with 004_logrotate in prairiedog

2021-07-17 Thread Michael Paquier
Hi all, prairiedog has failed in a way that seems a bit obscure to me: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=prairiedog&dt=2021-07-18%2000%3A23%3A29 Here are the details of the failure: server signaled to rotate log file could not read "/Users/buildfarm/bf-data/HEAD/pgsql.build/

Re: slab allocator performance issues

2021-07-17 Thread Andres Freund
Hi, On 2021-07-17 16:10:19 -0700, Andres Freund wrote: > Instead of populating a linked list with all chunks upon creation of a block - > which requires touching a fair bit of memory - keep a per-block pointer (or an > offset) into "unused" area of the block. When allocating from the block and > t

Re: slab allocator performance issues

2021-07-17 Thread Andres Freund
Hi, On 2021-07-18 00:46:09 +0200, Tomas Vondra wrote: > On 7/17/21 11:14 PM, Andres Freund wrote: > > Hm. I wonder if we should just not populate the freelist eagerly, to > > drive down the initialization cost. I.e. have a separate allocation path > > for chunks that have never been allocated, by

Re: slab allocator performance issues

2021-07-17 Thread Tomas Vondra
On 7/17/21 11:14 PM, Andres Freund wrote: Hi, On 2021-07-17 22:35:07 +0200, Tomas Vondra wrote: On 7/17/21 9:43 PM, Andres Freund wrote: 1) If allocations are short-lived slab.c, can end up constantly freeing/initializing blocks. Which requires fairly expensively iterating over all potentia

Re: corruption of WAL page header is never reported

2021-07-17 Thread Ranier Vilela
Em sáb., 17 de jul. de 2021 às 16:57, Yugo NAGATA escreveu: > Hello, > > I found that any corruption of WAL page header found during recovery is > never > reported in log messages. If wal page header is broken, it is detected in > XLogReaderValidatePageHeader called from XLogPageRead, but the er

Re: slab allocator performance issues

2021-07-17 Thread Andres Freund
Hi, On 2021-07-17 22:35:07 +0200, Tomas Vondra wrote: > On 7/17/21 9:43 PM, Andres Freund wrote: > > 1) If allocations are short-lived slab.c, can end up constantly > > freeing/initializing blocks. Which requires fairly expensively iterating > > over > > all potential chunks in the block and init

Re: slab allocator performance issues

2021-07-17 Thread Tomas Vondra
Hi, On 7/17/21 9:43 PM, Andres Freund wrote: Hi, I just tried to use the slab allocator for a case where aset.c was bloating memory usage substantially. First: It worked wonders for memory usage, nearly eliminating overhead. But it turned out to cause a *substantial* slowdown. With aset the al

corruption of WAL page header is never reported

2021-07-17 Thread Yugo NAGATA
Hello, I found that any corruption of WAL page header found during recovery is never reported in log messages. If wal page header is broken, it is detected in XLogReaderValidatePageHeader called from XLogPageRead, but the error messages are always reset and never reported. if (!XLogReade

Re: slab allocator performance issues

2021-07-17 Thread Andres Freund
Hi, On 2021-07-17 12:43:33 -0700, Andres Freund wrote: > 2) SlabChunkIndex() in SlabFree() is slow. It requires a 64bit division, > taking > up ~50% of the cycles in SlabFree(). A 64bit div, according to [1] , has a > latency of 35-88 cycles on skylake-x (and a reverse throughput of 21-83, > i.e.

slab allocator performance issues

2021-07-17 Thread Andres Freund
Hi, I just tried to use the slab allocator for a case where aset.c was bloating memory usage substantially. First: It worked wonders for memory usage, nearly eliminating overhead. But it turned out to cause a *substantial* slowdown. With aset the allocator is barely in the profile. With slab the

Re: UniqueKey on Partitioned table.

2021-07-17 Thread Andy Fan
On Sat, Jul 17, 2021 at 3:45 PM David Rowley wrote: > > On Sat, 17 Jul 2021 at 19:32, Andy Fan wrote: > > SELECT * FROM t1, t2 WHERE t1.pk = t2.pk; > > > > Then when I populate_baserel_uniquekeys for t1, we already have > > EC{Members={t1.pk, t2.pk}} in root->eq_classes already. Then I use > > th

Re: A micro-optimisation for ProcSendSignal()

2021-07-17 Thread Soumyadeep Chakraborty
Hi Thomas, You might have missed a spot to initialize SERIALIZABLE_XACT->pgprocno in InitPredicateLocks(), so: + PredXact->OldCommittedSxact->pgprocno = INVALID_PGPROCNO; Slightly tangential: we should add a comment to PGPROC.pgprocno, for more immediate understandability: + int pgprocno; /* in

Re: postgres_fdw - make cached connection functions tests meaningful

2021-07-17 Thread Tom Lane
Bharath Rupireddy writes: > On Thu, Jul 15, 2021 at 3:48 AM Tom Lane wrote: >> More generally, though, I am not sure that I believe the premise of >> this patch. AFAICS it's assuming that forcing debug_discard_caches >> off guarantees zero cache flushes, which it does not. > Can the setting deb

Re: What are exactly bootstrap processes, auxiliary processes, standalone backends, normal backends(user sessions)?

2021-07-17 Thread Tom Lane
Justin Pryzby writes: > On Sat, Jul 17, 2021 at 10:45:52AM -0400, Alvaro Herrera wrote: >> Process of an instance in >> charge of some specific, hardcoded background task. Examples are > And I think "hardcoded" doesn't mean anything beyond what "specific" means. > Maybe you'd say: .. process wh

Re: What are exactly bootstrap processes, auxiliary processes, standalone backends, normal backends(user sessions)?

2021-07-17 Thread Justin Pryzby
On Sat, Jul 17, 2021 at 10:45:52AM -0400, Alvaro Herrera wrote: > On 2021-Jul-17, Bharath Rupireddy wrote: > > > On Thu, Jul 15, 2021 at 8:17 PM Justin Pryzby wrote: > > > > > > On Fri, Jul 09, 2021 at 09:24:19PM +0530, Bharath Rupireddy wrote: > > > > I've always had a hard time distinguishing v

Re: What are exactly bootstrap processes, auxiliary processes, standalone backends, normal backends(user sessions)?

2021-07-17 Thread Alvaro Herrera
On 2021-Jul-17, Bharath Rupireddy wrote: > On Thu, Jul 15, 2021 at 8:17 PM Justin Pryzby wrote: > > > > On Fri, Jul 09, 2021 at 09:24:19PM +0530, Bharath Rupireddy wrote: > > > I've always had a hard time distinguishing various types of > > > processes/terms used in postgres. I look at the source

Re: postgres_fdw - make cached connection functions tests meaningful

2021-07-17 Thread Bharath Rupireddy
On Thu, Jul 15, 2021 at 3:48 AM Tom Lane wrote: > > Bharath Rupireddy writes: > > While working on [1], I got to know that there is a new GUC > > debug_invalidate_system_caches_always that has been introduced in v14. > > It can be used to switch off cache invalidation in > > CLOBBER_CACHE_ALWAYS

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2021-07-17 Thread Ryan Lambert
> On Sat, Jul 10, 2021 at 5:06 AM Tomas Vondra < tomas.von...@enterprisedb.com> wrote: > On 7/10/21 1:43 AM, Tom Lane wrote: >> Tomas Vondra writes: >>> The main question I have is whether this should include procedures. >> >> I feel a bit uncomfortable about sticking this sort of limited-purpose

Re: [PATCH] Use optimized single-datum tuplesort in ExecSort

2021-07-17 Thread David Rowley
On Sat, 17 Jul 2021 at 01:14, James Coleman wrote: > The only remaining question I have is whether or not costing needs to > change, given the very significant speedup for datum sort. I'm looking at cost_tuplesort and the only thing that I think might make sense would be to adjust how the input_

Re: UniqueKey on Partitioned table.

2021-07-17 Thread David Rowley
On Sat, 17 Jul 2021 at 19:32, Andy Fan wrote: > SELECT * FROM t1, t2 WHERE t1.pk = t2.pk; > > Then when I populate_baserel_uniquekeys for t1, we already have > EC{Members={t1.pk, t2.pk}} in root->eq_classes already. Then I use > this EC directly for t1's UniqueKey. The result is: > > T1's UniqueK

Re: UniqueKey on Partitioned table.

2021-07-17 Thread Andy Fan
On Tue, Apr 6, 2021 at 6:55 PM David Rowley wrote: > > On Tue, 6 Apr 2021 at 22:31, Andy Fan wrote: > > On Wed, Mar 31, 2021 at 9:12 PM Ashutosh Bapat > > wrote: > >> I think the reason we add ECs for sort expressions is to use > >> transitive relationship. The EC may start with a single member