Re: expose parallel leader in CSV and log_line_prefix

On Fri, Jul 31, 2020 at 03:04:48PM -0500, Justin Pryzby wrote: > On Tue, Jul 28, 2020 at 10:10:33AM +0900, Michael Paquier wrote: > > Except for those nits, I have tested the patch and things behave as we > > want (including padding and docs), so this looks good to me. > > Revised with your

Can a background worker exist without shared memory access for EXEC_BACKEND cases?

Hi, I tried to implement a static background(bg) worker without shared memory access (BGWORKER_SHMEM_ACCESS), it worked fine on Linux machine where EXEC_BACKEND is not defined(thanks to the fork() implementation which does great job to get the global state from the postmaster(parent) to bg

Re: windows config.pl question

cl -help works the same on z: or on c: and it’s equivalent to the output of cl /? from c: Z:\>cat cl_out.txt | grep favor /O1 maximum optimizations (favor space) /O2 maximum optimizations (favor speed) /Os favor code space/Ot favor code speed /Ox optimizations (favor speed)

Re: windows config.pl question

On Fri, Jul 31, 2020 at 10:41:46PM -0400, Dmitry Markman wrote: > but if I issue that command if the current folder is on z: > > Z:\>cl /? > Microsoft (R) C/C++ Optimizing Compiler Version 19.16.27042 for x64 > Copyright (C) Microsoft Corporation. All rights reserved. > > usage: cl [ option...

Re: Include access method in listTables output

On Wed, Jul 29, 2020 at 7:30 PM Georgios wrote: > > > I'm having issues understanding where you are going with the reviews, can you > fully describe > what is it that you wish to be done? > I'm happy with the patch, that was the last of the comments that I had from my side. Only idea here is to

Re: windows config.pl question

Hi Michael, I found the problem command cl /? gives different answer if you start that command from c: or from z: (where z: is mapped drive) if current directory is on c: then cl /? returns C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional>cl /? Microsoft (R) C/C++

Re: [PATCH] - Provide robust alternatives for replace_string

What happens if a replacement string happens to be split in the middle by the fgets buffering? I think it'll fail to be replaced. This applies to both versions. In the stringinfo version it seemed to me that using pnstrdup is possible to avoid copying trailing bytes. If you're asking for

Re: Concurrency bug in amcheck

On Wed, May 13, 2020 at 4:06 PM Peter Geoghegan wrote: > On Mon, May 11, 2020 at 5:56 AM Alexander Korotkov > wrote: > > Thank you. 2nd patch is proposed for master and makes btree page > > unlink remove all the items from the page being deleted. > > This looks good, but can we do the >

Re: Comment simplehash/dynahash trade-offs

On Sat, Aug 1, 2020 at 7:22 AM James Coleman wrote: > [v2 patch set] I ran it through pgindent which insisted on adding some newlines, I manually replaced some spaces with tabs to match nearby lines, I added some asterisks in your example function prototypes where is returned because they

Confusing behavior of create table like

Postgres provides serial and bigserial column types for which it implicitly creates sequence. As far as this mechanism is somehow hidden from user, it may be confusing that table created with CREATE TABLE LIKE has no associated sequence. But what is worse, even if experienced user knows that

Re: Cache relation sizes?

On Fri, Jul 31, 2020 at 2:36 PM Thomas Munro wrote: > There's still the matter of crazy numbers of lseeks in regular > backends; looking at all processes while running the above test, I get > 1,469,060 (9.18 per pgbench transaction) without -M prepared, and > 193,722 with -M prepared (1.21 per

Re: display offset along with block number in vacuum errors

Bcc: Subject: Re: display offset along with block number in vacuum errors Reply-To: In-Reply-To: On Wed, Jul 29, 2020 at 12:35:17AM +0530, Mahendra Singh Thalor wrote: > > Here: > > > > @@ -1924,14 +1932,22 @@ lazy_vacuum_page(Relation onerel, BlockNumber > > blkno, Buffer buffer, > >

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

On Tue, Jul 14, 2020 at 1:51 PM Anastasia Lubennikova wrote: > Questions from the first review pass: > > 1) Do we need XLH_INSERT_ALL_VISIBLE_SET ? IIUC, in the patch it is always > implied by XLH_INSERT_ALL_FROZEN_SET. I agree that it looks unnecessary to have two separate bits. > 2) What does

Re: [Patch] Optimize dropping of relation buffers using dlist

Hi, On 2020-07-31 15:50:04 -0400, Tom Lane wrote: > Andres Freund writes: > > Indeed. The buffer mapping hashtable already is visible as a major > > bottleneck in a number of workloads. Even in readonly pgbench if s_b is > > large enough (so the hashtable is larger than the cache). Not to speak

Re: Nicer error when connecting to standby with hot_standby=off

On Wed, Jul 29, 2020 at 11:24 AM Fujii Masao wrote: > > > > On 2020/04/03 22:49, James Coleman wrote: > > On Thu, Apr 2, 2020 at 5:53 PM David Zhang wrote: > >> > >> The following review has been posted through the commitfest application: > >> make installcheck-world: not tested > >> Implements

Re: expose parallel leader in CSV and log_line_prefix

On Tue, Jul 28, 2020 at 10:10:33AM +0900, Michael Paquier wrote: > Except for those nits, I have tested the patch and things behave as we > want (including padding and docs), so this looks good to me. Revised with your suggestions. -- Justin >From 72184bdb83eba6e477a12b743bb8d6bf938c5f41 Mon

Re: [Patch] Use internal pthreads reimplementation only when building with MSVC

> On 2 Jul 2020, at 16:35, Daniel Gustafsson wrote: > >> On 9 Apr 2020, at 23:57, Alvaro Herrera wrote: > >> Please do submit patches as separate attachments rather than in the >> email body. > > Since the CF app is unable to see that there is a patch at all, I took the > liberty to resubmit

Re: [Patch] Optimize dropping of relation buffers using dlist

Andres Freund writes: > Indeed. The buffer mapping hashtable already is visible as a major > bottleneck in a number of workloads. Even in readonly pgbench if s_b is > large enough (so the hashtable is larger than the cache). Not to speak > of things like a cached sequential scan with a cheap qual

Re: ModifyTable overheads in generic plans

On Fri, Jun 26, 2020 at 8:36 AM Amit Langote wrote: > 0001 and 0002 are preparatory patches. I read through these patches a bit but it's really unclear what the point of them is. I think they need better commit messages, or better comments, or both. -- Robert Haas EnterpriseDB:

Re: Improving connection scalability: GetSnapshotData()

> On 24 Jul 2020, at 03:11, Andres Freund wrote: > I've done that in the attached. As this is actively being reviewed but time is running short, I'm moving this to the next CF. cheers ./daniel

Re: Control your disk usage in PG: Introduction to Disk Quota Extension

> On 1 Jul 2020, at 10:36, Daniel Gustafsson wrote: > >> On 27 Mar 2020, at 11:22, Haozhou Wang wrote: > >> We rebased this patch with the newest master. > > This patch now fails to build according to Travis: > > smgr.c: In function ‘smgrtruncate’: > smgr.c:578:47: error: passing argument 4

Re: [HACKERS] [PATCH] Generic type subscripting

I started to look through this again, and really found myself wondering why we're going to all this work to invent what are fundamentally pretty bogus "features". The thing that particularly sticks in my craw is the 0005 patch, which tries to interpret a subscript of a JSON value as either

Re: Creating a function for exposing memory usage of backend process

On Fri, Jul 31, 2020 at 4:25 AM torikoshia wrote: > And as Fujii-san told me in person, exposing memory address seems > not preferable considering there are security techniques like > address space layout randomization. Yeah, exactly. ASLR wouldn't do anything to improve security if there were

Re: Comment simplehash/dynahash trade-offs

On Mon, Jul 20, 2020 at 1:29 AM Thomas Munro wrote: > > On Fri, May 1, 2020 at 1:53 PM James Coleman wrote: > > In another thread [1] I'd mused that "there might be some value in a > > README or comments > > addition that would be a guide to what the various hash > > implementations are useful

Re: [Patch] Optimize dropping of relation buffers using dlist

Hi, On 2020-07-31 13:39:37 -0400, Tom Lane wrote: > Robert Haas writes: > > Unfortunately, I don't have time for detailed review of this. I am > > suspicious that there are substantial performance regressions that you > > just haven't found yet. I would not take the position that this is a > >

Re: new heapcheck contrib module

On Fri, Jul 31, 2020 at 3:05 PM Andres Freund wrote: > The page could have been wrongly marked all-frozen. There could be > interactions between heap and toast table that are checked. Other bugs > could apply, like a broken hot chain or such. OK, at least the first two of these do sound like

Re: [PATCH] Support built control file in PGXS VPATH builds

> On 1 Jul 2020, at 13:34, Daniel Gustafsson wrote: > >> On 9 Apr 2020, at 05:54, Craig Ringer wrote: > >> Any thoughts here? I'd like to get it merged if possible and I hope my >> explanation of why I did it that way clears things up. > > According to the CFBot patch tester, this fails the

Re: new heapcheck contrib module

Hi, On 2020-07-31 12:42:51 -0400, Robert Haas wrote: > On Fri, Jul 31, 2020 at 12:33 PM Andres Freund wrote: > > I'm not sure what I was thinking "back then", but right now I'd argue > > that the best lock against vacuum isn't a SUE, but announcing the > > correct ->xmin, so you can be sure that

Re: [DOC] Document concurrent index builds waiting on each other

On Thu, Jul 16, 2020 at 7:34 PM David Johnston wrote: > > The following review has been posted through the commitfest application: > make installcheck-world: not tested > Implements feature: not tested > Spec compliant: not tested > Documentation:tested, passed > >

Re: recovering from "found xmin ... from before relfrozenxid ..."

On Fri, Jul 31, 2020 at 8:52 AM Ashutosh Sharma wrote: > Attached is the new version of patch that addresses the comments from Andrey > and Beena. +PGFILEDESC = "pg_surgery - perform surgery on the damaged heap table" the -> a I also suggest: heap table -> relation, because we might want to

Re: refactoring basebackup.c

On Fri, Jul 31, 2020 at 12:49 PM Andres Freund wrote: > Have you tested whether this still works against older servers? Or do > you think we should not have that as a goal? I haven't tested that recently but I intended to keep it working. I'll make sure to nail that down before I get to the

Re: [Patch] Optimize dropping of relation buffers using dlist

Robert Haas writes: > Unfortunately, I don't have time for detailed review of this. I am > suspicious that there are substantial performance regressions that you > just haven't found yet. I would not take the position that this is a > completely hopeless approach, or anything like that, but

Re: Why is pq_begintypsend so slow?

On Fri, Jul 31, 2020 at 1:00 PM Andres Freund wrote: > > Perhaps we could add a comment about this, e.g. > > Marking these pointers with pg_restrict tells the compiler that str > > and str->data can't overlap, which may allow the compiler to optimize > > better when this code is inlined. For

Re: Why is pq_begintypsend so slow?

Hi, On 2020-07-31 12:28:04 -0400, Robert Haas wrote: > On Fri, Jul 31, 2020 at 11:50 AM Andres Freund wrote: > > I hope the above makes this make sene now? It's about subsequent uses of > > the StringInfo, rather than the body of resetStringInfo itself. > > That does make sense, except that >

Re: refactoring basebackup.c

Hi, On 2020-07-29 11:31:26 -0400, Robert Haas wrote: > Here's an updated patch set. This is now rebased over master and > includes as 0001 the patch I posted separately at > http://postgr.es/m/CA+TgmobAczXDRO_Gr2euo_TxgzaH1JxbNxvFx=hyvbinefn...@mail.gmail.com > but drops some other patches that

Re: new heapcheck contrib module

On Fri, Jul 31, 2020 at 12:33 PM Andres Freund wrote: > I'm not sure what I was thinking "back then", but right now I'd argue > that the best lock against vacuum isn't a SUE, but announcing the > correct ->xmin, so you can be sure that clog entries won't be yanked out > from under you.

Re: new heapcheck contrib module

Hi, On 2020-07-31 08:51:50 -0700, Mark Dilger wrote: > In earlier versions of the patch, I was guarding (perhaps > unnecessarily) against clog truncation, (perhaps incorrectly) by > taking the CLogTruncationLock (aka XactTruncationLock.) . I thought > Andres was arguing that such locks were not

Re: proposal: type info support functions for functions that use "any" type

pá 31. 7. 2020 v 2:32 odesílatel Tom Lane napsal: > Daniel Gustafsson writes: > > This patch has been bumped in CFs for the past year, with the thread > stalled > > and the last review comment being in support of rejection. Tom, do you > still > > feel it should be rejected in light of Pavel's

Re: Why is pq_begintypsend so slow?

On Fri, Jul 31, 2020 at 11:50 AM Andres Freund wrote: > I hope the above makes this make sene now? It's about subsequent uses of > the StringInfo, rather than the body of resetStringInfo itself. That does make sense, except that https://en.cppreference.com/w/c/language/restrict says "During each

Re: new heapcheck contrib module

> On Jul 31, 2020, at 5:02 AM, Robert Haas wrote: > > On Thu, Jul 30, 2020 at 9:38 PM Mark Dilger > wrote: >>> On Jul 30, 2020, at 5:53 PM, Robert Haas wrote: >>> On Thu, Jul 30, 2020 at 6:10 PM Mark Dilger >> Since I can't think of a plausible concrete example of corruption which >> would

Re: Ought to use heap_multi_insert() for pg_attribute/depend insertions?

Hi, On 2020-07-30 23:23:38 +0200, Daniel Gustafsson wrote: > > On 30 Jul 2020, at 03:28, Michael Paquier wrote: > > On Wed, Jul 29, 2020 at 11:34:07PM +0200, Daniel Gustafsson wrote: > > >> Fair enough, let's break out pg_depend and I'll have another go at that. > > > > Thanks. Attached is a

Re: Why is pq_begintypsend so slow?

Hi, On 2020-07-31 11:14:46 -0400, Robert Haas wrote: > Here is some review for the first few patches in this series. Thanks! > I am generally in favor of applying 0001-0003 no matter what else we > decide to do here. However, as might be expected, I have a few > questions and comments. > >

Re: Why is pq_begintypsend so slow?

Here is some review for the first few patches in this series. I am generally in favor of applying 0001-0003 no matter what else we decide to do here. However, as might be expected, I have a few questions and comments. Regarding 0001: I dislike the name initStringInfoEx() because we do not use

Re: recovering from "found xmin ... from before relfrozenxid ..."

> 31 июля 2020 г., в 17:32, Ashutosh Sharma написал(а): > > > On Wed, Jul 29, 2020 at 9:58 AM Ashutosh Sharma wrote: > > > I think we should let VACUUM do that. > Sometimes VACUUM will not get to these pages, because they are marked All > Frozen. > An possibly some tuples will get stale

Re: Why is pq_begintypsend so slow?

On Tue, Jun 2, 2020 at 9:56 PM Andres Freund wrote: > I don't know what the best non-gross solution for the overhead of the > out/send functions is. There seems to be at least the following > major options (and a lots of variants thereof): > > 1) Just continue to incur significant overhead for

Re: Use of "long" in incremental sort code

On Thu, Jul 30, 2020 at 10:12 PM David Rowley wrote: > > On Fri, 3 Jul 2020 at 07:47, James Coleman wrote: > > Patch using int64 attached. > > I added this to the open items list for PG13. > > David I'd previously attached a patch [1], and there seemed to be agreement it was reasonable (lightly

Re: Is it worth accepting multiple CRLs?

Greetings, * Kyotaro Horiguchi (horikyota@gmail.com) wrote: > PostgreSQL server accepts only one CRL file. It is easy to expand > be_tls_init to accept a directory set in ssl_crl_file. But I'm not > sure CRL is actually even utilized in the field so that could ends > with just bloating the

Re: Is it worth accepting multiple CRLs?

A CA may issue a CRL infrequently, but issue a delta-CRL frequently. Does the logic support this properly? Personal email. hbh...@oxy.edu > On Jul 31, 2020, at 1:39 AM, Kyotaro Horiguchi > wrote: > > Hello. > > PostgreSQL server accepts only one CRL file. It is easy to expand > be_tls_init

Re: recovering from "found xmin ... from before relfrozenxid ..."

Attached is the new version of patch that addresses the comments from Andrey and Beena. On Wed, Jul 29, 2020 at 10:07 AM Ashutosh Sharma wrote: > Hi Beena, > > Thanks for the review. > > 1. We would be marking buffer dirty and writing wal even if we have >> not done any changes( ex if we pass

Re: track_planning causing performance regression

On Mon, Jul 6, 2020 at 10:29 AM Fujii Masao wrote: > > > On 2020/07/04 12:22, Pavel Stehule wrote: > > > > > > pá 3. 7. 2020 v 13:02 odesílatel Fujii Masao < > masao.fu...@oss.nttdata.com > napsal: > > > > > > > >

Re: Missing CFI in hlCover()?

Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> Here's a proposed patch along that line. > > > I've back-patched this to 11 (which was just a bit of fuzz) and tested > > it out with a couple of different queries that were

Re: recovering from "found xmin ... from before relfrozenxid ..."

On Wed, Jul 29, 2020 at 9:58 AM Ashutosh Sharma wrote: > > > I think we should let VACUUM do that. >> Sometimes VACUUM will not get to these pages, because they are marked All >> Frozen. >> An possibly some tuples will get stale on this page again >> > > Hmm, okay, will have a look into this.

[PATCH] - Provide robust alternatives for replace_string

Hi, In our testing framework, backed by pg_regress, there exists the ability to use special strings that can be replaced by environment based ones. Such an example is '@testtablespace@'. The function used for this replacement is replace_string which inline replaces these occurrences in

Re: new heapcheck contrib module

On Thu, Jul 30, 2020 at 9:38 PM Mark Dilger wrote: > > On Jul 30, 2020, at 5:53 PM, Robert Haas wrote: > > On Thu, Jul 30, 2020 at 6:10 PM Mark Dilger > Since I can't think of a plausible concrete example of corruption which would > elicit the problem I was worrying about, I'll withdraw the

Re: [bugfix]"make installcheck" could not work in PGXS

On Fri, Jul 31, 2020 at 08:31:56AM +, matsumura@fujitsu.com wrote: > I found that "make installcheck" could not work in PGXS. Yeah, that's a known problem. One way to counter that is for example to grab the path of pg_regress from pg_config --libdir and set $ENV{PG_REGRESS} to it, but

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

> Attaching an updated patch for the stats for streaming based on v2 of > Sawada's san replication slot stats framework and v44 of this patch series > . This is one patch that has both the stats framework from Sawada-san (1) > as well as my update for streaming, so it can be applied easily on top

Re: Is it useful to record whether plans are generic or custom?

On 2020-07-30 14:31, Fujii Masao wrote: On 2020/07/22 16:49, torikoshia wrote: On 2020-07-20 13:57, torikoshia wrote: As I proposed earlier in this thread, I'm now trying to add information about generic/cudstom plan to pg_stat_statements. I'll share the idea and the poc patch soon.

Re: windows config.pl question

On Thu, Jul 30, 2020 at 11:16:01PM -0400, Dmitry Markman wrote: > sorry I meant file src/tools/msvc/Solution.pm > > routine sub GetOpenSSLVersion > > has the follwing line: > > qq("$self->{options}->{openssl}\\bin\\openssl.exe" version 2>&1); > > in our distribution openssl.exe isn’t in the

Re: [PATCH] Remove useless distinct clauses

Hi Pierre, On Fri, Jul 31, 2020 at 2:11 PM Pierre Ducroquet wrote: > > Hi > > In a recent audit, I noticed that application developers have a tendency to > abuse the distinct clause. For instance they use an ORM and add a distinct at > the top level just because they don't know the cost it has,

[PATCH] Add section headings to index types doc

Hi hackers, Every time I have to look up what kinds of operations each index type is suitable for, I get annoyed by the index types page being virtually unskimmable due to not having headings for each index type. Attached is a patch that adds tags for each index type to make it easier to see

[PATCH] Remove useless distinct clauses

Hi In a recent audit, I noticed that application developers have a tendency to abuse the distinct clause. For instance they use an ORM and add a distinct at the top level just because they don't know the cost it has, or they don't know that using EXISTS is a better way to express their queries

Is it worth accepting multiple CRLs?

Hello. PostgreSQL server accepts only one CRL file. It is easy to expand be_tls_init to accept a directory set in ssl_crl_file. But I'm not sure CRL is actually even utilized in the field so that could ends with just bloating the documentation. Is it work doing? regards. -- Kyotaro Horiguchi

Re: fixing pg_ctl with relative paths

At Fri, 31 Jul 2020 09:42:42 +0800 (CST), ZHAOWANCHENG wrote in > At 2014-01-28 21:11:54, "Bruce Momjian" wrote: > >On Mon, Jul 1, 2013 at 08:10:14PM -0400, Josh Kupershmidt wrote: > >> On Thu, Jun 27, 2013 at 11:47 AM, Fujii Masao > >> wrote: > >> > On Thu, Jun 27, 2013 at 10:36 AM, Josh

[bugfix]"make installcheck" could not work in PGXS

Hello, I found that "make installcheck" could not work in PGXS. --[/src/foo_project/Makefile]-- SUBDIRS = foo TAP_TESTS = 1 PG_CONFIG = pg_config PGXS := $(shell $(PG_CONFIG) --pgxs) include $(PGXS) $(recurse) $(recurse_always) --[/src/foo_project/t/001_foo_test.pl] use strict; use

Re: Creating a function for exposing memory usage of backend process

On 2020-07-30 15:13, Kasahara Tatsuhito wrote: Hi, On Fri, Jul 10, 2020 at 5:32 PM torikoshia wrote: - whether information for identifying parent-child relation is necessary or it's an overkill I think it's important to understand the parent-child relationship of the context. Personally, I

Re: Ought to use heap_multi_insert() for pg_attribute/depend insertions?

> On 31 Jul 2020, at 04:42, Michael Paquier wrote: > > On Thu, Jul 30, 2020 at 11:23:38PM +0200, Daniel Gustafsson wrote: >> Looks good, thanks. Let's close the CF entry with this and open a new for >> the >> pg_depend part when that's done. > > I have applied the patch, thanks. > > And

Re: Internal key management system

On Tue, 23 Jun 2020 at 22:46, Masahiko Sawada wrote: > > On Fri, 19 Jun 2020 at 15:44, Fabien COELHO wrote: > > > > > > Hello Masahiko-san, > > > > > What I referred to "only one key" is KEK. > > > > Ok, sorry, I misunderstood. > > > > >>> Yeah, it depends on KMS, meaning we need different

Re: autovac issue with large number of tables

Hi, On Tue, Jul 28, 2020 at 3:49 AM Jim Nasby wrote: > I'm in favor of trying to improve scheduling (especially allowing users > to control how things are scheduled), but that's a far more invasive > patch. I'd like to get something like this patch in without waiting on a > significantly larger