Re: pgbench logging broken by time logic changes

2021-07-09 Thread Thomas Munro
On Fri, Jul 9, 2021 at 5:14 PM Fabien COELHO wrote: > Works for me: patch applies, global and local check ok. I'm fine with it. I hoped we were done here but I realised that your check for 1-3 log lines will not survive the harsh environment of the build farm. Adding sleep(2) before the final

Re: [PoC] Improve dead tuple storage for lazy vacuum

2021-07-09 Thread Andres Freund
Hi, On 2021-07-09 10:17:49 -0700, Andres Freund wrote: > On 2021-07-07 20:46:38 +0900, Masahiko Sawada wrote: > > Currently, the TIDs of dead tuples are stored in an array that is > > collectively allocated at the start of lazy vacuum and TID lookup uses > > bsearch(). There are the following

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2021-07-09 Thread Justin Pryzby
On Fri, Jul 09, 2021 at 07:43:02PM -0400, Tom Lane wrote: > Tomas Vondra writes: > > The main question I have is whether this should include procedures. > > I feel a bit uncomfortable about sticking this sort of limited-purpose > selectivity mechanism into pg_dump. I'd rather see a general

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2021-07-09 Thread Tom Lane
Tomas Vondra writes: > The main question I have is whether this should include procedures. I feel a bit uncomfortable about sticking this sort of limited-purpose selectivity mechanism into pg_dump. I'd rather see a general filter method that can select object(s) of any type. Pavel was doing

Re: [PATCH] Pull general SASL framework out of SCRAM

2021-07-09 Thread Jacob Champion
On Thu, 2021-07-08 at 16:27 +0900, Michael Paquier wrote: > I agree that this looks like an improvement in terms of the > expectations behind a SASL mechanism, so I have done the attached to > strengthen a bit all those checks. However, I don't really see a > point in back-patching any of that,

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2021-07-09 Thread Tomas Vondra
Hi, I took a quick look at the patch today. There was some minor bitrot requiring a rebase, so I attach the rebased patch as v3. The separate 0002 part contains some minor fixes - a couple typos/rewording in the docs (would be good if a native speaker looked at it, thought), and a slightly

Re: Preventing abort() and exit() calls in libpq

2021-07-09 Thread Noah Misch
On Fri, Jul 09, 2021 at 10:06:18AM -0400, Tom Lane wrote: > Noah Misch writes: > > On Sat, Jul 03, 2021 at 06:44:20PM -0400, Tom Lane wrote: > >> That'd require buildfarm owner intervention, as well as intervention > >> by users. Which seems like exporting our problems onto them. I'd > >>

Re: when the startup process doesn't (logging startup delays)

2021-07-09 Thread Robert Haas
Hi, I'd really like to see this enabled by default, say with a default interval of 10 seconds. If it has to be enabled explicitly, most people won't, but I think a lot of people would benefit from knowing why their system is slow to start up when that sort of thing happens. I don't see much

Re: Online verification of checksums

2021-07-09 Thread Ibrar Ahmed
On Tue, Mar 9, 2021 at 10:43 PM David Steele wrote: > On 11/30/20 6:38 PM, David Steele wrote: > > On 11/30/20 9:27 AM, Stephen Frost wrote: > >> * Michael Paquier (mich...@paquier.xyz) wrote: > >>> On Fri, Nov 27, 2020 at 11:15:27AM -0500, Stephen Frost wrote: > * Magnus Hagander

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2021-07-09 Thread Ibrar Ahmed
On Tue, Mar 30, 2021 at 12:12 PM Paul Guo wrote: > On 2021/3/27, 10:23 PM, "Alvaro Herrera" wrote: > > >Hmm, can you post a rebased set, where the points under discussion > > are marked in XXX comments explaining what the issue is? This thread > is > >long and old ago that it's

Re: enable_resultcache confusion

2021-07-09 Thread Robert Haas
On Fri, Jul 9, 2021 at 11:35 AM David Rowley wrote: > I really like that name. > > I'll wait to see if anyone else wants to voice their opinion before I > do any renaming work. I like it, too. -- Robert Haas EDB: http://www.enterprisedb.com

Re: Query about time zone patterns in to_char

2021-07-09 Thread David Zhang
The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: tested, passed Spec compliant: not tested Documentation:not tested Applied the patch `v2_support_of_tzh_tzm_patterns.patch` to

Re: Synchronous commit behavior during network outage

2021-07-09 Thread Andrey Borodin
> 3 июля 2021 г., в 23:44, Jeff Davis написал(а): > > On Sat, 2021-07-03 at 14:06 +0500, Andrey Borodin wrote: >>> But until you've disabled sync rep, the primary will essentially be >>> down for writes whether using this new feature or not. Even if you >>> can >>> terminate some backends to

Re: More time spending with "delete pending"

2021-07-09 Thread Alexander Lakhin
Hello Michael, 09.07.2021 08:52, Michael Paquier wrote: > On Thu, Jul 08, 2021 at 11:00:00PM +0300, Alexander Lakhin wrote: >> Beside the aforementioned test I can only propose the extended patch, >> that incorporates the undo of the changes brought by bed90759f. >> With this patch that test is

Re: Parallel scan with SubTransGetTopmostTransaction assert coredump

2021-07-09 Thread Tomas Vondra
Hi, I took a quick look on this - I'm no expert in the details of snapshots, so take my comments with a grain of salt. AFAICS both Greg Nancarrow and Pavel Borisov are kinda right. I think Greg is right the v3 patch does not seem like the right (or correct) solution, for a couple reasons: 1)

Re: [PoC] Improve dead tuple storage for lazy vacuum

2021-07-09 Thread Andres Freund
Hi, On 2021-07-07 20:46:38 +0900, Masahiko Sawada wrote: > Currently, the TIDs of dead tuples are stored in an array that is > collectively allocated at the start of lazy vacuum and TID lookup uses > bsearch(). There are the following challenges and limitations: > So I prototyped a new data

Re: short circuit suggestion in find_hash_columns()

2021-07-09 Thread Zhihong Yu
On Fri, Jul 9, 2021 at 8:28 AM David Rowley wrote: > On Sat, 10 Jul 2021 at 03:15, Zhihong Yu wrote: > > I was looking at find_hash_columns() in nodeAgg.c > > > > It seems the first loop tries to determine the max column number needed, > along with whether all columns are needed. > > > > The

Tab completion for CREATE SCHEMAAUTHORIZATION

2021-07-09 Thread Dagfinn Ilmari Mannsåker
Hi Hackers, I just noticed there's no tab completion for CREATE SCHEMA AUTHORIZATION, nor for anything after CREATE SCHEMA . Please find attached a patch that adds this. - ilmari >From db02df7ea8d3a5db41268edd8c311a3631c9e9ad Mon Sep 17 00:00:00 2001 From:

What are exactly bootstrap processes, auxiliary processes, standalone backends, normal backends(user sessions)?

2021-07-09 Thread Bharath Rupireddy
Hi, I've always had a hard time distinguishing various types of processes/terms used in postgres. I look at the source code every time to understand them, yet I don't feel satisfied with my understanding. I request any hacker (having a better idea than me) to help me with what each different

Re: enable_resultcache confusion

2021-07-09 Thread David Rowley
On Fri, 9 Jul 2021 at 06:00, Tom Lane wrote: > > Maybe name the plan node type Memoize, and the GUC "enable_memoize"? I really like that name. I'll wait to see if anyone else wants to voice their opinion before I do any renaming work. David

Re: short circuit suggestion in find_hash_columns()

2021-07-09 Thread David Rowley
On Sat, 10 Jul 2021 at 03:15, Zhihong Yu wrote: > I was looking at find_hash_columns() in nodeAgg.c > > It seems the first loop tries to determine the max column number needed, > along with whether all columns are needed. > > The loop can be re-written as shown in the patch. This runs during

Re: Record a Bitmapset of non-pruned partitions

2021-07-09 Thread David Rowley
On Thu, 1 Jul 2021 at 17:49, Amit Langote wrote: > Given that you're proposing more uses for live_parts, maybe he'd be > open to the idea. Just to make sure the new field in the 0001 patch gets good enough use, I've attached the patch which includes more usages of the field. 0002 adds a new

short circuit suggestion in find_hash_columns()

2021-07-09 Thread Zhihong Yu
Hi, I was looking at find_hash_columns() in nodeAgg.c It seems the first loop tries to determine the max column number needed, along with whether all columns are needed. The loop can be re-written as shown in the patch. In normal cases, we don't need to perform scanDesc->natts iterations. In

Re: Grammar railroad diagram

2021-07-09 Thread Domingo Alvarez Duarte
Hello Andres ! Another way that I tested and it's working is to use https://www.bottlecaps.de/convert/ paste the postgresql grammar there and press "convert" and after press "view diagram". Again optionally manually add the Tokens to a better diagram ! // Tokens from

Re: Query about time zone patterns in to_char

2021-07-09 Thread Tomas Vondra
On 5/20/21 8:25 PM, Bruce Momjian wrote: > On Thu, May 20, 2021 at 12:21:12PM +0530, Nitin Jadhav wrote: >> Thanks Suraj for reviewing the patch. >> >>> 1: >>> +RESET timezone; >>> + >>> + >>> CREATE TABLE TIMESTAMPTZ_TST (a int , b timestamptz); >>> >>> Extra line. >>> >>> 2: >>> +SET timezone

Re: Fwd: Grammar railroad diagram

2021-07-09 Thread grd
Domingo, Bruce,   sorry for the error. It was caused by my server using Tomcat's default maxPostSize of 2MB, which is not sufficient for the diagrams generated from the Postgres grammar. I have now extended it to 10MB.   By the way, I had already created diagrams for PostgreSQL back in

Re: Preventing abort() and exit() calls in libpq

2021-07-09 Thread Tom Lane
Noah Misch writes: > On Sat, Jul 03, 2021 at 06:44:20PM -0400, Tom Lane wrote: >> That'd require buildfarm owner intervention, as well as intervention >> by users. Which seems like exporting our problems onto them. I'd >> really rather not go that way if we can avoid it. > I like that goal,

Re: [CLOBBER_CACHE]Server crashed with segfault 11 while executing clusterdb

2021-07-09 Thread Alvaro Herrera
On 2021-Jul-09, Amul Sul wrote: > > On Tue, Jul 6, 2021 at 11:06 PM Tom Lane wrote: > > > The point of the static-inline function idea was to be cheap enough > > > that it isn't worth worrying about this sort of risky optimization. > > > Given that an smgr function is sure to involve some

Re: [CLOBBER_CACHE]Server crashed with segfault 11 while executing clusterdb

2021-07-09 Thread Amul Sul
On Wed, Jul 7, 2021 at 9:44 AM Amul Sul wrote: > > On Tue, Jul 6, 2021 at 11:06 PM Tom Lane wrote: > > > > Amul Sul writes: > > > On Tue, Apr 20, 2021 at 6:59 AM Kyotaro Horiguchi > > > wrote: > > >> I don't mind RelationGetSmgr(index)->smgr_rnode alone or > > >> >member alone and there's not

Re: Support kerberos authentication for postgres_fdw

2021-07-09 Thread Tom Lane
Peifeng Qiu writes: > I'd like to add kerberos authentication support for postgres_fdw by adding two > options to user mapping: krb_client_keyfile and gssencmode. As you note, this'd have to be restricted to superusers, which makes it seem like a pretty bad idea. We really don't want to be in a

Re: Transactions involving multiple postgres foreign servers, take 2

2021-07-09 Thread Masahiko Sawada
On Fri, Jul 9, 2021 at 3:26 PM Fujii Masao wrote: > > > > On 2021/06/30 10:05, Masahiko Sawada wrote: > > I've attached the new version patch that incorporates the comments > > from Fujii-san and Ikeda-san I got so far. > > Thanks for updating the patches! > > I'm now reading 0001 and 0002

Re: [PATCH] Automatic HASH and LIST partition creation

2021-07-09 Thread Robert Haas
On Tue, Mar 2, 2021 at 3:26 PM Justin Pryzby wrote: > I don't know what committers will say, but I think that "ALTER TABLE" might be > the essential thing for this patch to support, not "CREATE". (This is similar > to ALTER..SET STATISTICS, which is not allowed in CREATE.) > > The reason is that

Support kerberos authentication for postgres_fdw

2021-07-09 Thread Peifeng Qiu
Hi hackers, I'd like to add kerberos authentication support for postgres_fdw by adding two options to user mapping: krb_client_keyfile and gssencmode. In the backend we have krb_server_keyfile option to specify a keytab file to be used by postgres server, krb_client_keyfile is doing mostly the

Support kerberos authentication for postgres_fdw

2021-07-09 Thread Peifeng Qiu
Hi hackers, I'd like to add kerberos authentication support for postgres_fdw by adding two options to user mapping: krb_client_keyfile and gssencmode. In the backend we have krb_server_keyfile option to specify a keytab file to be used by postgres server, krb_client_keyfile is doing mostly the

Re: Why ALTER SUBSCRIPTION ... SET (slot_name='none') requires subscription disabled?

2021-07-09 Thread Ranier Vilela
Em qui., 8 de jul. de 2021 às 23:50, Japin Li escreveu: > > On Thu, 08 Jul 2021 at 18:17, Amit Kapila wrote: > > On Thu, Jul 8, 2021 at 3:43 PM Japin Li wrote: > >> > >> On Thu, 08 Jul 2021 at 17:51, Amit Kapila > wrote: > >> > On Wed, Jul 7, 2021 at 7:25 PM Japin Li wrote: > >> >> > >> >>

Re: ERROR: "ft1" is of the wrong type.

2021-07-09 Thread Kyotaro Horiguchi
At Fri, 9 Jul 2021 11:03:56 +0900, Michael Paquier wrote in > On Fri, Jul 09, 2021 at 10:44:13AM +0900, Kyotaro Horiguchi wrote: > > Sounds reasonable. So the attached are that for PG11-PG14. 11 and 12 > > shares the same patch. > > How much do the regression tests published upthread in >

Re: bugfix: when the blocksize is 32k, the function page_header of pageinspect returns negative numbers.

2021-07-09 Thread Bharath Rupireddy
On Fri, Jul 9, 2021 at 8:43 AM Quan Zongliang wrote: > Thanks for the comments. > Done Thanks for the patch. Few comments: 1) How about just adding a comment /* support for old extension version */ before INT2OID handling? + case INT2OID: + values[3] = UInt16GetDatum(page->pd_lower); + break;

Re: Transactions involving multiple postgres foreign servers, take 2

2021-07-09 Thread Masahiko Sawada
Sorry for the late reply. On Tue, Jul 6, 2021 at 3:15 PM r.takahash...@fujitsu.com wrote: > > Hi, > > > I'm interested in this patch and I also run the same test with Ikeda-san's > fxact_update.pgbench. Thank you for testing! > In my environment (poor spec VM), the result is following. > > *

Re: Transactions involving multiple postgres foreign servers, take 2

2021-07-09 Thread Masahiko Sawada
Sorry for the late reply. On Mon, Jul 5, 2021 at 3:29 PM Masahiro Ikeda wrote: > > > > On 2021/06/30 10:05, Masahiko Sawada wrote: > > On Fri, Jun 25, 2021 at 9:53 AM Masahiro Ikeda > > wrote: > >> > >> Hi Jamison-san, sawada-san, > >> > >> Thanks for testing! > >> > >> FWIF, I tested using

Introduce pg_receivewal gzip compression tests

2021-07-09 Thread Georgios
Hi, As suggested on a different thread [1], pg_receivewal can increase it's test coverage. There exists a non trivial amount of code that handles gzip compression. The current patch introduces tests that cover creation of gzip compressed WAL files and the handling of gzip partial segments.

Re: [PATCH] Automatic HASH and LIST partition creation

2021-07-09 Thread Pavel Borisov
> > > Thank you for your review! > > I've rebased the patch and made the changes mentioned. > > PFA v5. > > I've set this back to "needs review" in CF. > Thanks for the attention! I did the review of this patch, and the changes I've introduced in v5 are purely cosmetic. So I'd suppose the

Re: "debug_invalidate_system_caches_always" is too long

2021-07-09 Thread Noah Misch
On Thu, Jul 08, 2021 at 04:34:55PM -0400, Tom Lane wrote: > Robert Haas writes: > > On Wed, Jul 7, 2021 at 11:17 AM Tom Lane wrote: > >> Fair point. What do you think of the alternative proposals > >> "debug_flush_caches", "debug_discard_caches", etc? > > > I like debug_discard_caches best. >

Re: [PATCH] Automatic HASH and LIST partition creation

2021-07-09 Thread John Naylor
On Fri, Jul 9, 2021 at 6:30 AM Pavel Borisov wrote: > Thank you for your review! > I've rebased the patch and made the changes mentioned. > PFA v5. I've set this back to "needs review" in CF. -- John Naylor EDB: http://www.enterprisedb.com

Re: [PATCH] Automatic HASH and LIST partition creation

2021-07-09 Thread Pavel Borisov
> > I have reviewed the v4 patch. The patch does not get applied on the latest > source. Kindly rebase. > However I have found few comments. > > 1. > > +-- must fail because of wrong configuration > > +CREATE TABLE tbl_hash_fail (i int) PARTITION BY HASH (i) > > +CONFIGURATION (values in (1, 2),

Re: Preventing abort() and exit() calls in libpq

2021-07-09 Thread Noah Misch
On Sat, Jul 03, 2021 at 06:44:20PM -0400, Tom Lane wrote: > Noah Misch writes: > > On Sat, Jul 03, 2021 at 10:45:59AM -0400, Tom Lane wrote: > >> What I'm now thinking about is restricting the test to only be run on > >> platforms where use of foo.a libraries is deprecated, so that we can > >> be

RE: [HACKERS] logical decoding of two-phase transactions

2021-07-09 Thread tanghy.f...@fujitsu.com
On Friday, July 9, 2021 2:56 PM Ajin Cherian wrote: > > On Fri, Jul 9, 2021 at 9:13 AM Peter Smith wrote: > > > I tried the v95-0001 patch. > > > > - The patch applied cleanly and all build / testing was OK. > > - The documentation also builds OK. > > - I checked all v95-0001 / v93-0001

Re: [PoC] Improve dead tuple storage for lazy vacuum

2021-07-09 Thread Masahiko Sawada
On Thu, Jul 8, 2021 at 10:40 PM Hannu Krosing wrote: > > Very nice results. > > I have been working on the same problem but a bit different solution - > a mix of binary search for (sub)pages and 32-bit bitmaps for > tid-in-page. > > Even with currebnt allocation heuristics (allocate 291 tids per

Re: [PoC] Improve dead tuple storage for lazy vacuum

2021-07-09 Thread Masahiko Sawada
On Thu, Jul 8, 2021 at 7:51 AM Peter Geoghegan wrote: > > On Wed, Jul 7, 2021 at 1:24 PM Peter Geoghegan wrote: > > I wonder how much it would help to break up that loop into two loops. > > Make the callback into a batch operation that generates state that > > describes what to do with each and

Re: Grammar railroad diagram

2021-07-09 Thread Domingo Alvarez Duarte
Hello Andres ! There is two ways to achieve it: -1 I just add the bison grammar in CocoR format here https://github.com/mingodad/CocoR-CPP in the examples folder. -2 I created an small extension to Bison to do the same and published the patch here

Re: Incorrect usage of strtol, atoi for non-numeric junk inputs

2021-07-09 Thread Kyotaro Horiguchi
Thank you for the comments. At Fri, 9 Jul 2021 10:29:07 +0900, Michael Paquier wrote in > On Thu, Jul 08, 2021 at 05:30:23PM +0900, Kyotaro Horiguchi wrote: > > [1] is trying to expose pg_strtoint16/32 to frontend, but I don't see > > much point in doing that in conjunction with [2] or this

Re: Teach pg_receivewal to use lz4 compression

2021-07-09 Thread gkokolatos
‐‐‐ Original Message ‐‐‐ On Friday, July 9th, 2021 at 04:49, Michael Paquier wrote: > On Thu, Jul 08, 2021 at 02:18:40PM +, gkokola...@pm.me wrote: > > > please find v2 of the patch which tries to address the commends > > > > received so far. > > Thanks! > > > Michael Paquier

psql tab auto-complete for CREATE PUBLICATION

2021-07-09 Thread Peter Smith
I found that the psql tab auto-complete was not working for some cases of CREATE PUBLICATION [1]. CREATE PUBLICATION name [ FOR TABLE [ ONLY ] table_name [ * ] [, ...] | FOR ALL TABLES ] [ WITH ( publication_parameter [= value] [, ... ] ) ] ~~~ For example, the following scenarios

Re: [PoC] Improve dead tuple storage for lazy vacuum

2021-07-09 Thread Masahiko Sawada
On Fri, Jul 9, 2021 at 2:37 PM Andres Freund wrote: > > Hi, > > On 2021-07-08 20:53:32 -0700, Andres Freund wrote: > > On 2021-07-07 20:46:38 +0900, Masahiko Sawada wrote: > > > 1. Don't allocate more than 1GB. There was a discussion to eliminate > > > this limitation by using

Re: [PATCH] expand the units that pg_size_pretty supports on output

2021-07-09 Thread David Rowley
On Thu, 8 Jul 2021 at 05:44, David Christensen wrote: > Enclosed is the patch to change the return type to numeric, as well as one > for expanding units to > add PB and EB. I ended up not changing the return type of pg_size_bytes(). > I figured that PB and EB are probably good enough additions

Re: [HACKERS] logical decoding of two-phase transactions

2021-07-09 Thread Ajin Cherian
On Fri, Jul 9, 2021 at 9:13 AM Peter Smith wrote: > I tried the v95-0001 patch. > > - The patch applied cleanly and all build / testing was OK. > - The documentation also builds OK. > - I checked all v95-0001 / v93-0001 differences and found no problems. > - Furthermore, I noted that v95-0001

Re: Added schema level support for publication.

2021-07-09 Thread Greg Nancarrow
On Fri, Jul 9, 2021 at 1:28 PM houzj.f...@fujitsu.com wrote: > > Currently, postgres caches publication actions info in the > RelationData::rd_pubactions, but after applying the patch, it seems > rd_pubactions is not initialized when using schema level publication. > > It cound result in some

Re: [PoC] Improve dead tuple storage for lazy vacuum

2021-07-09 Thread Masahiko Sawada
On Fri, Jul 9, 2021 at 12:53 PM Andres Freund wrote: > > Hi, > > > On 2021-07-07 20:46:38 +0900, Masahiko Sawada wrote: > > 1. Don't allocate more than 1GB. There was a discussion to eliminate > > this limitation by using MemoryContextAllocHuge() but there were > > concerns about point 2[1]. > >

Re: Transactions involving multiple postgres foreign servers, take 2

2021-07-09 Thread Fujii Masao
On 2021/06/30 10:05, Masahiko Sawada wrote: I've attached the new version patch that incorporates the comments from Fujii-san and Ikeda-san I got so far. Thanks for updating the patches! I'm now reading 0001 and 0002 patches and wondering if we can commit them at first because they just

Re: Outdated comments about proc->sem in lwlock.c

2021-07-09 Thread Thomas Munro
On Thu, Jul 8, 2021 at 8:48 AM Daniel Gustafsson wrote: > > On 3 Jun 2021, at 04:07, Thomas Munro wrote: > > Here's a patch to remove the misleading comments. > > While not an expert in the area; reading the referenced commit and the code > with the now removed comments, I think this is correct.

Re: when the startup process doesn't (logging startup delays)

2021-07-09 Thread Amul Sul
Few comments for v4 patch: @@ -7351,6 +7363,8 @@ StartupXLOG(void) (errmsg("redo starts at %X/%X", LSN_FORMAT_ARGS(ReadRecPtr; + InitStartupProgress(); + /* * main redo apply loop */ @@ -7358,6