Re: Use generation context to speed up tuplesorts

2022-01-19 Thread Ronan Dunklau
Le jeudi 20 janvier 2022, 02:31:15 CET David Rowley a écrit : > On Tue, 18 Jan 2022 at 19:45, Julien Rouhaud wrote: > > I'm also a bit confused about which patch(es) should actually be reviewed > > in that thread. My understanding is that the original patch might not be > > relevant anymore but

RE: Support tab completion for upper character inputs in psql

2022-01-19 Thread tanghy.f...@fujitsu.com
On Sunday, January 16, 2022 3:51 AM, Tom Lane said: > Peter Eisentraut writes: > > The rest of the patch seems ok in principle, since AFAICT enums are the > > only query result in tab-complete.c that are not identifiers and thus > > subject to case issues. > > I spent some time looking at this

Re: Refactoring of compression options in pg_basebackup

2022-01-19 Thread Michael Paquier
On Wed, Jan 19, 2022 at 08:35:23AM -0500, Robert Haas wrote: > I think that this will reject something like --compress=nonetheless by > telling you that 't' is not a valid separator. I think it would be > better to code this so that we first identify the portion preceding > the first colon, or the

RE: [Proposal] Add foreign-server health checks infrastructure

2022-01-19 Thread kuroda.hay...@fujitsu.com
Dear Zhihong, Thank you for reviewing! And I have to apologize for the late. I'll post new patchset later. > + UnregisterAllCheckingRemoteServersCallback(); > > UnregisterAllCheckingRemoteServersCallback > -> UnregisterAllCheckingRemoteServersCallbacks Will fix. > +

Re: TAP test to cover "EndOfLogTLI != replayTLI" case

2022-01-19 Thread Amul Sul
On Tue, Jan 18, 2022 at 10:31 AM Kyotaro Horiguchi wrote: > > At Tue, 18 Jan 2022 12:25:15 +0800, Julien Rouhaud wrote > in > > Hi, > > > > On Mon, Jan 17, 2022 at 09:33:57PM +0800, Julien Rouhaud wrote: > > > > > > Thanks for the updated patch! Note that thanks to Andres and Thomas > > >

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2022-01-19 Thread Kyotaro Horiguchi
At Sun, 16 Jan 2022 12:43:03 +0800, Julien Rouhaud wrote in > Other than that, I see that the TAP tests are failing on all the environment, > due to Perl errors. For instance: Perl seems to have changed its behavior for undef hash. It is said that "if (%undef_hash)" is false but actually it

Re: Map WAL segment files on PMEM as WAL buffers

2022-01-19 Thread Takashi Menjo
Hi Justin, Here is patchset v8. It will have "make check-world" and Cirrus to pass. Would you try this one? The v8 squashes some patches in v7 into related ones, and adds the following patches: - v8-0003: Add wal_pmem_map to postgresql.conf.sample. It also helps v8-0011. - v8-0009: Fix wrong

Re: Refactoring of compression options in pg_basebackup

2022-01-19 Thread Michael Paquier
On Wed, Jan 19, 2022 at 12:50:44PM -0300, Alvaro Herrera wrote: > Note there is an extra [ before the {gzip bit. Thanks! -- Michael signature.asc Description: PGP signature

Re: POC: GROUP BY optimization

2022-01-19 Thread Andrey V. Lepikhov
I keep work on this patch. Here is intermediate results. On 7/22/21 3:58 AM, Tomas Vondra wrote: in the first loop. Which seems pretty bogus - why would there be just two groups? When processing the first expression, it's as if there was one big "prev group" with all the tuples, so why not to

Re: pg_upgrade should truncate/remove its logs before running

2022-01-19 Thread Michael Paquier
On Wed, Jan 19, 2022 at 09:59:14PM -0600, Justin Pryzby wrote: > We require that the dir not exist, by testing if (mkdir()). > So it's okay if someone specifies ../whatever or $CWD. What I am scared of here is the use of rmtree() if we allow something like that. So we should either keep the

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2022-01-19 Thread Kyotaro Horiguchi
At Mon, 17 Jan 2022 17:24:43 +0900 (JST), Kyotaro Horiguchi wrote in > At Sun, 16 Jan 2022 12:43:03 +0800, Julien Rouhaud wrote > in > > I'm not very familiar with windows, but maybe using strawberry perl instead > > ([1]) would fix your problem? I think it's also quite popular and is > >

Re: row filtering for logical replication

2022-01-19 Thread Peter Smith
On Thu, Jan 20, 2022 at 2:29 PM Amit Kapila wrote: > > On Thu, Jan 20, 2022 at 7:51 AM Peter Smith wrote: > > > > Here are some review comments for v68-0001. > > > > > > 3. src/backend/replication/pgoutput/pgoutput.c - pgoutput_row_filter_init > > > > + /* If no filter found, clean up the memory

Re: pg_upgrade should truncate/remove its logs before running

2022-01-19 Thread Justin Pryzby
On Thu, Jan 20, 2022 at 12:01:29PM +0900, Michael Paquier wrote: > On Wed, Jan 19, 2022 at 06:05:40PM -0600, Justin Pryzby wrote: > > > I'm not sure these restrictions are needed ? > > This could lead to issues with rmtree() if we are not careful enough, > no? We'd had our deal of argument

Re: row filtering for logical replication

2022-01-19 Thread Amit Kapila
On Thu, Jan 20, 2022 at 7:51 AM Peter Smith wrote: > > 5. DEBUG level 3 > > I found there are 3 debug logs in this patch and they all have DEBUG3 level. > > IMO it is probably OK as-is, > +1. > but just a comparison I noticed that the > most detailed logging for logical replication worker.c was

Re: row filtering for logical replication

2022-01-19 Thread Amit Kapila
On Thu, Jan 20, 2022 at 7:51 AM Peter Smith wrote: > > Here are some review comments for v68-0001. > > > 3. src/backend/replication/pgoutput/pgoutput.c - pgoutput_row_filter_init > > + /* If no filter found, clean up the memory and return */ > + if (!has_filter) > + { > + if

Re: Add last commit LSN to pg_last_committed_xact()

2022-01-19 Thread Andres Freund
Hi, On 2022-01-19 21:23:12 -0500, James Coleman wrote: > { oid => '3537', descr => 'get identification of SQL object', > diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h > index a5f9e9..2a026b0844 100644 > --- a/src/include/storage/proc.h > +++

Re: pg_upgrade should truncate/remove its logs before running

2022-01-19 Thread Michael Paquier
On Wed, Jan 19, 2022 at 06:05:40PM -0600, Justin Pryzby wrote: > I still don't know if it even needs to be configurable. I want this to be configurable to ease the switch of the pg_upgrade to TAP, moving the logs into a deterministic temporary location proper to each run. This makes reporting

Re: Null commitTS bug

2022-01-19 Thread Kyotaro Horiguchi
At Tue, 18 Jan 2022 13:48:11 +0900 (JST), Kyotaro Horiguchi wrote in > At Tue, 18 Jan 2022 10:43:55 +0900, Michael Paquier > wrote in > > On Sun, Jan 16, 2022 at 11:01:25PM -0500, Tom Lane wrote: > > > Isn't that a very bad way to write "i = j + 1"? > > > > > > I agree with Horiguchi-san

Re: Add checkpoint and redo LSN to LogCheckpointEnd log message

2022-01-19 Thread Kyotaro Horiguchi
At Thu, 20 Jan 2022 00:36:32 +, "Bossart, Nathan" wrote in > On 1/3/22, 5:52 PM, "Kyotaro Horiguchi" wrote: > > It seems to me "LSN" or just "location" is more confusing or > > mysterious than "REDO LSN" for the average user. If we want to avoid > > being technically too detailed, we would

Re: Document atthasmissing default optimization avoids verification table scan

2022-01-19 Thread David G. Johnston
On Wed, Jan 19, 2022 at 6:14 PM James Coleman wrote: > I'm open to the idea of wordsmithing here, of course, but I strongly > disagree that this is irrelevant data. Ok, but wording aside, only changing a tip in the DDL - Add Table section doesn't seem like a complete fix. The notes in alter

Re: Add last commit LSN to pg_last_committed_xact()

2022-01-19 Thread James Coleman
On Tue, Jan 18, 2022 at 9:19 PM Andres Freund wrote: > > > + LWLockAcquire(ProcArrayLock, LW_SHARED); > > + lsn = ShmemVariableCache->finishedProcsLastCommitLSN; > > + for (index = 0; index < ProcGlobal->allProcCount; index++) > > + { > > + XLogRecPtr procLSN = > >

Re: row filtering for logical replication

2022-01-19 Thread Peter Smith
Here are some review comments for v68-0001. ~~~ 1. Commit message "When a publication is defined or modified, rows that don't satisfy an optional WHERE clause will be filtered out." That wording seems strange to me - it sounds like the filtering takes place at the point of creating/altering.

Re: slowest tap tests - split or accelerate?

2022-01-19 Thread Andres Freund
Hi, On 2022-01-19 09:42:31 -0800, Andres Freund wrote: > Both ours have this. Unfortunately on windows cp doesn't natively > exist. Although git does provide it. I tried a few things that appear to be > natively available (time is best of three executions): > > gnu cp from git, cp -a

Re: Use generation context to speed up tuplesorts

2022-01-19 Thread David Rowley
On Tue, 18 Jan 2022 at 19:45, Julien Rouhaud wrote: > I'm also a bit confused about which patch(es) should actually be reviewed in > that thread. My understanding is that the original patch might not be > relevant > anymore but I might be wrong. The CF entry should probably be updated to >

Re: Replace uses of deprecated Python module distutils.sysconfig

2022-01-19 Thread Tom Lane
=?UTF-8?Q?Juan_Jos=C3=A9_Santamar=C3=ADa_Flecha?= writes: > On Wed, Jan 19, 2022 at 9:40 PM Tom Lane wrote: >> Done. (I couldn't find any equivalent logic in the MSVC build scripts >> though; is there something I missed?) > MSVC will use the path configured in src\tools\msvc\config.pl >

Re: Document atthasmissing default optimization avoids verification table scan

2022-01-19 Thread James Coleman
On Wed, Jan 19, 2022 at 7:51 PM David G. Johnston wrote: > > On Wed, Jan 19, 2022 at 5:08 PM Bossart, Nathan wrote: >> >> On 9/24/21, 7:30 AM, "James Coleman" wrote: >> > When PG11 added the ability for ALTER TABLE ADD COLUMN to set a constant >> > default value without rewriting the table the

Re: Remove redundant MemoryContextSwith in BeginCopyFrom

2022-01-19 Thread Japin Li
On Thu, 20 Jan 2022 at 08:50, Tom Lane wrote: > Japin Li writes: >> I see you have already push this patch on master (89f059bdf52), why not >> remove MemoryContextSwitchTo in the middle of BeginCopyTo in this commit? > > That was a different suggestion from a different person, so I didn't >

Re: Document atthasmissing default optimization avoids verification table scan

2022-01-19 Thread David G. Johnston
On Wed, Jan 19, 2022 at 5:08 PM Bossart, Nathan wrote: > On 9/24/21, 7:30 AM, "James Coleman" wrote: > > When PG11 added the ability for ALTER TABLE ADD COLUMN to set a constant > > default value without rewriting the table the doc changes did not note > > how the new feature interplayed with

Re: Remove redundant MemoryContextSwith in BeginCopyFrom

2022-01-19 Thread Tom Lane
Japin Li writes: > I see you have already push this patch on master (89f059bdf52), why not > remove MemoryContextSwitchTo in the middle of BeginCopyTo in this commit? That was a different suggestion from a different person, so I didn't want to muddle the credit. Also, it requires a bit of

Re: Remove redundant MemoryContextSwith in BeginCopyFrom

2022-01-19 Thread Japin Li
On Thu, 20 Jan 2022 at 00:38, Tom Lane wrote: > Bharath Rupireddy writes: >> +1. It looks like a thinko from c532d15d. There's no code in between, >> so switching to oldcontext doesn't make sense. > > Agreed. > >> I think we also need to remove MemoryContextSwitchTo(oldcontext); at >> the end

Re: Add checkpoint and redo LSN to LogCheckpointEnd log message

2022-01-19 Thread Bossart, Nathan
On 1/3/22, 5:52 PM, "Kyotaro Horiguchi" wrote: > It seems to me "LSN" or just "location" is more confusing or > mysterious than "REDO LSN" for the average user. If we want to avoid > being technically too detailed, we would use just "start LSN=%X/%X, > end LSN=%X/%X". And it is equivalent to

Re: Document atthasmissing default optimization avoids verification table scan

2022-01-19 Thread Bossart, Nathan
On 9/24/21, 7:30 AM, "James Coleman" wrote: > When PG11 added the ability for ALTER TABLE ADD COLUMN to set a constant > default value without rewriting the table the doc changes did not note > how the new feature interplayed with ADD COLUMN DEFAULT NOT NULL. > Previously such a new column

Re: pg_upgrade should truncate/remove its logs before running

2022-01-19 Thread Justin Pryzby
On Wed, Jan 19, 2022 at 05:13:18PM +0900, Michael Paquier wrote: > On Tue, Jan 11, 2022 at 10:08:13PM -0600, Justin Pryzby wrote: > > I asked about that before. Right now, it'll exit(1) when mkdir fails. > > > > I had written a patch to allow "." by skipping mkdir (or allowing it to > > fail if

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-01-19 Thread Tom Lane
"Bossart, Nathan" writes: > I think the other side of this is that we don't want checkpointing to > continually fail because of a noncritical failure. That could also > lead to problems down the road. Yeah, a persistent failure to complete checkpoints is very nasty. Your disk will soon fill

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-01-19 Thread Bossart, Nathan
On 1/19/22, 11:08 AM, "Andres Freund" wrote: > On 2022-01-19 13:34:21 -0500, Tom Lane wrote: >> As far as the patch itself goes, I agree that failure to unlink >> is noncritical, because such a file would have no further effect >> and we can just ignore it. > > I don't agree. We iterate through

Re: Replace uses of deprecated Python module distutils.sysconfig

2022-01-19 Thread Tom Lane
I wrote: > Peter Eisentraut writes: >> On 19.01.22 01:18, Tom Lane wrote: >>> ... I'm tempted to resurrect the >>> idea of changing configure's probe order to "python3 python python2" >>> in the meantime, just so we can see how much of the buildfarm is ready >>> for that. >> This seems sensible

Re: missing indexes in indexlist with partitioned tables

2022-01-19 Thread Alvaro Herrera
On 2022-Jan-19, Arne Roland wrote: > > a) make sure that the submitted patch updates these test results so > > that the test pass [...] > > Just for the record: I did run the tests, but I did miss that the > commit of Tomas fix for fractional optimization is already on the > master. Please note

Re: do only critical work during single-user vacuum?

2022-01-19 Thread Michael Paquier
On Wed, Jan 19, 2022 at 09:11:48PM +, Bossart, Nathan wrote: > I personally think VACUUM FOR WRAPAROUND is the best of the options > provided thus far. Could you avoid introducing a new grammar pattern in VACUUM? Any new option had better be within the parenthesized part as it is extensible

Re: missing indexes in indexlist with partitioned tables

2022-01-19 Thread Zhihong Yu
On Wed, Jan 19, 2022 at 1:50 PM Arne Roland wrote: > Hi, > > > I came up with a slightly more intuitive way to force the different > outcome for the fractional test, that does hardly change anything. > > > I'm not sure, whether the differentiation between fract_x and fract_t is > worth it, since

Re: Schema variables - new implementation for Postgres 15

2022-01-19 Thread Julien Rouhaud
Hi, On Wed, Jan 19, 2022 at 09:09:41PM +0100, Pavel Stehule wrote: > st 19. 1. 2022 v 9:01 odesílatel Julien Rouhaud napsal: > > RAISE NOTICE should use local variables, and in this case is a question if we > should raise a warning. There are two possible analogies - we can see session >

Re: missing indexes in indexlist with partitioned tables

2022-01-19 Thread Arne Roland
Hi, I came up with a slightly more intuitive way to force the different outcome for the fractional test, that does hardly change anything. I'm not sure, whether the differentiation between fract_x and fract_t is worth it, since there shouldn't be any difference, but as mentioned before I

Re: refactoring basebackup.c

2022-01-19 Thread Robert Haas
On Wed, Jan 19, 2022 at 7:16 AM Dipesh Pandit wrote: > I have done initial testing and > working on updating the test coverage. I spent some time thinking about test coverage for the server-side backup code today and came up with the attached (v12-0003). It does an end-to-end test that exercises

Re: Replace uses of deprecated Python module distutils.sysconfig

2022-01-19 Thread Juan José Santamaría Flecha
On Wed, Jan 19, 2022 at 9:40 PM Tom Lane wrote: > > Done. (I couldn't find any equivalent logic in the MSVC build scripts > though; is there something I missed?) > > MSVC will use the path configured in src\tools\msvc\config.pl > $config->{"python"}, there is no ambiguity. Regards, Juan José

Re: missing indexes in indexlist with partitioned tables

2022-01-19 Thread Arne Roland
Hi! > From: Alvaro Herrera > [...] > Hmm, these plan changes look valid to me. A left self-join using the > primary key column? That looks optimizable all right. > [...] > What I still don't know is whether this patch is actually desirable or > not. If the only cases it affects is self-joins,

Re: do only critical work during single-user vacuum?

2022-01-19 Thread Bossart, Nathan
On 1/19/22, 11:15 AM, "John Naylor" wrote: > This seems to be the motivating reason for wanting new configurability > on the server side. In any case, new knobs are out of scope for this > thread. If the use case is compelling enough, may I suggest starting a > new thread? Sure. Perhaps the new

Re: Adding CI to our tree

2022-01-19 Thread Tom Lane
Andres Freund writes: > I'm doubtful that it's good that we use BAIL_OUT so liberally, because it > prevents further tests from running (i.e. if 001 bails, 002+ doesn't run), > which doesn't generally seem like the right thing to do after a single test > fails. But that's really independent of

Re: Replace uses of deprecated Python module distutils.sysconfig

2022-01-19 Thread Tom Lane
Peter Eisentraut writes: > On 19.01.22 01:18, Tom Lane wrote: >> ... I'm tempted to resurrect the >> idea of changing configure's probe order to "python3 python python2" >> in the meantime, just so we can see how much of the buildfarm is ready >> for that. > This seems sensible in any case,

Re: Fix BUG #17335: Duplicate result rows in Gather node

2022-01-19 Thread David Rowley
On Fri, 31 Dec 2021 at 00:14, Yura Sokolov wrote: > Suggested quick (and valid) fix in the patch attached: > - If Append has single child, then copy its parallel awareness. I've been looking at this and I've gone through changing my mind about what's the right fix quite a number of times. My

Re: Adding CI to our tree

2022-01-19 Thread Andres Freund
Hi, On 2022-01-19 15:05:44 -0500, Tom Lane wrote: > And the cause of that is obvious: Cluster::start thinks that if "pg_ctl > start" failed, there couldn't possibly be a postmaster running. So it > doesn't bother to update self->_pid, and then the END hook thinks there is > nothing to do. Ah,

Re: Schema variables - new implementation for Postgres 15

2022-01-19 Thread Pavel Stehule
st 19. 1. 2022 v 9:01 odesílatel Julien Rouhaud napsal: > Hi, > > On Tue, Jan 18, 2022 at 10:01:01PM +0100, Pavel Stehule wrote: > > pá 14. 1. 2022 v 3:44 odesílatel Julien Rouhaud > napsal: > > > > > > > > =# let myvariable = 'AA'; > > > LET > > > > > > =# select 'AA' collate "en-x-icu" <

Re: Adding CI to our tree

2022-01-19 Thread Tom Lane
I wrote: > This test attempt revealed another problem too: the standby never > shut down, and thus the calling "make" never quit, until I intervened > manually. I'm not sure why. I see that Cluster::promote uses > system_or_bail() to run "pg_ctl promote" ... could it be that > BAIL_OUT causes

Re: do only critical work during single-user vacuum?

2022-01-19 Thread Bossart, Nathan
On 1/18/22, 9:47 PM, "Masahiko Sawada" wrote: > IIUC what we want to do here are two things: (1) select only old > tables and (2) set INDEX_CLEANUP = off, TRUNCATE = off, and FREEZE = > on. VACUUM LIMIT statement does both things at the same time. Although > I’m concerned a bit about its

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-01-19 Thread Peter Geoghegan
On Wed, Jan 19, 2022 at 6:56 AM Robert Haas wrote: > I don't think I've said anything on this thread that is an attack on > you. I am getting pretty frustrated with the tenor of the discussion, > though. I feel like you're the one attacking me, and I don't like it. "Attack" is a strong word

Re: Time to increase hash_mem_multiplier default?

2022-01-19 Thread John Naylor
On Sun, Jan 16, 2022 at 7:28 PM Peter Geoghegan wrote: > > The current hash_mem_multiplier default is 1.0, which is a fairly > conservative default: it preserves the historic behavior, which is > that hash-based executor nodes receive the same work_mem budget as > sort-based nodes. I propose that

Re: speed up text_position() for utf-8

2022-01-19 Thread John Naylor
I wrote: > 0001 puts the main implementation of pg_utf_mblen() into an inline > function and uses this in pg_mblen(). This is somewhat faster in the > strpos tests, so that gives some measure of the speedup expected for > other callers. Text search seems to call this a lot, so this might > have

Re: do only critical work during single-user vacuum?

2022-01-19 Thread John Naylor
On Wed, Jan 19, 2022 at 12:46 AM Masahiko Sawada wrote: > > On Fri, Jan 14, 2022 at 7:04 AM Bossart, Nathan wrote: > > > > I guess I'm ultimately imagining the new options as replacing the > > vacuumdb implementation. IOW vacuumdb would just use MIN_(M)XID_AGE > > behind the scenes (as would a

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-01-19 Thread Andres Freund
Hi, On 2022-01-19 13:34:21 -0500, Tom Lane wrote: > Bharath Rupireddy writes: > > On Sat, Jan 15, 2022 at 2:59 PM Julien Rouhaud wrote: > >> Maybe I'm missing something but wouldn't > >> https://commitfest.postgresql.org/36/3448/ better solve the problem? > > > The error can cause the new

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-01-19 Thread Tom Lane
I wrote: > Anyway, I think possibly we can drop the bottom half of the loop > (the part trying to fsync non-removed files) in favor of fsync'ing > the directory afterwards. Thoughts? Oh, scratch that --- *this* loop doesn't care about the file contents, but other code does. However, don't we

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-01-19 Thread Tom Lane
Bharath Rupireddy writes: > On Sat, Jan 15, 2022 at 2:59 PM Julien Rouhaud wrote: >> Maybe I'm missing something but wouldn't >> https://commitfest.postgresql.org/36/3448/ better solve the problem? > The error can cause the new background process proposed there in that > thread to restart,

Re: Compiling PostgreSQL for WIndows with 16kb blocksize

2022-01-19 Thread Andres Freund
Hi, On January 19, 2022 9:07:50 AM PST, Yannick Collette wrote: >Hello, > >For test purposes I need to compile PostgreSQL 14.1 using a 16kb blocksize. > >CFLAGS="-D WINVER=0x0600 -D _WIN32_WINNT=0x0600" LIBS="-ladvapi32" > ./configure --host=x86_64-w64-mingw32 --with-blocksize=16

Re: slowest tap tests - split or accelerate?

2022-01-19 Thread Andres Freund
Hi, On 2022-01-19 12:14:21 -0500, Tom Lane wrote: > No, it was largely the same as what you have here, I think. I dug > up my WIP patch and attach it below, just in case there's any ideas > worth borrowing. Heh, it does look quite similar. > + "cp -a

Re: Compiling PostgreSQL for WIndows with 16kb blocksize

2022-01-19 Thread Tom Lane
Yannick Collette writes: > For test purposes I need to compile PostgreSQL 14.1 using a 16kb blocksize. > CFLAGS="-D WINVER=0x0600 -D _WIN32_WINNT=0x0600" LIBS="-ladvapi32" > ./configure --host=x86_64-w64-mingw32 --with-blocksize=16 > --with-wal-blocksize=16 --with-openssl --with-libxml >

Re: slowest tap tests - split or accelerate?

2022-01-19 Thread Tom Lane
Andres Freund writes: > On 2022-01-19 11:54:01 -0500, Tom Lane wrote: >> Me too ;-). As I remarked earlier, I'd tried this once before and >> gave up because it didn't seem to be winning much. But that was >> before we had so many initdb's triggered by TAP tests, I think. > What approach did

Compiling PostgreSQL for WIndows with 16kb blocksize

2022-01-19 Thread Yannick Collette
Hello, For test purposes I need to compile PostgreSQL 14.1 using a 16kb blocksize. CFLAGS="-D WINVER=0x0600 -D _WIN32_WINNT=0x0600" LIBS="-ladvapi32" ./configure --host=x86_64-w64-mingw32 --with-blocksize=16 --with-wal-blocksize=16 --with-openssl --with-libxml --prefix=/c/postgresql/pg14/ 2>&1

Re: slowest tap tests - split or accelerate?

2022-01-19 Thread Andres Freund
Hi, On 2022-01-19 11:54:01 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2022-01-18 13:40:40 -0800, Andres Freund wrote: > >> Maybe we really should do at least the most simplistic caching for > >> initdbs, by > >> doing one initdb as part of the creation of temp_install. Then > >>

Re: slowest tap tests - split or accelerate?

2022-01-19 Thread Tom Lane
Andres Freund writes: > On 2022-01-18 13:40:40 -0800, Andres Freund wrote: >> Maybe we really should do at least the most simplistic caching for initdbs, >> by >> doing one initdb as part of the creation of temp_install. Then Cluster::init >> would need logic to only use that if $params{extra}

Re: Replace uses of deprecated Python module distutils.sysconfig

2022-01-19 Thread Peter Eisentraut
On 19.01.22 01:18, Tom Lane wrote: Anyway, based on these results, we might have better luck switching to sysconfig after we start forcing python3. I'm tempted to resurrect the idea of changing configure's probe order to "python3 python python2" in the meantime, just so we can see how much of

Re: Remove redundant MemoryContextSwith in BeginCopyFrom

2022-01-19 Thread Tom Lane
Bharath Rupireddy writes: > +1. It looks like a thinko from c532d15d. There's no code in between, > so switching to oldcontext doesn't make sense. Agreed. > I think we also need to remove MemoryContextSwitchTo(oldcontext); at > the end of BeginCopyTo in copyto.c, because we are not changing

Re: refactoring basebackup.c

2022-01-19 Thread tushar
On 1/18/22 8:12 PM, Jeevan Ladhe wrote: Similar to LZ4 server-side compression, I have also tried to add a ZSTD server-side compression in the attached patch. Thanks Jeevan. while testing found one scenario where the server is getting crash while performing pg_basebackup against

Re: Replace uses of deprecated Python module distutils.sysconfig

2022-01-19 Thread Tom Lane
I wrote: > Anyway, based on these results, we might have better luck switching to > sysconfig after we start forcing python3. On the other hand, that answer is not back-patchable, and we surely need a back-patchable fix, because people will try to build the back branches against newer pythons.

Re: A qsort template

2022-01-19 Thread John Naylor
On Tue, Jan 18, 2022 at 9:58 PM Peter Geoghegan wrote: > > On Tue, Jan 18, 2022 at 6:39 PM John Naylor > wrote: > > Editorializing the null position in queries is not very common in my > > experience. Not null is interesting since it'd be trivial to pass > > constant false to the same

Re: refactoring basebackup.c

2022-01-19 Thread Robert Haas
On Wed, Jan 19, 2022 at 7:16 AM Dipesh Pandit wrote: > I have added support for decompressing a gzip compressed tar file > at client. pg_basebackup can enable server side compression for > plain format backup with this change. > > Added a gzip extractor which decompresses the compressed archive >

Re: Refactoring of compression options in pg_basebackup

2022-01-19 Thread Alvaro Herrera
On 2022-Jan-19, Michael Paquier wrote: > + printf(_(" -Z, --compress=[{gzip,none}[:LEVEL] or [LEVEL]\n" > + " compress tar output with > given compression method or level\n")); Note there is an extra [ before the {gzip bit. -- Álvaro Herrera

Re: Remove redundant MemoryContextSwith in BeginCopyFrom

2022-01-19 Thread Bharath Rupireddy
On Wed, Jan 19, 2022 at 7:51 PM Japin Li wrote: > > > Hi, hackers > > When I read the code of COPY ... FROM ..., I find there is a redundant > MemoryContextSwith() in BeginCopyFrom(). In BeginCopyFrom, it creates > a COPY memory context and then switches to it, in the middle of this > function,

Re: Remove redundant MemoryContextSwith in BeginCopyFrom

2022-01-19 Thread Fabrízio de Royes Mello
On Wed, Jan 19, 2022 at 11:21 AM Japin Li wrote: > > > Hi, hackers > > When I read the code of COPY ... FROM ..., I find there is a redundant > MemoryContextSwith() in BeginCopyFrom(). In BeginCopyFrom, it creates > a COPY memory context and then switches to it, in the middle of this > function,

Re: New developer papercut - Makefile references INSTALL

2022-01-19 Thread Tom Lane
Daniel Gustafsson writes: > I ended up pushing this with the URL in place as there IMO was consensus in > the > thread for including it. We could if we want to update it to point to v15 > docs > once we branch off, but anything more than that is probably in the diminishing > returns territory

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-01-19 Thread Robert Haas
On Tue, Jan 18, 2022 at 1:48 PM Peter Geoghegan wrote: > That's what I was reacting to -- it had nothing to do with any > concerns you may have had. I wasn't thinking about long-idle cursors > at all. I was defending myself, because I was put in a position where > I had to defend myself. I don't

Re: [ERROR] Copy from CSV fails due to memory error.

2022-01-19 Thread Tomas Vondra
On 1/19/22 14:01, Kostas Chasialis wrote: Hey. I am facing an issue when I try to run the following command COPY FROM WITH DELIMITER E',’; This file, is rather large, it's around 178GBs. When I try to run this COPY command I get the following error: ERROR: out of memory DETAIL:

Re: Pluggable toaster

2022-01-19 Thread Nikita Malakhov
Hi, >I'm not convinced that's universally true. Yes, I'm sure certain TOAST >implementations would benefit from tighter control over compression, but >does that imply compression and toast are redundant? I doubt that, >because we compress non-toasted types too, for example. And layering has >a

Remove redundant MemoryContextSwith in BeginCopyFrom

2022-01-19 Thread Japin Li
Hi, hackers When I read the code of COPY ... FROM ..., I find there is a redundant MemoryContextSwith() in BeginCopyFrom(). In BeginCopyFrom, it creates a COPY memory context and then switches to it, in the middle of this function, it switches to the oldcontext and immediately switches back to

Re: New developer papercut - Makefile references INSTALL

2022-01-19 Thread Daniel Gustafsson
> On 18 Jan 2022, at 17:05, Peter Eisentraut > wrote: > On 18.01.22 16:51, Daniel Gustafsson wrote: >> I plan on applying the attached which address the feedback given. > > The indentation of the two INSTRUCTIONS= lines uses a different mix of tabs > and spaces, so it looks a bit weird

Re: Logical replication timeout problem

2022-01-19 Thread Fabrice Chapuis
Hello Amit, If it takes little work for you, can you please send me a piece of code with the change needed to do the test Thanks Regards, Fabrice On Fri, Jan 14, 2022 at 1:03 PM Amit Kapila wrote: > On Fri, Jan 14, 2022 at 3:47 PM Fabrice Chapuis > wrote: > > > > If I can follow

Re: Refactoring of compression options in pg_basebackup

2022-01-19 Thread Robert Haas
On Tue, Jan 18, 2022 at 11:27 PM Michael Paquier wrote: > WFM. Attached is a patch that extends --compress to handle a method > with an optional compression level. Some extra tests are added to > cover all that. I think that this will reject something like --compress=nonetheless by telling you

Re: pgsql: Modify pg_basebackup to use a new COPY subprotocol for base back

2022-01-19 Thread Robert Haas
On Tue, Jan 18, 2022 at 9:29 PM Tom Lane wrote: > Robert Haas writes: > > Here's a patch, based in part on some off-list discussion with Andres. > > I believe Andres has already confirmed that this fix works, but it > > wouldn't hurt if Tom wants to verify it also. > > WFM too --- at least,

Re: row filtering for logical replication

2022-01-19 Thread Amit Kapila
On Tue, Jan 18, 2022 at 10:23 AM Greg Nancarrow wrote: > > On Tue, Jan 18, 2022 at 2:31 PM Amit Kapila wrote: > > > > On Tue, Jan 18, 2022 at 8:41 AM Greg Nancarrow wrote: > > > > > > On Tue, Jan 18, 2022 at 2:31 AM houzj.f...@fujitsu.com > > > wrote: > > > > > > > > > (2)

[ERROR] Copy from CSV fails due to memory error.

2022-01-19 Thread Kostas Chasialis
Hey. I am facing an issue when I try to run the following command COPY FROM WITH DELIMITER E',’; This file, is rather large, it's around 178GBs. When I try to run this COPY command I get the following error: ERROR: out of memory DETAIL: Failed on request of size 2048 in memory context

Re: WIN32 pg_import_system_collations

2022-01-19 Thread Juan José Santamaría Flecha
On Wed, Jan 19, 2022 at 10:53 AM Julien Rouhaud wrote: > > This version doesn't apply anymore: > > Thanks for the heads up. Please find attached a rebased patch. I have also rewritten some comments to address previous reviews, code and test remain the same. Regards, Juan José Santamaría

Re: refactoring basebackup.c

2022-01-19 Thread Dipesh Pandit
Hi, I have added support for decompressing a gzip compressed tar file at client. pg_basebackup can enable server side compression for plain format backup with this change. Added a gzip extractor which decompresses the compressed archive and forwards it to the next streamer. I have done initial

Re: [PATCH] Add reloption for views to enable RLS

2022-01-19 Thread Christoph Heiss
Hi, On 1/19/22 09:30, Julien Rouhaud wrote: Hi, On Tue, Jan 18, 2022 at 04:16:53PM +0100, Christoph Heiss wrote: I've attached a v2 where I addressed the things you mentioned. This version unfortunately doesn't apply anymore: http://cfbot.cputube.org/patch_36_3466.log === Applying patches

Re: generic plans and "initial" pruning

2022-01-19 Thread Simon Riggs
On Wed, 19 Jan 2022 at 08:31, Amit Langote wrote: > > Maybe force all DDL, or just DDL that would cause safety issues, to > > update a hierarchy version number, so queries can tell whether they > > need to replan. Don't know, just looking for an O(1) solution. > > Yeah, it would be great if it

Re: a misbehavior of partition row movement (?)

2022-01-19 Thread Amit Langote
On Wed, Jan 19, 2022 at 6:26 PM Alvaro Herrera wrote: > On 2022-Jan-19, Amit Langote wrote: > > BTW, your tweaks patch added this: > > > > + * *inserted_tuple is the tuple that's effectively inserted; > > + * *inserted_destrel is the relation where it was inserted. > > + * These are

Re: pg_upgrade should truncate/remove its logs before running

2022-01-19 Thread Michael Paquier
On Wed, Jan 19, 2022 at 05:13:18PM +0900, Michael Paquier wrote: > I have noticed a couple of incorrect things in the docs, and some > other things. It is a bit late here, so I may have missed a couple of > things but I'll look at this stuff once again in a couple of days. And the docs failed to

Re: using extended statistics to improve join estimates

2022-01-19 Thread Julien Rouhaud
Hi, On Tue, Jan 04, 2022 at 03:55:50PM -0800, Andres Freund wrote: > On 2022-01-01 18:21:06 +0100, Tomas Vondra wrote: > > Here's an updated patch, rebased and fixing a couple typos reported by > > Justin Pryzby directly. > > FWIW, cfbot reports a few compiler warnings: Also the patch doesn't

Re: row filtering for logical replication

2022-01-19 Thread Amit Kapila
On Wed, Jan 19, 2022 at 7:45 AM houzj.f...@fujitsu.com wrote: > > On Tues, Jan 18, 2022 8:35 PM Amit Kapila wrote: > > Attach the V67 patch set which address the above comments. > Some more comments and suggestions: = 1. Can we do slot initialization in

Re: WIN32 pg_import_system_collations

2022-01-19 Thread Julien Rouhaud
Hi, On Mon, Dec 13, 2021 at 05:28:47PM +0100, Juan José Santamaría Flecha wrote: > On Mon, Dec 13, 2021 at 9:41 AM Juan José Santamaría Flecha < > juanjo.santama...@gmail.com> wrote: > > Per path tester. This version doesn't apply anymore: http://cfbot.cputube.org/patch_36_3450.log ===

Re: a misbehavior of partition row movement (?)

2022-01-19 Thread Alvaro Herrera
On 2022-Jan-19, Amit Langote wrote: > BTW, your tweaks patch added this: > > + * *inserted_tuple is the tuple that's effectively inserted; > + * *inserted_destrel is the relation where it was inserted. > + * These are only set on success. FIXME -- see what happens on > the "do

Re: Support for NSS as a libpq TLS backend

2022-01-19 Thread Daniel Gustafsson
> On 18 Jan 2022, at 17:37, Jacob Champion wrote: > > On Wed, 2021-12-15 at 23:10 +0100, Daniel Gustafsson wrote: >> I've attached a v50 which fixes the issues found by Joshua upthread, as well >> as >> rebases on top of all the recent SSL and pgcrypto changes. > > I'm currently tracking down

Re: Support tab completion for upper character inputs in psql

2022-01-19 Thread Julien Rouhaud
Hi, On Sat, Jan 15, 2022 at 01:51:26PM -0500, Tom Lane wrote: > > I spent some time looking at this patch. I'm not very happy with it, > for two reasons: > [...] On top of that the patch doesn't apply anymore: http://cfbot.cputube.org/patch_36_2979.log === Applying patches on top of

Re: Skipping logical replication transactions on subscriber side

2022-01-19 Thread Amit Kapila
On Wed, Jan 19, 2022 at 12:46 PM Masahiko Sawada wrote: > > On Wed, Jan 19, 2022 at 12:22 PM osumi.takami...@fujitsu.com > wrote: > > > > (6) apply_handle_stream_abort > > > > @@ -1209,6 +1300,13 @@ apply_handle_stream_abort(StringInfo s) > > > > logicalrep_read_stream_abort(s, , ); > >

Re: Showing I/O timings spent reading/writing temp buffers in EXPLAIN

2022-01-19 Thread Julien Rouhaud
Hi, On Tue, Nov 16, 2021 at 04:37:44PM +0900, Masahiko Sawada wrote: > > I've attached an updated patch. Please review it. It seems that the regression tests aren't entirely stable, per cfbot: https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest/36/3298. The failures look like:

  1   2   >