Re: Disallow cancellation of waiting for synchronous replication

2021-03-11 Thread David Steele
On 12/9/20 4:07 AM, Andrey Borodin wrote: 9 июня 2020 г., в 23:32, Jeff Davis написал(а): After using a patch for a while it became obvious that PANICing during termination is not a good idea. Even when we wait for synchronous replication. It generates undesired coredumps. I think in

Re: Disallow cancellation of waiting for synchronous replication

2021-03-11 Thread Fujii Masao
On 2020/12/09 18:07, Andrey Borodin wrote: 9 июня 2020 г., в 23:32, Jeff Davis написал(а): After using a patch for a while it became obvious that PANICing during termination is not a good idea. Even when we wait for synchronous replication. It generates undesired coredumps. I think

Re: pg_amcheck contrib application

2021-03-11 Thread Andrey Borodin
> 11 марта 2021 г., в 13:12, Peter Eisentraut > написал(а): > > client-side tools whose sole task is to execute server-side functionality in > a slightly filtered way By the way, can we teach pg_amcheck to verify database without creating local PGDATA and using bare minimum of file system

RE: Avoid CommandCounterIncrement in RI trigger when INSERT INTO referencing table

2021-03-11 Thread houzj.f...@fujitsu.com
> > After some more on how to support parallel insert into fk relation. > > It seems we do not have a cheap way to implement this feature. > > > > In RI_FKey_check, Currently, postgres execute "select xx for key > > share" to check that foreign key exists in PK table. > > However "select for

Re: non-HOT update not looking at FSM for large tuple update

2021-03-11 Thread Matthias van de Meent
On Tue, 9 Mar 2021 at 18:39, John Naylor wrote: > > I wrote: > > > That seems like the proper fix, and I see you've started a thread for that. > > I don't think that change in behavior would be backpatchable, but patch > > here might have a chance at that. > > I remembered after the fact that

Re: Allow batched insert during cross-partition updates

2021-03-11 Thread gkokolatos
‐‐‐ Original Message ‐‐‐ On Thursday, March 11, 2021 9:42 AM, Amit Langote wrote: > Hi Georgios, > > On Wed, Mar 10, 2021 at 9:30 PM Georgios gkokola...@protonmail.com wrote: > > > I continued looking a bit at the patch, yet I am either failing to see fix > > or I am > > looking

Re: New IndexAM API controlling index vacuum strategies

2021-03-11 Thread Masahiko Sawada
On Wed, Mar 3, 2021 at 12:40 PM Peter Geoghegan wrote: > > On Tue, Mar 2, 2021 at 6:44 PM Masahiko Sawada wrote: > > A scale type parameter seems good to me but I wonder if how users can > > tune that parameter. We already have tuple-based parameters such as > >

RE: Parallel INSERT (INTO ... SELECT ...)

2021-03-11 Thread houzj.f...@fujitsu.com
> I guess to have the finer granularity we'd have to go with > enable_parallel_insert, > which then would mean possibly having to later add enable_parallel_update, > should parallel update have similar potential overhead in the parallel-safety > checks (which to me, looks like it could, and

Re: OpenSSL 3.0.0 compatibility

2021-03-11 Thread Michael Paquier
On Thu, Mar 11, 2021 at 11:41:22AM +0100, Daniel Gustafsson wrote: > .. and apply the padding changes as proposed in a patch upthread > like this (these work for all OpenSSL versions I've tested, and I'm > rather more puzzled as to why we got away with not having them in > the past): No

Re: WIP: Aggregation push-down

2021-03-11 Thread David Steele
On 3/11/21 5:10 AM, Antonin Houska wrote: David Steele wrote: On 7/3/20 6:07 AM, Laurenz Albe wrote: On Thu, 2020-07-02 at 14:39 +0200, Daniel Gustafsson wrote: This version now fails to apply to HEAD, with what looks like like a trivial error in the expected test output. Can you please

Self-join optimisation

2021-03-11 Thread Hywel Carver
Hi, I asked this question in the Postgres Slack, and was recommended to ask here instead. A few times, I've been in a situation where I want to join a table to itself on its primary key. That typically happens because I have some kind of summary view, which I then want to join to the original

Re: WIP: System Versioned Temporal Table

2021-03-11 Thread Surafel Temesgen
On Wed, Mar 10, 2021 at 9:02 AM Vik Fearing wrote: > > I have plenty of objection. I'm sorry that I am taking so long with my > review. I am still working on it and it is coming soon, I promise. > > okay take your time regards Surafel

Re: first add newly loaded plugin to the list then invoke _PG_init

2021-03-11 Thread Julien Rouhaud
On Thu, Mar 11, 2021 at 12:29:49PM +0100, mickiew...@syncad.com wrote: > Hi All, > I'm working on C plugin for Postgres (ver. 10). One of the thing which I > need is to automatically add some SQL functions from the plugin during its > initialization ( inside _PG_init method ). It means that during

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-03-11 Thread Fujii Masao
On 2021/03/11 21:29, Masahiro Ikeda wrote: On 2021-03-11 11:52, Fujii Masao wrote: On 2021/03/11 9:38, Masahiro Ikeda wrote: On 2021-03-10 17:08, Fujii Masao wrote: On 2021/03/10 14:11, Masahiro Ikeda wrote: On 2021-03-09 17:51, Fujii Masao wrote: On 2021/03/05 8:38, Masahiro Ikeda

first add newly loaded plugin to the list then invoke _PG_init

2021-03-11 Thread mickiewicz
Hi All, I'm working on C plugin for Postgres (ver. 10). One of the thing which I need is to automatically add some SQL functions from the plugin during its initialization ( inside _PG_init method ). It means that during loading libmyplugin.so _PG_init calls SPI_execute( "CREATE FUNCTION fun()

A micro-optimisation for ProcSendSignal()

2021-03-11 Thread Thomas Munro
Hi, ProcSendSignal(pid) searches the ProcArray for the given pid and then sets that backend's procLatch. It has only two users: UnpinBuffer() and ReleasePredicateLocks(). In both cases, we could just as easily have recorded the pgprocno instead, avoiding the locking and the searching. We'd

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-03-11 Thread Masahiro Ikeda
On 2021-03-11 11:52, Fujii Masao wrote: On 2021/03/11 9:38, Masahiro Ikeda wrote: On 2021-03-10 17:08, Fujii Masao wrote: On 2021/03/10 14:11, Masahiro Ikeda wrote: On 2021-03-09 17:51, Fujii Masao wrote: On 2021/03/05 8:38, Masahiro Ikeda wrote: On 2021-03-05 01:02, Fujii Masao wrote: On

Re: [PATCH] Hooks at XactCommand level

2021-03-11 Thread David Steele
Hi Julien, On 12/8/20 5:15 AM, Gilles Darold wrote: Based on a PoC reported in a previous thread [1] I'd like to propose new hooks around transaction commands. The objective of this patch is to allow PostgreSQL extension to act at start and end (including abort) of a SQL statement in a

Re: [PATCH] Hooks at XactCommand level

2021-03-11 Thread Julien Rouhaud
Hi David, On Thu, Mar 11, 2021 at 07:41:35AM -0500, David Steele wrote: > Hi Julien, > > On 12/8/20 5:15 AM, Gilles Darold wrote: > > > > Based on a PoC reported in a previous thread [1] I'd like to propose new > > hooks around transaction commands. The objective of this patch is to > > allow

Re: OpenSSL 3.0.0 compatibility

2021-03-11 Thread Daniel Gustafsson
> On 11 Mar 2021, at 11:03, Peter Eisentraut > wrote: > The ssl tests fail with a small error message difference that must have been > introduced recently, because I think this was never reported before: This was mentioned by Michael on 26/11, it was earlier in the 3.0.0 cycle reported as

Re: Self-join optimisation

2021-03-11 Thread Matthias van de Meent
On Thu, 11 Mar 2021 at 15:15, Hywel Carver wrote: > > Hi, > > I asked this question in the Postgres Slack, and was recommended to ask here > instead. > > A few times, I've been in a situation where I want to join a table to itself > on its primary key. That typically happens because I have some

Re: non-HOT update not looking at FSM for large tuple update

2021-03-11 Thread John Naylor
On Thu, Mar 11, 2021 at 9:46 AM Matthias van de Meent < boekewurm+postg...@gmail.com> wrote: > Regarding the 2% slack logic, could we change it to use increments of > line pointers instead? That makes it more clear what problem this > solution is trying to work around; that is to say line

Re: pg_amcheck contrib application

2021-03-11 Thread Mark Dilger
> On Mar 11, 2021, at 3:36 AM, Andrey Borodin wrote: > > > >> 11 марта 2021 г., в 13:12, Peter Eisentraut >> написал(а): >> >> client-side tools whose sole task is to execute server-side functionality in >> a slightly filtered way > > By the way, can we teach pg_amcheck to verify

Re: documentation fix for SET ROLE

2021-03-11 Thread David G. Johnston
On Thu, Mar 11, 2021 at 7:58 AM Laurenz Albe wrote: > I think we should *not* document that under "server configuration". > This is confusing and will lead people to think that a role is > a configuration parameter. But you cannot add > >role = myrole > > to "postgresql.conf". A role is

Re: Self-join optimisation

2021-03-11 Thread Hywel Carver
Great! It looks like it's been in commitfest status for a few years. Is there anything someone like me (outside the pgsql-hackers community) can do to help it get reviewed this time around? On Thu, Mar 11, 2021 at 2:32 PM Matthias van de Meent < boekewurm+postg...@gmail.com> wrote: > On Thu, 11

Re: Disallow cancellation of waiting for synchronous replication

2021-03-11 Thread Andrey Borodin
Thanks for looking into this! > 11 марта 2021 г., в 19:15, Fujii Masao > написал(а): > > > > On 2020/12/09 18:07, Andrey Borodin wrote: >>> 9 июня 2020 г., в 23:32, Jeff Davis написал(а): >>> >>> >> After using a patch for a while it became obvious that PANICing during >> termination is

Re: documentation fix for SET ROLE

2021-03-11 Thread Laurenz Albe
I have had a look at the patch, and while I agree that this should be documented, I am not happy with the patch as it is. I think we should *not* document that under "server configuration". This is confusing and will lead people to think that a role is a configuration parameter. But you cannot

Re: pg_amcheck contrib application

2021-03-11 Thread Robert Haas
On Thu, Mar 11, 2021 at 3:12 AM Peter Eisentraut wrote: > (I'm still not a fan of adding more client-side tools whose sole task is > to execute server-side functionality in a slightly filtered way, but it > seems people are really interested in this, so ...) > > I want to register, if we are

Re: MultiXact\SLRU buffers configuration

2021-03-11 Thread Gilles Darold
Le 15/02/2021 à 18:17, Andrey Borodin a écrit : 23 дек. 2020 г., в 21:31, Gilles Darold написал(а): Sorry for the response delay, we have run several others tests trying to figure out the performances gain per patch but unfortunately we have very heratic results. With the same parameters

Re: pg_amcheck contrib application

2021-03-11 Thread Mark Dilger
> On Mar 11, 2021, at 12:12 AM, Peter Eisentraut > wrote: > > (I'm still not a fan of adding more client-side tools whose sole task is to > execute server-side functionality in a slightly filtered way, but it seems > people are really interested in this, so ...) > > I want to register, if

Re: pg_amcheck contrib application

2021-03-11 Thread Robert Haas
On Wed, Mar 10, 2021 at 11:02 PM Mark Dilger wrote: > > The documentation says that -D "does exclude any database that was > > listed explicitly as dbname on the command line, nor does it exclude > > the database chosen in the absence of any dbname argument." The first > > part of this makes

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-11 Thread Tom Lane
The buildfarm has revealed that this patch doesn't work under CLOBBER_CACHE_ALWAYS: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=husky=2021-03-10%2021%3A09%3A32 I initially thought that that was a problem with c3ffe34863, but after reproducing it here I get this stack trace: #0

Re: Self-join optimisation

2021-03-11 Thread Justin Pryzby
On Thu, Mar 11, 2021 at 03:32:16PM +0100, Matthias van de Meent wrote: > On Thu, 11 Mar 2021 at 15:15, Hywel Carver wrote: > > I asked this question in the Postgres Slack, and was recommended to ask > > here instead. > > > > A few times, I've been in a situation where I want to join a table to

Re: Huge memory consumption on partitioned table with FKs

2021-03-11 Thread Tom Lane
I wrote: > Now, maybe it's a coincidence that husky failed on a > partitioned-foreign-key test right after this patch went in, but I bet > not. Since husky runs CLOBBER_CACHE_ALWAYS, it looks to me like we've > overlooked some cache-reset scenario or other. After reproducing it here, that *is* a

Re: Autovacuum on partitioned table (autoanalyze)

2021-03-11 Thread David Steele
On 12/14/20 8:46 PM, yuzuko wrote: On Thu, Dec 3, 2020 at 10:28 PM Alvaro Herrera wrote: Attach the new patch based on his patch. What do you think? Álvaro, Justin, Kyotaro, thoughts on this latest patch? Regards, -- -David da...@pgmasters.net

Re: pg_amcheck contrib application

2021-03-11 Thread Andrey Borodin
> 11 марта 2021 г., в 20:30, Mark Dilger > написал(а): > > > pg_amcheck does not need a local data directory to check a remote database > server, though it does need to connect to that server. No, I mean it it would be great if we did not need to materialise whole DB anywhere. Let's say I

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2021-03-11 Thread Alvaro Herrera
On 2021-Mar-05, Alvaro Herrera wrote: > I'll take the weekend to think about the issue with conn->last_query and > conn->queryclass that I mentioned yesterday; other than that detail my > feeling is that this is committable, so I'll be looking at getting this > pushed early next weeks, barring

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-03-11 Thread Alvaro Herrera
Rebase to current sources, to appease CF bot; no other changes. -- Álvaro Herrera39°49'30"S 73°17'W diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml index b1de6d0674..ea3ae56991 100644 --- a/doc/src/sgml/catalogs.sgml +++

Re: Huge memory consumption on partitioned table with FKs

2021-03-11 Thread Tom Lane
Tatsuro Yamada writes: > Thanks for fixing the problem! :-D Hmm, I'm not sure we're done with this patch: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=husky=2021-03-10%2021%3A09%3A32 The critical log extract is 2021-03-11 05:10:13.012 CET [21574:1082] pg_regress/foreign_key LOG:

Re: Logical Replication - improve error message while adding tables to the publication in check_publication_add_relation

2021-03-11 Thread Bharath Rupireddy
On Wed, Mar 10, 2021 at 5:48 PM Euler Taveira wrote: > > On Wed, Mar 10, 2021, at 2:14 AM, Bharath Rupireddy wrote: > > While providing thoughts on [1], I observed that the error messages > that are emitted while adding foreign, temporary and unlogged tables > can be improved a bit from the

Re: pgbench - add pseudo-random permutation function

2021-03-11 Thread Dean Rasheed
On Thu, 11 Mar 2021 at 00:58, Bruce Momjian wrote: > > Maybe Dean Rasheed can help because of his math background --- CC'ing him. > Reading the thread I can see how such a function might be useful to scatter non-uniformly random values. The implementation looks plausible too, though it adds

Re: New IndexAM API controlling index vacuum strategies

2021-03-11 Thread Robert Haas
On Tue, Mar 9, 2021 at 3:35 PM Peter Geoghegan wrote: > Speaking of line pointer bloat (and "irreversible" bloat), I came > across something relevant today. I believe that this recent patch from > Matthias van de Meent is a relatively easy way to improve the > situation: > >

Re: pg_amcheck contrib application

2021-03-11 Thread Mark Dilger
> On Mar 11, 2021, at 9:10 AM, Andrey Borodin wrote: > > > >> 11 марта 2021 г., в 20:30, Mark Dilger >> написал(а): >> >> >> pg_amcheck does not need a local data directory to check a remote database >> server, though it does need to connect to that server. > No, I mean it it would be

Re: A qsort template

2021-03-11 Thread Andres Freund
Hi, I wish we had the same for bsearch... :) On 2021-03-03 17:17:13 +1300, Thomas Munro wrote: > As for which cases are actually worth specialising, I've attached the > example that Andres mentioned earlier; it seems like a reasonable > candidate to go ahead and commit too, but I realised that

Re: pgbench - add pseudo-random permutation function

2021-03-11 Thread David Bowen
The algorithm for generating a random permutation with a uniform distribution across all permutations is easy: for (i=0; i wrote: > On Thu, 11 Mar 2021 at 00:58, Bruce Momjian wrote: > > > > Maybe Dean Rasheed can help because of his math background --- CC'ing > him. > > > > Reading the thread I

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-11 Thread Justin Pryzby
On Thu, Mar 11, 2021 at 01:01:42PM +, houzj.f...@fujitsu.com wrote: > > I guess to have the finer granularity we'd have to go with > > enable_parallel_insert, > > which then would mean possibly having to later add enable_parallel_update, > > should parallel update have similar potential

Re: Performance degradation of REFRESH MATERIALIZED VIEW

2021-03-11 Thread Andres Freund
Hi, On 2021-03-11 17:44:37 +0900, Masahiko Sawada wrote: > The execution time of REFRESH MATERIALIZED VIEW are: > > w/ HEAP_INSERT_FROZEN flag : 42 sec > w/o HEAP_INSERT_FROZEN flag : 33 sec > > After investigation, I found that such performance degradation happens > on only HEAD code. It seems

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2021-03-11 Thread Tom Lane
Alvaro Herrera writes: > [ v35-libpq-pipeline.patch ] I think the changes in pqParseInput3() are broken. You should have kept the else-structure as-is and inserted the check for "not really idle" inside the else-clause that reports an error. As it stands, after successfully processing an

Re: [HACKERS] Custom compression methods

2021-03-11 Thread Justin Pryzby
On Thu, Mar 11, 2021 at 10:07:30AM +0530, Dilip Kumar wrote: > On Thu, Mar 11, 2021 at 8:50 AM Justin Pryzby wrote: > > > > Looking at v23-0002-alter-table-set-compression, ATRewriteTable() was > > calling > > CompareCompressionMethodAndDecompress(). > > While changing the compression method

Re: pg_amcheck contrib application

2021-03-11 Thread Robert Haas
On Wed, Mar 10, 2021 at 11:02 PM Mark Dilger wrote: > [ new patches ] Seems like this is mostly ready to commit now, modulo exactly what to do about the maintenance DB stuff, and whether to move it to src/bin. Since neither of those affects 0001, I went ahead and committed that part. -- Robert

Re: documentation fix for SET ROLE

2021-03-11 Thread David G. Johnston
On Thursday, March 11, 2021, Bossart, Nathan wrote: > Thanks for reviewing. > > On 3/11/21, 6:59 AM, "Laurenz Albe" wrote: > > I have had a look at the patch, and while I agree that this should > > be documented, I am not happy with the patch as it is. > > > > I think we should *not* document

Re: pgbench - add pseudo-random permutation function

2021-03-11 Thread Dean Rasheed
On Thu, 11 Mar 2021 at 19:06, David Bowen wrote: > > The algorithm for generating a random permutation with a uniform distribution > across all permutations is easy: > for (i=0; iswap a[n-i] with a[rand(n-i+1)] > } > > where 0 <= rand(x) < x and a[i] is initially i (see Knuth, Section 3.4.2

Re: Is Recovery actually paused?

2021-03-11 Thread Robert Haas
On Mon, Mar 1, 2021 at 12:08 AM Dilip Kumar wrote: > > > > I'll wait for a day before marking this RfC in case anyone have > > > > further comments. > > > > > > Okay. > > > > Hearing nothing, done that. > > Thanks. Committed with minor cosmetic changes. -- Robert Haas EDB:

Re: New IndexAM API controlling index vacuum strategies

2021-03-11 Thread Matthias van de Meent
On Thu, 11 Mar 2021 at 17:31, Robert Haas wrote: > > On Tue, Mar 9, 2021 at 3:35 PM Peter Geoghegan wrote: > > Speaking of line pointer bloat (and "irreversible" bloat), I came > > across something relevant today. I believe that this recent patch from > > Matthias van de Meent is a relatively

Re: libpq debug log

2021-03-11 Thread Tom Lane
Alvaro Herrera writes: > On 2021-Mar-10, Tom Lane wrote: >> After studying this further, I think we should apply the attached >> patch to remove that responsibility from pqParseInput3's subroutines. >> This will allow a single trace call near the bottom of pqParseInput3 >> to handle all cases

Re: Self-join optimisation

2021-03-11 Thread Tomas Vondra
On 3/11/21 3:39 PM, Hywel Carver wrote: > Great! It looks like it's been in commitfest status for a few years. Is > there anything someone like me (outside the pgsql-hackers community) can > do to help it get reviewed this time around? > Well, you could do a review, or at least test it with the

Re: shared-memory based stats collector

2021-03-11 Thread Kyotaro Horiguchi
At Fri, 12 Mar 2021 10:03:31 +0900, Fujii Masao wrote in > > I moved archiver from the current location to next to "walsenders" > > that is to be terminated along with archiver. > > The attached the only 0003 of the new version based on the last one > > from Fujii-san. > > Thanks for updating

Re: Procedures versus the "fastpath" API

2021-03-11 Thread Noah Misch
On Wed, Mar 10, 2021 at 10:03:24AM +0100, Laurenz Albe wrote: > On Tue, 2021-03-09 at 14:15 -0500, Tom Lane wrote: > > As for procedures, I'm of the opinion that we should just reject those > > too, but some other security team members were not happy with that > > idea. Conceivably we could

Re: pg_amcheck contrib application

2021-03-11 Thread Robert Haas
On Thu, Mar 11, 2021 at 11:09 AM Robert Haas wrote: > An alternate possibility would be to say that there should only ever > be EITHER a bare command-line argument OR options that require > querying for a list of databases OR neither BUT NOT both. Then it's > simple: > > 0. If you have both

Re: OpenSSL 3.0.0 compatibility

2021-03-11 Thread Peter Eisentraut
On 11.03.21 11:41, Daniel Gustafsson wrote: Then there are a few where we get padding back where we really should have ended up with the "Cipher cannot be initialized" error since DES is in the legacy provider: select decrypt_iv(decode('50735067b073bb93', 'hex'), '0123456', 'abcd', 'des'); -

Re: automatic analyze: readahead - add "IO read time" log message

2021-03-11 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: > On 3/12/21 1:11 AM, Stephen Frost wrote: > > * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: > >> On 3/8/21 8:42 PM, Stephen Frost wrote: > >>> * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: > On 2/10/21 11:10

Re: shared-memory based stats collector

2021-03-11 Thread Kyotaro Horiguchi
At Fri, 12 Mar 2021 10:07:10 +0900 (JST), Kyotaro Horiguchi wrote in > At Fri, 12 Mar 2021 09:23:12 +0900 (JST), Kyotaro Horiguchi > wrote in > > The attached the only 0003 of the new version based on the last one > > from Fujii-san. > > Please wait a moment. Something might be wrong. It

Re: Hybrid Hash/Nested Loop joins and caching results from subplans

2021-03-11 Thread Tom Lane
David Rowley writes: > On Tue, 23 Feb 2021 at 18:43, Tom Lane wrote: >> I doubt it's that bad. We could cache such info in RestrictInfo >> for quals, or PathTarget for tlists, without much new notational >> overhead. That doesn't cover everything the planner deals with >> of course, but it

Re: cursor sensitivity misunderstanding

2021-03-11 Thread Peter Eisentraut
On 09.03.21 00:22, David G. Johnston wrote: I came up with the attached patch to sort this out a bit.  It does not change any cursor behavior.  But the documentation now uses the terms more correctly and explains the differences between SQL and the PostgreSQL implementation

Re: [PATCH] Covering SPGiST index

2021-03-11 Thread Tom Lane
David Steele writes: > Tom, do the changes as enumerated in [1] look like they are going in the > right direction? I spent a little time looking at this, and realized something that may or may not be a serious problem. This form of the patch supposes that it can use the usual tuple form/deform

Re: Hybrid Hash/Nested Loop joins and caching results from subplans

2021-03-11 Thread David Rowley
Thanks for these suggestions. On Mon, 22 Feb 2021 at 14:21, Justin Pryzby wrote: > > On Tue, Feb 16, 2021 at 11:15:51PM +1300, David Rowley wrote: > > To summarise here, the planner performance gets a fair bit worse with > > the patched code. With master, summing the average planning time over >

Re: compression libraries and CF bot

2021-03-11 Thread Thomas Munro
On Sun, Feb 21, 2021 at 4:30 AM Justin Pryzby wrote: > Dilip's TOAST patch is passing on linux and bsd --with-lz4, so I think it's > desirable to install on mac now. Justin figured this out, so now this patch is using lz4 and passing on Linux, FreeBSD and macOS. > libzstd would be desirable for

Re: Is Recovery actually paused?

2021-03-11 Thread Dilip Kumar
On Fri, 12 Mar 2021 at 2:04 AM, Robert Haas wrote: > On Mon, Mar 1, 2021 at 12:08 AM Dilip Kumar wrote: > > > > > I'll wait for a day before marking this RfC in case anyone have > > > > > further comments. > > > > > > > > Okay. > > > > > > Hearing nothing, done that. > > > > Thanks. > >

Re: New IndexAM API controlling index vacuum strategies

2021-03-11 Thread Peter Geoghegan
On Thu, Mar 11, 2021 at 8:31 AM Robert Haas wrote: > I agree, but all you need is one long-lived tuple toward the end of > the array and you're stuck never being able to truncate it. It seems > like a worthwhile improvement, but whether it actually helps will be > workload-dependant. When it

Re: [Patch] Optimize dropping of relation buffers using dlist

2021-03-11 Thread Thomas Munro
While rebasing CF #2933 (which drops the _cached stuff and makes this optimisation always available, woo), I happened to notice that we're summing the size of many relations and forks into a variable nBlocksToInvalidate of type BlockNumber. That could overflow.

Re: automatic analyze: readahead - add "IO read time" log message

2021-03-11 Thread Tomas Vondra
On 3/12/21 1:11 AM, Stephen Frost wrote: > Greetings, > > * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: >> On 3/8/21 8:42 PM, Stephen Frost wrote: >>> * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: On 2/10/21 11:10 PM, Stephen Frost wrote: > * Heikki Linnakangas

Re: [PATCH] Covering SPGiST index

2021-03-11 Thread Tom Lane
I wrote: > I spent a little time looking at this, and realized something that may > or may not be a serious problem. This form of the patch supposes that > it can use the usual tuple form/deform logic for all columns of a leaf > tuple including the key column. However, that does not square with

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-11 Thread Greg Nancarrow
On Fri, Mar 12, 2021 at 5:00 AM Tom Lane wrote: > > The buildfarm has revealed that this patch doesn't work under > CLOBBER_CACHE_ALWAYS: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=husky=2021-03-10%2021%3A09%3A32 > > I initially thought that that was a problem with c3ffe34863,

Re: automatic analyze: readahead - add "IO read time" log message

2021-03-11 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: > On 3/8/21 8:42 PM, Stephen Frost wrote: > > * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: > >> On 2/10/21 11:10 PM, Stephen Frost wrote: > >>> * Heikki Linnakangas (hlinn...@iki.fi) wrote: > On 05/02/2021 23:22,

Re: OpenSSL 3.0.0 compatibility

2021-03-11 Thread Daniel Gustafsson
> On 12 Mar 2021, at 00:04, Peter Eisentraut > wrote: > > On 11.03.21 11:41, Daniel Gustafsson wrote: >> Then there are a few where we get padding back where we really should have >> ended up with the "Cipher cannot be initialized" error since DES is in the >> legacy provider: >> select

Re: shared-memory based stats collector

2021-03-11 Thread Kyotaro Horiguchi
At Thu, 11 Mar 2021 15:33:52 +0900, Fujii Masao wrote in > > > On 2021/03/11 13:42, Kyotaro Horiguchi wrote: > > At Wed, 10 Mar 2021 19:21:00 -0800, Andres Freund > > wrote in > >> Hi, > >> > >> Two minor nits: > > Thanks for the comments! > > > >> On 2021-03-10 21:47:51 +0900, Fujii Masao

Re: Hybrid Hash/Nested Loop joins and caching results from subplans

2021-03-11 Thread David Rowley
On Tue, 23 Feb 2021 at 14:22, Andy Fan wrote: > > On Mon, Feb 22, 2021 at 9:21 AM Justin Pryzby wrote: >> - Maybe this should be integrated into nestloop rather than being a separate >>plan node. That means that it could be dynamically enabled during >>execution, maybe after a few

Re: shared-memory based stats collector

2021-03-11 Thread Fujii Masao
On 2021/03/12 9:23, Kyotaro Horiguchi wrote: At Thu, 11 Mar 2021 15:33:52 +0900, Fujii Masao wrote in On 2021/03/11 13:42, Kyotaro Horiguchi wrote: At Wed, 10 Mar 2021 19:21:00 -0800, Andres Freund wrote in Hi, Two minor nits: Thanks for the comments! On 2021-03-10 21:47:51 +0900,

Re: shared-memory based stats collector

2021-03-11 Thread Kyotaro Horiguchi
At Fri, 12 Mar 2021 09:23:12 +0900 (JST), Kyotaro Horiguchi wrote in > The attached the only 0003 of the new version based on the last one > from Fujii-san. Please wait a moment. Something might be wrong. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: [HACKERS] Custom compression methods

2021-03-11 Thread Dilip Kumar
On Thu, Mar 11, 2021 at 11:55 PM Justin Pryzby wrote: > > On Wed, Mar 10, 2021 at 08:28:58PM -0600, Justin Pryzby wrote: > > This includes a patch to use pkgconfig, in an attempt to build on mac, which > > currently fails like: > > > >

Re: Huge memory consumption on partitioned table with FKs

2021-03-11 Thread Tatsuro Yamada
On 2021/03/12 2:44, Tom Lane wrote: I wrote: Now, maybe it's a coincidence that husky failed on a partitioned-foreign-key test right after this patch went in, but I bet not. Since husky runs CLOBBER_CACHE_ALWAYS, it looks to me like we've overlooked some cache-reset scenario or other. After

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-03-11 Thread Fujii Masao
On 2021/03/11 21:29, Masahiro Ikeda wrote: On 2021-03-11 11:52, Fujii Masao wrote: On 2021/03/11 9:38, Masahiro Ikeda wrote: On 2021-03-10 17:08, Fujii Masao wrote: On 2021/03/10 14:11, Masahiro Ikeda wrote: On 2021-03-09 17:51, Fujii Masao wrote: On 2021/03/05 8:38, Masahiro Ikeda

Re: [Patch] Optimize dropping of relation buffers using dlist

2021-03-11 Thread Thomas Munro
On Fri, Mar 12, 2021 at 5:20 PM Amit Kapila wrote: > uint64 +1

Re: shared-memory based stats collector

2021-03-11 Thread Kyotaro Horiguchi
At Fri, 12 Mar 2021 10:38:00 +0900 (JST), Kyotaro Horiguchi wrote in > At Fri, 12 Mar 2021 10:03:31 +0900, Fujii Masao > wrote in > > > I moved archiver from the current location to next to "walsenders" > > > that is to be terminated along with archiver. > > > The attached the only 0003 of

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-11 Thread vignesh C
On Thu, Mar 11, 2021 at 7:20 AM Peter Smith wrote: > > On Thu, Mar 11, 2021 at 12:46 PM Peter Smith wrote: > > > > Please find attached the latest patch set v57* > > > > Differences from v56* are: > > > > * Rebased to HEAD @ today > > > > * Addresses the following feedback issues: > > > > (24)

Re: [HACKERS] Custom compression methods

2021-03-11 Thread Dilip Kumar
On Fri, Mar 12, 2021 at 10:45 AM Justin Pryzby wrote: > > On Thu, Mar 11, 2021 at 12:25:26PM -0600, Justin Pryzby wrote: > > On Wed, Mar 10, 2021 at 08:28:58PM -0600, Justin Pryzby wrote: > > > This includes a patch to use pkgconfig, in an attempt to build on mac, > > > which > > > currently

Re: [PATCH] Hooks at XactCommand level

2021-03-11 Thread Julien Rouhaud
Hi, On Tue, Dec 08, 2020 at 11:15:12AM +0100, Gilles Darold wrote: > > Based on a PoC reported in a previous thread [1] I'd like to propose new > hooks around transaction commands. The objective of this patch is to > allow PostgreSQL extension to act at start and end (including abort) of > a SQL

Re: shared-memory based stats collector

2021-03-11 Thread Fujii Masao
On 2021/03/12 13:49, Kyotaro Horiguchi wrote: At Fri, 12 Mar 2021 10:38:00 +0900 (JST), Kyotaro Horiguchi wrote in At Fri, 12 Mar 2021 10:03:31 +0900, Fujii Masao wrote in I moved archiver from the current location to next to "walsenders" that is to be terminated along with archiver.

RE: [Patch] Optimize dropping of relation buffers using dlist

2021-03-11 Thread tsunakawa.ta...@fujitsu.com
From: Kyotaro Horiguchi > About the patch, it would be better to change the type of > BUF_DROP_FULL_SCAN_THRESHOLD to uint64, even though the current > value > doesn't harm. OK, attached, to be prepared for the distant future when NBuffers becomes 64-bit. Regards Takayuki Tsunakawa

Re: Hybrid Hash/Nested Loop joins and caching results from subplans

2021-03-11 Thread David Rowley
On Fri, 12 Mar 2021 at 14:59, Tom Lane wrote: > > David Rowley writes: > > The 0001 patch adds a has_volatile bool field to RestrictInfo and sets > > it when building the RestrictInfo. > > I'm -1 on doing it exactly that way, because you're expending > the cost of those lookups without certainty

Re: [HACKERS] Custom compression methods

2021-03-11 Thread Justin Pryzby
On Mon, Mar 01, 2021 at 08:53:09PM +0530, Dilip Kumar wrote: > On Mon, Mar 1, 2021 at 5:36 PM Dilip Kumar wrote: > > On Mon, Mar 1, 2021 at 11:06 AM Justin Pryzby wrote: > > > Thanks. It seems like that explains it. > > > I think if that's a problem with recent versions, then you'll have to > >

Re: [HACKERS] Custom compression methods

2021-03-11 Thread Dilip Kumar
On Fri, Mar 12, 2021 at 8:54 AM Justin Pryzby wrote: > > +#elif LZ4_VERSION_NUMBER < 10803 > + return lz4_cmdecompress(value); > +#else > > It occurred to me that this should actually compare the runtime version with > LZ4_versionNumber(). That way, a library upgrade can enable the slice >

Re: [HACKERS] Custom compression methods

2021-03-11 Thread Dilip Kumar
On Fri, Mar 12, 2021 at 9:03 AM Justin Pryzby wrote: > > On Fri, Mar 12, 2021 at 08:38:41AM +0530, Dilip Kumar wrote: > > On Thu, Mar 11, 2021 at 11:55 PM Justin Pryzby wrote: > > > On Wed, Mar 10, 2021 at 08:28:58PM -0600, Justin Pryzby wrote: > > > > This includes a patch to use pkgconfig, in

Re: Performance degradation of REFRESH MATERIALIZED VIEW

2021-03-11 Thread Masahiko Sawada
On Fri, Mar 12, 2021 at 3:13 AM Andres Freund wrote: > > Hi, > > On 2021-03-11 17:44:37 +0900, Masahiko Sawada wrote: > > The execution time of REFRESH MATERIALIZED VIEW are: > > > > w/ HEAP_INSERT_FROZEN flag : 42 sec > > w/o HEAP_INSERT_FROZEN flag : 33 sec > > > > After investigation, I found

Re: [Patch] Optimize dropping of relation buffers using dlist

2021-03-11 Thread Kyotaro Horiguchi
At Fri, 12 Mar 2021 05:26:02 +, "tsunakawa.ta...@fujitsu.com" wrote in > From: Thomas Munro > > > uint64 > > > > +1 > > Thank you, the patch is attached (we tend to forget how large our world is... > 64-bit) We're sorry to cause you trouble. BUF_DROP_FULL_SCAN_THRESHOLD cannot be

Re: Allow batched insert during cross-partition updates

2021-03-11 Thread Amit Langote
On Thu, Mar 11, 2021 at 8:36 PM wrote: > On Thursday, March 11, 2021 9:42 AM, Amit Langote > wrote: > > On Wed, Mar 10, 2021 at 9:30 PM Georgios gkokola...@protonmail.com wrote: > > > > > I continued looking a bit at the patch, yet I am either failing to see > > > fix or I am > > > looking at

Re: shared-memory based stats collector

2021-03-11 Thread Andres Freund
Hi, On 2021-03-10 20:26:56 -0800, Andres Freund wrote: > > +static void > > +attach_shared_stats(void) > > +{ > > + MemoryContext oldcontext; > > > + /* > > +* The first attacher backend may still reading the stats file, or the > > +* last detacher may writing it. Wait for the work

Re: [HACKERS] Custom compression methods

2021-03-11 Thread Dilip Kumar
On Fri, Mar 12, 2021 at 2:54 AM Justin Pryzby wrote: > > On Thu, Mar 11, 2021 at 10:07:30AM +0530, Dilip Kumar wrote: > > On Thu, Mar 11, 2021 at 8:50 AM Justin Pryzby wrote: > > > > > > Looking at v23-0002-alter-table-set-compression, ATRewriteTable() was > > > calling > > >

Re: shared-memory based stats collector

2021-03-11 Thread Kyotaro Horiguchi
At Thu, 11 Mar 2021 19:22:57 -0800, Andres Freund wrote in > Hi, > > On 2021-03-10 20:26:56 -0800, Andres Freund wrote: > > > +static void > > > +attach_shared_stats(void) > > > +{ > > > + MemoryContext oldcontext; > > > > > + /* > > > + * The first attacher backend may still reading the

Re: [HACKERS] Custom compression methods

2021-03-11 Thread Dilip Kumar
On Fri, Mar 12, 2021 at 11:55 AM Justin Pryzby wrote: > > It worked everywhere (but everytime someone mail this thread, it queues a > rebuild). > http://cfbot.cputube.org/dilip-kumar.html Okay > HAVE_LIBLZ4 was being set by AC_CHECK_LIB(), which is no longer used in favour > of pkgconfig. >

  1   2   >