Re: Confusing behavior of psql's \e

2021-03-10 Thread Juan José Santamaría Flecha
On Wed, Mar 10, 2021 at 6:14 AM Tom Lane wrote: > > PS: I seem to recall that some Microsoft filesystems have 2-second > resolution on file mod times, so maybe it needs to be "time(NULL) - 2"? > You are thinking about FAT:

Re: [PATCH] Provide more information to filter_prepare

2021-03-10 Thread Amit Kapila
On Tue, Mar 9, 2021 at 2:14 PM Markus Wanner wrote: > > Hi, > > currently, only the gid is passed on to the filter_prepare callback. > While we probably should not pass a full ReorderBufferTXN (as we do for > most other output plugin callbacks), a bit more information would be > nice, I think. >

RE: Avoid CommandCounterIncrement in RI trigger when INSERT INTO referencing table

2021-03-10 Thread tsunakawa.ta...@fujitsu.com
From: Hou, Zhijie/侯 志杰 > After some more on how to support parallel insert into fk relation. > It seems we do not have a cheap way to implement this feature. > > In RI_FKey_check, Currently, postgres execute "select xx for key share" to > check that foreign key exists in PK table. > However

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-10 Thread Amit Kapila
On Mon, Mar 8, 2021 at 7:19 PM Amit Langote wrote: > > Hi Amit > > On Mon, Mar 8, 2021 at 10:18 PM Amit Kapila wrote: > > On Mon, Mar 8, 2021 at 3:54 PM Greg Nancarrow wrote: > > > I've attached an updated set of patches with the suggested locking > > > changes. > > (Thanks Greg.) > > > Amit

Re: Confusing behavior of psql's \e

2021-03-10 Thread Laurenz Albe
On Wed, 2021-03-10 at 00:13 -0500, Tom Lane wrote: > I agree > that trying to get rid of the race condition inherent in the existing > file mtime test would be a good idea. However, I've got some > portability-related gripes about how you are doing the latter: > > 1. There is no principled

Re: [PATCH] Provide more information to filter_prepare

2021-03-10 Thread Markus Wanner
On 10.03.21 11:18, Amit Kapila wrote: On Tue, Mar 9, 2021 at 2:14 PM Markus Wanner wrote: currently, only the gid is passed on to the filter_prepare callback. While we probably should not pass a full ReorderBufferTXN (as we do for most other output plugin callbacks), a bit more information

Re: Procedures versus the "fastpath" API

2021-03-10 Thread Laurenz Albe
On Tue, 2021-03-09 at 14:15 -0500, Tom Lane wrote: > The security team received a report from Theodor-Arsenij > Larionov-Trichkin of PostgresPro that it's possible to crash the > backend with an assertion or null-pointer dereference by trying to > call a window function via the "fast path function

Do we support upgrade of logical replication?

2021-03-10 Thread vignesh C
Hi, I was reviewing logical decoding of two-phase transactions feature, while reviewing the feature I was checking if there is any impact on publisher/subscriber upgrade. I checked the existing pg_upgrade behaviour with logical replication. I made a logical replication data instance with

Re: Logical Replication - improve error message while adding tables to the publication in check_publication_add_relation

2021-03-10 Thread Bharath Rupireddy
On Wed, Mar 10, 2021 at 1:27 PM Jeevan Ladhe wrote: > > On Wed, Mar 10, 2021 at 10:44 AM Bharath Rupireddy > wrote: >> >> Hi, >> >> While providing thoughts on [1], I observed that the error messages >> that are emitted while adding foreign, temporary and unlogged tables >> can be improved a

Re: Release SPI plans for referential integrity with DISCARD ALL

2021-03-10 Thread yuzuko
Hello, I thought about this suggestion again. Amit's patch suggested in the thread [1] can eliminate SPI plans from INSERT/UPDATE triggers, so our memory pressure issue would be solved. But as far as I can see that thread, Amit's patch doesn't cover DELETE case. It is not a common case, but

Re: Improvements and additions to COPY progress reporting

2021-03-10 Thread Matthias van de Meent
On Tue, 9 Mar 2021 at 06:34, Michael Paquier wrote: > > On Mon, Mar 08, 2021 at 05:33:40PM +0100, Matthias van de Meent wrote: > > Seems reasonable. PFA updated patches. I've renamed the previous 0003 > > to 0002 to keep git-format-patch easy. > > Thanks for updating the patch. 0001 has been

Re: OpenSSL 3.0.0 compatibility

2021-03-10 Thread Daniel Gustafsson
> On 3 Mar 2021, at 14:55, Peter Eisentraut > wrote: > > This thread is still in the commit fest, but I don't see any actual proposed > patch still pending. Most of the activity has moved into other threads. The doc changes in the patch proposed on 29/9 still stands, although I see that it

Re: Add some tests for pg_stat_statements compatibility verification under contrib

2021-03-10 Thread Erica Zhang
Hi Julien, Thanks a lot for the quick review. Please see my answer below in blue. Attached is the new patch. -- Original -- From: "Julien

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-03-10 Thread Fujii Masao
On 2021/03/10 14:11, Masahiro Ikeda wrote: On 2021-03-09 17:51, Fujii Masao wrote: On 2021/03/05 8:38, Masahiro Ikeda wrote: On 2021-03-05 01:02, Fujii Masao wrote: On 2021/03/04 16:14, Masahiro Ikeda wrote: On 2021-03-03 20:27, Masahiro Ikeda wrote: On 2021-03-03 16:30, Fujii Masao

Re: pgsql: Enable parallel SELECT for "INSERT INTO ... SELECT ...".

2021-03-10 Thread Amit Kapila
On Wed, Mar 10, 2021 at 9:07 AM Tom Lane wrote: > > Amit Kapila writes: > > Enable parallel SELECT for "INSERT INTO ... SELECT ...". > > skink (valgrind) is unhappy: > > creating configuration files ... ok > running bootstrap script ... ok > performing post-bootstrap initialization ...

Re: cryptohash: missing locking functions for OpenSSL <= 1.0.2?

2021-03-10 Thread Michael Paquier
On Mon, Mar 08, 2021 at 06:06:32PM +, Jacob Champion wrote: > I had to convince myself that this logic is correct -- we set > destroy_needed even if crypto is not enabled, but then check later to > make sure that crypto_loaded is true before doing anything. What would > you think about moving

RE: Parallel Inserts in CREATE TABLE AS

2021-03-10 Thread houzj.f...@fujitsu.com
> Seems like v22 patch was failing in cfbot for one of the unstable test cases. > Attaching v23 patch set with modification in 0003 and 0004 patches. No > changes to 0001 and 0002 patches. Hopefully cfbot will be happy with v23. > > Please consider v23 for further review. Hi, I was looking into

Re: Change JOIN tutorial to focus more on explicit joins

2021-03-10 Thread David Steele
On 12/1/20 3:38 AM, Jürgen Purtz wrote: On 30.11.20 21:25, David G. Johnston wrote: Sorry, I managed to overlook the most recent patch. I admitted my use of parentheses was incorrect and I don't see anyone else defending them.  Please remove them. Minor typos: "the database compare" ->

Re: libpq debug log

2021-03-10 Thread 'alvhe...@alvh.no-ip.org'
On 2021-Mar-10, iwata@fujitsu.com wrote: > Hi all, > > Following all reviewer's advice, I have created a new patch. > > In this patch, I add only two tracing entry points; I call > pqTraceOutputMsg(PGconn conn, int msgCursor, PGCommSource commsource) in > pqParseInput3 () and pqPutMsgEnd

Re: get rid of tags in the docs?

2021-03-10 Thread John Naylor
On Thu, Feb 4, 2021 at 11:31 AM Tom Lane wrote: > > John Naylor writes: > > While looking at the proposed removal of the v2 protocol, I noticed that we > > italicize some, but not all, instances of 'per se', 'pro forma', and 'ad > > hoc'. I'd say these are widespread enough in formal registers

Re: [PATCH] SET search_path += octopus

2021-03-10 Thread David Steele
On 12/1/20 9:49 AM, Abhijit Menon-Sen wrote: Therefore, for lack of any plausible way to proceed, I do not plan to work on this feature any more. It sounds like this CF entry should be closed. I'll do that on March 12 unless somebody beats me to it. Regards, -- -David da...@pgmasters.net

Re: libpq debug log

2021-03-10 Thread 'alvhe...@alvh.no-ip.org'
I also found that DataRow messages are not being printed. This seems to fix that in the normal case and singlerow, at least in pipeline mode. Didn't verify the non-pipeline mode. -- Álvaro Herrera39°49'30"S 73°17'W "Nunca se desea ardientemente lo que solo se desea

Re: Logical Replication - improve error message while adding tables to the publication in check_publication_add_relation

2021-03-10 Thread Euler Taveira
On Wed, Mar 10, 2021, at 2:14 AM, Bharath Rupireddy wrote: > While providing thoughts on [1], I observed that the error messages > that are emitted while adding foreign, temporary and unlogged tables > can be improved a bit from the existing [2] to [3]. For instance, the > existing message when

Re: error_severity of brin work item

2021-03-10 Thread David Steele
On 12/1/20 5:25 PM, Justin Pryzby wrote: On Tue, Dec 01, 2020 at 03:57:24PM -0300, Alvaro Herrera wrote: Another idea is if perform_work_item() were responsible for discarding relations which disappear. Currently it does this, which is racy since it holds no lock. That has the property that

Re: Allow batched insert during cross-partition updates

2021-03-10 Thread Georgios
Hi, ‐‐‐ Original Message ‐‐‐ On Thursday, February 18, 2021 10:54 AM, Amit Langote wrote: > On Thu, Feb 18, 2021 at 6:52 PM Amit Langote amitlangot...@gmail.com wrote: > > > Based on the discussion at: > >

Re: Allow batched insert during cross-partition updates

2021-03-10 Thread Georgios
‐‐‐ Original Message ‐‐‐ On Wednesday, March 10, 2021 1:23 PM, Georgios wrote: > > > Hi, > > ‐‐‐ Original Message ‐‐‐ > On Thursday, February 18, 2021 10:54 AM, Amit Langote amitlangot...@gmail.com > wrote: > > > On Thu, Feb 18, 2021 at 6:52 PM Amit Langote

Re: Lowering the ever-growing heap->pd_lower

2021-03-10 Thread Matthias van de Meent
On Tue, 9 Mar 2021 at 22:35, Tom Lane wrote: > > Matthias van de Meent writes: > > The only two existing mechanisms that I could find (in the access/heap > > directory) that possibly could fail on shrunken line pointer arrays; > > being xlog recovery (I do not have enough knowledge on recovery

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-10 Thread Dilip Kumar
On Wed, Mar 10, 2021 at 2:48 PM Amit Kapila wrote: > > On Mon, Mar 8, 2021 at 7:19 PM Amit Langote wrote: > > > > Hi Amit > > > > On Mon, Mar 8, 2021 at 10:18 PM Amit Kapila wrote: > > > On Mon, Mar 8, 2021 at 3:54 PM Greg Nancarrow wrote: > > > > I've attached an updated set of patches with

Re: shared-memory based stats collector

2021-03-10 Thread Fujii Masao
On 2021/03/10 17:51, Kyotaro Horiguchi wrote: At Wed, 10 Mar 2021 15:20:43 +0900, Fujii Masao wrote in On 2021/03/10 12:10, Kyotaro Horiguchi wrote: Agreed. The code moved to the original place and added the crash handling code. And I added a phrase to the comment. +* Was

Re: proposal: unescape_text function

2021-03-10 Thread David Steele
On 12/2/20 1:30 PM, Pavel Stehule wrote: st 2. 12. 2020 v 11:37 odesílatel Pavel Stehule st 2. 12. 2020 v 9:23 odesílatel Peter Eisentraut Heh.  The fact that there is a table of two dozen possible representations kind of proves my point that we should be deliberate

Re: WIP: document the hook system

2021-03-10 Thread David Steele
On 3/9/21 12:20 PM, Bruce Momjian wrote: On Sat, Mar 6, 2021 at 08:32:43PM -0500, Tom Lane wrote: I think that the best you should hope for here is that people are willing to add a short, not-too-detailed para to a markup-free plain-text README file that lists all the hooks. As soon as it

Re: Boundary value check in lazy_tid_reaped()

2021-03-10 Thread Peter Eisentraut
On 10.03.21 02:29, Masahiko Sawada wrote: There is no noticeable effect of inlining lazy_tid_reaped(). So it would be better to not do that. Attached the patch that doesn't inline lazy_tid_reaped(). committed

Re: libpq debug log

2021-03-10 Thread 'alvhe...@alvh.no-ip.org'
On 2021-Mar-10, 'alvhe...@alvh.no-ip.org' wrote: > I also found that DataRow messages are not being printed. This seems to > fix that in the normal case and singlerow, at least in pipeline mode. > Didn't verify the non-pipeline mode. Hm, and RowDescription ('T') messages are also not being

Re: Occasional tablespace.sql failures in check-world -jnn

2021-03-10 Thread Andres Freund
Hi, On 2021-03-10 15:40:38 +0900, Michael Paquier wrote: > On Mon, Mar 08, 2021 at 11:53:57AM +0100, Peter Eisentraut wrote: > > On 09.12.20 08:55, Michael Paquier wrote: > >> ... Because we may still introduce this problem again if some new > >> stuff uses src/test/pg_regress in a way similar

Re: libpq debug log

2021-03-10 Thread Tom Lane
"'alvhe...@alvh.no-ip.org'" writes: > After staring at it a couple of times, I think that the places in > pqParseInput3() where there's a comment "... moves inStart itself" and > then "continue;" should have a call to pqTraceOutputMsg(), since AFAIU > those are the places where the message in

Re: libpq debug log

2021-03-10 Thread 'alvhe...@alvh.no-ip.org'
On 2021-Mar-10, Tom Lane wrote: > "'alvhe...@alvh.no-ip.org'" writes: > > After staring at it a couple of times, I think that the places in > > pqParseInput3() where there's a comment "... moves inStart itself" and > > then "continue;" should have a call to pqTraceOutputMsg(), since AFAIU > >

Re: pg_amcheck contrib application

2021-03-10 Thread Robert Haas
On Wed, Mar 10, 2021 at 11:10 AM Mark Dilger wrote: > Once again, I think you are right and have removed the objectionable > behavior, but > > The --startblock and --endblock options make the most sense when the user is > only checking one table, like > > pg_amcheck --startblock=17

Re: [HACKERS] Custom compression methods

2021-03-10 Thread Robert Haas
On Wed, Mar 10, 2021 at 6:52 AM Dilip Kumar wrote: > The pending comment is providing a way to rewrite a table and > re-compress the data with the current compression method. I spent some time poking at this yesterday and ran couldn't figure out what was going on here. There are two places where

Re: libpq debug log

2021-03-10 Thread 'alvhe...@alvh.no-ip.org'
On 2021-Mar-10, Tom Lane wrote: > Or we could rethink the logic. It's not quite clear to me, after > all this time, why getRowDescriptions() et al are allowed to > move inStart themselves rather than letting the main loop in > pqParseInput3 do it. It might well be an artifact of having not >

Re: WIP: System Versioned Temporal Table

2021-03-10 Thread Surafel Temesgen
hi Ibrar, thank you for rebasing On Mon, Mar 8, 2021 at 9:34 AM Ibrar Ahmed wrote: > >> Since the get_row_start_time_col_name() and get_row_end_time_col_name() >> are similar, IMO we can pass a flag to get StartTime/EndTime column name, >> thought? >> >> For me your option is better. i will

Re: Consider parallel for lateral subqueries with limit

2021-03-10 Thread David Steele
On 12/7/20 6:45 PM, James Coleman wrote: On Sun, Dec 6, 2020 at 7:34 PM Brian Davis wrote: Played around with this a bit, here's a non-correlated subquery that gets us to that if statement While I haven't actually tracked down to guarantee this is handled elsewhere, a thought experiment --

Re: Huge memory consumption on partitioned table with FKs

2021-03-10 Thread Tom Lane
Amit Langote writes: > On Wed, Mar 10, 2021 at 8:37 AM Tom Lane wrote: >> Hmm. So, the key point is that the values coming from the partitioned >> child table are injected into the test query as parameters, not as >> column references, thus it doesn't matter *to the test query* what >> numbers

Re: libpq debug log

2021-03-10 Thread Tom Lane
I wrote: > Or we could rethink the logic. It's not quite clear to me, after > all this time, why getRowDescriptions() et al are allowed to > move inStart themselves rather than letting the main loop in > pqParseInput3 do it. It might well be an artifact of having not > rewritten the v2 logic too

Re: libpq debug log

2021-03-10 Thread Tom Lane
"'alvhe...@alvh.no-ip.org'" writes: > After staring at it a couple of times, I think that the places in > pqParseInput3() where there's a comment "... moves inStart itself" and > then "continue;" should have a call to pqTraceOutputMsg(), since AFAIU > those are the places where the message in

Re: get rid of tags in the docs?

2021-03-10 Thread Tom Lane
John Naylor writes: > On Thu, Feb 4, 2021 at 11:31 AM Tom Lane wrote: >> +1, nobody italicizes those in normal usage. > Now that protocol v2 is gone, here's a patch to remove those tags. Pushed. regards, tom lane

Re: Evaluate expression at planning time for two more cases

2021-03-10 Thread Surafel Temesgen
Hi Ibrar, On Mon, Mar 8, 2021 at 8:13 AM Ibrar Ahmed wrote: > > It was a minor change therefore I rebased the patch, please take a look. > It is perfect thank you regards Surafel

Re: Add header support to text format and matching feature

2021-03-10 Thread David Steele
On 12/7/20 6:40 PM, Rémi Lapeyre wrote: Hi, here’s a rebased version of the patch. Michael, since the issue of duplicated options has been fixed do either of these patches look like they are ready for commit? Regards, -- -David da...@pgmasters.net

Re: Speeding up GIST index creation for tsvectors

2021-03-10 Thread John Naylor
On Mon, Mar 8, 2021 at 8:43 AM Amit Khandekar wrote: > > On Wed, 3 Mar 2021 at 23:32, John Naylor wrote: > > 0001: > > > > + /* > > + * We can process 64-bit chunks only if both are mis-aligned by the same > > + * number of bytes. > > + */ > > + if (b_aligned - b == a_aligned - a) > > > > The

Re: Huge memory consumption on partitioned table with FKs

2021-03-10 Thread Amit Langote
On Wed, Mar 10, 2021 at 8:37 AM Tom Lane wrote: > Amit Langote writes: > > On Fri, Mar 5, 2021 at 6:00 AM Tom Lane wrote: > >> This claim seems false on its face: > >>> All child constraints of a given foreign key constraint can use the > >>> same RI query and the resulting plan, that is, no

Re: WIP: System Versioned Temporal Table

2021-03-10 Thread Vik Fearing
On 3/10/21 5:49 PM, Surafel Temesgen wrote: > hi Ibrar, > thank you for rebasing > > On Mon, Mar 8, 2021 at 9:34 AM Ibrar Ahmed wrote: > >> >>> Since the get_row_start_time_col_name() and get_row_end_time_col_name() >>> are similar, IMO we can pass a flag to get StartTime/EndTime column name,

Re: cryptohash: missing locking functions for OpenSSL <= 1.0.2?

2021-03-10 Thread Jacob Champion
On Wed, 2021-03-10 at 17:21 +0900, Michael Paquier wrote: > Do you mean something like the attached? Yes! Patch LGTM. --Jacob

Re: [PATCH] Covering SPGiST index

2021-03-10 Thread David Steele
On 12/4/20 12:31 PM, Pavel Borisov wrote: The cfbot's still unhappy --- looks like you omitted a file from the patch? You are right, thank you. Corrected this. PFA v13 Tom, do the changes as enumerated in [1] look like they are going in the right direction? Regards, -- -David

Re: WIP: document the hook system

2021-03-10 Thread David Fetter
On Wed, Mar 10, 2021 at 09:38:39AM -0500, David Steele wrote: > On 3/9/21 12:20 PM, Bruce Momjian wrote: > > On Sat, Mar 6, 2021 at 08:32:43PM -0500, Tom Lane wrote: > > > I think that the best you should hope for here is that people are > > > willing to add a short, not-too-detailed para to a

Re: fdatasync performance problem with large number of DB files

2021-03-10 Thread Thomas Munro
On Thu, Mar 11, 2021 at 11:01 AM Michael Brown wrote: > * pg_basebackup receives a streaming backup (via [2] fsync_dir_recurse > or fsync_pgdata) unless --no-sync is specified > * postgres starts up unclean (via [3] SyncDataDirectory) > > We run multiple postgres clusters and some of those

Re: fdatasync performance problem with large number of DB files

2021-03-10 Thread Thomas Munro
On Thu, Mar 11, 2021 at 11:38 AM Thomas Munro wrote: > On Thu, Mar 11, 2021 at 11:01 AM Michael Brown > wrote: > > * is there a knob missing we can configure? > > * can we get an opt-in knob to use a single sync() call instead of a > > recursive fsync()? > > * would you be open to merging a

Re: fdatasync performance problem with large number of DB files

2021-03-10 Thread Tom Lane
Thomas Munro writes: > Thinking about this some more, if you were to propose a patch like > that syncfs() one but make it a configurable option, I'd personally be > in favour of trying to squeeze it into v14. Others might object on > commitfest procedural grounds, I dunno, but I think this is a

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-03-10 Thread Masahiro Ikeda
On 2021-03-10 17:08, Fujii Masao wrote: On 2021/03/10 14:11, Masahiro Ikeda wrote: On 2021-03-09 17:51, Fujii Masao wrote: On 2021/03/05 8:38, Masahiro Ikeda wrote: On 2021-03-05 01:02, Fujii Masao wrote: On 2021/03/04 16:14, Masahiro Ikeda wrote: On 2021-03-03 20:27, Masahiro Ikeda wrote:

Re: Huge memory consumption on partitioned table with FKs

2021-03-10 Thread Amit Langote
On Thu, Mar 11, 2021 at 4:25 AM Tom Lane wrote: > Amit Langote writes: > > On Wed, Mar 10, 2021 at 8:37 AM Tom Lane wrote: > >> Hmm. So, the key point is that the values coming from the partitioned > >> child table are injected into the test query as parameters, not as > >> column references,

Re: Improve join selectivity estimation using extended statistics

2021-03-10 Thread Tomas Vondra
Hi Konstantin, Thanks for working on this! Using extended statistics to improve join cardinality estimates was definitely on my radar, and this patch seems like a good start. I had two basic ideas about how we might improve join estimates: (a) use per-table extended statistics to estimate join

Re: Speeding up GIST index creation for tsvectors

2021-03-10 Thread John Naylor
I wrote: > I'll post a patch soon that builds on that, so you can see what I mean. I've attached where I was imagining this heading, as a text file to avoid distracting the cfbot. Here are the numbers I got with your test on the attached, as well as your 0001, on x86-64 Clang 10, default siglen:

Re: PROXY protocol support

2021-03-10 Thread Jacob Champion
On Tue, 2021-03-09 at 11:25 +0100, Magnus Hagander wrote: > I've also added some trivial tests (man that took an ungodly amount of > fighting perl -- it's clearly been a long time since I used perl > properly). Yeah. The tests I'm writing for this and NSS have been the same way; it's a real

Re: libpq debug log

2021-03-10 Thread Tom Lane
Alvaro Herrera writes: > On 2021-Mar-10, Tom Lane wrote: >> After studying this further, I think we should apply the attached >> patch to remove that responsibility from pqParseInput3's subroutines. >> This will allow a single trace call near the bottom of pqParseInput3 >> to handle all cases

Re: fdatasync performance problem with large number of DB files

2021-03-10 Thread Fujii Masao
On 2021/03/11 8:30, Thomas Munro wrote: On Thu, Mar 11, 2021 at 11:38 AM Thomas Munro wrote: On Thu, Mar 11, 2021 at 11:01 AM Michael Brown wrote: * is there a knob missing we can configure? * can we get an opt-in knob to use a single sync() call instead of a recursive fsync()? * would

fdatasync performance problem with large number of DB files

2021-03-10 Thread Michael Brown
I initially posted this on the pgsql-general mailing list [5] but was advised to also post this to the -hackers list as it deals with internals. We've encountered a production performance problem with pg13 related to how it fsyncs the whole data directory in certain scenarios, related to what

Re: GiST comment improvement

2021-03-10 Thread Bruce Momjian
On Tue, Mar 2, 2021 at 11:40:21AM -0500, Bruce Momjian wrote: > In talking to Teodor this week, I have written the attached C comment > patch which improves our description of GiST's NSN and GistBuildLSN > values. Patch applied. -- Bruce Momjian https://momjian.us EDB

Re: libpq debug log

2021-03-10 Thread Alvaro Herrera
On 2021-Mar-10, Tom Lane wrote: > After studying this further, I think we should apply the attached > patch to remove that responsibility from pqParseInput3's subroutines. > This will allow a single trace call near the bottom of pqParseInput3 > to handle all cases that that function processes.

Re: [HACKERS] Custom compression methods

2021-03-10 Thread Justin Pryzby
On Wed, Mar 10, 2021 at 03:50:48PM -0500, Robert Haas wrote: > On Wed, Mar 10, 2021 at 6:52 AM Dilip Kumar wrote: > > The pending comment is providing a way to rewrite a table and > > re-compress the data with the current compression method. > > I spent some time poking at this yesterday and ran

RE: Enhance traceability of wal_level changes for backup management

2021-03-10 Thread osumi.takami...@fujitsu.com
On Tuesday, March 9, 2021 11:13 PM David Steele > On 3/7/21 9:45 PM, osumi.takami...@fujitsu.com wrote: > > On Sun, Mar 7, 2021 3:48 AM Peter Eisentraut > wrote: > >> On 28.01.21 01:44, osumi.takami...@fujitsu.com wrote: > (1) writing the time or LSN in the control file to indicate >

Re: Replace buffer I/O locks with condition variables (reviving an old patch)

2021-03-10 Thread Thomas Munro
On Tue, Mar 9, 2021 at 6:24 PM Julien Rouhaud wrote: > > The old I/O lock array was the only user of struct > > LWLockMinimallyPadded, added in commit 6150a1b08a9, and it seems kinda > > strange to leave it in the tree with no user. Of course it's remotely > > possible there are extensions using

Re: fdatasync performance problem with large number of DB files

2021-03-10 Thread Thomas Munro
On Thu, Mar 11, 2021 at 1:16 PM Tom Lane wrote: > Thomas Munro writes: > > Thinking about this some more, if you were to propose a patch like > > that syncfs() one but make it a configurable option, I'd personally be > > in favour of trying to squeeze it into v14. Others might object on > >

Re: pgbench - add pseudo-random permutation function

2021-03-10 Thread Bruce Momjian
On Mon, Mar 8, 2021 at 11:50:43AM +0100, Fabien COELHO wrote: > > > > What are your current thoughts? > > > > Thanks for prodding. I still think it's a useful feature. However I > > don't think I'll have to time to get it done on the current commitfest. > > I suggest to let it sit in the

Re: Huge memory consumption on partitioned table with FKs

2021-03-10 Thread Keisuke Kuroda
Hi hackers, > > > > I did some cosmetic fooling with this (mostly, rewriting the comments > > YA time) and pushed it. > > Perfect. Thanks for your time on this. > Thank you for your help! I'm glad to solve it. -- Keisuke Kuroda NTT Software Innovation Center keisuke.kuroda.3...@gmail.com

Re: automatic analyze: readahead - add "IO read time" log message

2021-03-10 Thread Tomas Vondra
On 3/8/21 8:42 PM, Stephen Frost wrote: > Greetings, > > * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: >> On 2/10/21 11:10 PM, Stephen Frost wrote: >>> * Heikki Linnakangas (hlinn...@iki.fi) wrote: On 05/02/2021 23:22, Stephen Frost wrote: > Unless there's anything else on this,

Re: Columns correlation and adaptive query optimization

2021-03-10 Thread Tomas Vondra
On 3/10/21 3:00 AM, Tomas Vondra wrote: > Hello Konstantin, > > > Sorry for not responding to this thread earlier. I definitely agree the > features proposed here are very interesting and useful, and I appreciate > you kept rebasing the patch. > > I think the patch improving join estimates can

Re: shared-memory based stats collector

2021-03-10 Thread Kyotaro Horiguchi
At Wed, 10 Mar 2021 21:47:51 +0900, Fujii Masao wrote in > Attached is the updated version of the 0003 patch. Barring any > objection, > I will commit this patch. > > > -#include "storage/latch.h" > -#include "storage/proc.h" > > I removed these because they are no longer necessary. Mmm.

Re: Boundary value check in lazy_tid_reaped()

2021-03-10 Thread Masahiko Sawada
On Wed, Mar 10, 2021 at 11:53 PM Peter Eisentraut wrote: > > On 10.03.21 02:29, Masahiko Sawada wrote: > >> There is no noticeable effect of inlining lazy_tid_reaped(). So it > >> would be better to not do that. > > > > Attached the patch that doesn't inline lazy_tid_reaped(). > > committed

Re: Removing vacuum_cleanup_index_scale_factor

2021-03-10 Thread Peter Geoghegan
On Tue, Mar 9, 2021 at 7:42 PM Peter Geoghegan wrote: > My current plan is to commit everything within the next day or two. > This includes backpatching to Postgres 13 only. Pushed, thanks. -- Peter Geoghegan

Re: fdatasync performance problem with large number of DB files

2021-03-10 Thread Thomas Munro
On Thu, Mar 11, 2021 at 2:00 PM Fujii Masao wrote: > On 2021/03/11 8:30, Thomas Munro wrote: > > I've run into a couple of users who have just commented that recursive > > fsync() code out! > > BTW, we can skip that recursive fsync() by disabling fsync GUC even without > commenting out the code?

Re: fdatasync performance problem with large number of DB files

2021-03-10 Thread Thomas Munro
On Thu, Mar 11, 2021 at 2:25 PM Tom Lane wrote: > Trolling the net, I found a newer-looking version of the man page, > and behold it says > >In mainline kernel versions prior to 5.8, syncfs() will fail only >when passed a bad file descriptor (EBADF). Since Linux 5.8, >

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-03-10 Thread Fujii Masao
On 2021/03/11 9:38, Masahiro Ikeda wrote: On 2021-03-10 17:08, Fujii Masao wrote: On 2021/03/10 14:11, Masahiro Ikeda wrote: On 2021-03-09 17:51, Fujii Masao wrote: On 2021/03/05 8:38, Masahiro Ikeda wrote: On 2021-03-05 01:02, Fujii Masao wrote: On 2021/03/04 16:14, Masahiro Ikeda wrote:

RE: libpq debug log

2021-03-10 Thread tsunakawa.ta...@fujitsu.com
From: Kyotaro Horiguchi > The output functions copy message bytes into local variable but the > same effect can be obtained by just casing via typed pointer type. > > uint32 tmp4; > .. > memcpy(, buf + *cursor, 4); > result = (int) pg_ntoh32(tmp4); > > can be written as > > result =

Re: [HACKERS] Custom compression methods

2021-03-10 Thread Justin Pryzby
On Thu, Mar 11, 2021 at 08:17:46AM +0530, Dilip Kumar wrote: > On Thu, Mar 11, 2021 at 2:21 AM Robert Haas wrote: > > > > On Wed, Mar 10, 2021 at 6:52 AM Dilip Kumar wrote: > > > The pending comment is providing a way to rewrite a table and > > > re-compress the data with the current compression

Re: shared-memory based stats collector

2021-03-10 Thread Andres Freund
Hi, Two minor nits: On 2021-03-10 21:47:51 +0900, Fujii Masao wrote: > +/* Shared memory area for archiver process */ > +typedef struct PgArchData > +{ > + Latch *latch; /* latch to wake the archiver > up */ > + slock_t mutex; /* locks

Re: PostmasterIsAlive() in recovery (non-USE_POST_MASTER_DEATH_SIGNAL builds)

2021-03-10 Thread Thomas Munro
On Tue, Mar 2, 2021 at 2:10 PM Thomas Munro wrote: > ... One question I haven't > got to the bottom of: is it a problem for the startup process that CVs > use CHECK_FOR_INTERRUPTS()? This was a red herring. The startup process already reaches CFI() via various paths, as I figured out pretty

Re: [PATCH] Provide more information to filter_prepare

2021-03-10 Thread Amit Kapila
On Wed, Mar 10, 2021 at 4:26 PM Markus Wanner wrote: > > On 10.03.21 11:18, Amit Kapila wrote: > > On Tue, Mar 9, 2021 at 2:14 PM Markus Wanner > > wrote: > >> currently, only the gid is passed on to the filter_prepare callback. > >> While we probably should not pass a full ReorderBufferTXN (as

RE: libpq debug log

2021-03-10 Thread tsunakawa.ta...@fujitsu.com
Alvaro-san, Tom-san, Horiguchi-san, From: Kyotaro Horiguchi > +1 for the thanks for the quick work. I have some random comments > after a quick look on it. Thank you very much for giving many comments. And We're sorry to have caused you trouble. I told Iwata-san yesterday to modify the

RE: Parallel INSERT (INTO ... SELECT ...)

2021-03-10 Thread houzj.f...@fujitsu.com
> > 2. Should we keep the default value of GUC to on or off? It is > > currently off. I am fine keeping it off for this release and we can > > always turn it on in the later releases if required. Having said that, > > I see the value in keeping it on because in many cases Insert ... > > Select

Re: Replace buffer I/O locks with condition variables (reviving an old patch)

2021-03-10 Thread Julien Rouhaud
On Thu, Mar 11, 2021 at 03:54:06PM +1300, Thomas Munro wrote: > On Thu, Mar 11, 2021 at 3:27 PM Julien Rouhaud wrote: > > - /* > > -* It would be nice to include the I/O locks in the BufferDesc, but that > > -* would increase the size of a BufferDesc to more than one cache line, > > -

Re: libpq debug log

2021-03-10 Thread Kyotaro Horiguchi
At Thu, 11 Mar 2021 03:01:16 +, "tsunakawa.ta...@fujitsu.com" wrote in > From: Kyotaro Horiguchi > > The output functions copy message bytes into local variable but the > > same effect can be obtained by just casing via typed pointer type. > > > > uint32 tmp4; > > .. > > memcpy(,

Re: libpq debug log

2021-03-10 Thread Kyotaro Horiguchi
At Thu, 11 Mar 2021 04:12:57 +, "tsunakawa.ta...@fujitsu.com" wrote in > From: Kyotaro Horiguchi > > Right. So something like this? > > > > unsigned char p; > > > > p = buf + *cursor; > > result = (uint32) (*p << 24) + (*(p + 1)) << 16 + ...); > > Yes, that would work (if p is a

Re: shared-memory based stats collector

2021-03-10 Thread Kyotaro Horiguchi
At Wed, 10 Mar 2021 19:21:00 -0800, Andres Freund wrote in > Hi, > > Two minor nits: Thanks for the comments! > On 2021-03-10 21:47:51 +0900, Fujii Masao wrote: > > +/* Shared memory area for archiver process */ > > +typedef struct PgArchData > > +{ > > + Latch *latch;

Re: 64-bit XIDs in deleted nbtree pages

2021-03-10 Thread Peter Geoghegan
On Mon, Mar 1, 2021 at 7:25 PM Peter Geoghegan wrote: > Attached is v8 of the patch series, which has new patches. No real > changes compared to v7 for the first patch, though. Here is another bitrot-fix-only revision, v9. Just the recycling patch again. I'll commit this when we get your patch

Re: libpq debug log

2021-03-10 Thread Tom Lane
Kyotaro Horiguchi writes: > Maybe I'm missing something, but the above doesn't seem working. I > didn't see such message when making a SSL connection. As far as that goes, I don't see any point in trying to trace connection-related messages, because there is no way to enable tracing on a

Re: [HACKERS] Custom compression methods

2021-03-10 Thread Justin Pryzby
This includes a patch to use pkgconfig, in an attempt to build on mac, which currently fails like: https://cirrus-ci.com/task/5993712963551232?command=build#L126 checking for LZ4_compress in -llz4... no configure: error: library 'lz4' is required for LZ4 support -- Justin >From

Re: Replace buffer I/O locks with condition variables (reviving an old patch)

2021-03-10 Thread Thomas Munro
On Thu, Mar 11, 2021 at 3:27 PM Julien Rouhaud wrote: > - /* > -* It would be nice to include the I/O locks in the BufferDesc, but that > -* would increase the size of a BufferDesc to more than one cache line, > -* and benchmarking has shown that keeping every BufferDesc aligned on

Re: libpq debug log

2021-03-10 Thread Kyotaro Horiguchi
At Wed, 10 Mar 2021 20:38:20 -0500, Tom Lane wrote in > Kyotaro Horiguchi writes: > > Maybe I'm missing something, but the above doesn't seem working. I > > didn't see such message when making a SSL connection. > > As far as that goes, I don't see any point in trying to trace >

Re: libpq debug log

2021-03-10 Thread Kyotaro Horiguchi
At Thu, 11 Mar 2021 01:51:55 +, "tsunakawa.ta...@fujitsu.com" wrote in > Alvaro-san, Tom-san, Horiguchi-san, > > From: Kyotaro Horiguchi > > +1 for the thanks for the quick work. I have some random comments > > after a quick look on it. > > Thank you very much for giving many comments.

Re: shared-memory based stats collector

2021-03-10 Thread Andres Freund
Hi, On 2021-03-10 17:51:37 +0900, Kyotaro Horiguchi wrote: > From ed2fb2fca47fccbf9af1538688aab8334cf6470c Mon Sep 17 00:00:00 2001 > From: Kyotaro Horiguchi > Date: Fri, 13 Mar 2020 16:58:03 +0900 > Subject: [PATCH v52 1/7] sequential scan for dshash > > Dshash did not allow scan the all

Re: [HACKERS] Custom compression methods

2021-03-10 Thread Dilip Kumar
On Thu, Mar 11, 2021 at 8:50 AM Justin Pryzby wrote: > > Looking at v23-0002-alter-table-set-compression, ATRewriteTable() was calling > CompareCompressionMethodAndDecompress(). While changing the compression method user might be just interested to compress the future tuple with the new

Re: libpq debug log

2021-03-10 Thread Kyotaro Horiguchi
At Wed, 10 Mar 2021 10:31:27 -0300, "'alvhe...@alvh.no-ip.org'" wrote in > On 2021-Mar-10, iwata@fujitsu.com wrote: > > > Hi all, > > > > Following all reviewer's advice, I have created a new patch. > > > > In this patch, I add only two tracing entry points; I call > >

  1   2   >