[PATCH] Provide more information to filter_prepare

Hi, currently, only the gid is passed on to the filter_prepare callback. While we probably should not pass a full ReorderBufferTXN (as we do for most other output plugin callbacks), a bit more information would be nice, I think. Attached is a patch that adds the xid (still lacking docs

Re: Make stream_prepare an optional callback

On Tue, Mar 9, 2021 at 3:41 PM Markus Wanner wrote: > > On 09.03.21 10:37, Amit Kapila wrote: > I guess I don't quite understand the initial motivation for the patch. > It states: "This allows plugins to not allow the enabling of streaming > and two_phase at the same time in logical replication."

Re: cleanup temporary files after crash

On Tue, Mar 09, 2021 at 02:28:43AM +0100, Tomas Vondra wrote: > Let's move this patch forward. Based on the responses, I agree the > default behavior should be to remove the temp files, and I think we > should have the GUC (on the off chance that someone wants to preserve > the temporary files for

Re: Make stream_prepare an optional callback

On Tue, Mar 9, 2021 at 2:23 PM Markus Wanner wrote: > > On 09.03.21 09:39, Amit Kapila wrote: > > It is attached with the initial email. > > Oh, sorry, I looked up the initial email, but still didn't see the patch. > > > I think so. The behavior has to be similar to other optional callbacks > >

Re: Make stream_prepare an optional callback

On 09.03.21 10:37, Amit Kapila wrote: AFAICS, the error is removed by the patch as per below change: Ah, well, that does not seem right, then. We cannot just silently ignore the callback but not skip the prepare, IMO. That would lead to the output plugin missing the PREPARE, but still

Re: [POC] verifying UTF-8 using SIMD instructions

On Tue, Mar 9, 2021 at 5:00 AM Amit Khandekar wrote: > > Hi, > > Just a quick question before I move on to review the patch ... The > improvement looks like it is only meant for x86 platforms. Actually it's meant to be faster for all platforms, since the C fallback is quite a bit different from

Re: Make stream_prepare an optional callback

On 09.03.21 09:39, Amit Kapila wrote: > It is attached with the initial email. Oh, sorry, I looked up the initial email, but still didn't see the patch. I think so. The behavior has to be similar to other optional callbacks like message_cb, truncate_cb, stream_truncate_cb. Basically, we don't

Add some tests for pg_stat_statements compatibility verification under contrib

Hi All, On the master branch, it is possible to install multiple versions of pg_stat_statements with CREATE EXTENSION, but all the tests in sql/ on look at the latest version available, without testing past compatibility. Since we support to install lowest version 1.4 currently, add some tests

Re: [PATCH] regexp_positions ( string text, pattern text, flags text ) → setof int4range[]

út 9. 3. 2021 v 10:01 odesílatel Joel Jacobson napsal: > On Tue, Mar 9, 2021, at 09:29, Pavel Stehule wrote: > > > > https://www.postgresql.org/docs/current/plpgsql-control-structures.html#PLPGSQL-FOREACH-ARRAY > > but the SLICE argument should be constant. But this limit is artificial, > just

Re: [PATCH] regexp_positions ( string text, pattern text, flags text ) → setof int4range[]

On Tue, Mar 9, 2021, at 10:18, Pavel Stehule wrote: > What do you mean? >> More than one unnest() in the same query, e.g. SELECT unnest(..), unnest(..)? > > you can do unnest(array1, array2, ...) Right, I had forgotten about that variant. But isn't this a bit surprising then: \df unnest

Enlarge IOS vm cache

Hi, Our customer experienced a significant slowdown on queries involving Index Only Scan. As it turned out, the problem was constant pin-unpin of the visibility map page. IOS caches only one vm page, which corresponds to 8192 * 8 / 2 * 8192 bytes = 256 MB of data; if the table is larger and the

Questions about CommandIsReadOnly

Hi hackers, When reading the code, I found that in function CommandIsReadOnly[1], "select for update/share" is defined as "not read only". [1]- if (pstmt->rowMarks != NIL) return false; /* SELECT FOR [KEY] UPDATE/SHARE */ - And from

Re: [PATCH] regexp_positions ( string text, pattern text, flags text ) → setof int4range[]

út 9. 3. 2021 v 11:32 odesílatel Joel Jacobson napsal: > On Thu, Mar 4, 2021, at 16:40, Tom Lane wrote: > > My experience with working with parallel arrays in SQL has been unpleasant. > > > Could you please give an example on such an unpleasant experience? > it was more complex application with

Re: [POC] verifying UTF-8 using SIMD instructions

Hi, Just a quick question before I move on to review the patch ... The improvement looks like it is only meant for x86 platforms. Can this be done in a portable way by arranging for auto-vectorization ? Something like commit 88709176236caf. This way it would benefit other platforms as well. I

RE: POC: Cleaning up orphaned files using undo logs

I'm crawling like a snail to read the patch set. Below are my first set of review comments, which are all minor. (1) + tablespacetemporary temporary -> undo (2) undo_tablespaces (string) + ... +The value is a list of names of tablespaces. When there is more than +

Re: [HACKERS] logical decoding of two-phase transactions

On Mon, Mar 8, 2021 at 4:58 PM vignesh C wrote: > > + while (AnyTablesyncInProgress()) > + { > + process_syncing_tables(begin_data.final_lsn); > + > + /* This latch is to prevent 100% CPU looping. */ > +

Re: [HACKERS] logical decoding of two-phase transactions

On Tue, Mar 9, 2021 at 3:02 PM Peter Smith wrote: > > On Mon, Mar 8, 2021 at 4:58 PM vignesh C wrote: > > > > + while (AnyTablesyncInProgress()) > > + { > > + process_syncing_tables(begin_data.final_lsn); > > + > > + /* This

Re: PROXY protocol support

On Sat, Mar 6, 2021 at 5:30 PM Magnus Hagander wrote: > > On Sat, Mar 6, 2021 at 4:17 PM Magnus Hagander wrote: > > > > On Fri, Mar 5, 2021 at 8:11 PM Jacob Champion wrote: > > > > > > On Fri, 2021-03-05 at 10:22 +0100, Magnus Hagander wrote: > > > > On Fri, Mar 5, 2021 at 12:21 AM Jacob

Outdated comments about proc->sem in lwlock.c

Hi, In passing I noticed that lwlock.c contains 3 comments about bogus wakeups due to sharing proc->sem with the heavyweight lock manager and ProcWaitForSignal. Commit 675f55e (9.5) switched those things from proc->sem to proc->procLatch. ProcArrayGroupClearXid() and

Re: ResourceOwner refactoring

On 08/03/2021 18:47, Ibrar Ahmed wrote: The patchset does not apply successfully, there are some hunk failures. http://cfbot.cputube.org/patch_32_2834.log v6-0002-Make-resowners-more-easily-extensible.patch 1 out of 6 hunks FAILED -- saving

Re: Make stream_prepare an optional callback

On Tue, Mar 9, 2021 at 1:55 PM Markus Wanner wrote: > > On 09.03.21 07:40, Amit Kapila wrote: > > Sounds reasonable to me. I also don't see a reason why we need to make > > this a necessary callback. Some plugin authors might just want 2PC > > without streaming support. > > Sounds okay to me.

Re: About to add WAL write/fsync statistics to pg_stat_wal view

On 2021/03/05 8:38, Masahiro Ikeda wrote: On 2021-03-05 01:02, Fujii Masao wrote: On 2021/03/04 16:14, Masahiro Ikeda wrote: On 2021-03-03 20:27, Masahiro Ikeda wrote: On 2021-03-03 16:30, Fujii Masao wrote: On 2021/03/03 14:33, Masahiro Ikeda wrote: On 2021-02-24 16:14, Fujii Masao

Re: Add some tests for pg_stat_statements compatibility verification under contrib

Hi, On Tue, Mar 09, 2021 at 11:35:14AM +0800, Erica Zhang wrote: > Hi All, > On the master branch, it is possible to install multiple versions of > pg_stat_statements with CREATE EXTENSION, but all the tests in sql/ on look > at the latest version available, without testing past compatibility.

Re: [PATCH] regexp_positions ( string text, pattern text, flags text ) → setof int4range[]

On Thu, Mar 4, 2021, at 16:40, Tom Lane wrote: > My experience with working with parallel arrays in SQL has been unpleasant. Could you please give an example on such an unpleasant experience? I can see a problem if the arrays could possibly have difference dimensionality/cardinality, but

Re: SQL-standard function body

On 05.03.21 06:58, Jaime Casanova wrote: I was making some tests with this patch and found this problem: """ CREATE OR REPLACE FUNCTION public.make_table() RETURNS void LANGUAGE sql BEGIN ATOMIC CREATE TABLE created_table AS SELECT * FROM int8_tbl; END; ERROR: unrecognized token: "?"

Re: Nicer error when connecting to standby with hot_standby=off

On 2021-Mar-09, James Coleman wrote: > On Tue, Mar 9, 2021 at 8:47 AM Alvaro Herrera wrote: > > > > On 2021-Mar-07, Magnus Hagander wrote: > > > > > On Sun, Mar 7, 2021 at 3:39 PM Fujii Masao > > > wrote: > > > > Great, so we're agreed on the messages to emit. James, are you updating > > your

Re: Nicer error when connecting to standby with hot_standby=off

On Tue, Mar 9, 2021 at 9:07 AM Alvaro Herrera wrote: > > On 2021-Mar-09, James Coleman wrote: > > > On Tue, Mar 9, 2021 at 8:47 AM Alvaro Herrera > > wrote: > > > > > > On 2021-Mar-07, Magnus Hagander wrote: > > > > > > > On Sun, Mar 7, 2021 at 3:39 PM Fujii Masao > > > > wrote: > > > > > >

Re: Nicer error when connecting to standby with hot_standby=off

On Tue, Mar 9, 2021 at 9:17 AM Alvaro Herrera wrote: > > On 2021-Mar-09, James Coleman wrote: > > > Yes, I think they both agreed on the "DETAIL: Hot standby mode is > > disabled." message, but that alternative meant not needing to add any > > new signals and pm states, correct? > > Ah, I see!

Re: Any objections to implementing LogicalDecodeMessageCB for pgoutput?

Hi David, On 11/24/20 10:28 PM, Euler Taveira wrote: I also reviewed your patch. This feature would be really useful for replication scenarios. Supporting this feature means that you don't need to use a table to pass messages from one node to another one. Here are a few comments/ideas. Do

RE: non-HOT update not looking at FSM for large tuple update

Hi, > > This patch fails to consider that len may be bigger than MaxHeapTupleSize * > 0.98, which in this case triggers a reproducable > PANIC: Good catch! I've adapted the patch with your suggested fix. > > One different question I have, though, is why we can't "just" teach vacuum > to clean

Re: DROP relation IF EXISTS Docs and Tests - Bug Fix

Hi David, On 11/23/20 3:31 PM, Anastasia Lubennikova wrote: On 30.09.2020 05:00, David G. Johnston wrote: v5 attached, looking at this fresh and with some comments to consider. I ended up just combining both patches into one. I did away with the glossary changes altogether, and the

Re: DROP relation IF EXISTS Docs and Tests - Bug Fix

On 3/9/21 10:08 AM, David G. Johnston wrote: On Tuesday, March 9, 2021, David Steele > wrote: Further, I think we should close this entry at the end of the CF if it does not attract committer interest. Tom is not in favor of the patch and it appears

Re: shared-memory based stats collector

On 2021/03/09 16:51, Kyotaro Horiguchi wrote: At Sat, 6 Mar 2021 00:32:07 +0900, Fujii Masao wrote in On 2021/03/05 17:18, Kyotaro Horiguchi wrote: At Thu, 21 Jan 2021 12:03:48 +0900 (JST), Kyotaro Horiguchi wrote in Commit 960869da08 (database statistics) conflicted with this.

Re: row filtering for logical replication

Hi Euler, Please find some comments below: 1. If the where clause contains non-replica identity columns, the delete performed on a replicated row using DELETE from pub_tab where repl_ident_col = n; is not being replicated, as logical replication does not have any info whether the column has to

Lowering the ever-growing heap->pd_lower

Hi, The heap AMs' pages only grow their pd_linp array, and never shrink when trailing entries are marked unused. This means that up to 14% of free space (=291 unused line pointers) on a page could be unusable for data storage, which I think is a shame. With a patch in the works that allows the

Re: cleanup temporary files after crash

On Tue, Mar 9, 2021, at 9:31 AM, Michael Paquier wrote: > On Tue, Mar 09, 2021 at 02:28:43AM +0100, Tomas Vondra wrote: > > Let's move this patch forward. Based on the responses, I agree the > > default behavior should be to remove the temp files, and I think we > > should have the GUC (on the off

Re: authtype parameter in libpq

On 08.03.21 10:57, Peter Eisentraut wrote: On 04.03.21 16:06, Daniel Gustafsson wrote: authtype is completely dead in terms of reading back the value, to the point of it being a memleak if it indeed was found in as an environment variable. But I tend to think we should remove them both

Re: [PATCH] regexp_positions ( string text, pattern text, flags text ) → setof int4range[]

On Tue, Mar 9, 2021, at 13:16, Pavel Stehule wrote: > út 9. 3. 2021 v 11:32 odesílatel Joel Jacobson napsal: >> __On Thu, Mar 4, 2021, at 16:40, Tom Lane wrote: >>> My experience with working with parallel arrays in SQL has been unpleasant. >> >> Could you please give an example on such an

Re: Nicer error when connecting to standby with hot_standby=off

On 2021/03/09 23:19, James Coleman wrote: On Tue, Mar 9, 2021 at 9:17 AM Alvaro Herrera wrote: On 2021-Mar-09, James Coleman wrote: Yes, I think they both agreed on the "DETAIL: Hot standby mode is disabled." message, but that alternative meant not needing to add any new signals and pm

Re: Nicer error when connecting to standby with hot_standby=off

On 2021-Mar-07, Magnus Hagander wrote: > On Sun, Mar 7, 2021 at 3:39 PM Fujii Masao > wrote: > > > > Here's an idea: > > > > > > * hot_standby=on, before reaching consistent state > > >FATAL: database is not accepting connections > > >DETAIL: Consistent state has not yet been reached.

Re: Nicer error when connecting to standby with hot_standby=off

On Tue, Mar 9, 2021 at 8:47 AM Alvaro Herrera wrote: > > On 2021-Mar-07, Magnus Hagander wrote: > > > On Sun, Mar 7, 2021 at 3:39 PM Fujii Masao > > wrote: > > > > > > Here's an idea: > > > > > > > > * hot_standby=on, before reaching consistent state > > > >FATAL: database is not accepting

Re: default result formats setting

On 11/25/20 2:06 AM, Peter Eisentraut wrote: On 2020-11-16 16:15, Andrew Dunstan wrote: I think this is conceptually OK, although it feels a bit odd. Might it be better to have the values as typename={binary,text} pairs instead of oid={0,1} pairs, which are fairly opaque? That might make

Re: [PATCH] pg_permissions

On Tue, Mar 9, 2021, at 07:34, Joel Jacobson wrote: > On Tue, Mar 9, 2021, at 04:01, Chapman Flack wrote: >> 1. Is there a reason not to make 'grantor' and 'grantee' of type regrole? Having digested your idea, I actually agree with you. Since we have the regrole-type, I agree we should use it,

Re: Enhance traceability of wal_level changes for backup management

On 3/7/21 9:45 PM, osumi.takami...@fujitsu.com wrote: On Sun, Mar 7, 2021 3:48 AM Peter Eisentraut wrote: On 28.01.21 01:44, osumi.takami...@fujitsu.com wrote: (1) writing the time or LSN in the control file to indicate when/where wal_level is changed to 'minimal' from upper level to

Re: Nicer error when connecting to standby with hot_standby=off

On Tue, Mar 9, 2021 at 3:07 PM Alvaro Herrera wrote: > > On 2021-Mar-09, James Coleman wrote: > > > On Tue, Mar 9, 2021 at 8:47 AM Alvaro Herrera > > wrote: > > > > > > On 2021-Mar-07, Magnus Hagander wrote: > > > > > > > On Sun, Mar 7, 2021 at 3:39 PM Fujii Masao > > > > wrote: > > > > > >

Re: Nicer error when connecting to standby with hot_standby=off

On 2021-Mar-09, James Coleman wrote: > Yes, I think they both agreed on the "DETAIL: Hot standby mode is > disabled." message, but that alternative meant not needing to add any > new signals and pm states, correct? Ah, I see! I was thinking that you still needed the state and signal in order

Re: DROP relation IF EXISTS Docs and Tests - Bug Fix

On Tuesday, March 9, 2021, David Steele wrote: > > Further, I think we should close this entry at the end of the CF if it > does not attract committer interest. Tom is not in favor of the patch and > it appears Alexander decided not to commit it. > Pavel re-reviewed it and was fine with

Re: [patch] [doc] Minor variable related cleanup and rewording of plpgsql docs

On 11/30/20 10:37 AM, Pavel Stehule wrote: po 30. 11. 2020 v 16:06 odesílatel David G. Johnston ok This patch looks reasonable to me overall. A few comments: 1) PL/SQL seems to be used in a few places where I believe PL/pgSQL is meant. This was pre-existing but now seems like a good

Re: Lowering the ever-growing heap->pd_lower

> On Mar 9, 2021, at 7:13 AM, Matthias van de Meent > wrote: > > Hi, > > The heap AMs' pages only grow their pd_linp array, and never shrink > when trailing entries are marked unused. This means that up to 14% of > free space (=291 unused line pointers) on a page could be unusable for >

Re: A problem about partitionwise join

On 11/27/20 7:05 AM, Ashutosh Bapat wrote: On Tue, Nov 10, 2020 at 2:43 PM Richard Guo wrote: To recap, the problem we are fixing here is when generating join clauses from equivalence classes, we only select the joinclause with the 'best score', or the first joinclause with a score of 3. This

Re: Re: [PATCH] regexp_positions ( string text, pattern text, flags text ) → setof int4range[]

"Joel Jacobson" writes: > Tom - can you please give details on your unpleasant experiences with > parallel arrays? The problems I can recall running into were basically down to not having an easy way to iterate through parallel arrays. There are ways to do that in SQL, certainly, but they all

Re: non-HOT update not looking at FSM for large tuple update

I wrote: > That seems like the proper fix, and I see you've started a thread for that. I don't think that change in behavior would be backpatchable, but patch here might have a chance at that. I remembered after the fact that truncating line pointers would only allow for omitting the 2% slack

Re: Proposal: Save user's original authenticated identity for logging

On Mon, 2021-03-08 at 22:16 +, Jacob Champion wrote: > On Sat, 2021-03-06 at 18:33 +0100, Magnus Hagander wrote: > > As for log escaping, we report port->user_name already unescaped -- > > surely this shouldn't be a worse case than that? > > Ah, that's a fair point. I'll remove the TODO. v4

Re: Proposal: Save user's original authenticated identity for logging

On Tue, 2021-03-09 at 18:03 +, Jacob Champion wrote: > v4 removes the TODO and the extra allocation for peer_user. I'll hold > off on the other two suggestions pending that conversation. And v5 is rebased over this morning's SSL test changes. --Jacob From

Procedures versus the "fastpath" API

The security team received a report from Theodor-Arsenij Larionov-Trichkin of PostgresPro that it's possible to crash the backend with an assertion or null-pointer dereference by trying to call a window function via the "fast path function call" protocol message. fastpath.c doesn't set up any

Re: Re: [PATCH] regexp_positions ( string text, pattern text, flags text ) → setof int4range[]

"Joel Jacobson" writes: > On Tue, Mar 9, 2021, at 10:18, Pavel Stehule wrote: >> you can do unnest(array1, array2, ...) > Right, I had forgotten about that variant. > But isn't this a bit surprising then: > ... > Should there be an entry there showing the VARIADIC anyelement version as > well?

Re: [patch] [doc] Minor variable related cleanup and rewording of plpgsql docs

út 9. 3. 2021 v 18:03 odesílatel David Steele napsal: > On 11/30/20 10:37 AM, Pavel Stehule wrote: > > po 30. 11. 2020 v 16:06 odesílatel David G. Johnston > > > > ok > This patch looks reasonable to me overall. > > A few comments: > > 1) PL/SQL seems to be used in a few places where I believe

Re: partial heap only tuples

On Mon, Feb 15, 2021 at 08:19:40PM +, Bossart, Nathan wrote: > Yeah, this is something I'm concerned about. I think adding a bitmap > of modified columns to the header of PHOT-updated tuples improves > matters quite a bit, even for single-page vacuuming. Following is a > strategy I've been

Re: non-HOT update not looking at FSM for large tuple update

On Tue, Mar 9, 2021 at 9:40 AM Floris Van Nee wrote: > > Hi, > > > > > This patch fails to consider that len may be bigger than MaxHeapTupleSize * > > 0.98, which in this case triggers a reproducable > > PANIC: > > Good catch! I've adapted the patch with your suggested fix. Thank you both for

Re: Online checksums patch - once again

On Mon, Feb 15, 2021 at 02:02:02PM +0100, Daniel Gustafsson wrote: > > On 11 Feb 2021, at 14:10, Bruce Momjian wrote: > > I don't think anyone has done anything wrong --- rather, it is what we > > are _trying_ to do that is complex. > > Global state changes in a cluster are complicated, and are

Re: Fwd: Row description Metadata information

On Mon, Feb 15, 2021 at 05:25:55PM -0800, Aleksei Ivanov wrote: > Not sure that previous email was sent correctly. If it was duplicated, sorry > for the inconvenience. > > Hi, hackers, > > I have one question related to returned information in the row description for > prepared statement. > >

Re: [PATCH] pg_permissions

On 03/09/21 11:11, Joel Jacobson wrote: > On Tue, Mar 9, 2021, at 07:34, Joel Jacobson wrote: >> On Tue, Mar 9, 2021, at 04:01, Chapman Flack wrote: >>> 1. Is there a reason not to make 'grantor' and 'grantee' of type regrole? > > Having digested your idea, I actually agree with you. > > Since

Re: [PATCH] pg_permissions

On Tue, Mar 9, 2021, at 04:01, Chapman Flack wrote: > On Sat, Mar 06, 2021 at 08:03:17PM +0100, Joel Jacobson wrote: > >regclass | obj_desc | grantor | grantee | > privilege_type | is_grantable > > >

Re: default result formats setting

David Steele writes: > Andrew, Tom, does the latest patch address your concerns? [ reads patch quickly... ] I think the definition is fine now, modulo possible bikeshedding on the GUC name. (I have no great suggestion on that right now, but the current proposal seems mighty verbose.) The

Re: [patch] [doc] Minor variable related cleanup and rewording of plpgsql docs

David Steele writes: > 1) PL/SQL seems to be used in a few places where I believe PL/pgSQL is > meant. This was pre-existing but now seems like a good opportunity to > fix it, unless I am misunderstanding. PL/SQL is Oracle's function language, which PL/pgSQL is modeled on. At least some of the

Re: WIP: document the hook system

On Sat, Mar 6, 2021 at 08:32:43PM -0500, Tom Lane wrote: > I think that the best you should hope for here is that people are > willing to add a short, not-too-detailed para to a markup-free > plain-text README file that lists all the hooks. As soon as it > gets any more complex than that, either

Re: Online verification of checksums

On 11/30/20 6:38 PM, David Steele wrote: On 11/30/20 9:27 AM, Stephen Frost wrote: * Michael Paquier (mich...@paquier.xyz) wrote: On Fri, Nov 27, 2020 at 11:15:27AM -0500, Stephen Frost wrote: * Magnus Hagander (mag...@hagander.net) wrote: On Thu, Nov 26, 2020 at 8:42 AM Michael Paquier

Re: Online checksums patch - once again

> On 9 Mar 2021, at 19:12, Bruce Momjian wrote: Since this patch is de-facto rejected I'll mark it withdrawn in the CF app to save on cfbot bandwidth. > Do we support or document the ability to create a standby with checksums > from a primary without it, and is that a better approach? Michael

Re: [patch] [doc] Minor variable related cleanup and rewording of plpgsql docs

On Tue, Mar 9, 2021 at 10:45 AM Pavel Stehule wrote: > > > út 9. 3. 2021 v 18:03 odesílatel David Steele > napsal: > >> On 11/30/20 10:37 AM, Pavel Stehule wrote: >> > po 30. 11. 2020 v 16:06 odesílatel David G. Johnston >> > >> > ok >> This patch looks reasonable to me overall. >> >> A few

Re: partial heap only tuples

On 3/9/21, 8:24 AM, "Bruce Momjian" wrote: > On Mon, Feb 15, 2021 at 08:19:40PM +, Bossart, Nathan wrote: >> Yeah, this is something I'm concerned about. I think adding a bitmap >> of modified columns to the header of PHOT-updated tuples improves >> matters quite a bit, even for single-page

Re: Lowering the ever-growing heap->pd_lower

Matthias van de Meent writes: > The only two existing mechanisms that I could find (in the access/heap > directory) that possibly could fail on shrunken line pointer arrays; > being xlog recovery (I do not have enough knowledge on recovery to > determine if that may touch pages that have shrunken

Re: [PATCH] regexp_positions ( string text, pattern text, flags text ) → setof int4range[]

On Tue, Mar 9, 2021, at 17:42, Tom Lane wrote: > "Joel Jacobson" writes: > > Tom - can you please give details on your unpleasant experiences with > > parallel arrays? > > The problems I can recall running into were basically down to not having > an easy way to iterate through parallel arrays.

Re: Allow batched insert during cross-partition updates

Hi, thanks for the patch. I had a first look and played around with the code. The code seems clean, complete, and does what it says on the tin. I will need a bit more time to acclimatise with all the use cases for a more thorough review. I small question though is why expose

Re: Lowering the ever-growing heap->pd_lower

On Tue, Mar 9, 2021 at 7:13 AM Matthias van de Meent wrote: > The shrinking of the line pointer array is already common practice in > indexes (in which all LP_UNUSED items are removed), but this specific > implementation cannot be used for heap pages due to ItemId > invalidation. One available

Re: New IndexAM API controlling index vacuum strategies

On Mon, Mar 8, 2021 at 7:34 PM Peter Geoghegan wrote: > > One possible > > consequence that I'm concerned about is sequential scan performance. > > For an index scan, you just jump to the line pointer you want and then > > go get the tuple, but a sequential scan has to loop over all the line > >

Re: Lowering the ever-growing heap->pd_lower

On Tue, Mar 9, 2021 at 1:36 PM Tom Lane wrote: > > Matthias van de Meent writes: > > The only two existing mechanisms that I could find (in the access/heap > > directory) that possibly could fail on shrunken line pointer arrays; > > being xlog recovery (I do not have enough knowledge on recovery

Re: Procedures versus the "fastpath" API

On 3/9/21 2:15 PM, Tom Lane wrote: > So the question on the table is what to do about this. As far as > window functions go, it seems clear that fastpath.c should just reject > any attempt to call a window function that way (or an aggregate for > that matter; aggregates fail already, but with

Re: Lowering the ever-growing heap->pd_lower

> On Mar 9, 2021, at 1:35 PM, Tom Lane wrote: > > So, to accept a patch that shortens the line pointer array, what we need > to do is verify that every such code path checks for an out-of-range > offset before trying to fetch the target line pointer. I believed > back in 2007 that there

Re: Lowering the ever-growing heap->pd_lower

Mark Dilger writes: >> On Mar 9, 2021, at 1:35 PM, Tom Lane wrote: >> So, to accept a patch that shortens the line pointer array, what we need >> to do is verify that every such code path checks for an out-of-range >> offset before trying to fetch the target line pointer. > Much as Pavan asked

Re: pg_upgrade failing for 200+ million Large Objects

On Wed, Mar 03, 2021 at 11:36:26AM +, Tharakan, Robins wrote: > While reviewing a failed upgrade from Postgres v9.5 (to v9.6) I saw that the > instance had ~200 million (in-use) Large Objects. I was able to reproduce > this on a test instance which too fails with a similar error. If

Re: WIP: BRIN multi-range indexes

On Sun, Mar 7, 2021 at 8:53 PM Tomas Vondra wrote: [v20210308b] I managed to trap an assertion that somehow doesn't happen during the regression tests. The callers of fill_expanded_ranges() do math like this: /* both ranges and points are expanded into a separate element */ neranges =

Re: Lowering the ever-growing heap->pd_lower

On Tue, 9 Mar 2021 at 17:21, Mark Dilger wrote: > > For a prior discussion on this topic: > > https://www.postgresql.org/message-id/2e78013d0709130606l56539755wb9dbe17225ffe90a%40mail.gmail.com Thanks for the reference! I note that that thread mentions the old-style VACUUM FULL as a reason as to

Re: Lowering the ever-growing heap->pd_lower

Peter Geoghegan writes: > It occurs to me that we should also mark the hole in the middle of the > page (which includes the would-be LP_UNUSED line pointers at the end > of the original line pointer array space) as undefined to Valgrind > within PageRepairFragmentation(). +1

RE: Avoid CommandCounterIncrement in RI trigger when INSERT INTO referencing table

> Attaching the first version patch which avoid CCI in RI trigger when insert > into > referencing table. After some more on how to support parallel insert into fk relation. It seems we do not have a cheap way to implement this feature. Please see the explanation below: In RI_FKey_check,

Re: Columns correlation and adaptive query optimization

Hello Konstantin, Sorry for not responding to this thread earlier. I definitely agree the features proposed here are very interesting and useful, and I appreciate you kept rebasing the patch. I think the patch improving join estimates can be treated as separate, and I see it already has a

Re: Add some tests for pg_stat_statements compatibility verification under contrib

Hi Erica, On Wed, Mar 10, 2021 at 11:14:52AM +0800, Erica Zhang wrote: > Hi Julien, > Thanks a lot for the quick review. Please see my answer below in blue. > Attached is the new patch. Thanks! >> The upgrade scripts are already tested as postgres will install 1.4 and >> perform >> all

Re: Removing vacuum_cleanup_index_scale_factor

On Mon, Mar 8, 2021 at 10:21 PM Masahiko Sawada wrote: > Thank you for the patches. I looked at 0001 patch and have a comment: > > +* We don't report to the stats collector here because the stats collector > +* only tracks per-table stats. Reset the changes_since_analyze counter > +*

Re: a verbose option for autovacuum

On Tue, Mar 9, 2021 at 12:58 AM Euler Taveira wrote: > > On Mon, Mar 8, 2021, at 2:32 AM, Masahiko Sawada wrote: > > * Proposed idea > LOG: automatic vacuum of table "postgres.public.test": index scans: 1 > pages: 0 removed, 443 remain, 0 skipped due to pins, 0 skipped frozen > tuples: 1000

Re: Lowering the ever-growing heap->pd_lower

On Tue, Mar 9, 2021 at 1:54 PM Peter Geoghegan wrote: > It occurs to me that we should also mark the hole in the middle of the > page (which includes the would-be LP_UNUSED line pointers at the end > of the original line pointer array space) as undefined to Valgrind > within

Re: POC: GROUP BY optimization

Hi, I take a look at the patch today. Attached is the v12 and a separate patch with some comment tweaks and review comments. 1) I see the patch results in some plan changes in postgres_fdw. I assume it's somehow related to the sort costing changes, but I find it a bit suspicious. Why should the

Re: Huge memory consumption on partitioned table with FKs

Amit Langote writes: > On Fri, Mar 5, 2021 at 6:00 AM Tom Lane wrote: >> This claim seems false on its face: >>> All child constraints of a given foreign key constraint can use the >>> same RI query and the resulting plan, that is, no need to create as >>> many copies of the query and the plan

Re: WIP: BRIN multi-range indexes

On 3/9/21 9:51 PM, John Naylor wrote: > > On Sun, Mar 7, 2021 at 8:53 PM Tomas Vondra > mailto:tomas.von...@enterprisedb.com>> > wrote: > [v20210308b] > > I managed to trap an assertion that somehow doesn't happen during the > regression tests. The callers of fill_expanded_ranges() do math

Re: Optimising latch signals

On Tue, Mar 9, 2021 at 1:09 PM Thomas Munro wrote: > On Tue, Mar 9, 2021 at 12:20 PM Alvaro Herrera > wrote: > > Hi, I don't know if you realized but we have two new Illumos members > > now (haddock and hake), and they're both failing initdb on signalfd() > > problems. > I'll wait a short time

Re: [patch] [doc] Minor variable related cleanup and rewording of plpgsql docs

On 3/9/21 1:08 PM, Tom Lane wrote: David Steele writes: 1) PL/SQL seems to be used in a few places where I believe PL/pgSQL is meant. This was pre-existing but now seems like a good opportunity to fix it, unless I am misunderstanding. PL/SQL is Oracle's function language, which PL/pgSQL is

Re: Boundary value check in lazy_tid_reaped()

On Tue, Mar 9, 2021 at 9:57 AM Masahiko Sawada wrote: > > On Mon, Mar 8, 2021 at 7:16 PM Peter Eisentraut > wrote: > > > > On 21.01.21 14:11, Masahiko Sawada wrote: > > > Agreed. bsearch with bound check showed a reasonable improvement in my > > > evaluation in terms of performance. Regarding

Re: partial heap only tuples

On Tue, Mar 9, 2021 at 09:33:31PM +, Bossart, Nathan wrote: > I'm cautiously optimistic that index creation and deletion will not > require too much extra work. For example, if a new index needs to > point to a partial heap only tuple, it can do so (unlike HOT, which > would require that the

Re: [HACKERS] logical decoding of two-phase transactions

I ran a 5 cascaded setup of pub-subs on the latest patchset which starts pgbench on the first server and waits till the data on the fifth server matches the first. This is based on a test script created by Erik Rijkers. The tests run fine and the 5th server achieves data consistency in around a

Re: [PATCH] regexp_positions ( string text, pattern text, flags text ) → setof int4range[]

On Tue, Mar 9, 2021, at 09:29, Pavel Stehule wrote: > > https://www.postgresql.org/docs/current/plpgsql-control-structures.html#PLPGSQL-FOREACH-ARRAY > > but the SLICE argument should be constant. But this limit is artificial, just > for implementation simplicity. Important is behaviour. I

Re: [HACKERS] Custom compression methods

On Tue, Mar 9, 2021 at 1:56 AM Robert Haas wrote: > > With this design, we can support changing the compression method on a > column quite easily. It's just a hint, like the STORAGE parameter. It > has no bearing on what can be present in the table, but just controls > how new values are stored.

Re: Improvements and additions to COPY progress reporting

út 9. 3. 2021 v 6:34 odesílatel Michael Paquier napsal: > > On Mon, Mar 08, 2021 at 05:33:40PM +0100, Matthias van de Meent wrote: > > Seems reasonable. PFA updated patches. I've renamed the previous 0003 > > to 0002 to keep git-format-patch easy. > > Thanks for updating the patch. 0001 has been

  1   2   >