Re: Disallow quorum uncommitted (with synchronous standbys) txns in logical replication subscribers

2022-01-06 Thread SATYANARAYANA NARLAPURAM
On Thu, Jan 6, 2022 at 11:24 PM Jeff Davis wrote: > On Wed, 2022-01-05 at 23:59 -0800, SATYANARAYANA NARLAPURAM wrote: > > I would like to propose a GUC send_Wal_after_quorum_committed which > > when set to ON, walsenders corresponds to async standbys and logical > > replication workers wait

Re: preserve timestamps when installing headers

2022-01-06 Thread Peter Eisentraut
On 04.01.22 22:21, Tom Lane wrote: However, there's another problem with using INSTALL_DATA as a solution to this issue: why would you expect that to preserve timestamps? install-sh won't. I see that /usr/bin/install (which configure picks on my RHEL box) won't preserve them by default, but it

Re: row filtering for logical replication

2022-01-06 Thread Peter Smith
Below are some review comments for the v60 patches. V60-0001 Review Comments 1. src/backend/replication/pgoutput/pgoutput.c - pgoutput_row_filter unnecessary parens + /* + * NOTE: Multiple publication row filters have already been combined to a + * single exprstate (for

Re: Disallow quorum uncommitted (with synchronous standbys) txns in logical replication subscribers

2022-01-06 Thread Jeff Davis
On Wed, 2022-01-05 at 23:59 -0800, SATYANARAYANA NARLAPURAM wrote: > I would like to propose a GUC send_Wal_after_quorum_committed which > when set to ON, walsenders corresponds to async standbys and logical > replication workers wait until the LSN is quorum committed on the > primary before

Re: fix libpq comment

2022-01-06 Thread Michael Paquier
On Thu, Jan 06, 2022 at 09:33:07PM -0300, Euler Taveira wrote: > While checking the PQping code I noticed that pg_ctl does not rely on PQping > since commit f13ea95f9e4 (v10) so the attached patch removes a comment from > internal_ping(). Looking at the area, the rest looks fine. So, applied as

Re: Add jsonlog log_destination for JSON server logs

2022-01-06 Thread Michael Paquier
On Thu, Jan 06, 2022 at 06:28:26PM -0500, Andrew Dunstan wrote: > I have tested on an msys2 setup with your v8 patches and I am getting this: > > #   Failed test 'current_logfiles is sane' > #   at t/004_logrotate.pl line 96. > #   'stderr log/postgresql-2022-01-06_222419.log > #

Re: Refactoring of compression options in pg_basebackup

2022-01-06 Thread Michael Paquier
On Thu, Jan 06, 2022 at 09:27:19AM -0500, Robert Haas wrote: > Did you mean that -z would be a synonym for --compression-method=gzip? > It doesn't really make sense for -Z to be that, unless it's also > setting the compression level. Yes, I meant "-z", not "-Z", to be a synonym of

Re: row filtering for logical replication

2022-01-06 Thread Amit Kapila
On Fri, Jan 7, 2022 at 9:44 AM Amit Kapila wrote: > > On Thu, Jan 6, 2022 at 6:42 PM Euler Taveira wrote: > > > > IMO we shouldn't reuse ReorderBufferChangeType. For a long-term solution, > > it is > > fragile. ReorderBufferChangeType has values that do not matter for row > > filter > > and it

Re: Index-only scan for btree_gist turns bpchar to char

2022-01-06 Thread Japin Li
On Fri, 07 Jan 2022 at 03:21, Tom Lane wrote: > I looked at this and it does seem like it might work, as per attached > patch. The one thing that is troubling me is that the opclass is set > up to apply gbt_text_same, which is formally the Wrong Thing for bpchar, > because the equality

Re: Skipping logical replication transactions on subscriber side

2022-01-06 Thread Masahiko Sawada
On Fri, Jan 7, 2022 at 10:04 AM Masahiko Sawada wrote: > > On Wed, Jan 5, 2022 at 12:31 PM Amit Kapila wrote: > > > > On Mon, Dec 27, 2021 at 9:54 AM Masahiko Sawada > > wrote: > > > > > > On Thu, Dec 16, 2021 at 2:42 PM Masahiko Sawada > > > wrote: > > > > > > > > On Thu, Dec 16, 2021 at

Re: RFC: Logging plan of the running query

2022-01-06 Thread torikoshia
On 2021-11-26 12:39, torikoshia wrote: Since the patch could not be applied to the HEAD anymore, I also updated it. Updated the patch for fixing compiler warning about the format on windows. -- Regards, -- Atsushi Torikoshi NTT DATA CORPORATIONFrom b8367e22d7a9898e4b85627ba8c203be273fc22f

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-06 Thread Fujii Masao
On 2022/01/06 19:24, Simon Riggs wrote: On Thu, 30 Dec 2021 at 13:19, Maxim Orlov wrote: Your opinions are very much welcome! This is a review of the Int64 options patch, "v6-0001-Add-64-bit-GUCs-for-xids.patch" Do we really need to support both int32 and int64 options? Isn't it enough

RE: Support tab completion for upper character inputs in psql

2022-01-06 Thread tanghy.f...@fujitsu.com
On Friday, January 7, 2022 1:08 PM, Japin Li wrote: > +/* > + * pg_string_tolower - Fold a string to lower case if the string is not > quoted > + * and only contains ASCII characters. > + * For German/Turkish etc text, no change will be made. > + * > + * The returned value has to be freed. > +

Re: Map WAL segment files on PMEM as WAL buffers

2022-01-06 Thread Justin Pryzby
On Fri, Jan 07, 2022 at 12:50:01PM +0900, Takashi Menjo wrote: > > But in this case it really doesn't work :( > > > > running bootstrap script ... 2022-01-05 23:17:30.244 CST [12088] FATAL: > > file not on PMEM: path "pg_wal/00010001" > > Do you have a real PMEM device such as

Re: row filtering for logical replication

2022-01-06 Thread Peter Smith
On Wed, Jan 5, 2022 at 1:05 PM wangw.f...@fujitsu.com wrote: > > On Thu, Jan 4, 2022 at 00:54 PM Peter Smith wrote: > > Modified in v58 [1] as suggested > Thanks for updating the patches. > A few comments about v58-0001 and v58-0002. > > v58-0001 > 1. > How about modifying the following loop in

Re: Skipping logical replication transactions on subscriber side

2022-01-06 Thread Amit Kapila
On Fri, Jan 7, 2022 at 6:35 AM Masahiko Sawada wrote: > > On Wed, Jan 5, 2022 at 12:31 PM Amit Kapila wrote: > > > > On Mon, Dec 27, 2021 at 9:54 AM Masahiko Sawada > > wrote: > > > > > > On Thu, Dec 16, 2021 at 2:42 PM Masahiko Sawada > > > wrote: > > > > > > > > On Thu, Dec 16, 2021 at

Re: row filtering for logical replication

2022-01-06 Thread Amit Kapila
On Thu, Jan 6, 2022 at 6:42 PM Euler Taveira wrote: > > On Thu, Jan 6, 2022, at 1:18 AM, Amit Kapila wrote: > > On Thu, Jan 6, 2022 at 8:43 AM Peter Smith wrote: > > > > On Wed, Jan 5, 2022 at 9:52 PM Amit Kapila wrote: > > > > > ... > > > > > Another minor comment: > > > +static bool

Re: Support tab completion for upper character inputs in psql

2022-01-06 Thread Japin Li
On Fri, 07 Jan 2022 at 10:12, tanghy.f...@fujitsu.com wrote: > On Thursday, January 6, 2022 11:57 PM, Tom Lane wrote: >> >> Peter Eisentraut writes: >> > So the ordering of the suggested completions is different. I don't know >> > offhand how that ordering is determined. Perhaps it's

Re: Map WAL segment files on PMEM as WAL buffers

2022-01-06 Thread Takashi Menjo
Hi Justin, Thank you for your build test and comments. The v7 patchset attached to this email fixes the issues you reported. > The cfbot showed issues compiling on linux and windows. > http://cfbot.cputube.org/takashi-menjo.html > > https://cirrus-ci.com/task/6125740327436288 > [02:30:06.538]

Re: Deduplicate min restart_lsn calculation code

2022-01-06 Thread Bharath Rupireddy
On Thu, Jan 6, 2022 at 11:54 PM Alvaro Herrera wrote: > > On 2022-Jan-06, Bharath Rupireddy wrote: > > > Hi, > > > > It seems like the two functions ReplicationSlotsComputeRequiredLSN and > > ReplicationSlotsComputeLogicalRestartLSN more or less does the same > > thing which makes me optimize

Re: Bugs in pgoutput.c

2022-01-06 Thread Amit Kapila
On Fri, Jan 7, 2022 at 1:28 AM Tom Lane wrote: > > Amit Kapila writes: > > > + * *during* a callback if we do any syscache or table access in the > > + * callback. > > > As we don't take locks on tables, can invalidation events be accepted > > during table access? I could be missing something

Re: Add spin_delay() implementation for Arm in s_lock.h

2022-01-06 Thread Andres Freund
On 2022-01-06 22:23:38 -0500, Tom Lane wrote: > No; there's just one spinlock. I'm re-purposing the spinlock that > test_shm_mq uses to protect its setup operations (and thereafter > ignores). Oh, sorry, misread :( > AFAICS the N+1 shm_mq instances don't internally contain > spinlocks; they

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-06 Thread Alexander Korotkov
,On Thu, Jan 6, 2022 at 4:15 PM Finnerty, Jim wrote: > (Maxim) Re: -- If after upgrade page has no free space for special data, > tuples are >converted to "double xmax" format: xmin became virtual >FrozenTransactionId, xmax occupies the whole 64bit. >Page converted to new

Re: [PATCH] Add extra statistics to explain for Nested Loop

2022-01-06 Thread Lukas Fittl
On Sun, Nov 21, 2021 at 8:55 PM Justin Pryzby wrote: > I'm curious to hear what you and others think of the refactoring. > > It'd be nice if there's a good way to add a test case for verbose output > involving parallel workers, but the output is unstable ... > I've reviewed this patch, and it

Re: Add spin_delay() implementation for Arm in s_lock.h

2022-01-06 Thread Tom Lane
Andres Freund writes: > These separate shm_mq instances forward messages in a circle, > "leader"->worker_1->worker_2->...->"leader". So there isn't a single contended > spinlock, but a bunch of different spinlocks, each with at most two backends > accessing it? No; there's just one spinlock.

Re: Add spin_delay() implementation for Arm in s_lock.h

2022-01-06 Thread Andres Freund
Hi, On 2022-01-06 21:39:57 -0500, Tom Lane wrote: > Andres Freund writes: > > I wonder if this will show the full set of spinlock contention issues - > > isn't > > this only causing contention for one spinlock between two processes? > > I don't think so -- the point of using the "pipelined"

Re: Add spin_delay() implementation for Arm in s_lock.h

2022-01-06 Thread Tom Lane
Andres Freund writes: >> I landed on the idea of adding some intentional spinlock >> contention to src/test/modules/test_shm_mq, which is a prefab test >> framework for passing data among multiple worker processes. The >> attached quick-hack patch makes it grab and release a spinlock once >> per

Re: Add spin_delay() implementation for Arm in s_lock.h

2022-01-06 Thread Andres Freund
Hi, > I landed on the idea of adding some intentional spinlock > contention to src/test/modules/test_shm_mq, which is a prefab test > framework for passing data among multiple worker processes. The > attached quick-hack patch makes it grab and release a spinlock once > per passed message. I

Re: \dP and \dX use ::regclass without "pg_catalog."

2022-01-06 Thread Justin Pryzby
https://www.postgresql.org/message-id/flat/7ad8cd13-db5b-5cf6-8561-dccad1a934cb%40nttcom.co.jp https://www.postgresql.org/message-id/flat/20210827193151.GN26465%40telsasoft.com On Sat, Aug 28, 2021 at 08:57:32AM -0400, Álvaro Herrera wrote: > On 2021-Aug-27, Justin Pryzby wrote: > > I noticed

Re: Add index scan progress to pg_stat_progress_vacuum

2022-01-06 Thread Imseih (AWS), Sami
Thanks for the review. I am hesitant to make column name changes for obvious reasons, as it breaks existing tooling. However, I think there is a really good case to change "index_vacuum_count" as the name is confusing. "index_vacuum_cycles_completed" is the name I suggest if we agree to

RE: Support tab completion for upper character inputs in psql

2022-01-06 Thread tanghy.f...@fujitsu.com
On Thursday, January 6, 2022 11:57 PM, Tom Lane wrote: > > Peter Eisentraut writes: > > So the ordering of the suggested completions is different. I don't know > > offhand how that ordering is determined. Perhaps it's dependent on > > locale, readline version, or operating system. In any

Re: Add spin_delay() implementation for Arm in s_lock.h

2022-01-06 Thread Tom Lane
"Blake, Geoff" writes: > Hope everything is well going into the new year. I'd like to pick this > discussion back up and your thoughts on the patch with the data I posted 2 > weeks prior. Is there more data that would be helpful? Different setup? > Data on older versions of Postgresql to

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

2022-01-06 Thread Euler Taveira
On Thu, Jan 6, 2022, at 9:48 PM, Bossart, Nathan wrote: > After a quick glance, I didn't see an easy way to hold a session open > while the test does other things. If there isn't one, modifying > backup_fs_hot() to work with non-exclusive mode might be more trouble > than it is worth. You can use

Re: Skipping logical replication transactions on subscriber side

2022-01-06 Thread Masahiko Sawada
On Wed, Jan 5, 2022 at 12:31 PM Amit Kapila wrote: > > On Mon, Dec 27, 2021 at 9:54 AM Masahiko Sawada wrote: > > > > On Thu, Dec 16, 2021 at 2:42 PM Masahiko Sawada > > wrote: > > > > > > On Thu, Dec 16, 2021 at 2:21 PM Amit Kapila > > > wrote: > > > > > > > > On Thu, Dec 16, 2021 at 10:37

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

2022-01-06 Thread Bossart, Nathan
On 12/2/21, 1:34 PM, "Bossart, Nathan" wrote: > On 12/2/21, 9:50 AM, "David Steele" wrote: >> On 12/2/21 12:38, David Steele wrote: >>> On 12/2/21 11:00, Andrew Dunstan wrote: Should we really be getting rid of PostgreSQL::Test::Cluster::backup_fs_hot() ? >>> >>> Agreed, it would

fix libpq comment

2022-01-06 Thread Euler Taveira
Hi, While checking the PQping code I noticed that pg_ctl does not rely on PQping since commit f13ea95f9e4 (v10) so the attached patch removes a comment from internal_ping(). -- Euler Taveira EDB https://www.enterprisedb.com/ From 3055de48fe71f47df357114c7a42db05edcdb290 Mon Sep 17 00:00:00

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-01-06 Thread Peter Geoghegan
On Thu, Jan 6, 2022 at 2:45 PM Peter Geoghegan wrote: > But the "freeze early" heuristics work a bit like that anyway. We > won't freeze all the tuples on a whole heap page early if we won't > otherwise set the heap page to all-visible (not all-frozen) in the VM > anyway. I believe that

Re: Feature Proposal: Connection Pool Optimization - Change the Connection User

2022-01-06 Thread Todd Hubers
Hi Everyone, I have started working on this: - Benchmarking - increasingly more comprehensive benchmarking - Prototyping - to simulate the change of users (toggling back and forth) - Draft Implementation - of OPTION-1 (New Protocol Message) - (Then: Working with Odyssey and PgBouncer

Re: Add jsonlog log_destination for JSON server logs

2022-01-06 Thread Andrew Dunstan
On 1/6/22 13:06, Andrew Dunstan wrote: > On 1/5/22 02:32, Michael Paquier wrote: >> On Sun, Jan 02, 2022 at 01:34:45PM -0800, Andres Freund wrote: >>> The tests don't seem to pass on windows: >>> https://cirrus-ci.com/task/5412456754315264?logs=test_bin#L47 >>>

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-01-06 Thread Peter Geoghegan
On Thu, Jan 6, 2022 at 12:54 PM Robert Haas wrote: > On Fri, Dec 17, 2021 at 9:30 PM Peter Geoghegan wrote: > > Can we fully get rid of vacuum_freeze_table_age? Maybe even get rid of > > vacuum_freeze_min_age, too? Freezing tuples is a maintenance task for > > physical blocks, but we use logical

Re: row filtering for logical replication

2022-01-06 Thread Peter Smith
On Wed, Jan 5, 2022 at 1:05 PM wangw.f...@fujitsu.com wrote: > > On Thu, Jan 4, 2022 at 00:54 PM Peter Smith wrote: > > Modified in v58 [1] as suggested > Thanks for updating the patches. > A few comments about v58-0001 and v58-0002. > > v58-0001 > 1. > How about modifying the following loop in

Re: row filtering for logical replication

2022-01-06 Thread Peter Smith
On Wed, Jan 5, 2022 at 5:01 PM vignesh C wrote: > ... > 4) Should this be posted as a separate patch in a new thread, as it is > not part of row filtering: > --- a/src/include/parser/parse_node.h > +++ b/src/include/parser/parse_node.h > @@ -79,7 +79,7 @@ typedef enum ParseExprKind >

Re: row filtering for logical replication

2022-01-06 Thread Peter Smith
On Wed, Jan 5, 2022 at 4:34 PM Peter Smith wrote: > > I have reviewed again the source code for v58-0001. > > Below are my review comments. > > Actually, I intend to fix most of these myself for v59*, so this post > is just for records. > > v58-0001 Review Comments > > >

Re: row filtering for logical replication

2022-01-06 Thread Peter Smith
On Wed, Jan 5, 2022 at 4:26 PM Amit Kapila wrote: > > On Tue, Jan 4, 2022 at 12:15 PM Peter Smith wrote: > > > > On Fri, Dec 31, 2021 at 12:39 AM houzj.f...@fujitsu.com > > wrote: > > > > 3) v55-0002 > > > > +static bool pgoutput_row_filter_update_check(enum > > > > ReorderBufferChangeType

Re: row filtering for logical replication

2022-01-06 Thread Peter Smith
On Wed, Jan 5, 2022 at 4:56 PM Amit Kapila wrote: > ... > > > 4. > > > +#define IDX_PUBACTION_n 3 > > > + ExprState*exprstate[IDX_PUBACTION_n]; /* ExprState array for row > > > filter. > > > +One per publication action. */ > > > .. > > > .. > > > > > > I think we can have this define

Re: XMAX_LOCK_ONLY and XMAX_COMMITTED (fk/multixact code)

2022-01-06 Thread Bossart, Nathan
On 12/21/21, 11:42 AM, "Mark Dilger" wrote: > + /* pre-v9.3 lock-only bit pattern */ > + ereport(ERROR, > + (errcode(ERRCODE_DATA_CORRUPTED), > +errmsg_internal("found tuple with HEAP_XMAX_COMMITTED > and" > +

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Joel Jacobson
On Thu, Jan 6, 2022, at 21:38, Pavel Stehule wrote: > I say, semantically - how I understand the meaning of the word "in" is not > good to use it for generic alias of function arguments (when we have out > arguments too). Trying to imagine a situation where you would need a shorthand also for

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-01-06 Thread Robert Haas
On Fri, Dec 17, 2021 at 9:30 PM Peter Geoghegan wrote: > Can we fully get rid of vacuum_freeze_table_age? Maybe even get rid of > vacuum_freeze_min_age, too? Freezing tuples is a maintenance task for > physical blocks, but we use logical units (XIDs). I don't see how we can get rid of these. We

Re: psql: \dl+ to list large objects privileges

2022-01-06 Thread Pavel Luzanov
On 06.01.2022 21:13, Tom Lane wrote: I made the same changes to two files in the 'expected' directory: largeobject.out and largeobject_1.out. Although I must say that I still can't understand why more than one file with expected result is used for some tests. Because the output sometimes varies

Re: Make relfile tombstone files conditional on WAL level

2022-01-06 Thread Andres Freund
On 2022-01-06 08:52:01 -0500, Robert Haas wrote: > On Thu, Jan 6, 2022 at 3:47 AM Thomas Munro wrote: > > Another problem is that relfilenodes are normally allocated with > > GetNewOidWithIndex(), and initially match a relation's OID. We'd need > > a new allocator, and they won't be able to

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-06 Thread Peter Geoghegan
On Tue, Jan 4, 2022 at 9:40 PM Fujii Masao wrote: > Could you tell me what happens if new tuple with XID larger than xid_base + > 0x is inserted into the page? Such new tuple is not allowed to be > inserted into that page? I fear that this patch will have many bugs along these lines.

Re: Add index scan progress to pg_stat_progress_vacuum

2022-01-06 Thread Bossart, Nathan
On 12/29/21, 8:44 AM, "Imseih (AWS), Sami" wrote: > In "pg_stat_progress_vacuum", introduce 2 columns: > > * total_index_vacuum : This is the # of indexes that will be vacuumed. Keep > in mind that if failsafe mode kicks in mid-flight to the vacuum, Postgres may > choose to forgo index scans.

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Pavel Stehule
čt 6. 1. 2022 v 21:04 odesílatel Joel Jacobson napsal: > On Thu, Jan 6, 2022, at 20:24, Pavel Stehule wrote: > > But there is nothing similar in standard. > > Standard doesn't specify any column or table or label names in the > custom area. > > I think that's an unfair comparison. > This isn't

Re: Bugs in pgoutput.c

2022-01-06 Thread Robert Haas
On Thu, Jan 6, 2022 at 2:58 PM Tom Lane wrote: > That might be okay for the system catalog entries, but I don't see > how it prevents some other session from dropping the table entirely, > thereby causing the on-disk storage to go away. Is it guaranteed > that logical decoding will never try to

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Joel Jacobson
On Thu, Jan 6, 2022, at 20:24, Pavel Stehule wrote: > But there is nothing similar in standard. > Standard doesn't specify any column or table or label names in the custom > area. I think that's an unfair comparison. This isn't at all the same thing as dictating column or table names. I merely

Re: Bugs in pgoutput.c

2022-01-06 Thread Tom Lane
Amit Kapila writes: > On Thu, Jan 6, 2022 at 3:42 AM Tom Lane wrote: >> Also ... maybe I'm not looking in the right place, but I do not >> see anything anywhere in logical decoding that is taking any lock >> on the relation being processed. How can that be safe? > We don't need to acquire a

Re: Collecting statistics about contents of JSONB columns

2022-01-06 Thread Tomas Vondra
On 1/5/22 21:22, Simon Riggs wrote: On Fri, 31 Dec 2021 at 22:07, Tomas Vondra wrote: The patch does something far more elegant - it simply uses stavalues to store an array of JSONB documents, each describing stats for one path extracted from the sampled documents. Sounds good. I'm

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Pavel Stehule
čt 6. 1. 2022 v 20:03 odesílatel Joel Jacobson napsal: > On Thu, Jan 6, 2022, at 19:03, Pavel Stehule wrote: > > The possibility to define a label dynamically is a better solution (not > by some buildin keyword), > > because it allows some possibility for the end user to define what he >

Re: Collecting statistics about contents of JSONB columns

2022-01-06 Thread Tomas Vondra
On 1/1/22 22:16, Zhihong Yu wrote: Hi, +static JsonPathStats +jsonStatsFindPathStats(JsonStats jsdata, char *path, int pathlen) Stats appears twice in the method name. I think findJsonPathStats() should suffice. It should check `if (jsdata->nullfrac >= 1.0)` as jsonStatsGetPathStatsStr does.

Re: Index-only scan for btree_gist turns bpchar to char

2022-01-06 Thread Tom Lane
Japin Li writes: > On Thu, 06 Jan 2022 at 00:34, Tom Lane wrote: >> The minimum-effort fix would be to apply rtrim1 to both strings >> in gbt_bpchar_consistent, but I wonder if we can improve on that >> by pushing the ignore-trailing-spaces behavior further down. >> I didn't look yet at whether

Re: Collecting statistics about contents of JSONB columns

2022-01-06 Thread Tomas Vondra
On 1/1/22 16:33, Zhihong Yu wrote: Hi, For patch 1: +   List       *statisticsName = NIL;   /* optional stats estimat. procedure */ I think if the variable is named estimatorName (or something similar), it would be easier for people to grasp its purpose. I agree "statisticsName" might

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Joel Jacobson
On Thu, Jan 6, 2022, at 19:03, Pavel Stehule wrote: > The possibility to define a label dynamically is a better solution (not by > some buildin keyword), > because it allows some possibility for the end user to define what he prefers. I'm trying to understand why you think a user-defined

Storage for multiple variable-length attributes in a single row

2022-01-06 Thread Esteban Zimanyi
Dear all When ingesting mobility (IoT) data into MobilityDB https://mobilitydb.com/ we transform very wide (2K attributes) car mobility data of high frequence (every tenth of a second) from flat format (e.g. CSV) into MobilityDB format in which there is a single record per trip and each of the

Re: SQL:2011 application time

2022-01-06 Thread Corey Huinker
> > > But > the standard says that dropping system versioning should automatically > drop all historical records (2 under Part 2: Foundation, 11.30 system versioning clause>). That actually makes sense though: when you > do DML we automatically update the start/end columns, but we don't > save

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Pavel Stehule
> > > > I accept that both issues are valid, and I don't think we can find a good > design that solves both issues, because there are different objective > expectations. > I accept that both issues are valid, and I don't think we can find a **one** good design that solves both issues, because

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Pavel Stehule
čt 6. 1. 2022 v 18:18 odesílatel Joel Jacobson napsal: > On Thu, Jan 6, 2022, at 17:55, Tom Lane wrote: > > Even if it works today, we could be letting ourselves in for future > > trouble. The SQL standard is a moving target, and they could easily > > standardize some future syntax involving IN

Re: Deduplicate min restart_lsn calculation code

2022-01-06 Thread Alvaro Herrera
On 2022-Jan-06, Bharath Rupireddy wrote: > Hi, > > It seems like the two functions ReplicationSlotsComputeRequiredLSN and > ReplicationSlotsComputeLogicalRestartLSN more or less does the same > thing which makes me optimize (saving 40 LOC) it as attached. I'm > pretty much okay if it gets

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2022-01-06 Thread Peter Geoghegan
On Tue, Dec 21, 2021 at 11:57 PM Masahiko Sawada wrote: > I've looked at the patch and here are comments: Thanks! The patch bitrot again, so attached is a rebased version, v3. > I think we should set message_level. Otherwise, index AM will set an > invalid log level, although any index AM in

Re: psql: \dl+ to list large objects privileges

2022-01-06 Thread Tom Lane
Pavel Luzanov writes: >> I wonder if we shouldn't put these new tests in largeobject.sql instead. >> (That would mean there are two expected-files to change, which is a bit >> tedious, but it's not very hard.) > I made the same changes to two files in the 'expected' directory: > largeobject.out

Re: SQL:2011 application time

2022-01-06 Thread Paul A Jungwirth
On Thu, Jan 6, 2022 at 6:45 AM Vik Fearing wrote: > > On 1/5/22 11:03 PM, Corey Huinker wrote: > > > > There was similar work being done for system periods, which are a bit > > simpler but require a side (history) table to be created. > > This is false. SYSTEM_TIME periods do not need any kind

Re: Add jsonlog log_destination for JSON server logs

2022-01-06 Thread Andrew Dunstan
On 1/5/22 02:32, Michael Paquier wrote: > On Sun, Jan 02, 2022 at 01:34:45PM -0800, Andres Freund wrote: >> The tests don't seem to pass on windows: >> https://cirrus-ci.com/task/5412456754315264?logs=test_bin#L47 >>

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Pavel Stehule
čt 6. 1. 2022 v 17:48 odesílatel Joel Jacobson napsal: > On Thu, Jan 6, 2022, at 17:10, Pavel Stehule wrote: > > I understand well, and I don't think it's nice. > > > > Are there some similar features in other programming languages? > > It would be similar to "this" in Javascript/Java/C++, > but

Re: Suggestion: optionally return default value instead of error on failed cast

2022-01-06 Thread Corey Huinker
On Thu, Jan 6, 2022 at 12:18 PM Andrew Dunstan wrote: > > On 1/4/22 22:17, Corey Huinker wrote: > > > > currently a failed cast throws an error. It would be useful to have a > > way to get a default value instead. > > > > > > I've recently encountered situations where this would have

Re: Emit "checkpoint skipped because system is idle" message at LOG level if log_checkpoints is set

2022-01-06 Thread Justin Pryzby
On Wed, Jan 05, 2022 at 10:45:06AM +0530, Dilip Kumar wrote: > +1 to convert to LOG when log_checkpoints is set. On Thu, Jan 06, 2022 at 04:34:38PM +0900, Kyotaro Horiguchi wrote: > Agreed. -1 to just raising elevel of the message. On Thu, Jan 06, 2022 at 06:58:14PM +0800, Julien Rouhaud wrote:

Re: Suggestion: optionally return default value instead of error on failed cast

2022-01-06 Thread Andrew Dunstan
On 1/4/22 22:17, Corey Huinker wrote: > > currently a failed cast throws an error. It would be useful to have a > way to get a default value instead. > > > I've recently encountered situations where this would have been > helpful. Recently I came across some client code: > > CREATE

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Joel Jacobson
On Thu, Jan 6, 2022, at 17:55, Tom Lane wrote: > Even if it works today, we could be letting ourselves in for future > trouble. The SQL standard is a moving target, and they could easily > standardize some future syntax involving IN that creates a problem here. Perhaps the "in." notation could

Re: terminate called after throwing an instance of 'std::bad_alloc' (llvmjit)

2022-01-06 Thread Justin Pryzby
There's no leak after running for ~5 weeks. $ ps -O lstart,vsize,rss 17930 PID STARTEDVSZ RSS S TTY TIME COMMAND 17930 Tue Nov 30 15:35:26 2021 1019464 117424 S ? 7-04:54:03 postgres: telsasoft ts 192.168.122.13(57640) idle Unless you suggest otherwise ,

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Tom Lane
Pavel Stehule writes: > čt 6. 1. 2022 v 16:59 odesílatel Joel Jacobson napsal: >> How could the SQL parser have a problem with it, if "in" is currently >> never followed by "." (dot)? > you can check it. It is true, so IN is usually followed by "(", but until > check I am not able to say if

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Joel Jacobson
On Thu, Jan 6, 2022, at 17:10, Pavel Stehule wrote: > I understand well, and I don't think it's nice. > > Are there some similar features in other programming languages? It would be similar to "this" in Javascript/Java/C++, but instead using "in" to access function parameters. Currently, we

Deduplicate min restart_lsn calculation code

2022-01-06 Thread Bharath Rupireddy
Hi, It seems like the two functions ReplicationSlotsComputeRequiredLSN and ReplicationSlotsComputeLogicalRestartLSN more or less does the same thing which makes me optimize (saving 40 LOC) it as attached. I'm pretty much okay if it gets rejected on the grounds that it creates a lot of diff with

Re: SQL/JSON: functions

2022-01-06 Thread Andrew Dunstan
On 1/6/22 06:24, Himanshu Upadhyaya wrote: > I have one general question on the below scenario. > CREATE TABLE T (Id INTEGER PRIMARY KEY,Jcol CHARACTER VARYING ( 5000 > )CHECK ( Jcol IS JSON ) ); > insert into T values (1,323); > ORACLE is giving an error(check constraint...violated ORA-06512)

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-06 Thread Finnerty, Jim
Re:Most software has a one-stage upgrade model. What you propose would have us install 2 things, with a step in-between, which makes it harder to manage. The intended benefit would be that the code doesn't need to handle the possibility of 2 different XID representations for the

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Pavel Stehule
čt 6. 1. 2022 v 16:59 odesílatel Joel Jacobson napsal: > On Thu, Jan 6, 2022, at 15:05, Pavel Stehule wrote: > >>čt 6. 1. 2022 v 14:28 odesílatel Joel Jacobson > napsal: > >>How about using the existing reserved keyword "in" followed by "." (dot) > and then the function parameter name? > >> >

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Joel Jacobson
On Thu, Jan 6, 2022, at 15:05, Pavel Stehule wrote: >>čt 6. 1. 2022 v 14:28 odesílatel Joel Jacobson napsal: >>How about using the existing reserved keyword "in" followed by "." (dot) and >>then the function parameter name? >> >>This idea is based on the assumption "in." would always be a syntax

Re: [EXTERNAL] Re: PQcancel does not use tcp_user_timeout, connect_timeout and keepalive settings

2022-01-06 Thread Jelte Fennema
Attached are 3 patches that address the feedback from Andres about code duplication and splitting up commits. I completely removed internal_cancel now, since it only had one caller at this point. > IMO, this is a new feature not a bug fix IMO this is definitely a bugfix. Nowhere in the libpq

Re: sqlsmith: ERROR: XX000: bogus varno: 2

2022-01-06 Thread Tom Lane
Amit Langote writes: > Thanks for addressing that in the patch you posted. I guess fixing > only expression_tree_walker/mutator() suffices for now, but curious to > know if it was intentional that you decided not to touch the following > sites: > exprCollation(): it would perhaps make sense to

Re: Support tab completion for upper character inputs in psql

2022-01-06 Thread Tom Lane
Peter Eisentraut writes: > So the ordering of the suggested completions is different. I don't know > offhand how that ordering is determined. Perhaps it's dependent on > locale, readline version, or operating system. In any case, we need to > figure this out to make this test stable. I

Re: ICU for global collation

2022-01-06 Thread Julien Rouhaud
On Thu, Jan 06, 2022 at 01:55:55PM +, Finnerty, Jim wrote: > > I didn't notice anything version-specific about the patch. Would any > modifications be needed to backport it to pg13 and pg14? This is a new feature so it can't be backported. The changes aren't big and mostly touches places

Re: SQL:2011 application time

2022-01-06 Thread Vik Fearing
On 1/5/22 11:03 PM, Corey Huinker wrote: > > There was similar work being done for system periods, which are a bit > simpler but require a side (history) table to be created. This is false. SYSTEM_TIME periods do not need any kind of history. This was one of the problems I had with Surafel's

Re: sqlsmith: ERROR: XX000: bogus varno: 2

2022-01-06 Thread Robert Haas
On Mon, Dec 20, 2021 at 4:20 PM Tom Lane wrote: > pg_get_expr doesn't (or didn't) depend on expression_tree_walker, > so there wasn't a problem there before. I am worried that there > might be other code paths, now or in future, that could try to apply > expression_tree_walker/mutator to

Re: Refactoring of compression options in pg_basebackup

2022-01-06 Thread Robert Haas
On Thu, Jan 6, 2022 at 12:04 AM Michael Paquier wrote: > Yeah. There are cases for both. I just got to wonder whether it > makes sense to allow both server-side and client-side compression to > be used at the same time. That would be a rather strange case, but > well, with the correct set of

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-06 Thread Simon Riggs
On Thu, 6 Jan 2022 at 13:15, Finnerty, Jim wrote: > > (Maxim) Re: -- If after upgrade page has no free space for special data, > tuples are >converted to "double xmax" format: xmin became virtual >FrozenTransactionId, xmax occupies the whole 64bit. >Page converted to new

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Pavel Stehule
čt 6. 1. 2022 v 14:28 odesílatel Joel Jacobson napsal: > On Thu, Jan 6, 2022, at 10:05, Julien Rouhaud wrote: > > I agree, but on the other hand I don't see how defining a top level block > > alias identical for every single plpgsql function would really make > sense. > > Not every function has

Re: ICU for global collation

2022-01-06 Thread Finnerty, Jim
I didn't notice anything version-specific about the patch. Would any modifications be needed to backport it to pg13 and pg14? After this patch goes in, the big next thing would be to support nondeterministic collations for LIKE, ILIKE and pattern matching operators in general. Is anyone

Re: Make relfile tombstone files conditional on WAL level

2022-01-06 Thread Robert Haas
On Thu, Jan 6, 2022 at 3:47 AM Thomas Munro wrote: > Another problem is that relfilenodes are normally allocated with > GetNewOidWithIndex(), and initially match a relation's OID. We'd need > a new allocator, and they won't be able to match the OID in general > (while we have 32 bit OIDs at

Re: pl/pgsql feature request: shorthand for argument and local variable references

2022-01-06 Thread Joel Jacobson
On Thu, Jan 6, 2022, at 10:05, Julien Rouhaud wrote: > I agree, but on the other hand I don't see how defining a top level block > alias identical for every single plpgsql function would really make sense. > Not every function has a very long name and would benefit from it, and no one > can really

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-06 Thread Finnerty, Jim
(Maxim) Re: -- If after upgrade page has no free space for special data, tuples are converted to "double xmax" format: xmin became virtual FrozenTransactionId, xmax occupies the whole 64bit. Page converted to new format when vacuum frees enough space. A better way would be

Re: row filtering for logical replication

2022-01-06 Thread Euler Taveira
On Thu, Jan 6, 2022, at 1:18 AM, Amit Kapila wrote: > On Thu, Jan 6, 2022 at 8:43 AM Peter Smith wrote: > > > > On Wed, Jan 5, 2022 at 9:52 PM Amit Kapila wrote: > > > > > ... > > > > > Another minor comment: > > > +static bool pgoutput_row_filter(enum ReorderBufferChangeType changetype, > > > >

Re: sqlsmith: ERROR: XX000: bogus varno: 2

2022-01-06 Thread Amit Langote
On Thu, Jan 6, 2022 at 3:43 PM Amit Langote wrote: > On Tue, Dec 21, 2021 at 6:20 AM Tom Lane wrote: > > Robert Haas writes: > > > OK ... but my point is that dump and restore does work. So whatever > > > cases pg_get_expr() doesn't work must be cases that aren't needed for > > > that to

Re: a misbehavior of partition row movement (?)

2022-01-06 Thread Alvaro Herrera
On 2022-Jan-06, Amit Langote wrote: > On Thu, Jan 6, 2022 at 7:27 AM Alvaro Herrera wrote: > > I have pushed it thinking that we would not backpatch any of this fix. > > However, after running the tests and realizing that I didn't need an > > initdb for either patch, I wonder if maybe the whole

  1   2   >