RE: [PoC] pg_upgrade: allow to upgrade publisher node

2023-04-13 Thread Hayato Kuroda (Fujitsu)
Dear Peter, Thank you for checking. Then we can wait comments from others. PSA modified version. > 1. > There were a couple of comments that I thought would appear less > squished (aka more readable) if there was a blank line preceding the > XXX. > > 1a. This one is in

Build farm breakage over time

2023-04-13 Thread Thomas Munro
Just for fun, I broke time up into 15 minute intervals and counted how many machines were showing red on HEAD at each sample point (lateral join for last tick interpolation of data I collect from the BF), and plotted that over time. See attached. I excluded seawasp (it tells us about *future*

Re: Unexpected (wrong?) result querying boolean partitioned table with NULL partition

2023-04-13 Thread David Rowley
On Thu, 13 Apr 2023 at 15:45, David Rowley wrote: > > On Thu, 13 Apr 2023 at 15:30, Richard Guo wrote: > > BTW, I wonder if we should elog an Error here. > > > > default: > > - Assert(false); /* hmm? */ > > - return PARTCLAUSE_UNSUPPORTED; > >

RE: pg_upgrade and logical replication

2023-04-13 Thread Hayato Kuroda (Fujitsu)
Dear Julien, > I didn't really look into it, mostly because I don't think it's a sensible > use case. Logical sync of a relation is a heavy and time consuming operation > that requires to retain the xmin for quite some time. This can already lead > to > some bad effect on the publisher, so

segfault tied to "IS JSON predicate" commit

2023-04-13 Thread Peter Geoghegan
I find that if I run the following test against a standard debug build on HEAD, my local installation reliably segfaults: $ meson test --setup running --suite test_rls_hooks-running Attached is a "bt full" run from gdb against a core dump. The query "EXPLAIN (costs off) SELECT * FROM

Re: Various typo fixes

2023-04-13 Thread Michael Paquier
On Tue, Apr 11, 2023 at 10:44:43PM -0500, Justin Pryzby wrote: > It could be "an inadequate wal-level" or "a prohibitively low > wal-level", but Thom's language is better. "too low a wal-level" means > the same thing as "too low of a wal-level" (which would also be fine). I have been studying

Re: User functions for building SCRAM secrets

2023-04-13 Thread Michael Paquier
On Fri, Apr 14, 2023 at 01:27:46AM +0200, Daniel Gustafsson wrote: > What would be the intended usecase? I don’t have the RFC handy, does > it say anything about salt length? Hmm. I thought it did, but RFC 5802 has only these two paragraphs: If the authentication information is stolen from

Re: Should we remove vacuum_defer_cleanup_age?

2023-04-13 Thread Laurenz Albe
On Thu, 2023-04-13 at 12:16 -0400, Jonathan S. Katz wrote: > On 4/13/23 11:32 AM, Jonathan S. Katz wrote: > > On 4/12/23 11:34 PM, Amit Kapila wrote: > > > On Tue, Apr 11, 2023 at 11:50 PM Andres Freund > > > > +1 to do one of the above. I think there is a good chance that > > > somebody might

Re: User functions for building SCRAM secrets

2023-04-13 Thread Daniel Gustafsson
> On 14 Apr 2023, at 01:14, Michael Paquier wrote: > > On Tue, Apr 11, 2023 at 11:27:17AM +0200, Magnus Hagander wrote: >> Having the function always generate a random salt seems more >> reasonable though, and would perhaps be something that helps in some >> of the cases? It won't help with the

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

2023-04-13 Thread Tom Lane
Daniel Gustafsson writes: > Good points, it should of course be SOCK_ERRNO. The attached saves off errno > and reinstates it to avoid clobbering. Will test it on Windows in the morning > as well. I think instead of this: +SOCK_ERRNO_SET(save_errno); you could just do

Re: User functions for building SCRAM secrets

2023-04-13 Thread Michael Paquier
On Tue, Apr 11, 2023 at 11:27:17AM +0200, Magnus Hagander wrote: > Having the function always generate a random salt seems more > reasonable though, and would perhaps be something that helps in some > of the cases? It won't help with the password policy one, as it's too > secure for that, but it

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

2023-04-13 Thread Daniel Gustafsson
> On 14 Apr 2023, at 00:52, Tom Lane wrote: > > Daniel Gustafsson writes: >> The attached diff passes the tests on OpenSSL 1.0.1 through 3.1 as well as on >> LibreSSL. Thoughts? > > 1. You can't assume that errno starts out zero, unless you zero it > right before SSL_connect. Maybe we should

Re: Wrong results from Parallel Hash Full Join

2023-04-13 Thread Thomas Munro
On Thu, Apr 13, 2023 at 12:31 PM Melanie Plageman wrote: > On Wed, Apr 12, 2023 at 6:50 PM Thomas Munro wrote: > > I think "Discussion:" footers are supposed to use > > https://postgr.es/m/XXX shortened URLs. > > Hmm. Is the problem with mine that I included "flat"? Because I did use >

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

2023-04-13 Thread Tom Lane
Daniel Gustafsson writes: > The attached diff passes the tests on OpenSSL 1.0.1 through 3.1 as well as on > LibreSSL. Thoughts? 1. You can't assume that errno starts out zero, unless you zero it right before SSL_connect. 2. I wonder whether it's safe to assume that errno (a/k/a SOCK_ERRNO)

RE: [PATCH] Support % wildcard in extension upgrade filenames

2023-04-13 Thread Regina Obe
> Here are my thoughts of how this can work to satisfy our specific needs and > that of others who have many micro versions. > > 1) We define an additional file. I'll call this a paths file > > So for example postgis would have a > > postgis.paths file > > The format of the path file would be

Re: Clean up hba.c of code freeing regexps

2023-04-13 Thread Michael Paquier
On Thu, Apr 13, 2023 at 11:58:51AM +0200, Alvaro Herrera wrote: > I agree with the downthread votes to clean this up now rather than > waiting. Also, you're adding exactly zero lines of new code, so I don't > think feature freeze affects the decision. Thanks, done that. The commit log mentions

Re: Direct I/O

2023-04-13 Thread Christoph Berg
Re: Thomas Munro > Linux/tmpfs: 1..0 # SKIP pre-flight test if we can open a file with > O_DIRECT failed: Invalid argument I confirm it's working now: t/004_io_direct.pl .. skipped: pre-flight test if we can open a file with O_DIRECT failed: Invalid argument All tests successful.

Re: Can we do something to help stop users mistakenly using force_parallel_mode?

2023-04-13 Thread David Rowley
On Wed, 12 Apr 2023 at 09:53, Tom Lane wrote: > I don't see a reason to wait longer once the buildfarm is on board. I did a final sweep of the latest runs for each animal this morning. Everything has been switched over to debug_parallel_query, so I've gone and pushed the patch to remove the

RE: [PATCH] Support % wildcard in extension upgrade filenames

2023-04-13 Thread Regina Obe
Here are my thoughts of how this can work to satisfy our specific needs and that of others who have many micro versions. 1) We define an additional file. I'll call this a paths file So for example postgis would have a postgis.paths file The format of the path file would be of the form , =>

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

2023-04-13 Thread Daniel Gustafsson
> On 13 Apr 2023, at 18:42, Daniel Gustafsson wrote: > Regarding the thread; I hope to have a suggestion for a way forward regarding > the open issue later tonight. After reading OpenSSL code and documentation, I think the simplest solution is to explicitly check for X509 errors when OpenSSL

Re: Backends stunk in wait event IPC/MessageQueueInternal

2023-04-13 Thread Thomas Munro
On Sun, Aug 28, 2022 at 11:03 AM Thomas Munro wrote: > On Sun, Jun 26, 2022 at 11:18 AM Thomas Munro wrote: > > On Tue, May 17, 2022 at 3:31 PM Thomas Munro wrote: > > > On Mon, May 16, 2022 at 3:45 PM Japin Li wrote: > > > > Maybe use the __illumos__ macro more accurity. > > > > > > > >

Fix documentation for max_wal_size and min_wal_size

2023-04-13 Thread sirisha chamarthi
Hi, The documentation [1] says max_wal_size and min_wal_size defaults are 1GB and 80 MB respectively. However, these are configured based on the wal_segment_size and documentation is not clear about it. Attached a patch to fix the documentation. [1]

Re: COPY TO STDOUT Apache Arrow support

2023-04-13 Thread Adam Lippai
Hi, There are two bigger developments in this topic: 1. Pandas 2.0 is released and it can use Apache Arrow as a backend 2. Apache Arrow ADBC is released which standardizes the client API. Currently it uses the postgresql wire protocol underneath Best regards, Adam Lippai On Thu, Apr

Re: Issue in postgres_fdw causing unnecessary wait for cancel request reply

2023-04-13 Thread Fujii Masao
On 2023/04/13 15:13, Etsuro Fujita wrote: I am not 100% sure that it is a good idea to use the same error message "could not send cancel request" for the PQgetCancel() and PQcancel() cases, because they are different functions. How about "could not create PGcancel structure” or something

Re: Issue in postgres_fdw causing unnecessary wait for cancel request reply

2023-04-13 Thread Fujii Masao
On 2023/04/13 11:00, Kyotaro Horiguchi wrote: Agreed, it seems to be a leftover when we moved to parse_int_param() in that area. It looks like there was an oversight in commit e7a2217978. I've attached a patch (0002) that updates PQconnectPoll() to use parse_int_param() for parsing the

Re: pg_replslotdata - a tool for displaying replication slot information

2023-04-13 Thread Gurjeet
On Mon Jan 17, 2022 at 5:11 AM PST, Julien Rouhaud wrote: > On Mon, Jan 17, 2022 at 04:10:13PM +0530, Bharath Rupireddy wrote: > > > > Thanks Juilen. I'm okay if the patch gets dropped. > > Ok, I will take care of that soon. I find this utility interesting and useful, especially for the reason

Re: Temporary tables versus wraparound... again

2023-04-13 Thread Peter Geoghegan
On Thu, Apr 13, 2023 at 9:45 AM Robert Haas wrote: > > On Wed, Apr 12, 2023 at 4:23 PM Greg Stark wrote: > > Am I crazy or is the parenthetical comment there exactly backwards? If > > the horizon is *more recent* then fewer tuples are *non*-removable. > > I.e. *more* tuples are removable, no? >

Re: Temporary tables versus wraparound... again

2023-04-13 Thread Robert Haas
On Wed, Apr 12, 2023 at 4:23 PM Greg Stark wrote: > I'm trying to wrap my head around GetOldestNonRemovableTransactionId() > and whether it's the right thing here. This comment is not helping me: > > /* > * Return the oldest XID for which deleted tuples must be preserved in the > * passed

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

2023-04-13 Thread Daniel Gustafsson
> On 13 Apr 2023, at 18:39, Tom Lane wrote: > > Daniel Gustafsson writes: >>> On 12 Apr 2023, at 22:23, Tom Lane wrote: >>> 2. Run a second BF animal that's intentionally pointed at the MacPorts >>> environment, in hopes of testing what MacPorts users would see. > >> I think #2 would be a

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

2023-04-13 Thread Tom Lane
Daniel Gustafsson writes: >> On 12 Apr 2023, at 22:23, Tom Lane wrote: >> 2. Run a second BF animal that's intentionally pointed at the MacPorts >> environment, in hopes of testing what MacPorts users would see. > I think #2 would be a good addition. Most won't build OpenSSL themselves so >

Re: Should we remove vacuum_defer_cleanup_age?

2023-04-13 Thread Jonathan S. Katz
On 4/13/23 11:32 AM, Jonathan S. Katz wrote: On 4/12/23 11:34 PM, Amit Kapila wrote: On Tue, Apr 11, 2023 at 11:50 PM Andres Freund +1 to do one of the above. I think there is a good chance that somebody might be doing more harm by using it so removing this shouldn't be a problem.

Re: pg_init_privs corruption.

2023-04-13 Thread Robert Haas
On Fri, Feb 17, 2023 at 3:38 PM Tom Lane wrote: > Floris Van Nee writes: > > This is as far as I can see the same case as what I reported a few years > > ago here: > > https://www.postgresql.org/message-id/flat/1574068566573.13088%40Optiver.com#488bd647ce6f5d2c92764673a7c58289 > > There was a

Re: Partial aggregates pushdown

2023-04-13 Thread Robert Haas
On Wed, Nov 30, 2022 at 3:12 AM Alexander Pyhalov wrote: > 1) In previous version of the patch aggregates, which had partialaggfn, > were ok to push down. And it was a definite sign that aggregate can be > pushed down. Now we allow pushing down an aggregate, which prorettype is > not internal and

Re: doc: add missing "id" attributes to extension packaging page

2023-04-13 Thread Karl O. Pinc
On Thu, 13 Apr 2023 16:01:35 +0200 Brar Piening wrote: > On 13.04.2023 at 10:31, Peter Eisentraut wrote: > > The first patch has been committed. > > Yay - thank you! > > > The second patch should be sent to pgsql-www for integrating into > > the web site. > Done via [1]. Thanks for the

Re: doc: add missing "id" attributes to extension packaging page

2023-04-13 Thread Karl O. Pinc
On Thu, 13 Apr 2023 15:58:03 +0100 Dagfinn Ilmari Mannsåker wrote: > Peter Eisentraut writes: > > The first patch has been committed. > Another side note: I notice the links don't appear on > elements (e.g. > https://www.postgresql.org/docs/devel/sql-select.html#SQL-WITH), only > . This we

Re: Should we remove vacuum_defer_cleanup_age?

2023-04-13 Thread Jonathan S. Katz
On 4/12/23 11:34 PM, Amit Kapila wrote: On Tue, Apr 11, 2023 at 11:50 PM Andres Freund wrote: On 2023-04-11 11:33:01 -0500, Justin Pryzby wrote: On Wed, Mar 22, 2023 at 10:00:48AM -0700, Andres Freund wrote: I don't know whether others think we should apply it this release, given the "late

Re: doc: add missing "id" attributes to extension packaging page

2023-04-13 Thread Dagfinn Ilmari Mannsåker
Peter Eisentraut writes: > On 06.04.23 16:19, Brar Piening wrote: >> 001-make_html_ids_discoverable_v5.postgresql.patch which needs to be >> applied to the postgresql repository. It adds the XSLT to generate the >> id links and the CSS to hide/display them. I've added comments as >> suggested

Re: doc: add missing "id" attributes to extension packaging page

2023-04-13 Thread Brar Piening
On 13.04.2023 at 10:31, Peter Eisentraut wrote: The first patch has been committed. Yay - thank you! The second patch should be sent to pgsql-www for integrating into the web site. Done via [1]. Thanks for the hint. Side project: I noticed that these new hover links don't appear in the

Re: Allowing parallel-safe initplans

2023-04-13 Thread Tom Lane
Richard Guo writes: > * For the diff in standard_planner, I was wondering why not move the > initPlans up to the Gather node, just as we did before. So I tried that > way but did not notice the breakage of regression tests as stated in the > comments. Would you please confirm that? Try it with

Re: PGBuildfarm member pollock Branch HEAD Failed at Stage Make

2023-04-13 Thread Stephen Frost
Greetings, * buildfarm-adm...@lists.postgresql.org (buildfarm-adm...@lists.postgresql.org) wrote: > The PGBuildfarm member pollock had the following event on branch HEAD: > Failed at Stage: Make > The snapshot timestamp for the build is: 2023-04-13 13:06:34 > The specs of this machine are: >

Re: longfin missing gssapi_ext.h

2023-04-13 Thread Stephen Frost
Greetings, * Jonathan S. Katz (jk...@postgresql.org) wrote: > On 4/12/23 12:22 PM, Stephen Frost wrote: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > > Stephen Frost writes: > > > > Updated patch set attached. > > > > > > LGTM > > > > Great, thanks. > > > > I cleaned up the commit messages a

Re: psql: Add role's membership options to the \du+ command

2023-04-13 Thread Pavel Luzanov
After playing with the \du command, I found that we can't avoid translation. All attributes are translatable. Also, two of nine attributes shows in new line separated format (connection limit and password valid until). $ LANGUAGE=fr psql -c "ALTER ROLE postgres CONNECTION LIMIT 3 VALID UNTIL

Re: Unexpected (wrong?) result querying boolean partitioned table with NULL partition

2023-04-13 Thread David Kimura
On Wed, Apr 12, 2023 at 4:13 AM David Rowley wrote: > On Wed, 12 Apr 2023 at 22:13, David Kimura wrote: > > Is it fair to assume that, given the same data, a partitioned table should > > return the same results as a non-partitioned table? > > Yes, and also the same as when

Re: Protecting allocator headers with Valgrind

2023-04-13 Thread David Rowley
On Wed, 12 Apr 2023 at 01:28, David Rowley wrote: > Any objections? It seems there are none. I'll have another look at the patch tomorrow with the aim to get it in. (Unless someone objects to me doing that before then) David

Re: Fix incorrect start up costs for WindowAgg paths (bug #17862)

2023-04-13 Thread David Rowley
On Thu, 13 Apr 2023 at 10:09, David Rowley wrote: > I also see I might need to do a bit more work on this as the following > is not handled correctly: > > select count(*) over(rows between unbounded preceding and 10 > following) from tenk1; > > it's assuming all rows due to lack of ORDER BY, but

Re: Bufmgr possible overflow

2023-04-13 Thread Ranier Vilela
Em qua., 12 de abr. de 2023 às 22:29, Kyotaro Horiguchi < horikyota@gmail.com> escreveu: > Perhaps it's a good idea to seprate the patch for each issue. > > Thanks Kyotaro for taking a look. > At Wed, 12 Apr 2023 09:36:14 -0300, Ranier Vilela > wrote in> IMO I think that commit 31966b1 > >

Re: Should we remove vacuum_defer_cleanup_age?

2023-04-13 Thread Alvaro Herrera
On 2023-Apr-11, Andres Freund wrote: > Updated patch attached. I think we should either apply something like that > patch, or at least add a to the docs. I gave this patch a look. The only code change is that ComputeXidHorizons() and GetSnapshotData() no longer handle the case where

RE: Partial aggregates pushdown

2023-04-13 Thread fujii.y...@df.mitsubishielectric.co.jp
Hi Mr.Momjian. > > There is one more thing I would like your opinion on. > > As the major version of PostgreSQL increase, it is possible that the > > new builtin aggregate functions are added to the newer PostgreSQL. > > This patch assume that aggpartialfns definitions exist in BKI files. > > Due

Re: [PATCH] Use role name "system_user" instead of "user" for unsafe_tests

2023-04-13 Thread Aleksander Alekseev
Hi, > On Wed, Apr 12, 2023 at 03:30:03PM +0300, Aleksander Alekseev wrote: > > Any objections if we remove the tests for "user"? > > Based on some rather-recent experience in this area with > COERCE_SQL_SYNTAX, the relationship between the SQL keywords and the > way they can handled internally

Re: pg_upgrade and logical replication

2023-04-13 Thread Julien Rouhaud
Hi, On Thu, Apr 13, 2023 at 12:42:05PM +1000, Peter Smith wrote: > Here are some review comments for patch v4-0001 (not the test code) Thanks! > > (There are some overlaps here with what Kuroda-san already posted > yesterday because we were looking at the same patch code. Also, a few > of my

Re: Clean up hba.c of code freeing regexps

2023-04-13 Thread Alvaro Herrera
On 2023-Apr-13, Michael Paquier wrote: > With db4f21e in place, there is no need to worry about explicitely > freeing any regular expressions that may have been compiled when > loading HBA or ident files because MemoryContextDelete() would be > able to take care of that now that these are

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-04-13 Thread Peter Smith
Hi Kuroda-san. I do not have any more review comments for the v5 patch, but here are a few remaining nitpick items. == General 1. There were a couple of comments that I thought would appear less squished (aka more readable) if there was a blank line preceding the XXX. 1a. This one is in

Re: Support logical replication of DDLs

2023-04-13 Thread Amit Kapila
On Wed, Apr 12, 2023 at 4:53 PM Amit Kapila wrote: > > > Few comments on 0001 > === > Some more comments on 0001 == 1. +/* + * Subroutine for CREATE TABLE/CREATE DOMAIN deparsing. + * + * Given a table OID or domain OID, obtain its constraints and append

Re: pg_upgrade and logical replication

2023-04-13 Thread Julien Rouhaud
On Thu, Apr 13, 2023 at 10:51:10AM +0800, Julien Rouhaud wrote: > > On Wed, Apr 12, 2023 at 09:48:15AM +, Hayato Kuroda (Fujitsu) wrote: > > > > 5. AlterSubscription > > > > ``` > > + supported_opts = SUBOPT_RELID | > > SUBOPT_STATE | SUBOPT_LSN; > > +

Re: doc: add missing "id" attributes to extension packaging page

2023-04-13 Thread Peter Eisentraut
On 06.04.23 16:19, Brar Piening wrote: 001-make_html_ids_discoverable_v5.postgresql.patch which needs to be applied to the postgresql repository. It adds the XSLT to generate the id links and the CSS to hide/display them. I've added comments as suggested above.

Re: Allowing parallel-safe initplans

2023-04-13 Thread Richard Guo
On Thu, Apr 13, 2023 at 12:43 AM Tom Lane wrote: > Pursuant to the discussion at [1], here's a patch that removes our > old restriction that a plan node having initPlans can't be marked > parallel-safe (dating to commit ab77a5a45). That was really a special > case of the fact that we couldn't

Re: Partial aggregates pushdown

2023-04-13 Thread Bruce Momjian
On Thu, Apr 13, 2023 at 02:12:44AM -0400, Bruce Momjian wrote: > > In the next version of this patch, > > we can pushdown partial aggregate for an user-defined aggregate function > > only > > when the function pass through this check. > > Understood. In summary, we don't do any version check

Re: Clean up hba.c of code freeing regexps

2023-04-13 Thread Drouvot, Bertrand
Hi, On 4/13/23 5:48 AM, Michael Paquier wrote: On Wed, Apr 12, 2023 at 10:25:42PM -0400, Tom Lane wrote: +1 for cleanup, if this is new code. It does us no good in the long run for v16 to handle this differently from both earlier and later versions. Okidoki. Let me know if anybody has an

Re: Issue in postgres_fdw causing unnecessary wait for cancel request reply

2023-04-13 Thread Etsuro Fujita
Hi Fujii-san, On Wed, Apr 12, 2023 at 3:36 AM Fujii Masao wrote: > However, if PQgetCancel() returned NULL and no cancel request was issued, > I found that postgres_fdw could still wait for the reply to > the cancel request, causing unnecessary wait time with a 30 second timeout. Good catch! >

Re: Partial aggregates pushdown

2023-04-13 Thread Bruce Momjian
On Mon, Apr 10, 2023 at 01:18:37AM +, fujii.y...@df.mitsubishielectric.co.jp wrote: > > Uh, we actually want the patch to implement partial aggregate pushdown for > > all > > builtin data types that can support it. Is that done? I think it is only > > extension > > aggregates, which we do

Re: Add ps display while waiting for wal in read_local_xlog_page_guts

2023-04-13 Thread Drouvot, Bertrand
Hi, On 4/13/23 4:29 AM, Tom Lane wrote: sirisha chamarthi writes: pg_create_logical_replication_slot can take longer than usual on a standby when there is no activity on the primary. We don't have enough information in the pg_stat_activity or process title to debug why this is taking so long.

Re: Add ps display while waiting for wal in read_local_xlog_page_guts

2023-04-13 Thread Drouvot, Bertrand
Hi, On 4/13/23 12:43 AM, sirisha chamarthi wrote: Hi, pg_create_logical_replication_slot can take longer than usual on a standby when there is no activity on the primary. We don't have enough information in the pg_stat_activity or process title to debug why this is taking so long. Attached