Re: [doc] remove reference to pg_dump pre-8.1 switch behaviour

2020-11-30 Thread Michael Paquier
On Mon, Nov 30, 2020 at 03:46:19PM -0500, Tom Lane wrote: > Michael Paquier writes: > > So this comes down to 5 items, as per the attached. Thoughts? > > These items look fine to me, except this bit seems a bit awkward: > > + Note that the delayed indexing technique used for GIN > + (se

Re: autovac issue with large number of tables

2020-11-30 Thread Fujii Masao
On 2020/12/01 16:23, Masahiko Sawada wrote: On Tue, Dec 1, 2020 at 1:48 PM Kasahara Tatsuhito wrote: Hi, On Mon, Nov 30, 2020 at 8:59 PM Fujii Masao wrote: On 2020/11/30 10:43, Masahiko Sawada wrote: On Sun, Nov 29, 2020 at 10:34 PM Kasahara Tatsuhito wrote: Hi, Thanks for you com

Re: [HACKERS] logical decoding of two-phase transactions

2020-11-30 Thread Amit Kapila
On Mon, Nov 30, 2020 at 7:17 PM Amit Kapila wrote: > > On Mon, Nov 30, 2020 at 2:36 PM Ajin Cherian wrote: > > Sure, but you can see in your example above it got skipped due to > start_decoding_at not due to DecodingContextReady. So, the problem as > mentioned by me previously was how we distingu

Re: autovac issue with large number of tables

2020-11-30 Thread Masahiko Sawada
On Tue, Dec 1, 2020 at 1:48 PM Kasahara Tatsuhito wrote: > > Hi, > > On Mon, Nov 30, 2020 at 8:59 PM Fujii Masao > wrote: > > > > > > > > On 2020/11/30 10:43, Masahiko Sawada wrote: > > > On Sun, Nov 29, 2020 at 10:34 PM Kasahara Tatsuhito > > > wrote: > > >> > > >> Hi, Thanks for you comments.

Re: PG vs LLVM 12 on seawasp, next round

2020-11-30 Thread Andres Freund
Hi, On 2020-12-01 17:35:49 +1300, Thomas Munro wrote: > Since seawasp's bleeding edge LLVM installation moved to "trunk > 20201114 c8f4e06b 12.0.0" ~16 days ago, it has been red. Further > updates didn't help it and it's now on "trunk 20201127 6ee22ca6 > 12.0.0". I wonder if there is something i

Re: TAP test utility module 'PG_LSN.pm'

2020-11-30 Thread Michael Paquier
On Tue, Dec 01, 2020 at 03:14:06PM +0900, Fujii Masao wrote: > You mean the same function as the commit 9bae7e4cde provided? Completely forgot about this one, thanks. /me hides -- Michael signature.asc Description: PGP signature

Re: TAP test utility module 'PG_LSN.pm'

2020-11-30 Thread Fujii Masao
On 2020/12/01 14:58, Michael Paquier wrote: On Tue, Dec 01, 2020 at 12:03:41PM +0800, Craig Ringer wrote: I'd like to share the attached PG_LSN.pm module that I use when writing TAP tests. I suggest that it be considered for inclusion in core. It defines a Perl datatype PG_LSN with operator

Re: TAP test utility module 'PG_LSN.pm'

2020-11-30 Thread Craig Ringer
On Tue, 1 Dec 2020 at 13:58, Michael Paquier wrote: > > On Tue, Dec 01, 2020 at 12:03:41PM +0800, Craig Ringer wrote: > > I'd like to share the attached PG_LSN.pm module that I use when > > writing TAP tests. I suggest that it be considered for inclusion in > > core. > > > > It defines a Perl data

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-11-30 Thread Michael Paquier
On Mon, Nov 30, 2020 at 11:43:08PM -0600, Justin Pryzby wrote: > I eyeballed the patch and rebased the rest of the series on top if it to play > with. Looks fine - thanks. Cool, thanks. > FYI, the commit messages have the proper "author" for attribution. I proposed > and implemented the grammar

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-11-30 Thread Dilip Kumar
On Mon, Nov 30, 2020 at 6:49 PM Amit Kapila wrote: > > On Mon, Nov 30, 2020 at 3:14 AM Noah Misch wrote: > > > > On Mon, Sep 07, 2020 at 12:00:41PM +0530, Amit Kapila wrote: > > > Thanks, I have pushed the last patch. Let's wait for a day or so to > > > see the buildfarm reports > > > > https://b

Re: BUG #16663: DROP INDEX did not free up disk space: idle connection hold file marked as deleted

2020-11-30 Thread Michael Paquier
On Tue, Dec 01, 2020 at 04:06:48PM +1300, Thomas Munro wrote: > I felt on balance it was a "bug", since it causes operational > difficulties for people and was clearly not our intended behaviour, > and I announced this intention 6 weeks ago. Oops, sorry for missing this discussion for such a long

Re: Improving spin-lock implementation on ARM.

2020-11-30 Thread Tom Lane
Alexander Korotkov writes: > 2) None of the patches considered in this thread give a clear > advantage for PostgreSQL built with LSE. Yeah, I think so. > To further confirm this let's wait for Kunpeng 920 tests by Krunal > Bauskar and Amit Khandekar. Also it would be nice if someone will run >

Re: TAP test utility module 'PG_LSN.pm'

2020-11-30 Thread Michael Paquier
On Tue, Dec 01, 2020 at 12:03:41PM +0800, Craig Ringer wrote: > I'd like to share the attached PG_LSN.pm module that I use when > writing TAP tests. I suggest that it be considered for inclusion in > core. > > It defines a Perl datatype PG_LSN with operator support, so you can > write things like

RE: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-11-30 Thread Hou, Zhijie
Hi I look into the patch again and have some comments. 1. + Size oid_cmp_len = sizeof(Oid) * ind1->ncolumns; + + return ind1->ncolumns == ind2->ncolumns && + ind1->unique == ind2->unique && + memcmp(ind1->indexkeys, ind2->indexkeys, sizeof(int) * ind1->nc

Re: proposal: unescape_text function

2020-11-30 Thread Pavel Stehule
po 30. 11. 2020 v 22:56 odesílatel Pavel Stehule napsal: > > > po 30. 11. 2020 v 22:15 odesílatel Pavel Stehule > napsal: > >> >> >> po 30. 11. 2020 v 14:14 odesílatel Peter Eisentraut < >> peter.eisentr...@enterprisedb.com> napsal: >> >>> On 2020-11-29 18:36, Pavel Stehule wrote: >>> > >>> >

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-11-30 Thread Justin Pryzby
On Tue, Dec 01, 2020 at 11:46:55AM +0900, Michael Paquier wrote: > On Mon, Nov 30, 2020 at 05:12:42PM +0300, Alexey Kondratov wrote: > > Thanks. I have rebased the remaining patches on top of 873ea9ee to use > > 'utility_option_list' instead of 'common_option_list'. > > Thanks, that helps a lot.

RE: [POC] Fast COPY FROM command for the table with foreign partitions

2020-11-30 Thread tsunakawa.ta...@fujitsu.com
From: Amit Langote > Andrey's original patch had the flag to, as I understand it, make the > partitioning case work correctly. When inserting into a > non-partitioned table, there's only one relation to care about. In > that case, CopyFrom() can use either the new COPY interface or the > INSERT

Re: scram-sha-256 broken with FIPS and OpenSSL 1.0.2

2020-11-30 Thread Michael Paquier
On Mon, Nov 30, 2020 at 02:29:29PM +0100, Daniel Gustafsson wrote: > Yeah, that's along the lines of what I was thinking of. Hmm. I have looked at that, and thought first about having directly a reference to the resowner directly in pg_cryptohash_ctx, but that's not a good plan for two reasons: -

Re: Add statistics to pg_stat_wal view for wal related parameter tuning

2020-11-30 Thread Fujii Masao
On 2020/11/26 16:07, Masahiro Ikeda wrote: On 2020-11-25 20:19, Fujii Masao wrote: On 2020/11/19 16:31, Masahiro Ikeda wrote: On 2020-11-17 11:46, Fujii Masao wrote: On 2020/11/16 16:35, Masahiro Ikeda wrote: On 2020-11-12 14:58, Fujii Masao wrote: On 2020/11/06 10:25, Masahiro Ikeda wrote

Re: autovac issue with large number of tables

2020-11-30 Thread Kasahara Tatsuhito
Hi, On Mon, Nov 30, 2020 at 8:59 PM Fujii Masao wrote: > > > > On 2020/11/30 10:43, Masahiko Sawada wrote: > > On Sun, Nov 29, 2020 at 10:34 PM Kasahara Tatsuhito > > wrote: > >> > >> Hi, Thanks for you comments. > >> > >> On Fri, Nov 27, 2020 at 9:51 PM Fujii Masao > >> wrote: > >>> > >>> > >

PG vs LLVM 12 on seawasp, next round

2020-11-30 Thread Thomas Munro
Hi Since seawasp's bleeding edge LLVM installation moved to "trunk 20201114 c8f4e06b 12.0.0" ~16 days ago, it has been red. Further updates didn't help it and it's now on "trunk 20201127 6ee22ca6 12.0.0". I wonder if there is something in Fabien's scripting that needs to be tweaked, perhaps a sy

TAP test utility module 'PG_LSN.pm'

2020-11-30 Thread Craig Ringer
Hi all I'd like to share the attached PG_LSN.pm module that I use when writing TAP tests. I suggest that it be considered for inclusion in core. It defines a Perl datatype PG_LSN with operator support, so you can write things like cmp_ok($got_lsn, "<", $expected_lsn, "testname") in TAP test

Re: Printing backtrace of postgres processes

2020-11-30 Thread Tom Lane
Andres Freund writes: > It should be quite doable to emit such backtraces directly to stderr, > instead of using appendStringInfoString()/elog(). No, please no. (1) On lots of logging setups (think syslog), anything that goes to stderr is just going to wind up in the bit bucket. I realize that

Re: [HACKERS] logical decoding of two-phase transactions

2020-11-30 Thread Amit Kapila
On Tue, Dec 1, 2020 at 7:55 AM Ajin Cherian wrote: > > On Tue, Dec 1, 2020 at 12:46 AM Amit Kapila wrote: > > > > > Sure, but you can see in your example above it got skipped due to > > start_decoding_at not due to DecodingContextReady. So, the problem as > > mentioned by me previously was how we

Re: runtime error copying oids field

2020-11-30 Thread Zhihong Yu
Alvaro, et al: Please let me know how to proceed with the patch. Running test suite with the patch showed no regression. Cheers On Mon, Nov 30, 2020 at 3:24 PM Zhihong Yu wrote: > Hi, > See attached patch which is along the line Alvaro outlined. > > Cheers > > On Mon, Nov 30, 2020 at 3:01 PM A

Confusing behavior of psql's \e

2020-11-30 Thread Laurenz Albe
If you quit the editor without saving, the current query buffer or the last executed SQL statement get run. This can be annoying and disruptive, and it requires you to empty the file and save it if you don't want to execute anything. But when editing a script, it is a clear POLA violation: test=

Re: Printing backtrace of postgres processes

2020-11-30 Thread Craig Ringer
On Tue, 1 Dec 2020 at 11:31, Andres Freund wrote: > > Hi, > > On 2020-11-30 13:35:46 +0800, Craig Ringer wrote: > > I find that when I most often want a backtrace of a running, live > > backend, it's because the backend is doing something that isn't > > passing a CHECK_FOR_INTERRUPTS() so it's not

Re: Printing backtrace of postgres processes

2020-11-30 Thread Andres Freund
Hi, On 2020-11-30 13:35:46 +0800, Craig Ringer wrote: > I find that when I most often want a backtrace of a running, live > backend, it's because the backend is doing something that isn't > passing a CHECK_FOR_INTERRUPTS() so it's not responding to signals. So > it wouldn't help if a backend is wa

Re: Printing backtrace of postgres processes

2020-11-30 Thread Andres Freund
Hi, On 2020-11-22 01:25:08 -0500, Tom Lane wrote: > Surely this is *utterly* unsafe. You can't do that sort of stuff in > a signal handler. That's of course true for the current implementation - but I don't think it's a fundamental constraint. With a bit of care backtrace() and backtrace_symbols

Re: Improving spin-lock implementation on ARM.

2020-11-30 Thread Krunal Bauskar
On Tue, 1 Dec 2020 at 02:31, Alexander Korotkov wrote: > On Mon, Nov 30, 2020 at 7:00 AM Krunal Bauskar > wrote: > > 3. Problem with GCC approach is still a lot of distro don't support gcc > 9.4 as default. > > To use this approach: > > * PGSQL will have to roll out its packages using gc

Re: Huge memory consumption on partitioned table with FKs

2020-11-30 Thread Corey Huinker
On Mon, Nov 30, 2020 at 9:48 PM Tom Lane wrote: > Corey Huinker writes: > > Given that we're already looking at these checks, I was wondering if this > > might be the time to consider implementing these checks by directly > > scanning the constraint index. > > Yeah, maybe. Certainly ri_triggers

Re: Printing backtrace of postgres processes

2020-11-30 Thread Craig Ringer
On Tue, 1 Dec 2020 at 07:04, Tom Lane wrote: > I'd feel better about it if the mechanism had you specify exactly > one target process, and were restricted to a superuser requestor. Er, rather. I actually assumed the former was the case already, not having looked closely yet.

Re: BUG #16663: DROP INDEX did not free up disk space: idle connection hold file marked as deleted

2020-11-30 Thread Thomas Munro
On Tue, Dec 1, 2020 at 3:55 PM Michael Paquier wrote: > On Mon, Nov 30, 2020 at 06:59:40PM +1300, Thomas Munro wrote: > > So I fixed that, by adding a return value to do_truncate() and > > checking it. That's the version I plan to commit tomorrow, unless > > there are further comments or objectio

Re: BUG #16663: DROP INDEX did not free up disk space: idle connection hold file marked as deleted

2020-11-30 Thread Michael Paquier
On Mon, Nov 30, 2020 at 06:59:40PM +1300, Thomas Munro wrote: > So I fixed that, by adding a return value to do_truncate() and > checking it. That's the version I plan to commit tomorrow, unless > there are further comments or objections. I've also attached a > version suitable for REL_11_STABLE

Re: BUG #16663: DROP INDEX did not free up disk space: idle connection hold file marked as deleted

2020-11-30 Thread Thomas Munro
On Mon, Nov 30, 2020 at 6:59 PM Thomas Munro wrote: > On Wed, Nov 25, 2020 at 8:00 AM Pavel Borisov wrote: > > The new status of this patch is: Ready for Committer > ... That's the version I plan to commit tomorrow, unless > there are further comments or objections. ... Done, and back-patched

Re: Huge memory consumption on partitioned table with FKs

2020-11-30 Thread Tom Lane
Corey Huinker writes: > Given that we're already looking at these checks, I was wondering if this > might be the time to consider implementing these checks by directly > scanning the constraint index. Yeah, maybe. Certainly ri_triggers is putting a huge amount of effort into working around the S

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-11-30 Thread Michael Paquier
On Mon, Nov 30, 2020 at 05:12:42PM +0300, Alexey Kondratov wrote: > Thanks. I have rebased the remaining patches on top of 873ea9ee to use > 'utility_option_list' instead of 'common_option_list'. Thanks, that helps a lot. I have gone through 0002, and tweaked it as the attached (note that this pa

RE: [Patch] Optimize dropping of relation buffers using dlist

2020-11-30 Thread tsunakawa.ta...@fujitsu.com
From: Kyotaro Horiguchi > We are relying on the "fact" that the first lseek() call of a > (startup) process tells the truth. We added an assertion so that we > make sure that the cached value won't be cleared during recovery. A > possible remaining danger would be closing of an smgr object of a

Re: Huge memory consumption on partitioned table with FKs

2020-11-30 Thread Corey Huinker
> > I think this can be solved easily in the patch, by having > ri_BuildQueryKey() compare the parent's fk_attnums to the parent; if > they are equal then use the parent's constaint_id, otherwise use the > child constraint. That way, the cache entry is reused in the common > case where they are id

回复: 回复: [PATCH] BUG FIX: Core dump could happen when VACUUM FULL in standalone mode

2020-11-30 Thread Yulin PEI
I think you are right after reading code in compute_parallel_vacuum_workers() :) 发件人: Tom Lane 发送时间: 2020年12月1日 2:54 收件人: Yulin PEI 抄送: Masahiko Sawada ; pgsql-hackers@lists.postgresql.org 主题: Re: 回复: [PATCH] BUG FIX: Core dump could happen when VACUUM FULL in

Re: [HACKERS] logical decoding of two-phase transactions

2020-11-30 Thread Ajin Cherian
On Tue, Dec 1, 2020 at 12:46 AM Amit Kapila wrote: > So what caused it to skip due to start_decoding_at? Because the commit > where the snapshot became consistent is after Prepare. Does it happen > due to the below code in SnapBuildFindSnapshot() where we bump > start_decoding_at. > > { > ... > i

Re: Fix generate_useful_gather_paths for parallel unsafe pathkeys

2020-11-30 Thread James Coleman
On Sun, Nov 29, 2020 at 4:20 PM Tomas Vondra wrote: > > On 11/29/20 3:26 PM, James Coleman wrote: > > On Mon, Nov 23, 2020 at 8:35 AM James Coleman wrote: > >> > >> On Sun, Nov 22, 2020 at 4:59 PM Tomas Vondra > >> wrote: > >>> ... > >>> Thanks for the patches, I'll take a closer look next week.

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-30 Thread Masahiko Sawada
On Fri, Nov 20, 2020 at 8:47 PM Simon Riggs wrote: > > On Fri, 20 Nov 2020 at 10:15, Simon Riggs wrote: > > > > On Fri, 20 Nov 2020 at 01:40, Masahiko Sawada wrote: > > > > > > On Thu, Nov 19, 2020 at 8:02 PM Simon Riggs wrote: > > > > > > > > On Wed, 18 Nov 2020 at 17:59, Robert Haas wrote: >

RE: Disable WAL logging to speed up data loading

2020-11-30 Thread tsunakawa.ta...@fujitsu.com
From: Kyotaro Horiguchi > We're emitting only redo logs. So I think theoretically we don't need > anything other than the shutdown checkpoint record because we don't > perform recovery and checkpoint record is required at startup. > > RM_XLOG_ID: > XLOG_FPI_FOR_HINT - not needed? > XLOG_FPI

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2020-11-30 Thread Justin Pryzby
On Tue, Nov 03, 2020 at 08:56:06PM -0300, Alvaro Herrera wrote: > Here's an updated version of this patch. > > Apart from rebasing to current master, I made the following changes: > > * On the first transaction (the one that marks the partition as > detached), the partition is locked with ShareLo

Re: Why does create_gather_merge_plan need make_sort?

2020-11-30 Thread James Coleman
On Sun, Nov 29, 2020 at 4:10 PM Tomas Vondra wrote: > > > > On 11/29/20 3:44 PM, James Coleman wrote: > > On Mon, Nov 23, 2020 at 8:19 AM James Coleman wrote: > >> > >> .. > >> > >> So I'm +1 on turning it into an ERROR log instead, since it seems to > >> me that encountering this case would almo

Re: remove spurious CREATE INDEX CONCURRENTLY wait

2020-11-30 Thread Michael Paquier
On Mon, Nov 30, 2020 at 04:15:27PM -0300, Alvaro Herrera wrote: > In the interest of showing progress, I'm going to mark this CF item as > committed, and I'll submit the remaining pieces in a new thread. Thanks for splitting! -- Michael signature.asc Description: PGP signature

Re: Huge memory consumption on partitioned table with FKs

2020-11-30 Thread Kyotaro Horiguchi
At Mon, 30 Nov 2020 21:03:45 -0300, Alvaro Herrera wrote in > On 2020-Nov-26, Kyotaro Horiguchi wrote: > > > This shares RI_ConstraintInfo cache by constraints that shares the > > same parent constraints. But you forgot that the cache contains some > > members that can differ among partitions.

Re: vac_update_datfrozenxid will raise "wrong tuple length" if pg_database tuple contains toast attribute.

2020-11-30 Thread Ashwin Agrawal
On Sun, Nov 29, 2020 at 5:27 PM Michael Paquier wrote: > > One thing that strikes me as unwise is that we could run into similar > > problems with vac_update_relstats() in the future, and there have been > > recent talks about having more toast tables like for pg_class. That > > is not worth car

Re: [DOC] Document concurrent index builds waiting on each other

2020-11-30 Thread James Coleman
On Mon, Nov 30, 2020 at 4:53 PM Alvaro Herrera wrote: > > On 2020-Sep-30, Michael Paquier wrote: > > > + > > + CREATE INDEX (including the > > CONCURRENTLY > > + option) commands are included when VACUUM calculates > > what > > + dead tuples are safe to remove even on tables other than t

Re: error_severity of brin work item

2020-11-30 Thread Justin Pryzby
On Mon, Nov 30, 2020 at 08:47:32PM -0300, Alvaro Herrera wrote: > The more I look at this, the less I like it. This would set a precedent > that any action that can be initiated from an autovac work-item has a > requirement of silently being discarded when it referenced a > non-existant relation.

Re: enable_incremental_sort changes query behavior

2020-11-30 Thread James Coleman
On Tue, Nov 3, 2020 at 4:39 PM Tomas Vondra wrote: > I've pushed the 0001 part, i.e. the main fix. Not sure about the other > parts (comments and moving the code back to postgres_fdw) yet. I noticed the CF entry [1] got moved to the next CF; I'm thinking this entry should be marked as committed s

Re: Huge memory consumption on partitioned table with FKs

2020-11-30 Thread Alvaro Herrera
On 2020-Nov-26, Kyotaro Horiguchi wrote: > This shares RI_ConstraintInfo cache by constraints that shares the > same parent constraints. But you forgot that the cache contains some > members that can differ among partitions. > > Consider the case of attaching a partition that have experienced a >

Consider parallel for lateral subqueries with limit

2020-11-30 Thread James Coleman
I've been investigating parallelizing certain correlated subqueries, and during that work stumbled across the fact that set_rel_consider_parallel disallows parallel query on what seems like a fairly simple case. Consider this query: select t.unique1 from tenk1 t join lateral (select t.unique1 fro

Re: error_severity of brin work item

2020-11-30 Thread Alvaro Herrera
The more I look at this, the less I like it. This would set a precedent that any action that can be initiated from an autovac work-item has a requirement of silently being discarded when it referenced a non-existant relation. I'd rather have the code that drops the index go through the list of wo

Re: Online verification of checksums

2020-11-30 Thread David Steele
On 11/30/20 9:27 AM, Stephen Frost wrote: Greetings, * Michael Paquier (mich...@paquier.xyz) wrote: On Fri, Nov 27, 2020 at 11:15:27AM -0500, Stephen Frost wrote: * Magnus Hagander (mag...@hagander.net) wrote: On Thu, Nov 26, 2020 at 8:42 AM Michael Paquier wrote: But here the checksum is b

Re: Add Information during standby recovery conflicts

2020-11-30 Thread Masahiko Sawada
On Tue, Dec 1, 2020 at 3:25 AM Alvaro Herrera wrote: > > On 2020-Dec-01, Fujii Masao wrote: > > > + if (proc) > > + { > > + if (nprocs == 0) > > + appendStringInfo(&buf, "%d", > > proc->pid); >

Re: runtime error copying oids field

2020-11-30 Thread Zhihong Yu
Hi, See attached patch which is along the line Alvaro outlined. Cheers On Mon, Nov 30, 2020 at 3:01 PM Alvaro Herrera wrote: > On 2020-Nov-30, Zhihong Yu wrote: > > > This was the line runtime error was raised: > > > > memcpy(part_oids, partdesc->oids, sizeof(Oid) * nparts); > > > > Fro

Re: 回复: [PATCH] BUG FIX: Core dump could happen when VACUUM FULL in standalone mode

2020-11-30 Thread Masahiko Sawada
On Tue, Dec 1, 2020 at 3:54 AM Tom Lane wrote: > > Yulin PEI writes: > > Yes, I agree because (IsNormalProcessingMode() ) means that current process > > is not in bootstrap mode and postmaster process will not build index. > > So my new modified patch is attached. > > This is a good catch, but t

Re: Printing backtrace of postgres processes

2020-11-30 Thread Tom Lane
Craig Ringer writes: >> I would like to see some discussion of the security implications >> of such a feature, as well. ("There aren't any" is the wrong >> answer.) > If the stack only goes to the log, I actually don't think there are > significant security implications beyond those we already h

Re: runtime error copying oids field

2020-11-30 Thread Alvaro Herrera
On 2020-Nov-30, Zhihong Yu wrote: > This was the line runtime error was raised: > > memcpy(part_oids, partdesc->oids, sizeof(Oid) * nparts); > > From RelationBuildPartitionDesc we can see that: > > if (nparts > 0) > { > PartitionBoundInfo boundinfo; >

Re: [BUG] orphaned function

2020-11-30 Thread Tom Lane
"Drouvot, Bertrand" writes: > here is a scenario that produces an orphaned function (means it does not > belong to any namespace): > [ drop schema before committing function creation ] Hm. Historically we've not been too fussed about preventing such race conditions, and I wonder just how far is

runtime error copying oids field

2020-11-30 Thread Zhihong Yu
Hi, In our testPgRegressTrigger test log, I saw the following (this was for a relatively old version of PG): 197859 [ts-1] ../../../../../../src/postgres/src/backend/commands/indexcmds.c:1062:22: runtime error: null pointer passed as argument 2, which is declared to never be null 197860 [ts-1]

Re: proposal: unescape_text function

2020-11-30 Thread Pavel Stehule
po 30. 11. 2020 v 22:15 odesílatel Pavel Stehule napsal: > > > po 30. 11. 2020 v 14:14 odesílatel Peter Eisentraut < > peter.eisentr...@enterprisedb.com> napsal: > >> On 2020-11-29 18:36, Pavel Stehule wrote: >> > >> > I don't really get the point of this function. There is AFAICT no >> >

Re: [DOC] Document concurrent index builds waiting on each other

2020-11-30 Thread Alvaro Herrera
On 2020-Sep-30, Michael Paquier wrote: > + > + CREATE INDEX (including the > CONCURRENTLY > + option) commands are included when VACUUM calculates > what > + dead tuples are safe to remove even on tables other than the one being > indexed. > + > FWIW, this is true as well for REINDEX

Re: support IncrementalSortPath type in outNode()

2020-11-30 Thread Tom Lane
"Hou, Zhijie" writes: > I found IncrementalSortPath type is not analyzed in outNode. Indeed, that's supposed to be supported. Pushed, thanks for the patch! regards, tom lane

Re: proposal: unescape_text function

2020-11-30 Thread Pavel Stehule
po 30. 11. 2020 v 14:14 odesílatel Peter Eisentraut < peter.eisentr...@enterprisedb.com> napsal: > On 2020-11-29 18:36, Pavel Stehule wrote: > > > > I don't really get the point of this function. There is AFAICT no > > function to produce this escaped format, and it's not a recognized > >

Re: Printing LSN made easy

2020-11-30 Thread Andres Freund
Hi, On 2020-11-29 12:10:21 -0500, Tom Lane wrote: > Agreed. snprintf.c is meant to implement a recognized standard > (ok, %m is a GNU extension, but it's still pretty standard). > I'm not on board with putting PG-only extensions in there. I wonder if there's something we could layer on the elog.

Re: Improving spin-lock implementation on ARM.

2020-11-30 Thread Alexander Korotkov
On Mon, Nov 30, 2020 at 7:00 AM Krunal Bauskar wrote: > 3. Problem with GCC approach is still a lot of distro don't support gcc 9.4 > as default. > To use this approach: > * PGSQL will have to roll out its packages using gcc-9.4+ only so that > they are compatible with all aarch64 machin

Re: Improving spin-lock implementation on ARM.

2020-11-30 Thread Alexander Korotkov
On Mon, Nov 30, 2020 at 9:21 PM Tom Lane wrote: > Alexander Korotkov writes: > > I tend to think that LSE is enabled by default in Apple's clang based > > on your previous message[1]. In order to dispel the doubts could you > > please provide assembly of SpinLockAcquire for following clang > > o

Re: [doc] remove reference to pg_dump pre-8.1 switch behaviour

2020-11-30 Thread Tom Lane
Michael Paquier writes: > So this comes down to 5 items, as per the attached. Thoughts? These items look fine to me, except this bit seems a bit awkward: + Note that the delayed indexing technique used for GIN + (see for details) makes this advice + less necessary, but for very lar

Re: range_agg

2020-11-30 Thread Alexander Korotkov
On Mon, Nov 30, 2020 at 10:35 PM Alexander Korotkov wrote: > On Sun, Nov 29, 2020 at 11:53 PM Paul A Jungwirth > wrote: > > > > On Sun, Nov 29, 2020 at 11:43 AM Alexander Korotkov > > wrote: > > > Thank you. Could you please, update doc/src/sgml/catalogs.sgml, > > > because pg_type and pg_range

Re: Change JOIN tutorial to focus more on explicit joins

2020-11-30 Thread David G. Johnston
On Mon, Nov 30, 2020 at 1:15 PM Jürgen Purtz wrote: > On 30.11.20 20:45, Anastasia Lubennikova wrote: > > As far as I see something got committed and now the discussion is stuck > in arguing about parenthesis. > > FWIW, I think it is a matter of personal taste. Maybe we can compromise > on simply

Re: [patch] [doc] Introduce view updating options more succinctly

2020-11-30 Thread Anastasia Lubennikova
The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: not tested Spec compliant: not tested Documentation:tested, passed I wonder, why this patch didn't get a review during the CF. This minor im

Re: Change JOIN tutorial to focus more on explicit joins

2020-11-30 Thread Jürgen Purtz
On 30.11.20 20:45, Anastasia Lubennikova wrote: As far as I see something got committed and now the discussion is stuck in arguing about parenthesis. FWIW, I think it is a matter of personal taste. Maybe we can compromise on simply leaving this part unchanged. With or without parenthesis is a

Re: [DOC] Document concurrent index builds waiting on each other

2020-11-30 Thread Alvaro Herrera
On 2020-Nov-30, Anastasia Lubennikova wrote: > The commitfest is nearing the end and I wonder what is this discussion > waiting for. > It looks like the proposed patch received its fair share of review, so > I mark it as ReadyForCommitter and lay responsibility for the final > decision on them.

Re: [DOC] Document concurrent index builds waiting on each other

2020-11-30 Thread Anastasia Lubennikova
Status update for a commitfest entry. The commitfest is nearing the end and I wonder what is this discussion waiting for. It looks like the proposed patch received its fair share of review, so I mark it as ReadyForCommitter and lay responsibility for the final decision on them. The new status o

{CREATE INDEX, REINDEX} CONCURRENTLY improvements

2020-11-30 Thread Alvaro Herrera
Hello, In a previous thread [1], we added smarts so that processes running CREATE INDEX CONCURRENTLY would not wait for each other. One is adding the same to REINDEX CONCURRENTLY. I've attached patch 0002 here which does that. Why 0002, you ask? That's because preparatory patch 0001 simplifies

Re: Change JOIN tutorial to focus more on explicit joins

2020-11-30 Thread Anastasia Lubennikova
Status update for a commitfest entry. The commitfest is nearing the end and this thread was inactive for a while. As far as I see something got committed and now the discussion is stuck in arguing about parenthesis. FWIW, I think it is a matter of personal taste. Maybe we can compromise on simp

Re: range_agg

2020-11-30 Thread Alexander Korotkov
On Sun, Nov 29, 2020 at 11:53 PM Paul A Jungwirth wrote: > > On Sun, Nov 29, 2020 at 11:43 AM Alexander Korotkov > wrote: > > Thank you. Could you please, update doc/src/sgml/catalogs.sgml, > > because pg_type and pg_range catalogs are updated. > > Attached! :-) You're quick, thank you. Please

Re: remove spurious CREATE INDEX CONCURRENTLY wait

2020-11-30 Thread Alvaro Herrera
In the interest of showing progress, I'm going to mark this CF item as committed, and I'll submit the remaining pieces in a new thread. Thanks!

Re: Add docs stub for recovery.conf

2020-11-30 Thread David G. Johnston
On Mon, Nov 30, 2020 at 11:42 AM Bruce Momjian wrote: > > The downside is you end up with X-1 dummy sections just to allow for > references to old syntax, and you then have to find them all and remove > them when you implement the proper solution. I have no intention of > applying such an X-1 fi

Re: [PATCH] remove deprecated v8.2 containment operators

2020-11-30 Thread Tom Lane
Justin Pryzby writes: > I think this is waiting on me to provide a patch for the contrib/ modules with > update script removing the SQL operators, and documentating their deprecation. Right. We can remove the SQL operators, but not (yet) the C code support. I'm not sure that the docs change wou

Re: [PATCH] remove deprecated v8.2 containment operators

2020-11-30 Thread Justin Pryzby
On Mon, Nov 30, 2020 at 09:51:12PM +0300, Anastasia Lubennikova wrote: > On 16.11.2020 23:55, Justin Pryzby wrote: > > On Fri, Nov 13, 2020 at 10:03:43AM -0500, Stephen Frost wrote: > > > * Magnus Hagander (mag...@hagander.net) wrote: > > > > On Thu, Nov 12, 2020 at 11:28 PM Tom Lane wrote: > > >

Re: [PATCH] remove deprecated v8.2 containment operators

2020-11-30 Thread Tom Lane
Anastasia Lubennikova writes: > Status update for a commitfest entry. > The commitfest is nearing the end and this thread is "Waiting on > Author". As far as I see we don't have a patch here and discussion is a > bit stuck. > So, I am planning to return it with feedback. Any objections? AFAICS

Re: 回复: [PATCH] BUG FIX: Core dump could happen when VACUUM FULL in standalone mode

2020-11-30 Thread Tom Lane
Yulin PEI writes: > Yes, I agree because (IsNormalProcessingMode() ) means that current process > is not in bootstrap mode and postmaster process will not build index. > So my new modified patch is attached. This is a good catch, but the proposed fix still seems pretty random and unlike how it's

Re: [PATCH] remove deprecated v8.2 containment operators

2020-11-30 Thread Anastasia Lubennikova
On 16.11.2020 23:55, Justin Pryzby wrote: On Fri, Nov 13, 2020 at 10:03:43AM -0500, Stephen Frost wrote: * Magnus Hagander (mag...@hagander.net) wrote: On Thu, Nov 12, 2020 at 11:28 PM Tom Lane wrote: The changes to the contrib modules appear to be incomplete in some ways. In cube, hstore,

Re: Add docs stub for recovery.conf

2020-11-30 Thread Bruce Momjian
On Mon, Nov 30, 2020 at 11:31:35AM -0700, David G. Johnston wrote: > On Mon, Nov 30, 2020 at 11:25 AM Bruce Momjian wrote: > > On Mon, Nov 30, 2020 at 10:11:04AM +0800, Craig Ringer wrote: > > Can we please just address this docs issue? If you don't like my > solution > can > > y

Re: Improve handling of parameter differences in physical replication

2020-11-30 Thread Peter Eisentraut
On 2020-11-20 16:47, Sergei Kornilov wrote: Hmm... Good question. How about putting CheckForStandbyTrigger() in a wait loop, but reporting FATAL with an appropriate message, such as "promotion is not possible because of insufficient parameter settings"? Also it suits me if we only document that

Re: Add docs stub for recovery.conf

2020-11-30 Thread David G. Johnston
On Mon, Nov 30, 2020 at 11:25 AM Bruce Momjian wrote: > On Mon, Nov 30, 2020 at 10:11:04AM +0800, Craig Ringer wrote: > > Can we please just address this docs issue? If you don't like my > solution can > > you please supply a patch that you feel addresses the problem? Or > clearly state > > that

Re: Cost overestimation of foreign JOIN

2020-11-30 Thread Andrey Lepikhov
On 30.11.2020 22:38, Tom Lane wrote: Andrey Lepikhov writes: Maybe it is needed to swap lines 2908 and 2909 (see attachment)? No; as explained in the comment immediately above here, we're assuming that the join conditions will be applied on the cross product of the input relations. Thank yo

Re: Add docs stub for recovery.conf

2020-11-30 Thread Bruce Momjian
On Mon, Nov 30, 2020 at 10:11:04AM +0800, Craig Ringer wrote: > Can we please just address this docs issue? If you don't like my solution can > you please supply a patch that you feel addresses the problem? Or clearly > state > that you don't think there is a problem, and do so in a way that actua

Re: Add Information during standby recovery conflicts

2020-11-30 Thread Alvaro Herrera
On 2020-Dec-01, Fujii Masao wrote: > + if (proc) > + { > + if (nprocs == 0) > + appendStringInfo(&buf, "%d", proc->pid); > + else > +

Re: Improving spin-lock implementation on ARM.

2020-11-30 Thread Tom Lane
Alexander Korotkov writes: > I tend to think that LSE is enabled by default in Apple's clang based > on your previous message[1]. In order to dispel the doubts could you > please provide assembly of SpinLockAcquire for following clang > options. > "-O2" > "-O2 -march=armv8-a+lse" > "-O2 -march=ar

Re: Add Information during standby recovery conflicts

2020-11-30 Thread Fujii Masao
On 2020/11/30 16:26, Masahiko Sawada wrote: On Mon, Nov 30, 2020 at 3:46 PM Drouvot, Bertrand wrote: Hi, On 11/30/20 4:41 AM, Masahiko Sawada wrote: On Sun, Nov 29, 2020 at 3:47 PM Drouvot, Bertrand wrote: Hi Alvaro, On 11/28/20 6:36 PM, Alvaro Herrera wrote: Hi Bertrand, On 2020-Nov

Re: Add Information during standby recovery conflicts

2020-11-30 Thread Fujii Masao
On 2020/11/20 18:17, Drouvot, Bertrand wrote: Hi, On 11/17/20 4:44 PM, Fujii Masao wrote: Thanks for updating the patch! Here are review comments. +    Controls whether a log message is produced when the startup process +    is waiting longer than deadlock_timeout +    for reco

Re: Autovacuum on partitioned table (autoanalyze)

2020-11-30 Thread Alvaro Herrera
On 2020-Nov-10, Kyotaro Horiguchi wrote: > In second thought about the reason for the "toprel_oid". It is perhaps > to avoid "wrongly" propagated values to ancestors after a manual > ANALYZE on a partitioned table. But the same happens after an > autoanalyze iteration if some of the ancestors of

Re: Cost overestimation of foreign JOIN

2020-11-30 Thread Tom Lane
Andrey Lepikhov writes: > Maybe it is needed to swap lines 2908 and 2909 (see attachment)? No; as explained in the comment immediately above here, we're assuming that the join conditions will be applied on the cross product of the input relations. Now admittedly, that's a worst-case assumption,

Re: Notes on physical replica failover with logical publisher or subscriber

2020-11-30 Thread Alexey Kondratov
Hi Craig, On 2020-11-30 06:59, Craig Ringer wrote: https://wiki.postgresql.org/wiki/Logical_replication_and_physical_standby_failover Thank you for sharing these notes. I have not dealt a lot with physical/logical replication interoperability, so those were mostly new problems for me to kn

Re: Re: parallel distinct union and aggregate support patch

2020-11-30 Thread bu...@sohu.com
> 1. > +#define BATCH_SORT_MAX_BATCHES 512 > > Did you decide this number based on some experiment or is there some > analysis behind selecting this number? When there are too few batches, if a certain process works too slowly, it will cause unbalanced load. When there are too many batches, FD

  1   2   >