Re: pg_stat_progress_basebackup - progress reporting for pg_basebackup, in the server side

On Wed, Feb 5, 2020 at 3:36 PM Fujii Masao wrote: > On 2020/02/04 10:34, Amit Langote wrote: > > I like: > > Thanks for reviewing the patch! > > > 1. initializing > > 2-5 waiting for backup start checkpoint to finish > > Can we shorten this to "waiting for checkpoint"? IMO the simpler > phase

Re: [Proposal] Global temporary tables

On 05.02.2020 00:38, Robert Haas wrote: My guess it that the right time to do this work is just after we acquire locks, at the end of parse analysis. I think trying to do it during execution is too late, since the planner looks at indexes, and trying to do it in the planner instead of before

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

On Mon, Feb 3, 2020 at 6:20 PM Kasahara Tatsuhito wrote: > Therefore, from v12, Tid scan not only increases the value of > seq_scan, but also acquires a predicate lock. Based on further investigation and Fujii's advice, I've summarized this issue as follows. >From commit 147e3722f7, Tid Scan

Re: Expose lock group leader pid in pg_stat_activity

On Wed, Feb 05, 2020 at 10:48:31AM +0900, Michael Paquier wrote: > On Tue, Feb 04, 2020 at 03:27:25PM +0100, Julien Rouhaud wrote: > > I added some code comments to remind that we don't guarantee any consistency > > here. > > That's mostly fine. I have moved the comment related to >

Re: ALTER tbl rewrite loses CLUSTER ON index

Hi Justin, On Mon, Feb 3, 2020 at 1:17 AM Justin Pryzby wrote: > Other options are preserved by ALTER (and CLUSTER ON is and most obviously > should be preserved by CLUSTER's rewrite), so I think (SET) CLUSTER should be > preserved by ALTER, too. Yes. create table foo (a int primary key);

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

On Mon, Feb 3, 2020 at 8:03 PM Amit Kapila wrote: > > On Tue, Jun 26, 2018 at 12:47 PM Masahiko Sawada > wrote: > > > > On Fri, Apr 27, 2018 at 4:25 AM, Robert Haas wrote: > > > On Thu, Apr 26, 2018 at 3:10 PM, Andres Freund wrote: > > >>> I think the real question is whether the scenario is

Re: pg_stat_progress_basebackup - progress reporting for pg_basebackup, in the server side

On 2020/02/04 10:34, Amit Langote wrote: On Mon, Feb 3, 2020 at 11:04 PM Fujii Masao wrote: So we now have the following ideas about the phase names for pg_basebackup. 1. initializing 2. 2-1. starting backup 2-2. starting file transfer 2-3. performing pg_start_backup 2-4. performing

Re: [PATCH v1] Allow COPY "text" format to output a header

Hi, On Tue, Feb 4, 2020 at 4:25 PM Rémi Lapeyre wrote: > This patch adds the possibility to use the "header" option when using COPY > with > the text format. A todo entry was opened for this and I updated the tests > and > the documentation. > > This was previously discussed at >

A bug in LWLOCK_STATS

Hi, When I compiled PostgreSQL with -DLWLOCK_STATS and tried to check the statistics of light-weight locks, I observed that more than one statistics entries were output *for the same backend process and the same lwlock*. For example, I got the following four statistics when I checked how the

Re: Portal->commandTag as an enum

> On Feb 4, 2020, at 7:34 PM, Andres Freund wrote: > > Hi, Thanks for reviewing! I am pretty much in agreement with your comments, below. > On 2020-02-04 18:18:52 -0800, Mark Dilger wrote: >> In master, a number of functions pass a char *completionTag argument (really >> a char

Re: table partition and column default

On 2020/02/04 13:56, Amit Langote wrote: On Thu, Dec 26, 2019 at 6:21 PM Julien Rouhaud wrote: On Thu, Dec 26, 2019 at 7:12 AM Fujii Masao wrote: Thanks for reviewing the patch. Committed! I saw that you only pushed it on master, shouldn't we backpatch it down to pg10 as this is the

Re: [HACKERS] kqueue

On Wed, Jan 29, 2020 at 11:54 AM Thomas Munro wrote: > If there are no further objections, I'm planning to commit this sooner > rather than later, so that it gets plenty of air time on developer and > build farm machines. If problems are discovered on a particular > platform, there's a pretty

Re: base backup client as auxiliary backend process

On Mon, 3 Feb 2020 at 20:06, Andres Freund wrote: > > Hi, > > On 2020-01-11 10:52:30 +0100, Peter Eisentraut wrote: > > On 2020-01-10 04:32, Masahiko Sawada wrote: > > > I agreed that these patches are useful on its own and 0001 patch and > > > > committed 0001 > > over on -committers Robert

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

On Wed, Feb 5, 2020 at 9:27 AM Amit Kapila wrote: > > On Tue, Feb 4, 2020 at 11:00 AM Dilip Kumar wrote: > > > > On Tue, Jan 28, 2020 at 11:43 AM Amit Kapila > > wrote: > > > > > > > > > One more thing we can do is to identify whether the tuple belongs to > > > toast relation while decoding

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

On Fri, Jan 31, 2020 at 8:08 AM Amit Kapila wrote: > > On Thu, Jan 30, 2020 at 6:10 PM Dilip Kumar wrote: > > > > On Thu, Jan 30, 2020 at 4:11 PM Amit Kapila wrote: > > > > > > Also, if we need to copy the snapshot here, then do we need to again > > > copy it in ReorderBufferProcessTXN(in below

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

On Tue, Feb 4, 2020 at 11:00 AM Dilip Kumar wrote: > > On Tue, Jan 28, 2020 at 11:43 AM Amit Kapila wrote: > > > > > > One more thing we can do is to identify whether the tuple belongs to > > toast relation while decoding it. However, I think to do that we need > > to have access to relcache at

Re: Portal->commandTag as an enum

Hi, On 2020-02-04 18:18:52 -0800, Mark Dilger wrote: > In master, a number of functions pass a char *completionTag argument (really > a char completionTag[COMPLETION_TAG_BUFSIZE]) which gets filled in with the > string to return to the client from EndCommand. I have removed that kind of >

Re: Documentation patch for ALTER SUBSCRIPTION

>> On Feb 4, 2020, at 8:45 PM, Amit Kapila wrote: >> >> On Fri, Jan 24, 2020 at 2:05 AM David Christensen >> wrote: >> Greetings, >> Enclosed find a documentation patch that clarifies the behavior of ALTER >> SUBSCRIPTION … REFRESH PUBLICATION with new tables; I ran into a situation >>

Re: Documentation patch for ALTER SUBSCRIPTION

On Fri, Jan 24, 2020 at 2:05 AM David Christensen wrote: > > Greetings, > > Enclosed find a documentation patch that clarifies the behavior of ALTER > SUBSCRIPTION … REFRESH PUBLICATION with new tables; I ran into a situation > today where the docs were not clear that existing tables would not

Re: closesocket behavior in different platforms

On Thu, Jan 30, 2020 at 9:04 AM Amit Kapila wrote: > > On Wed, Jan 29, 2020 at 8:29 PM Robert Haas wrote: > > > > On Wed, Jan 29, 2020 at 6:04 AM vignesh C wrote: > > > Thanks for your review and suggestion. I have made a patch based on > > > similar lines. Attached patch has the doc update

Re: Expose lock group leader pid in pg_stat_activity

On Tue, Feb 04, 2020 at 03:27:25PM +0100, Julien Rouhaud wrote: > I added some code comments to remind that we don't guarantee any consistency > here. That's mostly fine. I have moved the comment related to AuxiliaryPidGetProc() within the inner part of its the "if" (or the comment should be

RE: Complete data erasure

From: Tom Lane > Up to now, we've sort of looked the other way with respect to failures > of file unlinks post-commit, reasoning that the worst that will happen > is disk space leakage from no-longer-referenced files that we failed to > unlink. (Which is bad, certainly, but not catastrophic;

Re: Delaying/avoiding BTreeTupleGetNAtts() call within _bt_compare()

On Thu, Jan 30, 2020 at 1:19 AM Floris Van Nee wrote: > I'd say applying just v2-0001 is actually slightly hurtful for single-core > performance. Applying all of them gives a good improvement though. It looks > like the performance improvement is also more noticeable at higher core > counts

Re: Memory-Bounded Hash Aggregation

On Wed, Feb 5, 2020 at 12:08 PM Peter Geoghegan wrote: > Parallel CREATE INDEX is currently accidentally disabled on the master > branch. That should be fixed in the next couple of days. You can > temporarily revert 74618e77 if you want to get it back for testing > purposes today. (Fixed --

Re: Memory-Bounded Hash Aggregation

On Mon, Feb 3, 2020 at 6:24 PM Jeff Davis wrote: > And now I'm attaching another version of the main Hash Aggregation > patch to be applied on top of the logtape.c patch. Have you tested this against tuplesort.c, particularly parallel CREATE INDEX? It would be worth trying to measure any

Re: Minimal logical decoding on standbys

Hi all, This is great stuff! My understanding is that this patch guarantees 0 data loss for a logical replication stream if the primary crashes and a standby which was marked as sync at failure time is promoted. Is this correct? James -- James Sewell, Chief Architect Suite 112, Jones Bay

Re: Complete data erasure

Tomas Vondra writes: > I think it depends how exactly it's implemented. As Tom pointed out in > his message [1], we can't do the erasure itself in the post-commit is > not being able to handle errors. But if the files are renamed durably, > and the erasure happens in a separate process, that

Re: [Proposal] Global temporary tables

On Mon, Feb 3, 2020 at 3:08 AM Konstantin Knizhnik wrote: > Thank you for explanation. > You convinced me that building indexes from _bt_getbuf is not good idea. > What do you think about idea to check and build indexes for GTT prior to > query execution? > > In this case we do not need to patch

Re: Complete data erasure

On Tue, Feb 04, 2020 at 12:53:44AM +, tsunakawa.ta...@fujitsu.com wrote: From: Tomas Vondra That's not really what I meant - let me explain. When I said DROP TABLE should do everything as usual, that includes catalog changes. I.e. after the commit there would not be any remaining entries

Re: [Proposal] Global temporary tables

On Sat, Feb 1, 2020 at 11:14 AM 曾文旌(义从) wrote: > As global_private_temp-8.patch, think about: > 1 session X tale several hours doing some statistical work with the GTT A, > which generated some data using transaction 100, The work is not over. > 2 Then session Y vacuumed A, and the GTT's

RE: Index Skip Scan

> this point me and Jesper inclined to go with the second option. But maybe > I'm missing something, are there any other suggestions? Unfortunately I figured this would need a more invasive fix. I tend to agree that it'd be better to not skip in situations like this. I think it'd make most

Re: Clarifying/rationalizing Vars' varno/varattno/varnoold/varoattno

Robert Haas writes: > On Fri, Dec 20, 2019 at 11:13 AM Tom Lane wrote: >> The alternatives that seem plausible at this point are >> (1) Create some sort of wrapper node indicating "the contents of this >> expression might be replaced by NULL". This is basically what the >> planner's

Re: Index Skip Scan

> On Mon, Jan 27, 2020 at 02:00:39PM +, Floris Van Nee wrote: > > Thanks for the new patch! I tested it and managed to find a case that causes > some issues. Here's how to reproduce: So, after a bit of investigation I found out the issue (it was actually there even in the previous version).

Re: Index only scan and ctid

Laurenz Albe writes: > On Mon, 2020-02-03 at 14:43 -0500, Tom Lane wrote: >> Laurenz Albe writes: >>> I noticed that "ctid" in the select list prevents an index only scan: >>> This strikes me as strange, since every index contains "ctid". >> There's no provision for an IOS to return a system

Re: Index only scan and ctid

On Mon, 2020-02-03 at 14:43 -0500, Tom Lane wrote: > Laurenz Albe writes: > > I noticed that "ctid" in the select list prevents an index only scan: > > This strikes me as strange, since every index contains "ctid". > > There's no provision for an IOS to return a system column, though. > Not sure

Re: BUG #16171: Potential malformed JSON in explain output

I wrote: > Daniel Gustafsson writes: >> I guess I'm leaning towards backpatching, but it's not entirely clear-cut. > That's where I stand too. I'll wait a day or so to see if anyone > else comments; but if not, I'll back-patch. Hearing no objections, done that way.

Re: [Proposal] Global temporary tables

On 04.02.2020 18:01, 曾文旌(义从) wrote: Yes, exactly. But it is still not clear to me why do we need some special handling for GTT? Shared memory is reinitialized and storage of temporary tables is removed. It is true for both local and global temp tables. Of course not. The local temp

Re: Memory-Bounded Hash Aggregation

On 03/02/2020 20:29, Jeff Davis wrote: 1. Use a minheap for the freelist. The original design used an array that had to be sorted between a read (which frees a block) and a write (which needs to sort the array to consume the lowest block number). The comments said: * sorted. This is an

Re: [Proposal] Global temporary tables

> 2020年2月3日 下午4:16，Konstantin Knizhnik 写道： > > > > On 01.02.2020 19:14, 曾文旌(义从) wrote: >> >> >>> 2020年1月27日 下午5:38，Konstantin Knizhnik >> > 写道： >>> >>> >>> >>> On 25.01.2020 18:15, 曾文旌(义从) wrote: I wonder why do we need some special check for GTT

Re: Expose lock group leader pid in pg_stat_activity

On Thu, Jan 30, 2020 at 10:03:01PM +0900, Michael Paquier wrote: > On Tue, Jan 28, 2020 at 02:52:08PM +0100, Tomas Vondra wrote: > > On Tue, Jan 28, 2020 at 02:26:34PM +0100, Julien Rouhaud wrote: > >> There were already some dependencies between the rows since parallel > >> queries were added, as

[PATCH v1] Allow COPY "text" format to output a header

This patch adds the possibility to use the "header" option when using COPY with the text format. A todo entry was opened for this and I updated the tests and the documentation. This was previously discussed at

More DSM slot exhaustion bugs

Hello, Here are two more bugs I found by running the regression tests repeatedly with dsm_create() hacked to fail at random. 1. If find_or_make_matching_shared_tupledesc() fails, we leave behind a null pointer in RecordCacheHash, so that a later lookup segfaults. 2. If we do a rescan, then

Re: Memory-Bounded Hash Aggregation

On Mon, Feb 03, 2020 at 06:24:14PM -0800, Jeff Davis wrote: > On Mon, 2020-02-03 at 10:29 -0800, Jeff Davis wrote: > > I ended up converting the freelist to a min heap. > > > > Attached is a patch which makes three changes to better support > > HashAgg: > > And now I'm attaching another version

RE: [Patch] Optimize dropping of relation buffers using dlist

Hi, I have rebased the patch to keep the CFbot happy. Apparently, in the previous patch there was a possibility of infinite loop when allocating buffers, so I fixed that part and also removed some whitespaces. Kindly check the attached V6 patch. Any thoughts on this? Regards, Kirk Jamison

Re: Implementing Incremental View Maintenance

"ROW LEVEL SECURITY" and INCREMENTAL MATERIALIZED VIEW. Hi. If ROW LEVEL SECURITY is set for the source table after creating the INCREMENTAL MATELIALIZED VIEW, the search results by that are not reflected. After setting ROW LEVEL SECURITY (similar to normal MATERIALIZED VIEW), you need to

Re: logical decoding : exceeded maxAllocatedDescs for .spill files

On Tue, Feb 4, 2020 at 10:15 AM Kuntal Ghosh wrote: > > On Sun, Jan 12, 2020 at 9:51 AM Tom Lane wrote: > > > > Tomas Vondra writes: > > > On Sat, Jan 11, 2020 at 10:53:57PM -0500, Tom Lane wrote: > > >> remind me where the win came from, exactly? > > > > > Well, the problem is that in 10 we

Re: Add %x to PROMPT1 and PROMPT2

On Tue, Feb 04, 2020 at 10:31:43AM +0900, Ian Barwick wrote: > The last change I recall affecting default psql behaviour was the addition > of COMP_KEYWORD_CASE in 9.2 (db84ba65), which personally I (and no doubt > others) > found annoying, but the world still turns. > > +1 one for this change,

Refactor compile-time assertion checks for C/C++

Hi all, As of [1], I have been playing with the compile time assertions that we have for expressions, declarations and statements. And it happens that it is visibly possible to consolidate the fallback implementations for C and C++. Attached is the result of what I am getting at. I am adding