Re: Parallel grouping sets

Hi Jesse, Thanks for reviewing these two patches. On Sat, Jan 25, 2020 at 6:52 AM Jesse Zhang wrote: > > I glanced over both patches. Just the opposite, I have a hunch that v3 > is always better than v5. Here's my 6-minute understanding of both. > > v5 (the one with a simple partial aggregate)

Re: [Proposal] Global temporary tables

On 31.01.2020 22:38, Robert Haas wrote: Now, your idea is not quite as crazy as that, but it has the same basic problem: you can't insert code into a low-level facility that uses a high level facility which may in turn use and depend on that very same low-level facility to not be in the

Re: base backup client as auxiliary backend process

Hi, On 2020-01-11 10:52:30 +0100, Peter Eisentraut wrote: > On 2020-01-10 04:32, Masahiko Sawada wrote: > > I agreed that these patches are useful on its own and 0001 patch and > > committed 0001 over on -committers Robert complained: On 2020-01-23 15:49:37 -0500, Robert Haas wrote: > On Tue,

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

On Mon, Feb 3, 2020 at 5:33 PM Fujii Masao wrote: > Thanks for explaining that! But heap_beginscan_internal() was really > called in TidScan case? Oh, you are right. Tid Scan started calling table_beginscan from v12 (commit 147e3722f7). So previously(-v11), Tid Scan might never calls

Re: BUG #16171: Potential malformed JSON in explain output

On Sun, Feb 02, 2020 at 11:48:32AM -0500, Tom Lane wrote: > > Does that prevent backpatching this, or are we Ok with EXPLAIN text output > > not > > being stable across minors? AFAICT Pg::Explain still works fine with this > > change, but mileage may vary for other parsers. > I'm not sure about

Re: [Proposal] Global temporary tables

On 01.02.2020 19:14, 曾文旌(义从) wrote: 2020年1月27日 下午5:38，Konstantin Knizhnik > 写道： On 25.01.2020 18:15, 曾文旌(义从) wrote: I wonder why do we need some special check for GTT here. From my point of view cleanup at startup of local storage of temp tables should

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

On 2020/02/03 16:39, Kasahara Tatsuhito wrote: On Mon, Feb 3, 2020 at 4:22 PM Fujii Masao wrote: On 2020/02/01 16:05, Kasahara Tatsuhito wrote: if (scan->rs_base.rs_flags & (SO_TYPE_SEQSCAN | SO_TYPE_SAMPLESCAN)) { /* * Ensure a missing snapshot is noticed

Re: Parallel grouping sets

Hi Amit, Thanks for reviewing these two patches. On Sat, Jan 25, 2020 at 6:31 PM Amit Kapila wrote: > > This is what I also understood after reading this thread. So, my > question is why not just review v3 and commit something on those lines > even though it would take a bit more time. It is

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

On Tue, Jun 26, 2018 at 12:47 PM Masahiko Sawada wrote: > > On Fri, Apr 27, 2018 at 4:25 AM, Robert Haas wrote: > > On Thu, Apr 26, 2018 at 3:10 PM, Andres Freund wrote: > >>> I think the real question is whether the scenario is common enough to > >>> worry about. In practice, you'd have to be

Re: Dumping/restoring fails on inherited generated column

On 2019-12-04 21:36, Tom Lane wrote: I wonder if the right fix is to not generate a DO_ATTRDEF object at all for generated columns in child tables. Am I right in guessing that we propagate generated-ness to child tables automatically? Right. New patch using that approach attached. (Could

Re: Dumping/restoring fails on inherited generated column

Peter Eisentraut writes: > On 2019-12-04 21:36, Tom Lane wrote: >> I wonder if the right fix is to not generate a DO_ATTRDEF >> object at all for generated columns in child tables. Am >> I right in guessing that we propagate generated-ness to >> child tables automatically? > Right. New patch

Re: [Proposal] Global temporary tables

po 3. 2. 2020 v 14:03 odesílatel 曾文旌(义从) napsal: > > > 2020年2月2日 上午2:00，Pavel Stehule 写道： > > > > so 1. 2. 2020 v 14:39 odesílatel 曾文旌(义从) > napsal: > >> >> >> 2020年1月30日 下午10:21，Pavel Stehule 写道： >> >> >> >> čt 30. 1. 2020 v 15:17 odesílatel 曾文旌(义从) >> napsal: >> >>> >>> >>> > 2020年1月29日

Re: Index only scan and ctid

Laurenz Albe writes: > I noticed that "ctid" in the select list prevents an index only scan: > This strikes me as strange, since every index contains "ctid". There's no provision for an IOS to return a system column, though. Not sure what it'd take to make that possible.

Re: Memory-Bounded Hash Aggregation

On Wed, 2020-01-29 at 14:48 -0800, Jeff Davis wrote: > 2) Use a different structure more capable of handling a large > fraction > of free space. A compressed bitmap might make sense, but that seems > like overkill to waste effort tracking a lot of space that is > unlikely > to ever be used. I

Re: Parallel grouping sets

On Mon, Feb 3, 2020 at 12:07 AM Richard Guo wrote: > > Hi Jesse, > > Thanks for reviewing these two patches. I enjoyed it! > > On Sat, Jan 25, 2020 at 6:52 AM Jesse Zhang wrote: >> >> >> I glanced over both patches. Just the opposite, I have a hunch that v3 >> is always better than v5. Here's

Re: Complete data erasure

On Mon, Feb 03, 2020 at 09:07:09AM -0500, Stephen Frost wrote: Greetings, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: On Tue, Jan 28, 2020 at 02:34:07PM -0500, Stephen Frost wrote: >We certainly can't run external commands during transaction COMMIT, so >this can't be part of a regular

Re: Portal->commandTag as an enum

> On Feb 2, 2020, at 6:14 PM, Tom Lane wrote: > > Mark Dilger writes: >> I put the CommandTag enum in src/common because there wasn’t any reason >> not to do so. It seems plausible that frontend test frameworks might want >> access to this enum. > Thanks for looking! > Au contraire,

Index only scan and ctid

I noticed that "ctid" in the select list prevents an index only scan: CREATE TABLE ios (id bigint NOT NULL, val text NOT NULL); INSERT INTO ios SELECT i, i::text FROM generate_series(1, 10) AS i; CREATE INDEX ON ios (id); VACUUM (ANALYZE) ios; EXPLAIN (VERBOSE, COSTS off) SELECT ctid, id

Re: Add %x to PROMPT1 and PROMPT2

> On 3 Feb 2020, at 08:08, Michael Paquier wrote: > FWIW, I am not really in favor of changing a default old enough that > it could vote (a45195a). That by itself doesn't seem a good reason to not change things. My concern would be that users who have never ever considered that the prompt can

Re: [Proposal] Global temporary tables

> 2020年2月2日 上午2:00，Pavel Stehule 写道： > > > > so 1. 2. 2020 v 14:39 odesílatel 曾文旌(义从) > napsal: > > >> 2020年1月30日 下午10:21，Pavel Stehule > > 写道： >> >> >> >> čt 30. 1. 2020 v 15:17 odesílatel 曾文旌(义从) >

Re: Complete data erasure

Greetings, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: > On Tue, Jan 28, 2020 at 02:34:07PM -0500, Stephen Frost wrote: > >We certainly can't run external commands during transaction COMMIT, so > >this can't be part of a regular DROP TABLE. > > IMO the best solution would be that the

Re: [PoC] Non-volatile WAL buffer

Hi, On 2020-01-27 13:54:38 -0500, Robert Haas wrote: > On Mon, Jan 27, 2020 at 2:01 AM Takashi Menjo > wrote: > > It sounds reasonable, but I'm sorry that I haven't tested such a program > > yet. I'll try it to compare with my non-volatile WAL buffer. For now, I'm > > a little worried about

Re: base backup client as auxiliary backend process

Hi, Comment: - It'd be good to split out the feature independent refactorings, like the introduction of InitControlFile(), into their own commit. Right now it's hard to separate out what should just should be moved code, and what should be behavioural changes. Especially when there's stuff

Re: Brokenness in dump/restore for GENERATED expressions

On 2020-01-30 19:54, Tom Lane wrote: Using today's HEAD, the regression database cannot be dumped and restored normally. Since the buildfarm isn't all red, I suppose it works in --binary-upgrade mode ... but if I just do $ make installcheck # to set up the test database $ pg_dump -Fc

Re: PATCH: standby crashed when replay block which truncated in standby but failed to truncate in master node

Hi, On 2020-01-21 15:41:54 +0900, Fujii Masao wrote: > On 2020/01/21 13:39, Michael Paquier wrote: > > On Tue, Jan 21, 2020 at 08:45:14AM +0530, Amit Kapila wrote: > > > The original email doesn't say so. I might be missing something, but > > > can you explain what makes you think so. > > > >

Re: Add %x to PROMPT1 and PROMPT2

Daniel Gustafsson writes: > On 3 Feb 2020, at 08:08, Michael Paquier wrote: >> FWIW, I am not really in favor of changing a default old enough that >> it could vote (a45195a). > That by itself doesn't seem a good reason to not change things. > My concern would be that users who have never ever

Re: explain HashAggregate to report bucket and memory stats

Hi, On 2020-01-03 10:19:26 -0600, Justin Pryzby wrote: > On Sun, Feb 17, 2019 at 11:29:56AM -0500, Jeff Janes wrote: > https://www.postgresql.org/message-id/CAMkU%3D1zBJNVo2DGYBgLJqpu8fyjCE_ys%2Bmsr6pOEoiwA7y5jrA%40mail.gmail.com > > What would I find very useful is [...] if the HashAggregate

Re: BUG #16171: Potential malformed JSON in explain output

Daniel Gustafsson writes: > On 1 Feb 2020, at 20:37, Tom Lane wrote: >> 0002 attached isn't committable, because nobody would want the overhead >> in production, but it seems like a good trick to keep up our sleeves. > Thats a neat trick, I wonder if it would be worth maintaining a curated list

Re: BUG #16171: Potential malformed JSON in explain output

> On 2 Feb 2020, at 17:48, Tom Lane wrote: > Thoughts? Keeping TEXT explain stable across minor versions is very appealing, but more so from a policy standpoint than a technical one. The real-world implication is probably quite small, but that's a very unscientific guess (searching Github

Re: Cache relation sizes?

Hi, On 2019-12-31 17:05:31 +1300, Thomas Munro wrote: > There is one potentially interesting case that doesn't require any > kind of shared cache invalidation AFAICS. XLogReadBufferExtended() > calls smgrnblocks() for every buffer access, even if the buffer is > already in our buffer pool. Yea,

Re: BUG #16171: Potential malformed JSON in explain output

Daniel Gustafsson writes: > I guess I'm leaning towards backpatching, but it's not entirely clear-cut. That's where I stand too. I'll wait a day or so to see if anyone else comments; but if not, I'll back-patch. regards, tom lane

Re: Experimenting with hash join prefetch

On Fri, Apr 12, 2019 at 4:23 PM Thomas Munro wrote: > ... if we also prefetch during > the build phase ... Here's an experimental patch to investigate just that part. I tried initiating a prefetch of the bucket just before we copy the tuple and then finally insert it, but it doesn't seem to be

Re: pg_stat_progress_basebackup - progress reporting for pg_basebackup, in the server side

On 2020/02/03 16:28, Amit Langote wrote: On Mon, Feb 3, 2020 at 1:17 PM Fujii Masao wrote: On 2020/02/02 14:59, Masahiko Sawada wrote: On Fri, 31 Jan 2020 at 02:29, Fujii Masao wrote: On 2020/01/30 12:58, Kyotaro Horiguchi wrote: +WHEN 3 THEN 'stopping backup'::text I'm not

Re: Add %x to PROMPT1 and PROMPT2

On 2020-Feb-03, Daniel Gustafsson wrote: > > On 3 Feb 2020, at 08:08, Michael Paquier wrote: > > > FWIW, I am not really in favor of changing a default old enough that > > it could vote (a45195a). > > That by itself doesn't seem a good reason to not change things. Yeah. > My concern would be

Re: Missing break in RelationFindReplTupleSeq

On 2020-Jan-31, Alvaro Herrera wrote: > On 2020-Jan-31, Konstantin Knizhnik wrote: > > > Eventually we find out that logical replication in the current version of > > Postgres works significantly slower on table with replica identity full than > > old pglogical implementation. > > > > The

Re: Global temporary tables

On 01.02.2020 14:49, Tomas Vondra wrote: Hi, this patch was marked as waiting on author since the beginning of the CF, most likely because it no longer applies (not sure). As there has been very little activity since then, I've marked it as returned with feedback. Feel free to re-submit an

Re: pg_stat_progress_basebackup - progress reporting for pg_basebackup, in the server side

On Mon, Feb 3, 2020 at 11:04 PM Fujii Masao wrote: > So we now have the following ideas about the phase names for pg_basebackup. > > 1. > initializing > > 2. > 2-1. starting backup > 2-2. starting file transfer > 2-3. performing pg_start_backup > 2-4. performing checkpoint > 2-5. waiting for [

Re: logical decoding : exceeded maxAllocatedDescs for .spill files

On Sun, Jan 12, 2020 at 9:51 AM Tom Lane wrote: > > Tomas Vondra writes: > > On Sat, Jan 11, 2020 at 10:53:57PM -0500, Tom Lane wrote: > >> remind me where the win came from, exactly? > > > Well, the problem is that in 10 we allocate tuple data in the main > > memory ReorderBuffer context, and

Re: Internal key management system

On Sun, 2 Feb 2020 at 17:05, Fabien COELHO wrote: > > > Hello Masahiko-san, > > > I've started a new separate thread from the previous long thread[1] > > for internal key management system to PostgreSQL. As I mentioned in > > the previous mail[2], we've decided to step back and focus on only > >

Re: error context for vacuum to include block number

On Sun, 2 Feb 2020 at 15:03, Justin Pryzby wrote: > > Thanks for reviewing again > > On Sun, Feb 02, 2020 at 10:45:12AM +0900, Masahiko Sawada wrote: > > Thank you for updating the patch. Here is some review comments: > > > > 1. > > +typedef struct > > +{ > > + char*relnamespace; > > +

Re: table partition and column default

On Thu, Dec 26, 2019 at 6:21 PM Julien Rouhaud wrote: > On Thu, Dec 26, 2019 at 7:12 AM Fujii Masao wrote: > > Thanks for reviewing the patch. Committed! > > I saw that you only pushed it on master, shouldn't we backpatch it > down to pg10 as this is the declarative partitioning behavior since >

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

On Mon, Feb 3, 2020 at 9:51 AM Amit Kapila wrote: > > On Fri, Jan 10, 2020 at 10:53 AM Dilip Kumar wrote: > > > > On Mon, Dec 30, 2019 at 3:43 PM Amit Kapila wrote: > > > > > > > > > > 2. During commit time in DecodeCommit we check whether we need to skip > > > > the changes of the transaction

RE: Complete data erasure

From: Tomas Vondra > That's not really what I meant - let me explain. When I said DROP TABLE > should do everything as usual, that includes catalog changes. I.e. after > the commit there would not be any remaining entries in system catalogs > or anything like that. > > The only thing we'd do

Re: Experimenting with hash join prefetch

HI, On 2020-02-04 01:48:49 +1300, Thomas Munro wrote: > On Fri, Apr 12, 2019 at 4:23 PM Thomas Munro wrote: > > ... if we also prefetch during > > the build phase ... > > Here's an experimental patch to investigate just that part. I tried > initiating a prefetch of the bucket just before we

Re: Add %x to PROMPT1 and PROMPT2

On 2020/02/03 23:40, Tom Lane wrote: Daniel Gustafsson writes: On 3 Feb 2020, at 08:08, Michael Paquier wrote: FWIW, I am not really in favor of changing a default old enough that it could vote (a45195a). That by itself doesn't seem a good reason to not change things. My concern would

Re: Experimenting with hash join prefetch

On Tue, Feb 4, 2020 at 2:31 PM Andres Freund wrote: > How much of the benefit here comes from the prefetching, and how much > just from writing the code in a manner that allows for more out-of-order > execution? Because there's no dependencies preventing execution of the > next queued tuple

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

On Tue, Jan 28, 2020 at 11:43 AM Amit Kapila wrote: > > On Tue, Jan 28, 2020 at 11:34 AM Dilip Kumar wrote: > > > > On Tue, Jan 28, 2020 at 11:28 AM Amit Kapila > > wrote: > > > > > > On Wed, Jan 22, 2020 at 10:30 AM Dilip Kumar > > > wrote: > > > > > > > > On Tue, Jan 14, 2020 at 10:44 AM

Re: base backup client as auxiliary backend process

On Mon, Feb 03, 2020 at 01:37:25AM -0800, Andres Freund wrote: > On 2020-01-23 15:49:37 -0500, Robert Haas wrote: >> I assume that's probably discussed on the thread that is linked here, >> but you shouldn't have to dig through the discussion thread to figure >> out what the benefits of a change

Re: Is custom MemoryContext prohibited?

Hi, On 2020-01-28 11:10:47 -0500, Robert Haas wrote: > I generally like this idea, but I'd like to propose that we instead > replace the NodeTag with a 4-byte magic number. I was studying how > feasible it would be to make memory contexts available in frontend > code, and it doesn't look all that

Re: Implementing Incremental View Maintenance

Hi, Attached is the latest patch (v12) to add support for Incremental Materialized View Maintenance (IVM). It is possible to apply to current latest master branch. Differences from the previous patch (v11) include: * support executing REFRESH MATERIALIZED VIEW command with IVM. * support

RE: Complete data erasure

>From tsunakawa.ta...@fujitsu.com > What is concerned about is that the need to erase and delete the data file > would be forgotten if the server crashes during step > 3. If so, postmaster can do the job at startup, just like it deletes > temporary files (although it delays the startup.) I

Re: pg_stat_progress_basebackup - progress reporting for pg_basebackup, in the server side

On Tue, Feb 4, 2020 at 10:34 AM Amit Langote wrote: > Reading this: > > + backup_total > + bigint > + > + Total amount of data that will be streamed. If progress reporting > + is not enabled in pg_basebackup > + (i.e., --progress option is not specified), > + this