Re: Identifying user-created objects

2020-02-05 Thread Julien Rouhaud
On Thu, Feb 6, 2020 at 8:53 AM Amit Langote wrote: > > On Thu, Feb 6, 2020 at 4:31 PM Michael Paquier wrote: > > On Thu, Feb 06, 2020 at 04:25:47PM +0900, Amit Langote wrote: > > > About the implementation, how about defining a static inline function, > > > say is_user_object(), next to

Re: Identifying user-created objects

2020-02-05 Thread Amit Langote
On Thu, Feb 6, 2020 at 4:31 PM Michael Paquier wrote: > On Thu, Feb 06, 2020 at 04:25:47PM +0900, Amit Langote wrote: > > About the implementation, how about defining a static inline function, > > say is_user_object(), next to FirstNormalObjectId's definition and > > make pg_is_user_object() call

SyncRepGetSyncStandbysPriority() vs. SIGHUP

2020-02-05 Thread Noah Misch
Buildfarm runs have triggered the assertion at the end of SyncRepGetSyncStandbysPriority(): sysname │ snapshot │branch │ bfurl

Re: Identifying user-created objects

2020-02-05 Thread Michael Paquier
On Thu, Feb 06, 2020 at 04:25:47PM +0900, Amit Langote wrote: > About the implementation, how about defining a static inline function, > say is_user_object(), next to FirstNormalObjectId's definition and > make pg_is_user_object() call it? There are a few placed in the > backend code that perform

Re: Identifying user-created objects

2020-02-05 Thread Amit Langote
On Wed, Feb 5, 2020 at 8:27 PM Masahiko Sawada wrote: > User can create database objects such as functions into pg_catalog. > But if I'm not missing something, currently there is no > straightforward way to identify if the object is a user created object > or a system object which is created

Re: In PG12, query with float calculations is slower than PG11

2020-02-05 Thread Amit Langote
Hi, On Thu, Feb 6, 2020 at 2:55 PM Andres Freund wrote: > On 2020-02-06 14:25:03 +0900, keisuke kuroda wrote: > > That's because check_float8_val() (in PG 12) is a function > > whose arguments must be evaluated before > > it is called (it is inline, but that's irrelevant), > > whereas

Re: typo in set_rel_consider_parallel()

2020-02-05 Thread Amit Kapila
On Thu, Feb 6, 2020 at 11:41 AM Amit Langote wrote: > > Hi, > > Attached fixes $subject. > LGTM. I will push this later today. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-05 Thread Fujii Masao
On 2020/02/06 15:04, Kasahara Tatsuhito wrote: Hi, On Thu, Feb 6, 2020 at 11:48 AM Andres Freund wrote: I think it'd be good if we could guard against b) via an isolation test. It's more painful to do that for a), due to the unreliability of stats at the moment (we have some tests, but

Re: Make ringbuffer threshold and ringbuffer sizes configurable?

2020-02-05 Thread Laurenz Albe
On Wed, 2020-02-05 at 20:00 -0800, Andres Freund wrote: > The ringbuffers we use for seqscans, vacuum, copy etc can cause very > drastic slowdowns (see e.g. [1]), an can cause some workloads to > practically never end up utilizing shared buffers. ETL workloads > e.g. regularly fight with that

typo in set_rel_consider_parallel()

2020-02-05 Thread Amit Langote
Hi, Attached fixes $subject. Thanks, Amit diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c index 8286d9cf34..905bbe77d8 100644 --- a/src/backend/optimizer/path/allpaths.c +++ b/src/backend/optimizer/path/allpaths.c @@ -610,7 +610,7 @@

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-05 Thread Kasahara Tatsuhito
Hi, On Thu, Feb 6, 2020 at 11:48 AM Andres Freund wrote: > I think it'd be good if we could guard against b) via an isolation > test. It's more painful to do that for a), due to the unreliability of > stats at the moment (we have some tests, but they take a long time). Thanks for your advise,

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-02-05 Thread Andy Fan
update the patch with considering the semi/anti join. Can anyone help to review this patch? Thanks On Fri, Jan 31, 2020 at 8:39 PM Andy Fan wrote: > Hi: > > I wrote a patch to erase the distinctClause if the result is unique by > definition, I find this because a user switch this code from

Re: In PG12, query with float calculations is slower than PG11

2020-02-05 Thread Andres Freund
Hi, On 2020-02-06 14:25:03 +0900, keisuke kuroda wrote: > That's because check_float8_val() (in PG 12) is a function > whose arguments must be evaluated before > it is called (it is inline, but that's irrelevant), > whereas CHECKFLOATVAL() (in PG11) is a macro > whose arguments are only

Re: [PATCH v1] Allow COPY "text" format to output a header

2020-02-05 Thread Surafel Temesgen
On Wed, Feb 5, 2020 at 4:19 PM Rémi Lapeyre wrote: > > > > FWIW there was more recent propose patch at > https://www.postgresql.org/message-id/flat/caf1-j-0ptcwmeltswwgv2m70u26n4g33gpe1rckqqe6wvqd...@mail.gmail.com > > and among feedback given is to adding header matching feature on to > this.

Re: A bug in LWLOCK_STATS

2020-02-05 Thread Fujii Masao
On 2020/02/05 17:13, Julien Rouhaud wrote: On Wed, Feb 05, 2020 at 02:43:49PM +0900, Fujii Masao wrote: Hi, When I compiled PostgreSQL with -DLWLOCK_STATS and tried to check the statistics of light-weight locks, I observed that more than one statistics entries were output *for the same

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2020-02-05 Thread Masahiko Sawada
On Thu, 6 Feb 2020 at 13:16, Amit Kapila wrote: > > On Tue, Jun 26, 2018 at 12:47 PM Masahiko Sawada > wrote: > > > > Type of table: normal table, unlogged table > > Number of child tables : 16, 64 (all tables are located on the same > > tablespace) > > Number of clients : 32 > > Number of

In PG12, query with float calculations is slower than PG11

2020-02-05 Thread keisuke kuroda
Hi, I am testing performance both PG12 and PG11. I found the case of performance degradation in PG12. Amit Langote help me to analyze and to create patch. Thanks! * environment CentOS Linux release 7.6.1810 (Core) postgresql 12.1 postgresql 11.6 * postgresql.conf shared_buffers = 2048MB

Re: Make ringbuffer threshold and ringbuffer sizes configurable?

2020-02-05 Thread Andres Freund
Hi, On 2020-02-06 05:12:11 +, tsunakawa.ta...@fujitsu.com wrote: > I think per-table reloption is necessary as well as or instead of GUC, > because the need for caching depends on the table (see below for Oracle's > manual.) I'm inclined to not do that initially. It's going to be

Re: typos in comments and user docs

2020-02-05 Thread Michael Paquier
On Thu, Feb 06, 2020 at 08:47:14AM +0530, Amit Kapila wrote: > Your changes look fine to me on the first read. I will push this to > HEAD unless there are any objections. If we want them in > back-branches, we might want to probably segregate the changes based > on the branch until those apply.

RE: Make ringbuffer threshold and ringbuffer sizes configurable?

2020-02-05 Thread tsunakawa.ta...@fujitsu.com
From: Andres Freund > While I think there's a number of improvements[2] we could make to the > ringbuffer logic, I think we should also just allow to make them > configurable. I think that'll allow a decent number of systems perform > better (especially on slightly bigger systems the the current

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2020-02-05 Thread Amit Kapila
On Tue, Jun 26, 2018 at 12:47 PM Masahiko Sawada wrote: > > Type of table: normal table, unlogged table > Number of child tables : 16, 64 (all tables are located on the same > tablespace) > Number of clients : 32 > Number of trials : 100 > Duration: 180 seconds for each trials > > The hardware

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2020-02-05 Thread Amit Kapila
On Thu, Feb 6, 2020 at 1:57 AM Mahendra Singh Thalor wrote: > > On Wed, 5 Feb 2020 at 12:07, Masahiko Sawada wrote: > > > > On Mon, Feb 3, 2020 at 8:03 PM Amit Kapila wrote: > > > > > > On Tue, Jun 26, 2018 at 12:47 PM Masahiko Sawada > > > wrote: > > > > > > > > On Fri, Apr 27, 2018 at 4:25

Make ringbuffer threshold and ringbuffer sizes configurable?

2020-02-05 Thread Andres Freund
Hi, The ringbuffers we use for seqscans, vacuum, copy etc can cause very drastic slowdowns (see e.g. [1]), an can cause some workloads to practically never end up utilizing shared buffers. ETL workloads e.g. regularly fight with that problem. While I think there's a number of improvements[2] we

Re: [Proposal] Global temporary tables

2020-02-05 Thread 曾文旌(义从)
> 2020年2月5日 下午10:15,Robert Haas 写道: > > On Wed, Feb 5, 2020 at 8:21 AM 曾文旌(义从) wrote: >> What do you mean by "catalog buffer"? >> Yes, cleanup of local temp table requires deletion of correspondent entry >> from catalog and GTT should not do it. >> But I am speaking only about cleanup of

Re: Is custom MemoryContext prohibited?

2020-02-05 Thread Andres Freund
Hi, On 2020-02-05 10:56:42 -0500, Tom Lane wrote: > Robert Haas writes: > > On Wed, Feb 5, 2020 at 10:09 AM Tom Lane wrote: > >> an enum field right in the context. You'll have to do an extra step to > >> discover the context's type, and if you jump to the wrong conclusion > >> and do, say, >

Re: typos in comments and user docs

2020-02-05 Thread Amit Kapila
On Thu, Feb 6, 2020 at 7:44 AM Justin Pryzby wrote: > > I sent earlier version of this a few times last year along with bunch of other > doc patches but it was never picked up. So maybe I'll try send one at a time > in more digestible chunks. >

Re: Is custom MemoryContext prohibited?

2020-02-05 Thread Andres Freund
On 2020-02-05 10:40:59 -0500, Robert Haas wrote: > I'm pretty unimpressed with the enum proposal - I think it's pretty > nasty for an extension author to have to make up a value that's not in > the enum. One, how are they supposed to know that they should do that? > Two, how are they supposed to

Re: Is custom MemoryContext prohibited?

2020-02-05 Thread Andres Freund
Hi, On 2020-02-05 09:28:08 -0500, Robert Haas wrote: > On Mon, Feb 3, 2020 at 7:26 PM Andres Freund wrote: > > Hm. I kinda like the idea of still having one NodeTag identifying memory > > contexts, and then some additional field identifying the actual > > type. Being able to continue to rely on

Re: Add %x to PROMPT1 and PROMPT2

2020-02-05 Thread Vik Fearing
On 06/02/2020 03:38, Michael Paquier wrote: > On Wed, Feb 05, 2020 at 10:21:11AM -0500, Tom Lane wrote: >> Robert Haas writes: >>> I'm not really against this change but, given how long it's been the >>> way that it is, I think we shouldn't make it without more plus votes. >>> If we've actually

Re: Assumptions about the number of parallel workers

2020-02-05 Thread Andres Freund
Hi, On 2020-02-05 10:50:05 +0100, Antonin Houska wrote: > I can't figure out why ExecGather/ExecGatherMerge do check whether num_workers > is non-zero. I think the code would be a bit clearer if these tests were > replaced with Assert() statements, as the attached patch does. It's probably

Re: Tid scan increments value of pg_stat_all_tables.seq_scan. (but not seq_tup_read)

2020-02-05 Thread Andres Freund
Hi, On 2020-02-05 16:25:25 +0900, Kasahara Tatsuhito wrote: > On Mon, Feb 3, 2020 at 6:20 PM Kasahara Tatsuhito > wrote: > > Therefore, from v12, Tid scan not only increases the value of > > seq_scan, but also acquires a predicate lock. > > Based on further investigation and Fujii's advice,

Re: Is custom MemoryContext prohibited?

2020-02-05 Thread Kyotaro Horiguchi
Hello. At Wed, 5 Feb 2020 13:09:48 -0500, Robert Haas wrote in > On Wed, Feb 5, 2020 at 12:15 PM Tom Lane wrote: > > The core problem that I'm unhappy about is that what you want to do > > adds cognitive burden (including an increased risk of mistakes) and > > extra debugging commands in

Re: Add %x to PROMPT1 and PROMPT2

2020-02-05 Thread Michael Paquier
On Wed, Feb 05, 2020 at 10:21:11AM -0500, Tom Lane wrote: > Robert Haas writes: >> I'm not really against this change but, given how long it's been the >> way that it is, I think we shouldn't make it without more plus votes. >> If we've actually got a broad consensus on it, sure, but I don't

Re: pg_stat_progress_basebackup - progress reporting for pg_basebackup, in the server side

2020-02-05 Thread Amit Langote
On Wed, Feb 5, 2020 at 4:29 PM Amit Langote wrote: > On Wed, Feb 5, 2020 at 3:36 PM Fujii Masao > wrote: > > Yeah, I understand your concern. The pg_basebackup document explains > > the risk when --progress is specified, as follows. Since I imagined that > > someone may explicitly disable

Re: Memory-Bounded Hash Aggregation

2020-02-05 Thread Jeff Davis
On Fri, 2020-01-24 at 17:01 -0800, Jeff Davis wrote: > New patch attached. Three minor independent refactoring patches: 1. Add new entry points for the tuple hash table: TupleHashTableHash() LookupTupleHashEntryHash() which are useful for saving and reusing hash values to avoid

typos in comments and user docs

2020-02-05 Thread Justin Pryzby
I sent earlier version of this a few times last year along with bunch of other doc patches but it was never picked up. So maybe I'll try send one at a time in more digestible chunks. https://www.postgresql.org/message-id/flat/20190427025647.GD3925%40telsasoft.com#e1731c33455145eadc1158042cc411f9

Re: pg_stat_progress_basebackup - progress reporting for pg_basebackup, in the server side

2020-02-05 Thread Amit Langote
On Thu, Feb 6, 2020 at 9:51 AM Kyotaro Horiguchi wrote: > > On Wed, Feb 5, 2020 at 18:25 Amit Langote wrote: > > > So, maybe you're saying that "waiting for checkpoint" is ambiguous and > > > most people will assume it means "...to start". As for me, I assume > > > it ends with "...to finish".

Re: Memory-Bounded Hash Aggregation

2020-02-05 Thread Jeff Davis
On Wed, 2020-02-05 at 11:56 -0800, Jeff Davis wrote: > Regarding the API, I'd like to change it, but I'm running into some > performance challenges when adding a layer of indirection. If I apply > the very simple attached patch, which simply makes a separate > allocation for the tapes array, it

Re: Index Skip Scan

2020-02-05 Thread Kyotaro Horiguchi
At Wed, 5 Feb 2020 17:37:30 +0100, Dmitry Dolgov <9erthali...@gmail.com> wrote in > > On Tue, Feb 04, 2020 at 08:34:09PM +, Floris Van Nee wrote: > > > > > this point me and Jesper inclined to go with the second option. But maybe > > > I'm missing something, are there any other suggestions?

Re: pg_stat_progress_basebackup - progress reporting for pg_basebackup, in the server side

2020-02-05 Thread Kyotaro Horiguchi
At Wed, 5 Feb 2020 18:53:19 +0900, Amit Langote wrote in > Last two messages weren’t sent to the list. Oops! Sorry, I made a mistake sending the mail. > On Wed, Feb 5, 2020 at 18:25 Amit Langote wrote: > > > On Wed, Feb 5, 2020 at 6:15 PM Kyotaro Horiguchi > > wrote: > > > 2020年2月5日(水)

Re: Expose lock group leader pid in pg_stat_activity

2020-02-05 Thread Michael Paquier
On Wed, Feb 05, 2020 at 07:57:20AM +0100, Julien Rouhaud wrote: > This looks good, thanks a lot! Thanks for double-checking. And done. -- Michael signature.asc Description: PGP signature

Re: Do not check unlogged indexes on standby

2020-02-05 Thread Peter Geoghegan
On Wed, Feb 5, 2020 at 1:27 PM Alvaro Herrera wrote: > So, I'm confused. There appear to be two bugfix patches in this thread, > with no relationship between them, and as far as I can tell only one of > them has been addressed. What was applied (6754fe65a4c6) is > significantly different from

Re: Do not check unlogged indexes on standby

2020-02-05 Thread Alvaro Herrera
On 2019-Sep-11, Peter Geoghegan wrote: > On Wed, Sep 11, 2019 at 7:10 PM Peter Geoghegan wrote: > > The patch has been committed already. > > Oh, wait. It hasn't. Andrey didn't create a new thread for his largely > independent patch, so I incorrectly assumed he created a CF entry for > his

Re: DROP OWNED CASCADE vs Temp tables

2020-02-05 Thread Alvaro Herrera
On 2020-Jan-23, Alvaro Herrera wrote: > This is not a 100% solution: there's the cases when the user is removed > from an ACL and from a policy, and those deletions are done directly > instead of accumulating to the end for a mass deletion. > > I had to export AcquireDeletionLock (previously a

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2020-02-05 Thread Mahendra Singh Thalor
On Wed, 5 Feb 2020 at 12:07, Masahiko Sawada wrote: > > On Mon, Feb 3, 2020 at 8:03 PM Amit Kapila wrote: > > > > On Tue, Jun 26, 2018 at 12:47 PM Masahiko Sawada wrote: > > > > > > On Fri, Apr 27, 2018 at 4:25 AM, Robert Haas wrote: > > > > On Thu, Apr 26, 2018 at 3:10 PM, Andres Freund

Re: widen vacuum buffer counters

2020-02-05 Thread Alvaro Herrera
On 2020-Jan-31, Tom Lane wrote: > Also, %zd is the wrong format code for int64. Recommended practice > these days is to use "%lld" with an explicit cast of the printf argument > to long long (just to be sure). That doesn't work safely before v12, > and if you did insist on back-patching

Re: Memory-Bounded Hash Aggregation

2020-02-05 Thread Jeff Davis
On Tue, 2020-02-04 at 18:10 +0200, Heikki Linnakangas wrote: > I'd love to change the LogicalTape API so that you could allocate > and > free tapes more freely. I wrote a patch to do that, as part of > replacing > tuplesort.c's polyphase algorithm with a simpler one (see [1]), but > I > never

Re: [Proposal] Global temporary tables

2020-02-05 Thread Pavel Stehule
st 5. 2. 2020 v 16:48 odesílatel Konstantin Knizhnik < k.knizh...@postgrespro.ru> napsal: > > > On 05.02.2020 17:10, Robert Haas wrote: > > On Wed, Feb 5, 2020 at 2:28 AM Konstantin Knizhnik > > wrote: > >> There is very important reason (from my point of view): allow other > >> sessions to use

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2020-02-05 Thread Mark Dilger
> On Feb 5, 2020, at 4:51 AM, Etsuro Fujita wrote: > > The patches apply and pass all tests. A review of the patch vs. master looks reasonable. The partition_join.sql test has multiple levels of partitioning, but when your patch extends that test with “advanced partition-wise join”,

Re: Memory-Bounded Hash Aggregation

2020-02-05 Thread Peter Geoghegan
On Wed, Feb 5, 2020 at 10:37 AM Jeff Davis wrote: > > LogicalTapeSetExtend() seems to work in a way that assumes that the > > tape is frozen. It would be good to document that assumption, and > > possible enforce it by way of an assertion. The same remark applies > > to > > any other assumptions

Re: Memory-Bounded Hash Aggregation

2020-02-05 Thread Jeff Davis
On Tue, 2020-02-04 at 15:08 -0800, Peter Geoghegan wrote: > Have you tested this against tuplesort.c, particularly parallel > CREATE > INDEX? It would be worth trying to measure any performance impact. > Note that most parallel CREATE INDEX tuplesorts will do a merge > within > each worker, and

Re: Memory-Bounded Hash Aggregation

2020-02-05 Thread Jeff Davis
On Tue, 2020-02-04 at 18:42 +0800, Adam Lee wrote: > The minheap looks good, have you tested the performance and aggregate > validation? Not sure exactly what you mean, but I tested the min heap with both Sort and HashAgg and it performs well. > About the "Cap the size of the LogicalTapeSet's

Re: Is custom MemoryContext prohibited?

2020-02-05 Thread Robert Haas
On Wed, Feb 5, 2020 at 12:15 PM Tom Lane wrote: > The core problem that I'm unhappy about is that what you want to do > adds cognitive burden (including an increased risk of mistakes) and > extra debugging commands in common cases around inspecting memory > contexts in a debugger. Sure, it's not

Re: Documentation patch for ALTER SUBSCRIPTION

2020-02-05 Thread Alvaro Herrera
OK, pushed that. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Is custom MemoryContext prohibited?

2020-02-05 Thread Tom Lane
Robert Haas writes: > I don't disagree with the factual statements that you are making but I > don't understand why any of them represent real problems. The core problem that I'm unhappy about is that what you want to do adds cognitive burden (including an increased risk of mistakes) and extra

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2020-02-05 Thread Mark Dilger
> On Feb 5, 2020, at 4:51 AM, Etsuro Fujita wrote: > > On Wed, Jan 29, 2020 at 9:15 PM Etsuro Fujita wrote: >> On Tue, Jan 28, 2020 at 1:39 PM Mark Dilger >> wrote: >>> I have added tests checking correctness and showing some partition pruning >>> limitations. Find three patches,

Re: Is custom MemoryContext prohibited?

2020-02-05 Thread Robert Haas
On Wed, Feb 5, 2020 at 10:56 AM Tom Lane wrote: > > Doesn't the proposed magic number address this concern? > > No, because (a) it will be a random magic number that nobody will > remember, and gdb won't print in any helpful form; (b) at least > as I understood the proposal, there'd be just one

Re: Index Skip Scan

2020-02-05 Thread Dmitry Dolgov
> On Tue, Feb 04, 2020 at 08:34:09PM +, Floris Van Nee wrote: > > > this point me and Jesper inclined to go with the second option. But maybe > > I'm missing something, are there any other suggestions? > > Unfortunately I figured this would need a more invasive fix. I tend to agree > that

Re: Add %x to PROMPT1 and PROMPT2

2020-02-05 Thread Fabien COELHO
+1 one for this change, it's something I also add to every .psqlrc I setup. So.. We have: +1: Vik, Ian, Daniel, Alvaro, Christoph +-0: Tom (?), Fabien (?) I did not know I had a vote. I'm "+1" on this change, if that matters. Just this morning I had a case where I wished I had the

Re: Add %x to PROMPT1 and PROMPT2

2020-02-05 Thread Magnus Hagander
On Wed, Feb 5, 2020 at 3:30 PM Robert Haas wrote: > > On Tue, Feb 4, 2020 at 3:20 AM Michael Paquier wrote: > > So.. We have: > > +1: Vik, Ian, Daniel, Alvaro, Christoph > > +-0: Tom (?), Fabien (?) > > -1: Michael P. > > I'm not really against this change but, given how long it's been the >

bad logging around broken restore_command

2020-02-05 Thread Jeff Janes
If the restore command claims to have succeeded, but fails to have created a file with the right name (due to typos or escaping or quoting issues, for example), no complaint is issued at the time, and it then fails later with a relatively uninformative error message like "could not locate required

Re: Is custom MemoryContext prohibited?

2020-02-05 Thread Tom Lane
Chapman Flack writes: > On 2/5/20 10:20 AM, Tom Lane wrote: >> * Third-party context types would have to force the compiler to take >> context-type values that weren't among the known enum values --- > Doesn't that seem like a long run for a short slide? Well, one thing we could do is assign an

Re: Is custom MemoryContext prohibited?

2020-02-05 Thread Tom Lane
Robert Haas writes: > I'm pretty unimpressed with the enum proposal - I think it's pretty > nasty for an extension author to have to make up a value that's not in > the enum. One, how are they supposed to know that they should do that? > Two, how are they supposed to know that the code doesn't

Re: Is custom MemoryContext prohibited?

2020-02-05 Thread Tom Lane
Robert Haas writes: > On Wed, Feb 5, 2020 at 10:09 AM Tom Lane wrote: >> an enum field right in the context. You'll have to do an extra step to >> discover the context's type, and if you jump to the wrong conclusion >> and do, say, >> p *(AllocSetContext *) ptr_value >> when it's really

Re: Is custom MemoryContext prohibited?

2020-02-05 Thread Chapman Flack
On 2/5/20 10:20 AM, Tom Lane wrote: > * Third-party context types would have to force the compiler to take > context-type values that weren't among the known enum values --- Doesn't that seem like a long run for a short slide? An extension author gets expected to do something

Re: [Proposal] Global temporary tables

2020-02-05 Thread Konstantin Knizhnik
On 05.02.2020 17:10, Robert Haas wrote: On Wed, Feb 5, 2020 at 2:28 AM Konstantin Knizhnik wrote: There is very important reason (from my point of view): allow other sessions to use created index and so provide compatible behavior with regular tables (and with Oracle). So we should be able

Re: Is custom MemoryContext prohibited?

2020-02-05 Thread Robert Haas
On Wed, Feb 5, 2020 at 10:20 AM Tom Lane wrote: > Good summary. I think that the combination of a magic number to identify > "this is a memory context struct" and an enum to identify the specific > type of context should meet all these goals moderately well: > > * Third-party context types would

Re: Is custom MemoryContext prohibited?

2020-02-05 Thread Robert Haas
On Wed, Feb 5, 2020 at 10:09 AM Tom Lane wrote: > an enum field right in the context. You'll have to do an extra step to > discover the context's type, and if you jump to the wrong conclusion > and do, say, > p *(AllocSetContext *) ptr_value > when it's really some other context type,

Re: Add %x to PROMPT1 and PROMPT2

2020-02-05 Thread Tom Lane
Robert Haas writes: > I'm not really against this change but, given how long it's been the > way that it is, I think we shouldn't make it without more plus votes. > If we've actually got a broad consensus on it, sure, but I don't think > 4 votes is a broad consensus. Fair point. I'm still

Re: Is custom MemoryContext prohibited?

2020-02-05 Thread Tom Lane
Robert Haas writes: > On Mon, Feb 3, 2020 at 7:26 PM Andres Freund wrote: >> It seems like a good idea to still have an additional identifier for >> each node type, for some cross checking. How about just frobbing the >> pointer to the MemoryContextMethod slightly, and storing that in an >>

Re: Is custom MemoryContext prohibited?

2020-02-05 Thread Robert Haas
On Wed, Feb 5, 2020 at 10:09 AM Tom Lane wrote: > So I really want to also have an enum > field of some sort, and it does not seem to me that there's any > compelling reason not to have one. I mean, then it's not extensible, right? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The

Re: [Patch] Optimize dropping of relation buffers using dlist

2020-02-05 Thread Robert Haas
On Tue, Feb 4, 2020 at 4:57 AM k.jami...@fujitsu.com wrote: > Kindly check the attached V6 patch. > Any thoughts on this? Unfortunately, I don't have time for detailed review of this. I am suspicious that there are substantial performance regressions that you just haven't found yet. I would not

Re: Is custom MemoryContext prohibited?

2020-02-05 Thread Tom Lane
Robert Haas writes: > Hmm. What if we put a "const char *name" in the methods array? Then > even if you couldn't print out the function pointers, you would at > least see the name. Yeah, that idea had occurred to me too. It'd definitely be better than relying on the ability to interpret

Re: Documentation patch for ALTER SUBSCRIPTION

2020-02-05 Thread David Christensen
> On Feb 5, 2020, at 7:56 AM, Alvaro Herrera wrote: > > On 2020-Feb-05, Amit Kapila wrote: > >> It is possible that one might not understand how this option works by >> reading the already existing text in docs, but I think writing in a >> different language the same thing also doesn't seem

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-02-05 Thread Julien Rouhaud
On Sat, Feb 1, 2020 at 12:30 PM Tomas Vondra wrote: > > On Fri, Nov 29, 2019 at 09:39:09AM +0100, Julien Rouhaud wrote: > >On Fri, Nov 29, 2019 at 7:21 AM Michael Paquier wrote: > >> > >> On Fri, Nov 29, 2019 at 03:19:49PM +0900, Michael Paquier wrote: > >> > On Wed, Nov 13, 2019 at 12:53:09PM

Re: Add %x to PROMPT1 and PROMPT2

2020-02-05 Thread Robert Haas
On Tue, Feb 4, 2020 at 3:20 AM Michael Paquier wrote: > So.. We have: > +1: Vik, Ian, Daniel, Alvaro, Christoph > +-0: Tom (?), Fabien (?) > -1: Michael P. I'm not really against this change but, given how long it's been the way that it is, I think we shouldn't make it without more plus votes.

Re: Is custom MemoryContext prohibited?

2020-02-05 Thread Robert Haas
On Mon, Feb 3, 2020 at 7:26 PM Andres Freund wrote: > Hm. I kinda like the idea of still having one NodeTag identifying memory > contexts, and then some additional field identifying the actual > type. Being able to continue to rely on IsA() etc imo is nice. I think > nodes.h itself only would be

Re: Is custom MemoryContext prohibited?

2020-02-05 Thread Robert Haas
On Tue, Jan 28, 2020 at 9:54 PM Tom Lane wrote: > Interesting. But I bet there are platform and version dependencies > in the mix, too. I'd still not wish to rely on this for debugging. Hmm. What if we put a "const char *name" in the methods array? Then even if you couldn't print out the

Re: [Proposal] Global temporary tables

2020-02-05 Thread Robert Haas
On Wed, Feb 5, 2020 at 8:21 AM 曾文旌(义从) wrote: > What do you mean by "catalog buffer"? > Yes, cleanup of local temp table requires deletion of correspondent entry > from catalog and GTT should not do it. > But I am speaking only about cleanup of data files of temp relations. It is > done in the

Re: [Proposal] Global temporary tables

2020-02-05 Thread Robert Haas
On Wed, Feb 5, 2020 at 2:28 AM Konstantin Knizhnik wrote: > There is very important reason (from my point of view): allow other > sessions to use created index and > so provide compatible behavior with regular tables (and with Oracle). > So we should be able to populate index with existed GTT

Re: Documentation patch for ALTER SUBSCRIPTION

2020-02-05 Thread Alvaro Herrera
On 2020-Feb-05, Amit Kapila wrote: > It is possible that one might not understand how this option works by > reading the already existing text in docs, but I think writing in a > different language the same thing also doesn't seem advisable. I > think if we want to explain it better, then maybe

WAL usage calculation patch

2020-02-05 Thread Kirill Bychik
Hello pgsql-hackers, Submitting a patch that would enable gathering of per-statement WAL generation statistics, similar to how it is done for buffer usage. Collected is the number of records added to WAL and number of WAL bytes written. The data collected was found valuable to analyze

Re: Internal key management system

2020-02-05 Thread Sehrope Sarkuni
On Sat, Feb 1, 2020 at 7:02 PM Masahiko Sawada < masahiko.saw...@2ndquadrant.com> wrote: > On Sun, 2 Feb 2020 at 00:37, Sehrope Sarkuni wrote: > > > > On Fri, Jan 31, 2020 at 1:21 AM Masahiko Sawada > > wrote: > > > On Thu, 30 Jan 2020 at 20:36, Sehrope Sarkuni wrote: > > > > That > > > > would

Re: [Proposal] Global temporary tables

2020-02-05 Thread 曾文旌(义从)
> 2020年2月5日 上午12:47,Konstantin Knizhnik 写道: > > > > On 04.02.2020 18:01, 曾文旌(义从) wrote: >> >> >>> >>> Yes, exactly. >>> But it is still not clear to me why do we need some special handling for >>> GTT? >>> Shared memory is reinitialized and storage of temporary tables is removed. >>> It

Re: [PATCH v1] Allow COPY "text" format to output a header

2020-02-05 Thread Rémi Lapeyre
> > FWIW there was more recent propose patch at > https://www.postgresql.org/message-id/flat/caf1-j-0ptcwmeltswwgv2m70u26n4g33gpe1rckqqe6wvqd...@mail.gmail.com > and among feedback given is to adding header matching feature on to this. Thanks for the feedback. What should happen now? Can I

Re: BUG #15858: could not stat file - over 4GB

2020-02-05 Thread Emil Iggland
The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: tested, passed Spec compliant: not tested Documentation:not tested I ran into this problem when using psql.exe and copy command. I have

Identifying user-created objects

2020-02-05 Thread Masahiko Sawada
Hi, User can create database objects such as functions into pg_catalog. But if I'm not missing something, currently there is no straightforward way to identify if the object is a user created object or a system object which is created during initdb. If we can do that user will be able to check if

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-02-05 Thread Amit Kapila
On Wed, Feb 5, 2020 at 9:46 AM Dilip Kumar wrote: > > Fixed in the latest version sent upthread. > Okay, thanks. I haven't looked at the latest version of patch series as I was reviewing the previous version and I think all of these comments are in the patch which is not modified. Here are my

Re: Complete data erasure

2020-02-05 Thread Masahiko Sawada
On Tue, 4 Feb 2020 at 09:53, tsunakawa.ta...@fujitsu.com wrote: > > From: Tomas Vondra > > That's not really what I meant - let me explain. When I said DROP TABLE > > should do everything as usual, that includes catalog changes. I.e. after > > the commit there would not be any remaining entries

Re: Implementing Incremental View Maintenance

2020-02-05 Thread Yugo NAGATA
On Tue, 4 Feb 2020 18:40:45 +0900 nuko yokohama wrote: > "ROW LEVEL SECURITY" and INCREMENTAL MATERIALIZED VIEW. > > Hi. > > If ROW LEVEL SECURITY is set for the source table after creating the > INCREMENTAL MATELIALIZED VIEW, the search results by that are not reflected. > After setting ROW

Re: [Proposal] Global temporary tables

2020-02-05 Thread 曾文旌(义从)
> 2020年2月5日 上午4:57,Robert Haas 写道: > > On Sat, Feb 1, 2020 at 11:14 AM 曾文旌(义从) wrote: >> As global_private_temp-8.patch, think about: >> 1 session X tale several hours doing some statistical work with the GTT A, >> which generated some data using transaction 100, The work is not over. >> 2

Assumptions about the number of parallel workers

2020-02-05 Thread Antonin Houska
I can't figure out why ExecGather/ExecGatherMerge do check whether num_workers is non-zero. I think the code would be a bit clearer if these tests were replaced with Assert() statements, as the attached patch does. In addition, if my assumptions are correct, I think that only Gather node needs

Re: pg_stat_progress_basebackup - progress reporting for pg_basebackup, in the server side

2020-02-05 Thread Amit Langote
On Wed, Feb 5, 2020 at 5:32 PM Kyotaro Horiguchi wrote: > At Wed, 5 Feb 2020 16:29:54 +0900, Amit Langote > wrote in > > On Wed, Feb 5, 2020 at 3:36 PM Fujii Masao > > wrote: > > > On 2020/02/04 10:34, Amit Langote wrote: > > > > I like: > > > > > > Thanks for reviewing the patch! > > > > > >

Re: ALTER tbl rewrite loses CLUSTER ON index

2020-02-05 Thread Justin Pryzby
On Wed, Feb 05, 2020 at 03:53:45PM +0900, Amit Langote wrote: > Hi Justin, > > On Mon, Feb 3, 2020 at 1:17 AM Justin Pryzby wrote: > > Other options are preserved by ALTER (and CLUSTER ON is and most obviously > > should be preserved by CLUSTER's rewrite), so I think (SET) CLUSTER should > > be

Re: pg_stat_progress_basebackup - progress reporting for pg_basebackup, in the server side

2020-02-05 Thread Kyotaro Horiguchi
At Wed, 5 Feb 2020 16:29:54 +0900, Amit Langote wrote in > On Wed, Feb 5, 2020 at 3:36 PM Fujii Masao > wrote: > > On 2020/02/04 10:34, Amit Langote wrote: > > > I like: > > > > Thanks for reviewing the patch! > > > > > 1. initializing > > > 2-5 waiting for backup start checkpoint to finish >

Re: A bug in LWLOCK_STATS

2020-02-05 Thread Kyotaro Horiguchi
At Wed, 5 Feb 2020 14:43:49 +0900, Fujii Masao wrote in > The cause of this issue is that the key variable used for lwlock hash > table was not fully initialized. The key consists of two fields and > they are initialized as follows. But the following 4 bytes allocated > for the alignment was

Re: Documentation patch for ALTER SUBSCRIPTION

2020-02-05 Thread Amit Kapila
On Wed, Feb 5, 2020 at 8:44 AM David Christensen wrote: > > >> On Feb 4, 2020, at 8:45 PM, Amit Kapila wrote: > >> > >> On Fri, Jan 24, 2020 at 2:05 AM David Christensen > >> wrote: > >> Greetings, > >> Enclosed find a documentation patch that clarifies the behavior of ALTER > >>

Re: A bug in LWLOCK_STATS

2020-02-05 Thread Julien Rouhaud
On Wed, Feb 05, 2020 at 02:43:49PM +0900, Fujii Masao wrote: > Hi, > > When I compiled PostgreSQL with -DLWLOCK_STATS and tried to check > the statistics of light-weight locks, I observed that more than one > statistics entries were output *for the same backend process and > the same lwlock*. For