Re: pl/pgsql feature request: shorthand for argument and local variable references

2021-03-17 Thread Michael Paquier
On Wed, Mar 17, 2021 at 02:06:57PM +0800, Julien Rouhaud wrote: > I also think that there should be a single usable top label, otherwise it will > lead to confusing code and it can be a source of bug. Okay, that's fine by me. Could it be possible to come up with an approach that does not hijack

Re: Type of wait events WalReceiverWaitStart and WalSenderWaitForWAL

2021-03-17 Thread Kyotaro Horiguchi
At Wed, 17 Mar 2021 15:31:37 +0900 (JST), Kyotaro Horiguchi wrote in > WAIT_EVENT_WAL_RECEIVER_MAIN(WalReceiverMain) is waiting for new data > to arrive. This looks like an activity to me. > > WAIT_EVENT_WAL_RECEIVER_WAIT_START is waiting for waiting for starup > process to kick me. So it

Re: Code comment fix

2021-03-17 Thread Michael Paquier
On Wed, Mar 17, 2021 at 08:31:16AM +0100, Vik Fearing wrote: > When table oids were removed by commit 578b229718e, the function > CatalogTupleInsert() was modified to return void but its comment was > left intact. Here is a trivial patch to fix that. Thanks, Vik. Good catch. I'll fix that in a

Re: pl/pgsql feature request: shorthand for argument and local variable references

2021-03-17 Thread Julien Rouhaud
On Wed, Mar 17, 2021 at 05:30:30AM +0100, Pavel Stehule wrote: > st 17. 3. 2021 v 4:52 odesílatel Michael Paquier > > > I am wondering whether it would be better to allow multiple aliases > > though, and if it would bring more readability to the routines written > > if these are treated equal to

Re: Type of wait events WalReceiverWaitStart and WalSenderWaitForWAL

2021-03-17 Thread Kyotaro Horiguchi
At Tue, 16 Mar 2021 15:42:27 +0900, Fujii Masao wrote in > > > On 2021/03/16 11:59, Kyotaro Horiguchi wrote: > > At Tue, 16 Mar 2021 03:12:54 +0900, Fujii Masao > > wrote in > >> The wait event WalReceiverWaitStart has been categorized in the type > >> Client. > >> But why? Walreceiver is

RE: [PATCH] pgbench: improve \sleep meta command

2021-03-17 Thread kuroda.hay...@fujitsu.com
Dear Fujii-san, Thank you for updating the patch. I understand that you don't want to change the current specification. ```diff + if (usec == 0) + { + char *c = var; + + /* Skip sign */ + if (*c

Code comment fix

2021-03-17 Thread Vik Fearing
When table oids were removed by commit 578b229718e, the function CatalogTupleInsert() was modified to return void but its comment was left intact. Here is a trivial patch to fix that. -- Vik Fearing diff --git a/src/backend/catalog/indexing.c b/src/backend/catalog/indexing.c index

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-17 Thread Peter Smith
On Fri, Mar 12, 2021 at 8:38 PM vignesh C wrote: > ... > 1) I felt twophase_given can be a local variable, it need not be added > as a function parameter as it is not used outside the function. > --- a/src/backend/commands/subscriptioncmds.c > +++ b/src/backend/commands/subscriptioncmds.c > @@

Re: PITR promote bug: Checkpointer writes to older timeline

2021-03-17 Thread Michael Paquier
On Mon, Mar 15, 2021 at 04:38:08PM +0900, Michael Paquier wrote: > On Mon, Mar 15, 2021 at 03:01:09PM +0900, Kyotaro Horiguchi wrote: >> Logical decoding stuff is (I think) designed to turn any backend into >> a walsender, which may need to maintain ThisTimeLineID. It seems to >> me that logical

Re: Getting better results from valgrind leak tracking

2021-03-17 Thread Andres Freund
Hi, On Wed, Mar 17, 2021, at 07:16, Tom Lane wrote: > Andres Freund writes: > > On 2021-03-16 20:50:17 -0700, Andres Freund wrote: > Meanwhile, I'm still trying to understand why valgrind is whining > about the rd_indexcxt identifier strings. AFAICS it shouldn't. I found a way around that late

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-17 Thread Bruce Momjian
On Sun, Mar 14, 2021 at 04:06:45PM +0800, Julien Rouhaud wrote: > Recent conflict, thanks to cfbot. v18 attached. We are reaching the two-year mark on this feature, that everyone seems to agree is needed. Is any committer going to work on this to get it into PG 14? Should I take it? I just

Re: Calendar support in localization

2021-03-17 Thread Surafel Temesgen
On Tue, Mar 16, 2021 at 12:20 PM Thomas Munro wrote: > On Wed, Mar 17, 2021 at 6:31 AM Surafel Temesgen > wrote: > > Ethiopice calendar have 13 months so it can not be stored as date and > timestamp type and you approach seems more complicated and i suggest to > have this feature on the purpose

RE: libpq debug log

2021-03-17 Thread iwata....@fujitsu.com
Hi Tsunakawa san, Alvaro san, Thank you very much for your review. It helped me a lot to make the patch better. I update patch to v26. This patch has been updated according to Tsunakawa san and Alvaro san review comments. The output is following; ``` 2021-03-17 14:43:16.411238 > Terminate

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-17 Thread Tom Lane
Bruce Momjian writes: > We are reaching the two-year mark on this feature, that everyone seems > to agree is needed. Is any committer going to work on this to get it > into PG 14? Should I take it? I still say that it's a serious mistake to sanctify a query ID calculation method that was

Re: Calendar support in localization

2021-03-17 Thread Robert Haas
On Wed, Mar 17, 2021 at 9:54 AM Surafel Temesgen wrote: > As you mention above whatever the calendar type is we ended up storing an > integer that represent the date so rather than re-implementing every > function and operation for every calendar we can use existing Gerigorian >

Re: Getting better results from valgrind leak tracking

2021-03-17 Thread Tom Lane
Andres Freund writes: > On 2021-03-16 20:50:17 -0700, Andres Freund wrote: >> What I meant was that I didn't understand how there's not a leak >> danger when compilation fails halfway through, given that the context >> in question is below TopMemoryContext and that I didn't see a relevant >> TRY

Re: Get memory contexts of an arbitrary backend process

2021-03-17 Thread torikoshia
On 2021-03-05 14:22, Fujii Masao wrote: On 2021/03/04 18:32, torikoshia wrote: On 2021-01-14 19:11, torikoshia wrote: Since pg_get_target_backend_memory_contexts() waits to dump memory and it could lead dead lock as below.   - session1   BEGIN; TRUNCATE t;   - session2   BEGIN; TRUNCATE t;

Re: pgsql: Add libpq pipeline mode support to pgbench

2021-03-17 Thread Daniel Verite
Fabien COELHO wrote: > For consistency with the existing \if … \endif, ISTM that it could have > been named \batch … \endbatch or \pipeline … \endpipeline? "start" mirrors "end". To me, the analogy with \if-\endif is not obvious. Grammatically \if is meant to introduce the expression

Re: Calendar support in localization

2021-03-17 Thread Tom Lane
Robert Haas writes: > It's not very obvious how to scale this kind of approach to a wide > variety of calendar types, and as Thomas says, it would much cooler to > be able to handle all of the ones that ICU knows how to support rather > than just one. But, the problem I see with using timestamptz

Re: logical replication worker accesses catalogs in error context callback

2021-03-17 Thread Bharath Rupireddy
On Tue, Mar 16, 2021 at 2:21 AM Tom Lane wrote: > > Thanks for pointing to the changes in the commit message. I corrected > > them. Attaching v4 patch set, consider it for further review. > > I took a quick look at this. I'm quite worried about the potential > performance cost of the v4-0001

Re: Assertion failure with barriers in parallel hash join

2021-03-17 Thread Thomas Munro
On Wed, Mar 17, 2021 at 6:58 PM Thomas Munro wrote: > According to BF animal elver there is something wrong with this > commit. Looking into it. Assertion failure reproduced here and understood, but unfortunately it'll take some more time to fix this. I've reverted the commit for now to

Re: Code comment fix

2021-03-17 Thread Vik Fearing
On 3/17/21 9:11 AM, Michael Paquier wrote: > On Wed, Mar 17, 2021 at 08:31:16AM +0100, Vik Fearing wrote: >> When table oids were removed by commit 578b229718e, the function >> CatalogTupleInsert() was modified to return void but its comment was >> left intact. Here is a trivial patch to fix

Re: Should we improve "PID XXXX is not a PostgreSQL server process" warning for pg_terminate_backend(<>)?

2021-03-17 Thread Bharath Rupireddy
On Wed, Mar 17, 2021 at 8:05 AM torikoshia wrote: > > I have not gone through that thread though. Is there any way we can > > detect those child processes(stats collector, sys logger) that are > > forked by the postmaster from a backend process? Thoughts? > > I couldn't find good ways to do that,

Re: fdatasync performance problem with large number of DB files

2021-03-17 Thread Paul Guo
On Wed, Mar 17, 2021 at 11:45 AM Thomas Munro wrote: > > On Tue, Mar 16, 2021 at 9:29 PM Fujii Masao > wrote: > > On 2021/03/16 8:15, Thomas Munro wrote: > > > I don't want to add a hypothetical sync_after_crash=none, because it > > > seems like generally a bad idea. We already have a > > >

RE: Parallel Inserts in CREATE TABLE AS

2021-03-17 Thread houzj.f...@fujitsu.com
> > Seems like v22 patch was failing in cfbot for one of the unstable test > > cases. > > Attaching v23 patch set with modification in 0003 and 0004 patches. No > > changes to 0001 and 0002 patches. Hopefully cfbot will be happy with v23. > > > > Please consider v23 for further review. > Hi, >

Re: a verbose option for autovacuum

2021-03-17 Thread Euler Taveira
On Wed, Mar 10, 2021, at 12:46 AM, Masahiko Sawada wrote: > Attached a patch. I've slightly modified the format for consistency > with heap statistics. Since commit 5f8727f5a6, this patch doesn't apply anymore. Fortunately, it is just a small hunk. I reviewed this patch and it looks good to me.

Re: pgsql: Add libpq pipeline mode support to pgbench

2021-03-17 Thread Fabien COELHO
Bonjour Daniel, Ola Alvaro, Add libpq pipeline mode support to pgbench New metacommands \startpipeline and \endpipeline allow the user to run queries in libpq pipeline mode. Author: Daniel Vérité Reviewed-by: Álvaro Herrera Discussion:

Re: A new function to wait for the backend exit after termination

2021-03-17 Thread Zhihong Yu
Hi, w.r.t. WaitLatch(), if its return value is WL_TIMEOUT, we know the specified timeout has elapsed. It seems WaitLatch() can be enhanced to also return the actual duration of the wait. This way, the caller can utilize the duration directly. As for other places where WaitLatch() is called,

Re: proposal: schema variables - doc

2021-03-17 Thread Erik Rijkers
> On 2021.03.13. 07:01 Pavel Stehule wrote: > Hi > fresh rebase > [schema-variables-20210313.patch.gz] Hi Pavel, I notice that the phrase 'schema variable' is not in the index at the end ('bookindex.html'). Not good. It is also not in the index at the front of the manual - also not good.

Re: A new function to wait for the backend exit after termination

2021-03-17 Thread Bharath Rupireddy
On Wed, Mar 17, 2021 at 8:28 AM Kyotaro Horiguchi wrote: > > At Wed, 17 Mar 2021 07:01:39 +0530, Bharath Rupireddy > wrote in > > Attaching v10 patch for further review. > > The time-out mechanism doesn't count remainingtime as expected, > concretely it does the following. > > do { > kill();

Re: Transactions involving multiple postgres foreign servers, take 2

2021-03-17 Thread Zhihong Yu
Hi, For v35-0007-Prepare-foreign-transactions-at-commit-time.patch : With this commit, the foreign server modified within the transaction marked as 'modified'. transaction marked -> transaction is marked +#define IsForeignTwophaseCommitRequested() \ +(foreign_twophase_commit >

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-17 Thread Pavel Stehule
st 17. 3. 2021 v 17:03 odesílatel Tom Lane napsal: > Bruce Momjian writes: > > On Wed, Mar 17, 2021 at 11:28:38AM -0400, Tom Lane wrote: > >> I still say that it's a serious mistake to sanctify a query ID > calculation > >> method that was designed only for pg_stat_statement's needs as the one

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2021-03-17 Thread Vik Fearing
On 3/17/21 6:00 PM, Lætitia Avrot wrote: > Hey hackers, > > I had this idea, that I raised and cherished like my baby to add a switch > in `pg_dump` to allow exporting stored functions (and procedures) only. > > However, when I finally got the time to look at it in detail, I found out > there

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2021-03-17 Thread Fabrízio de Royes Mello
On Wed, Mar 17, 2021 at 2:00 PM Lætitia Avrot wrote: > > Hey hackers, > > I had this idea, that I raised and cherished like my baby to add a switch in `pg_dump` to allow exporting stored functions (and procedures) only. > > However, when I finally got the time to look at it in detail, I found out

Re: [HACKERS] Custom compression methods

2021-03-17 Thread Robert Haas
On Wed, Mar 17, 2021 at 7:41 AM Dilip Kumar wrote: > 0002: > - Wrapper over heap_form_tuple and used in ExecEvalRow() and > ExecEvalFieldStoreForm() Instead of having heap_form_flattened_tuple(), how about heap_flatten_values(tupleDesc, values, isnull) that is documented to modify the values

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-03-17 Thread Alvaro Herrera
On 2021-Mar-15, Alvaro Herrera wrote: > Here's a fixup patch to do it that way. I tried running the commands > you showed and one of them immediately dies with the new error message; > I can't cause the bogus constraint to show up anymore. Actually, that was a silly fix that didn't actually

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-17 Thread Bruce Momjian
On Wed, Mar 17, 2021 at 11:28:38AM -0400, Tom Lane wrote: > Bruce Momjian writes: > > We are reaching the two-year mark on this feature, that everyone seems > > to agree is needed. Is any committer going to work on this to get it > > into PG 14? Should I take it? > > I still say that it's a

Re: pl/pgsql feature request: shorthand for argument and local variable references

2021-03-17 Thread Pavel Stehule
st 17. 3. 2021 v 9:20 odesílatel Michael Paquier napsal: > On Wed, Mar 17, 2021 at 02:06:57PM +0800, Julien Rouhaud wrote: > > I also think that there should be a single usable top label, otherwise > it will > > lead to confusing code and it can be a source of bug. > > Okay, that's fine by me.

Re: WIP: WAL prefetch (another approach)

2021-03-17 Thread Tomas Vondra
On 2/15/21 12:18 AM, Stephen Frost wrote: > Greetings, > > ... > I think there's potential for some significant optimization going forward, but I think it's basically optimization over what we're doing today. As this is already a nontrivial patch, I'd argue for doing so

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-17 Thread Julien Rouhaud
On Wed, Mar 17, 2021 at 12:24:44PM -0400, Bruce Momjian wrote: > On Wed, Mar 17, 2021 at 05:16:50PM +0100, Pavel Stehule wrote: > > > > > > st 17. 3. 2021 v 17:03 odesílatel Tom Lane napsal: > > > > Bruce Momjian writes: > > > On Wed, Mar 17, 2021 at 11:28:38AM -0400, Tom Lane wrote:

Re: PoC/WIP: Extended statistics on expressions

2021-03-17 Thread Tomas Vondra
Hi, On 3/17/21 4:55 PM, Dean Rasheed wrote: > On Sun, 7 Mar 2021 at 21:10, Tomas Vondra > wrote: >> >> 2) ndistinct >> >> There's one thing that's bugging me, in how we handle "partial" matches. >> For each expression we track both the original expression and the Vars >> we extract from it. If

Re: Index Skip Scan (new UniqueKeys)

2021-03-17 Thread Tomas Vondra
On 3/17/21 6:02 PM, Dmitry Dolgov wrote: >> On Wed, Mar 17, 2021 at 03:28:00AM +0100, Tomas Vondra wrote: >> Hi, >> >> I took a look at the new patch series, focusing mostly on the uniquekeys >> part. It'd be a bit tedious to explain all the review comments here, so >> attached is a patch

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-17 Thread Bruce Momjian
On Wed, Mar 17, 2021 at 12:01:38PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Wed, Mar 17, 2021 at 11:28:38AM -0400, Tom Lane wrote: > >> I still say that it's a serious mistake to sanctify a query ID calculation > >> method that was designed only for pg_stat_statement's needs as the

Re: libpq debug log

2021-03-17 Thread Alvaro Herrera
Hello In pqTraceOutputString(), you can use the return value from fprintf to move the cursor -- no need to count chars. I still think that the message-type specific functions should print the message type, rather than having the string arrays. -- Álvaro Herrera Valdivia, Chile "La gente

Re: Index Skip Scan (new UniqueKeys)

2021-03-17 Thread Dmitry Dolgov
> On Wed, Mar 17, 2021 at 03:28:00AM +0100, Tomas Vondra wrote: > Hi, > > I took a look at the new patch series, focusing mostly on the uniquekeys > part. It'd be a bit tedious to explain all the review comments here, so > attached is a patch series with a "review" patch for some of the parts.

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2021-03-17 Thread Tom Lane
Vik Fearing writes: > On 3/17/21 6:00 PM, Lætitia Avrot wrote: >> However, when I finally got the time to look at it in detail, I found out >> there was no way to solve the dependencies in the functions and procedures, >> so that the exported file, when re-played could lead to invalid objects. >>

Re: PoC/WIP: Extended statistics on expressions

2021-03-17 Thread Dean Rasheed
On Sun, 7 Mar 2021 at 21:10, Tomas Vondra wrote: > > 2) ndistinct > > There's one thing that's bugging me, in how we handle "partial" matches. > For each expression we track both the original expression and the Vars > we extract from it. If we can't find a statistics matching the whole >

pg_dump new feature: exporting functions only. Bad or good idea ?

2021-03-17 Thread Lætitia Avrot
Hey hackers, I had this idea, that I raised and cherished like my baby to add a switch in `pg_dump` to allow exporting stored functions (and procedures) only. However, when I finally got the time to look at it in detail, I found out there was no way to solve the dependencies in the functions and

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-17 Thread Tom Lane
Bruce Momjian writes: > On Wed, Mar 17, 2021 at 11:28:38AM -0400, Tom Lane wrote: >> I still say that it's a serious mistake to sanctify a query ID calculation >> method that was designed only for pg_stat_statement's needs as the one >> true way to do it. But that's what exposing it in a core

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-17 Thread Bruce Momjian
On Wed, Mar 17, 2021 at 05:16:50PM +0100, Pavel Stehule wrote: > > > st 17. 3. 2021 v 17:03 odesílatel Tom Lane napsal: > > Bruce Momjian writes: > > On Wed, Mar 17, 2021 at 11:28:38AM -0400, Tom Lane wrote: > >> I still say that it's a serious mistake to sanctify a query ID >

Re: [patch] [doc] Minor variable related cleanup and rewording of plpgsql docs

2021-03-17 Thread Tom Lane
I wrote: > I took a stab at doing that, just to see what it might look like. > I thought it comes out pretty well, really -- see what you think. Hearing nothing further, I pushed that after another round of copy-editing. There's still plenty of time to revise it if anybody has further comments.

Re: WIP: BRIN multi-range indexes

2021-03-17 Thread John Naylor
On Wed, Mar 17, 2021 at 3:16 PM Tomas Vondra wrote: > Ummm, no attachment ;-) Oops, here it is. -- John Naylor EDB: http://www.enterprisedb.com jcn-costing-test.sql Description: Binary data

Re: Getting better results from valgrind leak tracking

2021-03-17 Thread Tom Lane
Andres Freund writes: >> I found a way around that late last night. Need to mark the context >> itself as an allocation. But I made a mess on the way to that and need >> to clean the patch up before sending it (and need to drop my >> girlfriend off first). > Unfortunately I didn't immediately

Re: Calendar support in localization

2021-03-17 Thread Thomas Munro
On Thu, Mar 18, 2021 at 3:48 AM Tom Lane wrote: > Robert Haas writes: > > It's not very obvious how to scale this kind of approach to a wide > > variety of calendar types, and as Thomas says, it would much cooler to > > be able to handle all of the ones that ICU knows how to support rather > >

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-17 Thread Robert Haas
On Wed, Mar 17, 2021 at 12:48 PM Julien Rouhaud wrote: > I originally suggested to make it clearer by having an enum GUC rather than a > boolean, say compute_queryid = [ none | core | external ], and if set to > external then a hook would be explicitely called. Right now, "none" and > "external"

Re: non-HOT update not looking at FSM for large tuple update

2021-03-17 Thread John Naylor
On Fri, Mar 12, 2021 at 8:45 AM Matthias van de Meent < boekewurm+postg...@gmail.com> wrote: > > If this case isn't added, the lower else branch will fail to find > fitting pages for len > maxPaddedFsmRequest tuples; potentially > extending the relation when there is actually still enough space >

Re: PoC/WIP: Extended statistics on expressions

2021-03-17 Thread Dean Rasheed
On Wed, 17 Mar 2021 at 20:48, Dean Rasheed wrote: > > For reference, here is the test case I was using (which isn't really very > good for > catching dependence between columns): > And here's a test case with much more dependence between the columns: DROP TABLE IF EXISTS foo; CREATE TABLE foo

Re: PoC/WIP: Extended statistics on expressions

2021-03-17 Thread Tomas Vondra
On 3/17/21 9:58 PM, Dean Rasheed wrote: > On Wed, 17 Mar 2021 at 20:48, Dean Rasheed wrote: >> >> For reference, here is the test case I was using (which isn't really very >> good for >> catching dependence between columns): >> > > And here's a test case with much more dependence between the

Re: WIP: WAL prefetch (another approach)

2021-03-17 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: > Right, I was just going to point out the FPIs are not necessary - what > matters is the presence of long streaks of WAL records touching the same > set of blocks. But people with workloads where this is common likely > don't need

Re: [HACKERS] Custom compression methods

2021-03-17 Thread Andres Freund
Hi, On 2021-03-17 16:01:58 -0400, Robert Haas wrote: > > - why is HIDE_TOAST_COMPRESSION useful? Doesn't quite seem to be > > comparable to HIDE_TABLEAM? > > Andres, what do you mean by this exactly? It's exactly the same issue: > without this, if you change the default compression method,

Re: pl/pgsql feature request: shorthand for argument and local variable references

2021-03-17 Thread Hannu Krosing
why are you using yet another special syntax for this ? would it not be better to do something like this: CREATE FUNCTION a_reall_long_and_winding_function_name(i int, out o int) LANGUAGE plpgsql AS $plpgsql$ DECLARE args function_name_alias BEGIN args.o = 2 * args.i END; $plpgsql$; or at

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-17 Thread Bruce Momjian
On Wed, Mar 17, 2021 at 04:04:44PM -0400, Robert Haas wrote: > On Wed, Mar 17, 2021 at 12:48 PM Julien Rouhaud wrote: > > I originally suggested to make it clearer by having an enum GUC rather than > > a > > boolean, say compute_queryid = [ none | core | external ], and if set to > > external

Re: pl/pgsql feature request: shorthand for argument and local variable references

2021-03-17 Thread Michael Paquier
On Wed, Mar 17, 2021 at 05:04:48PM +0100, Pavel Stehule wrote: > This tree has a different direction than is usual, and then replacing the > root node is not simple. Yeah, it is not like we should redesign this whole part just for the feature discussed here, and that may impact performance as the

Re: Getting better results from valgrind leak tracking

2021-03-17 Thread Andres Freund
Hi, (really need to fix my mobile phone mail program to keep the CC list...) On 2021-03-17 08:15:43 -0700, Andres Freund wrote: > On Wed, Mar 17, 2021, at 07:16, Tom Lane wrote: > > Andres Freund writes: > > > On 2021-03-16 20:50:17 -0700, Andres Freund wrote: > > Meanwhile, I'm still trying to

Re: WIP: BRIN multi-range indexes

2021-03-17 Thread John Naylor
On Thu, Mar 11, 2021 at 12:26 PM Tomas Vondra wrote: > > Hi, > > Here is an updated version of the patch series. > > It fixes the assert failure (just remove the multiplication from it) and > adds a simple regression test that exercises this. > > Based on the discussion so far, I've decided to

Re: PoC/WIP: Extended statistics on expressions

2021-03-17 Thread Tomas Vondra
On 3/17/21 7:54 PM, Dean Rasheed wrote: > On Wed, 17 Mar 2021 at 17:26, Tomas Vondra > wrote: >> >> My concern is that the current behavior (where we prefer expression >> stats over multi-column stats to some extent) works fine as long as the >> parts are independent, but once there's

Re: WIP: BRIN multi-range indexes

2021-03-17 Thread Tomas Vondra
On 3/17/21 7:59 PM, John Naylor wrote: > On Thu, Mar 11, 2021 at 12:26 PM Tomas Vondra > mailto:tomas.von...@enterprisedb.com>> > wrote: >> >> Hi, >> >> Here is an updated version of the patch series. >> >> It fixes the assert failure (just remove the multiplication from it) and >> adds a

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-03-17 Thread Justin Pryzby
The v8 patch has the "broken constraint" problem. Also, it "fails to avoid" adding duplicate constraints: Check constraints: "c" CHECK (i IS NOT NULL AND i > 1 AND i < 2) "cc" CHECK (i IS NOT NULL AND i >= 1 AND i < 2) "p1_check" CHECK (true) "p1_i_check" CHECK (i IS NOT NULL AND

Re: [HACKERS] Custom compression methods

2021-03-17 Thread Robert Haas
).On Mon, Mar 15, 2021 at 6:58 PM Andres Freund wrote: > - Adding all these indirect function calls via toast_compression[] just > for all of two builtin methods isn't fun either. Yeah, it feels like this has too many layers of indirection now. Like, toast_decompress_datum() first gets

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2021-03-17 Thread Simon Riggs
On Fri, 12 Mar 2021 at 22:16, Tomas Vondra wrote: > > On 1/28/21 2:33 PM, Simon Riggs wrote: > > On Thu, 28 Jan 2021 at 12:53, Masahiko Sawada wrote: > > > >> This entry has been "Waiting on Author" status and the patch has not > >> been updated since Nov 30. Are you still planning to work on

Re: PoC/WIP: Extended statistics on expressions

2021-03-17 Thread Dean Rasheed
On Wed, 17 Mar 2021 at 19:07, Tomas Vondra wrote: > > On 3/17/21 7:54 PM, Dean Rasheed wrote: > > > > it might have been better to estimate the first case as > > > > ndistinct((a+b)) * ndistinct(c) * ndistinct(d) > > > > and the second case as > > > > ndistinct((a+b)) * ndistinct((c+d))

Re: [HACKERS] Custom compression methods

2021-03-17 Thread Andres Freund
Hi, On 2021-03-17 13:31:14 -0400, Robert Haas wrote: > On Wed, Mar 17, 2021 at 7:41 AM Dilip Kumar wrote: > > 0002: > > - Wrapper over heap_form_tuple and used in ExecEvalRow() and > > ExecEvalFieldStoreForm() > > Instead of having heap_form_flattened_tuple(), how about >

Re: PoC/WIP: Extended statistics on expressions

2021-03-17 Thread Dean Rasheed
On Wed, 17 Mar 2021 at 17:26, Tomas Vondra wrote: > > My concern is that the current behavior (where we prefer expression > stats over multi-column stats to some extent) works fine as long as the > parts are independent, but once there's dependency it's probably more > likely to produce

Re: [HACKERS] Custom compression methods

2021-03-17 Thread Robert Haas
On Wed, Mar 17, 2021 at 2:17 PM Andres Freund wrote: > OTOH heap_form_flattened_tuple() has the advantage that we can optimize > it further (e.g. to do the conversion to flattened values in fill_val()) > without changing the outside API. Well, in my view, that does change the outside API,

Re: replication slot stats memory bug

2021-03-17 Thread Tom Lane
Andres Freund writes: > I saw a leak in pgstat_read_statsfiles(), more precisely: > /* Allocate the space for replication slot statistics */ > replSlotStats = palloc0(max_replication_slots * > sizeof(PgStat_ReplSlotStats)); Yeah, I just found that myself. I think your fix is good.

Re: Getting better results from valgrind leak tracking

2021-03-17 Thread Andres Freund
Hi, On 2021-03-17 18:07:36 -0400, Tom Lane wrote: > Andres Freund writes: > >> I found a way around that late last night. Need to mark the context > >> itself as an allocation. But I made a mess on the way to that and need > >> to clean the patch up before sending it (and need to drop my > >>

replication slot stats memory bug

2021-03-17 Thread Andres Freund
Hi, in the course of https://postgr.es/m/3471359.1615937770%40sss.pgh.pa.us I saw a leak in pgstat_read_statsfiles(), more precisely: /* Allocate the space for replication slot statistics */ replSlotStats = palloc0(max_replication_slots * sizeof(PgStat_ReplSlotStats)); the issue

Re: replication slot stats memory bug

2021-03-17 Thread Andres Freund
Hi, On 2021-03-17 16:04:47 -0700, Andres Freund wrote: > I'll push the minimal fix of forcing the allocation to happen in > pgStatLocalContext and setting it to NULL in pgstat_clear_snapshot(). Done:

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-17 Thread Julien Rouhaud
On Wed, Mar 17, 2021 at 06:32:16PM -0400, Bruce Momjian wrote: > On Wed, Mar 17, 2021 at 04:04:44PM -0400, Robert Haas wrote: > > On Wed, Mar 17, 2021 at 12:48 PM Julien Rouhaud wrote: > > > I originally suggested to make it clearer by having an enum GUC rather > > > than a > > > boolean, say

Re: WIP: WAL prefetch (another approach)

2021-03-17 Thread Tomas Vondra
Hi, On 3/17/21 10:43 PM, Stephen Frost wrote: > Greetings, > > * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: >> Right, I was just going to point out the FPIs are not necessary - what >> matters is the presence of long streaks of WAL records touching the same >> set of blocks. But people

Re: Permission failures with WAL files in 13~ on Windows

2021-03-17 Thread Michael Paquier
On Tue, Mar 16, 2021 at 11:40:12AM +0100, Magnus Hagander wrote: > If we can provide a new .EXE built with exactly the same flags as the > EDB downloads that they can just drop into a directory, I think it's a > lot easier to get that done. Yeah, multiple people have been complaining about that

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-17 Thread Amit Kapila
On Wed, Mar 17, 2021 at 11:27 AM Amit Kapila wrote: > > 5. I have modified the comments atop worker.c to explain the design > and some of the problems clearly. See attached. If you are fine with > this, please include it in the next version of the patch. > I have further expanded these comments

RE: libpq debug log

2021-03-17 Thread tsunakawa.ta...@fujitsu.com
From: Alvaro Herrera > In pqTraceOutputString(), you can use the return value from fprintf to > move the cursor -- no need to count chars. Yes, precisely, 2 bytes for the double quotes needs to be subtracted as follows: len = fprintf(...); *cursor += (len - 2); > I still think

Re: PITR promote bug: Checkpointer writes to older timeline

2021-03-17 Thread Michael Paquier
On Wed, Mar 17, 2021 at 05:09:50PM +0900, Michael Paquier wrote: > Currently with HEAD and back branches, nothing would be broken as > logical contexts cannot exist in recovery. Still it would be easy > to miss the new behavior for anybody attempting to work more on this > feature in the future

Re: pg_amcheck contrib application

2021-03-17 Thread Mark Dilger
> On Mar 16, 2021, at 12:52 PM, Robert Haas wrote: > > On Mon, Mar 15, 2021 at 10:10 PM Mark Dilger > wrote: >> It is unfortunate that the failing test only runs pg_amcheck after creating >> numerous corruptions, as we can't know if pg_amcheck would have complained >> about pg_statistic

Re: pl/pgsql feature request: shorthand for argument and local variable references

2021-03-17 Thread Pavel Stehule
st 17. 3. 2021 v 23:32 odesílatel Michael Paquier napsal: > On Wed, Mar 17, 2021 at 05:04:48PM +0100, Pavel Stehule wrote: > > This tree has a different direction than is usual, and then replacing the > > root node is not simple. > > Yeah, it is not like we should redesign this whole part just

Re: pg_amcheck contrib application

2021-03-17 Thread Tom Lane
Mark Dilger writes: >> On Mar 16, 2021, at 12:52 PM, Robert Haas wrote: >> Since we now know that shutting autovacuum off makes the problem go >> away, I don't see a reason to commit 0001. We should fix pg_amcheck >> instead, if, as presently seems to be the case, that's where the >> problem is.

Re: pl/pgsql feature request: shorthand for argument and local variable references

2021-03-17 Thread Pavel Stehule
st 17. 3. 2021 v 23:16 odesílatel Hannu Krosing napsal: > why are you using yet another special syntax for this ? > > would it not be better to do something like this: > CREATE FUNCTION a_reall_long_and_winding_function_name(i int, out o int) > LANGUAGE plpgsql AS $plpgsql$ > DECLARE >args

Re: make the stats collector shutdown without writing the statsfiles if the immediate shutdown is requested.

2021-03-17 Thread Fujii Masao
On 2021/03/18 11:59, kuroda.hay...@fujitsu.com wrote: Dear Ikeda-san, I confirmed new patch and no problem was found. Thanks. (I'm not a native English speaker, so I cannot check your comments correctly, sorry) One user-visible side-effect by this change is; with the patch, the stats is

Re: Getting better results from valgrind leak tracking

2021-03-17 Thread Andres Freund
On 2021-03-17 22:33:59 -0400, Tom Lane wrote: > >> I've not yet tried to grok the comment that purports to justify it, > >> but I fail to see why it'd ever be useful to drop values inherited > >> from the postmaster. > > > I can't really make sense of it either. I think it may be trying to > >

Re: New IndexAM API controlling index vacuum strategies

2021-03-17 Thread Peter Geoghegan
On Wed, Mar 17, 2021 at 7:16 PM Masahiko Sawada wrote: > Since I was thinking that always skipping index vacuuming on > anti-wraparound autovacuum is legitimate, skipping index vacuuming > only when we're really close to the point of going into read-only mode > seems a bit conservative, but maybe

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-17 Thread Amit Kapila
On Thu, Mar 18, 2021 at 8:30 AM Justin Pryzby wrote: > > diff --git a/doc/src/sgml/ref/create_table.sgml > b/doc/src/sgml/ref/create_table.sgml > index ff1b642722..d5d356f2de 100644 > --- a/doc/src/sgml/ref/create_table.sgml > +++ b/doc/src/sgml/ref/create_table.sgml > @@ -1338,8 +1338,10 @@

RE: Parallel INSERT (INTO ... SELECT ...)

2021-03-17 Thread houzj.f...@fujitsu.com
> > If a table parameter value is set and the > > equivalent toast. parameter is not, the TOAST table > > will use the table's parameter value. > > -Specifying these parameters for partitioned tables is not supported, > > -but you may specify them for individual leaf

Re: Calendar support in localization

2021-03-17 Thread Vik Fearing
On 3/17/21 3:48 PM, Tom Lane wrote: > Also, the SQL spec says in so many words > that the SQL-defined datetime types follow the Gregorian calendar. We already don't follow the SQL spec for timestamps (and I, for one, think we are better for it) so I don't think we should worry about that. -- Vik

Re: Getting better results from valgrind leak tracking

2021-03-17 Thread Tom Lane
Andres Freund writes: > On 2021-03-17 18:07:36 -0400, Tom Lane wrote: >> Huh, interesting. I wonder why that makes the ident problem go away? > I spent a bit of time looking at valgrind, and it looks to be explicit > behaviour: > >// Our goal is to construct a set of chunks that includes

Re: replication slot stats memory bug

2021-03-17 Thread Amit Kapila
On Thu, Mar 18, 2021 at 4:55 AM Andres Freund wrote: > > Hi, > > On 2021-03-17 16:04:47 -0700, Andres Freund wrote: > > I'll push the minimal fix of forcing the allocation to happen in > > pgStatLocalContext and setting it to NULL in pgstat_clear_snapshot(). > > Done: >

Re: [HACKERS] Custom compression methods

2021-03-17 Thread Justin Pryzby
On Wed, Mar 17, 2021 at 02:50:34PM -0700, Andres Freund wrote: > On 2021-03-17 16:01:58 -0400, Robert Haas wrote: > > > - why is HIDE_TOAST_COMPRESSION useful? Doesn't quite seem to be > > > comparable to HIDE_TABLEAM? > > > > Andres, what do you mean by this exactly? It's exactly the same

Re: Permission failures with WAL files in 13~ on Windows

2021-03-17 Thread Andres Freund
Hi, On 2021-03-16 16:20:37 +0900, Michael Paquier wrote: > Fujii-san has mentioned that on twitter, but one area that has changed > during the v13 cycle is aaa3aed, where the code recycling segments has > been switched from a pgrename() (with a retry loop) to a > CreateHardLinkA()+pgunlink()

Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW

2021-03-17 Thread Japin Li
On Tue, 16 Mar 2021 at 20:13, Bharath Rupireddy wrote: > On Tue, Mar 16, 2021 at 1:15 AM Tom Lane wrote: >> >> [ Sorry for not looking at this thread sooner ] >> >> Bharath Rupireddy writes: >> > Currently, $subject is not allowed. We do plan the mat view query >> > before every refresh. I

Re: replication slot stats memory bug

2021-03-17 Thread Andres Freund
Hi, On 2021-03-17 19:36:46 -0400, Tom Lane wrote: > > But it seems like we just shouldn't allocate it dynamically at all? > > max_replication_slots doesn't change during postmaster lifetime, so it > > seems like it should just be allocated once? > > Meh. I don't see a need to wire in such an

  1   2   >