Re: A micro-optimisation for ProcSendSignal()

2021-07-20 Thread Thomas Munro
Hi Soumyadeep and Ashwin, Thanks for looking! On Sun, Jul 18, 2021 at 6:58 AM Soumyadeep Chakraborty wrote: > You might have missed a spot to initialize SERIALIZABLE_XACT->pgprocno in > InitPredicateLocks(), so: > > + PredXact->OldCommittedSxact->pgprocno = INVALID_PGPROCNO; The magic

Re: Doc necessity for superuser privileges to execute pg_import_system_collations()

2021-07-20 Thread Fujii Masao
On 2021/07/19 13:30, Fujii Masao wrote: On 2021/07/19 11:45, torikoshia wrote: Hi. It seems that only superusers can execute pg_import_system_collations(), but this is not mentioned in the manual. Since other functions that require superuser privileges describe that in the manual, I

Re: Addition of authenticated ID to pg_stat_activity

2021-07-20 Thread Michael Paquier
On Mon, Jul 19, 2021 at 04:56:24PM +0300, Aleksander Alekseev wrote: > Any reason why we shouldn't simply exclude internal processes from the > view? Do they have a connection that the VIEW could show? Yeah, they don't really have any useful information. Still, I kept that mainly as a matter of

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2021-07-20 Thread David Rowley
On Mon, 19 Jul 2021 at 18:32, Ronan Dunklau wrote: > regression=# explain select sum(unique1 order by ten, two), sum(unique1 order > by four), sum(unique1 order by two, four) from tenk2 group by ten; >QUERY PLAN >

Re: Have I found an interval arithmetic bug?

2021-07-20 Thread Bruce Momjian
On Tue, Jul 20, 2021 at 05:13:37PM -0700, Zhihong Yu wrote: > On Tue, Jul 20, 2021 at 3:53 PM Bruce Momjian wrote: > With your patch, the following example (Coutesy Bryn) still shows fraction: > > # select (interval '1 month')*1.123; >        ?column? > --- >  1 mon 3 days

Re: add 'noError' to euc_tw_and_big5.c

2021-07-20 Thread Michael Paquier
On Wed, Jul 21, 2021 at 02:15:14AM +, wangyu...@fujitsu.com wrote: > 'noError' argument was added at commit ea1b99a661, > but it seems to be neglected in euc_tw_and_big5.c Line 289. > please see the attachment. That sounds right to me. Double-checking the area, I am not seeing another

ORDER BY pushdowns seem broken in postgres_fdw

2021-07-20 Thread David Rowley
I'm working on a patch [1] to get the planner to consider adding PathKeys to satisfy ORDER BY / DISTINCT aggregates. I think this has led me to discover some problems with postgres_fdw's handling of pushing down ORDER BY clauses into the foreign server. The following test exists in the

Re: 回复: Why is XLOG_FPI_FOR_HINT always need backups?

2021-07-20 Thread Fujii Masao
On 2021/07/19 10:16, Kyotaro Horiguchi wrote: At Sat, 17 Jul 2021 00:14:34 +0900, Fujii Masao wrote in Thanks for updating the patch! It basically looks good to me. * Full-page image (FPI) records contain nothing else but a backup * block (or multiple

Re: wrong relkind error messages

2021-07-20 Thread Michael Paquier
On Tue, Jul 20, 2021 at 05:08:53PM +0200, Peter Eisentraut wrote: > While reviewing the logical decoding of sequences patch, I found a few more > places that could be updated in the new style introduced by this thread. > See attached patch. Those changes look fine. I am spotting one instance in

add 'noError' to euc_tw_and_big5.c

2021-07-20 Thread wangyu...@fujitsu.com
Hi, Heikki 'noError' argument was added at commit ea1b99a661, but it seems to be neglected in euc_tw_and_big5.c Line 289. please see the attachment. Regards, Yukun Wang add-noError.patch Description: add-noError.patch

Re: pgbench: using prepared BEGIN statement in a pipeline could cause an error

2021-07-20 Thread Yugo NAGATA
On Mon, 19 Jul 2021 10:51:36 +0900 Yugo NAGATA wrote: > Hello Fabien, > > On Sat, 17 Jul 2021 07:03:01 +0200 (CEST) > Fabien COELHO wrote: > > > > > Hello Yugo-san, > > > > > [...] One way to avoid these errors is to send Parse messages before > > > pipeline mode starts. I attached a patch

Re: [PATCH] Use optimized single-datum tuplesort in ExecSort

2021-07-20 Thread James Coleman
On Tue, Jul 20, 2021 at 4:35 AM David Rowley wrote: > > On Tue, 20 Jul 2021 at 01:10, James Coleman wrote: > > To be clear up front: I'm in favor of the patch, and I don't want to > > put unnecessary stumbling blocks up for it getting committed. So if we > > decide to proceed as is, that's fine

Re: Bitmap reuse

2021-07-20 Thread Tom Lane
David Rowley writes: > On Wed, 21 Jul 2021 at 11:25, Tom Lane wrote: >> Uh it's not the "exact same bitmap each time", because the selected >> rows vary depending on the value of g.i. > I imagined Jeff was meaning the bitmap from the scan of foo_x_idx, not > the combined ANDed bitmap from

Re: Question about non-blocking mode in libpq

2021-07-20 Thread Yugo NAGATA
Hello Alvaro, On Tue, 20 Jul 2021 12:05:11 -0400 Alvaro Herrera wrote: > On 2021-Jul-19, Yugo NAGATA wrote: > > > On Tue, 13 Jul 2021 11:59:49 +0900 > > Yugo NAGATA wrote: > > > > However, looking into the code, PQsendQuery seems not to return an error > > > in non-bloking mode even if

Re: [PATCH] Use optimized single-datum tuplesort in ExecSort

2021-07-20 Thread David Rowley
On Tue, 20 Jul 2021 at 23:28, Ranier Vilela wrote: > I took a look at cost_tuplesort and I think that some small adjustments could > be made as part of the improvement process. > It is attached. > 1. long is a very problematic type; better int64? > 2. 1024 can be int, not long? > 3. 2 changed

Re: [PATCH] Use optimized single-datum tuplesort in ExecSort

2021-07-20 Thread David Rowley
On Fri, 16 Jul 2021 at 15:44, David Rowley wrote: > > On Fri, 16 Jul 2021 at 02:53, Ronan Dunklau wrote: > > Please find attached a v9 just moving the flag setting to ExecInitSort, and > > my > > apologies if I misunderstood your point. > > I took this and adjusted a few things and ended up

Re: Micro-optimizations to avoid some strlen calls.

2021-07-20 Thread David G. Johnston
On Tue, Jul 20, 2021 at 5:28 PM Michael Paquier wrote: > On Mon, Jul 19, 2021 at 07:48:55PM -0300, Ranier Vilela wrote: > > There are some places, where strlen can have an overhead. > > This patch tries to fix this. > > > > Pass check-world at linux ubuntu (20.04) 64 bits. > > Why does it

Re: CLUSTER on partitioned index

2021-07-20 Thread Michael Paquier
On Tue, Jul 20, 2021 at 08:27:02PM -0400, Alvaro Herrera wrote: > I have to wonder if there really *is* a use case for CLUSTER in the > first place on regular tables, let alone on partitioned tables, which > are likely to be large and thus take a lot of time. What justifies > spending so much

Re: Bitmap reuse

2021-07-20 Thread David Rowley
On Wed, 21 Jul 2021 at 11:25, Tom Lane wrote: > > Jeff Janes writes: > > For some queries PostgreSQL can spend most of its time creating the exact > > same bitmap over and over. For example, in the below case: (also attached > > as a file because line-wrapping is going to make a mess of it) > >

Re: Micro-optimizations to avoid some strlen calls.

2021-07-20 Thread Michael Paquier
On Mon, Jul 19, 2021 at 07:48:55PM -0300, Ranier Vilela wrote: > There are some places, where strlen can have an overhead. > This patch tries to fix this. > > Pass check-world at linux ubuntu (20.04) 64 bits. Why does it matter? No code paths you are changing here are performance-critical,

Re: CLUSTER on partitioned index

2021-07-20 Thread Alvaro Herrera
I have to wonder if there really *is* a use case for CLUSTER in the first place on regular tables, let alone on partitioned tables, which are likely to be large and thus take a lot of time. What justifies spending so much time on this implementation? My impression is that CLUSTER is pretty much

Re: Column Filtering in Logical Replication

2021-07-20 Thread Alvaro Herrera
Hello, I think this looks good regarding the PublicationRelationInfo API that was discussed. Looking at OpenTableList(), I think you forgot to update the comment -- it says "open relations specified by a RangeVar list", but the list is now of PublicationTable. Also I think it would be good to

Re: Have I found an interval arithmetic bug?

2021-07-20 Thread Zhihong Yu
On Tue, Jul 20, 2021 at 3:53 PM Bruce Momjian wrote: > On Tue, Jul 20, 2021 at 02:33:07PM -0700, Zhihong Yu wrote: > > On Mon, Jul 19, 2021 at 9:14 PM Bruce Momjian wrote: > > > Obviously this should return '1 mon 26 days', but with my most > recent > > > patch, it returned '1 mon 25

Re: badly calculated width of emoji in psql

2021-07-20 Thread Jacob Champion
On Mon, 2021-07-19 at 13:13 +0200, Laurenz Albe wrote: > On Mon, 2021-07-19 at 16:46 +0900, Michael Paquier wrote: > > > In your opinion, would the current one-line patch proposal make things > > > strictly better than they are today, or would it have mixed results? > > > I'm wondering how to help

Bitmap reuse

2021-07-20 Thread Jeff Janes
For some queries PostgreSQL can spend most of its time creating the exact same bitmap over and over. For example, in the below case: (also attached as a file because line-wrapping is going to make a mess of it) drop table if exists foo; create table foo (x daterange, i int, t text); insert into

Re: Bitmap reuse

2021-07-20 Thread Tom Lane
Jeff Janes writes: > For some queries PostgreSQL can spend most of its time creating the exact > same bitmap over and over. For example, in the below case: (also attached > as a file because line-wrapping is going to make a mess of it) Uh it's not the "exact same bitmap each time", because

Re: POC: GROUP BY optimization

2021-07-20 Thread Tomas Vondra
On 7/20/21 3:12 PM, Tomas Vondra wrote: > ... > > The other comments from the review still apply - I'm particularly > concerned about the (1) point, i.e. plan changes in postgres_fdw. Those > seem to be rather strange (LIMIT not being pushed down in queries > without any grouping). I'd bet this is

Re: Have I found an interval arithmetic bug?

2021-07-20 Thread Bruce Momjian
On Tue, Jul 20, 2021 at 02:33:07PM -0700, Zhihong Yu wrote: > On Mon, Jul 19, 2021 at 9:14 PM Bruce Momjian wrote: > > Obviously this should return '1 mon 26 days', but with my most recent > > patch, it returned '1 mon 25 days'.  Turns out I had not properly used > > rint() in

Re: Signed vs Unsigned (take 2) (src/backend/storage/ipc/procarray.c)

2021-07-20 Thread Ranier Vilela
Em sex., 16 de jul. de 2021 às 09:41, Ranier Vilela escreveu: > Em sex., 16 de jul. de 2021 às 09:05, Aleksander Alekseev < > aleksan...@timescale.com> escreveu: > >> Hi Rainer, >> >> > Here are the two patches. >> > As suggested, reclassified as refactoring only. >> >> Please don't change the

Re: OpenSSL 3.0.0 compatibility

2021-07-20 Thread Daniel Gustafsson
> On 20 Jul 2021, at 09:54, Michael Paquier wrote: > > On Tue, Jul 20, 2021 at 01:23:42AM +0200, Daniel Gustafsson wrote: >> Another aspect of OpenSSL 3 compatibility is that of legacy cipher support, >> and >> as we concluded upthread it's best to leave that to the user to define in >>

Re: logical decoding and replication of sequences

2021-07-20 Thread Tomas Vondra
Here's a rebased version of the patch, no other changes. I think the crucial aspect of this that needs discussion/feedback the most is the transactional vs. non-transactional behavior. All the other questions are less important / cosmetic. regards -- Tomas Vondra EnterpriseDB:

Re: Have I found an interval arithmetic bug?

2021-07-20 Thread Zhihong Yu
On Mon, Jul 19, 2021 at 9:14 PM Bruce Momjian wrote: > On Wed, Jul 14, 2021 at 09:03:21AM -0700, Zhihong Yu wrote: > > On Thu, Jul 8, 2021 at 10:22 AM Zhihong Yu wrote: > > On Wed, Jun 30, 2021 at 9:35 AM Bruce Momjian > wrote: > > > > On Tue, Jun 29, 2021 at 06:49:45PM +0200,

Re: speed up verifying UTF-8

2021-07-20 Thread John Naylor
> On Mon, Jul 19, 2021 at 9:43 AM Vladimir Sitnikov < sitnikov.vladi...@gmail.com> wrote: > > An alternative idea: should we optimize for validation of **valid** inputs rather than optimizing the worst case? > > In other words, what if the implementation processes all characters always and uses a

Re: logical decoding and replication of sequences

2021-07-20 Thread Tomas Vondra
On 7/20/21 5:30 PM, Peter Eisentraut wrote: > On 08.06.21 00:28, Tomas Vondra wrote: >> >> new "sequence" publication action >> - >> >> The publications now have a new "sequence" publication action, which is >> enabled by default. This determines whether the

Re: Rename of triggers for partitioned tables

2021-07-20 Thread Alvaro Herrera
On 2021-Jul-19, Alvaro Herrera wrote: > Well, it does rename a trigger named 'name' on the table 'table_name', > as well as all its descendant triggers. I guess I am surprised that > anybody would rename a descendant trigger in the first place. I'm not > wedded to the decision of removing the

Re: refactoring basebackup.c

2021-07-20 Thread Mark Dilger
> On Jul 20, 2021, at 11:57 AM, Robert Haas wrote: > > I don't really understand what your problem is with how the patch set > leaves pg_basebackup. I don't have a problem with how the patch set leaves pg_basebackup. > On the server side, because I dropped the > bbarchiver stuff,

Re: Fix pkg-config file for static linking

2021-07-20 Thread Filip Gospodinov
On 06.07.21 15:13, Peter Eisentraut wrote: This doesn't work. This patch adds libpgcommon and libpgport to Requires.private.  But they are not pkg-config names but library names, so they should be added to Libs.private. Then, I must admit that I have misunderstood this patch at first

Re: [PATCH] Automatic HASH and LIST partition creation

2021-07-20 Thread Robert Haas
On Tue, Jul 20, 2021 at 3:13 PM Justin Pryzby wrote: > On Tue, Jul 20, 2021 at 02:42:16PM -0400, Robert Haas wrote: > > The bigger issue IMHO with on-the-fly > > partition creation is avoiding deadlocks in the presence of current > > inserters; I submit that without at least some kind of attempt

Re: [PATCH] Automatic HASH and LIST partition creation

2021-07-20 Thread Justin Pryzby
On Tue, Jul 20, 2021 at 02:42:16PM -0400, Robert Haas wrote: > The bigger issue IMHO with on-the-fly > partition creation is avoiding deadlocks in the presence of current > inserters; I submit that without at least some kind of attempt to > avoid deadlocks and spurious errors there, it's not

Re: [bug?] Missed parallel safety checks, and wrong parallel safety

2021-07-20 Thread Robert Haas
On Sun, Jul 4, 2021 at 1:44 AM Dilip Kumar wrote: > In general, for the non-partitioned table, where we don't have much > overhead of checking the parallel safety and invalidation is also not > a big problem so I am tempted to provide an automatic parallel safety > check. This would enable

Re: refactoring basebackup.c

2021-07-20 Thread Robert Haas
On Mon, Jul 19, 2021 at 2:51 PM Mark Dilger wrote: > The difficulty in v3-0007 with pg_basebackup only knowing how to parse tar > archives seems to be a natural consequence of not sufficiently abstracting > out the handling of the tar format. If the bbsink and bbstreamer > abstractions fully

Re: something is wonky with pgbench pipelining

2021-07-20 Thread Alvaro Herrera
On 2021-Jul-20, Andres Freund wrote: > I think what's happening is that the first recvfrom() actually gets all 7 > connection results. The server doesn't have any queries to process at that > point. But we ask the kernel whether there is new network input over and over > again, despite having

Re: [PATCH] Automatic HASH and LIST partition creation

2021-07-20 Thread Robert Haas
On Wed, Jul 14, 2021 at 7:28 AM Pavel Borisov wrote: > What do you think will the described approach lead to a useful patch? Should > it be done as a whole or it's possible to commit it in smaller steps? (E.g. > first part without AUTOMATIC capability, then add AUTOMATIC capability. Or > with

Re: .ready and .done files considered harmful

2021-07-20 Thread Robert Haas
On Tue, Jul 6, 2021 at 9:34 AM Stephen Frost wrote: > As was suggested on that subthread, it seems like it should be possible > to just track the current timeline and adjust what we're doing if the > timeline changes, and we should even know what the .history file is at > that point and likely

Re: Use extended statistics to estimate (Var op Var) clauses

2021-07-20 Thread Tomas Vondra
Hi, On 7/5/21 2:46 PM, Dean Rasheed wrote: > On Sun, 13 Jun 2021 at 21:28, Tomas Vondra > wrote: >> >> Here is a slightly updated version of the patch >> > > Hi, > > I have looked at this in some more detail, and it all looks pretty > good, other than some mostly cosmetic stuff. > Thanks for

something is wonky with pgbench pipelining

2021-07-20 Thread Andres Freund
Hi, I think something is slightly off with pgbench (or libpq) pipelining. Consider e.g. the following pgbench workload: \startpipeline SELECT 1; SELECT 1; SELECT 1; SELECT 1; SELECT 1; SELECT 1; SELECT 1; \endpipeline A pgbench run using that results in in endless repetitions of the below:

Re: Question about non-blocking mode in libpq

2021-07-20 Thread Alvaro Herrera
On 2021-Jul-19, Yugo NAGATA wrote: > On Tue, 13 Jul 2021 11:59:49 +0900 > Yugo NAGATA wrote: > > However, looking into the code, PQsendQuery seems not to return an error > > in non-bloking mode even if unable to send all data. In such cases, > > pqSendSome will return 1 but it doesn't cause an

Re: Avoid stack frame setup in performance critical routines using tail calls

2021-07-20 Thread Andres Freund
Hi, On 2021-07-20 19:37:46 +1200, David Rowley wrote: > On Tue, 20 Jul 2021 at 19:04, Andres Freund wrote: > > > * AllocateSetAlloc.txt > > > * palloc.txt > > > * percent.txt > > > > Huh, that's interesting. You have some control flow enforcement stuff > > turned on (the endbr64). And it looks

Re: Early Sort/Group resjunk column elimination.

2021-07-20 Thread Ronan Dunklau
Le vendredi 16 juillet 2021, 17:37:15 CEST James Coleman a écrit : > Thanks for hacking on this; as you're not surprised given I made the > original suggestion, I'm particularly interested in this for > incremental sort benefits, but I find the other examples you gave > compelling also. > > Of

Re: logical decoding and replication of sequences

2021-07-20 Thread Peter Eisentraut
On 08.06.21 00:28, Tomas Vondra wrote: new "sequence" publication action - The publications now have a new "sequence" publication action, which is enabled by default. This determines whether the publication decodes sequences or what. FOR ALL SEQUENCES

Re: wrong relkind error messages

2021-07-20 Thread Peter Eisentraut
While reviewing the logical decoding of sequences patch, I found a few more places that could be updated in the new style introduced by this thread. See attached patch. From 92a06ebfa6f856246c2642a4e45c5b2af69a911d Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Tue, 20 Jul 2021

Re: Rename of triggers for partitioned tables

2021-07-20 Thread Alvaro Herrera
On 2021-Jul-20, Arne Roland wrote: > Thank you! I get a compile time error > trigger.c:1588:17: note: in expansion of macro ‘PG_USED_FOR_ASSERTS_ONLY’ > int found = 0 PG_USED_FOR_ASSERTS_ONLY; > ^~~~ Oh, I misplaced the annotation of course. It has to be

Re: slab allocator performance issues

2021-07-20 Thread Ranier Vilela
Em ter., 20 de jul. de 2021 às 11:15, David Rowley escreveu: > On Tue, 20 Jul 2021 at 10:04, Ranier Vilela wrote: > > Perhaps you would agree with me that in the most absolute of times, > malloc will not fail. > > So it makes more sense to test: > > if (ret != NULL) > > than > > if (ret ==

Re: slab allocator performance issues

2021-07-20 Thread David Rowley
On Tue, 20 Jul 2021 at 10:04, Ranier Vilela wrote: > Perhaps you would agree with me that in the most absolute of times, malloc > will not fail. > So it makes more sense to test: > if (ret != NULL) > than > if (ret == NULL) I think it'd be better to use unlikely() for that. David

Re: POC: GROUP BY optimization

2021-07-20 Thread Tomas Vondra
Hi, here is an updated version of this patch, with some significant changes. The main change that instead of calling get_cheapest_group_keys_order directly, the planner now calls get_useful_group_keys_orderings and gets multiple "interesting" pathkey orderings instead of just a single one. The

Re: Next Steps with Hash Indexes

2021-07-20 Thread Simon Riggs
On Tue, Jul 20, 2021 at 1:26 PM Amit Kapila wrote: > One more thing we need to think about here is when to find the right > bucket page in the chain where we can insert the new tuple. Do we > first try to complete the uniqueness check (which needs to scan > through the entire bucket chain) and

Re: Next Steps with Hash Indexes

2021-07-20 Thread Simon Riggs
On Tue, Jul 20, 2021 at 1:00 PM Amit Kapila wrote: > > On Thu, Jul 15, 2021 at 10:11 PM Simon Riggs > wrote: > > > > 2. Unique Hash Indexes have been summarized here: > > https://www.postgresql.org/message-id/CAA4eK1KATC1TA5bR5eobYQVO3RWsnH6djNpk3P376em4V8MuUA%40mail.gmail.com > > which also

Re: Next Steps with Hash Indexes

2021-07-20 Thread Amit Kapila
On Tue, Jul 20, 2021 at 5:30 PM Amit Kapila wrote: > > On Thu, Jul 15, 2021 at 10:11 PM Simon Riggs > wrote: > > > > 2. Unique Hash Indexes have been summarized here: > > https://www.postgresql.org/message-id/CAA4eK1KATC1TA5bR5eobYQVO3RWsnH6djNpk3P376em4V8MuUA%40mail.gmail.com > > which also

Re: row filtering for logical replication

2021-07-20 Thread Amit Kapila
On Tue, Jul 20, 2021 at 5:13 PM Greg Nancarrow wrote: > > On Tue, Jul 20, 2021 at 6:29 PM Amit Kapila wrote: > > > > I think in terms of referring to old and new rows, we already have > > terminology which we used at various other similar places. See Create > > Rule docs [1]. For where clause,

Re: Next Steps with Hash Indexes

2021-07-20 Thread Amit Kapila
On Thu, Jul 15, 2021 at 10:11 PM Simon Riggs wrote: > > 2. Unique Hash Indexes have been summarized here: > https://www.postgresql.org/message-id/CAA4eK1KATC1TA5bR5eobYQVO3RWsnH6djNpk3P376em4V8MuUA%40mail.gmail.com > which also seems to have two parts to it. > > 2.1 Uniqueness Check > Amit: "to

Re: improvements in Unicode tables generation code

2021-07-20 Thread Peter Eisentraut
On 23.06.21 10:55, Peter Eisentraut wrote: v1-0001-Make-Unicode-makefile-more-parallel-safe.patch The makefile rule that calls UCS_to_most.pl was written incorrectly for parallel make.  The script writes all output files in one go, but the rule as written would call the command once for each

Re: row filtering for logical replication

2021-07-20 Thread Greg Nancarrow
On Tue, Jul 20, 2021 at 6:29 PM Amit Kapila wrote: > > I think in terms of referring to old and new rows, we already have > terminology which we used at various other similar places. See Create > Rule docs [1]. For where clause, it says "Within condition and > command, the special table names NEW

Re: [PATCH] Use optimized single-datum tuplesort in ExecSort

2021-07-20 Thread Ranier Vilela
Em ter., 20 de jul. de 2021 às 05:35, David Rowley escreveu: > On Tue, 20 Jul 2021 at 01:10, James Coleman wrote: > > To be clear up front: I'm in favor of the patch, and I don't want to > > put unnecessary stumbling blocks up for it getting committed. So if we > > decide to proceed as is,

Re: [PATCH] Allow multiple recursive self-references

2021-07-20 Thread Denis Hirn
> In the next version of the patch, would you be so kind as to include > updated documentation of the feature and at least one regression test > of same? As requested, this new version of the patch contains regression tests and documentation. Best wishes, -- Denis

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2021-07-20 Thread David Rowley
On Mon, 19 Jul 2021 at 18:32, Ronan Dunklau wrote: > It means the logic for appending the order by pathkeys to the existing group > by pathkeys would ideally need to remove the redundant group by keys from the > order by keys, considering this example: > > regression=# explain select sum(unique1

Re: row filtering for logical replication

2021-07-20 Thread Dilip Kumar
On Tue, Jul 20, 2021 at 3:43 PM Tomas Vondra wrote: > > Do we log the TOAST-ed values that were not updated? No, we don't, I have submitted a patch sometime back to fix that [1] [1] https://commitfest.postgresql.org/33/3162/ -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: row filtering for logical replication

2021-07-20 Thread Tomas Vondra
On 7/20/21 11:42 AM, Amit Kapila wrote: > On Tue, Jul 20, 2021 at 2:39 PM Tomas Vondra > wrote: >> >> On 7/20/21 7:23 AM, Amit Kapila wrote: >>> On Mon, Jul 19, 2021 at 7:02 PM Tomas Vondra >>> wrote: >> So maybe the best thing is to stick to the simple approach already used e.g. by

Re: row filtering for logical replication

2021-07-20 Thread Amit Kapila
On Tue, Jul 20, 2021 at 3:19 PM Dilip Kumar wrote: > > On Tue, Jul 20, 2021 at 3:12 PM Amit Kapila wrote: > > > > > > Why? I think it would just need similar restrictions as we are > > > > planning for Delete operation such that filter columns must be either > > > > present in primary or replica

Re: row filtering for logical replication

2021-07-20 Thread Dilip Kumar
On Tue, Jul 20, 2021 at 3:12 PM Amit Kapila wrote: > > > > Why? I think it would just need similar restrictions as we are > > > planning for Delete operation such that filter columns must be either > > > present in primary or replica identity columns. > > > > > > > How else would you turn UPDATE

Re: row filtering for logical replication

2021-07-20 Thread Amit Kapila
On Tue, Jul 20, 2021 at 2:39 PM Tomas Vondra wrote: > > On 7/20/21 7:23 AM, Amit Kapila wrote: > > On Mon, Jul 19, 2021 at 7:02 PM Tomas Vondra > > wrote: > > >> So maybe the best thing is to stick to the simple approach already used > >> e.g. by pglogical, which simply user the new row when

Re: Skipping logical replication transactions on subscriber side

2021-07-20 Thread Amit Kapila
On Tue, Jul 20, 2021 at 6:56 AM Masahiko Sawada wrote: > > On Mon, Jul 19, 2021 at 8:38 PM houzj.f...@fujitsu.com > wrote: > > > > 3) For 0003 patch, if user set skip_xid to a wrong xid which have not been > >assigned, and then will the change be skipped when the xid is assigned in > >

Re: Rename of triggers for partitioned tables

2021-07-20 Thread Arne Roland
Thank you! I get a compile time error gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation

Re: Numeric x^y for negative x

2021-07-20 Thread Yugo NAGATA
On Wed, 7 Jul 2021 18:36:56 +0100 Dean Rasheed wrote: > On Thu, 1 Jul 2021 at 14:17, Dean Rasheed wrote: > > > > On Tue, 29 Jun 2021 at 12:08, Dean Rasheed wrote: > > > > > > Numeric x^y is supported for x < 0 if y is an integer, but this > > > currently fails if y is outside the range of an

Re: row filtering for logical replication

2021-07-20 Thread Tomas Vondra
On 7/20/21 7:23 AM, Amit Kapila wrote: > On Mon, Jul 19, 2021 at 7:02 PM Tomas Vondra > wrote: >> >> On 7/19/21 1:00 PM, Dilip Kumar wrote: >>> On Mon, Jul 19, 2021 at 3:12 PM Amit Kapila wrote: a. Just log it and move to the next row b. send to stats collector some info about this

Re: row filtering for logical replication

2021-07-20 Thread Amit Kapila
On Wed, Jul 14, 2021 at 2:08 AM Euler Taveira wrote: > > On Tue, Jul 13, 2021, at 12:25 AM, Peter Smith wrote: > > I have reviewed the latest v18 patch. Below are some more review > comments and patches. > > Peter, thanks for quickly check the new patch. I'm attaching a new patch > (v19). > The

Re: row filtering for logical replication

2021-07-20 Thread Amit Kapila
On Tue, Jul 20, 2021 at 9:54 AM Amit Kapila wrote: > > On Mon, Jul 19, 2021 at 4:31 PM Dilip Kumar wrote: > > > > On Mon, Jul 19, 2021 at 3:12 PM Amit Kapila wrote: > > > > > > Maybe a second option is to have replication change any UPDATE into > > > > either an INSERT or a DELETE, if the old

Re: [PATCH] Use optimized single-datum tuplesort in ExecSort

2021-07-20 Thread David Rowley
On Tue, 20 Jul 2021 at 01:10, James Coleman wrote: > To be clear up front: I'm in favor of the patch, and I don't want to > put unnecessary stumbling blocks up for it getting committed. So if we > decide to proceed as is, that's fine with me. Thanks for making that clear. > But I'm not sure

Re: row filtering for logical replication

2021-07-20 Thread Amit Kapila
On Tue, Jul 20, 2021 at 11:38 AM Greg Nancarrow wrote: > > On Tue, Jul 20, 2021 at 2:25 PM Amit Kapila wrote: > > > > Today, while studying the behavior of this particular operation in > > other databases, I found that IBM's InfoSphere Data Replication does > > exactly this. See [1]. I think

RE: ECPG bug fix: DECALRE STATEMENT and DEALLOCATE, DESCRIBE

2021-07-20 Thread wangsh.f...@fujitsu.com
Hi kuroda-san: I find another problem about declare statement. The test source looks like: > EXEC SQL AT con1 DECLARE stmt STATEMENT; > EXEC SQL PREPARE stmt AS SELECT version(); > EXEC SQL DECLARE cur CURSOR FOR stmt; > EXEC SQL WHENEVER SQLERROR STOP; The outout about ecpg: >test.pgc:14:

Re: OpenSSL 3.0.0 compatibility

2021-07-20 Thread Michael Paquier
On Tue, Jul 20, 2021 at 01:23:42AM +0200, Daniel Gustafsson wrote: > Another aspect of OpenSSL 3 compatibility is that of legacy cipher support, > and > as we concluded upthread it's best to leave that to the user to define in > openssl.cnf. The attached 0002 adds alternative output files for

Re: Avoid stack frame setup in performance critical routines using tail calls

2021-07-20 Thread David Rowley
On Tue, 20 Jul 2021 at 19:04, Andres Freund wrote: > > * AllocateSetAlloc.txt > > * palloc.txt > > * percent.txt > > Huh, that's interesting. You have some control flow enforcement stuff turned > on (the endbr64). And it looks like it has a non zero cost (or maybe it's > just skid). Did you

Re: Avoid stack frame setup in performance critical routines using tail calls

2021-07-20 Thread Andres Freund
Hi, On Mon, Jul 19, 2021, at 23:53, David Rowley wrote: > On Tue, 20 Jul 2021 at 18:17, Andres Freund wrote: > > Any chance you could show a `perf annotate AllocSetAlloc` and `perf annotate > > palloc` from a patched run? And perhaps how high their percentages of the > > total work are. E.g.

Re: Avoid stack frame setup in performance critical routines using tail calls

2021-07-20 Thread David Rowley
On Tue, 20 Jul 2021 at 18:17, Andres Freund wrote: > Any chance you could show a `perf annotate AllocSetAlloc` and `perf annotate > palloc` from a patched run? And perhaps how high their percentages of the > total work are. E.g. using something like > perf report -g none|grep -E

Re: Avoid stack frame setup in performance critical routines using tail calls

2021-07-20 Thread Andres Freund
Hi, On 2021-07-20 16:50:09 +1200, David Rowley wrote: > I've not taken the time to study the patch but I was running some > other benchmarks today on a small scale pgbench readonly test and I > took this patch for a spin to see if I could see the same performance > gains. Thanks! > This is an

Re: row filtering for logical replication

2021-07-20 Thread Greg Nancarrow
On Tue, Jul 20, 2021 at 2:25 PM Amit Kapila wrote: > > Today, while studying the behavior of this particular operation in > other databases, I found that IBM's InfoSphere Data Replication does > exactly this. See [1]. I think there is a merit if want to follow this > idea. > So in this model