Re: Is a modern build system acceptable for older platforms

2018-05-28 Thread Pierre Ducroquet
On Monday, May 28, 2018 4:37:06 AM CEST Yuriy Zhuravlev wrote: > > Can't see getting rid of those entirely. None of the github style > > platforms copes with reasonable complex discussions. > > I disagree. A good example of complex discussions on github is Rust > language tracker for RFCs: >

Re: effect of JIT tuple deform?

2018-07-02 Thread Pierre Ducroquet
50 ms (1.5ms spent optimizing) - jit without tuple deforming : 820ms (0.2ms) - jit with tuple deforming with optimization (-O3) : 770ms (105ms) - jit with tuple deforming with patch (-O1) : 725ms (54ms) I will look at the generated code for tuple deforming, but I think we should pre-optimize the LL

Re: [RFC] Add an until-0 loop in psql

2018-04-30 Thread Pierre Ducroquet
On Monday, April 30, 2018 1:01:25 PM CEST Daniel Verite wrote: > Corey Huinker wrote: > > As of v11, DO blocks can do transactions. I think this will meet your > > needs. > They do support COMMIT and ROLLBACK in the current > development tree, but not VACUUM as in Pierre's example. > >

Re: JIT compiling with LLVM v9.0

2018-01-24 Thread Pierre Ducroquet
has no member named ‘getBaseObject’ fs = llvm::cast(gvs->getBaseObject()); ^ That one was a bit uglier. I'm not sure how to test everything properly, so the patch is attached for both these issues, do as you wish with it… :) Regards Pierre Ducroquet >F

Re: JIT compiling with LLVM v9.0

2018-01-25 Thread Pierre Ducroquet
On Thursday, January 25, 2018 7:38:16 AM CET Andres Freund wrote: > Hi, > > On 2018-01-24 22:33:30 -0800, Jeff Davis wrote: > > On Wed, Jan 24, 2018 at 1:35 PM, Pierre Ducroquet <p.p...@pinaraf.info> wrote: > > > In LLVM 5.0, it looks like DebugInfo.h is

Re: JIT compiling with LLVM v9.0

2018-01-28 Thread Pierre Ducroquet
On Thursday, January 25, 2018 8:02:54 AM CET Andres Freund wrote: > Hi! > > On 2018-01-24 22:51:36 -0800, Jeff Davis wrote: > > Can we store the bitcode in pg_proc, simplifying deployment and > > allowing extensions to travel over replication? > > Yes, we could. You'd need to be a bit careful

Re: JIT compiling with LLVM v9.0

2018-01-28 Thread Pierre Ducroquet
On Thursday, January 25, 2018 8:12:42 PM CET Andres Freund wrote: > Hi, > > On 2018-01-25 10:00:14 +0100, Pierre Ducroquet wrote: > > I don't know when this would be released, > > August-October range. > > > but the minimal supported LLVM > >

Re: JIT compiling with LLVM v9.0

2018-01-29 Thread Pierre Ducroquet
On Monday, January 29, 2018 10:46:13 AM CET Andres Freund wrote: > Hi, > > On 2018-01-28 23:02:56 +0100, Pierre Ducroquet wrote: > > I have fixed the build issues with LLVM 3.9 and 4.0. The LLVM > > documentation is really lacking when it comes to porting from version x >

Re: JIT compiling with LLVM v9.1

2018-02-02 Thread Pierre Ducroquet
On Monday, January 29, 2018 10:53:50 AM CET Andres Freund wrote: > Hi, > > On 2018-01-23 23:20:38 -0800, Andres Freund wrote: > > == Code == > > > > As the patchset is large (500kb) and I'm still quickly evolving it, I do > > not yet want to attach it. The git tree is at > > > >

Re: JIT compiling with LLVM v9.1

2018-02-02 Thread Pierre Ducroquet
On Friday, February 2, 2018 10:48:16 AM CET Pierre Ducroquet wrote: > On Monday, January 29, 2018 10:53:50 AM CET Andres Freund wrote: > > Hi, > > > > On 2018-01-23 23:20:38 -0800, Andres Freund wrote: > > > == Code == > > > > > > As the patchset

Re: JIT compiling with LLVM v9.1

2018-02-05 Thread Pierre Ducroquet
On Sunday, February 4, 2018 12:45:50 AM CET Andreas Karlsson wrote: > On 02/02/2018 10:48 AM, Pierre Ducroquet wrote: > > I have successfully built the JIT branch against LLVM 4.0.1 on Debian > > testing. This is not enough for Debian stable (LLVM 3.9 is the latest > > avail

Re: JIT compiling with LLVM v9.1

2018-02-05 Thread Pierre Ducroquet
On Sunday, February 4, 2018 12:45:50 AM CET Andreas Karlsson wrote: > On 02/02/2018 10:48 AM, Pierre Ducroquet wrote: > > I have successfully built the JIT branch against LLVM 4.0.1 on Debian > > testing. This is not enough for Debian stable (LLVM 3.9 is the latest > > avail

Re: JIT compiling with LLVM v10.0

2018-02-07 Thread Pierre Ducroquet
expr.c b/src/backend/jit/llvm/llvmjit_expr.c index a06319b1b6..4b3c5367e5 100644 --- a/src/backend/jit/llvm/llvmjit_expr.c +++ b/src/backend/jit/llvm/llvmjit_expr.c @@ -170,7 +170,11 @@ get_LifetimeEnd(LLVMModuleRef mod) LLVMTypeRef sig; LLVMValueRef fn; LLVMTypeRef param_types[2]; +#if LLVM_VERSION_MAJOR > 4

Re: JIT compiling with LLVM v9.1

2018-02-05 Thread Pierre Ducroquet
On Monday, February 5, 2018 10:20:27 PM CET Andreas Karlsson wrote: > OK that fixed the issue, but you have a typo in your patch set. > > diff --git a/src/backend/lib/llvmjit_inline.cpp > b/src/backend/lib/llvmjit_inline.cpp > index a785261bea..51f38e10d2 100644 > ---

Re: JIT compiling with LLVM v10.1

2018-02-14 Thread Pierre Ducroquet
cleaning and then pushing some of the preliminary > patches (fixed tupledesc, grouping) over the next few days. > > Greetings, > > Andres Freund Hi Here are the LLVM4 and LLVM3.9 compatibility patches. Successfully built, and executed some silly queries with JIT forced to

Re: Query is over 2x slower with jit=on

2018-08-22 Thread Pierre Ducroquet
On Wednesday, August 22, 2018 6:51:55 PM CEST Andres Freund wrote: > On 2018-08-22 18:39:18 +0200, Andreas Joseph Krogh wrote: > > Just to be clear; The query really runs slower (wall-clock time), it's not > > just the timing. > > I bet it's not actually running slower, it "just" takes longer to

Re: [PATCH] LLVM tuple deforming improvements

2018-07-13 Thread Pierre Ducroquet
On Friday, July 13, 2018 11:08:45 PM CEST Andres Freund wrote: > Hi, > > Thanks for looking at this! > > On 2018-07-13 10:20:42 +0200, Pierre Ducroquet wrote: > > 2) improve the LLVM IR code > > > > The code generator in llvmjit-deform.c currently rely on the L

Re: function lca('{}'::ltree[]) caused DB Instance crash

2018-07-13 Thread Pierre Ducroquet
ence it in lca_inner. The attached basic patch fixes it. Regards Pierre >From 4e59747cea428d39c80974c408e95ba86bf63ecc Mon Sep 17 00:00:00 2001 From: Pierre Ducroquet Date: Fri, 13 Jul 2018 12:47:36 +0200 Subject: [PATCH] Fix segfault with lca('{}'::ltree[]) --- contrib/ltree/_ltree_op.c | 3

[PATCH] LLVM tuple deforming improvements

2018-07-13 Thread Pierre Ducroquet
rom: Pierre Ducroquet Date: Wed, 11 Jul 2018 23:41:59 +0200 Subject: [PATCH 1/2] Check for the hasnulls attribute before checking individual fields --- src/backend/jit/llvm/llvmjit_deform.c | 26 +++--- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/src/backend/jit/l

Re: Build fails with different versions of clang and LLVM

2018-04-23 Thread Pierre Ducroquet
On Monday, April 23, 2018 10:33:04 AM CEST Heikki Linnakangas wrote: > Hi! > > I tried compiling with --with-llvm on my laptop, and got this: > > $ make -s > > All of PostgreSQL successfully made. Ready to install. > > $ make -s install > > Invalid Summary Block: version expected > > error: can't

[RFC] Add an until-0 loop in psql

2018-04-24 Thread Pierre Ducroquet
Hi When running database migrations with .sql files on a live database, it's not uncommon to have to run a migration in a loop to prevent a big lock on a table. For instance if one want to delete some old datas from a big table one would write : DELETE FROM big_table WHERE id IN (SELECT id

Re: JIT compiling with LLVM v12

2018-03-29 Thread Pierre Ducroquet
On Thursday, March 29, 2018 2:39:17 PM CEST Jesper Pedersen wrote: > Hi Andres, > > On 03/28/2018 05:27 PM, Andres Freund wrote: > > On 2018-03-27 10:34:26 -0700, Andres Freund wrote: > >> On 2018-03-27 10:05:47 -0400, Peter Eisentraut wrote: > >>> On 3/13/18 19:40, Andres Freund wrote: >

Re: ALTER TABLE does not check for column existence before starting operations

2018-03-02 Thread Pierre Ducroquet
On Friday, March 2, 2018 2:44:16 PM CET David Steele wrote: > Hi Pierre, > > On 3/2/18 6:36 AM, Pierre Ducroquet wrote: > > While working on a big table recently, I noticed that ALTER TABLE does not > > check for column existence in operations like SET NOT NULL before

ALTER TABLE does not check for column existence before starting operations

2018-03-02 Thread Pierre Ducroquet
Hi While working on a big table recently, I noticed that ALTER TABLE does not check for column existence in operations like SET NOT NULL before starting working on the table, for instance adding a primary key. It is thus possible, if a typo has been made, to generate a long lock and a lot of

Re: Poor plan when using EXISTS in the expression list

2018-10-04 Thread Pierre Ducroquet
On Thursday, October 4, 2018 4:46:26 PM CEST Geoff Winkless wrote: > On Thu, 4 Oct 2018 at 13:11, Pierre Ducroquet wrote: > > Our developpers ORM (Django's) sadly can not use EXISTS in the where > > clauses > > without having it in the expression part of the SELECT stateme

Poor plan when using EXISTS in the expression list

2018-10-04 Thread Pierre Ducroquet
Hello Our developpers ORM (Django's) sadly can not use EXISTS in the where clauses without having it in the expression part of the SELECT statement. I was expecting it to create queries performing a bit worse than queries without this useless expression, but it turns out this trigger an

[Patch] Invalid permission check in pg_stats for functional indexes

2019-04-06 Thread Pierre Ducroquet
this by introducing a second path in privilege check in pg_stats view. I have not written a regression test yet, mainly because I'm not 100% certain where to write it. Given some hints, I would happily add it to this patch. Regards Pierre Ducroquet >From c2f638a9491e103311161208715dfcbcb55a2fbd Mon

Re: Row Level Security − leakproof-ness and performance implications

2019-02-21 Thread Pierre Ducroquet
On Wednesday, February 20, 2019 5:24:17 PM CET Tom Lane wrote: > Pierre Ducroquet writes: > > For simple functions like enum_eq/enum_ne, marking them leakproof is an > > obvious fix (patch attached to this email, including also textin/textout). > > This is not nearly as &

Re: Row Level Security − leakproof-ness and performance implications

2019-02-19 Thread Pierre Ducroquet
On Wednesday, February 20, 2019 12:43:50 AM CET Laurenz Albe wrote: > Pierre Ducroquet wrote: > > In order to increase our security, we have started deploying row-level > > security in order to add another safety net if any issue was to happen in > > our applications

Row Level Security − leakproof-ness and performance implications

2019-02-19 Thread Pierre Ducroquet
>From a4dd6eb9a16e7e2d2e44b03c5a04c485841aab25 Mon Sep 17 00:00:00 2001 From: Pierre Ducroquet Date: Tue, 19 Feb 2019 17:21:12 +0100 Subject: [PATCH] Mark textin/out and enum_eq/ne as leakproof --- src/include/catalog/pg_proc.dat | 8 1 file changed, 4 insertions(+), 4 deletions(-) d

[Patch] Add a reset_computed_values function in pg_stat_statements

2019-09-01 Thread Pierre Ducroquet
the aforementioned scenario. Regards Pierre Ducroquet>From 5e6d16d738e5279968321c5f3695e72ded2432db Mon Sep 17 00:00:00 2001 From: Pierre Date: Thu, 14 Feb 2019 14:37:48 +0100 Subject: [PATCH] Add a function to reset the cumulative statistics pg_stat_statements has two parts : raw statistics that are sim

Re: [Patch] Invalid permission check in pg_stats for functional indexes

2019-09-03 Thread Pierre Ducroquet
t; also not sure which regression file best suites for these tests. Thank you very much for your review Pierre >From d1d99104f8f157257a30cb9bf9cfea1cc467b1a7 Mon Sep 17 00:00:00 2001 From: Pierre Ducroquet Date: Sat, 6 Apr 2019 13:22:29 +0200 Subject: [PATCH] Use a different permission check pa

Re: [Patch] Add a reset_computed_values function in pg_stat_statements

2019-09-03 Thread Pierre Ducroquet
On Monday, September 2, 2019 3:25:00 AM CEST Euler Taveira wrote: > Em dom, 1 de set de 2019 às 06:04, Pierre Ducroquet > > escreveu: > > Hello > > > > pg_stat_statements is a great tool to track performance issue in live > > databases, especially when a

[Patch] optimizer - simplify $VAR1 IS NULL AND $VAR1 IS NOT NULL

2019-11-06 Thread Pierre Ducroquet
. But compared to the possible benefits, and the very low risk of n being high enough to have a real planification-time impact, I feel this optimization would be worth it. Regards Pierre >From 0a300b6fdd934daa6fb79e9bc67bdc2adfa3cc72 Mon Sep 17 00:00:00 2001 From: Pierre Ducroquet Date: Wed

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-02-02 Thread Pierre Ducroquet
On Saturday, February 1, 2020 3:24:46 PM CET Tomas Vondra wrote: > On Sat, Feb 01, 2020 at 08:51:04AM +0100, Pierre Ducroquet wrote: > >Hello > > > >At my current job, we have a lot of multi-tenant databases, thus with > >tables containing a tenant_id column. Such a

PATCH: add support for IN and @> in functional-dependency statistics use

2020-02-01 Thread Pierre Ducroquet
ude array contains. I tested the patches on current HEAD, but I can test and provide back-ported versions of the patch for other versions if needed (this code path hardly changed since its introduction in 10). Best regards Pierre Ducroquet >From d2b89748e1b520c276875538149eb466e97c21b4 Mon Sep

[PATCH] fix a performance issue with multiple logical-decoding walsenders

2019-12-26 Thread Pierre Ducroquet
Hello Our current setup uses logical replication to build a BI replication server along our primary clusters (running PG 10.10 so far). This implies having one logical replication slot per database. After some analysis, we identified two hot spots behind this issue. Fixing them gave us a 10

Re: [PATCH] fix a performance issue with multiple logical-decoding walsenders

2019-12-30 Thread Pierre Ducroquet
On Sunday, December 29, 2019 1:32:31 PM CET Julien Rouhaud wrote: > On Sat, Dec 28, 2019 at 1:56 PM Pierre Ducroquet wrote: > > Thank you for your comments. > > Attached to this email is a patch with better comments regarding the > > XLogSendLogical change. > &

Re: [PATCH] fix a performance issue with multiple logical-decoding walsenders

2020-01-06 Thread Pierre Ducroquet
On Monday, January 6, 2020 6:57:33 PM CET Tom Lane wrote: > Pierre Ducroquet writes: > > Attached to this email is a patch with better comments regarding the > > XLogSendLogical change. > > Hi, > This patch entirely fails to apply for me (and for the cfbot too).

Re: [PATCH] fix a performance issue with multiple logical-decoding walsenders

2019-12-28 Thread Pierre Ducroquet
On Thursday, December 26, 2019 8:18:46 PM CET Julien Rouhaud wrote: > Hello Pierre, > > On Thu, Dec 26, 2019 at 5:43 PM Pierre Ducroquet wrote: > > The second one was tested on PG 10 and PG 12 (with 48 lines offset). It > > has on PG12 the same effect it has on a PG10+i

[PATCH] Remove useless distinct clauses

2020-07-31 Thread Pierre Ducroquet
. If it is possible to optimize, we then iterate through the JOINs. Any comment on this would be more than welcome! Regards Pierre >From bc71aea5a0c1911c1f63567c21a870241b76231b Mon Sep 17 00:00:00 2001 From: Pierre Ducroquet Date: Fri, 31 Jul 2020 10:28:25 +0200 Subject: [PATCH] Add a

Re: Improvements in pg_dump/pg_restore toc format and performances

2023-09-19 Thread Pierre Ducroquet
On Monday, September 18, 2023 11:54:42 PM CEST Nathan Bossart wrote: > On Mon, Sep 18, 2023 at 02:52:47PM -0700, Nathan Bossart wrote: > > On Thu, Jul 27, 2023 at 10:51:11AM +0200, Pierre Ducroquet wrote: > >> I ended up writing several patches that shaved some time for

Re: Improvements in pg_dump/pg_restore toc format and performances

2023-09-18 Thread Pierre Ducroquet
On Monday, September 18, 2023 11:52:47 PM CEST Nathan Bossart wrote: > On Thu, Jul 27, 2023 at 10:51:11AM +0200, Pierre Ducroquet wrote: > > I ended up writing several patches that shaved some time for pg_restore > > -l, > > and reduced the toc.dat size. > > I've only

Re: Improvements in pg_dump/pg_restore toc format and performances

2023-09-19 Thread Pierre Ducroquet
On Monday, September 18, 2023 11:54:42 PM CEST Nathan Bossart wrote: > On Mon, Sep 18, 2023 at 02:52:47PM -0700, Nathan Bossart wrote: > > On Thu, Jul 27, 2023 at 10:51:11AM +0200, Pierre Ducroquet wrote: > >> I ended up writing several patches that shaved some time for

log_line_prefix: make it possible to add the search_path

2022-07-25 Thread Pierre Ducroquet
neric" method should be used (I thought of %{name} to fetch an arbitrary GUC, but did not implement due to a lack of need for that feature) >From d28ea4452c6e78bf8e67db49d1c72dbf4bd1ca48 Mon Sep 17 00:00:00 2001 From: Pierre Ducroquet Date: Mon, 25 Jul 2022 09:33:49 +0200 Subject: [PATCH] lo

Improvements in pg_dump/pg_restore toc format and performances

2023-07-27 Thread Pierre Ducroquet
functions that do that in a better way. And the location of string tables in the file and in the structures is probably not acceptable, I suppose these should go to the toc header instead. I still submit these for comments and first review. Best regards Pierre Ducroquet >From cf5afd22a6e754ccfab0cd

Re: Inefficiency in parallel pg_restore with many tables

2023-07-24 Thread Pierre Ducroquet
On Saturday, July 15, 2023 7:47:12 PM CEST Tom Lane wrote: > I'm not sure how big a deal this is in practice: in most situations > the individual jobs are larger than they are in this toy example, > plus the initial non-parallelizable part of the restore is a bigger > bottleneck anyway with this