Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-07-01 Thread Daniel Gustafsson
> On 29 Mar 2020, at 17:27, Tomas Vondra wrote: > On Sun, Mar 29, 2020 at 10:22:25AM +0100, Dean Rasheed wrote: >> On Sat, 28 Mar 2020 at 13:18, Dean Rasheed wrote: >>> OK, I've pushed that with your recommendation for that function name. >> >> Does this now complete everything that you wanted

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-03-29 Thread Tomas Vondra
On Sun, Mar 29, 2020 at 10:22:25AM +0100, Dean Rasheed wrote: On Sat, 28 Mar 2020 at 13:18, Dean Rasheed wrote: OK, I've pushed that with your recommendation for that function name. Does this now complete everything that you wanted to do for functional dependency stats for PG13? Re-reading

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-03-29 Thread Dean Rasheed
On Sat, 28 Mar 2020 at 13:18, Dean Rasheed wrote: > > OK, I've pushed that with your recommendation for that function name. > Does this now complete everything that you wanted to do for functional dependency stats for PG13? Re-reading the thread, I couldn't see anything else that needed looking

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-03-28 Thread Dean Rasheed
On Wed, 25 Mar 2020 at 00:28, Tomas Vondra wrote: > > Seems OK to me. > > I'd perhaps name deps_clauselist_selectivity differently, it's a bit too > similar to dependencies_clauselist_selectivity. Perhaps something like > clauselist_apply_dependencies? But that's a minor detail. > OK, I've

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-03-24 Thread Tomas Vondra
On Thu, Mar 19, 2020 at 07:53:39PM +, Dean Rasheed wrote: On Wed, 18 Mar 2020 at 00:29, Tomas Vondra wrote: OK, I took a look. I think from the correctness POV the patch is OK, but I think the dependencies_clauselist_selectivity() function now got a bit too complex. I've been able to

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-03-19 Thread Dean Rasheed
On Wed, 18 Mar 2020 at 00:29, Tomas Vondra wrote: > > OK, I took a look. I think from the correctness POV the patch is OK, but > I think the dependencies_clauselist_selectivity() function now got a bit > too complex. I've been able to parse it now, but I'm sure I'll have > trouble in the future

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-03-18 Thread Tomas Vondra
On Tue, Mar 17, 2020 at 04:37:06PM +0100, Tomas Vondra wrote: On Tue, Mar 17, 2020 at 12:42:52PM +, Dean Rasheed wrote: On Sat, 14 Mar 2020 at 18:45, Tomas Vondra wrote: I realized there's one more thing that probably needs discussing. Essentially, these two clause types are the same:

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-03-17 Thread Tomas Vondra
On Tue, Mar 17, 2020 at 06:05:17PM +0100, Tomas Vondra wrote: On Tue, Mar 17, 2020 at 04:14:26PM +, Dean Rasheed wrote: On Tue, 17 Mar 2020 at 15:37, Tomas Vondra wrote: On Tue, Mar 17, 2020 at 12:42:52PM +, Dean Rasheed wrote: The other thing that I'm still concerned about is the

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-03-17 Thread Tomas Vondra
On Tue, Mar 17, 2020 at 04:14:26PM +, Dean Rasheed wrote: On Tue, 17 Mar 2020 at 15:37, Tomas Vondra wrote: On Tue, Mar 17, 2020 at 12:42:52PM +, Dean Rasheed wrote: >The other thing that I'm still concerned about is the possibility of >returning estimates with P(a,b) > P(a) or P(b).

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-03-17 Thread Dean Rasheed
On Tue, 17 Mar 2020 at 15:37, Tomas Vondra wrote: > > On Tue, Mar 17, 2020 at 12:42:52PM +, Dean Rasheed wrote: > > >The other thing that I'm still concerned about is the possibility of > >returning estimates with P(a,b) > P(a) or P(b). I think that such a > >thing becomes much more likely

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-03-17 Thread Tomas Vondra
On Tue, Mar 17, 2020 at 12:42:52PM +, Dean Rasheed wrote: On Sat, 14 Mar 2020 at 18:45, Tomas Vondra wrote: I realized there's one more thing that probably needs discussing. Essentially, these two clause types are the same: a IN (1, 2, 3) (a = 1 OR a = 2 OR a = 3) but with

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-03-17 Thread Dean Rasheed
On Sat, 14 Mar 2020 at 18:45, Tomas Vondra wrote: > > I realized there's one more thing that probably needs discussing. > Essentially, these two clause types are the same: > >a IN (1, 2, 3) > >(a = 1 OR a = 2 OR a = 3) > > but with 8f321bd1 we only recognize the first one as compatible

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-03-14 Thread Tomas Vondra
Hi, I realized there's one more thing that probably needs discussing. Essentially, these two clause types are the same: a IN (1, 2, 3) (a = 1 OR a = 2 OR a = 3) but with 8f321bd1 we only recognize the first one as compatible with functional dependencies. It was always the case that we

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-03-14 Thread Tomas Vondra
Hi, I've pushed the first part of the patch, adding ScalarArrayOpExpr as supported clause for functional dependencies, and then also doing the same for MCV lists. As discussed, I'm not going to do anything about the array containment clauses for PG13, that needs more discussion. I have a bunch

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-03-13 Thread Tomas Vondra
On Fri, Mar 13, 2020 at 08:42:49AM +, Dean Rasheed wrote: On Thu, 12 Mar 2020 at 17:30, Tomas Vondra wrote: I'm sorry, but I don't see how we could do this for arbitrary clauses. I think we could do that for clauses that have equality semantics and reference column values as a whole. So I

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-03-13 Thread Bruce Momjian
On Fri, Mar 13, 2020 at 08:42:49AM +, Dean Rasheed wrote: > On Thu, 12 Mar 2020 at 17:30, Tomas Vondra > wrote: > > > > I'm sorry, but I don't see how we could do this for arbitrary clauses. I > > think we could do that for clauses that have equality semantics and > > reference column values

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-03-13 Thread Dean Rasheed
On Thu, 12 Mar 2020 at 17:30, Tomas Vondra wrote: > > I'm sorry, but I don't see how we could do this for arbitrary clauses. I > think we could do that for clauses that have equality semantics and > reference column values as a whole. So I think it's possible to do this > for IN clauses (which is

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-03-12 Thread Tomas Vondra
On Thu, Mar 12, 2020 at 10:25:41AM +, Dean Rasheed wrote: [ For the sake of the archives, some of the discussion on the other thread [1-3] should really have been on this thread. ] On Sun, 2 Feb 2020 at 18:41, Tomas Vondra wrote: I think the challenge here is in applying the functional

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-03-12 Thread Dean Rasheed
[ For the sake of the archives, some of the discussion on the other thread [1-3] should really have been on this thread. ] On Sun, 2 Feb 2020 at 18:41, Tomas Vondra wrote: > > I think the challenge here is in applying the functional dependency > computed for the whole array to individual

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-03-04 Thread Tomas Vondra
Hi Pierre, I've looked at this patch series, hoping to get it close to committable. Here is a somewhat improved version of the patch series, split into 5 pieces. The first 4 parts are about applying functional dependencies to ScalarArrayOpExpr clauses. The last part is about doing the same thing

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-02-02 Thread Tomas Vondra
On Sun, Feb 02, 2020 at 10:59:32AM +0100, Pierre Ducroquet wrote: On Saturday, February 1, 2020 3:24:46 PM CET Tomas Vondra wrote: On Sat, Feb 01, 2020 at 08:51:04AM +0100, Pierre Ducroquet wrote: >Hello > >At my current job, we have a lot of multi-tenant databases, thus with >tables containing

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-02-02 Thread Pierre Ducroquet
On Saturday, February 1, 2020 3:24:46 PM CET Tomas Vondra wrote: > On Sat, Feb 01, 2020 at 08:51:04AM +0100, Pierre Ducroquet wrote: > >Hello > > > >At my current job, we have a lot of multi-tenant databases, thus with > >tables containing a tenant_id column. Such a column introduces a severe >

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-02-01 Thread Tomas Vondra
On Sat, Feb 01, 2020 at 08:51:04AM +0100, Pierre Ducroquet wrote: Hello At my current job, we have a lot of multi-tenant databases, thus with tables containing a tenant_id column. Such a column introduces a severe bias in statistics estimation since any other FK in the next columns is very

PATCH: add support for IN and @> in functional-dependency statistics use

2020-02-01 Thread Pierre Ducroquet
Hello At my current job, we have a lot of multi-tenant databases, thus with tables containing a tenant_id column. Such a column introduces a severe bias in statistics estimation since any other FK in the next columns is very likely to have a functional dependency on the tenant id. We found