Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-04-17 Thread Ashutosh Bapat
On Tue, Apr 17, 2018 at 2:05 PM, Etsuro Fujita wrote: > (2018/02/20 18:13), Ashutosh Bapat wrote: >> >> Here's patchset implementing this solution. >> >> 0001 adds PVC_*_CONVERTROWTYPEEXPR to pull_var_clause() and adjusts its >> callers. >> >>

Re: Postgres 10 problem with UNION ALL of null value in "subselect"

2018-04-16 Thread Ashutosh Bapat
expected with that commit. We can work around this problem by casting null to integer like null::integer. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Problem while updating a foreign table pointing to a partitioned table on foreign server

2018-04-16 Thread Ashutosh Bapat
er for the first and local foreign table's oid for the second. Right now, I don't see a simple way to do that. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c index 6e2fa14.

Re: [HACKERS] path toward faster partition pruning

2018-04-12 Thread Ashutosh Bapat
ng? I think we need to have some testcases somwhere to test constraint exclusion on partitions and partitioned tables, those do not necessarily fit partition pruning. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Creation of wiki page for open items of v11

2018-04-12 Thread Ashutosh Bapat
#x27;s what I had in mind. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Creation of wiki page for open items of v11

2018-04-12 Thread Ashutosh Bapat
On Thu, Apr 12, 2018 at 6:27 PM, Alvaro Herrera wrote: > Ashutosh Bapat wrote: > >> We use commitfest app for tracking the patches submitted. It has done >> well. Can we re-purpose the same for tracking open items? > > I think the rules are too different to cram both in

Re: Creation of wiki page for open items of v11

2018-04-11 Thread Ashutosh Bapat
re patches to solve those and owners responsible in resolving them. We have all the infrastructure in the commitfest app to track those things, may be we could just relabel things. I haven't seen the commitfest app code (and don't plan to do that myself) so may be this is just a wild idea.. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Creation of wiki page for open items of v11

2018-04-11 Thread Ashutosh Bapat
ore 11beta1" section is pretty much useless. > > Hm. There's definitely an argument to be made that it's not worth > tracking resolved items till after beta1. Once betas exist, the list > becomes useful to beta testers who may not be tracking events as closely > as hackers do. > +1 -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Creation of wiki page for open items of v11

2018-04-11 Thread Ashutosh Bapat
for resolved items, we can keep track of all items. If we just delete the resolved items, we wouldn't know if it was a mistake or it was intentional removal. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] path toward faster partition pruning

2018-04-11 Thread Ashutosh Bapat
ch is the point of having the inserts and the select just > above the actual pruning related tests. So, I'm not sure if adding the > satisfies_hash_partition against each pruning tests adds much. +1. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: crash with sql language partition support function

2018-04-10 Thread Ashutosh Bapat
LOBBER_CACHE_ALWAYS should show failure. I am missing something here. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] path toward faster partition pruning

2018-04-10 Thread Ashutosh Bapat
On Tue, Apr 10, 2018 at 5:32 PM, David Rowley wrote: > On 10 April 2018 at 20:56, Amit Langote wrote: >> On 2018/04/10 13:27, Ashutosh Bapat wrote: >>> On Mon, Apr 9, 2018 at 8:56 PM, Robert Haas wrote: >>>> CREATE OR REPLACE FUNCTION hashint4_noop(int4, int8)

Re: [HACKERS] path toward faster partition pruning

2018-04-09 Thread Ashutosh Bapat
ould produce for a given input except by running the code. > If you do the kind of thing shown above, though, then you can easily > see by inspection that you're getting the right answer. +1. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Optimizing nested ConvertRowtypeExpr execution

2018-04-09 Thread Ashutosh Bapat
typeExpr, arg)->arg; > > This runs depth-first so the while loop seems to run at most > once. I suppose that the "arg =" and the while loop are > transposed as intention. Yes. I have modelled it after RelableType case few lines above in the same function. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Optimizing nested ConvertRowtypeExpr execution

2018-04-09 Thread Ashutosh Bapat
On Mon, Apr 9, 2018 at 4:29 PM, Kyotaro HORIGUCHI wrote: > At Mon, 9 Apr 2018 16:07:33 +0530, Ashutosh Bapat > wrote in > >> On Mon, Apr 9, 2018 at 3:53 PM, Ashutosh Bapat >> wrote: >> > On Mon, Apr 9, 2018 at 3:49 PM, Kyotaro HORIGUCHI >> > wrote: &

Re: Optimizing nested ConvertRowtypeExpr execution

2018-04-09 Thread Ashutosh Bapat
On Mon, Apr 9, 2018 at 3:53 PM, Ashutosh Bapat wrote: > On Mon, Apr 9, 2018 at 3:49 PM, Kyotaro HORIGUCHI > wrote: >> >> I don't think it is not only on constatns. With the patch, >> non-constants are .. getting a rather strange conversion. >> >> >&g

Re: Optimizing nested ConvertRowtypeExpr execution

2018-04-09 Thread Ashutosh Bapat
r values cannot be assumed safely > composed each other for general inputs but it is known to be safe > for the ConvertRowtypeExpr case.. I think. I am not able to parse this statement. What output do you see without the patch? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread Ashutosh Bapat
On Sat, Apr 7, 2018 at 8:55 AM, David Rowley wrote: > On 7 April 2018 at 15:14, Ashutosh Bapat > wrote: >> On Sat, Apr 7, 2018 at 8:37 AM, David Rowley >>> Why is writing tests that produce the same output required? >>> >>> We have many tests with alt

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread Ashutosh Bapat
On Sat, Apr 7, 2018 at 8:37 AM, David Rowley wrote: > On 7 April 2018 at 15:03, Ashutosh Bapat > wrote: >> On Sat, Apr 7, 2018 at 7:25 AM, David Rowley >>> The only alternative would be to change all the hash functions so that >>> they normalise their endianness. It

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread Ashutosh Bapat
see a reason why this is bad? I don't think the concept is bad by itself. That's expected, in fact, we have added an option to pg_dump (dump through parent or some such) to handle exactly this case. What Amit seems to be complaining though is the regression test. We need to write regression tests so that they produce the same plans, pruning same partitions by name, on all architectures. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Get the name of the target Relation from Query struct? SOLVED!

2018-04-06 Thread Ashutosh Bapat
ble, > Query->resultRelation - 1) As I suggested in the other mail, please use rt_fetch() or planner_rt_fetch() that's future-proof in case we change relids to be something-other-than-one based (an unlikely event, but who knows). -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Get the name of the target Relation from Query struct?

2018-04-06 Thread Ashutosh Bapat
the lists are 0 based. But relations indexes are 1 based. Please use rt_fetch() or planner_rt_fetch() as suitable. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Optimizing nested ConvertRowtypeExpr execution

2018-04-05 Thread Ashutosh Bapat
On Tue, Apr 3, 2018 at 10:48 AM, Ashutosh Bapat wrote: >> >> Why is this done appropriately at ExecInitExpr() time, rather than at >> plan time? Seems like eval_const_expressions() would be a bit more >> appropriate (being badly named aside...)? > > That seems to

Re: Comment update in BuildTupleFromCStrings()

2018-04-03 Thread Ashutosh Bapat
Thanks Bruce. On Tue, Apr 3, 2018 at 11:31 PM, Bruce Momjian wrote: > On Fri, Mar 23, 2018 at 03:00:37PM +0530, Ashutosh Bapat wrote: >> Hi, >> BuildTupleFromCStrings() has comment "/* Call the "in" function for >> each non-dropped attribute */". It the

Re: Optimizing nested ConvertRowtypeExpr execution

2018-04-02 Thread Ashutosh Bapat
On Mon, Apr 2, 2018 at 1:40 AM, Andres Freund wrote: > Hi, > > On 2018-02-26 17:20:05 +0530, Ashutosh Bapat wrote: >> In a multi-level partitioned table, a parent whole-row reference gets >> translated into nested ConvertRowtypeExpr with child whole-row >> refere

Re: Diagonal storage model

2018-04-02 Thread Ashutosh Bapat
ability to add extra dimensions to provide additional views like double diagonal view. Alas! it all collapses since I was late to the party. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2018-03-30 Thread Ashutosh Bapat
k those inside the comparison function e.g. partition_rbound_cmp(). We could argue that that will benefit intermediate functions like partition_range_bound_cmp(), which just pass those values down, but there is all the possibility that its future caller may not have that packing structure available readily. So, I am inclined not to add a new structure just for this. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Additional Statistics Hooks

2018-03-30 Thread Ashutosh Bapat
On Thu, Mar 15, 2018 at 7:59 PM, Tomas Vondra wrote: > > > On 03/15/2018 06:00 AM, Ashutosh Bapat wrote: >> On Tue, Mar 13, 2018 at 8:55 PM, Mat Arye wrote: >>>> >>>> Like cost associated with a function, we may associate mapping >>>> cardinalit

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-29 Thread Ashutosh Bapat
On Wed, Mar 28, 2018 at 7:21 PM, Ashutosh Bapat wrote: > > Ah sorry, I was wrong about remote_conds. remote_conds and local_conds > are basically the conditions on the relation being pushed down. > havingQuals are conditions on a grouped relation so treating them like > baserestr

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-28 Thread Ashutosh Bapat
sense. So better to have > a separate member in PgFdwRelationInfo. Ah sorry, I was wrong about remote_conds. remote_conds and local_conds are basically the conditions on the relation being pushed down. havingQuals are conditions on a grouped relation so treating them like baserestrictinfo or join

Re: Oddity in COPY FROM handling of check constraints on partition tables

2018-03-28 Thread Ashutosh Bapat
ost. We can not add this to v11 open items since it isn't a v11 bug exactly. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-26 Thread Ashutosh Bapat
ing scan, so + * use root's all_baserels. */ This is correct only for "other" grouping relations. We are yet to decide what to do for the other upper relations. -if (IS_UPPER_REL(rel)) +if (IS_UPPER_REL(rel) && !IS_OTHER_REL(rel)) I guess, this condition boi

Re: Odd procedure resolution

2018-03-23 Thread Ashutosh Bapat
On Fri, Mar 23, 2018 at 7:53 PM, Tom Lane wrote: > Ashutosh Bapat writes: >> Incidently the fix looks quite simple. See patch attached. > > ISTM this patch effectively proposes to make procedures have their own > namespace yet still live in pg_proc. That is the worst of all

Re: Odd procedure resolution

2018-03-23 Thread Ashutosh Bapat
;. But I haven't fixed the error messages in this patch. I need to first see if the changes are acceptable. On Fri, Mar 23, 2018 at 3:53 PM, Ashutosh Bapat wrote: > Hi, > Consider following scenario > > create function foo(a int) returns integer as $$begin return a; end; &g

Odd procedure resolution

2018-03-23 Thread Ashutosh Bapat
drop function foo(a int); DROP FUNCTION call foo(1); CALL Functions and Procedures are two different objects and we enforce different methods to invoke those, SELECT and CALL resp. So, we should be able to filter out one or the other and try to find best candidate of a given kind. -- Best Wi

Comment update in BuildTupleFromCStrings()

2018-03-23 Thread Ashutosh Bapat
the attribute whose in() function caused an error. May be we could pass an optional error context to the later and use it. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company From c8faf979d34a04099597bdbbe952ca7a936e5dbe Mon Sep 17 00:00:00 2001 From: Ashutosh Bap

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2018-03-22 Thread Ashutosh Bapat
On Thu, Mar 22, 2018 at 4:32 AM, Dmitry Dolgov <9erthali...@gmail.com> wrote: >> On 12 March 2018 at 06:00, Ashutosh Bapat >> wrote: >> Thanks for the note. Here are rebased patches. > > Since I started to look at this patch, I can share few random notes (althoug

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-21 Thread Ashutosh Bapat
ation will not be created when it wants to just fetch an existing relation (and error out/assert if it doesn't find one.). -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-19 Thread Ashutosh Bapat
On Mon, Mar 19, 2018 at 11:15 PM, Robert Haas wrote: > On Fri, Mar 16, 2018 at 1:50 PM, Ashutosh Bapat > wrote: >>> This patch also renames can_parallel_agg to >>> can_partial_agg and removes the parallelism-specific bits from it. >> >> I think we need to upd

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-16 Thread Ashutosh Bapat
o an FDW. I think some FDWs and extensions will be happy if we provide them readymade decisions for can_sort, can_hash, can_partial_agg etc. It will be good if they don't have to translate the grouping target and havingQual for every child twice, once for core and second time in the FDW. In all it looks like we need some structure to hold that information so that we can pass it down the hook. I am fine with two structures one variable and other invariable. An upper operation can have one of them or both. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-16 Thread Ashutosh Bapat
. So, this sentence seems to contradict itself. I am confused. > Moreover, even if it's > correct now, I think that the chances that the next person who > modifies this code will manage to keep it correct are not great. I > think we need to create the partial grouping rel somewhere in the code > that's closer to where it's actually needed, so that we don't have so > much action at a distance, or at least have a simpler and more > transparent set of tests. +1. I agree with that. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-16 Thread Ashutosh Bapat
lso remove the need for 0004, as that case would > also end up being handled in a different way. However, the jury is > still out on whether or not the approach I've proposed there is any > good. Feel free to opine over on that thread. Will take a look at that patch next week. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-16 Thread Ashutosh Bapat
On Thu, Mar 15, 2018 at 7:46 PM, Robert Haas wrote: > On Thu, Mar 15, 2018 at 6:08 AM, Ashutosh Bapat > wrote: >> In current create_grouping_paths() (without any of your patches >> applied) we first create partial paths in partially grouped rel and >> then add parallel p

Re: parallel append vs. simple UNION ALL

2018-03-16 Thread Ashutosh Bapat
On Wed, Mar 14, 2018 at 2:09 AM, Robert Haas wrote: > On Tue, Mar 13, 2018 at 12:35 AM, Ashutosh Bapat > wrote: >> It looks like it was not changed in all the places. make falied. I >> have fixed all the instances of these two functions in the attached >> patchset (on

Re: [HACKERS] Another oddity in handling of WCO constraints in postgres_fdw

2018-03-16 Thread Ashutosh Bapat
. If WCO is defined on the view on the remote > server, then it should operate properly and not require anything from the > local side. I agree with this analysis. I have no objection about the patch anymore. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: "SELECT ... FROM DUAL" is not quite as silly as it appears

2018-03-16 Thread Ashutosh Bapat
one row without any column, I guess, there will be at least one row in the output. I am curious how would we handle cases which do not return any row like create function set_ret_func() returns setof record as $$select * from pg_class where oid = 0;$$ language sql; select set_ret_func(); set_ret_

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-15 Thread Ashutosh Bapat
comments. > Also incorporated few comments improvements provided off-list by Ashutosh > Bapat. > The patchset needs rebase. I have rebased those on the latest head and made following changes. Argument force_partial_agg is added after output arguments to make_grouping_rels(). Moved it before

Re: Additional Statistics Hooks

2018-03-14 Thread Ashutosh Bapat
output. The planner can then use this statistics to arrive at various estimates. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Comment fixes in create_grouping_paths() add_paths_to_append_rel()

2018-03-14 Thread Ashutosh Bapat
On Wed, Mar 14, 2018 at 10:11 PM, Stephen Frost wrote: > Greetings Ashutosh, > > * Ashutosh Bapat (ashutosh.ba...@enterprisedb.com) wrote: >> Here are two patches fixing comments. > > Thanks! > >> 0001 >> A comment in add_paths_to_append_rel() mentions the num

Comment fixes in create_grouping_paths() add_paths_to_append_rel()

2018-03-14 Thread Ashutosh Bapat
grouping sets to create_grouping_paths(), but forgot to update the prologue of the function. Fixed it. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company From 053a0f4e77ad14a08f7fb8fb96152fae4406bd9a Mon Sep 17 00:00:00 2001 From: Ashutosh Bapat Date: Wed, 14 Mar

Re: Ambigous Plan - Larger Table on Hash Side

2018-03-13 Thread Ashutosh Bapat
on and can we fix this ? After analysing the small table, the first plan is chosen as the cheapest. This means that the plan with smaller table being hashed has cost higher than the plan with larger table being hashed. We need to examine that costing to see what went wrong in costing. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Additional Statistics Hooks

2018-03-13 Thread Ashutosh Bapat
e, I mean input argument tuple. In Mat's case the mapping cardinality will be 12. The number of distinct values that function may output is estimated as number of estimated rows / mapping cardinality of that function. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Ambigous Plan - Larger Table on Hash Side

2018-03-13 Thread Ashutosh Bapat
ANALYZE on this query, produce outputs of two plans: one with larger table being hashed and second with the smaller one being hashed, you will see which of them performs better. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] why not parallel seq scan for slow functions

2018-03-12 Thread Ashutosh Bapat
targetlist of parallel seq scan plan. The solution here addresses only parallel scan requirement. In future if we implement a solution which also addresses requirements of FDW and custom plan (i.e. ability to handle targetlists by FDW and custom plan), the changes made here will need to be reverted. That would be a painful exercsize. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: parallel append vs. simple UNION ALL

2018-03-12 Thread Ashutosh Bapat
; > Good point. Changed. It looks like it was not changed in all the places. make falied. I have fixed all the instances of these two functions in the attached patchset (only 0003 changes). Please check. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporat

Re: [HACKERS] Another oddity in handling of WCO constraints in postgres_fdw

2018-03-12 Thread Ashutosh Bapat
On Mon, Mar 12, 2018 at 1:25 PM, Etsuro Fujita wrote: > (2018/03/09 20:55), Etsuro Fujita wrote: >> >> (2018/03/08 14:24), Ashutosh Bapat wrote: >>> >>> For local constraints to be enforced, we use remote >>> constraints. For local WCO we need to use

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-12 Thread Ashutosh Bapat
On Mon, Mar 12, 2018 at 7:49 PM, Jeevan Chalke wrote: > > > On Mon, Mar 12, 2018 at 6:07 PM, Ashutosh Bapat > wrote: >> >> On Fri, Mar 9, 2018 at 4:21 PM, Ashutosh Bapat >> >> GroupPathExtraData now completely absorbs members from and replaces >> OtherU

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-12 Thread Ashutosh Bapat
On Fri, Mar 9, 2018 at 4:21 PM, Ashutosh Bapat wrote: > On Thu, Mar 8, 2018 at 7:31 PM, Robert Haas wrote: >> >> This kind of goes along with the suggestion I made yesterday to >> introduce a new function, which at the time I proposed calling >> initialize_grouping_r

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-09 Thread Ashutosh Bapat
the grouped relation and partially grouped relation do not have bare partition keys. So, for a relation sitting on top of this (partially) grouped relation the partition key doesn't exist. So, we can't consider grouped or partially grouped relation as partitioned. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-08 Thread Ashutosh Bapat
upper-rel PathTargets we just computed into * root->upper_targets[]. The core code doesn't use this, but it * provides a convenient place for extensions to get at the info. For * consistency, we save all the intermediate targets, even though some * of t

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-07 Thread Ashutosh Bapat
uch paths have different shapes from the ones that we create now and will they be better? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Another oddity in handling of WCO constraints in postgres_fdw

2018-03-07 Thread Ashutosh Bapat
eed to use remote WCO. That means we create many foreign tables pointing to same local table on the foreign server through many views, but it's not impossible. [1] https://www.postgresql.org/message-id/20180117130021.GC2416%40tamriel.snowman.net [2] https://www.postgresql.org/message-id/5A058F21.2040201%40lab.ntt.co.jp [3] https://www.postgresql.org/message-id/a3955a1d-ad07-5b0a-7618-b6ef5ff0e1c5%40lab.ntt.co.jp -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-07 Thread Ashutosh Bapat
On Wed, Mar 7, 2018 at 10:04 AM, Ashutosh Bapat wrote: > On Tue, Mar 6, 2018 at 7:52 PM, Jeevan Chalke > wrote: > >> >> >> Changes look good to me and refactoring will be useful for partitionwise >> patches. >> >> However, will it be good if we add

Missing break statement after transformCallStmt in transformStmt

2018-03-06 Thread Ashutosh Bapat
g to cause problems in future when somebody adds a case after CallStmt. Here's patch with fix. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c index c3a9617..cf1a34e 100644 --- a/src

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-06 Thread Ashutosh Bapat
al_paths_to_grouping_rel() and > add_paths_to_grouping_rel() to avoid passing can_sort, can_hash and costs > related details individually to them? I think so too. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-06 Thread Ashutosh Bapat
t * -make_partial_grouping_target(PlannerInfo *root, PathTarget *grouping_target) +make_partial_grouping_target(PlannerInfo *root, + PathTarget *grouping_target, + Node *havingQual) This looks like a refactoring change. Should go to one of the r

Re: [HACKERS] PoC: full merge join on comparison clause

2018-03-04 Thread Ashutosh Bapat
pstream}, but there are other ways also. Then you can use git rebase to rebase your patches periodically. If you are already doing that, sorry for the noise. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: postgres_fdw: perform UPDATE/DELETE .. RETURNING on a join directly

2018-03-04 Thread Ashutosh Bapat
has grown really large over the time, as we have added more pushdown. Since most of the queries in that file use the same tables created at the beginning of the file, changes somewhere in-between (esp. DMLs) affect the following queries. It's getting hard to add a test query in that file an

Re: parallel append vs. simple UNION ALL

2018-03-01 Thread Ashutosh Bapat
th = make_union_unique(op, ppath, tlist, root); We could probably push the grouping/sorting down to the parallel workers. But again not part of this patchset. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] path toward faster partition pruning

2018-02-28 Thread Ashutosh Bapat
On Thu, Mar 1, 2018 at 6:57 AM, Amit Langote wrote: > On 2018/02/28 19:14, Ashutosh Bapat wrote: >> On Wed, Feb 28, 2018 at 6:42 AM, Amit Langote wrote: >>> BTW, should there be a relevant test in partition_join.sql? If yes, >>> attached a patch (partitionwise-join-col

Re: [HACKERS] path toward faster partition pruning

2018-02-28 Thread Ashutosh Bapat
't explain why and if it did so it should do that in find_partition_scheme() not here. partsupfunc is another property of partition keys that is cached like parttyplen, parttypbyval. Why does it deserve a separate comment when others don't? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] path toward faster partition pruning

2018-02-28 Thread Ashutosh Bapat
be a relevant test in partition_join.sql? If yes, > attached a patch (partitionwise-join-collation-test-1.patch) to add one. A partition-wise join path will be created but discarded because of higher cost. This test won't see it in that case. So, please add some data like other tes

Optimizing nested ConvertRowtypeExpr execution

2018-02-26 Thread Ashutosh Bapat
code which handles nested ConvertRowtypeExprs like is_converted_whole_row_reference(). But I haven't tried that approach yet. Suggestions/comments welcome. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company diff --git a/src/backend/executor/execExpr.c b/src/backend/exe

Re: PlaceHolderVars in pushed down child-join cause error

2018-02-25 Thread Ashutosh Bapat
On Thu, Feb 22, 2018 at 8:36 PM, Robert Haas wrote: > On Thu, Feb 22, 2018 at 7:41 AM, Ashutosh Bapat > wrote: >> postgres_fdw isn't expected to push down joins with placeholder vars. >> But the check for that in foreign_join_ok() only considers >> joinrel->

PlaceHolderVars in pushed down child-join cause error

2018-02-22 Thread Ashutosh Bapat
o push down a child-join with PlaceHolderVars in it and fails with error "unsupported expression type for deparse: 198". 198 being T_PlaceHolderVar. The fix is to use joinrel->top_parent_relids for a child-join. Attached patch for the same. -- Best Wishes, Ashutosh Bapat EnterpriseD

Re: NEXT VALUE FOR sequence

2018-02-21 Thread Ashutosh Bapat
ed row" thing gets pretty complicated. Consider simple case. What happens when NextValueExpr appears in one of the conditions and that row gets eliminated, do we consider that as a processed row and increment the NextValueExpr or do not increment it? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-02-20 Thread Ashutosh Bapat
On Tue, Feb 13, 2018 at 6:21 PM, Ashutosh Bapat wrote: > > 1. Push down ConvertRowtypeExpr and include it in the pushed down targetlist. > This would solve both the problems described above. Both set_plan_ref() and > get_relation_column_alias_ids() will find ConvertRowtypeExpr, they

Re: spelling of enable_partition_wise_join

2018-02-18 Thread Ashutosh Bapat
On Mon, Feb 19, 2018 at 9:08 AM, Ashutosh Bapat wrote: > > Thanks. There are functions like try_partition_wise_join(), > generate_partition_wise_join_paths() which use partition_wise > spelling. Should we update those as well? I see that the commit already did that. Sorry. Although,

Re: [HACKERS] Partition-wise aggregation/grouping

2018-02-18 Thread Ashutosh Bapat
Commit 2fb1abaeb016aeb45b9e6d0b81b7a7e92bb251b9, changed enable_partition_wise_join to enable_partitionwise_join. This patch too should use enable_partitionwise_agg instead of enable_partition_wise_agg.

Re: [HACKERS] why not parallel seq scan for slow functions

2018-02-18 Thread Ashutosh Bapat
On Mon, Feb 19, 2018 at 9:35 AM, Ashutosh Bapat wrote: > On Sat, Feb 17, 2018 at 8:17 AM, Amit Kapila wrote: >> On Fri, Feb 16, 2018 at 9:29 AM, Ashutosh Bapat >> wrote: >>> On Thu, Feb 15, 2018 at 7:47 PM, Amit Kapila >>> wrote: >>>> On

Re: [HACKERS] why not parallel seq scan for slow functions

2018-02-18 Thread Ashutosh Bapat
On Sat, Feb 17, 2018 at 8:17 AM, Amit Kapila wrote: > On Fri, Feb 16, 2018 at 9:29 AM, Ashutosh Bapat > wrote: >> On Thu, Feb 15, 2018 at 7:47 PM, Amit Kapila wrote: >>> On Thu, Feb 15, 2018 at 4:18 PM, Ashutosh Bapat >>> wrote: >>>> I happened to look

Re: spelling of enable_partition_wise_join

2018-02-18 Thread Ashutosh Bapat
On Fri, Feb 16, 2018 at 9:09 PM, Peter Eisentraut wrote: > On 2/14/18 03:28, Ashutosh Bapat wrote: >> On Wed, Feb 14, 2018 at 2:15 AM, Alvaro Herrera >> wrote: >>> Peter Eisentraut wrote: >>>> I wonder how others feel about this, but the spelling of >>

Re: [HACKERS] why not parallel seq scan for slow functions

2018-02-15 Thread Ashutosh Bapat
On Thu, Feb 15, 2018 at 7:47 PM, Amit Kapila wrote: > On Thu, Feb 15, 2018 at 4:18 PM, Ashutosh Bapat > wrote: >> I happened to look at the patch for something else. But here are some >> comments. If any of those have been already discussed, please feel >> free to ignore.

Changing baserel to foreignrel in postgres_fdw functions

2018-02-15 Thread Ashutosh Bapat
Hi, I noticed that functions is_foreign_expr(), classifyConditions() and appendOrderByClause() had variables/arguments named baserel when the relations passed to those could be join or upper relation as well. Here's patch renaming those as foreignrel. -- Best Wishes, Ashutosh Bapat Enterpr

Re: [HACKERS] why not parallel seq scan for slow functions

2018-02-15 Thread Ashutosh Bapat
e output will show that the parallel seq scan's targetlist has costly_func() in it. Isn't that what we want to test here? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-02-14 Thread Ashutosh Bapat
partitioned table. That will be too restrictive. I think the case of PHV and ConvertRowtypeExprs are different. The the former we need a subquery to push PHVs, whereas to deparse ConvertRowtypeExpr on the nullable side we can use the same strategy as whole-row expression deparsing. -- Best

Re: spelling of enable_partition_wise_join

2018-02-14 Thread Ashutosh Bapat
e. Alvaro, Peter, Gavin have voted for partitionwise in this thread and Robert had similar objections earlier. Looks like we should change it. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: reorganizing partitioning code (was: Re: [HACKERS] path toward faster partition pruning)

2018-02-13 Thread Ashutosh Bapat
itioning related fields. Various partition > bound comparison and search functions (and then some) that work off of the > cached information are moved. Are you moving partition bound comparison functions to partcache.c? They will also used by optimizer, so may be leave them out of partcache.c? --

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-02-13 Thread Ashutosh Bapat
t_relation_column_alias_ids() encounters a ConvertRowtypeExpr, it pulls out the embedded whole row reference and returns the corresponding column id. deparseExpr() calls deparseVar() by pulling out the embedded whole-row reference Var when it encouters ConvertRowtypeExpr. Comments, thoughts any other solutions? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Wait event names mismatch: oldserxid

2018-02-09 Thread Ashutosh Bapat
On Fri, Feb 9, 2018 at 7:23 PM, Michael Paquier wrote: > On Fri, Feb 09, 2018 at 06:04:39PM +0530, Ashutosh Bapat wrote: >> Name for wait event "LWTRANCHE_OLDSERXID_BUFFERS" is printed as >> "oldserxid", but documentation at >> https://www.postgresql.org/

Wait event names mismatch: oldserxid

2018-02-09 Thread Ashutosh Bapat
CHE_OLDSERXID_BUFFERS); -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Creation of wiki page for open items of v11

2018-02-08 Thread Ashutosh Bapat
ail.com > > Are there any objections in creating a wiki page to track all those bugs > and issues? We don't want to lose track of all that. +1. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2018-02-08 Thread Ashutosh Bapat
On Fri, Feb 9, 2018 at 11:26 AM, Amit Langote wrote: > On 2018/02/09 14:31, Ashutosh Bapat wrote: >>> I also noticed that a later patch adds partsupfunc to PartitionScheme, >>> which the pruning patch needs too. So, perhaps would be nice to take out >>> that porti

Re: Proposal: partition pruning by secondary attributes

2018-02-08 Thread Ashutosh Bapat
nsertion and update of secondary attribute will touch catalog and update if required. That will increase the contention on catalog. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2018-02-08 Thread Ashutosh Bapat
On Thu, Feb 8, 2018 at 10:41 AM, Amit Langote wrote: > On 2018/02/08 11:55, Amit Langote wrote: >> Hi Ashutosh. >> >> On 2018/02/07 13:51, Ashutosh Bapat wrote: >>> Here's a new patchset with following changes >>> >>> 1. Rebased on the late

Re: [HACKERS] path toward faster partition pruning

2018-02-08 Thread Ashutosh Bapat
On Wed, Feb 7, 2018 at 7:17 PM, Robert Haas wrote: > On Wed, Feb 7, 2018 at 8:37 AM, Ashutosh Bapat > wrote: >> While looking at the changes in partition.c I happened to look at the >> changes in try_partition_wise_join(). They mark partitions deemed >> dummy by pruning a

Re: [HACKERS] path toward faster partition pruning

2018-02-07 Thread Ashutosh Bapat
On Wed, Feb 7, 2018 at 6:49 PM, Robert Haas wrote: > On Wed, Feb 7, 2018 at 3:42 AM, Ashutosh Bapat > wrote: >> partprune.c looks to much tied to one feature. I am sure that the >> functions used for partition pruning can be used by other >> optimizations as well. > &g

Re: [HACKERS] path toward faster partition pruning

2018-02-07 Thread Ashutosh Bapat
from bounds etc. which are metadata management kind 2. partition bound comparison functions, and other optimizer related functions. May be we should divide the file that way. The first category code remains in catalog/ as it is today. The second catagory functions move to optimizer/. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2018-02-07 Thread Ashutosh Bapat
On Fri, Dec 22, 2017 at 3:00 PM, Rajkumar Raghuwanshi wrote: > On Wed, Dec 20, 2017 at 5:21 PM, Ashutosh Bapat > wrote: >> >> Thanks. Here are some comments >> > Thanks Ashutosh for review and suggestions. > >> >> +-- test default partition beha

Re: Crash in partition-wise join involving dummy partitioned relation

2018-02-05 Thread Ashutosh Bapat
On Tue, Feb 6, 2018 at 4:04 AM, Robert Haas wrote: > On Mon, Feb 5, 2018 at 4:46 AM, Ashutosh Bapat > wrote: >> Here's patch taking that approach. > > I rewrote the comment in relation.h like this, which I think is more clear: > > /* > * Is given relation

<    5   6   7   8   9   10   11   >