Re: Report error position in partition bound check

2020-04-10 Thread Ashutosh Bapat
On Fri, 10 Apr 2020 at 14:31, Amit Langote wrote: > On Thu, Apr 9, 2020 at 11:04 PM Tom Lane wrote: > > While I'm quite on board with providing useful error cursors, > > the example cases in this patch don't seem all that useful: > > > > -- trying to create range partition with empty range > >

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2020-04-10 Thread Ashutosh Bapat
On Fri, 10 Apr 2020 at 20:44, Jeff Janes wrote: > On Thu, Apr 9, 2020 at 10:04 AM Ashutosh Bapat < > ashutosh.bapat@gmail.com> wrote: > >> On Thu, Apr 9, 2020 at 12:03 PM Etsuro Fujita >> wrote: >> > >> > On Thu, Apr 9, 2020 at 2:36 PM

Re: A problem about partitionwise join

2020-04-09 Thread Ashutosh Bapat
to the current EC infrastructure. So we still have to look > through restrictlist. > When I wrote that function and even today, EC didn't accommodate outer join equality conditions. If we can somehow do that, have_partkey_equi_join() can be completely eliminated. -- Best Wishes, Ashutosh Bapat

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2020-04-09 Thread Ashutosh Bapat
tion looks more purposed for join, but it's not difficult to convert it to be used for UNION as well. In that case those set of functions will have many more callers. So, I will vote to keep that assertion now that we have it there. -- Best Wishes, Ashutosh Bapat

Re: Report error position in partition bound check

2020-04-09 Thread Ashutosh Bapat
st answer file. > -- Best Wishes, Ashutosh Bapat

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2020-04-08 Thread Ashutosh Bapat
On Wed, 8 Apr 2020 at 15:42, Etsuro Fujita wrote: > Hi Kuntal, > > On Wed, Apr 8, 2020 at 4:30 PM Kuntal Ghosh > wrote: > > I'm getting the following warning during compilation. > > > > partbounds.c: In function ‘partition_bounds_merge’: > > partbounds.c:1024:21: warning: unused variable ‘inner_

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2020-04-08 Thread Ashutosh Bapat
Thanks Kuntal for the report. Let me know if this patch works for you. On Wed, 8 Apr 2020 at 13:00, Kuntal Ghosh wrote: > Hi, > > On Wed, Apr 8, 2020 at 7:07 AM Etsuro Fujita > wrote: > > > > Pushed after modifying some comments further, based on the suggestions > > of Ashutosh. > I'm getting t

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2020-04-07 Thread Ashutosh Bapat
Thanks a lot Fujita-san. Thanks Dmitry, Rajkumar, Amul, Mark, Robert, Antonin, Amit, Justin,Thomas and Tomas for all your help and review. On Wed, 8 Apr 2020 at 07:07, Etsuro Fujita wrote: > On Wed, Apr 8, 2020 at 2:24 AM Etsuro Fujita > wrote: > > On Wed, Apr 8, 2020 at 12:15 AM Tomas Vondra >

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2020-04-06 Thread Ashutosh Bapat
Here are some changes suggested on top of v34 as per my previous mail. These are mostly comment changes. On Mon, 6 Apr 2020 at 13:59, Etsuro Fujita wrote: > On Sat, Apr 4, 2020 at 12:15 AM Etsuro Fujita > wrote: > > Attached is the original patch (0001) and one patch (0002) with > > changes inc

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2020-04-06 Thread Ashutosh Bapat
On Fri, 3 Apr 2020 at 20:45, Etsuro Fujita wrote: > Hi, > > On Thu, Apr 2, 2020 at 2:12 AM Ashutosh Bapat > wrote: > > On Thu, 26 Mar 2020 at 00:35, Tomas Vondra > wrote: > >> I've started reviewing the patch a couple days ago. I haven't done any >

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2020-04-01 Thread Ashutosh Bapat
On Thu, 26 Mar 2020 at 00:35, Tomas Vondra wrote: > Hi, > > I've started reviewing the patch a couple days ago. I haven't done any > extensive testing, but I do have a bunch of initial comments that I can > share now. > > 1) I wonder if this needs to update src/backend/optimizer/README, which > d

Re: A bug when use get_bit() function for a long bytea string

2020-03-31 Thread Ashutosh Bapat
Thanks for the changes, + int64 res,resultlen; It's better to have them on separate lines. -unsigned +int64 hex_decode(const char *src, unsigned len, char *dst) Do we want to explicitly cast the return value to int64? Will build on some platform crib if not done so? I don't know of such a platf

Re: A bug when use get_bit() function for a long bytea string

2020-03-26 Thread Ashutosh Bapat
On Thu, 26 Mar 2020 at 19:39, Tom Lane wrote: > Ashutosh Bapat writes: > > On Wed, 18 Mar 2020 at 08:18, movead...@highgo.ca > > wrote: > >> if we change return type of all those functions to int64, we won't need > >> this cast. > >> I change

Re: A bug when use get_bit() function for a long bytea string

2020-03-26 Thread Ashutosh Bapat
On Wed, 18 Mar 2020 at 08:18, movead...@highgo.ca wrote: > > Hello thanks for the detailed review, > > >I think the second argument indicates the bit position, which would be > max bytea length * 8. If max bytea length covers whole int32, the second > argument >needs to be wider i.e. int64. > Yes

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2020-03-23 Thread Ashutosh Bapat
t one partition on one side > > will > > join with multiple partitions on the other side. So exact partition bounds > > match will work in most of the cases. The cases otherwise are not so common > > to > > spend the effort in coding and planning. > > > > I have added this explanation in the patch. > > I also think it would be great if we can perform generic partitionwise > join for hash partitioned tables, so I'd like to propose to add > something like this, instead: "Currently we support partitionwise join > for hash partitioned tables only when the partition bounds for them > exactly match, but later it might be worth the effort to relax the > restriction." That's good too. But please include an explanation about the case when modulo/max no. of partitions itself differs. That case is not likely to get addressed in nearer future. -- Best Wishes, Ashutosh Bapat

Re: BEFORE ROW triggers for partitioned tables

2020-03-18 Thread Ashutosh Bapat
I was expecting that documentation somewhere covered the fact that BR triggers are not supported on a partitioned table. And this patch would remove/improve that portion of the code. But I don't see any documentation changes in this patch. On Tue, Mar 17, 2020 at 10:11 PM Ashutosh Bapat

Re: BEFORE ROW triggers for partitioned tables

2020-03-17 Thread Ashutosh Bapat
On Fri, 13 Mar 2020 at 21:55, Alvaro Herrera wrote: > On 2020-Mar-11, Ashutosh Bapat wrote: > > > On Thu, Feb 27, 2020 at 10:22 PM Alvaro Herrera > > wrote: > > > > * The new function I added, ReportTriggerPartkeyChange(), contains one > > > serious bug (n

Re: Re: A bug when use get_bit() function for a long bytea string

2020-03-16 Thread Ashutosh Bapat
On Fri, 13 Mar 2020 at 08:48, movead...@highgo.ca wrote: > Thanks for the reply. > > >Why have you used size? Shouldn't we use int64? > Yes, thanks for the point, I have changed the patch. > > Thanks for the patch. > >If get_bit()/set_bit() accept the second argument as int32, it can not > >be

Re: A bug when use get_bit() function for a long bytea string

2020-03-12 Thread Ashutosh Bapat
8, when performing that calculation, we have to widen the result. So, instead of changing the datatype of len, it might be better to perform the calculation as (int64)len * 8. If we use int64, we could also use INT64_FORMAT instead of using %ld. Since this is a bug it shouldn't wait another commitfest, but still add this patch to the commitfest so that it's not forgotten. -- Best Wishes, Ashutosh Bapat

Re: BEFORE ROW triggers for partitioned tables

2020-03-11 Thread Ashutosh Bapat
On Wed, Mar 11, 2020 at 8:53 PM Ashutosh Bapat wrote: > > On Thu, Feb 27, 2020 at 10:22 PM Alvaro Herrera > wrote: > > > > * The "root" is not necessarily the root partitioned table, but instead > > it's the table that was named in the command. Bec

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-03-11 Thread Ashutosh Bapat
On Wed, Mar 11, 2020 at 4:19 AM David Rowley wrote: > > On Wed, 11 Mar 2020 at 02:50, Ashutosh Bapat > wrote: > > > > On Tue, Mar 10, 2020 at 1:49 PM Andy Fan wrote: > > > In my current implementation, it calculates the uniqueness for each > > > BaseRel o

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-03-11 Thread Ashutosh Bapat
complex of it deserves > another thread > to discuss it, any thought? I think if the relation underlying an Agg node is know to be unique for given groupByClause, we could safely use AGG_SORTED strategy. Though the input is not ordered, it's sorted thus for every row Agg node will combine/finalize the aggregate result. -- Best Wishes, Ashutosh Bapat

Re: BEFORE ROW triggers for partitioned tables

2020-03-11 Thread Ashutosh Bapat
ge in the partition key values when it contains expression. Using RelationGetPartitionQual() will fix this problem and also problem of attribute remapping across the partition hierarchy. -- Best Wishes, Ashutosh Bapat

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-03-10 Thread Ashutosh Bapat
m2.b" won't need DISTINCT node since the result of joining m1 and m2 has unique value of m2.a for each row. In your example the join will produce two rows (m1.a, m1.b, m2.a, m2.b) (1, 1, 1, 1) and (1, 1, 2, 1) where m2.a is unique key. -- Best Wishes, Ashutosh Bapat

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-03-10 Thread Ashutosh Bapat
that patch, but as discussed upthread, in this case we want the uniqueness associated with the RelOptInfo and not the path. > > I also believe that the existing code in analyzejoins.c should be > edited to make use of unique keys rather than how it looks at unique > indexes and group by / distinct clauses. > +1. -- Best Wishes, Ashutosh Bapat

Re: Option to dump foreign data in pg_dump

2020-03-04 Thread Ashutosh Bapat
TableForWorker() using the same logic that getTables() > uses. > > > > I think parallel data dump would be an especially interesting option > > when using foreign tables, so it's worth figuring this out. > > What do you think of Peter's comment? > > Regards, > -- > -David > da...@pgmasters.net > > > -- Best Wishes, Ashutosh Bapat

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2020-03-03 Thread Ashutosh Bapat
; Seq Scan on alpha_pos_pos alpha (cost=0.00..1.06 rows=1 > width=16) > > > Filter: ((b = '1'::double precision) AND (a = '1'::double > precision)) > > >-> Seq Scan on beta_hi beta (cost=0.00..1.04 rows=1 width=16) > > >

Re: LOCK TABLE and DROP TABLE on temp tables of other sessions

2020-02-14 Thread Ashutosh Bapat
On Fri, Feb 14, 2020 at 11:35 AM Michael Paquier wrote: > On Thu, Feb 13, 2020 at 09:05:01PM +0530, Ashutosh Bapat wrote: > > That looks odd. Other sessions are able to see temporary tables of a > given > > session because they are stored in the same catalog which is accessibl

Re: LOCK TABLE and DROP TABLE on temp tables of other sessions

2020-02-13 Thread Ashutosh Bapat
ematic. > > Regards, > > -- > Fujii Masao > NTT DATA CORPORATION > Advanced Platform Technology Group > Research and Development Headquarters > > > -- -- Best Wishes, Ashutosh Bapat

Updating row and width estimates in postgres_fdw

2020-02-12 Thread Ashutosh Bapat
inPaths into two - one to get estimates and the other to setup fpinfo. Comments? -- Best Wishes, Ashutosh Bapat

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-02-11 Thread Ashutosh Bapat
On Tue, Feb 11, 2020 at 8:27 AM Andy Fan wrote: > > > On Tue, Feb 11, 2020 at 12:22 AM Ashutosh Bapat < > ashutosh.bapat@gmail.com> wrote: > >> >> >>> >>> [PATCH] Erase the distinctClause if the result is unique by >>> defini

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-02-11 Thread Ashutosh Bapat
On Mon, Feb 10, 2020 at 10:57 PM Tom Lane wrote: > Ashutosh Bapat writes: > >> On Sat, Feb 8, 2020 at 12:53 PM Andy Fan > wrote: > >> Do you mean adding some information into PlannerInfo, and when we > create > >> a node for Unique/HashAggregate/Gro

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-02-10 Thread Ashutosh Bapat
On Sat, Feb 8, 2020 at 12:53 PM Andy Fan wrote: > Hi Ashutosh: >Thanks for your time. > > On Fri, Feb 7, 2020 at 11:54 PM Ashutosh Bapat < > ashutosh.bapat@gmail.com> wrote: > >> Hi Andy, >> What might help is to add more description to your emai

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-02-07 Thread Ashutosh Bapat
s well. >> >> This patch doesn't work for a well-written SQL, but some drawback >> of a SQL may be not very obvious, since the cost of checking is pretty >> low as well, so I think it would be ok to add.. >> >> Please see the patch for details. >> >> Thank you. >> > -- -- Best Wishes, Ashutosh Bapat

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2019-05-14 Thread Ashutosh Bapat
de to handle the cases when the partitions are missing on the nullable side. Tom mentioned the idea of using just projection to produce join tuples with rows on the outer side appended with null columns from the nullable side. If we can implement that, we can remove the restrictions in this patch. -- Best Wishes, Ashutosh Bapat

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2019-03-11 Thread Ashutosh Bapat
;> >> > > Ok, that can be fixed. > > Similarly, I think merge_null_partitions should set null_index instead of > asserting when null partitions missing from both the side, make sense? > I think not. null_index, once set shouldn't change and hence does not change with each pair of partitions being matched. So, it makes sense to make sure that null_index remains invalid if none of the tables have null partition. -- Best Wishes, Ashutosh Bapat

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2019-03-10 Thread Ashutosh Bapat
g merged_index explicitly somewhere else, other places may miss that explicit assignment. So it's better to move it inside this function. -- Best Wishes, Ashutosh Bapat

Re: Query with high planning time at version 11.1 compared versions 10.5 and 11.0

2019-01-14 Thread Ashutosh Bapat
to > add such details here than in the header as it's better to keep the latter > more stable. > > About the comment you added, I think we could clarify the note further as: > > Note: here we abuse the consider_partitionwise_join flag by setting it > *even* for child rels that are not partitioned. In that case, we set it > to tell try_partitionwise_join() that it doesn't need to generate their > targetlists and EC entries as they have already been generated here, as > opposed to the dummy child rels for which the flag is left set to false so > that it will generate them. > > Maybe it's a bit wordy, but it helps get the intention across more clearly. > > Thanks, > Amit > > -- -- Best Wishes, Ashutosh Bapat

Re: Query with high planning time at version 11.1 compared versions 10.5 and 11.0

2019-01-10 Thread Ashutosh Bapat
er way would be to avoid these translations and use Parent var to represent a Var of the child being dealt with. That will be a massive churn on inheritance based planner code, but it will improve planning time for queries involving thousands of partitions. -- Best Wishes, Ashutosh Bapat

Re: [HACKERS] Macros bundling RELKIND_* conditions

2018-12-20 Thread Ashutosh Bapat
gt; needs a better place. > > BTW are there other opinions on the RELKIND_HAS_STORAGE vs. > RELKIND_CAN_HAVE_STORAGE debate? I'm inclined to change it to the > former. > +1 I liked the idea. -- Best Wishes, Ashutosh Bapat

Re: Query with high planning time at version 11.1 compared versions 10.5 and 11.0

2018-12-07 Thread Ashutosh Bapat
On Fri, Dec 7, 2018 at 11:13 AM Ashutosh Bapat wrote: > > > >> >> >> Robert, Ashutosh, any comments on this? I'm unfamiliar with the >> partitionwise join code. >> > > As the comment says it has to do with the equivalence classes being u

Re: Query with high planning time at version 11.1 compared versions 10.5 and 11.0

2018-12-06 Thread Ashutosh Bapat
dea to make partition-wise join code do this may work, but will add a similar overhead esp. in N-way partition-wise join once those equivalence classes are added. -- Best Wishes, Ashutosh Bapat

Re: Is it really difficult for postgres_fdw to implement READ COMMITTED isolation?

2018-09-18 Thread Ashutosh Bapat
hen we could use READ > COMMITTED on both sides? > > I guess the problem is 1. exporting snapshots is not cheap 2. tracking prepared transactions is not implemented. See a nearby thread on "transaction involving multiple foreign server". -- Best Wishes, Ashutosh Bapat

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2018-09-17 Thread Ashutosh Bapat
On Thu, Sep 13, 2018 at 1:45 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > On Fri, 31 Aug 2018 at 08:23, Ashutosh Bapat < > ashutosh.ba...@enterprisedb.com> wrote: > > > > On Thu, Aug 30, 2018 at 2:23 PM, Dmitry Dolgov <9erthali...@gmail.com> > wrot

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2018-08-30 Thread Ashutosh Bapat
ry as "returned with feedback" in the >> next commitfest. > > Since I'm more or less familiar with the code and I believe it's an > interesting > feature, I can try to take over it for now if you don't mind (but without any > strong commitments to make it

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2018-08-29 Thread Ashutosh Bapat
On Thu, Aug 23, 2018 at 4:01 PM, Dmitry Dolgov <9erthali...@gmail.com> wrote: >> On Fri, 27 Jul 2018 at 20:13, Robert Haas wrote: >> >> On Fri, Jul 27, 2018 at 3:17 AM, Ashutosh Bapat >> wrote: >> > Apart from the complexity there's also a possibi

Re: TupleTableSlot abstraction

2018-08-28 Thread Ashutosh Bapat
On Fri, Aug 24, 2018 at 6:46 AM, Andres Freund wrote: > Hi, > > On 2018-08-20 19:51:33 +0530, Ashutosh Bapat wrote: >> Sorry, forgot about that. Here's the patch set with that addressed. > > Btw, you attach files as tar.zip, but they're actually gzip > compre

Re: TupleTableSlot abstraction

2018-08-09 Thread Ashutosh Bapat
On Wed, Aug 8, 2018 at 5:07 PM, Ashutosh Bapat wrote: Amit Khandekar offlist told me that the previous patch-set doesn't apply cleanly on the latest head. PFA patches rebased on 31380bc7c204e7cfa9c9e1c62947909e2b38577c >>>>> >>>>> 3. compile with LLVM and

Re: TupleTableSlot abstraction

2018-08-06 Thread Ashutosh Bapat
On Mon, Aug 6, 2018 at 10:15 AM, Andres Freund wrote: > Hi, > > On 2018-08-06 10:08:24 +0530, Ashutosh Bapat wrote: >> I think, I explained why getattr() needs to be a separate callback. >> There's a reason why slot_getattr() more code than just calling >> sl

Re: TupleTableSlot abstraction

2018-08-05 Thread Ashutosh Bapat
On Sun, Aug 5, 2018 at 3:49 PM, Andres Freund wrote: > Hi, > > Working on rebasing the pluggable storage patch on the current version > of this. Thanks. Please let me know if you see any issues. > > On 2018-07-26 17:09:08 +0530, Ashutosh Bapat wrote: >> Done. I also n

Re: Problem while updating a foreign table pointing to a partitioned table on foreign server

2018-08-02 Thread Ashutosh Bapat
o you think we need it to be added to the relation descriptor. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] [PATCH] Overestimated filter cost and its mitigation

2018-07-27 Thread Ashutosh Bapat
On Thu, Jul 26, 2018 at 10:30 PM, Dean Rasheed wrote: > On 26 July 2018 at 07:12, Ashutosh Bapat > wrote: >> In the patch clauselist_selectivity() gets called repeatedly for every >> new qual added to the clause list. Instead, if we try to combine the >>

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2018-07-27 Thread Ashutosh Bapat
On Thu, Jul 26, 2018 at 8:37 PM, Dmitry Dolgov <9erthali...@gmail.com> wrote: >> On Mon, 23 Jul 2018 at 10:38, Ashutosh Bapat >> wrote: >> >> On Fri, Jul 20, 2018 at 3:13 AM, Dmitry Dolgov <9erthali...@gmail.com> wrote: >> > >> > It's

Re: [HACKERS] [PATCH] Overestimated filter cost and its mitigation

2018-07-25 Thread Ashutosh Bapat
by Tom, it will be more useful to have this feature work with not just scans but joins as well. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-25 Thread Ashutosh Bapat
ou suggesting that we change the reltype of a partition being attached? What happens when we detach a partition and let the table live independent of the partitioned table? Do we create a new reltype? I am not even going into the problem when we try to attach a partition with different tuple layout.

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-23 Thread Ashutosh Bapat
be fine if > the change you were making were only cosmetic, but it's not. > Agree with all that, including the clipped paras. > After looking over both patches, I think Ashutosh Bapat has basically > the right idea. What he is proposing is a localized fix that doesn't >

Re: [HACKERS] Two pass CheckDeadlock in contentent case

2018-07-23 Thread Ashutosh Bapat
the backend starts releasing its locks and releases even the first lock. Some different backend gets its first exclusive lock because of that but can not get the next one since it's held by the second backend holding shared locks. This might not happen if by "more than one" the comment means all the locks. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: de-deduplicate code in DML execution hooks in postgres_fdw

2018-07-23 Thread Ashutosh Bapat
. I don't like "parameterized" since that gets too tied to the method we are using rather than what's actually being done. In short I don't find any of the suggestions to be significantly better or worse than the name I have chosen. Said that, I am not wedded to any of those. A committer is free to choose anything s/he likes. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2018-07-23 Thread Ashutosh Bapat
further is not the worth the rare use cases. If we get complaints from the field, we can certainly improve it in future. But I would wait for those complaints before improving it further. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: de-deduplicate code in DML execution hooks in postgres_fdw

2018-07-19 Thread Ashutosh Bapat
On Thu, Jul 19, 2018 at 2:05 PM, Etsuro Fujita wrote: > (2018/04/18 19:34), Ashutosh Bapat wrote: >> >> Hi, >> While working on a fix related to non-direct DML [1], I noticed that >> postgresExecForeignInsert(), postgresExecForeignUpdate() and >> postgresExecFor

Re: print_path is missing GatherMerge and CustomScan support

2018-07-18 Thread Ashutosh Bapat
On Thu, Jul 19, 2018 at 11:56 AM, Michael Paquier wrote: > On Thu, Jul 19, 2018 at 10:41:10AM +0530, Ashutosh Bapat wrote: >> I had extensive testcases in my original patch-set to exercise that >> code but 1. that testset was too extensive; even today >> partition_join.sql i

Re: print_path is missing GatherMerge and CustomScan support

2018-07-18 Thread Ashutosh Bapat
On Thu, Jul 19, 2018 at 5:37 AM, Michael Paquier wrote: > On Wed, Jul 18, 2018 at 12:15:25PM +0530, Ashutosh Bapat wrote: >> Yes that's right. Thanks for taking care of it. > > Okay, I have pushed a fix for this one as that's wrong and > back-patc

Re: [HACKERS] PoC: full merge join on comparison clause

2018-07-18 Thread Ashutosh Bapat
support one inequality clause. Thanks for the commit messages. I would use word "in-equality" instead of "comparison" since equality is also a comparison. > > Some particular points: > > On 07/06/2018 04:01 PM, Ashutosh Bapat wrote: >> >> -

Re: print_path is missing GatherMerge and CustomScan support

2018-07-17 Thread Ashutosh Bapat
. Yes that's right. Thanks for taking care of it. > Robert, Ashutosh, am I missing > something? You used my personal email id by mistake, I think. I have removed it and added by EDB email address. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: foreign key to foreign table

2018-07-17 Thread Ashutosh Bapat
ign key constraints will become invalid without local server knowing about them. This looks like a question for general mailing list to me. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: How to make partitioning scale better for larger numbers of partitions

2018-07-16 Thread Ashutosh Bapat
hich are never used. In this case, even if we improve the planning time by 100%, it hardly matters since planning time is neglegible compared to the execution time because of huge data where partitioning is useful. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: partition pruning doesn't work with IS NULL clause in multikey range partition case

2018-07-16 Thread Ashutosh Bapat
On Sat, Jul 14, 2018 at 2:41 AM, Alvaro Herrera wrote: > On 2018-Jul-13, Ashutosh Bapat wrote: > >> On Fri, Jul 13, 2018 at 1:15 PM, Amit Langote >> wrote: >> >> >> >> I don't think this is true. When equality conditions and IS NULL clauses

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-13 Thread Ashutosh Bapat
n [1]. And if we go that > way, my patch seems to fit into that, because in that approach the > Append/MergeAppend could be removed after adjusting the targetlists for its > subplans in create_append_plan/create_merge_append_plan. Anyway, I'd like > to join in that work for PG12. Whatever may be the outcome of that work, I think what we fix here shouldn't require to reverted in a few months from now, just so that that patch works. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: BUG #15212: Default values in partition tables don't work as expected and allow NOT NULL violation

2018-07-13 Thread Ashutosh Bapat
On Thu, Jul 12, 2018 at 2:29 PM, Amit Langote wrote: > Thanks Ashutosh. > > On 2018/07/10 22:50, Ashutosh Bapat wrote: >> I didn't see any hackers thread linked to this CF entry. Hence sending this >> mail through CF app. > > Hmm, yes. I hadn't posted the pa

Re: partition pruning doesn't work with IS NULL clause in multikey range partition case

2018-07-13 Thread Ashutosh Bapat
it's only then that we could eliminate the partition >> containing >> NULL values, not otherwise. > > Actually, there is only one case where the pruning step generated by that > block of code is useful -- to prune a list partition that accepts only > nulls. List partitioning only allows one partition, key so this works, > but let's say only accidentally. I modified the condition as follows: > > +else if (!generate_opsteps && !bms_is_empty(notnullkeys) && > + bms_num_members(notnullkeys) == part_scheme->partnatts) > Hmm. Ok. May be it's better to explicitly say that it's only useful in case of list partitioned table. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: How to make partitioning scale better for larger numbers of partitions

2018-07-13 Thread Ashutosh Bapat
have really large that which really requires 1100 partitions and then compare performance by putting that data in 1100 partitions and in an unpartitioned table. Even with that kind of data, you will see some difference in performance, but that won't be as dramatic as you report. I mi

Re: partition pruning doesn't work with IS NULL clause in multikey range partition case

2018-07-12 Thread Ashutosh Bapat
I think we should add this to open items list so that it gets tracked. On Thu, Jul 12, 2018 at 6:31 PM, Ashutosh Bapat wrote: > On Thu, Jul 12, 2018 at 11:10 AM, Amit Langote > wrote: >>> >>> I think your fix is correct. I slightly modified it along with updating >&

Re: partition pruning doesn't work with IS NULL clause in multikey range partition case

2018-07-12 Thread Ashutosh Bapat
partition. I don't understand this. When there are IS NOT NULL clauses for all the partition keys, it's only then that we could eliminate the partition containing NULL values, not otherwise. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-11 Thread Ashutosh Bapat
On Thu, Jul 12, 2018 at 9:02 AM, Etsuro Fujita wrote: > (2018/07/11 20:02), Ashutosh Bapat wrote: >> >> On Wed, Jul 11, 2018 at 1:23 PM, Etsuro Fujita >> wrote: >>> >>> Actually, even if we could create such an index on the child table and >>> th

Re: [HACKERS] Two pass CheckDeadlock in contentent case

2018-07-11 Thread Ashutosh Bapat
se their respective locks. Also there's a comment at the end about the order in which to release the lock /* * And release locks. We do this in reverse order for two reasons: (1) * Anyone else who needs more than one of the locks will be trying to lock * them in increasing order; we don't want to release the other process * until it can get all the locks it needs. (2) This avoids O(N^2) * behavior inside LWLockRelease. */ check_done: for (i = NUM_LOCK_PARTITIONS; --i >= 0;) LWLockRelease(LockHashPartitionLockByIndex(i)); Is that scenario applicable when the shared locks are released in the first phase? [1] https://www.postgresql.org/docs/11/static/runtime-config-locks.html -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Preferring index-only-scan when the cost is equal

2018-07-11 Thread Ashutosh Bapat
h() instead to add index only path before any index paths. For that the index list needs to be sorted by the possibility of using index only scan. But I think in your case, it might be better to first check whether there is any costing error because of which index only scan's path has the same

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-11 Thread Ashutosh Bapat
so it think it needs to add a ConvertRowtypeExpr on that. But when the plan is created, there is ConvertRowtypeExpr already, but there is no way to know that a new ConvertRowtypeExpr is not needed anymore. So, we may have two ConvertRowtypeExprs giving wrong results. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

partition pruning doesn't work with IS NULL clause in multikey range partition case

2018-07-11 Thread Ashutosh Bapat
x27;t investigate further to see why nullkeys is NULL, but it looks like that's the problem and we are missing an optimization. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] PoC: full merge join on comparison clause

2018-07-10 Thread Ashutosh Bapat
On Tue, Jul 10, 2018 at 12:05 AM, Alexander Kuzmenkov wrote: > On 07/09/2018 04:12 PM, Ashutosh Bapat wrote: >> >> On Fri, Jul 6, 2018 at 6:31 PM, Ashutosh Bapat >> wrote: >>> >>> I will continue reviewing the patches. >>> >> Here are some

Re: BUG #15212: Default values in partition tables don't work as expected and allow NOT NULL violation

2018-07-10 Thread Ashutosh Bapat
I didn't see any hackers thread linked to this CF entry. Hence sending this mail through CF app. The patch looks good to me. It applies cleanly, compiles cleanly and make check passes. I think you could avoid condition + /* Do not override parent's NOT NULL constraint. */

Re: Test for trigger condition accessing system attributes

2018-07-10 Thread Ashutosh Bapat
Thanks Heikki. On Tue, Jul 10, 2018 at 6:47 PM, Heikki Linnakangas wrote: > On 10/05/18 11:58, Ashutosh Bapat wrote: >> >> Hi, >> I was investigating the cases when the system attributes are accessed >> beyond the scans. After investigating set_plan_references(),

Re: [PATCH] Add support for ON UPDATE/DELETE actions on ALTER CONSTRAINT

2018-07-10 Thread Ashutosh Bapat
don't think that it will be acceptable to add that huge an output to the regression test. You will need to provide a patch with much smaller output addition and may be a smaller test as well. > > You think this should be applied to all versions that support ALTER > CONSTRAINT, right? I think so. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-10 Thread Ashutosh Bapat
On Tue, Jul 10, 2018 at 9:08 AM, Etsuro Fujita wrote: > (2018/07/09 20:43), Ashutosh Bapat wrote: >>> >>> >>> I don't have any numbers right now, so that is nothing but a concern. But >>> as >>> I said in a previous email, in the approach I pr

Re: [HACKERS] PoC: full merge join on comparison clause

2018-07-09 Thread Ashutosh Bapat
On Fri, Jul 6, 2018 at 6:31 PM, Ashutosh Bapat wrote: > > I will continue reviewing the patches. > Here are some more review comments - * sort ordering for each merge key. The mergejoinable operator is an - * equality operator in the opfamily, and the two inputs are guaranteed to be

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-09 Thread Ashutosh Bapat
eExpr, albeit only when there is a whole-row expression in the targetlist, when creating plans. That's not there in my patch. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-09 Thread Ashutosh Bapat
e some cases where the test affects > performance. What are those cases? Can you please provide any numbers supporting your claim? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] PoC: full merge join on comparison clause

2018-07-06 Thread Ashutosh Bapat
ssage that gets added to the patch. That way, it leaves there for anybody to review. I am having a difficult time reading the next patch. There are various changes in the second patch, which I don't understand the reason behind. I think some comments will help, in as commit message or in the code. I

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-06 Thread Ashutosh Bapat
On Fri, Jul 6, 2018 at 4:29 PM, Etsuro Fujita wrote: > (2018/07/04 21:37), Ashutosh Bapat wrote: >> >> On Wed, Jul 4, 2018 at 5:36 PM, Etsuro Fujita >> wrote: >>> >>> (2018/07/04 19:04), Ashutosh Bapat wrote: >>>> >>>&g

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-05 Thread Ashutosh Bapat
don't think that's a great idea since such a plan might turn out cheaper esp. when there are very few children which could use parallel query and parallel append is possible at the top parent. But anyway, that's what it is today. But I think, we shouldn't write code assuming tha

Re: TupleTableSlot abstraction

2018-07-05 Thread Ashutosh Bapat
On Mon, Jun 11, 2018 at 6:13 PM, Ashutosh Bapat wrote: > >> >> The slightly bigger issue is that several parts of the code currently >> require heaptuples they can scribble upon. E.g. nodeModifyTable.c >> materializes the tuple - those can be solved by using a local

Re: Remove mention in docs that foreign keys on partitioned tables are not supported

2018-07-04 Thread Ashutosh Bapat
munication. But I try to get clarity on that confusion. > I understand that you don't agree, which is > fine, but I stand by my position. > I am fine with disagreement, now that I know why there's disagreement. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-04 Thread Ashutosh Bapat
On Wed, Jul 4, 2018 at 5:36 PM, Etsuro Fujita wrote: > (2018/07/04 19:04), Ashutosh Bapat wrote: >> >> On Fri, Jun 29, 2018 at 6:21 PM, Etsuro Fujita >> wrote: >>> >>> (2018/06/22 22:54), Ashutosh Bapat wrote: >>>> >>>> + if (

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-04 Thread Ashutosh Bapat
eful, but I don't think we can theoretically eliminate the need for such a plan. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-04 Thread Ashutosh Bapat
On Fri, Jun 29, 2018 at 6:21 PM, Etsuro Fujita wrote: > (2018/06/22 22:54), Ashutosh Bapat wrote: >> >> I have started reviewing the patch. > > > Thanks for the review! > >> + if (enable_partitionwise_join&& rel->top_parent_is_partitioned) >

Re: Desirability of client-side expressions in psql?

2018-07-03 Thread Ashutosh Bapat
So I am marking this as RWF. The discussion here can continue. Please submit a patch (may be a PoC) to the commitfest whenever there's one. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Remove mention in docs that foreign keys on partitioned tables are not supported

2018-07-02 Thread Ashutosh Bapat
On Tue, Jul 3, 2018 at 8:19 AM, Robert Haas wrote: > On Mon, Jul 2, 2018 at 1:46 AM, Ashutosh Bapat > wrote: >> This constraint was added to the partitioned table and inherited from >> there. If user wants to drop that constraint for some reason, this >> error message

Re: Remove mention in docs that foreign keys on partitioned tables are not supported

2018-07-01 Thread Ashutosh Bapat
onstraint was inherited like the error message itself reads like "can not drop constraint "p_a_check" on relation "p1" inherited from "partitioned table's name" OR a hint "you may drop constraint "p_a_check" on the partitioned table "partitioned table's name". It might even suffice to say "partition p1" instead "relation p1" so that the user gets a clue. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: partitioning - changing a slot's descriptor is expensive

2018-06-28 Thread Ashutosh Bapat
On Fri, Jun 29, 2018 at 11:29 AM, Andres Freund wrote: > On 2018-06-29 11:20:53 +0530, Amit Khandekar wrote: >> On 27 June 2018 at 18:33, Ashutosh Bapat >> wrote: >> > On Wed, Jun 27, 2018 at 10:39 AM, Andres Freund wrote: >> >> Unfortunately calling ExecS

Re: partition tree inspection functions

2018-06-28 Thread Ashutosh Bapat
those cases are not covered by object modification/creation tests as well. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2018-06-27 Thread Ashutosh Bapat
On Wed, Jun 27, 2018 at 12:28 PM, Amit Langote wrote: > On 2018/06/26 18:02, Ashutosh Bapat wrote: >> On Tue, Jun 26, 2018 at 2:27 PM, Amit Langote >> wrote: >>> Hi Ashutosh, >>> >>> On 2018/05/14 20:14, Ashutosh Bapat wrote: >>>> 0001-Has

<    3   4   5   6   7   8   9   10   11   >